AI media pipelines move large input videos and generated clips across unreliable networks. Incomplete S3 multipart uploads are not finished objects, but their parts can still incur storage cost. This runbook explains state transitions, cleanup thresholds, cancellation races and evidence requirements without assuming that a single retry makes a distributed workflow consistent.
Model the unfinished object
After initiation, S3 assigns an upload identifier; parts remain separate until completion assembles an object. Record that identifier with bucket, object key, application job and creation timestamp. Distinguish upload success from media validation and public release. A database row saying ‘rendered’ must never substitute for object completion evidence. Investigate partial bytes separately from objects visible in a normal listing.
Validation should record the expected state, the observed state, the identity of the responsible component and a concrete rollback condition. Repeat the scenario with a delayed callback and a duplicate request so that a passing ordinary case does not mask an unstable failure path. Compare results against a documented baseline and preserve dated evidence for later audits.
Design explicit state transitions
Use initiated, transferring, commit-pending, complete, cancel-requested and aborted states. Transitions require an idempotency token and durable evidence. A worker that resumes after a timeout must reconcile S3 before issuing another commit. Do not mark an upload failed merely because its completion response was lost. Test crashes at each boundary between remote storage and the local job database.
Validation should record the expected state, the observed state, the identity of the responsible component and a concrete rollback condition. Repeat the scenario with a delayed callback and a duplicate request so that a passing ordinary case does not mask an unstable failure path. Compare results against a documented baseline and preserve dated evidence for later audits.
Choose lifecycle rules deliberately
AWS describes AbortIncompleteMultipartUpload as a lifecycle action using days since initiation. Select a threshold longer than legitimate recovery windows, and document whether prefix filters are used. The lifecycle action applies to unfinished uploads, not completed objects. Treat it as a safety net, while the application performs explicit cancellation for user-directed job aborts. Verify the rule against the correct bucket environment.
Validation should record the expected state, the observed state, the identity of the responsible component and a concrete rollback condition. Repeat the scenario with a delayed callback and a duplicate request so that a passing ordinary case does not mask an unstable failure path. Compare results against a documented baseline and preserve dated evidence for later audits.
The relevant technical rules are detailed in AWS: Aborting a multipart upload (accessed 2026-11-04).
Build a storage cost ledger
Measure unfinished part bytes by age bracket and origin workflow. An illustrative estimate multiplies average unfinished bytes by storage price and duration, then adds any relevant request or investigation overhead. Do not describe this estimate as an observed vendor charge. Compare how much waste is prevented by explicit cancellation versus lifecycle cleanup. Publish input assumptions so future prices and media sizes can be substituted.
Validation should record the expected state, the observed state, the identity of the responsible component and a concrete rollback condition. Repeat the scenario with a delayed callback and a duplicate request so that a passing ordinary case does not mask an unstable failure path. Compare results against a documented baseline and preserve dated evidence for later audits.
Handle races during cancellation
A user can cancel while the final part or completion request is in flight. Application workers must coordinate a terminal cancellation marker and verify remote state. Do not publish the asset after cancellation simply because an old callback arrives late. An abort call can race with ongoing transfers, so reconcile and retry according to documented semantics. Preserve one stable job identity across all branches.
Validation should record the expected state, the observed state, the identity of the responsible component and a concrete rollback condition. Repeat the scenario with a delayed callback and a duplicate request so that a passing ordinary case does not mask an unstable failure path. Compare results against a documented baseline and preserve dated evidence for later audits.
Test before broad rollout
In a test bucket, initiate a multipart upload, send benign test parts and deliberately stop. Confirm that the incomplete upload can be listed, explicitly aborted and later cleaned through lifecycle eligibility. Verify that completed test objects remain intact. Track recovery attempts, unfinished bytes and the time required to close each incident. Keep secrets, user videos and private object keys out of diagnostic reports.
Validation should record the expected state, the observed state, the identity of the responsible component and a concrete rollback condition. Repeat the scenario with a delayed callback and a duplicate request so that a passing ordinary case does not mask an unstable failure path. Compare results against a documented baseline and preserve dated evidence for later audits.
Worked scenario and acceptance review
Consider an illustrative 600 MB creator-video upload divided into 12 parts. A client disconnects after part seven; the UI reports a network error, but the database still holds an active upload ID. The operator should first check if the upload later completed, then confirm whether the user requested cancellation. If it remains incomplete, an explicit abort can reclaim the stored parts. A lifecycle rule provides a backstop after the agreed age threshold. The important observables are not a fabricated dollar saving but the bytes stored, the age of the unfinished upload, the state in S3 and the final application decision. A different scenario has the complete request succeed but the response disappear; blindly aborting that logical job could produce a false user error. A reconciliation test must distinguish these outcomes before either publishing the video or cleaning it up.
Sources
- AWS: Aborting a multipart upload (accessed 2026-11-04)
- AWS: Lifecycle cleanup of incomplete multipart uploads (accessed 2026-11-04)
All example operations and budgets are illustrative, not claimed production data.