[CONF] Apache Sling > Chunked File Upload Support

[confluence]

Chunked File Upload Support Page edited by Shashank Gupta Changes (30) ... h1. Protocol Specification h4. Advertising chunk upload support in sling Sling advertise its support for chunk upload by adding POST and PATCH methods to the listing of allowed methods in the "Allow" OPTIONS response header defined in HTTP/1.1. [request] {code:title= OPTION request} OPTIONS /content/dam/dam-folder/catalog.pdf.chunk HTTP/1.1 {code} [response] {code:title= OPTION response} HTTP/1.1 200 OK Allow: GET, PUT, POST, OPTIONS, HEAD, DELETE, PATCH Accept-Patch: append/binary-patch, application/json-patch, application/json-patch {code} [ Accept-Patch|http://greenbytes.de/tech/webdav/rfc5789.html#accept-patch] used to specify the patch document formats accepted by the server. Chunk upload supports three append/binary-patch, application/json-patch, application/json-patch formats. h3. Upload chunk using POST h4. Upload of first and intermediate chunks Client uses PATCH POST method to upload binary chunk of file. [request] {code:title= Upload the first/intermediate chunk request} Request: PATCH POST /content/dam/dam-folder/catalog.pdf.chunk.<chunk_number>.res HTTP/1.1 Host: localhost:4502 If-Match: "e0023aa4e" Content-Type: Date: <date> Content-Length: 5242880 {code} [response] {code:title= First/Intermediate chunk upload response} Response: HTTP/1.1 201 CREATED Content-Length: 0 ETag: "e0023aa4f" Location: /content/dam/dam-folder/catalog.pdf.chunk Connection: close ... {code} The location header provides path to retrieve chunk upload attributes. Concurrent request from multiple client requests will corrupt the resource. Clients should use a conditional request such that the request will fail if the resource has been updated since the client last accessed the resource. For example, the client can use a strong [ ETag | http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.19] in an [ If-Match | http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.24] header on the chunk request. h4. Upload of last chunk Client appends "/last" to upload url to signify that it would be last that needs to be uploaded. The last chunk upload request can optionally be multipart request if client requires to send additional parameters to sling. [request] {code:title= Upload last chunk request} Request: PATCH POST /content/dam/dam-folder/catalog.pdf.chunk.<chunk_number>.res/last HTTP/1.1 Host: localhost:4502 Date: <date> ... {code} [response] {code:title= Upload last chunk response} Response: HTTP/1.1 201/CREATED Content-Length: 0 Location: /content/dam/dam-folder/catalog.pdf Content-MD5: <md5 checksum of the merged file> Content-MD5: $md5_hash Connection: close {code} The location header provides location of fully uploaded asset. h3. Upload chunk using PATCH h3. Query upload h4. Query to retrieve last successful chunk upload Returns the number of last chunk upload and bytes uploaded successfully on sling. [request] {code:title= Get last successful chunk upload request} Request GET /content/dam/dam-folder/catalog.pdf.chunk.json HTTP/1.1 Host: localhost:4502 ... {code} [response] {code:title= Get last successful upload chunk response} Response: HTTP/1.1 200/OK Connection: close ... If no broken/discontinued upload sling returns 404 not found response. [response] {code:title= Get erroneous last successful upload chunk response} Response: HTTP/1.1 404/Not Found {code} h43. Abort chunked upload [request] {code:title= Abort chunked upload Request} Request DELETE /content/dam/dam-folder/catalog.pdf.chunk.res HTTP/1.1 Host: localhost:4502 ... {code} [response] {code:title= Abort chunked upload response} Response HTTP/1.1 200/OK Connection: close {code} h4. Query sling's capability to support chunked upload Client can use option method to query Sling's capability to support chunked upload. In response, the Allow header contains PATCH method, indicating that Sling supports chunked upload on the url. {code:title= Options chunked upload request} Request OPTIONS /content/dam/dam-folder/catalog.pdf.chunk.res HTTP/1.1 Host: localhost:4502 Date: <date> {code} {code:title= Options chunked upload response} Response HTTP/1.1 200 OK Connection: Keep-Alive Allow: OPTIONS, GET, PATCH {code} Full Content 1 Chunked File Upload 2 Use Cases: 3 Approach 4 Protocol Specification 4.1.1 Advertising chunk upload support in sling 4.2 Upload chunk using POST 4.2.1 Upload of first and intermediate chunks 4.2.2 Upload of last chunk 4.3 Upload chunk using PATCH 4.4 Query upload 4.4.1 Query to retrieve last successful chunk upload 4.5 Abort chunked upload Chunked File Upload Status: DRAFT Created: 20. January 2013 Author: shgupta JIRA: SLING-2707 References: - http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html Updated: - Use Cases: 1. Large file upload - With high speed internet connections, advent of cloud and HD going mainstream, Sling support large files (> 2GB) upload. 2. Fault tolerant uploads - Sling provides capability to resume upload from failure point. It doesn't require client to restart the complete upload process. Approach Sling provides an endpoint which accepts file chunks in accordance with a specified protocol. Sling client slices the file in chunks, and upload the chunks in serial manner to server. Each chunk is numbered serially and the last chunk has additional identifier to distinguish from the rest chunks. Sling endpoint upon receiving the last chunk, stitches all chunks into a single file. The file inputstream is passed to inner layers for further processing. In case of upload failures, sling provides support to query the last chunk uploaded till failure point. Client resumes chunk upload from last failure point. Protocol Specification Advertising chunk upload support in sling Sling advertise its support for chunk upload by adding POST and PATCH methods to the listing of allowed methods in the "Allow" OPTIONS response header defined in HTTP/1.1. request OPTION request OPTIONS /content/dam/dam-folder/catalog.pdf.chunk HTTP/1.1 response OPTION response HTTP/1.1 200 OK Allow: GET, PUT, POST, OPTIONS, HEAD, DELETE, PATCH Accept-Patch: append/binary-patch, application/json-patch, application/json-patch [ Accept-Patch|http://greenbytes.de/tech/webdav/rfc5789.html#accept-patch] used to specify the patch document formats accepted by the server. Chunk upload supports three append/binary-patch, application/json-patch, application/json-patch formats. Upload chunk using POST Upload of first and intermediate chunks Client uses POST method to upload binary chunk of file. request Upload the first/intermediate chunk request POST /content/dam/dam-folder/catalog.pdf.chunk.<chunk_number>.res HTTP/1.1 Host: localhost:4502 If-Match: "e0023aa4e" Content-Type: Date: <date> Content-Length: 5242880 response First/Intermediate chunk upload response HTTP/1.1 201 CREATED Content-Length: 0 ETag: "e0023aa4f" Location: /content/dam/dam-folder/catalog.pdf.chunk Connection: close Server: localhost:4502 The location header provides path to retrieve chunk upload attributes. Concurrent request from multiple client requests will corrupt the resource. Clients should use a conditional request such that the request will fail if the resource has been updated since the client last accessed the resource. For example, the client can use a strong [ ETag | http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.19] in an [ If-Match | http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.24] header on the chunk request. Upload of last chunk Client appends "/last" to upload url to signify that it would be last that needs to be uploaded. The last chunk upload request can optionally be multipart request if client requires to send additional parameters to sling. request Upload last chunk request POST /content/dam/dam-folder/catalog.pdf.chunk.<chunk_number>.res/last HTTP/1.1 Host: localhost:4502 Date: <date> Content-type: multipart/form-data, boundary=AaB03x --AaB03x content-disposition: form-data; name="fileSize" 210000000 --AaB03x content-disposition: form-data; name="file"; fileName="catalog.pdf" Content-Type: application/pdf Content-Transfer-Encoding: binary $binarydata --AaB03x-- response Upload last chunk response HTTP/1.1 201/CREATED Content-Length: 0 Location: /content/dam/dam-folder/catalog.pdf Content-MD5: $md5_hash Connection: close The location header provides location of fully uploaded asset. Upload chunk using PATCH Query upload Query to retrieve last successful chunk upload Returns the number of last chunk upload and bytes uploaded successfully on sling. request Get last successful chunk upload request GET /content/dam/dam-folder/catalog.pdf.chunk.json HTTP/1.1 Host: localhost:4502 Date: <date> Content-Length: 0 response Get last successful upload chunk response HTTP/1.1 200/OK Connection: close { "chunkNumber": 8 "bytesuploaded": 19872 } If no broken/discontinued upload sling returns 404 not found response. response Get erroneous last successful upload chunk response HTTP/1.1 404/Not Found Abort chunked upload request Abort chunked upload Request DELETE /content/dam/dam-folder/catalog.pdf.chunk.res HTTP/1.1 Host: localhost:4502 Date: <date> response Abort chunked upload response HTTP/1.1 200/OK Connection: close Change Notification Preferences View Online | View Changes | Add Comment