[
https://issues.apache.org/jira/browse/SLING-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashank Gupta updated SLING-2707:
----------------------------------
Attachment: uploadclient.jar
> What was used as server/endpoint for the above comparison?
CQ 5.6 server with S3 datastore.
I have implemented a servlet endpoint on amazon ec2 instance which can accept
parallel file chunks and merge them. Also attached upload client which can
upload chunks in parallel.
Environment: Amazon ec2 instance
OS: RHEL-6.3
Server: CQ 5.6 Load 20
CRX datastore: file system
Server endpoint: http://ec2-23-20-237-244.compute-1.amazonaws.com:4502
Usage: java -DparallelThreads=<number of parallel connections>
-DbaseUrl=http://ec2-23-20-237-244.compute-1.amazonaws.com:4502
-DfilePath=<file-path> -jar uploadclient.jar
if number of parallel connections = 1 : all chunks uploaded in serial manner
file-path = path of file which is being uploaded. file is uploaded into
/content/dam/<file_name>
Results:
client : my laptop (my org network)
File size : 54 MB
Chunk size : 5 MB
Number of chunks : 11
Time taken in serial upload : 1735 sec ~= 29 minutes
Time taken in parallel upload with 6 parallel connections: 691 sec ~= 11
minutes
There is clear advantage of faster upload using parallel connections.
PS: Since crx datastore detects duplicate binary, please change binary a bit so
that their SHA-1 hash is different.
> Support of chunked file upload into Sling
> -----------------------------------------
>
> Key: SLING-2707
> URL: https://issues.apache.org/jira/browse/SLING-2707
> Project: Sling
> Issue Type: New Feature
> Components: General
> Reporter: Shashank Gupta
> Attachments: uploadclient.jar
>
>
> Use cases:
> 1. Large file upload - With high speed internet connections, advent of cloud
> and HD going mainstream, Sling should support large files (> 2GB) upload.
> 2. Fault tolerant uploads - Sling should provide capability to resume upload
> from failure point. It should not require client to restart the complete
> upload process.
> 3. Faster upload: Sling should support its clients to initiate multiple
> connection and upload file chunks in parallel.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
