Hi Hansel, Here's the documentation for browser-based POSTing using S3: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingHTTPPOST.html
If you want to implement chunking the file and stitching together yourself, here's quite a nice JavaScript library to help with some of the heavy lifting on the front-end. We've used this will some success for large files: http://www.resumablejs.com/ Kind regards, James Browne On 11 April 2017 at 11:32, William Mayor <m...@williammayor.co.uk> wrote: > > > On 11 Apr 2017, at 11:21, Hansel Dunlop <han...@interpretthis.org> > wrote: > > > > Hello all > > > > I'm working on an application that has to accept large uploads. Think ~ > 2GB+ size files getting uploaded over slowish connections. These files are > eventually going to end up in S3. > > > > Uploading smallish files is not a problem. But things get a bit > complicated when you're dealing with large files and load balanced servers. > Servers that can be replaced at any time. Has anyone done something similar? > > > > My current plan is: > > > > 1. Accept chunked uploads. So the app/browser sends individual POST > requests with ~10mb chunks. Once that upload is complete the server > responds with a chunk id and the current offset > > 2. The server stores each intermediate chunk in a temporary S3 bucket > > 3. Once the final chunk has been uploaded the server kicks off another > process that stitches the pieces together and puts the whole file into it's > final location. And then deletes the intermediate pieces. > > > > I think I have to do the file in chunks like this but maybe there is > some way to stream the files somewhere? > > > > -- > > > > Hansel > > _______________________________________________ > > python-uk mailing list > > python-uk@python.org > > https://mail.python.org/mailman/listinfo/python-uk > > I’ve don’t something similar in the past where I ended up having the > users’ browsers upload directly to S3 and then send the file’s key back to > me. That way my server didn’t have to manage any of the files and > (hopefully) the process was quicker from the user’s point of view. > > I used STS to create temporary AWS credentials that could only be used to > upload files to a very specific bucket and “folder”. I then used the AWS JS > SDK to manage the uploads client-side. > > I liked this solution as it really reduced the amount of complex code that > I had to write/maintain. All I had to do was glue the AWS SDKs together. > > Billy > > _______________________________________________ > python-uk mailing list > python-uk@python.org > https://mail.python.org/mailman/listinfo/python-uk > -- James Browne j...@jamesbrowne.me 07779 804 426
_______________________________________________ python-uk mailing list python-uk@python.org https://mail.python.org/mailman/listinfo/python-uk
