Hi Hugh, >> > I don't know about this at all. There was some stuff about streaming >> > on Redhanded, but I think it was downloads, so.... Does S3 assume a >> > perfect, fast internet, or does it have another part of the API which >> > would allow you to split the file into chunks, so that when your dialup >> > connection falls over, or a passing train scatters your wireless connection >> > to smithereens, or,... you can still send the whole file eventually? >> >> S3 does not support incremental uploads. Its a pure HTTP post of a >> large file. But it works fine ... just as long as the software doesn't >> try and load 1GB files directly into memory and then post them out. >> That's what I mean by streaming ... its streaming off the hard >> disk.... read a bit, add it to the post, read a bit more, add it to >> the post.... etc. > > OK, the only thing I can find about this is that post takes a block so > things can be dealt with in chunks. But the docs: > http://www.ruby-doc.org/stdlib/libdoc/net/http/rdoc/classes/Net/HTTP.html#M001227 > make no sense to me: they talk about reading stuff, which is in the > wrong direction. Maybe examining the source would clarify. Can't > find anythng with search engines (except that one says post is to > handle incoming post requests on the server), but looking at the > code of Mechanize might also be of some use.
I've actually already got this working with Net::Http. Did so by copying the S3Sync code here: http://github.com/willcodeforfoo/s3sync/blob/745fef915a225933f4a1156c14a1ed237b43409f/HTTPStreaming.rb ... which is why if Net:HTTP domain resolution worked in windows, I'd be tickled pink. (I'm thinking one kludge hack might be to run a download on the same url first, just before doing the net:http one... which, from tests, seems to allow net::http to resolve). Essentially,
