[BangPypers] How to send 1 GB Zip file to some remote machine using HTTP/HTTPS?
Hi All, How to send 1 GB Zip file to some remote machine using HTTP/HTTPS? As of now : I am running a http listner in to remote machine. I want to copy a file from local system to the remote machine where my HTTP/HTTPS Listner is running. Currently i am comressing the file, then reading the file and sending the contents to the remote agent as a string. At Remote agent , i am reading the string content and writing the same in to the file. this methord is OK when the file size is small ,like 100 MB or 200 MB. But it will give problem when i will transfer the large file say 1GB or 2 GB. because till the file writing contents will be in the memory, so possible errors can come as out of memory or it may not be able to send the large file so socket error can also come. So is there any other way to transfer the large file, size i have mention above using HTTP/HTTPS. Please mention if i can do it by using some other protocol. Thanks and Regards, Deepak Looking for local information? Find it on Yahoo! Local http://in.local.yahoo.com/___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] How to send 1 GB Zip file to some remote machine using HTTP/HTTPS?
On Thu, Sep 10, 2009 at 11:01 PM, deepak gupta dg288_m...@yahoo.co.in wrote: [..] Please mention if i can do it by using some other protocol. [..] Perhaps you should consider something more tuned to handle file transfers like rsync or even just sftp/scp. Is there a reason you *have* to use HTTP? -- ~noufal http://nibrahim.net.in ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] How to send 1 GB Zip file to some remote machine using HTTP/HTTPS?
Hi Noufal, Thanks for your quick reply. Actually at remote site we can not open the other ports except 443. thats why i am using HTTPS for this. I don't know exaxtly, how to use rsync or even just sftp/scp or how to create the listner for these. Thanks, Deepak --- On Thu, 10/9/09, Noufal Ibrahim nou...@gmail.com wrote: From: Noufal Ibrahim nou...@gmail.com Subject: Re: [BangPypers] How to send 1 GB Zip file to some remote machine using HTTP/HTTPS? To: Bangalore Python Users Group - India bangpypers@python.org Date: Thursday, 10 September, 2009, 11:05 PM On Thu, Sep 10, 2009 at 11:01 PM, deepak gupta dg288_m...@yahoo.co.in wrote: [..] Please mention if i can do it by using some other protocol. [..] Perhaps you should consider something more tuned to handle file transfers like rsync or even just sftp/scp. Is there a reason you *have* to use HTTP? -- ~noufal http://nibrahim.net.in ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers Love Cricket? Check out live scores, photos, video highlights and more. Click here http://cricket.yahoo.com___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] How to send 1 GB Zip file to some remote machine using HTTP/HTTPS?
On Thu, Sep 10, 2009 at 11:16 PM, deepak gupta dg288_m...@yahoo.co.in wrote: Hi Noufal, Thanks for your quick reply. Actually at remote site we can not open the other ports except 443. thats why i am using HTTPS for this. I don't know exaxtly, how to use rsync or even just sftp/scp or how to create the listner for these. Are your remote end servers written in Python? If not, you can setup servers on the remote end for rsync or ssh. ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] How to send 1 GB Zip file to some remote machine using HTTP/HTTPS?
Yes it is return in python. --- On Thu, 10/9/09, Noufal Ibrahim nou...@gmail.com wrote: From: Noufal Ibrahim nou...@gmail.com Subject: Re: [BangPypers] How to send 1 GB Zip file to some remote machine using HTTP/HTTPS? To: Bangalore Python Users Group - India bangpypers@python.org Date: Thursday, 10 September, 2009, 11:19 PM On Thu, Sep 10, 2009 at 11:16 PM, deepak gupta dg288_m...@yahoo.co.in wrote: Hi Noufal, Thanks for your quick reply. Actually at remote site we can not open the other ports except 443. thats why i am using HTTPS for this. I don't know exaxtly, how to use rsync or even just sftp/scp or how to create the listner for these. Are your remote end servers written in Python? If not, you can setup servers on the remote end for rsync or ssh. ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers Love Cricket? Check out live scores, photos, video highlights and more. Click here http://cricket.yahoo.com___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] How to send 1 GB Zip file to some remote machine using HTTP/HTTPS?
On Thu, Sep 10, 2009 at 11:24 PM, deepak gupta dg288_m...@yahoo.co.in wrote: Yes it is return in python.[..] I expect it will be slow then. If you can install an rsync server on the remote end, it would be nice. Then you can just rsync the files over. If you can't do that, one option is to split your compressed file into pieces and transfer them over one by one and assemble them on the remote end. -- ~noufal http://nibrahim.net.in ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
Re: [BangPypers] How to send 1 GB Zip file to some remote machine using HTTP/HTTPS?
On Thu, Sep 10, 2009 at 11:50 PM, Noufal Ibrahim nou...@gmail.com wrote: On Thu, Sep 10, 2009 at 11:24 PM, deepak gupta dg288_m...@yahoo.co.in wrote: Yes it is return in python.[..] I expect it will be slow then. If you can install an rsync server on the remote end, it would be nice. Then you can just rsync the files over. +1. It's worth having rsync. Here's what you'd do the first time: rsync -W -t --progress --partial datafile.zip u...@server:/export/home/user/data/datafile.zip What that does is: -W - Just transfer, don't check for any existing file on the receiving side -t- Use timestamps to compare and decide whether the file has even changed or not. --progress - Display progress --partial - Keep partially transferred files. The next time you want to transfer the same file, or in case you need to resume the transfer: rsync -t --progress --partial datafile.zip u...@server:/export/home/user/data/datafile.zip Simply remove the -W command line argument. If you can't do that, one option is to split your compressed file into pieces and transfer them over one by one and assemble them on the remote end. In such a case, remember to generate md5sums of the split parts, and then verify the checksums on the server side. -- ~noufal http://nibrahim.net.in ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers -- Sriram ___ BangPypers mailing list BangPypers@python.org http://mail.python.org/mailman/listinfo/bangpypers
