Re: [Bug-wget] Does wget has the "outstanding read data" problem?
> Just try it out and report back if there is a problem. It is hard to reproduce the problem. -- Regards, Peng
Re: [Bug-wget] Does wget has the "outstanding read data" problem?
On 18.05.19 13:39, Peng Yu wrote: > On 5/18/19, Tim Rühsen wrote: >> On 17.05.19 23:09, Peng Yu wrote: >>> Hi, >>> >>> curl has this problem "(18) transfer closed with outstanding read data >>> remaining". Does wget have a similar problem like this? Thanks. >>> >>> https://curl.haxx.se/mail/archive-2018-10/0015.html >> >> The connection is closed prematurely and you do not want to re-download >> everything. >> >> Wget has -c / --continue for that. >> >> If you have a recent version of wget, it will use a if-modified-since >> request header - and sometimes this goes wrong, some servers have issues >> with that header. In this case add --no-if-modified-since to use a >> different strategy (HEAD request first, then GET). > > But what if the request causing the problem was POST instead of GET? Just try it out and report back if there is a problem. Regards, Tim signature.asc Description: OpenPGP digital signature
Re: [Bug-wget] Does wget has the "outstanding read data" problem?
On 18.05.19 16:22, Daniel Stenberg wrote: > On Sat, 18 May 2019, Tim Rühsen wrote: > >> The connection is closed prematurely and you do not want to >> re-download everything. >> >> Wget has -c / --continue for that. > > I could be noted that transfers with chunked encoding are rarely resumable. > I never tried - so I can't say for sure. But even if the structure of chunks is different, the content / payload is the same (else wget would detect that and re-download from the start). I see no technical problem here. Then, chunked transfer makes mostly sense when transferring dynamically generated content. And that likely means two successive requests will see two (slightly) different contents. Regards, Tim signature.asc Description: OpenPGP digital signature
Re: [Bug-wget] Does wget has the "outstanding read data" problem?
On Sat, 18 May 2019, Tim Rühsen wrote: The connection is closed prematurely and you do not want to re-download everything. Wget has -c / --continue for that. I could be noted that transfers with chunked encoding are rarely resumable. -- / daniel.haxx.se
Re: [Bug-wget] Does wget has the "outstanding read data" problem?
On 5/18/19, Tim Rühsen wrote: > On 17.05.19 23:09, Peng Yu wrote: >> Hi, >> >> curl has this problem "(18) transfer closed with outstanding read data >> remaining". Does wget have a similar problem like this? Thanks. >> >> https://curl.haxx.se/mail/archive-2018-10/0015.html > > The connection is closed prematurely and you do not want to re-download > everything. > > Wget has -c / --continue for that. > > If you have a recent version of wget, it will use a if-modified-since > request header - and sometimes this goes wrong, some servers have issues > with that header. In this case add --no-if-modified-since to use a > different strategy (HEAD request first, then GET). But what if the request causing the problem was POST instead of GET? -- Regards, Peng
Re: [Bug-wget] Does wget has the "outstanding read data" problem?
On 17.05.19 23:09, Peng Yu wrote: > Hi, > > curl has this problem "(18) transfer closed with outstanding read data > remaining". Does wget have a similar problem like this? Thanks. > > https://curl.haxx.se/mail/archive-2018-10/0015.html The connection is closed prematurely and you do not want to re-download everything. Wget has -c / --continue for that. If you have a recent version of wget, it will use a if-modified-since request header - and sometimes this goes wrong, some servers have issues with that header. In this case add --no-if-modified-since to use a different strategy (HEAD request first, then GET). Regards, Tim signature.asc Description: OpenPGP digital signature
Re: [Bug-wget] Does wget has the "outstanding read data" problem?
The question is whether wget can fix the problem by resuming from where it failed. Curl can not do it as mentioned in the email. On Fri, May 17, 2019 at 4:20 PM Daniel Stenberg wrote: > On Fri, 17 May 2019, Peng Yu wrote: > > > curl has this problem "(18) transfer closed with outstanding read data > > remaining". Does wget have a similar problem like this? Thanks. > > That's not a "problem" curl has. That's a feature. It detects prematurely > cut > off transfers when receiving chunked encoded data. > > I would presume wget does the same, as so should all HTTP clients. > > But of course, a retry might repair the damage as the transfer can be > tried > again. > > -- > > / daniel.haxx.se > -- Regards, Peng
Re: [Bug-wget] Does wget has the "outstanding read data" problem?
On Fri, 17 May 2019, Peng Yu wrote: curl has this problem "(18) transfer closed with outstanding read data remaining". Does wget have a similar problem like this? Thanks. That's not a "problem" curl has. That's a feature. It detects prematurely cut off transfers when receiving chunked encoded data. I would presume wget does the same, as so should all HTTP clients. But of course, a retry might repair the damage as the transfer can be tried again. -- / daniel.haxx.se
