wget2 will not download any files, and I think there's some sort of disk
access issue.
this is on Windows Subsystem for Linux Debian 9.3 Stretch. (Ubuntu 16.04
LTS had the same issue.)
Here's the output of strace -o strace.txt -ff wget2 https://www.google.com
https://pastebin.com/4MEL88qs
wget
gt; But I can reproduce a hang with HTTPS and (repeating) errors with HTTP.
>
>
> This might be an issue with Windows Sockets... maybe someone has a
> faster machine to do some testing !?
>
>
> Regards, Tim
>
> On 02.04.2018 19:30, Jeffrey Fetterman wrote:
> > I can
oh, and the hang with HTTPS and repeating errors with HTTP is exactly the
same issue I'm experiencing, yes.
On Mon, Apr 2, 2018 at 3:59 PM, Jeffrey Fetterman
wrote:
> Why'd you use your wife's laptop? You can have Debian and Ubuntu installed
> on the same machine. Typ
.
I also tried this with openSUSE aside from learning that WSL openSUSE is a
mess, once I got it working I ran into the same issues as on WSL Debian &
WSL Ubuntu.
On Mon, Apr 2, 2018 at 3:59 PM, Jeffrey Fetterman
wrote:
> oh, and the hang with HTTPS and repeating errors with HTTP is exactl
I think I may have found the problem...
*https://github.com/Microsoft/WSL/issues/1419
<https://github.com/Microsoft/WSL/issues/1419>*
There's no workaround posted so I may be SOL unless anyone has any ideas.
On Mon, Apr 2, 2018 at 4:45 PM, Jeffrey Fetterman
wrote:
> Few other
It looks like there is a way to fix it:
https://github.com/Rich-Harris/port-authority/pull/5
On Mon, Apr 2, 2018 at 6:02 PM, Jeffrey Fetterman
wrote:
> I think I may have found the problem...
>
> *https://github.com/Microsoft/WSL/issues/1419
> <https://github.com/Microsoft/
I've submitted an issue here: https://gitlab.com/gnuwget/wget2/issues/370
On Mon, Apr 2, 2018 at 6:12 PM, Jeffrey Fetterman
wrote:
> It looks like there is a way to fix it: https://github.com/Rich-
> Harris/port-authority/pull/5
>
> On Mon, Apr 2, 2018 at 6:02 PM, Jeffrey F
Tim, do you know when you'll be able to examine and come up with a
workaround for the issue? There are alternatives to wget2 but either
they're not high performance or they're not really cut out for site
scraping.
On Mon, Apr 2, 2018 at 12:30 PM, Jeffrey Fetterman
wrote:
>
> > hickups with WSL have been solved one or another way.
> >
> >
> > With Best Regards, Tim
> >
> >
> > On 04/04/2018 09:01 AM, Jeffrey Fetterman wrote:
> >> Tim, do you know when you'll be able to examine and come up with a
> >> worka
I've successfully built wget2 through msys2 as a Windows binary, and it
appears to be working (granted I've not used it much yet), but I'm
concerned about some of the warnings that occurred during compilation.
Unsurprisingly they seem to be socket-related.
https://spit.mixtape.moe/view/9f38bd83
Thanks to the fix that Tim posted on gitlab, I've got wget2 running just
fine in WSL. Unfortunately it means I don't have TCP Fast Open, but given
how fast it's downloading a ton of files at once, it seems like it must've
been only a small gain.
I've come across a few annoyances however.
1. Ther
gh performance, just how
much am I missing out on by not being able to take advantage of tcp fast
open?
On Fri, Apr 6, 2018 at 5:01 PM, Tim Rühsen wrote:
> Hi Jeffrey,
>
>
> thanks for your feedback !
>
>
> On 06.04.2018 23:30, Jeffrey Fetterman wrote:
> > Thanks to th
On Sat, Apr 7, 2018 at 3:04 AM, Darshit Shah wrote:
> Hi Jefferey,
>
> Thanks a lot for your feedback. This is what helps us improve.
>
> * Tim Rühsen [180407 00:01]:
> >
> > On 06.04.2018 23:30, Jeffrey Fetterman wrote:
> > > Thanks to the fix that Tim post
r 8, 2018 at 10:11 AM, Darshit Shah wrote:
> * Jeffrey Fetterman [180408 04:53]:
> > Yes! Multiplexing was indeed partially the culprit, I've changed it
> > to --http2-request-window=5
> >
> > However the download queue (AKA 'Todo') still gets enormous. It
}
However, I still end up with multiple 'Failed to write 305 bytes (32:
Broken pipe)' errors when resuming a previous download with TLS Resume.
On Sun, Apr 8, 2018 at 4:38 PM, Jeffrey Fetterman
wrote:
> > The URLs are added first because of the way Wget will traverse
; is forcequitting out of a download and then
starting from the same session file as last time.
On Mon, Apr 9, 2018 at 3:36 AM, Tim Rühsen wrote:
> On 04/09/2018 10:10 AM, Jeffrey Fetterman wrote:
> > I've tested wget2 with the following changes to .\libwget\ssl_gnut
So I wanted to see how scraping a large site compared with multiplexing
off. I used the -http2=off parameter, but I got a segfault.
So I decided I'd configure wget2 without the http2 library and just try the
same command again (without -http2=off since it wasn't compiled with it
anyway) and it wor
God damnit, I just got it to happen with ./configure --without-libnghttp2
Now I'm not sure what is triggering it.
On Mon, Apr 9, 2018 at 6:04 AM, Jeffrey Fetterman
wrote:
> So I wanted to see how scraping a large site compared with multiplexing
> off. I used the -http2=off parameter
I'm going to do some more testing first. I'm not sure what changed.
On Mon, Apr 9, 2018 at 6:18 AM, Tim Rühsen wrote:
> On 04/09/2018 01:04 PM, Jeffrey Fetterman wrote:
> > So I wanted to see how scraping a large site compared with multiplexing
> > off. I used the -h
--retry_connrefused is mentioned in the documentation but it doesn't seem
to be an option anymore. I can't find a replacement for it, either. My VPN
is being a bit fussy today and I keep having to restart my script because
of 111 errors.
with --tries=5 set, Failed to connect (111) will still instantly abort the
operation.
On Tue, Apr 10, 2018 at 2:45 AM, Tim Rühsen wrote:
> On 04/10/2018 03:12 AM, Jeffrey Fetterman wrote:
> > --retry_connrefused is mentioned in the documentation but it doesn't seem
> > to
500 internal server error isn't being re-tried either.
On Tue, Apr 10, 2018 at 2:45 AM, Tim Rühsen wrote:
> On 04/10/2018 03:12 AM, Jeffrey Fetterman wrote:
> > --retry_connrefused is mentioned in the documentation but it doesn't seem
> > to be an option anymore. I can
There is so much crap in stdin that the text file for the debug output is
so big I can't even open it. It only occurs on occasion, I can't replicate
it reliably.
On Tue, Apr 10, 2018 at 3:27 PM, Tim Rühsen wrote:
>
>
> On 10.04.2018 20:37, Jeffrey Fetterman wrote:
>
So there's a directory in a site I've been using wget2 on that has a bunch
of files I don't need, but I can't figure out how to filter it out.
--exclude-directories is in the documentation but it says it's an unknown
option.
Was it replaced by a different option? How do I filter out a certain
dir
I'm using the latest version of wget2 as of 12/29/2018 (just freshly
compiled it to make sure a bug I've reported is still an issue) and I've
been running into problems.
My first problem is something I've reported on the wget2 gitlab, which is
'--convert-links' breaking if '--adjust-extension' or
ignore my last email, aside from the gitlab
issue I posted regarding --convert-links of course.
On Sat, Dec 29, 2018 at 1:51 PM Jeffrey Fetterman
wrote:
> I'm using the latest version of wget2 as of 12/29/2018 (just freshly
> compiled it to make sure a bug I've reported is still
If you specify --html-extension or --adjust-extension when downloading a
page that does not end with an extension (might also be a problem with any
site that doesn't end in .html), wget2 can't find the file to convert the
links afterward.
Can this please get looked into? It's been 3 weeks since I'
27 matches
Mail list logo
