Re: Python 2 and test/vm/netbsd
Eduardo Habkost, le ven. 18 oct. 2019 13:41:43 -0300, a ecrit: > On Fri, Oct 18, 2019 at 06:00:19PM +0200, Samuel Thibault wrote: > > It was implemented at the time of introduction of IPv6 in SLIRP. Perhaps > > NetBSD has a slightly different behavior which makes the implementation > > fail to notice the error. > > If anybody is interested in investigating it, a network traffic > dump generated by `-object filter-dump` is attached. The dump does show the Destination Unreachable icmp message, so it seems it's the guest which does not notice it for some reason. Samuel
Re: Python 2 and test/vm/netbsd
On 22.10.2019 15:16, Samuel Thibault wrote: > Eduardo Habkost, le ven. 18 oct. 2019 13:41:43 -0300, a ecrit: >> On Fri, Oct 18, 2019 at 06:00:19PM +0200, Samuel Thibault wrote: >>> It was implemented at the time of introduction of IPv6 in SLIRP. Perhaps >>> NetBSD has a slightly different behavior which makes the implementation >>> fail to notice the error. >> >> If anybody is interested in investigating it, a network traffic >> dump generated by `-object filter-dump` is attached. > > The dump does show the Destination Unreachable icmp message, so it seems > it's the guest which does not notice it for some reason. > > Samuel > I will try to investigate it on the NetBSD kernel level.
Re: Python 2 and test/vm/netbsd
Samuel Thibault, le ven. 18 oct. 2019 18:00:19 +0200, a ecrit: > Philippe Mathieu-Daudé, le ven. 18 oct. 2019 16:58:00 +0200, a ecrit: > > On 10/18/19 4:29 PM, Eduardo Habkost wrote: > > > In addition to that, the connect() error should be generating a > > > ICMP6_UNREACH message, and I'd expect the NetBSD guest to notice > > > it instead of waiting for timeout. > > > > Is this missing in SLiRP? > > It was implemented at the time of introduction of IPv6 in SLIRP. Perhaps > NetBSD has a slightly different behavior which makes the implementation > fail to notice the error. It definitely is there in tcp_input(): on tcp_fconnect() error an ICMP6_UNREACH message is sent. I can confirm that this works with a Linux host and Linux guest. Samuel
Re: Python 2 and test/vm/netbsd
On Fri, Oct 18, 2019 at 06:00:19PM +0200, Samuel Thibault wrote: > Philippe Mathieu-Daudé, le ven. 18 oct. 2019 16:58:00 +0200, a ecrit: > > On 10/18/19 4:29 PM, Eduardo Habkost wrote: > > > On Fri, Oct 18, 2019 at 12:44:39PM +0200, Gerd Hoffmann wrote: > > > >Hi, > > > > > > > > > > Running with V=1, I see packages being downloaded at reasonable > > > > > > speeds, but > > > > > > there's a huge interval (of various minutes) between each package > > > > > > download. > > > > > > > > > > I've found the cause for the slowness I'm seeing: for each file > > > > > being downloaded, the guest spents at least 75 seconds trying to > > > > > connect to the IPv6 address of ftp.NetBSD.org, before trying > > > > > IPv4. > > > > > > > > Ah, that nicely explains why it worked just fine for me. First, I have > > > > a local proxy configured so the installer isn't going to connect to > > > > ftp.NetBSD.org directly. Second I have IPv6 connectivity. > > > > > > > > > I don't know if this is a NetBSD bug, or a slirp bug. > > > > > > > > Both I'd say ... > > > > > > > > First, by default slirp should not send IPv6 router announcements > > > > to the user network if the host has no IPv6 connectivity. > > > > > > > > Second, the recommended way to connect is to try ipv4 and ipv6 in > > > > parallel, then use whatever connects first. Web browsers typically > > > > do it that way. wget and curl don't do that though, they try one > > > > address after the other, and I guess this is where the delay comes > > > > from ... > > > > > > In addition to that, the connect() error should be generating a > > > ICMP6_UNREACH message, and I'd expect the NetBSD guest to notice > > > it instead of waiting for timeout. > > > > Is this missing in SLiRP? > > It was implemented at the time of introduction of IPv6 in SLIRP. Perhaps > NetBSD has a slightly different behavior which makes the implementation > fail to notice the error. If anybody is interested in investigating it, a network traffic dump generated by `-object filter-dump` is attached. -- Eduardo dump.pcap.gz Description: application/gzip
Re: Python 2 and test/vm/netbsd
Philippe Mathieu-Daudé, le ven. 18 oct. 2019 16:58:00 +0200, a ecrit: > On 10/18/19 4:29 PM, Eduardo Habkost wrote: > > On Fri, Oct 18, 2019 at 12:44:39PM +0200, Gerd Hoffmann wrote: > > >Hi, > > > > > > > > Running with V=1, I see packages being downloaded at reasonable > > > > > speeds, but > > > > > there's a huge interval (of various minutes) between each package > > > > > download. > > > > > > > > I've found the cause for the slowness I'm seeing: for each file > > > > being downloaded, the guest spents at least 75 seconds trying to > > > > connect to the IPv6 address of ftp.NetBSD.org, before trying > > > > IPv4. > > > > > > Ah, that nicely explains why it worked just fine for me. First, I have > > > a local proxy configured so the installer isn't going to connect to > > > ftp.NetBSD.org directly. Second I have IPv6 connectivity. > > > > > > > I don't know if this is a NetBSD bug, or a slirp bug. > > > > > > Both I'd say ... > > > > > > First, by default slirp should not send IPv6 router announcements > > > to the user network if the host has no IPv6 connectivity. > > > > > > Second, the recommended way to connect is to try ipv4 and ipv6 in > > > parallel, then use whatever connects first. Web browsers typically > > > do it that way. wget and curl don't do that though, they try one > > > address after the other, and I guess this is where the delay comes > > > from ... > > > > In addition to that, the connect() error should be generating a > > ICMP6_UNREACH message, and I'd expect the NetBSD guest to notice > > it instead of waiting for timeout. > > Is this missing in SLiRP? It was implemented at the time of introduction of IPv6 in SLIRP. Perhaps NetBSD has a slightly different behavior which makes the implementation fail to notice the error. Samuel
Re: Python 2 and test/vm/netbsd
On 10/18/19 4:29 PM, Eduardo Habkost wrote: On Fri, Oct 18, 2019 at 12:44:39PM +0200, Gerd Hoffmann wrote: Hi, Running with V=1, I see packages being downloaded at reasonable speeds, but there's a huge interval (of various minutes) between each package download. I've found the cause for the slowness I'm seeing: for each file being downloaded, the guest spents at least 75 seconds trying to connect to the IPv6 address of ftp.NetBSD.org, before trying IPv4. Ah, that nicely explains why it worked just fine for me. First, I have a local proxy configured so the installer isn't going to connect to ftp.NetBSD.org directly. Second I have IPv6 connectivity. I don't know if this is a NetBSD bug, or a slirp bug. Both I'd say ... First, by default slirp should not send IPv6 router announcements to the user network if the host has no IPv6 connectivity. Second, the recommended way to connect is to try ipv4 and ipv6 in parallel, then use whatever connects first. Web browsers typically do it that way. wget and curl don't do that though, they try one address after the other, and I guess this is where the delay comes from ... In addition to that, the connect() error should be generating a ICMP6_UNREACH message, and I'd expect the NetBSD guest to notice it instead of waiting for timeout. Is this missing in SLiRP?
Re: Python 2 and test/vm/netbsd
On Fri, Oct 18, 2019 at 09:13:24AM +0200, Thomas Huth wrote: > On 18/10/2019 00.55, Eduardo Habkost wrote: > > On Thu, Oct 17, 2019 at 07:05:41PM -0300, Eduardo Habkost wrote: > >> On Wed, Oct 16, 2019 at 07:41:24PM -0300, Eduardo Habkost wrote: > >>> On Wed, Oct 16, 2019 at 08:11:57AM +0200, Thomas Huth wrote: > On 16/10/2019 05.00, Eduardo Habkost wrote: > > On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote: > >> On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote: > >>> On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote: > > [...] > The configure check also spits out deprecation warnings for > the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice > to get those updated. > >>> > >>> CCing the test/vm maintainers. > >>> > >>> Fam, Alex, are you able to fix this and create new BSD VM images > >>> with Python 3 available? I thought the VM image configurations > >>> were stored in the source tree, but they are downloaded from > >>> download.patchew.org. > >> > >> Fam, Alex, can you help us on this? Python 2 won't be supported > >> anymore, so we need the VM images to be updated. > > > > Anyone? > > > > I'm about to submit patches to remove Python 2 support, and this > > will break tests/vm/netbsd. > > > > I'm powerless to fix this issue, because the netbsd image is > > hosted at download.patchew.org. > > Gerd had a patch to convert the netbsd VM script to ad hoc image > creation, too: > > https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html > > But there was a regression with the serial port between QEMU v3.0 and > v4.x, so it was not included: > > https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html > >>> > >>> The URL above has this error: > >>> > >>> con recv: x: Exitqj > >>> con recv: To be able to use the network, we need answers to the > >>> following:Network media type > >>> con send: > >>> con recv: : qqqk Perform autoconfiguration? > >>> >a: Yes b: Noq > >>> console: *** read timeout *** > >>> console: waiting for: 'a: Yes' > >>> console: line buffer: > >>> > >>> con recv: qqj > >>> > >>> I believe that problem was solved in v4, because v4 was reading > >>> the serial output 1 byte at a time. > >>> > >>> The issue that caused the netbsd patch to be dropped was: > >>> https://lore.kernel.org/qemu-devel/cafeaca8k9qja9ie-kwiaphr0fy_2zg7jrx5uv4aassjxcss...@mail.gmail.com/ > >>> > >>> Possibly this is the same issue we saw at: > >>> https://lore.kernel.org/qemu-devel/20190607034214.gb22...@habkost.net/ > >>> > >>> The test script must either close the console socket, or keep > >>> reading from it. Otherwise, the QEMU VCPU threads might get > >>> stuck waiting for the chardev to be writeable. > >> > >> It doesn't seem to be the same issue. Even if the console socket is > >> closed, > >> I'm seeing results similar to the ones reported by Peter (the "pkgin -y > >> install" step is unreasonably slow). > >> > >> Running with V=1, I see packages being downloaded at reasonable speeds, but > >> there's a huge interval (of various minutes) between each package download. > > > > I've found the cause for the slowness I'm seeing: for each file > > being downloaded, the guest spents at least 75 seconds trying to > > connect to the IPv6 address of ftp.NetBSD.org, before trying > > IPv4. I don't know if this is a NetBSD bug, or a slirp bug. > > Does it work better if you turn IPv6 off? E.g.: > > diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py > --- a/tests/vm/basevm.py > +++ b/tests/vm/basevm.py > @@ -81,7 +81,7 @@ class BaseVM(object): > self._args = [ \ > "-nodefaults", "-m", "4G", > "-cpu", "max", > -"-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22", > +"-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22,ipv6=off", > "-device", "virtio-net-pci,netdev=vnet", > "-vnc", "127.0.0.1:0,to=20"] > if vcpus and vcpus > 1: Yes, it is much better. Thanks! I will send a series disabling ipv6 in tests/vm/netbsd as a workaround. -- Eduardo
Re: Python 2 and test/vm/netbsd
On Fri, Oct 18, 2019 at 12:44:39PM +0200, Gerd Hoffmann wrote: > Hi, > > > > Running with V=1, I see packages being downloaded at reasonable speeds, > > > but > > > there's a huge interval (of various minutes) between each package > > > download. > > > > I've found the cause for the slowness I'm seeing: for each file > > being downloaded, the guest spents at least 75 seconds trying to > > connect to the IPv6 address of ftp.NetBSD.org, before trying > > IPv4. > > Ah, that nicely explains why it worked just fine for me. First, I have > a local proxy configured so the installer isn't going to connect to > ftp.NetBSD.org directly. Second I have IPv6 connectivity. > > > I don't know if this is a NetBSD bug, or a slirp bug. > > Both I'd say ... > > First, by default slirp should not send IPv6 router announcements > to the user network if the host has no IPv6 connectivity. > > Second, the recommended way to connect is to try ipv4 and ipv6 in > parallel, then use whatever connects first. Web browsers typically > do it that way. wget and curl don't do that though, they try one > address after the other, and I guess this is where the delay comes > from ... In addition to that, the connect() error should be generating a ICMP6_UNREACH message, and I'd expect the NetBSD guest to notice it instead of waiting for timeout. -- Eduardo
Re: Python 2 and test/vm/netbsd
Hi, > > Running with V=1, I see packages being downloaded at reasonable speeds, but > > there's a huge interval (of various minutes) between each package download. > > I've found the cause for the slowness I'm seeing: for each file > being downloaded, the guest spents at least 75 seconds trying to > connect to the IPv6 address of ftp.NetBSD.org, before trying > IPv4. Ah, that nicely explains why it worked just fine for me. First, I have a local proxy configured so the installer isn't going to connect to ftp.NetBSD.org directly. Second I have IPv6 connectivity. > I don't know if this is a NetBSD bug, or a slirp bug. Both I'd say ... First, by default slirp should not send IPv6 router announcements to the user network if the host has no IPv6 connectivity. Second, the recommended way to connect is to try ipv4 and ipv6 in parallel, then use whatever connects first. Web browsers typically do it that way. wget and curl don't do that though, they try one address after the other, and I guess this is where the delay comes from ... cheers, Gerd
Re: Python 2 and test/vm/netbsd
On 18/10/2019 00.55, Eduardo Habkost wrote:
> On Thu, Oct 17, 2019 at 07:05:41PM -0300, Eduardo Habkost wrote:
>> On Wed, Oct 16, 2019 at 07:41:24PM -0300, Eduardo Habkost wrote:
>>> On Wed, Oct 16, 2019 at 08:11:57AM +0200, Thomas Huth wrote:
On 16/10/2019 05.00, Eduardo Habkost wrote:
> On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote:
>> On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote:
>>> On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote:
> [...]
The configure check also spits out deprecation warnings for
the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice
to get those updated.
>>>
>>> CCing the test/vm maintainers.
>>>
>>> Fam, Alex, are you able to fix this and create new BSD VM images
>>> with Python 3 available? I thought the VM image configurations
>>> were stored in the source tree, but they are downloaded from
>>> download.patchew.org.
>>
>> Fam, Alex, can you help us on this? Python 2 won't be supported
>> anymore, so we need the VM images to be updated.
>
> Anyone?
>
> I'm about to submit patches to remove Python 2 support, and this
> will break tests/vm/netbsd.
>
> I'm powerless to fix this issue, because the netbsd image is
> hosted at download.patchew.org.
Gerd had a patch to convert the netbsd VM script to ad hoc image
creation, too:
https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html
But there was a regression with the serial port between QEMU v3.0 and
v4.x, so it was not included:
https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html
>>>
>>> The URL above has this error:
>>>
>>> con recv: x: Exitqj
>>> con recv: To be able to use the network, we need answers to the
>>> following:Network media type
>>> con send:
>>> con recv: : qqqk Perform autoconfiguration?
>>> >a: Yes b: Noq
>>> console: *** read timeout ***
>>> console: waiting for: 'a: Yes'
>>> console: line buffer:
>>>
>>> con recv: qqj
>>>
>>> I believe that problem was solved in v4, because v4 was reading
>>> the serial output 1 byte at a time.
>>>
>>> The issue that caused the netbsd patch to be dropped was:
>>> https://lore.kernel.org/qemu-devel/cafeaca8k9qja9ie-kwiaphr0fy_2zg7jrx5uv4aassjxcss...@mail.gmail.com/
>>>
>>> Possibly this is the same issue we saw at:
>>> https://lore.kernel.org/qemu-devel/20190607034214.gb22...@habkost.net/
>>>
>>> The test script must either close the console socket, or keep
>>> reading from it. Otherwise, the QEMU VCPU threads might get
>>> stuck waiting for the chardev to be writeable.
>>
>> It doesn't seem to be the same issue. Even if the console socket is closed,
>> I'm seeing results similar to the ones reported by Peter (the "pkgin -y
>> install" step is unreasonably slow).
>>
>> Running with V=1, I see packages being downloaded at reasonable speeds, but
>> there's a huge interval (of various minutes) between each package download.
>
> I've found the cause for the slowness I'm seeing: for each file
> being downloaded, the guest spents at least 75 seconds trying to
> connect to the IPv6 address of ftp.NetBSD.org, before trying
> IPv4. I don't know if this is a NetBSD bug, or a slirp bug.
Does it work better if you turn IPv6 off? E.g.:
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -81,7 +81,7 @@ class BaseVM(object):
self._args = [ \
"-nodefaults", "-m", "4G",
"-cpu", "max",
-"-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
+"-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22,ipv6=off",
"-device", "virtio-net-pci,netdev=vnet",
"-vnc", "127.0.0.1:0,to=20"]
if vcpus and vcpus > 1:
Thomas
> Output of `strace -e trace=network` below:
>
> 1571352260.348566 recvfrom(30,
> "~[\201\200\0\1\0\1\0\0\0\0\3ftp\6NetBSD\3org\0\0\1\0\1"..., 1500, 0,
> {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.5.30.160")},
> [128->16]) = 48 <0.16>
> 1571352260.349030 socket(AF_INET6, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_IP) = 31
> <0.41>
> 1571352260.349142 setsockopt(31, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
> <0.09>
> 1571352260.349179 setsockopt(31, SOL_SOCKET, SO_OOBINLINE, [1], 4) = 0
> <0.07>
> 1571352260.349207 setsockopt(31, SOL_TCP, TCP_NODELAY, [1], 4) = 0 <0.08>
> 1571352260.349239 connect(31, {sa_family=AF_INET6, sin6_port=htons(80),
> sin6_flowinfo=htonl(67108864), inet_pton(AF_INET6, "2001:470:a085:999::21",
> _addr), sin6_scope_id=377348480}, 28) = -1 ENETUNREACH (Network is
> unreachable) <0.000
> 021>
> 1571352266.350112 socket(AF_INET6, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_IP) = 31
> <0.000131>
> 1571352266.350603 setsockopt(31,
Re: Python 2 and test/vm/netbsd
On Thu, Oct 17, 2019 at 07:05:41PM -0300, Eduardo Habkost wrote:
> On Wed, Oct 16, 2019 at 07:41:24PM -0300, Eduardo Habkost wrote:
> > On Wed, Oct 16, 2019 at 08:11:57AM +0200, Thomas Huth wrote:
> > > On 16/10/2019 05.00, Eduardo Habkost wrote:
> > > > On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote:
> > > >> On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote:
> > > >>> On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote:
> > > > [...]
> > > The configure check also spits out deprecation warnings for
> > > the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice
> > > to get those updated.
> > > >>>
> > > >>> CCing the test/vm maintainers.
> > > >>>
> > > >>> Fam, Alex, are you able to fix this and create new BSD VM images
> > > >>> with Python 3 available? I thought the VM image configurations
> > > >>> were stored in the source tree, but they are downloaded from
> > > >>> download.patchew.org.
> > > >>
> > > >> Fam, Alex, can you help us on this? Python 2 won't be supported
> > > >> anymore, so we need the VM images to be updated.
> > > >
> > > > Anyone?
> > > >
> > > > I'm about to submit patches to remove Python 2 support, and this
> > > > will break tests/vm/netbsd.
> > > >
> > > > I'm powerless to fix this issue, because the netbsd image is
> > > > hosted at download.patchew.org.
> > >
> > > Gerd had a patch to convert the netbsd VM script to ad hoc image
> > > creation, too:
> > >
> > > https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html
> > >
> > > But there was a regression with the serial port between QEMU v3.0 and
> > > v4.x, so it was not included:
> > >
> > > https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html
> >
> > The URL above has this error:
> >
> > con recv: x: Exitqj
> > con recv: To be able to use the network, we need answers to the
> > following:Network media type
> > con send:
> > con recv: : qqqk Perform autoconfiguration?
> > >a: Yes b: Noq
> > console: *** read timeout ***
> > console: waiting for: 'a: Yes'
> > console: line buffer:
> >
> > con recv: qqj
> >
> > I believe that problem was solved in v4, because v4 was reading
> > the serial output 1 byte at a time.
> >
> > The issue that caused the netbsd patch to be dropped was:
> > https://lore.kernel.org/qemu-devel/cafeaca8k9qja9ie-kwiaphr0fy_2zg7jrx5uv4aassjxcss...@mail.gmail.com/
> >
> > Possibly this is the same issue we saw at:
> > https://lore.kernel.org/qemu-devel/20190607034214.gb22...@habkost.net/
> >
> > The test script must either close the console socket, or keep
> > reading from it. Otherwise, the QEMU VCPU threads might get
> > stuck waiting for the chardev to be writeable.
>
> It doesn't seem to be the same issue. Even if the console socket is closed,
> I'm seeing results similar to the ones reported by Peter (the "pkgin -y
> install" step is unreasonably slow).
>
> Running with V=1, I see packages being downloaded at reasonable speeds, but
> there's a huge interval (of various minutes) between each package download.
I've found the cause for the slowness I'm seeing: for each file
being downloaded, the guest spents at least 75 seconds trying to
connect to the IPv6 address of ftp.NetBSD.org, before trying
IPv4. I don't know if this is a NetBSD bug, or a slirp bug.
Output of `strace -e trace=network` below:
1571352260.348566 recvfrom(30,
"~[\201\200\0\1\0\1\0\0\0\0\3ftp\6NetBSD\3org\0\0\1\0\1"..., 1500, 0,
{sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.5.30.160")},
[128->16]) = 48 <0.16>
1571352260.349030 socket(AF_INET6, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_IP) = 31
<0.41>
1571352260.349142 setsockopt(31, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
<0.09>
1571352260.349179 setsockopt(31, SOL_SOCKET, SO_OOBINLINE, [1], 4) = 0
<0.07>
1571352260.349207 setsockopt(31, SOL_TCP, TCP_NODELAY, [1], 4) = 0 <0.08>
1571352260.349239 connect(31, {sa_family=AF_INET6, sin6_port=htons(80),
sin6_flowinfo=htonl(67108864), inet_pton(AF_INET6, "2001:470:a085:999::21",
_addr), sin6_scope_id=377348480}, 28) = -1 ENETUNREACH (Network is
unreachable) <0.000
021>
1571352266.350112 socket(AF_INET6, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_IP) = 31
<0.000131>
1571352266.350603 setsockopt(31, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
<0.000183>
1571352266.350946 setsockopt(31, SOL_SOCKET, SO_OOBINLINE, [1], 4) = 0
<0.47>
1571352266.351109 setsockopt(31, SOL_TCP, TCP_NODELAY, [1], 4) = 0 <0.43>
1571352266.351260 connect(31, {sa_family=AF_INET6, sin6_port=htons(80),
sin6_flowinfo=htonl(67108864), inet_pton(AF_INET6, "2001:470:a085:999::21",
_addr), sin6_scope_id=377348480}, 28) = -1 ENETUNREACH (Network is
unreachable) <0.70>
1571352278.357962 socket(AF_INET6, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_IP) = 29
<0.000166>
1571352278.358524 setsockopt(29, SOL_SOCKET, SO_REUSEADDR,
Re: Python 2 and test/vm/netbsd
On Wed, Oct 16, 2019 at 07:41:24PM -0300, Eduardo Habkost wrote: > On Wed, Oct 16, 2019 at 08:11:57AM +0200, Thomas Huth wrote: > > On 16/10/2019 05.00, Eduardo Habkost wrote: > > > On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote: > > >> On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote: > > >>> On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote: > > > [...] > > The configure check also spits out deprecation warnings for > > the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice > > to get those updated. > > >>> > > >>> CCing the test/vm maintainers. > > >>> > > >>> Fam, Alex, are you able to fix this and create new BSD VM images > > >>> with Python 3 available? I thought the VM image configurations > > >>> were stored in the source tree, but they are downloaded from > > >>> download.patchew.org. > > >> > > >> Fam, Alex, can you help us on this? Python 2 won't be supported > > >> anymore, so we need the VM images to be updated. > > > > > > Anyone? > > > > > > I'm about to submit patches to remove Python 2 support, and this > > > will break tests/vm/netbsd. > > > > > > I'm powerless to fix this issue, because the netbsd image is > > > hosted at download.patchew.org. > > > > Gerd had a patch to convert the netbsd VM script to ad hoc image > > creation, too: > > > > https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html > > > > But there was a regression with the serial port between QEMU v3.0 and > > v4.x, so it was not included: > > > > https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html > > The URL above has this error: > > con recv: x: Exitqj > con recv: To be able to use the network, we need answers to the > following:Network media type > con send: > con recv: : qqqk Perform autoconfiguration? > >a: Yes b: Noq > console: *** read timeout *** > console: waiting for: 'a: Yes' > console: line buffer: > > con recv: qqj > > I believe that problem was solved in v4, because v4 was reading > the serial output 1 byte at a time. > > The issue that caused the netbsd patch to be dropped was: > https://lore.kernel.org/qemu-devel/cafeaca8k9qja9ie-kwiaphr0fy_2zg7jrx5uv4aassjxcss...@mail.gmail.com/ > > Possibly this is the same issue we saw at: > https://lore.kernel.org/qemu-devel/20190607034214.gb22...@habkost.net/ > > The test script must either close the console socket, or keep > reading from it. Otherwise, the QEMU VCPU threads might get > stuck waiting for the chardev to be writeable. It doesn't seem to be the same issue. Even if the console socket is closed, I'm seeing results similar to the ones reported by Peter (the "pkgin -y install" step is unreasonably slow). Running with V=1, I see packages being downloaded at reasonable speeds, but there's a huge interval (of various minutes) between each package download. -- Eduardo
Re: Python 2 and test/vm/netbsd
On Wed, Oct 16, 2019 at 08:11:57AM +0200, Thomas Huth wrote: > On 16/10/2019 05.00, Eduardo Habkost wrote: > > On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote: > >> On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote: > >>> On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote: > > [...] > The configure check also spits out deprecation warnings for > the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice > to get those updated. > >>> > >>> CCing the test/vm maintainers. > >>> > >>> Fam, Alex, are you able to fix this and create new BSD VM images > >>> with Python 3 available? I thought the VM image configurations > >>> were stored in the source tree, but they are downloaded from > >>> download.patchew.org. > >> > >> Fam, Alex, can you help us on this? Python 2 won't be supported > >> anymore, so we need the VM images to be updated. > > > > Anyone? > > > > I'm about to submit patches to remove Python 2 support, and this > > will break tests/vm/netbsd. > > > > I'm powerless to fix this issue, because the netbsd image is > > hosted at download.patchew.org. > > Gerd had a patch to convert the netbsd VM script to ad hoc image > creation, too: > > https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html > > But there was a regression with the serial port between QEMU v3.0 and > v4.x, so it was not included: > > https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html The URL above has this error: con recv: x: Exitqj con recv: To be able to use the network, we need answers to the following:Network media type con send: con recv: : qqqk Perform autoconfiguration? >a: Yes b: Noq console: *** read timeout *** console: waiting for: 'a: Yes' console: line buffer: con recv: qqj I believe that problem was solved in v4, because v4 was reading the serial output 1 byte at a time. The issue that caused the netbsd patch to be dropped was: https://lore.kernel.org/qemu-devel/cafeaca8k9qja9ie-kwiaphr0fy_2zg7jrx5uv4aassjxcss...@mail.gmail.com/ Possibly this is the same issue we saw at: https://lore.kernel.org/qemu-devel/20190607034214.gb22...@habkost.net/ The test script must either close the console socket, or keep reading from it. Otherwise, the QEMU VCPU threads might get stuck waiting for the chardev to be writeable. -- Eduardo
Re: Python 2 and test/vm/netbsd
On 10/16/19 12:59 PM, Alex Bennée wrote: Kamil Rytarowski writes: On 16.10.2019 08:11, Thomas Huth wrote: On 16/10/2019 05.00, Eduardo Habkost wrote: On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote: On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote: On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote: [...] The configure check also spits out deprecation warnings for the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice to get those updated. CCing the test/vm maintainers. Fam, Alex, are you able to fix this and create new BSD VM images with Python 3 available? I thought the VM image configurations were stored in the source tree, but they are downloaded from download.patchew.org. Fam, Alex, can you help us on this? Python 2 won't be supported anymore, so we need the VM images to be updated. Anyone? I'm about to submit patches to remove Python 2 support, and this will break tests/vm/netbsd. I'm powerless to fix this issue, because the netbsd image is hosted at download.patchew.org. Gerd had a patch to convert the netbsd VM script to ad hoc image creation, too: https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html But there was a regression with the serial port between QEMU v3.0 and v4.x, so it was not included: https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html I guess someone™ needs to bisect that regression, so we can fix that bug and finally include Gerd's patch... Thomas Is this a regression in qemu? How to reproduce the problem? "make vm-build-netbsd V=1" ? You'll need to apply the patch from that series: tests/vm: netbsd autoinstall, using serial (all the others got merged) I can have a look but I need to know exact specifics of the problem. IIRC this is not a NetBSD specific issue, but the NetBSD image triggers the chardev bug reliably. Make sure you've cleared out any cached images. As was mentioned in the thread it seems to be a little host dependant - some host systems it was working and some it was not.
Re: Python 2 and test/vm/netbsd
On 16/10/2019 12.59, Alex Bennée wrote: > > Kamil Rytarowski writes: > >> On 16.10.2019 08:11, Thomas Huth wrote: >>> On 16/10/2019 05.00, Eduardo Habkost wrote: On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote: > On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote: >> On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote: [...] >>> The configure check also spits out deprecation warnings for >>> the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice >>> to get those updated. >> >> CCing the test/vm maintainers. >> >> Fam, Alex, are you able to fix this and create new BSD VM images >> with Python 3 available? I thought the VM image configurations >> were stored in the source tree, but they are downloaded from >> download.patchew.org. > > Fam, Alex, can you help us on this? Python 2 won't be supported > anymore, so we need the VM images to be updated. Anyone? I'm about to submit patches to remove Python 2 support, and this will break tests/vm/netbsd. I'm powerless to fix this issue, because the netbsd image is hosted at download.patchew.org. >>> >>> Gerd had a patch to convert the netbsd VM script to ad hoc image >>> creation, too: >>> >>> https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html >>> >>> But there was a regression with the serial port between QEMU v3.0 and >>> v4.x, so it was not included: >>> >>> https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html >>> >>> I guess someone™ needs to bisect that regression, so we can fix that bug >>> and finally include Gerd's patch... >>> >>> Thomas >>> >> >> Is this a regression in qemu? How to reproduce the problem? "make >> vm-build-netbsd V=1" ? > > You'll need to apply the patch from that series: > > tests/vm: netbsd autoinstall, using serial Patch (mbox for applying with "git am") is still available in Patchew if you don't have it in your local qemu-devel folder anymore: http://next.patchew.org/QEMU/20190520124716.30472-13-kra...@redhat.com/mbox Thomas
Re: Python 2 and test/vm/netbsd
Kamil Rytarowski writes: > On 16.10.2019 08:11, Thomas Huth wrote: >> On 16/10/2019 05.00, Eduardo Habkost wrote: >>> On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote: On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote: > On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote: >>> [...] >> The configure check also spits out deprecation warnings for >> the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice >> to get those updated. > > CCing the test/vm maintainers. > > Fam, Alex, are you able to fix this and create new BSD VM images > with Python 3 available? I thought the VM image configurations > were stored in the source tree, but they are downloaded from > download.patchew.org. Fam, Alex, can you help us on this? Python 2 won't be supported anymore, so we need the VM images to be updated. >>> >>> Anyone? >>> >>> I'm about to submit patches to remove Python 2 support, and this >>> will break tests/vm/netbsd. >>> >>> I'm powerless to fix this issue, because the netbsd image is >>> hosted at download.patchew.org. >> >> Gerd had a patch to convert the netbsd VM script to ad hoc image >> creation, too: >> >> https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html >> >> But there was a regression with the serial port between QEMU v3.0 and >> v4.x, so it was not included: >> >> https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html >> >> I guess someone™ needs to bisect that regression, so we can fix that bug >> and finally include Gerd's patch... >> >> Thomas >> > > Is this a regression in qemu? How to reproduce the problem? "make > vm-build-netbsd V=1" ? You'll need to apply the patch from that series: tests/vm: netbsd autoinstall, using serial (all the others got merged) > I can have a look but I need to know exact specifics of the problem. Make sure you've cleared out any cached images. As was mentioned in the thread it seems to be a little host dependant - some host systems it was working and some it was not. -- Alex Bennée
Re: Python 2 and test/vm/netbsd
On 16.10.2019 08:11, Thomas Huth wrote: On 16/10/2019 05.00, Eduardo Habkost wrote: On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote: On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote: On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote: [...] The configure check also spits out deprecation warnings for the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice to get those updated. CCing the test/vm maintainers. Fam, Alex, are you able to fix this and create new BSD VM images with Python 3 available? I thought the VM image configurations were stored in the source tree, but they are downloaded from download.patchew.org. Fam, Alex, can you help us on this? Python 2 won't be supported anymore, so we need the VM images to be updated. Anyone? I'm about to submit patches to remove Python 2 support, and this will break tests/vm/netbsd. I'm powerless to fix this issue, because the netbsd image is hosted at download.patchew.org. Gerd had a patch to convert the netbsd VM script to ad hoc image creation, too: https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html But there was a regression with the serial port between QEMU v3.0 and v4.x, so it was not included: https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html I guess someone™ needs to bisect that regression, so we can fix that bug and finally include Gerd's patch... Thomas Is this a regression in qemu? How to reproduce the problem? "make vm-build-netbsd V=1" ? I can have a look but I need to know exact specifics of the problem.
Re: Python 2 and test/vm/netbsd
On 16/10/2019 05.00, Eduardo Habkost wrote: > On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote: >> On Mon, Jul 01, 2019 at 07:25:27PM -0300, Eduardo Habkost wrote: >>> On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote: > [...] The configure check also spits out deprecation warnings for the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice to get those updated. >>> >>> CCing the test/vm maintainers. >>> >>> Fam, Alex, are you able to fix this and create new BSD VM images >>> with Python 3 available? I thought the VM image configurations >>> were stored in the source tree, but they are downloaded from >>> download.patchew.org. >> >> Fam, Alex, can you help us on this? Python 2 won't be supported >> anymore, so we need the VM images to be updated. > > Anyone? > > I'm about to submit patches to remove Python 2 support, and this > will break tests/vm/netbsd. > > I'm powerless to fix this issue, because the netbsd image is > hosted at download.patchew.org. Gerd had a patch to convert the netbsd VM script to ad hoc image creation, too: https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg04459.html But there was a regression with the serial port between QEMU v3.0 and v4.x, so it was not included: https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06784.html I guess someone™ needs to bisect that regression, so we can fix that bug and finally include Gerd's patch... Thomas
