Re: virtio support
Ritesh Raj Sarraf, on Mon 08 Feb 2016 19:09:49 +0530, wrote: > Does the current state of Debian GNU/HUrd support virtio drivers ? No, the emulated devices work fine for now, we haven't taken the time to write virtio drivers. Samuel
virtio support
Hello, Does the current state of Debian GNU/HUrd support virtio drivers ? I haven't tracked Hurd status and just wanted to check here first. -- Given the large number of mailing lists I follow, I request you to CC me in replies for quicker response signature.asc Description: This is a digitally signed message part
Processed: [bts-link] source package hurd
Processing commands for cont...@bugs.debian.org: > # > # bts-link upstream status pull for source package hurd > # see http://lists.debian.org/debian-devel-announce/2006/05/msg1.html > # > user bts-link-upstr...@lists.alioth.debian.org Setting user to bts-link-upstr...@lists.alioth.debian.org (was bts-link-de...@lists.alioth.debian.org). > # remote status report for #42079 (http://bugs.debian.org/42079) > # Bug title: hurd: exec doesn't like zip'ed binaries > # * http://savannah.gnu.org/bugs/?func=detailitem_id=15329 > # * remote status changed: Open -> Closed > usertags 42079 - status-Open Usertags were: resolution-Wont-Fix status-Open. Usertags are now: resolution-Wont-Fix. > usertags 42079 + status-Closed Usertags were: resolution-Wont-Fix. Usertags are now: resolution-Wont-Fix status-Closed. > # remote status report for #47998 (http://bugs.debian.org/47998) > # Bug title: msgget IPC not implemented > # * http://savannah.gnu.org/bugs/?func=detailitem_id=11509 > # * remote status changed: Open -> Closed > # * remote resolution changed: Confirmed -> Fixed > # * closed upstream > tags 47998 + fixed-upstream Bug #47998 [hurd] msgget IPC not implemented Added tag(s) fixed-upstream. > usertags 47998 - status-Open resolution-Confirmed Usertags were: status-Open resolution-Confirmed. Usertags are now: . > usertags 47998 + status-Closed resolution-Fixed There were no usertags set. Usertags are now: status-Closed resolution-Fixed. > thanks Stopping processing here. Please contact me if you need assistance. -- 47998: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=47998 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Re: virtio support
On Mon, 2016-02-08 at 23:24 +0530, Ritesh Raj Sarraf wrote: > In today's upgrade, 2 observations I have that I'd like to share. > > 1) I'm using the rtl8139 network driver emulated with qemu. I fetched > some data (around 200 MiB) over the network from the host (Linux) > machine. The network speed I got was average 250 KiB/Sec And I think part of the reason may be the emulation. Especially for the block device, when you do I/O, the CPU usage is persistently 100%. Right now, I tried again, with just 1 VM i.e. Hurd. And I got a better performance. Close to 700 KiB/Sec. Which will degrade as an when contention increases either on the Host or the Guest. -- Given the large number of mailing lists I follow, I request you to CC me in replies for quicker response signature.asc Description: This is a digitally signed message part
Re: virtio support
Ritesh Raj Sarraf, on Mon 08 Feb 2016 23:33:53 +0530, wrote: > On Mon, 2016-02-08 at 23:24 +0530, Ritesh Raj Sarraf wrote: > > In today's upgrade, 2 observations I have that I'd like to share. > > > > 1) I'm using the rtl8139 network driver emulated with qemu. I fetched > > some data (around 200 MiB) over the network from the host (Linux) > > machine. The network speed I got was average 250 KiB/Sec > > And I think part of the reason may be the emulation. Especially for the > block device, when you do I/O, the CPU usage is persistently 100%. That's way too much. On my laptop (nothing fancy, just a Core i5), I typically get 15MB/s transfers. Again, do you really have KVM enabled? Samuel
Re: virtio support
Ritesh Raj Sarraf, on Mon 08 Feb 2016 23:24:30 +0530, wrote: > When you say "emulated devices work fine", is it as in they kinda work > ? They just work. > Sorry. I ask this because I have my Debian GNU/Hurd setup, that at > times, just breaks apart during upgrades. What are the symptoms? There may be simply memory management issues for instance. > 1) I'm using the rtl8139 network driver emulated with qemu. I fetched > some data (around 200 MiB) over the network from the host (Linux) > machine. The network speed I got was average 250 KiB/Sec That's very odd. Our buildd daemons typically get several MiB/sec from the Internet. Do you have KVM really enabled? > 2) For the block driver, I'm still on IDE. I believe reading some > documentation from you that now SATA is supported. But the last time I > tried with SATA, things did not work that great. With IDE drives, a > common problem is I/O Throughput. SATA won't be better: the protocol is the same, only the channel is different. > But more than that, during (big) upgrades, most of the times the file > system or block translator hangs. That's most probably rather issues in memory management or ext2fs. How much memory did you give to the VM? (I'm surprised to see such reports, when buildds do go fine for days long installing/removing software) Samuel
Re: virtio support
Samuel Thibault, on Mon 08 Feb 2016 19:09:48 +0100, wrote: > Ritesh Raj Sarraf, on Mon 08 Feb 2016 23:33:53 +0530, wrote: > > On Mon, 2016-02-08 at 23:24 +0530, Ritesh Raj Sarraf wrote: > > > In today's upgrade, 2 observations I have that I'd like to share. > > > > > > 1) I'm using the rtl8139 network driver emulated with qemu. I fetched > > > some data (around 200 MiB) over the network from the host (Linux) > > > machine. The network speed I got was average 250 KiB/Sec > > > > And I think part of the reason may be the emulation. Especially for the > > block device, when you do I/O, the CPU usage is persistently 100%. > > That's way too much. On my laptop (nothing fancy, just a Core i5), I > typically get 15MB/s transfers. > > Again, do you really have KVM enabled? Also, do you use cache=writeback as advised in our documentations? Samuel
Re: virtio support
On Mon, 2016-02-08 at 16:54 +0100, Samuel Thibault wrote: > Ritesh Raj Sarraf, on Mon 08 Feb 2016 19:09:49 +0530, wrote: > > Does the current state of Debian GNU/HUrd support virtio drivers ? > > No, the emulated devices work fine for now, we haven't taken the time > to > write virtio drivers. Hello Samuel. Thank you for confirming the state of drivers. When you say "emulated devices work fine", is it as in they kinda work ? Sorry. I ask this because I have my Debian GNU/Hurd setup, that at times, just breaks apart during upgrades. Luckily, this time I do have a snapshot of a working image, to which I can revert back to when things break. In today's upgrade, 2 observations I have that I'd like to share. 1) I'm using the rtl8139 network driver emulated with qemu. I fetched some data (around 200 MiB) over the network from the host (Linux) machine. The network speed I got was average 250 KiB/Sec 2) For the block driver, I'm still on IDE. I believe reading some documentation from you that now SATA is supported. But the last time I tried with SATA, things did not work that great. With IDE drives, a common problem is I/O Throughput. But more than that, during (big) upgrades, most of the times the file system or block translator hangs. That leaves the user with no choice but to reset the VM, which then is screwed up on next boot because then the file system is inconsistent and mostly does not repair. I haven't file bug reports because just taking a screenshot might not be useful. How do you debug these issues ? Is there any hidden knob in GNU/Hurd or Qemu that we can use to do a better root cause ? -- Given the large number of mailing lists I follow, I request you to CC me in replies for quicker response signature.asc Description: This is a digitally signed message part
Re: virtio support
On Mon, 2016-02-08 at 19:12 +0100, Samuel Thibault wrote: > > Again, do you really have KVM enabled? > > Also, do you use cache=writeback as advised in our documentations? Yes. The cache=writeback option is setup. The same is seen in the process arg too. id=drive-ide0-0-0,cache=writeback Libvirt and virt-manager have a nice reporting interface. It can also regularly graph the Guest CPU consumption. So what I reported you was about the Guest OS. So on my setup, Hurd is problematic in performance. I guess I need to be sure of "KVM is enabled" or not. -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Re: virtio support
On Mon, 2016-02-08 at 23:59 +0530, Ritesh Raj Sarraf wrote: > Libvirt and virt-manager have a nice reporting interface. It can also > regularly graph the Guest CPU consumption. So what I reported you was > about the Guest OS. So on my setup, Hurd is problematic in > performance. > > I guess I need to be sure of "KVM is enabled" or not. OKay!! Turns out to be an issue in Libvirt. I think I reported it to libvirt but forgot about the issue. Editing the domain for the VM is required. The GUI for virt-manager doesn't allow one to set kvm as the emulator. So virsh => edit Domain works. Specify /usr/bin/kvm as the emulator. It still results in the following, but the overall performance is better and more closer in line to what you mentioned earlier. qemu-system-x86_64 -enable-kvm -name Hurd -S -machine pc-i440fx- 2.4,accel=tcg,usb=off,vmport=off -cpu kvm32 -m 2048 It is almost the same as before, just the important addition of "- enable-kvm" option. Thanks. -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." signature.asc Description: This is a digitally signed message part
Processed: closing 42079
Processing commands for cont...@bugs.debian.org: > close 42079 Bug #42079 [hurd] hurd: exec doesn't like zip'ed binaries Marked Bug as done > thanks Stopping processing here. Please contact me if you need assistance. -- 42079: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=42079 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Processed: bug 47998 is not forwarded
Processing commands for cont...@bugs.debian.org: > notforwarded 47998 Bug #47998 [hurd] msgget IPC not implemented Unset Bug forwarded-to-address > thanks Stopping processing here. Please contact me if you need assistance. -- 47998: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=47998 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Re: virtio support
On Mon, 2016-02-08 at 19:32 +0100, Samuel Thibault wrote: > Ritesh Raj Sarraf, on Mon 08 Feb 2016 23:54:15 +0530, wrote: > > Since you asked about memory, I've now increased it to 2048. I > think > > for historical reasons, I had kept it to 1048 till now. > > 1024 should be plenty already. IIRC, Hurd only addressed memory up till 1024 MiB. Is that still the status ? Sorry to ask but the Hurd FAQ [1] does not talk much about these limitations. [1] https://www.gnu.org/software/hurd/faq.html -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Re: virtio support
Ritesh Raj Sarraf, on Tue 09 Feb 2016 00:18:27 +0530, wrote: > On Mon, 2016-02-08 at 19:32 +0100, Samuel Thibault wrote: > > Ritesh Raj Sarraf, on Mon 08 Feb 2016 23:54:15 +0530, wrote: > > > Since you asked about memory, I've now increased it to 2048. I > > think > > > for historical reasons, I had kept it to 1048 till now. > > > > 1024 should be plenty already. > > IIRC, Hurd only addressed memory up till 1024 MiB. Is that still the > status ? > > Sorry to ask but the Hurd FAQ [1] does not talk much about these > limitations. > > [1] https://www.gnu.org/software/hurd/faq.html https://www.gnu.org/software/hurd/faq/ram_limit.html is the latest achieved news. Some work is currently done to exploit more memory. Samuel
Re: virtio support
On Mon, 2016-02-08 at 19:03 +0100, Samuel Thibault wrote: > > > 1) I'm using the rtl8139 network driver emulated with qemu. I > fetched > > some data (around 200 MiB) over the network from the host (Linux) > > machine. The network speed I got was average 250 KiB/Sec > > That's very odd. Our buildd daemons typically get several MiB/sec > from > the Internet. Do you have KVM really enabled? Until recently my setup was using VirtualBox. Now that I'm on KVM/Libvirt, I checked the running image. Earlier it was running with qemu32, and now I've changed it to kvm32. /usr/bin/qemu-system-x86_64 -name Hurd -S -machine pc-i440fx- 2.4,accel=tcg,usb=off,vmport=off -cpu kvm32 -m 1024 Since you asked about memory, I've now increased it to 2048. I think for historical reasons, I had kept it to 1048 till now. -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Re: virtio support
Ritesh Raj Sarraf, on Mon 08 Feb 2016 23:54:15 +0530, wrote: > Since you asked about memory, I've now increased it to 2048. I think > for historical reasons, I had kept it to 1048 till now. 1024 should be plenty already. Samuel
Re: virtio support
On Tue, 2016-02-09 at 00:13 +0530, Ritesh Raj Sarraf wrote: > qemu-system-x86_64 -enable-kvm -name Hurd -S -machine pc-i440fx- > 2.4,accel=tcg,usb=off,vmport=off -cpu kvm32 -m 2048 > > > It is almost the same as before, just the important addition of "- > enable-kvm" option. > > Thanks. Okay!! The overall system performance is better. But still many things may be broken. Trying to ssh into the Hurd VM hangs. When trying to SSH connect, in the Guest, the CPU tops to 70%+ On the host: rrs@learner:~$ ssh root@172.16.230.201 root@172.16.230.201's password: Timeout, server 172.16.230.201 not responding. 2016-02-09 / 00:27:03 ♒♒♒☹ => 255 Then, there's another interesting bug with networking. The first attempt to restart the network fails. The follow-up attempt passes. And finally, my app: apt-offline has some zipfile related exceptions that trigger only on the Hurd platform. The same dataset, when used under Linux, works fine. The exception is: hurd-dev_1%3a0.7.git20160114-1_hurd-i386.deb file synced. Traceback (most recent call last): File "/usr/bin/apt-offline", line 28, in main() File "/usr/lib/python2.7/dist- packages/apt_offline_core/AptOfflineCoreLib.py", line 1997, in main args.func(args) File "/usr/lib/python2.7/dist- packages/apt_offline_core/AptOfflineCoreLib.py", line 1368, in installer data.file.write( zipBugFile.read( filename ) ) File "/usr/lib/python2.7/zipfile.py", line 935, in read return self.open(name, "r", pwd).read() File "/usr/lib/python2.7/zipfile.py", line 630, in read data = self.read1(n) File "/usr/lib/python2.7/zipfile.py", line 684, in read1 max(n - len_readbuffer, self.MIN_READ_SIZE) zlib.error: Error -3 while decompressing: invalid block type Which I think may be a memory issue, what you mentioned earlier. I think all these should be filed individually as bugs ? Or should I just keep reporting them on the mailing list ? -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." signature.asc Description: This is a digitally signed message part
Re: virtio support
Ritesh Raj Sarraf, on Tue 09 Feb 2016 00:34:17 +0530, wrote: > Trying to ssh into the Hurd VM hangs. When trying to SSH connect, in > the Guest, the CPU tops to 70%+ I have never seen that. > Then, there's another interesting bug with networking. The first > attempt to restart the network fails. The follow-up attempt passes. Neither have I. > And finally, my app: apt-offline has some zipfile related exceptions > that trigger only on the Hurd platform. I have never used apt-offline. > Which I think may be a memory issue, what you mentioned earlier. No, memory issues are usually that the box slows down and eventually panics. You can monitor memory usage along what you run. > I think all these should be filed individually as bugs ? Or should I > just keep reporting them on the mailing list ? Well, more importantly, these issues should be investigated. I have never seen these, and will probably not be able to reproduce them, and least fix them. Without investigation on the systems where they happen, there's little hope we get any kind of idea of what is happening. Samuel
