[uml-devel] [PATCH 2/3] BUG: Memory corruption on startup

2014-03-07 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com The reverse case of this race (you must msync before read) is well known. This is the not so common one. It can be triggered only on systems which do a lot of task switching and only at UML startup. If you are starting 200+ UMLs ~ 0.5% will always die

Re: [uml-devel] [PATCH 3/3] BUG: PIPE is the wrong IPC for this type of inter-thread communication.

2014-03-07 Thread Anton Ivanov
that the request disposal and handling of failed requests in the UBD driver is wrong somewhere else. No idea where. A. On 08/03/14 06:51, anton.iva...@kot-begemot.co.uk wrote: From: Anton Ivanov antiv...@cisco.com For more details see: http://stackoverflow.com/questions/4624071/pipe-buffer-size

[uml-devel] [PATCH 1/3] BUG: Missing pipe handling

2014-03-08 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com UML does not handle sigpipe. As a result when running it under expect or redirecting the IO from the console to an external program it will crash if the program stops or exits. Signed-off-by: Anton Ivanov antiv...@cisco.com --- arch/um/os-Linux/main.c

[uml-devel] [PATCH 2/3] BUG: Memory corruption on startup

2014-03-08 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com The reverse case of this race (you must msync before read) is well known. This is the not so common one. It can be triggered only on systems which do a lot of task switching and only at UML startup. If you are starting 200+ UMLs ~ 0.5% will always die

Re: [uml-devel] [PATCH 3/3] BUG: Inapprorpiate choice of IPC

2014-03-08 Thread Anton Ivanov
-list. A. On 08/03/14 23:33, Richard Weinberger wrote: Am 07.03.2014 19:37, schrieb anton.iva...@kot-begemot.co.uk: From: Anton Ivanov antiv...@cisco.com PIPE is the wrong IPC for this type of inter-thread communication. For more details see: http://stackoverflow.com/questions/4624071/pipe

[uml-devel] [PATCH 08/10] Minor performance optimization for ubd

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com Obvious performance optimization - it is not necessary to read the requests one at a time in the IRQ handler Signed-off-by: Anton Ivanov antiv...@cisco.com --- arch/um/drivers/ubd_kern.c | 29 ++--- 1 file changed, 22 insertions

[uml-devel] [PATCH 02/10] Remove unnecessary 'reactivate' statements

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com The epoll based controller has real (not emulated) edge and level semantics and the edge/level is handled by epoll. There is no toggling of the poll set any more, thus it is removed throughout Signed-off-by: Anton Ivanov antiv...@cisco.com --- arch/um

[uml-devel] [PATCH 06/10] RAW Ethernet transport for UML

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This is an alternative to the well known pcap transport. In the absense of special hardware support pcap is slow, guaranteed to be slow and with significant penalties on NUMA/SMP systems due to the timestamping of every packet. This transport does not incur

[uml-devel] [PATCH 07/10] Performance and NUMA improvements for ubd

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com The use of the seek()/read() and seek()/write() is a terminal disease on NUMA. Intense use of this on shared files (f.e. the master for a COW image) can cause anything up to and including killing CPUs on unhandled NMIs. This patch deals with this UML major

[uml-devel] [PATCH 01/10] Epoll based interrupt controller

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com 1. Minimum kernel 2.5.99 2. No walk the list lookups for received IRQs - immediate identification of the correct handler to invoke 3. Full set of IRQ semantics - edge, level, read, write 3.1. Write is now a *REAL* write - so if you (ab)use the write

[uml-devel] [PATCH 05/10] GRE transport for UML

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This transport allows a UML to connect to another UML local or remote, the Linux host or any other network device running the industry standard Ethernet over GRE protocol. The transport supports all features of RFC 2784. The transport supports a common set

[uml-devel] [PATCH 09/10] Better IPC for UBD

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com socketpair() is a better IPC choice for lots of small requests as it allows deeper (and configurable) queues than pipe() As a result UBD will process nearly all of the requests submitted to it instead of bouncing a significant percentage under load Signed

[uml-devel] [PATCH 03/10] High performance networking subsystem

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com Support for multi-packet vector IO - multiple packets read in one syscall and written in one syscall. Should work with legacy UML, thorough tested only for the epoll based IRQ controller Minimal host kernel version for RX - 2.6.32 Minimal host kernel version

[uml-devel] [PATCH 10/10] High Resolution Timer subsystem for UML

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This patch adds an extra timer source which has correct timing and uses an up-to-date OS API and. Results - correct kernel behaviour on timer related tasks. 1. Improvement in network performance (TCP state machines are now fed correct time). 2

[uml-devel] [PATCH 04/10] L2TPv3 Transport Driver for UML

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This transport allows a UML to connect to another UML local or remote, the Linux host or any other network device running the industry standard Ethernet over L2TPv3 protocol as per RFC 3931 (and successors). The transport supports a common set of features

[uml-devel] [PATCHv2 3/10] High performance networking subsystem

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com Support for multi-packet vector IO - multiple packets read in one syscall and written in one syscall. Should work with legacy UML, thorough tested only for the epoll based IRQ controller Minimal host kernel version for RX - 2.6.32 Minimal

[uml-devel] [PATCHv2 10/10] High Resolution Timer subsystem for UML

2014-08-29 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This patch adds an extra timer source which has correct timing and uses an up-to-date OS API and. Results - correct kernel behaviour on timer related tasks. 1. Improvement in network performance (TCP state machines are now fed correct time). 2

[uml-devel] UML Performance improvement patchset

2014-09-04 Thread anton . ivanov
Patch dependencies: [PATCH v3 01/10] Epoll based interrupt controller Full redesign of the existing UML poll based controller. The old poll controller incurs huge penalties for IRQ sharing and many devices setup due to the device list being walked twice. Additionally, the current controller

[uml-devel] [PATCH v3 06/10] RAW Ethernet transport for UML

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This is an alternative to the well known pcap transport. In the absense of special hardware support pcap is slow, guaranteed to be slow and with significant penalties on NUMA/SMP systems due to the timestamping of every packet. This transport does not incur

[uml-devel] [PATCH v3 02/10] Remove unnecessary 'reactivate' statements

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com The epoll based controller has real (not emulated) edge and level semantics and the edge/level is handled by epoll. There is no toggling of the poll set any more, thus it is removed throughout Signed-off-by: Anton Ivanov antiv...@cisco.com --- arch/um

[uml-devel] [PATCH v3 08/10] Minor performance optimization for ubd

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com Obvious performance optimization - it is not necessary to read the requests one at a time in the IRQ handler Signed-off-by: Anton Ivanov antiv...@cisco.com --- arch/um/drivers/ubd_kern.c | 29 ++--- 1 file changed, 22 insertions

[uml-devel] [PATCH v3 07/10] Performance and NUMA improvements for ubd

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com The use of the seek()/read() and seek()/write() is a terminal disease on NUMA. Intense use of this on shared files (f.e. the master for a COW image) can cause anything up to and including killing CPUs on unhandled NMIs. This patch deals with this UML major

[uml-devel] [PATCH v3 05/10] GRE transport for UML

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This transport allows a UML to connect to another UML local or remote, the Linux host or any other network device running the industry standard Ethernet over GRE protocol. The transport supports all features of RFC 2784. The transport supports a common set

[uml-devel] [PATCH v3 04/10] L2TPv3 Transport Driver for UML

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This transport allows a UML to connect to another UML local or remote, the Linux host or any other network device running the industry standard Ethernet over L2TPv3 protocol as per RFC 3931 (and successors). The transport supports a common set of features

[uml-devel] [PATCH v3 01/10] Epoll based interrupt controller

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com 1. Minimum kernel 2.5.99 2. No walk the list lookups for received IRQs - immediate identification of the correct handler to invoke 3. Full set of IRQ semantics - edge, level, read, write 3.1. Write is now a *REAL* write - so if you (ab)use the write

[uml-devel] [PATCH v3 09/10] Better IPC for UBD

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com socketpair() is a better IPC choice for lots of small requests as it allows deeper (and configurable) queues than pipe() As a result UBD will process nearly all of the requests submitted to it instead of bouncing a significant percentage under load Signed

[uml-devel] [PATCH v3 03/10] High performance networking subsystem

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com Support for multi-packet vector IO - multiple packets read in one syscall and (optionally) written in one syscall. Support for (optional) queueing on EAGAIN/ENOBUFS - applies only to socket transports. Sorry TAP, -EYOULOSE - it will remain

[uml-devel] [PATCH v3 10/10] High Resolution Timer subsystem for UML

2014-09-04 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This patch adds an extra timer source which has correct timing and uses an up-to-date OS API and. Results - correct kernel behaviour on timer related tasks. 1. Improvement in network performance (TCP state machines are now fed correct time). 2

Re: [uml-devel] UML Performance improvement patchset

2014-09-20 Thread Anton Ivanov
Hi list, Some benchmarks (current version, I believe this can do more once it gains full epoll write support). 4 UMLs running in parallel, each pinned to a core, new raw driver - vlan per UML, 10G NIC on 4 core 3.5GHz A8 connected back to back to an 8 core machine running iperf server.

[uml-devel] Updated Performance Improvements patchset

2014-09-22 Thread anton . ivanov
This is an update of the performance improvement patchset, it addresses a number of issues resulting from porting what was originally written for 3.3.8 to the current linux kernel. Changes (where applicable) are annotated in actual patchsets. A.

[uml-devel] [PATCH v4 08/11] Minor performance optimization for ubd

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com Obvious performance optimization - it is not necessary to read the requests one at a time in the IRQ handler Signed-off-by: Anton Ivanov antiv...@cisco.com --- arch/um/drivers/ubd_kern.c | 29 ++--- 1 file changed, 22 insertions

[uml-devel] [PATCH v4 06/11] RAW Ethernet transport for UML

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This is an alternative to the well known pcap transport. In the absense of special hardware support pcap is slow, guaranteed to be slow and with significant penalties on NUMA/SMP systems due to the timestamping of every packet. This transport does not incur

[uml-devel] [PATCH v4 09/11] Better IPC for UBD

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com socketpair() is a better IPC choice for lots of small requests as it allows deeper (and configurable) queues than pipe() As a result UBD will process nearly all of the requests submitted to it instead of bouncing a significant percentage under load Signed

[uml-devel] [PATCH v4 07/11] Performance and NUMA improvements for ubd

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com The use of the seek()/read() and seek()/write() is a terminal disease on NUMA. Intense use of this on shared files (f.e. the master for a COW image) can cause anything up to and including killing CPUs on unhandled NMIs. This patch deals with this UML major

[uml-devel] [PATCH v4 02/11] Remove unnecessary 'reactivate' statements

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com The epoll based controller has real (not emulated) edge and level semantics and the edge/level is handled by epoll. There is no toggling of the poll set any more, thus it is removed throughout Signed-off-by: Anton Ivanov antiv...@cisco.com --- arch/um

[uml-devel] [PATCH v4 03/11] High performance networking subsystem

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com Support for multi-packet vector IO - multiple packets read in one syscall and (optionally) written in one syscall. Support for (optional) queueing on EAGAIN/ENOBUFS - applies only to socket transports. Sorry TAP, -EYOULOSE - it will remain

[uml-devel] [PATCH v4 01/11] Epoll based interrupt controller

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com 1. Minimum kernel 2.5.44 2. No walk the list lookups for received IRQs - immediate identification of the correct handler to invoke 3. Full set of IRQ semantics - edge, level, read, write 3.1. Write is now a *REAL* write - so if you (ab)use the write

[uml-devel] [PATCH v4 05/11] GRE transport for UML

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This transport allows a UML to connect to another UML local or remote, the Linux host or any other network device running the industry standard Ethernet over GRE protocol. The transport supports all features of RFC 2784. The transport supports a common set

[uml-devel] [PATCH v4 10/11] High Resolution Timer subsystem for UML

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This patch adds an extra timer source which has correct timing and uses an up-to-date OS API and. Results - correct kernel behaviour on timer related tasks. 1. Improvement in network performance (TCP state machines are now fed correct time). 2

[uml-devel] [PATCH v4 11/11] Replace fsync() with sync_file_range()

2014-09-22 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com If the ubd is asynchronous the amount of time uml will spend fsync-ing on FLUSH is unpredictable. There are bits in the upper fs layers that really dislike this. Replacing fsync() with an async flush request via sync_file_range mitigates this (somewhat

[uml-devel] Fwd: Mail delivery failed: returning message to sender

2014-09-22 Thread Anton Ivanov
for sender-domain -0.0 SPF_PASS SPF: sender matches SPF record X-Headers-End: 1XW6ro-0005DR-7p From: Anton Ivanov antiv...@cisco.com This transport allows a UML to connect to another UML local or remote, the Linux host or any other network device running

Re: [uml-devel] Fwd: Mail delivery failed: returning message to sender

2014-09-23 Thread Anton Ivanov
On 23/09/14 15:58, Rob Landley wrote: On 09/22/14 13:39, Anton Ivanov wrote: Hi RIchard, hi list, There is some problem with the list on sourceforge. It has bounced one of the patches - no 4. I can resend this for book-keeping purposes. If memory serves me right there are no changes

Re: [uml-devel] Updated Performance Improvements patchset

2014-09-26 Thread Anton Ivanov
I finally nailed the last stability issue related to the patchset this morning. It is an extremely old issue and I tried to locate the root cause for it with Richard off-list 2 years ago. Unfortunately it was without success at that time. This issue is inherent in stock UML, but very difficult

[uml-devel] [PATCH] Fix for occasional userspace process in D/Z state bug

2014-09-26 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com This is a fix for a very old UML bug which can be triggered with stock UML. It takes a lot of effort to trigger it there because the lseek()/read() | write() mechanics of the UBD driver implicitly sync the memory all the time by hitting the appropriate

[uml-devel] [PATCH v2] Fix for occasional userspace process in D/Z state

2014-09-26 Thread anton . ivanov
From: Anton Ivanov antiv...@cisco.com Occasionally, under very heavy load inside UML, on host or both one of the processes in UML will remain in D state or will fail to reap a child which will sit in Z state. This is very difficult to reproduce with stock UML because the lseek()/read()|write

Re: [uml-devel] [PATCH] Fix for occasional userspace process in D/Z state bug

2014-09-28 Thread Anton Ivanov
On 26/09/14 12:56, Richard Weinberger wrote: Am 26.09.2014 13:49, schrieb anton.iva...@kot-begemot.co.uk: From: Anton Ivanov antiv...@cisco.com This is a fix for a very old UML bug which can be triggered with stock UML. It takes a lot of effort to trigger it there because the lseek()/read

Re: [uml-devel] kernel stalls in balance_dirty_pages_ratelimited()

2014-10-13 Thread Anton Ivanov
How does the stall manifest itself? Do you have the journal thread (and sometimes a couple of other threads) sitting in D state? A. On 13/10/14 22:48, Thomas Meyer wrote: #0 balance_dirty_pages_ratelimited (mapping=0x792cc618) at mm/page-writeback.c:1587 #1 0x600ba54f in

Re: [uml-devel] kernel stalls in balance_dirty_pages_ratelimited()

2014-10-14 Thread Anton Ivanov
schrieb Anton Ivanov: On 14/10/14 06:38, Anton Ivanov wrote: How does the stall manifest itself? Do you have the journal thread (and sometimes a couple of other threads) sitting in D state? Sorry, should not be asking questions at 6 am before the 3rd double espresso. I think it is the same bug I

Re: [uml-devel] kernel stalls in balance_dirty_pages_ratelimited()

2014-10-23 Thread Anton Ivanov
schrieb Thomas Meyer: Am Sonntag, den 19.10.2014, 17:02 +0100 schrieb Anton Ivanov: On 19/10/14 15:59, Thomas Meyer wrote: Am Dienstag, den 14.10.2014, 08:31 +0100 schrieb Anton Ivanov: I see a very similar stall on writeout to ubd with my patches (easy) and without (difficult - takes running an IO

Re: [uml-devel] kernel stalls in balance_dirty_pages_ratelimited()

2014-10-23 Thread Anton Ivanov
On 23/10/14 09:36, Richard Weinberger wrote: Anton, Am 23.10.2014 um 09:34 schrieb Anton Ivanov: Hi Richard, I have had a question in my mind on this for quite a while around this bug (and a quite a few others). Fell free to ask, I'll try hand to find time to answer. UML by the nature

Re: [uml-devel] starting an UML image sometimes failed, next (identical) attempt works

2014-11-19 Thread Anton Ivanov
I have seen this a couple of times as well. I am a bit snowed under at the moment, when I have some spare time I will try to dig through my trees to see exactly what did I come up with as a root cause. A. On 19/11/14 17:06, Toralf Förster wrote: /me wonders, why the same call few seconds later

Re: [uml-devel] [UM] Long loop in __getnsdayoftime() after resume from ram

2015-04-27 Thread Anton Ivanov
On 26/04/15 22:00, Richard Weinberger wrote: Can you give the attached patch a try? Let's see if it proves my theory. Looks like UML's clocksource needs fixing. Hi Richard, I did run this for an hour and did 4 suspend/resume cycles and it seems not to hang any more! Yay! BTW: Changing

Re: [uml-devel] [UM] Long loop in __getnsdayoftime() after resume from ram

2015-04-27 Thread Anton Ivanov
On 27/04/15 08:23, Richard Weinberger wrote: On Mon, Apr 27, 2015 at 7:47 AM, Anton Ivanov anton.iva...@kot-begemot.co.uk wrote: On 26/04/15 22:00, Richard Weinberger wrote: Can you give the attached patch a try? Let's see if it proves my theory. Looks like UML's clocksource needs fixing

Re: [uml-devel] [PATCH v5] um: Add a high resolution timer subsystem

2015-05-02 Thread Anton Ivanov
On 02/05/15 10:48, Thomas Meyer wrote: Hi, I did port Anton's v4 patch to v4.1-rc1-56-g3d99e3f and run it the last two days. Original v4 from Anton can be found here: https://sourceforge.net/p/user-mode-linux/mailman/message/32856805/ Issues addressed in v5 version: - Ported to

Re: [uml-devel] [PATCH v5] um: Add a high resolution timer subsystem

2015-05-03 Thread Anton Ivanov
Anton Ivanov: On 02/05/15 10:48, Thomas Meyer wrote: Hi, I did port Anton's v4 patch to v4.1-rc1-56-g3d99e3f and run it the last two days. Original v4 from Anton can be found here: https://sourceforge.net/p/user-mode-linux/mailman/message/32856805/ Issues addressed in v5 version

Re: [uml-devel] [PATCH v5] um: Add a high resolution timer subsystem

2015-05-04 Thread Anton Ivanov
that cannot be helped at this stage. A. On 03/05/15 16:46, Thomas Meyer wrote: Am Samstag, den 02.05.2015, 12:08 +0100 schrieb Anton Ivanov: On 02/05/15 10:48, Thomas Meyer wrote: Hi, I did port Anton's v4 patch to v4.1-rc1-56-g3d99e3f and run it the last two days. Original v4 from Anton

Re: [uml-devel] [PATCH v6] um: Add a high resolution timer subsystem

2015-05-11 Thread Anton Ivanov
Hurray, Houston we have ignition. We now have working userspace timers. It is still schizophrenic - userspace is HZ, kernel is NOHZ because the userpace has to keep checking did the kernel timer fire yet at a HZ interval. However, even that is a major progress compared to having userspace timer

Re: [uml-devel] [PATCH v6] um: Add a high resolution timer subsystem

2015-05-11 Thread Anton Ivanov
). On the positive side - the behavior we are getting now is better, so we just need to figure out the root cause for the hang(s) and stabilize it. A. On 11/05/15 13:52, Anton Ivanov wrote: Hurray, Houston we have ignition. We now have working userspace timers. It is still schizophrenic - userspace is HZ

Re: [uml-devel] [PATCH v6] um: Add a high resolution timer subsystem

2015-05-11 Thread Anton Ivanov
replacement where the timer fires exactly when it is expected to fire does not quite work. So what is the idea here? A. On 11/05/15 16:05, Anton Ivanov wrote: Hi Thomas, hi Richard, It is now possible to reproducibly hang it. I have not been able to concoct a synthetic test (yet), but a non

Re: [uml-devel] [PATCH v5] um: Add a high resolution timer subsystem

2015-05-05 Thread Anton Ivanov
On 05/05/15 08:00, Thomas Meyer wrote: Am 04.05.2015 um 18:22 schrieb Anton Ivanov anton.iva...@kot-begemot.co.uk: Hi Thomas, Hi Anton, I read it and like it except idle sleep. See my comments inline. I killed that nanosleep quite on purpose :) Okay. I think I understand now

Re: [uml-devel] [PATCH v6] um: Add a high resolution timer subsystem

2015-05-10 Thread Anton Ivanov
On 10/05/15 15:34, Thomas Meyer wrote: Am 10.05.2015 um 14:35 schrieb Richard Weinberger richard.weinber...@gmail.com: On Sun, May 10, 2015 at 1:14 AM, Thomas Meyer tho...@m3y3r.de wrote: Hi, Changes: - also create posix timer in stub_clone_handler() - incorporated antons remarks Hm,

Re: [uml-devel] [PATCH v6] um: Add a high resolution timer subsystem

2015-05-10 Thread Anton Ivanov
On 10/05/15 13:35, Richard Weinberger wrote: On Sun, May 10, 2015 at 1:14 AM, Thomas Meyer tho...@m3y3r.de wrote: Hi, Changes: - also create posix timer in stub_clone_handler() - incorporated antons remarks Hm, this patch does a *lot* more than the changelog says. Richard, I think this is

Re: [uml-devel] [PATCH v6] um: Add a high resolution timer subsystem

2015-05-11 Thread Anton Ivanov
On 11/05/15 18:00, Thomas Meyer wrote: Hi, maybe there is a bug in how the timers are created for all user space processes. In the latest patch I use os__timer_remain for the initial interval. The idea was to launch all timers at the same time. But I now think this can never work using

Re: [uml-devel] [PATCH v6] um: Add a high resolution timer subsystem

2015-05-11 Thread Anton Ivanov
On 11/05/15 18:41, Thomas Meyer wrote: Am 11.05.2015 7:31 nachm. schrieb Anton Ivanov anton.iva...@kot-begemot.co.uk: On 11/05/15 18:20, Anton Ivanov wrote: On 11/05/15 18:00, Thomas Meyer wrote: Hi, maybe there is a bug in how the timers are created for all user space processes

Re: [uml-devel] [PATCH v6] um: Add a high resolution timer subsystem

2015-05-11 Thread Anton Ivanov
On 11/05/15 18:20, Anton Ivanov wrote: On 11/05/15 18:00, Thomas Meyer wrote: Hi, maybe there is a bug in how the timers are created for all user space processes. In the latest patch I use os__timer_remain for the initial interval. The idea was to launch all timers at the same time. But I

Re: [uml-devel] [PATCH] um: Switch clocksource to hrtimers

2015-05-19 Thread Anton Ivanov
On 19/05/15 17:39, Richard Weinberger wrote: On Tue, May 19, 2015 at 6:19 PM, Anton Ivanov anton.iva...@kot-begemot.co.uk wrote: Same story. Overall works quite well, the moment I give it the dselect upgrade test the userspace hangs. Kernel continues to work - forwarding, icmp

Re: [uml-devel] [PATCH] um: Switch clocksource to hrtimers

2015-06-04 Thread Anton Ivanov
I will give it a spin before the end of the weekend and run the full test suite on it. A. [snip] Cascading the kernel timer looks like the correct solution. Hi, below patch get's rid of all userspace timers. When a kernel timer interrupt is received the userspace process of the

Re: [uml-devel] [PATCH] um: Switch clocksource to hrtimers

2015-05-31 Thread Anton Ivanov
On 31/05/15 20:00, Thomas Meyer wrote: Am Sonntag, den 31.05.2015, 13:15 +0200 schrieb Richard Weinberger: Am 20.05.2015 um 07:26 schrieb Thomas Meyer: Am 20.05.2015 12:12 vorm. schrieb Richard Weinberger richard.weinber...@gmail.com: On Sun, May 17, 2015 at 11:25 AM, Thomas Meyer

Re: [uml-devel] [PATCH] um: Switch clocksource to hrtimers

2015-05-19 Thread Anton Ivanov
Same story. Overall works quite well, the moment I give it the dselect upgrade test the userspace hangs. Kernel continues to work - forwarding, icmp, uml_mconsole are fine. A. On 17/05/15 10:25, Thomas Meyer wrote: Switch the UML clocksource from interval timers to posix interval timers and

Re: [uml-devel] [PATCH] um: Switch clocksource to hrtimers

2015-08-18 Thread Anton Ivanov
If Thomas is busy I can pick it up ~ middle of next week and finish it. A. On 18/08/15 10:34, Richard Weinberger wrote: On Sat, Aug 15, 2015 at 6:27 PM, Anton Ivanov anton.iva...@kot-begemot.co.uk wrote: On 15/08/15 09:15, Richard Weinberger wrote: Am 09.08.2015 um 19:53 schrieb Thomas Meyer

Re: [uml-devel] [PATCH v3] um: Switch clocksource to hrtimers

2015-10-29 Thread Anton Ivanov
ing a week or so. IMHO 25% is worth it :) A. On 26/10/15 10:12, Richard Weinberger wrote: > Hi! > > Am 25.10.2015 um 19:46 schrieb Anton Ivanov: >> Hi List, hi Richard, >> >> I am going to sort out the UBD patchset next as that has no dependencies on >> timer

Re: [uml-devel] [PATCH v3] um: Switch clocksource to hrtimers

2015-10-25 Thread Anton Ivanov
of that for ages). I should be able to get these out of the door early on this week. A. P.S. If there is anything to fix on the timer one, please shout. A. On 21/10/15 09:42, Anton Ivanov wrote: > Background: UML is using an obsolete itimer call for > all timers and "polls" for kernel sp

[uml-devel] [PATCH v2] EPOLL Interrupt Controller V2.0

2015-11-09 Thread Anton Ivanov
. Signed-off-by: Anton Ivanov <aiva...@brocade.com> --- arch/um/drivers/line.c| 5 +- arch/um/drivers/mconsole_kern.c | 2 - arch/um/drivers/net_kern.c| 1 - arch/um/drivers/port_kern.c | 1 - arch/um/drivers/random.c | 1 - arch/um/drivers/ubd_

Re: [uml-devel] [PATCH v2] EPOLL Interrupt Controller V2.0

2015-11-09 Thread Anton Ivanov
some point in the future. I have fixed the file descriptor leak in the reboot case. A. On 09/11/15 14:33, Anton Ivanov wrote: > Epoll based interrupt controller. > > IMPROVES: IO loop performance - no per fd lookups, allowing for > 15% IO speedup in minimal config going to 100s of % with many &g

Re: [uml-devel] [PATCH v2] EPOLL Interrupt Controller V2.0

2015-11-10 Thread Anton Ivanov
ssion", do not use yet. A On 09/11/15 15:03, Anton Ivanov wrote: > It throws a couple of harmless "epoll del fd" warnings on reboot which > result the fact that disable_fd/enable_fd are not removed in the > terminal/line code. > > These are harmless and will g

[uml-devel] [PATCH] Update UBD to use pread/pwrite family of functions

2015-11-08 Thread Anton Ivanov
This decreases the number of syscalls per read/write by half. Signed-off-by: Anton Ivanov <aiva...@brocade.com> --- arch/um/drivers/ubd_kern.c | 27 +-- arch/um/include/shared/os.h | 2 ++ arch/um/os-Linux/file.c | 19 +++ 3 files chang

[uml-devel] [PATCH] Missing includes in start up

2015-11-08 Thread Anton Ivanov
Modifying RLIMIT requires including sys/time.h and sys/resource.h Signed-off-by: Anton Ivanov <aiva...@brocade.com> --- arch/um/os-Linux/start_up.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/um/os-Linux/start_up.c b/arch/um/os-Linux/start_up.c index 425162e..c1811df

Re: [uml-devel] [PATCH] Missing includes in start up

2015-11-08 Thread Anton Ivanov
Hi Richard, This is vs your tree, 4.x master branch in linus repo actually has sys/resource.h Pls ignore the patch. A. On 08/11/15 14:58, Anton Ivanov wrote: > Modifying RLIMIT requires including sys/time.h and sys/resource.h > > Signed-off-by: Anton Ivanov <aiva...

Re: [uml-devel] [PATCH] Update UBD to use pread/pwrite family of functions

2015-11-08 Thread Anton Ivanov
I am going to send in a couple of more which bulk-up the transactions (this forms most of the improvement) later this week. This one is obvious - qemu has been using it for ages. A. On 08/11/15 15:20, Anton Ivanov wrote: > This decreases the number of syscalls per read/write by h

Re: [uml-devel] [PATCH] Missing includes in start up

2015-11-08 Thread Anton Ivanov
multipacket reads/multipacket writes which will be next. A. On 08/11/15 22:38, Richard Weinberger wrote: > On Sun, Nov 8, 2015 at 4:18 PM, Anton Ivanov <aiva...@brocade.com> wrote: >> Hi Richard, >> >> This is vs your tree, 4.x master branch in linus repo actually has

[uml-devel] [PATCH] EPOLL Interrupt Controller V2.0

2015-11-08 Thread Anton Ivanov
. Signed-off-by/: Anton Ivanov <aiva...@brocade.com> --- arch/um/drivers/line.c| 5 +- arch/um/drivers/mconsole_kern.c | 2 - arch/um/drivers/net_kern.c| 1 - arch/um/drivers/port_kern.c | 1 - arch/um/drivers/random.c | 1 - arch/um/drivers/ubd_

Re: [uml-devel] [PATCH] EPOLL Interrupt Controller V2.0

2015-11-08 Thread Anton Ivanov
different IRQs to different network devices. A. On 08/11/15 22:50, Anton Ivanov wrote: > Epoll based interrupt controller. > > IMPROVES: IO loop performance - no per fd lookups, allowing for > 15% IO speedup in minimal config going to 100s of % with many > devices - a N^N lookup

Re: [uml-devel] [PATCH] EPOLL Interrupt Controller V2.0

2015-11-08 Thread Anton Ivanov
I just noted one minor issue with it (which existed in the earlier version as well) - it leaks one FD per reboot. I will fix it later on during the week in a revised version. A. On 08/11/15 22:50, Anton Ivanov wrote: > Epoll based interrupt controller. > > IMPROVES: IO loop pe

Re: [uml-devel] [PATCH v2] EPOLL Interrupt Controller V2.0

2015-11-12 Thread Anton Ivanov
On 11/11/15 21:05, Richard Weinberger wrote: > On Wed, Nov 11, 2015 at 9:46 PM, Thomas Meyer <tho...@m3y3r.de> wrote: >> Am Montag, den 09.11.2015, 15:03 +0000 schrieb Anton Ivanov: >>> It throws a couple of harmless "epoll del fd" warnings on reboot >>>

Re: [uml-devel] WARNING: CPU: 0 PID: 908 at include/linux/thread_info.h:128 sigsuspend+0xab/0xc0()

2015-11-16 Thread Anton Ivanov
On 16/11/15 11:53, Richard Weinberger wrote: > Am 16.11.2015 um 12:49 schrieb Vegard Nossum: >> On 11/16/2015 12:44 PM, Richard Weinberger wrote: >>> Am 16.11.2015 um 10:43 schrieb Vegard Nossum: Starting UML like this: ./vmlinux rootfstype=hostfs rw ignore_console_loglevel

Re: [uml-devel] [PATCH v2] EPOLL Interrupt Controller V2.0

2015-11-16 Thread Anton Ivanov
On 12/11/15 16:03, Anton Ivanov wrote: > On 12/11/15 15:23, Anton Ivanov wrote: >> [snip] >> >>>> Hmmm, UML is UP and does not support PREEMPT, so all spinlocks >>>> should be a no-op. >>> In that case, if I understand correctly what is going on,

Re: [uml-devel] [PATCH v2] EPOLL Interrupt Controller V2.0

2015-11-12 Thread Anton Ivanov
[snip] >> Hmmm, UML is UP and does not support PREEMPT, so all spinlocks >> should be a no-op. > In that case, if I understand correctly what is going on, there are a > couple of places - the free_irqs(), activate_fd and the sigio handler > itself, where it should not be a mutex, not a spinlock.

[uml-devel] [PATCH v3] EPOLL Interrupt Controller V2.0 - Work in Progress

2015-11-12 Thread Anton Ivanov
which do not get along with poll() very well. Signed-off-by: Anton Ivanov <aiva...@brocade.com> --- arch/um/drivers/chan_kern.c | 10 +- arch/um/drivers/line.c| 5 +- arch/um/drivers/mconsole_kern.c | 2 - arch/um/drivers/net_kern.c| 1 - arch/um/d

Re: [uml-devel] [PATCH v2] EPOLL Interrupt Controller V2.0

2015-11-11 Thread Anton Ivanov
Thomas Meyer <tho...@m3y3r.de> wrote: >> Am Montag, den 09.11.2015, 15:03 + schrieb Anton Ivanov: >>> It throws a couple of harmless "epoll del fd" warnings on reboot >>> which >>> result the fact that disable_fd/enable_fd are not removed in the

Re: [uml-devel] [PATCH v2] EPOLL Interrupt Controller V2.0

2015-11-11 Thread Anton Ivanov
There is for some reason IRQ/sigio handler reentrancy. Both the debug "by hand" I stuck into irq.c and Thomas traces confirm it. I have no clue how this is possible as the code in signal.c should guard against that. I will sleep on it and try to tackle this tomorrow - no idea what is going

Re: [uml-devel] [PATCH v3] um: Switch clocksource to hrtimers

2015-11-02 Thread Anton Ivanov
On 02/11/15 10:01, Richard Weinberger wrote: > Am 02.11.2015 um 10:53 schrieb Anton Ivanov: >> [snip] >> >>>>> I'm pretty sure that you don't see the issue as your Jessy userspace uses >>>>> nanosleep periodically. >>>> There are

Re: [uml-devel] [PATCH v4] um: Switch clocksource to hrtimers

2015-11-02 Thread Anton Ivanov
On 02/11/15 22:13, Anton Ivanov wrote: > On 02/11/15 21:50, Richard Weinberger wrote: >> Am 02.11.2015 um 17:16 schrieb Anton Ivanov: >>> -void idle_sleep(unsigned long long nsecs) >>> +/** >>> + * os_idle_sleep() - sleep for a given time of nsecs

Re: [uml-devel] [PATCH v3] um: Switch clocksource to hrtimers

2015-11-02 Thread Anton Ivanov
I was testing under similar conditions (CPU pinning using taskset -c 0 on a multicore). I have removed it and run some retests - I cannot reproduce the hang at this point with my config I am going to run a defconfig and compare the results to see if this will give me any insights on the root

Re: [uml-devel] [PATCH v3] um: Switch clocksource to hrtimers

2015-11-02 Thread Anton Ivanov
On 02/11/15 08:37, Richard Weinberger wrote: > Hi! > > Am 02.11.2015 um 09:14 schrieb Anton Ivanov: >> I was testing under similar conditions (CPU pinning using taskset -c 0 on a >> multicore). >> >> I have removed it and run some retests - I cannot reproduce t

Re: [uml-devel] [PATCH v4] um: Switch clocksource to hrtimers

2015-11-07 Thread Anton Ivanov
Cool, Is there a tree I can pull so I can have the next one (the epoll based IRQ controller) incremental? A. On 06/11/15 21:56, Richard Weinberger wrote: > Am 02.11.2015 um 17:16 schrieb Anton Ivanov: >> Background: UML is using an obsolete itimer call for >> all timers and &qu

Re: [uml-devel] [PATCH v3] um: Switch clocksource to hrtimers

2015-11-02 Thread Anton Ivanov
On 02/11/15 15:25, Richard Weinberger wrote: > Am 02.11.2015 um 15:30 schrieb Anton Ivanov: >> On 02/11/15 10:01, Richard Weinberger wrote: >>> Am 02.11.2015 um 10:53 schrieb Anton Ivanov: >>>> [snip] >>>> >>>>>>> I'm pretty sure tha

[uml-devel] [PATCH v4] um: Switch clocksource to hrtimers

2015-11-02 Thread Anton Ivanov
Signed-off-by: Thomas Meyer <tho...@m3y3r.de> Signed-off-by: Anton Ivanov <aiva...@brocade.com> --- arch/um/Makefile| 2 +- arch/um/include/shared/os.h | 18 ++- arch/um/include/shared/skas/stub-data.h | 7 +- arch/um/include/shared/timer-intern

Re: [uml-devel] [PATCH v4] um: Switch clocksource to hrtimers

2015-11-02 Thread Anton Ivanov
On 02/11/15 21:50, Richard Weinberger wrote: > Am 02.11.2015 um 17:16 schrieb Anton Ivanov: >> -void idle_sleep(unsigned long long nsecs) >> +/** >> + * os_idle_sleep() - sleep for a given time of nsecs >> + * @nsecs: nanoseconds to sleep >> + */ >> +void

Re: [uml-devel] [PATCH v3] um: Switch clocksource to hrtimers

2015-11-02 Thread Anton Ivanov
On 02/11/15 08:52, Richard Weinberger wrote: > Am 02.11.2015 um 09:41 schrieb Anton Ivanov: >> On 02/11/15 08:37, Richard Weinberger wrote: >>> Hi! >>> >>> Am 02.11.2015 um 09:14 schrieb Anton Ivanov: >>>> I was testing under similar conditions (CPU

Re: [uml-devel] [PATCH v3] um: Switch clocksource to hrtimers

2015-11-02 Thread Anton Ivanov
[snip] >>> I'm pretty sure that you don't see the issue as your Jessy userspace uses >>> nanosleep periodically. >> There are quite a few things running so this may indeed be the case. >> >> What do you use for userspace (so I can try to reproduce this and debug it)? > Debian Squeeze amd64 with

  1   2   3   >