That graph looks a bit funky .... Perhaps you should try to run hwlat ? ( module ) ... to see if there are some SMI or similar, you can also use latency tracing in the kernel to see where the offender is .... Can you see what rt-priority the system is running with ? 110 looks like a bit over the top.
How about asking on the rt-preempt mailing list ? linux-rt-us...@vger.kernel.org 2012/9/14 Jan de Kruyf <jan.de.kr...@gmail.com>: > Hallo, > this is the plot from this board with a slightly different cpu, under > reasonable load: > https://www.osadl.org/Latency-plot-of-system-in-rack-4-slot.qa-latencyplot-r4s7.0.html > You might look up there also exactly how they do their tests. > > The Pagefault message says that the LCNC memory is not properly locked in > place (i.e. the kernel needs to page, it should not) > So I would say, speak to your friendly software developer, or if you really > want to try, throw lots of memory at it and hope the kernel stops paging > after a while. > > By the way you can also look up the kernel compile switches here: > https://www.osadl.org/Profile-of-system-in-rack-4-slot-7.qa-profile-r4s7.0.html > and try to build a new kernel. At first I thought the problem might be > there, but the pagefault message convinced me otherwise. > > cheers, > > j. > > > > On Fri, Sep 14, 2012 at 6:30 AM, Kent A. Reed <kentallanr...@gmail.com>wrote: > >> Gentle persons: >> >> First, a statement. >> >> On my ASUS AT5NM10-I board (Intel Atom D510 CPU, 2GB RAM, yada yada >> yada) I followed in Charles Steinkeuhler's footsteps >> ( >> http://wiki.linuxcnc.org/cgi-bin/wiki.pl?Debian_Wheezy_Linux-Rt_Compile_LinuxCNC >> ) >> to build a 64-bit Debian Wheezy RT PREEMPT Linux system running the >> 3.2.0-3-rt-amd64 kernel and Linuxcnc 2.6.0~pre. Such a build has given >> Charles some decent albeit not (yet) great latency numbers on several >> non-Atom boards. >> >> My ASUS board has given me excellent latency results with what I'll call >> classic Ubuntu 10.04LTS LinuxCNC running the 2.6.32-122-rtai kernel: sub >> 10us for both threads with Hyperthreading disabled in the BIOS and one >> cpu isolated during the boot process. >> >> With the RT PREEMPT system, this board gives lousy results even running >> headless. Using latency-test, for some minutes I see the base (25us) >> thread showing a max latency of anywhere from 25us to 40us and the servo >> (1000us) thread 40us to 50us. Then, the servo thread pops a bit and the >> base thread pops a lot, to about 110us. At the same time, the kernel >> throws three lines to the console >> >> "ERROR: Missed scheduling deadline for task 0 [nnnn times] >> "Now is xxxxx.xxx, deadline is xxxxx.xxx >> "Absolute number of pagefaults in realtime context: 1030" >> >> This process repeats but not at regular intervals. Using latencyplot, I >> can see that, with nothing else running, both threads mostly show good >> latency numbers, typically < 10us, once we've settled down after >> invoking latencyplot. If I run a copy of glxgears, the servo thread >> latency gets jumpy but stays below about 40us. Running several copies of >> glxgears doesn't seem to cause any more damage, nor does invoking du or >> some other disk access-intensive command. Sooner or later, though, the >> above big-time event happens. Repeat ad nauseum. >> >> I've done everything I can think of. I've diddled all available BIOS >> settings (this is not an enthusiast board; it has only a limited number >> of settings); stopped the kernel from loading just about any >> non-essential module; preventing many services from starting. No gdm, no >> X, no Intel i915 driver, no acpid, no alsa, no pulseaudio, yada yada yada. >> >> About the only things I haven't tried are (1) trying Charles' suggestion >> of playing with cpusets, mostly because I don't understand them well >> enough yet to trust myself, (2) ripping out some more modules that >> relate to sound (with names snd*; alsa and pulseaudio stuff is already >> gone), mostly because I'm not sure who's loading them, and (3) redoing >> all this with a 32-bit Debian Wheezy, mostly because getting Debian >> Wheezy systems into this machine is a bore (the Wheezy installer is >> broken somewhere in the disk partitioning process and why should I be >> the one to fix it when others have been complaining forever). >> >> Now, my question. >> >> I'm wondering if my results are intrinsic to the Atom architecture or >> specific to the ASUS BIOS. >> >> Has anyone with a different Atom board, preferably an Intel-branded >> board, loaded and tested Wheezy with Linux-RT and LinuxCNC? If so, >> what's your experience? >> >> Regards, >> Kent >> >> >> >> >> >> ------------------------------------------------------------------------------ >> Got visibility? >> Most devs has no idea what their production app looks like. >> Find out how fast your code is with AppDynamics Lite. >> http://ad.doubleclick.net/clk;262219671;13503038;y? >> http://info.appdynamics.com/FreeJavaPerformanceDownload.html >> _______________________________________________ >> Emc-developers mailing list >> Emc-developers@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/emc-developers >> > ------------------------------------------------------------------------------ > Got visibility? > Most devs has no idea what their production app looks like. > Find out how fast your code is with AppDynamics Lite. > http://ad.doubleclick.net/clk;262219671;13503038;y? > http://info.appdynamics.com/FreeJavaPerformanceDownload.html > _______________________________________________ > Emc-developers mailing list > Emc-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/emc-developers ------------------------------------------------------------------------------ Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html _______________________________________________ Emc-developers mailing list Emc-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-developers
