I'm using 2.6.24.2 with -rt1 applied
(applies cleanly with some offsets) on x86.
I've *not* configured PROVE_LOCKING (too expensive).
However, I selected all the latency tracings, timings and histograms,
among them CRITICAL_IRQSOFF_TIMING.
This implicitly configures TRACE_IRQFLAGS.
However,
When a process running with RT priority dumps core,
I get the following BUG:
Apr 11 13:44:23 OF455 kern.err kernel: BUG: rtc2:833 RT task
yield()-ing!
Apr 11 13:44:23 OF455 kern.warn kernel: [c026dad1] yield+0x61/0x70
(8)
Apr 11 13:44:23 OF455 kern.warn kernel: [c0151e49]
From the three sources of multi-millisecond latency in my experiments
(console messages to dead serial console, USB I/O, CF Card bulk read),
I've analyzed one:
The latency of around 70 milliseconds in low-priority RT processes
when running a dd if=/dev/hda of=/dev/null in parallel (where hda
is
When trying a dd if=/dev/sda of=/dev/null (where /dev/sda is an USB
memory stick), this works fine for some seconds (and actually transfers
data at around 700-1000 KB/s), but ends up with some I/O errors sooner
or later, which cause the device to go offline (the stick must be
replugged to make it
I've written a small test program which enables periodic RTC interrupts
at 8192 Hz and then goes into a loop reading /dev/rtc and collecting
timing statistics (using the rdtscl macro).
The program runs at highest realtime scheduling priority (99) with
memory
locked. In the loop, it doesn't do
From: Bartlomiej Zolnierkiewicz [mailto:[EMAIL PROTECTED]
On Wed, 30 Mar 2005 13:52:05 +0200, kus Kusche Klaus
[EMAIL PROTECTED] wrote:
However, things break seriously when exercising the CF card
in parallel
(e.g. with a dd if=/dev/hda of=/dev/null):
* The rtc *interrupt handler
I'm experimenting with a realtime-preempt-2.6.12-rc1-V0.7.41-11 kernel,
preemption fully enabled, and a small RTC test program (see my previous
mail for details).
If my program runs at rtprio 99, most of the time everything is fine:
* My program records intervals of 122 +- 40 microseconds (which
I'm performing realtime latency tests (for details about the hardware
and software, see my mail [BUG] 2.6.11: Random SCSI/USB errors when
reading from USB memory stick erlier today).
Even when the errors described in my previous mail does not occur,
massive USB stick transfers cause latencies of
i have released the -V0.7.41-25 Real-Time Preemption patch,
which can be
downloaded from the usual place:
1. Does not compile without RT_DEADLOCK_DETECT:
kernel/rt.c: In function `change_owner':
kernel/rt.c:556: error: structure has no member named `debug'
kernel/rt.c: In function
The test system runs a 2.6.11 kernel (no SMP) on a Pentium3 500 MHz
embedded hardware.
which probably has memory bandwidth of at most a couple hundred MB/s,
which is really horrible by modern standards.
It's my job to find out if linux can be used for control systems
on this (and
The latencies are almost certainly caused by the USB host controller
driver. I'm planning improvements to uhci-hcd which should
help reduce
the latency, but it will still be on the large side. And I
won't have
time to write the changes to the driver for several months.
Any numbers
Latency is the subject of a separate email. Does this
increase in latency
occur only when you see the errors, or whenever you do a large data
transfer? In fact, I would suspect the errors to _decrease_
the latency
with respect to a normal transfer.
I observe from 1 to 2 ms on
I couldn't find that previous email in the MARC archives.
Regardless, you'd have to provide a small bit of information about
your hardware configuration. What device speed: full or high?
What controller: EHCI, OHCI, UHCI, something else? Which driver
for the stick: usb-storage, or ub?
getting /dev/rtc handling right for latency measurement is
... tricky.
The method i'm using under PREEMPT_RT is:
chrt -f 84 -p `pidof 'IRQ 0'`
chrt -f 95 -p `pidof 'IRQ 8'`
./rtc_wakeup -f 1024 -t 10
I tried it your way.
First impressions with
Latency is the subject of a separate email. Does this
increase in latency
occur only when you see the errors, or whenever you do a
large data
transfer? In fact, I would suspect the errors to _decrease_
the latency
with respect to a normal transfer.
I observe from 1
The latencies are almost certainly caused by the USB host
controller
driver. I'm planning improvements to uhci-hcd which should
help reduce
the latency, but it will still be on the large side. And I
won't have
time to write the changes to the driver for several months.
The following tests are made with 'IRQ 8' at 95, rtc_wakeup
at 89(99):
* Heavy mmap load, no oom: max jitter: 42.1% ( 51 usec)
* Heavy mmap load, oom:max jitter: 11989.2% (14635 usec)
(but still missed irqs: 0, so IRQ 8 was also blocked for 14 ms)
did you get any kernel
Even when the errors described in my previous mail does not occur,
massive USB stick transfers cause latencies of 1 to 2 milliseconds,
which is way too much for realtime control systems.
do these occur under PREEMPT_RT? If yes, do you get any
useful trace if
you enable all the
* kus Kusche Klaus [EMAIL PROTECTED] wrote:
IRQ 7-724 0d..11us : end_8259A_irq (do_hardirq)
IRQ 7-724 0d..11us!: enable_8259A_irq (do_hardirq)
IRQ 7-724 0d... 832us : do_hardirq (do_irqd)
IRQ 7-724 0d... 833us : trace_irqs_on (do_hardirq)
mmap-1000
When trying a "dd if=/dev/sda of=/dev/null" (where /dev/sda is an USB
memory stick), this works fine for some seconds (and actually transfers
data at around 700-1000 KB/s), but ends up with some I/O errors sooner
or later, which cause the device to go offline (the stick must be
replugged to make
I've written a small test program which enables periodic RTC interrupts
at 8192 Hz and then goes into a loop reading /dev/rtc and collecting
timing statistics (using the rdtscl macro).
The program runs at highest realtime scheduling priority (99) with
memory
locked. In the loop, it doesn't do
> From: Bartlomiej Zolnierkiewicz [mailto:[EMAIL PROTECTED]
>
> On Wed, 30 Mar 2005 13:52:05 +0200, kus Kusche Klaus
> <[EMAIL PROTECTED]> wrote:
> > However, things break seriously when exercising the CF card
> in parallel
> > (e.g. with a dd if=/dev/hda
I'm experimenting with a realtime-preempt-2.6.12-rc1-V0.7.41-11 kernel,
preemption fully enabled, and a small RTC test program (see my previous
mail for details).
If my program runs at rtprio 99, most of the time everything is fine:
* My program records intervals of 122 +- 40 microseconds (which
I'm performing realtime latency tests (for details about the hardware
and software, see my mail "[BUG] 2.6.11: Random SCSI/USB errors when
reading from USB memory stick" erlier today).
Even when the errors described in my previous mail does not occur,
massive USB stick transfers cause latencies
> i have released the -V0.7.41-25 Real-Time Preemption patch,
> which can be
> downloaded from the usual place:
1. Does not compile without RT_DEADLOCK_DETECT:
kernel/rt.c: In function `change_owner':
kernel/rt.c:556: error: structure has no member named `debug'
kernel/rt.c: In function
> > The test system runs a 2.6.11 kernel (no SMP) on a Pentium3 500 MHz
> > embedded hardware.
>
> which probably has memory bandwidth of at most a couple hundred MB/s,
> which is really horrible by modern standards.
It's my job to find out if linux can be used for control systems
on this (and
> The latencies are almost certainly caused by the USB host controller
> driver. I'm planning improvements to uhci-hcd which should
> help reduce
> the latency, but it will still be on the large side. And I
> won't have
> time to write the changes to the driver for several months.
Any
> Latency is the subject of a separate email. Does this
> increase in latency
> occur only when you see the errors, or whenever you do a large data
> transfer? In fact, I would suspect the errors to _decrease_
> the latency
> with respect to a normal transfer.
I observe from <1 to 2 ms on
> I couldn't find that previous email in the MARC archives.
>
> Regardless, you'd have to provide a small bit of information about
> your hardware configuration. What device speed: full or high?
> What controller: EHCI, OHCI, UHCI, something else? Which driver
> for the stick: usb-storage,
> getting /dev/rtc handling right for latency measurement is
> ... tricky.
> The method i'm using under PREEMPT_RT is:
>
> chrt -f 84 -p `pidof 'IRQ 0'`
> chrt -f 95 -p `pidof 'IRQ 8'`
> ./rtc_wakeup -f 1024 -t 10
I tried it your way.
First impressions with
> > > Latency is the subject of a separate email. Does this
> > > increase in latency
> > > occur only when you see the errors, or whenever you do a
> large data
> > > transfer? In fact, I would suspect the errors to _decrease_
> > > the latency
> > > with respect to a normal transfer.
> >
> > > The latencies are almost certainly caused by the USB host
> controller
> > > driver. I'm planning improvements to uhci-hcd which should
> > > help reduce
> > > the latency, but it will still be on the large side. And I
> > > won't have
> > > time to write the changes to the driver
> > The following tests are made with 'IRQ 8' at 95, rtc_wakeup
> at 89(99):
> > * Heavy mmap load, no oom: max jitter: 42.1% ( 51 usec)
> > * Heavy mmap load, oom:max jitter: 11989.2% (14635 usec)
> > (but still "missed irqs: 0", so IRQ 8 was also blocked for 14 ms)
>
> did you get
> > Even when the errors described in my previous mail does not occur,
> > massive USB stick transfers cause latencies of 1 to 2 milliseconds,
> > which is way too much for realtime control systems.
>
> do these occur under PREEMPT_RT? If yes, do you get any
> useful trace if
> you enable all
> * kus Kusche Klaus <[EMAIL PROTECTED]> wrote:
> >IRQ 7-724 0d..11us : end_8259A_irq (do_hardirq)
> >IRQ 7-724 0d..11us!: enable_8259A_irq (do_hardirq)
> >IRQ 7-724 0d... 832us : do_hardirq (do_irqd)
> >IRQ 7-724 0d...
When a process running with RT priority dumps core,
I get the following BUG:
Apr 11 13:44:23 OF455 kern.err kernel: BUG: rtc2:833 RT task
yield()-ing!
Apr 11 13:44:23 OF455 kern.warn kernel: [] yield+0x61/0x70
(8)
Apr 11 13:44:23 OF455 kern.warn kernel: []
coredump_wait+0x79/0xc0 (20)
Apr 11
>From the three sources of multi-millisecond latency in my experiments
(console messages to dead serial console, USB I/O, CF Card bulk read),
I've analyzed one:
The latency of around 70 milliseconds in low-priority RT processes
when running a "dd if=/dev/hda of=/dev/null" in parallel (where hda
I'm using 2.6.24.2 with -rt1 applied
(applies cleanly with some offsets) on x86.
I've *not* configured PROVE_LOCKING (too expensive).
However, I selected all the latency tracings, timings and histograms,
among them CRITICAL_IRQSOFF_TIMING.
This implicitly configures TRACE_IRQFLAGS.
However,
38 matches
Mail list logo
