Re: [Emc-users] BBB 14: unexpected realtime delay on RT thread 1
I’ll start looking. Thanks for the pointer On 14 Jan 2014, at 19:57, Michael Haberler wrote: > > Am 14.01.2014 um 19:33 schrieb Charles Steinkuehler > : > >> On 1/14/2014 12:21 PM, Bas de Bruijn wrote: >>> Yes, the BeagleBone Black, >>> the PRU is hat 10us step length and 1us step time. > > The PRU code cannot cause an "unexpected realtime delay", since this is a > message generated by the motion component. Since missing steps have no way to > find their way back into HAL except if you are running a stepper servo loop > with encoders (which you likely dont), this cannot be the source either. > > The logic for "unexpected realtime delay" is here: > https://github.com/mhaberler/linuxcnc/blob/unified-build-candidate-3/src/emc/motion/control.c#L249 > > > What motion does is: > - observes its invocation interval over a window of samples, > - once the window is filled with samples, start checking: > - compare the current invocation time to all samples > - if the current interval is longer than 1.2 of any of the samples then > generate an "unexpected realtime delay" message. > > What you want to do is investigate the functions on the servo thread. > > see the 'show thread' and 'show param' commands > http://www.linuxcnc.org/docs/devel/html/hal/tutorial.html, in particular the > '.time' and '.tmax' values, which might give a hint which thread function > takes too long. > > It might be worth looking at motion execution times, see the PARAMETERS > section ofhttp://www.linuxcnc.org/docs/devel/html/man/man9/motion.9.html. > > - Michael > >>> The servo task period, in nano sec is 100. >>> When i run a latency test i get a jitter of approximately 56000 to 6 ns >>> max Jitter >>> >>> I’ll disable the screensaver to check. >> >> The latency you report is typical for the BeagleBone. The "unexpected >> realtime delay" is something I see occasionally on my systems. I >> haven't fully dug into the cause, but it seems like something in >> LinuxCNC has hard-coded jitter numbers and doesn't like the BeagleBone >> (which has higher IRQ jitter than a typical x86 system). I've never had >> it cause issues, however, and it is not correlated with a spike in >> latency as measured by either the LinuxCNC latency-test or the xenomai >> latency utility. >> >> I also run with the screen saver and display power management enabled on >> my BeagleBone and have not had any issues with "missed steps". >> >> I suggest you review the current setting on your stepper drivers, >> cooling on the stepper drivers, the mechanical system (make sure >> everything moves freely and can't bind), and review your machine setup. >> >> I did have to dial my acceleration down to around 2400 mm/s/s to avoid >> missing steps with some Slic3r generated infill gcode, but that had >> nothing to do with the BeagleBone, my steppers and drivers just weren't >> up to the task when running off 12V. >> >> -- >> Charles Steinkuehler >> char...@steinkuehler.net >> >> -- >> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >> Learn Why More Businesses Are Choosing CenturyLink Cloud For >> Critical Workloads, Development Environments & Everything In Between. >> Get a Quote or Start a Free Trial Today. >> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk___ >> Emc-users mailing list >> Emc-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/emc-users > > > -- > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > ___ > Emc-users mailing list > Emc-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/emc-users -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk ___ Emc-users mailing list Emc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-users
Re: [Emc-users] BBB 14: unexpected realtime delay on RT thread 1
Am 14.01.2014 um 19:33 schrieb Charles Steinkuehler : > On 1/14/2014 12:21 PM, Bas de Bruijn wrote: >> Yes, the BeagleBone Black, >> the PRU is hat 10us step length and 1us step time. The PRU code cannot cause an "unexpected realtime delay", since this is a message generated by the motion component. Since missing steps have no way to find their way back into HAL except if you are running a stepper servo loop with encoders (which you likely dont), this cannot be the source either. The logic for "unexpected realtime delay" is here: https://github.com/mhaberler/linuxcnc/blob/unified-build-candidate-3/src/emc/motion/control.c#L249 What motion does is: - observes its invocation interval over a window of samples, - once the window is filled with samples, start checking: - compare the current invocation time to all samples - if the current interval is longer than 1.2 of any of the samples then generate an "unexpected realtime delay" message. What you want to do is investigate the functions on the servo thread. see the 'show thread' and 'show param' commands http://www.linuxcnc.org/docs/devel/html/hal/tutorial.html, in particular the '.time' and '.tmax' values, which might give a hint which thread function takes too long. It might be worth looking at motion execution times, see the PARAMETERS section of http://www.linuxcnc.org/docs/devel/html/man/man9/motion.9.html. - Michael >> The servo task period, in nano sec is 100. >> When i run a latency test i get a jitter of approximately 56000 to 6 ns >> max Jitter >> >> I’ll disable the screensaver to check. > > The latency you report is typical for the BeagleBone. The "unexpected > realtime delay" is something I see occasionally on my systems. I > haven't fully dug into the cause, but it seems like something in > LinuxCNC has hard-coded jitter numbers and doesn't like the BeagleBone > (which has higher IRQ jitter than a typical x86 system). I've never had > it cause issues, however, and it is not correlated with a spike in > latency as measured by either the LinuxCNC latency-test or the xenomai > latency utility. > > I also run with the screen saver and display power management enabled on > my BeagleBone and have not had any issues with "missed steps". > > I suggest you review the current setting on your stepper drivers, > cooling on the stepper drivers, the mechanical system (make sure > everything moves freely and can't bind), and review your machine setup. > > I did have to dial my acceleration down to around 2400 mm/s/s to avoid > missing steps with some Slic3r generated infill gcode, but that had > nothing to do with the BeagleBone, my steppers and drivers just weren't > up to the task when running off 12V. > > -- > Charles Steinkuehler > char...@steinkuehler.net > > -- > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk___ > Emc-users mailing list > Emc-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/emc-users -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk ___ Emc-users mailing list Emc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-users
Re: [Emc-users] BBB 14: unexpected realtime delay on RT thread 1
Maybe the error generated has nothing to do with the missing steps per se, it just got me triggered. I have acceleration about 2000 and I’ll turn it down some more. Other parts got printed without problems. I’ll review the tension on my belts. On the arduino clone I have approx 1000 mm/s^2 because the extruder on the effector introduces some vibrations. My new frame is very rigid so everything will be transferred directly to the motor etc. My new extruder will be half the weight so I expect i’ll be able to do a but faster acceleration (if needed). The slic3r generated code (no arc although I have tried to generate with arc support) will slow down in the radii. but that’s to be expected. I’ll do some experimenting with G64 P0.02 Q0.02 On 14 Jan 2014, at 19:33, Charles Steinkuehler wrote: > On 1/14/2014 12:21 PM, Bas de Bruijn wrote: >> Yes, the BeagleBone Black, >> the PRU is hat 10us step length and 1us step time. >> The servo task period, in nano sec is 100. >> When i run a latency test i get a jitter of approximately 56000 to 6 ns >> max Jitter >> >> I’ll disable the screensaver to check. > > The latency you report is typical for the BeagleBone. The "unexpected > realtime delay" is something I see occasionally on my systems. I > haven't fully dug into the cause, but it seems like something in > LinuxCNC has hard-coded jitter numbers and doesn't like the BeagleBone > (which has higher IRQ jitter than a typical x86 system). I've never had > it cause issues, however, and it is not correlated with a spike in > latency as measured by either the LinuxCNC latency-test or the xenomai > latency utility. > > I also run with the screen saver and display power management enabled on > my BeagleBone and have not had any issues with "missed steps". > > I suggest you review the current setting on your stepper drivers, > cooling on the stepper drivers, the mechanical system (make sure > everything moves freely and can't bind), and review your machine setup. > > I did have to dial my acceleration down to around 2400 mm/s/s to avoid > missing steps with some Slic3r generated infill gcode, but that had > nothing to do with the BeagleBone, my steppers and drivers just weren't > up to the task when running off 12V. > > -- > Charles Steinkuehler > char...@steinkuehler.net > > -- > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk___ > Emc-users mailing list > Emc-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/emc-users -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk ___ Emc-users mailing list Emc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-users
Re: [Emc-users] BBB 14: unexpected realtime delay on RT thread 1
On 1/14/2014 12:21 PM, Bas de Bruijn wrote: > Yes, the BeagleBone Black, > the PRU is hat 10us step length and 1us step time. > The servo task period, in nano sec is 100. > When i run a latency test i get a jitter of approximately 56000 to 6 ns > max Jitter > > I’ll disable the screensaver to check. The latency you report is typical for the BeagleBone. The "unexpected realtime delay" is something I see occasionally on my systems. I haven't fully dug into the cause, but it seems like something in LinuxCNC has hard-coded jitter numbers and doesn't like the BeagleBone (which has higher IRQ jitter than a typical x86 system). I've never had it cause issues, however, and it is not correlated with a spike in latency as measured by either the LinuxCNC latency-test or the xenomai latency utility. I also run with the screen saver and display power management enabled on my BeagleBone and have not had any issues with "missed steps". I suggest you review the current setting on your stepper drivers, cooling on the stepper drivers, the mechanical system (make sure everything moves freely and can't bind), and review your machine setup. I did have to dial my acceleration down to around 2400 mm/s/s to avoid missing steps with some Slic3r generated infill gcode, but that had nothing to do with the BeagleBone, my steppers and drivers just weren't up to the task when running off 12V. -- Charles Steinkuehler char...@steinkuehler.net signature.asc Description: OpenPGP digital signature -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk___ Emc-users mailing list Emc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-users
Re: [Emc-users] BBB 14: unexpected realtime delay on RT thread 1
Yes, the BeagleBone Black, the PRU is hat 10us step length and 1us step time. The servo task period, in nano sec is 100. When i run a latency test i get a jitter of approximately 56000 to 6 ns max Jitter I’ll disable the screensaver to check. On 14 Jan 2014, at 19:13, Sebastian Kuzminsky wrote: > On 1/14/14 11:07 , Bas de Bruijn wrote: >> I’m having an error resulting in missing steps so the layers I’m printing >> shift. The file I print is pretty big (17MB). But the error does not happen >> at the same hight. How can i find what’s happening? Do i need to switch off >> the screensaver for example? or is there something else i can check? > > You're on the Beaglebone Black, running Xenomai? I'm not sure exactly > what the realtime performance and jitter on that platform should be > expected to me. > > Try running the latency measuring tool (it used to be called > 'latency-test', there's also latencyhistogram and latencyplot) and see > if turning off the screensaver helps. > > > -- > Sebastian Kuzminsky > > -- > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > ___ > Emc-users mailing list > Emc-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/emc-users -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk ___ Emc-users mailing list Emc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-users
Re: [Emc-users] BBB 14: unexpected realtime delay on RT thread 1
On 1/14/14 11:07 , Bas de Bruijn wrote: > I’m having an error resulting in missing steps so the layers I’m printing > shift. The file I print is pretty big (17MB). But the error does not happen > at the same hight. How can i find what’s happening? Do i need to switch off > the screensaver for example? or is there something else i can check? You're on the Beaglebone Black, running Xenomai? I'm not sure exactly what the realtime performance and jitter on that platform should be expected to me. Try running the latency measuring tool (it used to be called 'latency-test', there's also latencyhistogram and latencyplot) and see if turning off the screensaver helps. -- Sebastian Kuzminsky -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk ___ Emc-users mailing list Emc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-users
[Emc-users] BBB 14: unexpected realtime delay on RT thread 1
I’m having an error resulting in missing steps so the layers I’m printing shift. The file I print is pretty big (17MB). But the error does not happen at the same hight. How can i find what’s happening? Do i need to switch off the screensaver for example? or is there something else i can check? Thanks, Bas -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk ___ Emc-users mailing list Emc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/emc-users
