Re: [etherlab-users] yet another datagrams UNMATCHED - DC 0.2ms / 5kHz // igb kernel 3.18

[Sebastien Blanchet]

Hi Daniel,

It was done in a RTAI kernel-thread.
I had modified the example supplied by IgH Ethercat Master.
I had plugged a digital scope to see the jitter, while checking missing packets 
in dmesg. The goal was just to see how fast EtherCAT was.


best regards
---
Sebastien BLANCHET

On 03/08/2018 03:54 PM, PAUL-SOFTWARE wrote:

Hi Sebastien,

did you implement your code as a kernel-thread or as a rtai-userspace-process 
getting these results?

Mit freundlichen Grüßen / Best Regards

i.A. Daniel Koch

Elektronik/Software

Paul Maschinenfabrik GmbH & Co. KG
Max-Paul-Str. 1 | 88525 Duermentingen | Deutschland/Germany
Phone: +49 (7371) 500 - 0 | Fax: +49 (7371) 500 - 111
Mail: softw...@paul.eu | Web: www.paul.eu

Kommanditgesellschaft, Sitz Duermentingen, Registergericht Ulm HRA 650073, 
Pers. haftende Gesellschafterin:
Paul Maschinenfabrik GmbH, Sitz Riedlingen, Registergericht Ulm HRB 650013, GF: 
Werner Paul, USt-IdNr. DE146544409

-Ursprüngliche Nachricht-
Von: etherlab-users [mailto:etherlab-users-boun...@etherlab.org] Im Auftrag von 
Sebastien Blanchet
Gesendet: Mittwoch, 14. Februar 2018 14:37
An: etherlab-users@etherlab.org
Betreff: Re: [etherlab-users] yet another datagrams UNMATCHED - DC 0.2ms / 5kHz 
// igb kernel 3.18

Hi Jürgen,

I can share with you some results with the 8139too and r8169 ethercat native 
drivers when used with RTAI and Preempt-RT kernel.

- with RTAI and 8139too you can achieve 10 KHz for the ethercat control loop 
without losing datagram. (tested on Debian 6.0 i386 with RTAI-3.8.1). The 
computer was a DELL Precision 390 with a Core Duo 2 processor, the slave were 4 
Beckhoff Ethercat terminals (EL1004, EL2004, EL3102, EL4102).

- with Preempt-RT and r8169 you can achieve 1 KHz for the ethercat control loop 
without losing datagram. (tested on Debian 7.0 i386 with Debian RT kernel).
At 2 KHz, it does not work very well (sometime you miss datagrams), so the 
limit is somewhere in the 1...2 KHz range. The computer is a Beckhoff 
C6920-0040 with the PCIe extension slots, and a Celeron B810 processor. The 
slaves are 4 Kollmorgen AKD servo drives + 13 other Ethercat terminals from 
Beckhoff.

Finally Preempt-RT is slower than RTAI, but it is really easier than RTAI.

best regards,
--
Sebastien BLANCHET




On 02/13/2018 02:12 PM, Christoph Schroeder wrote:

Hi Jürgen,

On 02/13/2018 10:01 AM, Jürgen Walter • DATATRONiQ wrote:

I see- will get another Intel card (although compatible ones (kernel driver
e1000, e1000e) seem to be hard to come by these days) and try with anther
kernel driver.

I also did some tests with the EtherCAT master, Xenomai and different network
devices compatible with the ec_e1000e driver.
Test system was a Core-i5 (4th generation) with a Debian Wheezy  (Kernel 3.2)
with two slaves attached (Microship LAN9252 in simple Digital I/O mode). I
observed the following:

   * even with the PREEMPT patch and the native driver ec_e1000e I could barely
 achieve 2000Hz cycle rate without frame losses
   * with Xenomai I could go over 1Hz without a problem, but there are some
 other issues with Xenomai:
   o the EtherCAT master is not ready to use the newer Xenomai 3.x branch 
and
 the Xenomai 2.x branch is not supported anymore
   o I got some issues with long term stability on my test system (Kernel
 panic) and decided not to use it on a productive system
   o I would suggest you go for RTAI since you will also get far more help
 from the EtherCAT community, I could barely find other people here who
 also use Xenomai with the EtherCAT master
   * while using Xenomai I observed, that there is also a huge difference 
between
 different network chips:
   o I initially used a card with an Intel 82572EI (introduced Q4'05) and
 also tried a Dual port card with an Intel 82571EB (introduced Q3'05)
 which had basically the same results on my test system
   o the best results I got were achieved with an Intel 82574L (introduced 
Q2'0)

I didn't try RTAI till now, but I think you will get the same or even better
results than I got with Xenomai.

The cards with Intel 82574L we got were sold as "Intel Gigabit CT Desktop
Adapter". The chip will be produced until 2020, so getting a card with it should
be no problem. Maybe this will help you.


Best regards,
Christoph



Helmholtz-Zentrum Berlin für Materialien und Energie GmbH

Mitglied der Hermann von Helmholtz-Gemeinschaft Deutscher Forschungszentren e.V.

Aufsichtsrat: Vorsitzender Dr. Karl Eugen Huthmacher, stv. Vorsitzende Dr. Jutta
Koch-Unterseher
Geschäft

Re: [etherlab-users] yet another datagrams UNMATCHED - DC 0.2ms / 5kHz // igb kernel 3.18

[Sebastien Blanchet]

Hi Jürgen,

I can share with you some results with the 8139too and r8169 ethercat native 
drivers when used with RTAI and Preempt-RT kernel.


- with RTAI and 8139too you can achieve 10 KHz for the ethercat control loop 
without losing datagram. (tested on Debian 6.0 i386 with RTAI-3.8.1). The 
computer was a DELL Precision 390 with a Core Duo 2 processor, the slave were 4 
Beckhoff Ethercat terminals (EL1004, EL2004, EL3102, EL4102).


- with Preempt-RT and r8169 you can achieve 1 KHz for the ethercat control loop 
without losing datagram. (tested on Debian 7.0 i386 with Debian RT kernel).
At 2 KHz, it does not work very well (sometime you miss datagrams), so the limit 
is somewhere in the 1...2 KHz range. The computer is a Beckhoff C6920-0040 with 
the PCIe extension slots, and a Celeron B810 processor. The slaves are 4 
Kollmorgen AKD servo drives + 13 other Ethercat terminals from Beckhoff.


Finally Preempt-RT is slower than RTAI, but it is really easier than RTAI.

best regards,
--
Sebastien BLANCHET




On 02/13/2018 02:12 PM, Christoph Schroeder wrote:

Hi Jürgen,

On 02/13/2018 10:01 AM, Jürgen Walter • DATATRONiQ wrote:
I see- will get another Intel card (although compatible ones (kernel driver 
e1000, e1000e) seem to be hard to come by these days) and try with anther 
kernel driver.
I also did some tests with the EtherCAT master, Xenomai and different network 
devices compatible with the ec_e1000e driver.
Test system was a Core-i5 (4th generation) with a Debian Wheezy  (Kernel 3.2) 
with two slaves attached (Microship LAN9252 in simple Digital I/O mode). I 
observed the following:


  * even with the PREEMPT patch and the native driver ec_e1000e I could barely
achieve 2000Hz cycle rate without frame losses
  * with Xenomai I could go over 1Hz without a problem, but there are some
other issues with Xenomai:
  o the EtherCAT master is not ready to use the newer Xenomai 3.x branch and
the Xenomai 2.x branch is not supported anymore
  o I got some issues with long term stability on my test system (Kernel
panic) and decided not to use it on a productive system
  o I would suggest you go for RTAI since you will also get far more help
from the EtherCAT community, I could barely find other people here who
also use Xenomai with the EtherCAT master 
  * while using Xenomai I observed, that there is also a huge difference between

different network chips:
  o I initially used a card with an Intel 82572EI (introduced Q4'05) and
also tried a Dual port card with an Intel 82571EB (introduced Q3'05)
which had basically the same results on my test system
  o the best results I got were achieved with an Intel 82574L (introduced Q2'0) 

I didn't try RTAI till now, but I think you will get the same or even better 
results than I got with Xenomai.


The cards with Intel 82574L we got were sold as "Intel Gigabit CT Desktop 
Adapter". The chip will be produced until 2020, so getting a card with it should 
be no problem. Maybe this will help you.



Best regards,
Christoph



Helmholtz-Zentrum Berlin für Materialien und Energie GmbH

Mitglied der Hermann von Helmholtz-Gemeinschaft Deutscher Forschungszentren e.V.

Aufsichtsrat: Vorsitzender Dr. Karl Eugen Huthmacher, stv. Vorsitzende Dr. Jutta 
Koch-Unterseher

Geschäftsführung: Prof. Dr. Bernd Rech (kommissarisch), Thomas Frederking

Sitz Berlin, AG Charlottenburg, 89 HRB 5583

Postadresse:
Hahn-Meitner-Platz 1
D-14109 Berlin

http://www.helmholtz-berlin.de


___
etherlab-users mailing list
etherlab-users@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-users





___
etherlab-users mailing list
etherlab-users@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-users

Re: [etherlab-users] yet another datagrams UNMATCHED - DC 0.2ms / 5kHz // igb kernel 3.18

[Christoph Schroeder]

Hi Jürgen,

On 02/13/2018 10:01 AM, Jürgen Walter • DATATRONiQ wrote:
I see- will get another Intel card (although compatible ones (kernel driver 
e1000, e1000e) seem to be hard to come by these days) and try with anther 
kernel driver.
I also did some tests with the EtherCAT master, Xenomai and different network 
devices compatible with the ec_e1000e driver.
Test system was a Core-i5 (4th generation) with a Debian Wheezy  (Kernel 3.2) 
with two slaves attached (Microship LAN9252 in simple Digital I/O mode). I 
observed the following:

 *   even with the PREEMPT patch and the native driver ec_e1000e I could barely 
achieve 2000Hz cycle rate without frame losses
 *   with Xenomai I could go over 1Hz without a problem, but there are some 
other issues with Xenomai:
*   the EtherCAT master is not ready to use the newer Xenomai 3.x branch 
and the Xenomai 2.x branch is not supported anymore
*   I got some issues with long term stability on my test system (Kernel 
panic) and decided not to use it on a productive system
*   I would suggest you go for RTAI since you will also get far more help 
from the EtherCAT community, I could barely find other people here who also use 
Xenomai with the EtherCAT master
 *   while using Xenomai I observed, that there is also a huge difference 
between different network chips:
*   I initially used a card with an Intel 82572EI (introduced Q4'05) and 
also tried a Dual port card with an Intel 82571EB (introduced Q3'05) which had 
basically the same results on my test system
*   the best results I got were achieved with an Intel 82574L (introduced 
Q2'0)

I didn't try RTAI till now, but I think you will get the same or even better 
results than I got with Xenomai.

The cards with Intel 82574L we got were sold as "Intel Gigabit CT Desktop 
Adapter". The chip will be produced until 2020, so getting a card with it should be 
no problem. Maybe this will help you.

Best regards,
Christoph



Helmholtz-Zentrum Berlin für Materialien und Energie GmbH

Mitglied der Hermann von Helmholtz-Gemeinschaft Deutscher Forschungszentren e.V.

Aufsichtsrat: Vorsitzender Dr. Karl Eugen Huthmacher, stv. Vorsitzende Dr. 
Jutta Koch-Unterseher
Geschäftsführung: Prof. Dr. Bernd Rech (kommissarisch), Thomas Frederking

Sitz Berlin, AG Charlottenburg, 89 HRB 5583

Postadresse:
Hahn-Meitner-Platz 1
D-14109 Berlin

http://www.helmholtz-berlin.de
___
etherlab-users mailing list
etherlab-users@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-users

Re: [etherlab-users] yet another datagrams UNMATCHED - DC 0.2ms / 5kHz // igb kernel 3.18

[Jürgen Walter • DATATRONiQ]

Hello Gavin,

many thanks for the detailed reply!

On 13 Feb 2018, at 5:04, Gavin Lambert wrote:

The ec_igb driver is relatively new (and only exists in 3.18 in the 
stable-1.5 branch, although the unofficial default-branch patchset 
 
 extends this to later kernels), so it’s certainly possible that it 
has some bugs.


I see- will get another Intel card (although compatible ones (kernel 
driver e1000, e1000e) seem to be hard to come by these days) and try 
with anther kernel driver.


But the most likely thing is that your hardware or kernel 
configuration can’t sustain that cycle time.  Sub-millisecond cycle 
times are possible with PREEMPT_RT (provided that you are not using 
ec_generic), but are highly dependent on your hardware (and its 
general latency) and require a very carefully written realtime loop.  
You may have better luck using RTAI or Xenomai, however.


Understood -will try first an RT kernel (I read that Debian brings out 
of the box support for linux-image-rt-deb - will try that first; 
else going to recompile)


Another possibility is that you might need to specify a different 
shift time in your call to ecrt_slave_config_dc.  If you have the 
ability to hook a scope to your slave to measure the ECAT SOF vs. 
SYNC0, this can help to determine if the ECAT frame is arriving with 
the correct phase – you usually want SOF and SYNC0/SYNC1 to cleanly 
alternate, not letting it jitter sometimes on one side and sometimes 
the other.  Check your slave’s documentation for guidelines on 
specifying the shift time.
that is a bit beyond my abilities right now - however, the slave works 
fine under Twincat using 200us as cycle time, and 0 for shift (but that 
is just in the Twincat GUI - who knows if Twincat actually negotiates / 
adapts some phase shift behind the scenes?)


Another option, if your slave supports it or if you can switch to an 
alternate slave, is to use oversampling.  In one application I sample 
data at 4000 Hz with a 1ms ECAT cycle time (PREEMPT_RT with e1000e) by 
using a slave that is configured to capture 4 samples per ECAT cycle 
(triggered by the DC sync clock).
Yes, this one already does use oversampling: the measurement device 
(slave) generates 50.000 samples per second (actually: 8x because 8 
channels) - the slaves stuffs 10 samples into one PDO so I have to make 
sure to read/exchange the PDO 5.000x per second (if I understand 
correctly, this is how the vendor of the slave calculates the 200us)



Again, many thanks for your help - I will report back with my results!! 
Jürgen

___
etherlab-users mailing list
etherlab-users@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-users

Re: [etherlab-users] yet another datagrams UNMATCHED - DC 0.2ms / 5kHz // igb kernel 3.18

[Gavin Lambert]

The ec_igb driver is relatively new (and only exists in 3.18 in the stable-1.5 
branch, although the unofficial default-branch patchset 
<https://sourceforge.net/u/uecasm/etherlab-patches/ci/default/tree/#readme>  
extends this to later kernels), so it’s certainly possible that it has some 
bugs.

 

But the most likely thing is that your hardware or kernel configuration can’t 
sustain that cycle time.  Sub-millisecond cycle times are possible with 
PREEMPT_RT (provided that you are not using ec_generic), but are highly 
dependent on your hardware (and its general latency) and require a very 
carefully written realtime loop.  You may have better luck using RTAI or 
Xenomai, however.

 

Another possibility is that you might need to specify a different shift time in 
your call to ecrt_slave_config_dc.  If you have the ability to hook a scope to 
your slave to measure the ECAT SOF vs. SYNC0, this can help to determine if the 
ECAT frame is arriving with the correct phase – you usually want SOF and 
SYNC0/SYNC1 to cleanly alternate, not letting it jitter sometimes on one side 
and sometimes the other.  Check your slave’s documentation for guidelines on 
specifying the shift time.

 

Another option, if your slave supports it or if you can switch to an alternate 
slave, is to use oversampling.  In one application I sample data at 4000 Hz 
with a 1ms ECAT cycle time (PREEMPT_RT with e1000e) by using a slave that is 
configured to capture 4 samples per ECAT cycle (triggered by the DC sync clock).

 

From: Jürgen Walter • DATATRONiQ
Sent: Tuesday, 13 February 2018 14:23
To: etherlab-users@etherlab.org
Subject: [etherlab-users] yet another datagrams UNMATCHED - DC 0.2ms / 5kHz // 
igb kernel 3.18

 

Hello list, 

I have been fighting with an ethercat slave (measurement device from imc) over 
the past week. I got it somewhat (mostly?) working but I am seeing really a lot 
of the dreaded "datagrams UNMATCHED" in syslog.

My setup is an APU2 (AMD Chipset but "Ethernet controller: Intel Corporation 
I210 Gigabit Network Connection (rev 03)" -  <https://pcengines.ch/apu2.htm> 
https://pcengines.ch/apu2.htm) with Ubuntu 12.04 and a "low latency" kernel 
3.18 installed - to my delight (after doing ./bootstrap.sh) I found the option 
"--enable-igb" available with "./configure --help" (it was not available on the 
3.2 and 3.13 (or some earlier experiments with a newer distro and Kernel 4.4)). 
I enabled it and after "make && make modules && make install && make 
modules_install" I can do "/etc/init.d/ethercat start" and the module loads 
with:

ec_igb :01:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow 
Control: RX/TX

Next, I have adapted the main.c in "examples/dc_user"

1.  customising the PDO section and inserting my output of "ethercat 
cstruct"

2.  changing the frequency to: "#define FREQUENCY 5000" (this is so I can 
get a DC cycle time of 200 microseconds (working fine in TwinCat 3)) 

3.  reading (some of the) mapped PDOs in the cyclic task

4.  customizing "ecrt_slave_config_dc" (fixed DC cycle time, nothing else 
works with this slave): ecrt_slave_config_dc(sc, 0x0300, 20, 0, 0, 0);

I can see from the logs that the DC setup succeeds and the slaves goes into OP 
(it is the only slave, a 1:1 direct connection).
I can also confirm that the PDO exchange works so far because the data I am 
reading makes sense.

Yet, my syslog essentially gets flooded with those two

EtherCAT WARNING 0: 58 datagrams UNMATCHED!
EtherCAT WARNING: Datagram f66b3b8c (domain0-0-main) was SKIPPED 29 times.

In addition - when leaving the "check_domain_state()" call enabled, I am seeing 
those (warnings?):

Domain1: WC 0.
Domain1: State 0.
Domain1: WC 1.
Domain1: State 2.
Domain1: WC 0.
Domain1: State 0.
Domain1: WC 1.

When I reduce the "FREQUENCY" back to 1000 the problems pretty much disappear; 
however, my problem is that I need to run the DC cycle at 200 microseconds 
(each channel uses PDOs with 10 real/float values - I need to read each channel 
5000x per second in order to get all 50.000 measurement samples).

I have also tried this setup on a newer Ubuntu with Kernel 4.4 and the generic 
driver - the results are essentially the same.

I am really not sure how to proceed from here - should I give the Xenomai a 
try? Or buy another PCI/Intel network card (PRO/GT - what model works still 
with the e1000/e drivers?) - or am I in general out of luck because 200us DC 
cycles are completely unrealistic with the ec_master? 

Any help would be greatly appreciated!! Jürgen 

___
etherlab-users mailing list
etherlab-users@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-users

[etherlab-users] yet another datagrams UNMATCHED - DC 0.2ms / 5kHz // igb kernel 3.18

[Jürgen Walter • DATATRONiQ]

Hello list,

I have been fighting with an ethercat slave (measurement device from 
imc) over the past week. I got it somewhat (mostly?) working but I am 
seeing really a lot of the dreaded "datagrams UNMATCHED" in syslog.


My setup is an APU2 (AMD Chipset but "Ethernet controller: Intel 
Corporation I210 Gigabit Network Connection (rev 03)" - 
https://pcengines.ch/apu2.htm) with Ubuntu 12.04 and a "low latency" 
kernel 3.18 installed - to my delight (after doing ./bootstrap.sh) I 
found the option "--enable-igb" available with "./configure --help" (it 
was not available on the 3.2 and 3.13 (or some earlier experiments with 
a newer distro and Kernel 4.4)). I enabled it and after "make && make 
modules && make install && make modules_install" I can do 
"/etc/init.d/ethercat start" and the module loads with:


	ec_igb :01:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full 
Duplex, Flow Control: RX/TX


Next, I have adapted the main.c in "examples/dc_user"

1. customising the PDO section and inserting my output of "ethercat 
cstruct"
2. changing the frequency to: "#define FREQUENCY 5000" (this is so I can 
get a DC cycle time of 200 microseconds (working fine in TwinCat 3))

3. reading (some of the) mapped PDOs in the cyclic task
4. customizing "ecrt_slave_config_dc" (fixed DC cycle time, nothing else 
works with this slave): ecrt_slave_config_dc(sc, 0x0300, 20, 0, 0, 
0);


I can see from the logs that the DC setup succeeds and the slaves goes 
into OP (it is the only slave, a 1:1 direct connection).
I can also confirm that the PDO exchange works so far because the data I 
am reading makes sense.


Yet, my syslog essentially gets flooded with those two

EtherCAT WARNING 0: 58 datagrams UNMATCHED!
	EtherCAT WARNING: Datagram f66b3b8c (domain0-0-main) was SKIPPED 29 
times.


In addition - when leaving the "check_domain_state()" call enabled, I am 
seeing those (warnings?):


Domain1: WC 0.
Domain1: State 0.
Domain1: WC 1.
Domain1: State 2.
Domain1: WC 0.
Domain1: State 0.
Domain1: WC 1.


When I reduce the "FREQUENCY" back to 1000 the problems pretty much 
disappear; however, my problem is that I need to run the DC cycle at 200 
microseconds (each channel uses PDOs with 10 real/float values - I need 
to read each channel 5000x per second in order to get all 50.000 
measurement samples).


I have also tried this setup on a newer Ubuntu with Kernel 4.4 and the 
generic driver - the results are essentially the same.


I am really not sure how to proceed from here - should I give the 
Xenomai a try? Or buy another PCI/Intel network card (PRO/GT - what 
model works still with the e1000/e drivers?) - or am I in general out of 
luck because 200us DC cycles are completely unrealistic with the 
ec_master?



Any help would be *greatly* appreciated!! Jürgen

___
etherlab-users mailing list
etherlab-users@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-users