Re: [etherlab-users] Control loop at higher frequencies

[Gavin Lambert]

When you use the master debug interface (or the newly added “ethercat pcap” 
command, although you won’t have that in your version yet), the receive 
timestamps are the time that they are received by the master, which is the time 
that you called ecrt_master_receive.

You can get a better understanding of the “real” wire transfer time by 
connecting a separate monitoring device between the master and first slave – a 
proper network monitor is ideal, but you can made do with a standard Ethernet 
hub/switch with three ports connected (master, first slave, monitoring laptop 
running Wireshark), although for best results (especially if it’s a Windows 
laptop) you should disable all the network protocols on the laptop adapter 
other than the minimum it needs to do the Wireshark monitoring, so that it 
doesn’t inadvertently transmit packets of its own.  (These won’t hurt the 
network itself, since both the master and slave will drop non-EtherCAT packets, 
but it may worsen the latency.)  This will still be somewhat inaccurate since 
it’s still timestamping when the packet was processed rather than when it was 
actually received in hardware, but it’s likely to be more accurate than doing 
it on the master since it’s not waiting for the cycle delay.

As Graeme said, you can also put the monitoring device elsewhere in the network 
(all packets will always pass every point in the network – except the end), 
although then it becomes harder to meaningfully use the timestamps.


Gavin Lambert
Senior Software Developer

[cid:logo_compac_5dcf97ef-52f5-498c-8b9b-728410ddffaf.png]
[cid:compacicon_82e8a8c7-154a-4a32-9720-a5badb6258e0.png]
 [cid:facebook_fa85b924-53b9-45cc-8162-0564f64ec3a3.png] 
  
[cid:linkedin_4ec016ad-84fa-443c-85a3-b9615a4ccef8.png] 
  
[cid:youtube_32142163-fc27-4aed-b14d-e8a377f98a6d.png] 
  
[cid:twitter_d89338d8-98c8-4b65-9a9e-7b1333160b0d.png] 
  
[cid:insta2_1cd85de9-b3a2-4971-9904-52b2481a7c82.png] 


COMPAC SORTING EQUIPMENT LTD | 4 Henderson Pl | Onehunga | Auckland 1061 | New 
Zealand
Switchboard: +64 96 34 00 88 | tomra.com

The information contained in this communication and any attachment is 
confidential and may be legally privileged. It should only be read by the 
person(s) to whom it is addressed. If you have received this communication in 
error, please notify the sender and delete the communication.

From: Graeme Foot
Sent: Thursday, 31 October 2019 11:49
To: Jordan Palacios 
Cc: etherlab-users@etherlab.org
Subject: Re: [etherlab-users] Control loop at higher frequencies

Hi,

It is sounding like the data time on the wire is taking too long (> 250us).

Besides doubling the "DC system time transmission delay" of the last slave, you 
can also check the "Diff [ns]" value of your first slave.  This may give you a 
more accurate idea if you have a star topology, as the return trip of the frame 
from the last slave bypasses the fingers of the stars.  e.g.:

ethercat slaves -v -p0

=== Master 0, Slave 0 ===
Alias: 10001
Device: Main
State: OP
Flag: +
Identity:
  Vendor Id:   0x0002
  Product code:0x04562c52
  Revision number: 0x0011
  Serial number:   0x
DL information:
  FMMU bit operation: no
  Distributed clocks: yes, 64 bit
  DC system time transmission delay: 0 ns
Port  Type  Link  LoopSignal  NextSlave  RxTime [ns]  Diff [ns]   NextDc 
[ns]
   0* EBUS  upopenyes -   3638440690   0   0
   1  MII   upopenyes 1   36384424101720 560
   2  N/A   down  closed  no  --   -   -
   3  N/C   down  closed  no  --   -   -

You will also need to add on 2 * "master to first slave transmission delay" 
which you will probably need to guess.  If you use a Beckhoff CX2020 (or 
similar) where the CX2100 is directly connected to the EBus this will likely be 
around 150us.  If your master has an ethernet port and the first slave is an 
amp or a coupler this may be around 550us or more (depending on cable length).  
You could also calc the frame transmission time.  Your frame length is 782 
bytes.  So that should take ~ 6 - 7us.  Plus whatever other overheads the 
ethernet card / driver has.

Reducing the number of configured slaves (even with them still physically 
plugged in) will reduce the ec_master_domain_queue() and ec_master_send() 
overhead time, getting the frame to the wire quicker.  It will also reduce the 
frame length and the related frame transmission time (especially if you remove 
slaves with bigger datagram overhead).  It looks like this is enough to result 
in the total frame roundtrip time being reduce enough for it to return and be 
ready by the time that 

Re: [etherlab-users] Control loop at higher frequencies

[Graeme Foot]

Hi,

It is sounding like the data time on the wire is taking too long (> 250us).

Besides doubling the "DC system time transmission delay" of the last slave, you 
can also check the "Diff [ns]" value of your first slave.  This may give you a 
more accurate idea if you have a star topology, as the return trip of the frame 
from the last slave bypasses the fingers of the stars.  e.g.:

ethercat slaves -v -p0

=== Master 0, Slave 0 ===
Alias: 10001
Device: Main
State: OP
Flag: +
Identity:
  Vendor Id:   0x0002
  Product code:0x04562c52
  Revision number: 0x0011
  Serial number:   0x
DL information:
  FMMU bit operation: no
  Distributed clocks: yes, 64 bit
  DC system time transmission delay: 0 ns
Port  Type  Link  LoopSignal  NextSlave  RxTime [ns]  Diff [ns]   NextDc 
[ns]
   0* EBUS  upopenyes -   3638440690   0   0
   1  MII   upopenyes 1   36384424101720 560
   2  N/A   down  closed  no  --   -   -
   3  N/C   down  closed  no  --   -   -

You will also need to add on 2 * "master to first slave transmission delay" 
which you will probably need to guess.  If you use a Beckhoff CX2020 (or 
similar) where the CX2100 is directly connected to the EBus this will likely be 
around 150us.  If your master has an ethernet port and the first slave is an 
amp or a coupler this may be around 550us or more (depending on cable length).  
You could also calc the frame transmission time.  Your frame length is 782 
bytes.  So that should take ~ 6 - 7us.  Plus whatever other overheads the 
ethernet card / driver has.

Reducing the number of configured slaves (even with them still physically 
plugged in) will reduce the ec_master_domain_queue() and ec_master_send() 
overhead time, getting the frame to the wire quicker.  It will also reduce the 
frame length and the related frame transmission time (especially if you remove 
slaves with bigger datagram overhead).  It looks like this is enough to result 
in the total frame roundtrip time being reduce enough for it to return and be 
ready by the time that ec_master_receive() is called.


Just FYI, I have a machine downstairs at the moment with 42 slaves (14 of which 
are amps), linear topology.  The last slaves transmission delay is 19560ns.  
The first slaves Diff is 39390ns.  The EtherCAT frame size is 886 bytes.  The 
ec_master_receive() to ec_master_send() time is approx 23us.  So your values 
sound in line with what I have, except that the wireshark output is showing a 
large cycle time.

In general it's looking like the time on the wire is taking longer than it 
should.  How are you capturing the EtherCAT frame data?  e.g.: a switch between 
your master and first slave?  If so, try moving it further down the chain, 
doing an "ethercat rescan" and check the transmission delays.  See if the 
switch is causing a large delay between those slaves.

What is your network card and driver used by the master?  What is your realtime 
system?  If you are using tshark on the master PC there might be delays being 
introduced in the network card driver.


Regards,
Graeme.


From: etherlab-users  On Behalf Of Jordan 
Palacios
Sent: Thursday, 31 October 2019 4:36 AM
To: etherlab-users@etherlab.org
Subject: [etherlab-users] Control loop at higher frequencies

Hi.

We've been working with the etherlab master for some time now. On our current 
setup we have around 40 slaves and our control loop runs at 1Khz without any 
issues. We use the stable version of etherlab with the 20180622 patchset.

Recently we've tried increasing the loop frequency to achieve a better control. 
The target frequency is 4Khz. After some troubles we managed a stable control 
at 2Khz. Then we went for the 4Khz and this is where we have hit a roadblock.

A warning like this is generated each second:

[11547.183302] EtherCAT WARNING 0: 4000 datagrams UNMATCHED!
[11547.619399] EtherCAT WARNING: Datagram 88040b6bf318 (domain0-0-main) was 
SKIPPED 4004 times.

As per what Florian explained 
here this 
means that the answer to the last datagram sent has not been received yet. I 
don't think this is related to the rate at which we execute the control loop. 
We have instrumented it and is steady at 250us with a jitter below 10us (see 
attachment). Furthermore, we have also enabled the ethercat master debug 
interface. Here is a sample of the traffic output using tshark:

 7807 0.975750173 MS-NLB-PhysServer-19_95:2e:e1:b0 → BroadcastECAT 810 
'LRW': Len: 782, Adp 0x0, Ado 0x0, Wc 69
 7808 0.975757725 Congatec_2e:e1:b0 → BroadcastECAT 810 'LRW': Len: 782, 
Adp 0x0, Ado 0x0, Wc 0
 7809 0.976000284 MS-NLB-PhysServer-19_95:2e:e1:b0 → BroadcastECAT 810 
'LRW': Len: 782, Adp 0x0, Ado 0x0, Wc 69
 7810 0.976007872 Congatec_2e:e1:b0 → BroadcastECAT 810 'LRW': Len: 782, 

Re: [etherlab-users] ELM3004 (ADC) and SDOs requests

[Gavin Lambert]

It is normal for a slave to drop to SAFEOP+ERROR when it loses communication 
with the master app.

If a slave is remaining in OP+ERROR with the master app not running, that is 
almost certainly a fault in the slave itself.  (However: it's also worthwhile 
making sure that the slave is not the first one in the network -- try putting 
in an EK1100 as the first slave if you have nothing else.  In particular, the 
stock stable branch code will try to keep the first slave in OP all the time, 
which is usually not a good thing (but is relatively harmless when it's an 
infrastructure component such as the EK1100).

For your other questions (and also for the above), you may find it helpful to 
try the unofficial patchset at 
https://sourceforge.net/u/uecasm/etherlab-patches/ci/default/tree/.  This 
includes some feature and reliability improvements for CoE as well as a command 
allowing you to soft-reboot compatible slaves.  It also disables the 
first-slave-in-OP behaviour.


Gavin Lambert
Senior Software Developer

 


COMPAC SORTING EQUIPMENT LTD | 4 Henderson Pl | Onehunga | Auckland 1061 | New 
Zealand
Switchboard: +49 2630 96520 | https://www.tomra.com

The information contained in this communication and any attachment is 
confidential and may be legally privileged. It should only be read by the 
person(s) to whom it is addressed. If you have received this communication in 
error, please notify the sender and delete the communication.
-Original Message-
From: Romain Virot
Sent: Thursday, 31 October 2019 01:38
To: etherlab-users@etherlab.org
Subject: [etherlab-users] ELM3004 (ADC) and SDOs requests

Hi,

I'm quite new to EtherCAT and while I managed to run a simple system (2 EL2004 
modules), I'm facing two issues (maybe related) with a Beckhoff ELM3004 module 
(and SDOs in general). I already searched heavily for solutions and tried many 
things but I'm now kind of lost...

First issue is :

I can activate the master and enter the cylic task without issue, and the 
ELM3004 goes into OP state. If I now stop my master (getting out of the cyclic 
task loop and releasing the master), the ELM3004 is stuck in OP (with the Error 
tag) state, while the 2 EL2004 modules do properly return to the PREOP state. I 
do no understand this behavior nor have any idea of how to diagnosis the 
problem.

Second one :

I cannot make RT SDOs (using requests) work. The request creation, before 
master activation, does work. However calling a request, either read or write, 
always ends up with "busy' states only for the request until the timeout is 
reached, and then the request state naturally goes into 'error'. This is for 
various frequency checks (from 1kHz to 1 Hz). I tried to replicate the mini 
exemple, and behavior is similar. Note that everything related to the EtherCAT 
system is on the same thread, if it matters (I have a QT Gui on another 
thread). FYI, I want to use RT SDO to be able to test modules easily and also 
run diagnosis.


On a side note, is there any possibility to reset//reboot an EtherCAT module 
(or whole domain//master) from a software perspective only? When the ELM3004 is 
stuck the only way for me to clean thing is to reboot the system...

Thanks a lot,

Romain
___
etherlab-users mailing list
etherlab-users@etherlab.org
https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.etherlab.org%2Fmailman%2Flistinfo%2Fetherlab-usersdata=02%7C01%7Cgavin.lambert%40tomra.com%7C13c13d49111d45372eb608d75d35fb99%7C4308d118edd143008a37cfeba8ad5898%7C0%7C1%7C637080358744179207sdata=urMi8xlYYC%2BIGYoMcMIN%2BO%2Fm5vyFy1%2Bie7XSnHfTo3w%3Dreserved=0
___
etherlab-users mailing list
etherlab-users@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-users

[etherlab-users] ELM3004 (ADC) and SDOs requests

[Romain Virot]

Hi,

I'm quite new to EtherCAT and while I managed to run a simple system (2 EL2004 
modules), I'm facing two issues (maybe related) with a Beckhoff ELM3004 module 
(and SDOs in general). I already searched heavily for solutions and tried many 
things but I'm now kind of lost...

First issue is :

I can activate the master and enter the cylic task without issue, and the 
ELM3004 goes into OP state. If I now stop my master (getting out of the cyclic 
task loop and releasing the master), the ELM3004 is stuck in OP (with the Error 
tag) state, while the 2 EL2004 modules do properly return to the PREOP state. I 
do no understand this behavior nor have any idea of how to diagnosis the 
problem.

Second one :

I cannot make RT SDOs (using requests) work. The request creation, before 
master activation, does work. However calling a request, either read or write, 
always ends up with "busy' states only for the request until the timeout is 
reached, and then the request state naturally goes into 'error'. This is for 
various frequency checks (from 1kHz to 1 Hz). I tried to replicate the mini 
exemple, and behavior is similar. Note that everything related to the EtherCAT 
system is on the same thread, if it matters (I have a QT Gui on another 
thread). FYI, I want to use RT SDO to be able to test modules easily and also 
run diagnosis.


On a side note, is there any possibility to reset//reboot an EtherCAT module 
(or whole domain//master) from a software perspective only? When the ELM3004 is 
stuck the only way for me to clean thing is to reboot the system...

Thanks a lot,

Romain
___
etherlab-users mailing list
etherlab-users@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-users