Hi Jan,
I finally found some time to address the problems I had last month.
Unfortunately, I still need help.
I have added a udev rule, so unnecessary device renaming does not occur
anymore.
I do have two rteth devices on one of the boards, but I checked the right
one is connected to the other board.
I can now execute the manual ping test (insert all rtnet modules,
rtifconfig, rtroute, rtping). It works well and I get 70-100 us for rtt
times from both boards.
However, I still get similar outcome for the multi-node tdma test, even
though it seems that master and slave go through all the stages. I still get
successive "TDMA: Failed to transmit sync frame!" messages on the master
node and rtping does not work afterwords. I looked a bit into the code and
realized that this message is generated because at some point alloc_rtskb
cannot allocate rtskb (returns NULL).
If possible, please see below the output of the './rtnet -v start' command
and let me know if you have any tip about this. Note that on the slave side
there is a 'RTmac: received unsupported RTmac protocol version on device
rteth0' message.
Thanks,
Bodan
##### MASTER console output
Turning on verbose mode
/usr/local/rtnet/sbin/rtifconfig rtlo up 127.0.0.1
/usr/local/rtnet/sbin/rtcfg rteth0 server
/usr/local/rtnet/sbin/rtifconfig rteth0 up 10.0.0.2 netmask 255.255.255.0
/usr/local/rtnet/sbin/tdmacfg rteth0 master 5000
/usr/local/rtnet/sbin/tdmacfg rteth0 slot 0 0
/usr/local/rtnet/sbin/rtcfg rteth0 add 10.0.0.1 -stage1 -
Waiting for all slaves.../usr/local/rtnet/sbin/rtcfg rteth0 wait
/usr/local/rtnet/sbin/rtcfg rteth0 ready
##### MASTER kernel output
*** RTnet 0.9.10 - built on Nov 3 2008 10:28:22 ***
RTnet: initialising real-time networking
Intel(R) PRO/1000 Network Driver - version 7.1.9
Copyright (c) 1999-2006 Intel Corporation.
PCI: Setting latency timer of device 0000:00:19.0 to 64
e1000: 0000:00:19.0: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
00:16:d3:3e:8c:b5
RTnet: registered rteth0
e1000: rteth0: e1000_probe: Intel(R) PRO/1000 Network Connection
initializing loopback...
RTnet: registered rtlo
RTcfg: init real-time configuration distribution protocol
RTmac: init realtime media access control
RTmac/TDMA: init time division multiple access control mechanism
e1000: rteth0: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex
vnic0: no IPv6 routers present
TDMA: Failed to transmit sync frame!
TDMA: Failed to transmit sync frame!
TDMA: Failed to transmit sync frame!
...
TDMA: Failed to transmit sync frame!
TDMA: Failed to transmit sync frame!
e1000: rteth0: e1000_clean_tx_irq: Detected Tx Unit Hang
Tx Queue <0>
TDH <ec>
TDT <c>
next_to_use <c>
next_to_clean <ec>
buffer_info[next_to_clean]
time_stamp <1004d8047>
next_to_watch <ec>
jiffies <1004d895d>
next_to_watch.status <0>
TDMA: Failed to transmit sync frame!
TDMA: Failed to transmit sync frame!
...
#### SLAVE console output
Turning on verbose mode
/usr/local/rtnet/sbin/rtifconfig rtlo up 127.0.0.1
/usr/local/rtnet/sbin/tdmacfg rteth0 slave
/usr/local/rtnet/sbin/rtifconfig rteth0 up 10.0.0.1
Stage 1: searching for master...$TDMACFG rteth0 slot 0 200;ifconfig vnic0 up
$IPADDR $NETMASK_OPT
/usr/local/rtnet/sbin/tdmacfg rteth0 slot 0 200
Stage 2: waiting for other slaves.../usr/local/rtnet/sbin/rtcfg rteth0
announce
Stage 3: waiting for common setup completion.../usr/local/rtnet/sbin/rtcfg
rteth0 ready
#### SLAVE kernel output
*** RTnet 0.9.10 - built on Oct 2 2008 10:59:23 ***
RTnet: initialising real-time networking
Intel(R) PRO/1000 Network Driver - version 7.1.9
Copyright (c) 1999-2006 Intel Corporation.
ACPI: PCI Interrupt 0000:0f:00.0[A] -> GSI 18 (level, low) -> IRQ 18
PCI: Setting latency timer of device 0000:0f:00.0 to 64
e1000: 0000:0f:00.0: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
00:e0:81:b0:0d:37
RTnet: registered rteth0
e1000: rteth0: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI Interrupt 0000:0f:00.1[B] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:0f:00.1 to 64
e1000: 0000:0f:00.1: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
00:e0:81:b0:0d:36
RTnet: registered rteth1
e1000: rteth1: e1000_probe: Intel(R) PRO/1000 Network Connection
initializing loopback...
RTnet: registered rtlo
RTcfg: init real-time configuration distribution protocol
RTmac: init realtime media access control
RTmac/TDMA: init time division multiple access control mechanism
e1000: rteth0: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex
TDMA: calibrated master-to-slave packet delay: 29 us (min/max: 29/30 us)
RTmac: received unsupported RTmac protocol version on device rteth0
vnic0: no IPv6 routers present
On Sun, Oct 5, 2008 at 4:37 AM, Jan Kiszka <[EMAIL PROTECTED]> wrote:
> Slobodan Matic wrote:
> > Hi all,
> >
> > I have a problem using rtnet-0.9.10 with two machines (lenovo D10 and
> lenovo
> > x61) running rtai-3.6.1.
> > On each machine "Testing with a single node" procedure from the RTnet
> wiki
> > instructions works without any problems.
> > When I try "Testing with multiple nodes" procedure, the two connected
> nodes
> > seem to establish the protocol correctly, since they return to the
> command
> > prompt after "Waiting for all slaves..." and "Stage 3: waiting for common
> > setup completion..." messages. However, immediately after that, the
> master
> > node fails with the periodic "TDMA: Failed to transmit sync frame!"
> output
> > errors. In addition, rtping commands do not work.
> >
> > Please, see below the rtnet.conf, console and kernel (dmesg) output for
> both
> > the master and slave nodes.
> >
> > I would appreciate any ideas on how to solve this problem.
> >
> > Thanks,
> > Bodan
> >
> >
> > +++++ master rtnet.conf
> > RT_DRIVER="rt_e1000"
> > RT_DRIVER_OPTIONS=""
> > IPADDR="10.0.0.1"
> > NETMASK="255.255.255.0"
> > RT_LOOPBACK="yes"
> > RTCAP="yes"
> > TDMA_MODE="master"
> > TDMA_SLAVES="10.0.0.3"
> > TDMA_CYCLE="5000"
> > TDMA_OFFSET="200"
> >
> > +++++ master console output
> > # ifconfig eth0 down
> > # rmmod e1000
> > # insmod /usr/realtime/modules/rtai_hal.ko
> > # insmod /usr/realtime/modules/rtai_lxrt.ko
> > # insmod /usr/realtime/modules/rtai_sem.ko
> > # insmod /usr/realtime/modules/rtai_rtdm.ko
> > # mknod /dev/rtnet c 10 240
> > # cd /usr/local/rtnet/sbin/
> > # ./rtnet start
> > vnic0: unknown interface: No such device
> > SIOCSIFADDR: No such device
> > vnic0: unknown interface: No such device
> > SIOCSIFNETMASK: No such device
> > SIOCGIFADDR: No such device
> > SIOCSIFBROADCAST: No such device
> > Waiting for all slaves...
> > # ./rtnet stop
> >
> > +++++ master kernel output
> > I-pipe: Domain RTAI registered.
> > RTAI[hal]: <3.6.1> mounted over IPIPE-NOTHREADS 1.4-03.
> > RTAI[hal]: compiled with gcc version 4.1.2 20071124 (Red Hat 4.1.2-42).
> > RTAI[hal]: mounted (IPIPE-NOTHREADS, IMMEDIATE (INTERNAL IRQs
> DISPATCHED),
> > ISOL_CPUS_MASK: 0).
> > PIPELINE layers:
> > ffffffff881c5380 9ac15d93 RTAI 200
> > ffffffff8062e940 0 Linux 100
> > RTAI[malloc]: global heap size = 2097152 bytes, <BSD>.
> > RTAI[sched]: IMMEDIATE, MP, USER/KERNEL SPACE: <without RTAI OWN KTASKs>,
> > kstacks pool size = 524288 bytes.
> > RTAI[sched]: hard timer type/freq = APIC/20781000(Hz); default timing:
> > periodic; black/red timed lists.
> > RTAI[sched]: Linux timer freq = 1000 (Hz), CPU freq = 2992499000 hz.
> > RTAI[sched]: timer setup = 999 ns, resched latency = 2944 ns.
> > RTDM started.
> > rtnet: no version for "struct_module" found: kernel tainted.
>
> Is this issue of RTAI still unfixed? That makes it hard to detect if you
> are accidentally building against a wrong version. However, let's assume
> it was the right one.
>
> >
> > *** RTnet 0.9.10 - built on Oct 2 2008 10:59:23 ***
> >
> > RTnet: initialising real-time networking
> > Intel(R) PRO/1000 Network Driver - version 7.1.9
> > Copyright (c) 1999-2006 Intel Corporation.
> > ACPI: PCI Interrupt 0000:0f:00.0[A] -> GSI 18 (level, low) -> IRQ 18
> > PCI: Setting latency timer of device 0000:0f:00.0 to 64
> > e1000: 0000:0f:00.0: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
> > 00:e0:81:b0:0d:37
> > RTnet: registered rteth0
> > e1000: rteth0: e1000_probe: Intel(R) PRO/1000 Network Connection
> > ACPI: PCI Interrupt 0000:0f:00.1[B] -> GSI 19 (level, low) -> IRQ 19
> > PCI: Setting latency timer of device 0000:0f:00.1 to 64
> > e1000: 0000:0f:00.1: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
> > 00:e0:81:b0:0d:36
> > RTnet: registered rteth1
> > e1000: rteth1: e1000_probe: Intel(R) PRO/1000 Network Connection
> > initializing loopback...
> > RTnet: registered rtlo
> > RTcap: real-time capturing interface
> > RTcfg: init real-time configuration distribution protocol
> > RTmac: init realtime media access control
> > RTmac/TDMA: init time division multiple access control mechanism
> > e1000: rteth0: e1000_watchdog: NIC Link is Up 100 Mbps Half Duplex
> > net eth0: device_rename: sysfs_create_symlink failed (-17)
>
> udev tries to shuffle network device names around, here probably
> vnic0->eth0. The rtnet script is not prepared for this, you should
> therefore add an appropriate udev rule to prevent it.
>
> > rtlo: no IPv6 routers present
> > rteth0: no IPv6 routers present
> > rteth0-mac: no IPv6 routers present
> > eth0: no IPv6 routers present
> > TDMA: Failed to transmit sync frame!
> > TDMA: Failed to transmit sync frame!
> > TDMA: Failed to transmit sync frame!
> > TDMA: Failed to transmit sync frame!
> > TDMA: Failed to transmit sync frame!
> > TDMA: Failed to transmit sync frame!
> > TDMA: Failed to transmit sync frame!
> > ...
> > ...
> > ...
> > TDMA: Failed to transmit sync frame!
> > e1000: rteth0: e1000_clean_tx_irq: Detected Tx Unit Hang
>
> OK, you have obviously two rteth devices on your master board. Are you
> sure you connected the right one with the slave box (rteth0)? And leave
> RT_DRIVER_OPTIONS as it is. The default value should ensure that the
> driver only grabs one device.
>
> Instead of the multi-node test, you could also try the manual ping test
> from rtnet/README (see "Special notes" section). That will exclude any
> RTmac effect and focus on the core issue: packet exchange between both
> nodes.
>
> > Tx Queue <0>
> > TDH <2a>
> > TDT <5a>
> > next_to_use <5a>
> > next_to_clean <2a>
> > buffer_info[next_to_clean]
> > time_stamp <10005d9d3>
> > next_to_watch <2a>
> > jiffies <10005de35>
> > next_to_watch.status <0>
> > TDMA: Failed to transmit sync frame!
> > ...
> > ...
> > ...
> > TDMA: Failed to transmit sync frame!
> > RTmac/TDMA: unloaded
> > RTmac: unloaded
> > RTcfg: unloaded
> > RTcap: unloaded
> > removing loopback...
> > RTnet: unregistered rtlo
> > RTnet: unregistered rteth1
> > ACPI: PCI interrupt for device 0000:0f:00.1 disabled
> > RTnet: unregistered rteth0
> > ACPI: PCI interrupt for device 0000:0f:00.0 disabled
> > slab error in kmem_cache_destroy(): cache `rtskb_slab_pool': Can't free
> all
> > objects
>
> That's not nice to fail like this, but probably a the result of the
> earlier TX error in rt_e1000.
>
> >
> > Call Trace:
> > [<ffffffff80282923>] kmem_cache_destroy+0x72/0xa5
> > [<ffffffff8839c69b>] :rtnet:cleanup_module+0x2b/0x3f
> > [<ffffffff802530d8>] sys_delete_module+0x196/0x1c5
> > [<ffffffff8020bea3>] tracesys+0xdc/0xe1
> >
> > RTnet: unloaded
> >
> > +++++ slave rtnet.conf
> > RT_DRIVER="rt_e1000"
> > RT_DRIVER_OPTIONS=""
> > IPADDR="10.0.0.3"
> > NETMASK="255.255.255.0"
> > RT_LOOPBACK="yes"
> > RTCAP="yes"
> > TDMA_MODE="slave"
> > TDMA_SLAVES="10.0.0.3"
> > TDMA_CYCLE="5000"
> > TDMA_OFFSET="200"
> >
> > +++++ slave console output
> > # ifconfig eth0 down
> > # rmmod e1000
> > # insmod /usr/realtime/modules/rtai_hal.ko
> > # insmod /usr/realtime/modules/rtai_lxrt.ko
> > # insmod /usr/realtime/modules/rtai_sem.ko
> > # insmod /usr/realtime/modules/rtai_rtdm.ko
> > # mknod /dev/rtnet c 10 240
> > # cd /usr/local/rtnet/sbin
> > # ./rtnet start
> > Stage 1: searching for master...
> > vnic0: unknown interface: No such device
> > SIOCSIFADDR: No such device
> > vnic0: unknown interface: No such device
> > SIOCSIFNETMASK: No such device
> > SIOCGIFADDR: No such device
> > SIOCSIFBROADCAST: No such device
> >
> > Stage 2: waiting for other slaves...
> > Stage 3: waiting for common setup completion...
> > # ./rtnet stop
> >
> > +++++ slave kernel output
> > I-pipe: Domain RTAI registered.
> > RTAI[hal]: <3.6.1> mounted over IPIPE-NOTHREADS 1.4-03.
> > RTAI[hal]: compiled with gcc version 4.1.2 20070626 (Red Hat 4.1.2-14).
> > RTAI[hal]: mounted (IPIPE-NOTHREADS, IMMEDIATE (INTERNAL IRQs
> DISPATCHED),
> > ISOL_CPUS_MASK: 0).
> > PIPELINE layers:
> > ffffffff8813b380 9ac15d93 RTAI 200
> > ffffffff805b67c0 0 Linux 100
> > RTAI[malloc]: global heap size = 2097152 bytes, <BSD>.
> > RTAI[sched]: IMMEDIATE, MP, USER/KERNEL SPACE: <without RTAI OWN KTASKs>,
> > kstacks pool size = 524288 bytes.
> > RTAI[sched]: hard timer type/freq = APIC/12469000(Hz); default timing:
> > periodic; linear timed lists.
> > RTAI[sched]: Linux timer freq = 1000 (Hz), CPU freq = 1596051000 hz.
> > RTAI[sched]: timer setup = 999 ns, resched latency = 2943 ns.
> > RTDM started.
> > rtnet: no version for "struct_module" found: kernel tainted.
> >
> > *** RTnet 0.9.10 - built on Oct 2 2008 13:27:23 ***
> >
> > RTnet: initialising real-time networking
> > Intel(R) PRO/1000 Network Driver - version 7.1.9
> > Copyright (c) 1999-2006 Intel Corporation.
> > PCI: Setting latency timer of device 0000:00:19.0 to 64
> > e1000: 0000:00:19.0: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
> > 00:16:d3:3e:8c:b5
> > RTnet: registered rteth0
> > e1000: rteth0: e1000_probe: Intel(R) PRO/1000 Network Connection
> > initializing loopback...
> > RTnet: registered rtlo
> > RTcap: real-time capturing interface
> > RTcfg: init real-time configuration distribution protocol
> > RTmac: init realtime media access control
> > RTmac/TDMA: init time division multiple access control mechanism
> > net eth0: device_rename: sysfs_create_symlink failed (-17)
>
> Uhh, even uglier. Also here: add an udev rule to keep vnic0 as it is.
>
> > BUG eth0 code -11 qlen 0
> > BUG eth0 code -11 qlen 0
> > BUG eth0 code -11 qlen 0
> > BUG eth0 code -11 qlen 0
> > BUG eth0 code -11 qlen 0
> > BUG eth0 code -11 qlen 0
> > BUG eth0 code -11 qlen 0
> > BUG eth0 code -11 qlen 0
> > BUG eth0 code -11 qlen 0
> > BUG eth0 code -11 qlen 0
> > e1000: rteth0: e1000_watchdog: NIC Link is Up 100 Mbps Half Duplex
> > TDMA: calibrated master-to-slave packet delay: 184467440737036 us
> (min/max:
> > 9223372036854716/9223372036854717 us)
>
> Weird...
>
> > rtlo: no IPv6 routers present
> > rteth0-mac: no IPv6 routers present
> > eth0: no IPv6 routers present
> > rteth0: no IPv6 routers present
> > RTmac/TDMA: unloaded
> > RTmac: unloaded
> > RTcfg: unloaded
> > RTcap: unloaded
> > removing loopback...
> > RTnet: unregistered rtlo
> > RTnet: unregistered rteth0
> > RTnet: unloaded
> >
>
> Jan
>
>
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
RTnet-users mailing list
RTnet-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rtnet-users
