Re: [Query]: DSA Understanding
Hi Florian, On Mon, Aug 13, 2018 at 7:57 PM Florian Fainelli wrote: > > On 08/13/2018 08:58 AM, Lad, Prabhakar wrote: > > Hi Andrew/Florain, > > > > On Mon, Aug 13, 2018 at 2:38 PM Andrew Lunn wrote: > >> > I agree, this should be padding packets correctly, can you still > instrument cpsw to make sure that what comes to its ndo_start_xmit() is > ETH_ZLEN + tag_len or more? > > >>> Yes I can confirm the skb->len is always >= 62 (ETH_ZLEN + 2) > >> > >> Which switch are you using? > >> > >> Marvell switches use either 4 or 8 bytes of tag. Broadcom has 4, KSZ > >> has 1 for packets going to the switch, lan9303 has 4, mtd uses 4, qca > >> has 2. > >> > > I am using the KSZ switch. for Ingress it has 1 byte and for Egress it > > has 2 bytes. > > I came across patch [1] and padded 2 more bytes in ksz_xmit() and I was > > successfully able to ping from lan4 to PC. Thank you very much for > > your guidance/support. > > > > Now I have stumbled into a different issue: > > > > Case 1 Works: > > = > > lan0 = 192.168.0.1 > > PC1 = 192.168.0.10 > > For the above ping works from both directions. > > > > CASE 2 Doesn’t Work: > > = > > lan0 = 192.168.0.1 > > PC1 = 192.168.0.10 > > lan4 = 192.168.0.4 > > PC2 = 192.168.0.11 > > > > Ping from lan0 to PC1 and PC1 to lan0 works > > But ping from PC2 to lan4 and lan4 to PC2 fails. > > > > CASE 3 Works: > > = > > lan0 = 192.168.0.1 > > PC1 = 192.168.0.10 > > lan4 = 192.168.4.4 > > PC2 = 192.168.4.11 > > > > With the above setup ping works. > > > > [Query] Why does ping fail in case 2. Any thoughts what I am missing here ? > > or is it the expected behaviour ? > > For case 2, what I suspect is happening is that the machine that has > lan1/lan4, because you have put lan1/lan4 in the same subnet, does not > know how to respond to PC2 because it is unable to select an appropriate > output interface. In such cases, you might have to add an explicit /32 > route that forces telling the kernel that PC2 is accessible via lan2. > That did the trick thank you, following is my setup lan0 = 192.168.0.1 PC1 = 192.168.0.10 lan4 = 192.168.0.4 PC2 = 192.168.0.11 route add 192.168.0.11 gw 192.168.0.4 route add 192.168.0.10 gw 192.168.0.1 And now ping works either ways. Is this setup for adding the route a valid way to do it ? Or is there some standard way I need to follow which I am missing. > Andrew, do you see an other explanation for that? > > > > > [1] https://lore.kernel.org/patchwork/patch/851457/ > > This patch works, but I think it is still working by "accident" in that > if you have both VLAN tags + KSZ tag, you would likely still be too > short by a few bytes. > too take care of this now I making sure the skb->len is padded to VLAN_ETH_ZLEN in tag_ksz.c ksz_xmit() function. Cheers, --Prabhakar Lad
Re: [Query]: DSA Understanding
On 08/13/2018 08:58 AM, Lad, Prabhakar wrote: > Hi Andrew/Florain, > > On Mon, Aug 13, 2018 at 2:38 PM Andrew Lunn wrote: >> I agree, this should be padding packets correctly, can you still instrument cpsw to make sure that what comes to its ndo_start_xmit() is ETH_ZLEN + tag_len or more? >>> Yes I can confirm the skb->len is always >= 62 (ETH_ZLEN + 2) >> >> Which switch are you using? >> >> Marvell switches use either 4 or 8 bytes of tag. Broadcom has 4, KSZ >> has 1 for packets going to the switch, lan9303 has 4, mtd uses 4, qca >> has 2. >> > I am using the KSZ switch. for Ingress it has 1 byte and for Egress it > has 2 bytes. > I came across patch [1] and padded 2 more bytes in ksz_xmit() and I was > successfully able to ping from lan4 to PC. Thank you very much for > your guidance/support. > > Now I have stumbled into a different issue: > > Case 1 Works: > = > lan0 = 192.168.0.1 > PC1 = 192.168.0.10 > For the above ping works from both directions. > > CASE 2 Doesn’t Work: > = > lan0 = 192.168.0.1 > PC1 = 192.168.0.10 > lan4 = 192.168.0.4 > PC2 = 192.168.0.11 > > Ping from lan0 to PC1 and PC1 to lan0 works > But ping from PC2 to lan4 and lan4 to PC2 fails. > > CASE 3 Works: > = > lan0 = 192.168.0.1 > PC1 = 192.168.0.10 > lan4 = 192.168.4.4 > PC2 = 192.168.4.11 > > With the above setup ping works. > > [Query] Why does ping fail in case 2. Any thoughts what I am missing here ? > or is it the expected behaviour ? For case 2, what I suspect is happening is that the machine that has lan1/lan4, because you have put lan1/lan4 in the same subnet, does not know how to respond to PC2 because it is unable to select an appropriate output interface. In such cases, you might have to add an explicit /32 route that forces telling the kernel that PC2 is accessible via lan2. Andrew, do you see an other explanation for that? > > [1] https://lore.kernel.org/patchwork/patch/851457/ This patch works, but I think it is still working by "accident" in that if you have both VLAN tags + KSZ tag, you would likely still be too short by a few bytes. -- Florian
Re: [Query]: DSA Understanding
Hi Andrew/Florain, On Mon, Aug 13, 2018 at 2:38 PM Andrew Lunn wrote: > > > > I agree, this should be padding packets correctly, can you still > > > instrument cpsw to make sure that what comes to its ndo_start_xmit() is > > > ETH_ZLEN + tag_len or more? > > > > > Yes I can confirm the skb->len is always >= 62 (ETH_ZLEN + 2) > > Which switch are you using? > > Marvell switches use either 4 or 8 bytes of tag. Broadcom has 4, KSZ > has 1 for packets going to the switch, lan9303 has 4, mtd uses 4, qca > has 2. > I am using the KSZ switch. for Ingress it has 1 byte and for Egress it has 2 bytes. I came across patch [1] and padded 2 more bytes in ksz_xmit() and I was successfully able to ping from lan4 to PC. Thank you very much for your guidance/support. Now I have stumbled into a different issue: Case 1 Works: = lan0 = 192.168.0.1 PC1 = 192.168.0.10 For the above ping works from both directions. CASE 2 Doesn’t Work: = lan0 = 192.168.0.1 PC1 = 192.168.0.10 lan4 = 192.168.0.4 PC2 = 192.168.0.11 Ping from lan0 to PC1 and PC1 to lan0 works But ping from PC2 to lan4 and lan4 to PC2 fails. CASE 3 Works: = lan0 = 192.168.0.1 PC1 = 192.168.0.10 lan4 = 192.168.4.4 PC2 = 192.168.4.11 With the above setup ping works. [Query] Why does ping fail in case 2. Any thoughts what I am missing here ? or is it the expected behaviour ? [1] https://lore.kernel.org/patchwork/patch/851457/ Cheers, --Prabhakar Lad
Re: [Query]: DSA Understanding
> > I agree, this should be padding packets correctly, can you still > > instrument cpsw to make sure that what comes to its ndo_start_xmit() is > > ETH_ZLEN + tag_len or more? > > > Yes I can confirm the skb->len is always >= 62 (ETH_ZLEN + 2) Which switch are you using? Marvell switches use either 4 or 8 bytes of tag. Broadcom has 4, KSZ has 1 for packets going to the switch, lan9303 has 4, mtd uses 4, qca has 2. Andrew
Re: [Query]: DSA Understanding
Hi Florian, On Fri, Aug 10, 2018 at 6:36 PM Florian Fainelli wrote: > > On 08/10/2018 04:26 AM, Lad, Prabhakar wrote: > > Hi Andrew, > > > > On Thu, Aug 9, 2018 at 6:23 PM Andrew Lunn wrote: > >> > >>> Its coming from the switch lan4 I have attached the png, where > >>> C4:F3:12:08:FE:7F is > >>> the mac of lan4, which is broadcast to ff:ff:ff:ff:ff:ff, which is > >>> causing rx counter on > >>> PC to go up. > >> > >> So, big packets are making it from the switch to the PC. But the small > >> ARP packets are not. > >> > >> This is what Florian was suggesting. > >> > >> ARP packets are smaller than 64 bytes, which is the minimum packet > >> size for Ethernet. Any packets smaller than 64 bytes are called runt > >> packets. They have to be padded upto 64 bytes in order to make them > >> valid. Otherwise the destination, or any switch along the path, might > >> throw them away. > >> > >> What could be happening is that the CSPW driver or hardware is padding > >> the packet to 64 bytes. But that packet has a DSA header in it. The > >> switch removes the header, recalculate the checksum and sends the > >> packet. It is now either 4 or 8 bytes smaller, depending on what DSA > >> header was used. It then becomes a runt packet. > >> > > Thank you for the clarification, this really helped me out. > > > >> Florian had to fix this problem recently. > >> > >> http://patchwork.ozlabs.org/patch/836534/ > >> > > But seems like this patch was never accepted, instead > > brcm_tag_xmit_ll() does it if I am understanding it correctly. > > similarly to this ksz_xmit() is taking care of padding. > > net/dsa/tag_brcm.c ended up doing the padding because that was a more > generic and central location: > > https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git/tree/net/dsa/tag_brcm.c#n73 > > > > >> You probably need something similar for the cpsw. > >> > > looking at the tag_ksz.c in xmit function this is taken care of > > I agree, this should be padding packets correctly, can you still > instrument cpsw to make sure that what comes to its ndo_start_xmit() is > ETH_ZLEN + tag_len or more? > Yes I can confirm the skb->len is always >= 62 (ETH_ZLEN + 2) Cheers, --Prabhakar
Re: [Query]: DSA Understanding
On 08/10/2018 04:26 AM, Lad, Prabhakar wrote:
> Hi Andrew,
>
> On Thu, Aug 9, 2018 at 6:23 PM Andrew Lunn wrote:
>>
>>> Its coming from the switch lan4 I have attached the png, where
>>> C4:F3:12:08:FE:7F is
>>> the mac of lan4, which is broadcast to ff:ff:ff:ff:ff:ff, which is
>>> causing rx counter on
>>> PC to go up.
>>
>> So, big packets are making it from the switch to the PC. But the small
>> ARP packets are not.
>>
>> This is what Florian was suggesting.
>>
>> ARP packets are smaller than 64 bytes, which is the minimum packet
>> size for Ethernet. Any packets smaller than 64 bytes are called runt
>> packets. They have to be padded upto 64 bytes in order to make them
>> valid. Otherwise the destination, or any switch along the path, might
>> throw them away.
>>
>> What could be happening is that the CSPW driver or hardware is padding
>> the packet to 64 bytes. But that packet has a DSA header in it. The
>> switch removes the header, recalculate the checksum and sends the
>> packet. It is now either 4 or 8 bytes smaller, depending on what DSA
>> header was used. It then becomes a runt packet.
>>
> Thank you for the clarification, this really helped me out.
>
>> Florian had to fix this problem recently.
>>
>> http://patchwork.ozlabs.org/patch/836534/
>>
> But seems like this patch was never accepted, instead
> brcm_tag_xmit_ll() does it if I am understanding it correctly.
> similarly to this ksz_xmit() is taking care of padding.
net/dsa/tag_brcm.c ended up doing the padding because that was a more
generic and central location:
https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git/tree/net/dsa/tag_brcm.c#n73
>
>> You probably need something similar for the cpsw.
>>
> looking at the tag_ksz.c in xmit function this is taken care of
I agree, this should be padding packets correctly, can you still
instrument cpsw to make sure that what comes to its ndo_start_xmit() is
ETH_ZLEN + tag_len or more?
>
> /* For Ingress (Host -> KSZ), 2 bytes are added before FCS.
> * ---
> * DA(6bytes)|SA(6bytes)||Data(nbytes)|tag0(1byte)|tag1(1byte)|FCS(4bytes)
> * ---
> * tag0 : Prioritization (not used now)
> * tag1 : each bit represents port (eg, 0x01=port1, 0x02=port2, 0x10=port5)
> *
> * For Egress (KSZ -> Host), 1 byte is added before FCS.
> * ---
> * DA(6bytes)|SA(6bytes)||Data(nbytes)|tag0(1byte)|FCS(4bytes)
> * ---
> * tag0 : zero-based value represents port
> * (eg, 0x00=port1, 0x02=port3, 0x06=port7)
> */
>
> #defineKSZ_INGRESS_TAG_LEN2
> #defineKSZ_EGRESS_TAG_LEN1
>
> static struct sk_buff *ksz_xmit(struct sk_buff *skb, struct net_device *dev)
> {
> struct dsa_slave_priv *p = netdev_priv(dev);
> struct sk_buff *nskb;
> int padlen;
> u8 *tag;
>
> padlen = (skb->len >= ETH_ZLEN) ? 0 : ETH_ZLEN - skb->len;
>
> if (skb_tailroom(skb) >= padlen + KSZ_INGRESS_TAG_LEN) {
> /* Let dsa_slave_xmit() free skb */
> if (__skb_put_padto(skb, skb->len + padlen, false))
> return NULL;
>
> nskb = skb;
> } else {
> nskb = alloc_skb(NET_IP_ALIGN + skb->len +
> padlen + KSZ_INGRESS_TAG_LEN, GFP_ATOMIC);
> if (!nskb)
> return NULL;
> skb_reserve(nskb, NET_IP_ALIGN);
>
> skb_reset_mac_header(nskb);
> skb_set_network_header(nskb,
>skb_network_header(skb) - skb->head);
> skb_set_transport_header(nskb,
> skb_transport_header(skb) - skb->head);
> skb_copy_and_csum_dev(skb, skb_put(nskb, skb->len));
>
> /* Let skb_put_padto() free nskb, and let dsa_slave_xmit() free
> * skb
> */
> if (skb_put_padto(nskb, nskb->len + padlen))
> return NULL;
>
> consume_skb(skb);
> }
>
> tag = skb_put(nskb, KSZ_INGRESS_TAG_LEN);
> tag[0] = 0;
> tag[1] = 1 << p->dp->index; /* destination port */
>
> return nskb;
> }
>
> Cheers,
> --Prabhakar Lad
>
--
Florian
Re: [Query]: DSA Understanding
Hi Andrew,
On Thu, Aug 9, 2018 at 6:23 PM Andrew Lunn wrote:
>
> > Its coming from the switch lan4 I have attached the png, where
> > C4:F3:12:08:FE:7F is
> > the mac of lan4, which is broadcast to ff:ff:ff:ff:ff:ff, which is
> > causing rx counter on
> > PC to go up.
>
> So, big packets are making it from the switch to the PC. But the small
> ARP packets are not.
>
> This is what Florian was suggesting.
>
> ARP packets are smaller than 64 bytes, which is the minimum packet
> size for Ethernet. Any packets smaller than 64 bytes are called runt
> packets. They have to be padded upto 64 bytes in order to make them
> valid. Otherwise the destination, or any switch along the path, might
> throw them away.
>
> What could be happening is that the CSPW driver or hardware is padding
> the packet to 64 bytes. But that packet has a DSA header in it. The
> switch removes the header, recalculate the checksum and sends the
> packet. It is now either 4 or 8 bytes smaller, depending on what DSA
> header was used. It then becomes a runt packet.
>
Thank you for the clarification, this really helped me out.
> Florian had to fix this problem recently.
>
> http://patchwork.ozlabs.org/patch/836534/
>
But seems like this patch was never accepted, instead
brcm_tag_xmit_ll() does it if I am understanding it correctly.
similarly to this ksz_xmit() is taking care of padding.
> You probably need something similar for the cpsw.
>
looking at the tag_ksz.c in xmit function this is taken care of
/* For Ingress (Host -> KSZ), 2 bytes are added before FCS.
* ---
* DA(6bytes)|SA(6bytes)||Data(nbytes)|tag0(1byte)|tag1(1byte)|FCS(4bytes)
* ---
* tag0 : Prioritization (not used now)
* tag1 : each bit represents port (eg, 0x01=port1, 0x02=port2, 0x10=port5)
*
* For Egress (KSZ -> Host), 1 byte is added before FCS.
* ---
* DA(6bytes)|SA(6bytes)||Data(nbytes)|tag0(1byte)|FCS(4bytes)
* ---
* tag0 : zero-based value represents port
* (eg, 0x00=port1, 0x02=port3, 0x06=port7)
*/
#defineKSZ_INGRESS_TAG_LEN2
#defineKSZ_EGRESS_TAG_LEN1
static struct sk_buff *ksz_xmit(struct sk_buff *skb, struct net_device *dev)
{
struct dsa_slave_priv *p = netdev_priv(dev);
struct sk_buff *nskb;
int padlen;
u8 *tag;
padlen = (skb->len >= ETH_ZLEN) ? 0 : ETH_ZLEN - skb->len;
if (skb_tailroom(skb) >= padlen + KSZ_INGRESS_TAG_LEN) {
/* Let dsa_slave_xmit() free skb */
if (__skb_put_padto(skb, skb->len + padlen, false))
return NULL;
nskb = skb;
} else {
nskb = alloc_skb(NET_IP_ALIGN + skb->len +
padlen + KSZ_INGRESS_TAG_LEN, GFP_ATOMIC);
if (!nskb)
return NULL;
skb_reserve(nskb, NET_IP_ALIGN);
skb_reset_mac_header(nskb);
skb_set_network_header(nskb,
skb_network_header(skb) - skb->head);
skb_set_transport_header(nskb,
skb_transport_header(skb) - skb->head);
skb_copy_and_csum_dev(skb, skb_put(nskb, skb->len));
/* Let skb_put_padto() free nskb, and let dsa_slave_xmit() free
* skb
*/
if (skb_put_padto(nskb, nskb->len + padlen))
return NULL;
consume_skb(skb);
}
tag = skb_put(nskb, KSZ_INGRESS_TAG_LEN);
tag[0] = 0;
tag[1] = 1 << p->dp->index; /* destination port */
return nskb;
}
Cheers,
--Prabhakar Lad
Re: [Query]: DSA Understanding
> Its coming from the switch lan4 I have attached the png, where > C4:F3:12:08:FE:7F is > the mac of lan4, which is broadcast to ff:ff:ff:ff:ff:ff, which is > causing rx counter on > PC to go up. So, big packets are making it from the switch to the PC. But the small ARP packets are not. This is what Florian was suggesting. ARP packets are smaller than 64 bytes, which is the minimum packet size for Ethernet. Any packets smaller than 64 bytes are called runt packets. They have to be padded upto 64 bytes in order to make them valid. Otherwise the destination, or any switch along the path, might throw them away. What could be happening is that the CSPW driver or hardware is padding the packet to 64 bytes. But that packet has a DSA header in it. The switch removes the header, recalculate the checksum and sends the packet. It is now either 4 or 8 bytes smaller, depending on what DSA header was used. It then becomes a runt packet. Florian had to fix this problem recently. http://patchwork.ozlabs.org/patch/836534/ You probably need something similar for the cpsw. Andrew
Re: [Query]: DSA Understanding
> > > The received packets captured on the PC are MDNS and DHPC, these MDNS > > > are causing the rx > > > packet counter go up: > > > > And where are these packets coming from? The target device? Or some > > other device on your network? > > > AFIK, MDNS is also kind of a bcast its sending MDNS requests and > receiving itself, > that’s the reason rx packets are incrementing (correct me if I am wrong) Your Ethernet device should not be receiving its own transmissions. Looping back for broadcast and multicast packets happens higher up in the network stack. Look at the source MAC address for these packets. Where are the coming from? > ~$ ethtool -S lan4 > NIC statistics: ... > tx_hi: 0 > tx_late_col: 0 > tx_pause: 0 > tx_bcast: 749 > tx_mcast: 212 This suggest the switch is putting packets onto the wire. > Only weird thing I notice on target, when its replying for ping > requests ( (oui Unknown) is that something which is causing issues ? These are not ping requests. These are ARP requests/replies. > 08:11:20.230704 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has > VB4-SN tell tango-charlie.local, length 46 > 08:11:20.230749 ARP, Ethernet (len 6), IPv4 (len 4), Reply > VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 > 08:11:21.230629 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has > VB4-SN tell tango-charlie.local, length 46 > 08:11:21.230657 ARP, Ethernet (len 6), IPv4 (len 4), Reply > VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 > 08:11:22.247831 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has > VB4-SN tell tango-charlie.local, length 46 > 08:11:22.247857 ARP, Ethernet (len 6), IPv4 (len 4), Reply > VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 c4:f3:12 is the OUI. It is actually registers to TI: https://aruljohn.com/mac/C4F312 But tcpdump probably does not know this, or the build you have has the oui table removed to keep the binary small. Andrew
Re: [Query]: DSA Understanding
Hi, On Thu, Aug 9, 2018 at 1:56 PM Andrew Lunn wrote: > > On Thu, Aug 09, 2018 at 01:45:52PM +0100, Lad, Prabhakar wrote: > > On Thu, Aug 9, 2018 at 1:02 PM Andrew Lunn wrote: > > > > > > On Thu, Aug 09, 2018 at 12:31:31PM +0100, Lad, Prabhakar wrote: > > > > Hi Andrew, > > > > > > > > On Thu, Aug 2, 2018 at 5:05 PM Andrew Lunn wrote: > > > > > > > > > > > I dont see any Reply's on the PC with tcpdump on PC > > > > > > > > > > So try ethool -S on the PC. Any packets dropped because of errors? > > > > > > > > > I dont see any drops/errors on the PC, following is the dump from PC: > > > > > > > > sudo ethtool -S enx00e04c68c229 > > > > [sudo] password for prabhakar: > > > > NIC statistics: > > > > tx_packets: 1659 > > > > rx_packets: 485 > > > > tx_errors: 0 > > > > rx_errors: 0 > > > > rx_missed: 0 > > > > align_errors: 0 > > > > tx_single_collisions: 0 > > > > tx_multi_collisions: 0 > > > > rx_unicast: 18 > > > > rx_broadcast: 295 > > > > rx_multicast: 172 > > > > tx_aborted: 0 > > > > tx_underrun: 0 > > > > > > So there are received packets at the PC. Not many unicast, mostly > > > broadcast, which fits with ARP. What does wireshark tell you about > > > these received packets? Are they ARP replies? Are they something else? > > > If they are ARP replies, why are they being ignored? I don't know if > > > tshark will show you CRC problems. Wireshark does, when you unfold a > > > packet, and look at the fields in detail. > > > > > > > Seems like the packet is not being transmitted from the switch at all > > > > ? (as ping from switch lan4 to PC fails) > > > > > > I don't think you can make that conclusion yet. The PC is receiving > > > something, rx_packets=485. What are those packets? > > > > > The received packets captured on the PC are MDNS and DHPC, these MDNS > > are causing the rx > > packet counter go up: > > And where are these packets coming from? The target device? Or some > other device on your network? > AFIK, MDNS is also kind of a bcast its sending MDNS requests and receiving itself, that’s the reason rx packets are incrementing (correct me if I am wrong) On the PC where lan4 is connected , tx has high count because of ping requests prabhakar@tango-charlie:~/Desktop/test$ ifconfig enx00e04c68c229 enx00e04c68c229 Link encap:Ethernet HWaddr 00:e0:4c:68:c2:29 inet addr:169.254.78.251 Bcast:169.254.255.255 Mask:255.255.0.0 inet6 addr: fe80::2f12:3d45:7cca:57fa/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:502 errors:0 dropped:0 overruns:0 frame:0 TX packets:4811 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:79843 (79.8 KB) TX bytes:252647 (252.6 KB) > > I don’t see any packets reaching the PC for the ping request. I can see the > > RX and TX on the switch for lan4 increasing every second. seems like the > > switch itself is consuming it and not forwarding(but then lan4 TX > > shouldn’t have incremented ?). > > Which lan4 counters are going up? tx_packets, rx_packets, tx_errors, > rx_errors are software counters, and are incremented by the DSA > core. Other counters are hardware counters, and the DSA driver will > read them from the actual switch port. If the hardware counters show > packets are being transmitted, then the packets are probably on the > cable. > I can see the RX and TX incrementing every second, no errors counters go up $ watch -n1 ifconfig lan4 lan4 Link encap:Ethernet HWaddr C4:F3:12:08:FE:7F inet addr:169.254.126.126 Bcast:169.254.255.255 Mask:255.255.0.0 inet6 addr: fe80::18b9:d16c:7ff:ab73%3201178264/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2168 errors:0 dropped:0 overruns:0 frame:0 TX packets:1724 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:117816 (115.0 KiB) TX bytes:99940 (97.5 KiB) ~$ ethtool -S eth1 NIC statistics: Good Rx Frames: 800 Broadcast Rx Frames: 729 Multicast Rx Frames: 71 Pause Rx Frames: 0 Rx CRC Errors: 0 Rx Align/Code Errors: 0 Oversize Rx Frames: 0 Rx Jabbers: 0 Undersize (Short) Rx Frames: 0 Rx Fragments: 0 Rx Octets: 65472 Good Tx Frames: 369 Broadcast Tx Frames: 16 Multicast Tx Frames: 139 Pause Tx Frames: 0 Deferred Tx Frames: 0 Collisions: 0 Single Collision Tx Frames: 0 Multiple Collision Tx Frames: 0 Excessive Collisions: 0 Late Collisions: 0 Tx Underrun: 0 Carrier Sense Errors: 0 Tx Octets: 40990 Rx + Tx 64 Octet Frames: 0 Rx + Tx 65-127 Octet Frames: 1031 Rx + Tx 128-255 Octet Frames: 73 Rx + Tx 256-511 Octet Frames: 65 Rx + Tx 512-1023 Octet Frames: 0 Rx + Tx 1024-Up Octet Frames: 0 Net Octets: 106462 Rx Start of Frame Overruns: 0 Rx
Re: [Query]: DSA Understanding
On Thu, Aug 09, 2018 at 01:45:52PM +0100, Lad, Prabhakar wrote: > On Thu, Aug 9, 2018 at 1:02 PM Andrew Lunn wrote: > > > > On Thu, Aug 09, 2018 at 12:31:31PM +0100, Lad, Prabhakar wrote: > > > Hi Andrew, > > > > > > On Thu, Aug 2, 2018 at 5:05 PM Andrew Lunn wrote: > > > > > > > > > I dont see any Reply's on the PC with tcpdump on PC > > > > > > > > So try ethool -S on the PC. Any packets dropped because of errors? > > > > > > > I dont see any drops/errors on the PC, following is the dump from PC: > > > > > > sudo ethtool -S enx00e04c68c229 > > > [sudo] password for prabhakar: > > > NIC statistics: > > > tx_packets: 1659 > > > rx_packets: 485 > > > tx_errors: 0 > > > rx_errors: 0 > > > rx_missed: 0 > > > align_errors: 0 > > > tx_single_collisions: 0 > > > tx_multi_collisions: 0 > > > rx_unicast: 18 > > > rx_broadcast: 295 > > > rx_multicast: 172 > > > tx_aborted: 0 > > > tx_underrun: 0 > > > > So there are received packets at the PC. Not many unicast, mostly > > broadcast, which fits with ARP. What does wireshark tell you about > > these received packets? Are they ARP replies? Are they something else? > > If they are ARP replies, why are they being ignored? I don't know if > > tshark will show you CRC problems. Wireshark does, when you unfold a > > packet, and look at the fields in detail. > > > > > Seems like the packet is not being transmitted from the switch at all > > > ? (as ping from switch lan4 to PC fails) > > > > I don't think you can make that conclusion yet. The PC is receiving > > something, rx_packets=485. What are those packets? > > > The received packets captured on the PC are MDNS and DHPC, these MDNS > are causing the rx > packet counter go up: And where are these packets coming from? The target device? Or some other device on your network? > I don’t see any packets reaching the PC for the ping request. I can see the > RX and TX on the switch for lan4 increasing every second. seems like the > switch itself is consuming it and not forwarding(but then lan4 TX > shouldn’t have incremented ?). Which lan4 counters are going up? tx_packets, rx_packets, tx_errors, rx_errors are software counters, and are incremented by the DSA core. Other counters are hardware counters, and the DSA driver will read them from the actual switch port. If the hardware counters show packets are being transmitted, then the packets are probably on the cable. Andrew
Re: [Query]: DSA Understanding
On Thu, Aug 9, 2018 at 1:02 PM Andrew Lunn wrote: > > On Thu, Aug 09, 2018 at 12:31:31PM +0100, Lad, Prabhakar wrote: > > Hi Andrew, > > > > On Thu, Aug 2, 2018 at 5:05 PM Andrew Lunn wrote: > > > > > > > I dont see any Reply's on the PC with tcpdump on PC > > > > > > So try ethool -S on the PC. Any packets dropped because of errors? > > > > > I dont see any drops/errors on the PC, following is the dump from PC: > > > > sudo ethtool -S enx00e04c68c229 > > [sudo] password for prabhakar: > > NIC statistics: > > tx_packets: 1659 > > rx_packets: 485 > > tx_errors: 0 > > rx_errors: 0 > > rx_missed: 0 > > align_errors: 0 > > tx_single_collisions: 0 > > tx_multi_collisions: 0 > > rx_unicast: 18 > > rx_broadcast: 295 > > rx_multicast: 172 > > tx_aborted: 0 > > tx_underrun: 0 > > So there are received packets at the PC. Not many unicast, mostly > broadcast, which fits with ARP. What does wireshark tell you about > these received packets? Are they ARP replies? Are they something else? > If they are ARP replies, why are they being ignored? I don't know if > tshark will show you CRC problems. Wireshark does, when you unfold a > packet, and look at the fields in detail. > > > Seems like the packet is not being transmitted from the switch at all > > ? (as ping from switch lan4 to PC fails) > > I don't think you can make that conclusion yet. The PC is receiving > something, rx_packets=485. What are those packets? > The received packets captured on the PC are MDNS and DHPC, these MDNS are causing the rx packet counter go up: 685682.9562669630.0.0.0255.255.255.255DHCP422 DHCP Discover - Transaction ID 0x9d832fac 550555.5433222000.0.0.0255.255.255.255DHCP422 DHCP Discover - Transaction ID 0x9d832fac 572576.738682378fe80::2f12:3d45:7cca:57faff02::fbMDNS 136Standard query 0x VB4-SN.local, "QM" question SRV VB4-SN._sftp-ssh._tcp.local, "QM" question 630630.706578680169.254.126.126224.0.0.251MDNS168 Standard query response 0x , cache flush fe80::c6f3:12ff:fe08:fe7f SRV, cache flush 0 0 22 VB4-SN.local A, cache flush 169.254.126.126 732728.982449369169.254.78.251224.0.0.251MDNS122 Standard query 0x VB4-SN.local, "QM" question A VB4-SN.local, "QM" question SRV VB4-SN._sftp-ssh._tcp.local, "QM" question 733728.983534948169.254.126.126224.0.0.251MDNS168 Standard query response 0x SRV, cache flush 0 0 22 VB4-SN.local , cache flush fe80::c6f3:12ff:fe08:fe7f A, cache flush 169.254.126.126 > Look at the statistics along the chain, from the target to the PC. > Look at the master device, lan4 and the PC. You should see about one > packet per second transmitted on the master device. One packet per > second transmitted on lan4, and one packet per second received on the > PC. Where does this break down. > I don’t see any packets reaching the PC for the ping request. I can see the RX and TX on the switch for lan4 increasing every second. seems like the switch itself is consuming it and not forwarding(but then lan4 TX shouldn’t have incremented ?). any thoughts where I could focusing on the switch or cpsw ? Cheers, --Prabhakar Lad
Re: [Query]: DSA Understanding
On Thu, Aug 09, 2018 at 12:33:30PM +0100, Lad, Prabhakar wrote: > Hi Florain, > > Thanks for your reply. > > On Thu, Aug 2, 2018 at 5:24 PM Florian Fainelli wrote: > > > > > > > > On 08/02/2018 09:05 AM, Andrew Lunn wrote: > > >> I dont see any Reply's on the PC with tcpdump on PC > > > > > > So try ethool -S on the PC. Any packets dropped because of errors? > > > > > > Try turning off hardware checksums on the switch. ethtool -K. > > > > Also make sure that cpsw is properly sending 64 bytes (including FCS) > > packets to the switch, some switches will just discard packets when > > packets are RUNT > > how can I dump the FCS from the switch (can I use tshark ?) tskark running on the target will not show you the FCS. The Ethernet hardware should calculate that as it sends the packet. Wireshark on the PC might, but some Ethernet cards strip off the FCS before passing it to the stack. What is important here is the idea of a runt packet. You appear to be receiving some packets at the PC, if we can trust the statistics you showed. Are those received packets all big? Are the small ARP replies missing? Andrew
Re: [Query]: DSA Understanding
On Thu, Aug 09, 2018 at 12:31:31PM +0100, Lad, Prabhakar wrote: > Hi Andrew, > > On Thu, Aug 2, 2018 at 5:05 PM Andrew Lunn wrote: > > > > > I dont see any Reply's on the PC with tcpdump on PC > > > > So try ethool -S on the PC. Any packets dropped because of errors? > > > I dont see any drops/errors on the PC, following is the dump from PC: > > sudo ethtool -S enx00e04c68c229 > [sudo] password for prabhakar: > NIC statistics: > tx_packets: 1659 > rx_packets: 485 > tx_errors: 0 > rx_errors: 0 > rx_missed: 0 > align_errors: 0 > tx_single_collisions: 0 > tx_multi_collisions: 0 > rx_unicast: 18 > rx_broadcast: 295 > rx_multicast: 172 > tx_aborted: 0 > tx_underrun: 0 So there are received packets at the PC. Not many unicast, mostly broadcast, which fits with ARP. What does wireshark tell you about these received packets? Are they ARP replies? Are they something else? If they are ARP replies, why are they being ignored? I don't know if tshark will show you CRC problems. Wireshark does, when you unfold a packet, and look at the fields in detail. > Seems like the packet is not being transmitted from the switch at all > ? (as ping from switch lan4 to PC fails) I don't think you can make that conclusion yet. The PC is receiving something, rx_packets=485. What are those packets? Look at the statistics along the chain, from the target to the PC. Look at the master device, lan4 and the PC. You should see about one packet per second transmitted on the master device. One packet per second transmitted on lan4, and one packet per second received on the PC. Where does this break down. Andrew
Re: [Query]: DSA Understanding
Hi Florain, Thanks for your reply. On Thu, Aug 2, 2018 at 5:24 PM Florian Fainelli wrote: > > > > On 08/02/2018 09:05 AM, Andrew Lunn wrote: > >> I dont see any Reply's on the PC with tcpdump on PC > > > > So try ethool -S on the PC. Any packets dropped because of errors? > > > > Try turning off hardware checksums on the switch. ethtool -K. > > Also make sure that cpsw is properly sending 64 bytes (including FCS) > packets to the switch, some switches will just discard packets when > packets are RUNT how can I dump the FCS from the switch (can I use tshark ?) Cheers, --Prabhakar Lad
Re: [Query]: DSA Understanding
Hi Andrew, On Thu, Aug 2, 2018 at 5:05 PM Andrew Lunn wrote: > > > I dont see any Reply's on the PC with tcpdump on PC > > So try ethool -S on the PC. Any packets dropped because of errors? > I dont see any drops/errors on the PC, following is the dump from PC: sudo ethtool -S enx00e04c68c229 [sudo] password for prabhakar: NIC statistics: tx_packets: 1659 rx_packets: 485 tx_errors: 0 rx_errors: 0 rx_missed: 0 align_errors: 0 tx_single_collisions: 0 tx_multi_collisions: 0 rx_unicast: 18 rx_broadcast: 295 rx_multicast: 172 tx_aborted: 0 tx_underrun: 0 > Try turning off hardware checksums on the switch. ethtool -K. > Following is the dump from the switch, the checksums are off ~$ ethtool -k eth1 Features for eth1: Cannot get device udp-fragmentation-offload settings: Operation not supported rx-checksumming: off [fixed] tx-checksumming: off tx-checksum-ipv4: off [fixed] tx-checksum-ip-generic: off [fixed] tx-checksum-ipv6: off [fixed] tx-checksum-fcoe-crc: off [fixed] tx-checksum-sctp: off [fixed] scatter-gather: off tx-scatter-gather: off [fixed] tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: off tx-tcp-segmentation: off [fixed] tx-tcp-ecn-segmentation: off [fixed] tx-tcp-mangleid-segmentation: off [fixed] tx-tcp6-segmentation: off [fixed] udp-fragmentation-offload: off generic-segmentation-offload: off [requested on] generic-receive-offload: on large-receive-offload: off [fixed] rx-vlan-offload: off [fixed] tx-vlan-offload: off [fixed] ntuple-filters: off [fixed] receive-hashing: off [fixed] highdma: off [fixed] rx-vlan-filter: on [fixed] vlan-challenged: off [fixed] tx-lockless: off [fixed] netns-local: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: off [fixed] tx-gre-segmentation: off [fixed] tx-gre-csum-segmentation: off [fixed] tx-ipxip4-segmentation: off [fixed] tx-ipxip6-segmentation: off [fixed] tx-udp_tnl-segmentation: off [fixed] tx-udp_tnl-csum-segmentation: off [fixed] tx-gso-partial: off [fixed] tx-sctp-segmentation: off [fixed] tx-esp-segmentation: off [fixed] fcoe-mtu: off [fixed] tx-nocache-copy: off loopback: off [fixed] rx-fcs: off [fixed] rx-all: off [fixed] tx-vlan-stag-hw-insert: off [fixed] rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] l2-fwd-offload: off [fixed] hw-tc-offload: off [fixed] esp-hw-offload: off [fixed] esp-tx-csum-hw-offload: off [fixed] rx-udp_tunnel-port-offload: off [fixed] Tshark dump on the switch: ~$ tshark -i lan4 Running as user "root" and group "root". This could be dangerous. Capturing on 'lan4' [ 1482.987520] device lan4 entered promiscuous mode [ 1482.992169] device eth1 entered promiscuous mode 1 0.0 RealtekS_68:c2:29 ��→ BroadcastARP 60 Who has 169.254.126.126? Tell 169.254.78.251 2 0.62952 c4:f3:12:08:fe:7f ��→ RealtekS_68:c2:29 ARP 42 169.254.126.126 is at c4:f3:12:08:fe:7f 3 0.997115432 RealtekS_68:c2:29 ��→ BroadcastARP 60 Who has 169.254.126.126? Tell 169.254.78.251 4 0.997142272 c4:f3:12:08:fe:7f ��→ RealtekS_68:c2:29 ARP 42 169.254.126.126 is at c4:f3:12:08:fe:7f 5 1.997036539 RealtekS_68:c2:29 ��→ BroadcastARP 60 Who has 169.254.126.126? Tell 169.254.78.251 6 1.997063379 c4:f3:12:08:fe:7f ��→ RealtekS_68:c2:29 ARP 42 169.254.126.126 is at c4:f3:12:08:fe:7f 7 3.014232032 RealtekS_68:c2:29 ��→ BroadcastARP 60 Who has 169.254.126.126? Tell 169.254.78.251 8 3.014252528 c4:f3:12:08:fe:7f ��→ RealtekS_68:c2:29 ARP 42 169.254.126.126 is at c4:f3:12:08:fe:7f 9 4.013008290 RealtekS_68:c2:29 ��→ BroadcastARP 60 Who has 169.254.126.126? Tell 169.254.78.251 10 4.013031064 c4:f3:12:08:fe:7f ��→ RealtekS_68:c2:29 ARP 42 169.254.126.126 is at c4:f3:12:08:fe:7f 11 5.012951194 RealtekS_68:c2:29 ��→ BroadcastARP 60 Who has 169.254.126.126? Tell 169.254.78.251 12 5.012970552 c4:f3:12:08:fe:7f ��→ RealtekS_68:c2:29 ARP 42 169.254.126.126 is at c4:f3:12:08:fe:7f 13 6.030173853 RealtekS_68:c2:29 ��→ BroadcastARP 60 Who has 169.254.126.126? Tell 169.254.78.251 14 6.030192234 c4:f3:12:08:fe:7f ��→ RealtekS_68:c2:29 ARP 42 169.254.126.126 is at c4:f3:12:08:fe:7f 15 7.028911559 RealtekS_68:c2:29 ��→ BroadcastARP 60 Who has 169.254.126.126? Tell 169.254.78.251 16 7.028947183 c4:f3:12:08:fe:7f ��→ RealtekS_68:c2:29 ARP 42 169.254.126.126 is at c4:f3:12:08:fe:7f ^C[ 1494.020087] device lan4 left promiscuous mode [ 1494.024475] device eth1 left promiscuous mode 16 packets captured Seems like the packet is not being transmitted from the switch at all ? (as ping from switch lan4 to PC fails) ~$ ping -I lan4 169.254.78.251 PING 169.254.78.251 (169.254.78.251): 56 data bytes ^C --- 169.254.78.251 ping statistics --- 24 packets transmitted, 0 packets received, 100% packet loss Cheers, --Prabhakar
Re: [Query]: DSA Understanding
On 08/02/2018 09:05 AM, Andrew Lunn wrote: >> I dont see any Reply's on the PC with tcpdump on PC > > So try ethool -S on the PC. Any packets dropped because of errors? > > Try turning off hardware checksums on the switch. ethtool -K. Also make sure that cpsw is properly sending 64 bytes (including FCS) packets to the switch, some switches will just discard packets when packets are RUNT -- Florian
Re: [Query]: DSA Understanding
> I dont see any Reply's on the PC with tcpdump on PC So try ethool -S on the PC. Any packets dropped because of errors? Try turning off hardware checksums on the switch. ethtool -K. Andrew
Re: [Query]: DSA Understanding
Hi Andrew, On Thu, Aug 2, 2018 at 3:45 PM, Andrew Lunn wrote: >> I have PC connected to lan4(ip = 169.254..126.126) and the PC ip is >> 169.254.78.251, >> but when I ping from PC to lan4 I get Destination Host Unreachable, >> but where as I can see >> that in the tcpdump log for lan4 it does reply back, but it doesn’t >> reach the PC, Is there I am missing >> something here ? >> >> ~$ tcpdump -i lan4 -v >> [ 661.057166] device lan4 entered promiscuous mode >> [ 661.061814] device eth1 entered promiscuous mode > >> 07:40:21.254161 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has >> VB4-SN tell tango-charlie.local, length 46 >> 07:40:21.254181 ARP, Ethernet (len 6), IPv4 (len 4), Reply >> VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 > > Having names here does not help when you gave IP addresses above. > Sorry about that. VB4-SN is the switch device and tango-charlie.local is the PC. > Am i reading this correct? The PC is ARPing the switch device. The > switch device is replying? > Yes the PC is ARPing to the switch device and from the tcpdump on swicth device lan4 I can see that its trying to send Reply ~$ tcpdump -i lan4 -v [ 1675.526326] device lan4 entered promiscuous mode [ 1675.531503] device eth1 entered promiscuous mode tcpdump: listening on lan4, link-type EN10MB (Ethernet), capture size 262144 bytes 07:57:06.133853 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has VB4-SN tell tango-charlie.local, length 46 07:57:06.133893 ARP, Ethernet (len 6), IPv4 (len 4), Reply VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 07:57:07.151100 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has VB4-SN tell tango-charlie.local, length 46 07:57:07.151133 ARP, Ethernet (len 6), IPv4 (len 4), Reply VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 07:57:07.703063 IP (tos 0x0, ttl 64, id 41824, offset 0, flags [DF], proto UDP (17), length 224) tango-charlie.local.netbios-dgm > 169.254.255.255.netbios-dgm: NBT UDP PACKET(138) 07:57:07.804780 IP6 (flowlabel 0x42aaa, hlim 255, next-header UDP (17) payload length: 54) VB4-SN.mdns > ff02::fb.mdns: [udp sum o) 07:57:07.804961 IP (tos 0x0, ttl 255, id 13070, offset 0, flags [DF], proto UDP (17), length 74) ^CVB4-SN.mdns > 224.0.0.251.mdns: 0 PTR (QM)? 255.255.[ 1689.046815] device lan4 left promiscuous mode 254.169.in-addr.arpa. (46) 7 packets captured 53 packets rec[ 1689.055592] device eth1 left promiscuous mode eived by filter 40 packets dropped by kernel > What does tcpdump on the PC show? Are the ARP replies getting to it? > Is the PC dropping the ARP replies? > I dont see any Reply's on the PC with tcpdump on PC > If the PC is dropping the ARP replies, take a look at the > checksums. Wireshark is good at that. > Nor do I see anything in the wireshark. > If the ARP replies are not making it to the PC, look at the switch > statistics. ethtool -S lan4. Are the TX counts going up? Any error > counts going up? > Yes the Tx counts are going up, without any errors, following is the log: ~$ ethtool -S lan4 NIC statistics: tx_packets: 499 tx_bytes: 37105 rx_packets: 462 rx_bytes: 34138 rx_hi: 0 rx_undersize: 0 rx_fragments: 0 rx_oversize: 0 rx_jabbers: 0 rx_symbol_err: 0 rx_crc_err: 0 rx_align_err: 0 rx_mac_ctrl: 0 rx_pause: 0 rx_bcast: 402 rx_mcast: 69 rx_ucast: 0 rx_64_or_less: 361 rx_65_127: 45 rx_128_255: 34 rx_256_511: 31 rx_512_1023: 0 rx_1024_1522: 0 rx_1523_2000: 0 rx_2001: 0 tx_hi: 0 tx_late_col: 0 tx_pause: 0 tx_bcast: 14 tx_mcast: 196 tx_ucast: 0 tx_deferred: 0 tx_total_col: 0 tx_exc_col: 0 tx_single_col: 0 tx_mult_col: 0 rx_total: 43248 tx_total: 33759 rx_discards: 0 tx_discards: 0 Cheers, --Prabhakar
Re: [Query]: DSA Understanding
> I have PC connected to lan4(ip = 169.254..126.126) and the PC ip is > 169.254.78.251, > but when I ping from PC to lan4 I get Destination Host Unreachable, > but where as I can see > that in the tcpdump log for lan4 it does reply back, but it doesn’t > reach the PC, Is there I am missing > something here ? > > ~$ tcpdump -i lan4 -v > [ 661.057166] device lan4 entered promiscuous mode > [ 661.061814] device eth1 entered promiscuous mode > 07:40:21.254161 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has > VB4-SN tell tango-charlie.local, length 46 > 07:40:21.254181 ARP, Ethernet (len 6), IPv4 (len 4), Reply > VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 Having names here does not help when you gave IP addresses above. Am i reading this correct? The PC is ARPing the switch device. The switch device is replying? What does tcpdump on the PC show? Are the ARP replies getting to it? Is the PC dropping the ARP replies? If the PC is dropping the ARP replies, take a look at the checksums. Wireshark is good at that. If the ARP replies are not making it to the PC, look at the switch statistics. ethtool -S lan4. Are the TX counts going up? Any error counts going up? Andrew
Re: [Query]: DSA Understanding
Hi Andrew, Thank for your reply. On Thu, Jul 26, 2018 at 4:39 PM, Andrew Lunn wrote: >> I am bit confused on how dsa needs to be actually working, >> Q's >> 1] should I be running a dhcp server on eth1 (where switch is connected) >> so that devices connected on lan* devices get an ip ? > > Nope. You need eth1 up, but otherwise you do not use it. Use the lanX > interfaces like normal Linux interfaces. Run your dhclient on lanX, etc. >> >> 2] From the device where switch is connected if the cpu port wants to send >>any data to any other user ports lan* how do i do it (just open >> socket on eth1 or lan*) ? > > Just treat the lanX interfaces as normal Linux interfaces. > I have some more query’s on DSA. I have manged to get the TI's cpsw slave1 connected to ksz9897 Ethernet switch chip partially working, I have PC connected to lan4(ip = 169.254..126.126) and the PC ip is 169.254.78.251, but when I ping from PC to lan4 I get Destination Host Unreachable, but where as I can see that in the tcpdump log for lan4 it does reply back, but it doesn’t reach the PC, Is there I am missing something here ? Log from the device on which switch is present: === ~$ ifconfig eth0 Link encap:Ethernet HWaddr C4:F3:12:08:FE:7E UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:102 eth1 Link encap:Ethernet HWaddr C4:F3:12:08:FE:7F inet6 addr: fe80::c6f3:12ff:fe08:fe7f%lo/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:436 errors:0 dropped:0 overruns:0 frame:0 TX packets:516 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:37254 (36.3 KiB) TX bytes:51585 (50.3 KiB) lan4 Link encap:Ethernet HWaddr C4:F3:12:08:FE:7F inet addr:169.254.126.126 Bcast:169.254.255.255 Mask:255.255.0.0 inet6 addr: fe80::c6f3:12ff:fe08:fe7f%lo/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:436 errors:0 dropped:0 overruns:0 frame:0 TX packets:444 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:28970 (28.2 KiB) TX bytes:35214 (34.3 KiB) loLink encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1%1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:67 errors:0 dropped:0 overruns:0 frame:0 TX packets:67 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:6618 (6.4 KiB) TX bytes:6618 (6.4 KiB) ~$ tcpdump -i lan4 -v [ 661.057166] device lan4 entered promiscuous mode [ 661.061814] device eth1 entered promiscuous mode tcpdump: listening on lan4, link-type EN10MB (Ethernet), capture size 262144 bytes 07:40:20.255355 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has VB4-SN tell tango-charlie.local, length 46 07:40:20.255393 ARP, Ethernet (len 6), IPv4 (len 4), Reply VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 07:40:20.360936 IP6 (flowlabel 0x970aa, hlim 255, next-header UDP (17) payload length: 53) VB4-SN.mdns > ff02::fb.mdns: [udp sum o) 07:40:20.361085 IP (tos 0x0, ttl 255, id 17259, offset 0, flags [DF], proto UDP (17), length 73) VB4-SN.mdns > 224.0.0.251.mdns: 0 PTR (QM)? 251.78.254.169.in-addr.arpa. (45) 07:40:20.361848 IP (tos 0x0, ttl 255, id 1808, offset 0, flags [DF], proto UDP (17), length 100) tango-charlie.local.mdns > 224.0.0.251.mdns: 0*- [0q] 1/0/0 251.78.254.169.in-addr.arpa. (Cache flush) PTR tango-charlie.local. (72) 07:40:20.465933 IP6 (flowlabel 0x970aa, hlim 255, next-header UDP (17) payload length: 98) VB4-SN.mdns > ff02::fb.mdns: [udp sum o) 07:40:20.466124 IP (tos 0x0, ttl 255, id 17288, offset 0, flags [DF], proto UDP (17), length 118) VB4-SN.mdns > 224.0.0.251.mdns: 0 PTR (QM)? b.f.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.f.f.ip6.arpa. (90) 07:40:21.254161 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has VB4-SN tell tango-charlie.local, length 46 07:40:21.254181 ARP, Ethernet (len 6), IPv4 (len 4), Reply VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 07:40:25.470288 IP6 (flowlabel 0x970aa, hlim 255, next-header UDP (17) payload length: 50) VB4-SN.mdns > ff02::fb.mdns: [udp sum o) 07:40:31.301929 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has VB4-SN tell tango-charlie.local, length 46 07:40:31.301957 ARP, Ethernet (len 6), IPv4 (len 4), Reply VB4-SN is-at c4:f3:12:08:fe:7f (oui Unknown), length 28 07:40:32.319104 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has VB4-SN tell tango-charlie.local, length
Re: [Query]: DSA Understanding
> I am bit confused on how dsa needs to be actually working, > Q's > 1] should I be running a dhcp server on eth1 (where switch is connected) > so that devices connected on lan* devices get an ip ? Nope. You need eth1 up, but otherwise you do not use it. Use the lanX interfaces like normal Linux interfaces. Run your dhclient on lanX, etc. > > 2] From the device where switch is connected if the cpu port wants to send >any data to any other user ports lan* how do i do it (just open > socket on eth1 or lan*) ? Just treat the lanX interfaces as normal Linux interfaces. Andrew
Re: [Query]: DSA Understanding
On Thu, Jul 26, 2018 at 3:08 PM, Andrew Lunn wrote:
>> Yes I am using fixed phy on slave1, following is my dts:
>
> Posting the original DTS file is better, not the decompiled version.
>
My bad will take care of it next time.
>>
>> ethernet@48484000 {
>> compatible = "ti,dra7-cpsw", "ti,cpsw";
>> ti,hwmods = "gmac";
>> clocks = <0x124 0x125>;
>> clock-names = "fck", "cpts";
>> cpdma_channels = <0x8>;
>> ale_entries = <0x400>;
>> bd_ram_size = <0x2000>;
>> mac_control = <0x20>;
>> slaves = <0x2>;
>> active_slave = <0x0>;
>> cpts_clock_mult = <0x784cfe14>;
>> cpts_clock_shift = <0x1d>;
>> reg = <0x48484000 0x1000 0x48485200 0x2e00>;
>> #address-cells = <0x1>;
>> #size-cells = <0x1>;
>> ti,no-idle;
>> interrupts = <0x0 0x14e 0x4 0x0 0x14f 0x4 0x0 0x150 0x4
>> 0x0 0x151 0x4>;
>> ranges;
>> syscon = <0x8>;
>> status = "okay";
>> pinctrl-names = "default", "sleep";
>> pinctrl-0 = <0x126 0x127>;
>> pinctrl-1 = <0x128 0x129>;
>> dual_emac;
>> linux,phandle = <0x500>;
>> phandle = <0x500>;
>
> So here is 0x500
>
>>
>> mdio@48485000 {
>> compatible = "ti,cpsw-mdio", "ti,davinci_mdio";
>> #address-cells = <0x1>;
>> #size-cells = <0x0>;
>> ti,hwmods = "davinci_mdio";
>> bus_freq = <0xf4240>;
>> reg = <0x48485000 0x100>;
>> status = "okay";
>> pinctrl-names = "default", "sleep";
>> pinctrl-0 = <0x12a>;
>> pinctrl-1 = <0x12b>;
>>
>> ethernet-phy@1 {
>> reg = <0x1>;
>> linux,phandle = <0x12c>;
>> phandle = <0x12c>;
>> };
>> };
>>
>> slave@48480200 {
>> mac-address = [00 00 00 00 00 00];
>> status = "okay";
>> phy-handle = <0x12c>;
>> phy-mode = "rgmii";
>> dual_emac_res_vlan = <0x1>;
>> };
>>
>> slave@48480300 {
>> mac-address = [00 00 00 00 00 00];
>> status = "okay";
>> phy-mode = "rgmii";
>> dual_emac_res_vlan = <0x2>;
>> linux,phandle = <0xf3>;
>> phandle = <0xf3>;
>
> This is the actual interface you are using and it has a phandle of
> 0xf3.
>
The dsa of_find_net_device_by_node() looks for ethernet device node,
but the above node is for the cpsw slave. I tried adding 0xf3 and got
probe defer,
that is the reason I added 0x500 (mdio) reference to cpu switch and
patched to pick
up eth1 instead of eth0.
>>
>> fixed-link {
>> speed = <0x3e8>;
>> full-duplex;
>> };
>> };
>>
>> cpsw-phy-sel@4a002554 {
>> compatible = "ti,dra7xx-cpsw-phy-sel";
>> reg = <0x4a002554 0x4>;
>> reg-names = "gmii-sel";
>> };
>> };
>>
>> spi@480ba000 {
>> compatible = "ti,omap4-mcspi";
>> reg = <0x480ba000 0x200>;
>> interrupts = <0x0 0x2b 0x4>;
>> #address-cells = <0x1>;
>> #size-cells = <0x0>;
>> ti,hwmods = "mcspi4";
>> ti,spi-num-cs = <0x1>;
>> dmas = <0xb2 0x46 0xb2 0x47>;
>> dma-names = "tx0", "rx0";
>> status = "okay";
>> ti,pindir-d0-out-d1-in;
>>
>> ksz9477@0 {
>> compatible = "microchip,ksz9477";
>> reg = <0x0>;
>> spi-max-frequency = <0x2dc6c00>;
>> spi-cpha;
>> spi-cpol;
>>
>> ports {
>> #address-cells = <0x1>;
>> #size-cells = <0x0>;
>>
>> port@0 {
>> reg = <0x0>;
>> label = "lan0";
>> };
>>
>> port@1 {
>> reg = <0x1>;
>> label = "lan1";
>> };
>>
>> port@2 {
>> reg = <0x2>;
>> label = "lan2";
>> };
>>
>> port@3 {
>> reg = <0x3>;
>> label = "lan3";
>> };
>>
>> port@4 {
>> reg = <0x4>;
>> label = "lan4";
>> };
>>
>> port@5 {
>> reg = <0x5>;
>> label = "cpu";
>> ethernet =
Re: [Query]: DSA Understanding
> Yes I am using fixed phy on slave1, following is my dts:
Posting the original DTS file is better, not the decompiled version.
>
> ethernet@48484000 {
> compatible = "ti,dra7-cpsw", "ti,cpsw";
> ti,hwmods = "gmac";
> clocks = <0x124 0x125>;
> clock-names = "fck", "cpts";
> cpdma_channels = <0x8>;
> ale_entries = <0x400>;
> bd_ram_size = <0x2000>;
> mac_control = <0x20>;
> slaves = <0x2>;
> active_slave = <0x0>;
> cpts_clock_mult = <0x784cfe14>;
> cpts_clock_shift = <0x1d>;
> reg = <0x48484000 0x1000 0x48485200 0x2e00>;
> #address-cells = <0x1>;
> #size-cells = <0x1>;
> ti,no-idle;
> interrupts = <0x0 0x14e 0x4 0x0 0x14f 0x4 0x0 0x150 0x4
> 0x0 0x151 0x4>;
> ranges;
> syscon = <0x8>;
> status = "okay";
> pinctrl-names = "default", "sleep";
> pinctrl-0 = <0x126 0x127>;
> pinctrl-1 = <0x128 0x129>;
> dual_emac;
> linux,phandle = <0x500>;
> phandle = <0x500>;
So here is 0x500
>
> mdio@48485000 {
> compatible = "ti,cpsw-mdio", "ti,davinci_mdio";
> #address-cells = <0x1>;
> #size-cells = <0x0>;
> ti,hwmods = "davinci_mdio";
> bus_freq = <0xf4240>;
> reg = <0x48485000 0x100>;
> status = "okay";
> pinctrl-names = "default", "sleep";
> pinctrl-0 = <0x12a>;
> pinctrl-1 = <0x12b>;
>
> ethernet-phy@1 {
> reg = <0x1>;
> linux,phandle = <0x12c>;
> phandle = <0x12c>;
> };
> };
>
> slave@48480200 {
> mac-address = [00 00 00 00 00 00];
> status = "okay";
> phy-handle = <0x12c>;
> phy-mode = "rgmii";
> dual_emac_res_vlan = <0x1>;
> };
>
> slave@48480300 {
> mac-address = [00 00 00 00 00 00];
> status = "okay";
> phy-mode = "rgmii";
> dual_emac_res_vlan = <0x2>;
> linux,phandle = <0xf3>;
> phandle = <0xf3>;
This is the actual interface you are using and it has a phandle of
0xf3.
>
> fixed-link {
> speed = <0x3e8>;
> full-duplex;
> };
> };
>
> cpsw-phy-sel@4a002554 {
> compatible = "ti,dra7xx-cpsw-phy-sel";
> reg = <0x4a002554 0x4>;
> reg-names = "gmii-sel";
> };
> };
>
> spi@480ba000 {
> compatible = "ti,omap4-mcspi";
> reg = <0x480ba000 0x200>;
> interrupts = <0x0 0x2b 0x4>;
> #address-cells = <0x1>;
> #size-cells = <0x0>;
> ti,hwmods = "mcspi4";
> ti,spi-num-cs = <0x1>;
> dmas = <0xb2 0x46 0xb2 0x47>;
> dma-names = "tx0", "rx0";
> status = "okay";
> ti,pindir-d0-out-d1-in;
>
> ksz9477@0 {
> compatible = "microchip,ksz9477";
> reg = <0x0>;
> spi-max-frequency = <0x2dc6c00>;
> spi-cpha;
> spi-cpol;
>
> ports {
> #address-cells = <0x1>;
> #size-cells = <0x0>;
>
> port@0 {
> reg = <0x0>;
> label = "lan0";
> };
>
> port@1 {
> reg = <0x1>;
> label = "lan1";
> };
>
> port@2 {
> reg = <0x2>;
> label = "lan2";
> };
>
> port@3 {
> reg = <0x3>;
> label = "lan3";
> };
>
> port@4 {
> reg = <0x4>;
> label = "lan4";
> };
>
> port@5 {
> reg = <0x5>;
> label = "cpu";
> ethernet = <0x500>;
So here you are pointing to the container node, not the actual
interface. You talk of having to patch DSA. That i would say is
wrong. You should be using the phandle of the actual interface.
> };
>
> port@6 {
> reg = <0x6>;
> label = "lan6";
> };
> };
> };
> };
>
> Where port 6 of ksz9897 chip is connected to slave 1 (eth1) of
Re: [Query]: DSA Understanding
Hi Andrew,
Thanks for the reply.
On Wed, Jul 25, 2018 at 5:19 PM, Andrew Lunn wrote:
> On Wed, Jul 25, 2018 at 09:39:50AM +0100, Lad, Prabhakar wrote:
>> Hi,
>>
>> We are trying to integrate a MAC to an external switch as following:
>>
>> +---++--+
>> || RGMII ||
>> |++ KSZ9897 +-- 1000baseT MDI ("LAN1")
>> || PORT 6 | 7-port +-- 1000baseT MDI
>> ("LAN2")
>> |AM572x| | Ethernet +-- 1000baseT MDI ("LAN3")
>> | CPSW |MIImgmt | switch+-- 1000baseT MDI ("LAN4")
>> |+-+ +--
>> 1000baseT MDI ("LAN5")
>> ||SPI||
>> || | |
>> || | +--
>> 1000baseT MII ("LAN7")
>> || | |
>> +--+ +--+
>
> Please use a fixed size font, otherwise this is unreadable.
>
sorry about that, Ill take care of it henceforth.
>>
>> I have done all the configuration required for the MAC controller and
>> enabled the KSZ driver in the kernel, I can see the following:
>> ~$ ip link show
>> 1: lo: mtu 65536 qdisc noqueue state UNKNOWN
>> mode DEFAULT group default qlen 1000
>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>> 2: eth0: mtu 1500 qdisc mq state
>> DOWN mode DEFAULT group default qlen 1000
>
> eth0 is you master device. NO-CARRIER is bad, since that means you
> cannot send/receive from from the switch. Are you using a fixed phy on
> the CPSW? What does your device tree look like?
>
With my earlier post, the dsa was linking to eth0, which is incorrect
as the switch is connected to eth1.
I had to patch the dsa to pickup eth1 instead of eth0.
CPSW can support two ports internally one port is connected to KSZ9031
phy chip which is eth0 and the
other port is connected to KSZ9897 chip eth1, following is the ip link
dump with my patch applied.
~$ ip link show
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: mtu 1500 qdisc mq state
DOWN mode DEFAULT group default qlen 1000
link/ether c4:f3:12:08:fe:7e brd ff:ff:ff:ff:ff:ff
3: eth1: mtu 1500 qdisc mq state UP
mode DEFAULT group default qlen 1000
link/ether c4:f3:12:08:fe:7f brd ff:ff:ff:ff:ff:ff
4: sit0@NONE: mtu 1480 qdisc noop state DOWN mode DEFAULT
group default qlen 1000
link/sit 0.0.0.0 brd 0.0.0.0
5: lan0@eth1: mtu 1500 qdisc noop state DOWN
mode DEFAULT group default qlen 1000
link/ether c4:f3:12:08:fe:7f brd ff:ff:ff:ff:ff:ff
6: lan1@eth1: mtu 1500 qdisc noop state DOWN
mode DEFAULT group default qlen 1000
link/ether c4:f3:12:08:fe:7f brd ff:ff:ff:ff:ff:ff
7: lan2@eth1: mtu 1500 qdisc noop state DOWN
mode DEFAULT group default qlen 1000
link/ether c4:f3:12:08:fe:7f brd ff:ff:ff:ff:ff:ff
8: lan3@eth1: mtu 1500 qdisc noop state DOWN
mode DEFAULT group default qlen 1000
link/ether c4:f3:12:08:fe:7f brd ff:ff:ff:ff:ff:ff
9: lan4@eth1: mtu 1500 qdisc noop state DOWN
mode DEFAULT group default qlen 1000
link/ether c4:f3:12:08:fe:7f brd ff:ff:ff:ff:ff:ff
10: lan6@eth1: mtu 1500 qdisc noop state DOWN
mode DEFAULT group default qlen 1000
link/ether c4:f3:12:08:fe:7f brd ff:ff:ff:ff:ff:ff
~$
Yes I am using fixed phy on slave1, following is my dts:
ethernet@48484000 {
compatible = "ti,dra7-cpsw", "ti,cpsw";
ti,hwmods = "gmac";
clocks = <0x124 0x125>;
clock-names = "fck", "cpts";
cpdma_channels = <0x8>;
ale_entries = <0x400>;
bd_ram_size = <0x2000>;
mac_control = <0x20>;
slaves = <0x2>;
active_slave = <0x0>;
cpts_clock_mult = <0x784cfe14>;
cpts_clock_shift = <0x1d>;
reg = <0x48484000 0x1000 0x48485200 0x2e00>;
#address-cells = <0x1>;
#size-cells = <0x1>;
ti,no-idle;
interrupts = <0x0 0x14e 0x4 0x0 0x14f 0x4 0x0 0x150 0x4
0x0 0x151 0x4>;
ranges;
syscon = <0x8>;
status = "okay";
pinctrl-names = "default", "sleep";
pinctrl-0 = <0x126 0x127>;
pinctrl-1 = <0x128 0x129>;
dual_emac;
linux,phandle = <0x500>;
phandle = <0x500>;
mdio@48485000 {
compatible = "ti,cpsw-mdio", "ti,davinci_mdio";
#address-cells = <0x1>;
#size-cells = <0x0>;
ti,hwmods = "davinci_mdio";
bus_freq = <0xf4240>;
reg = <0x48485000 0x100>;
status = "okay";
pinctrl-names = "default", "sleep";
Re: [Query]: DSA Understanding
On Wed, Jul 25, 2018 at 09:39:50AM +0100, Lad, Prabhakar wrote:
> Hi,
>
> We are trying to integrate a MAC to an external switch as following:
>
> +---++--+
> || RGMII ||
> |++ KSZ9897 +-- 1000baseT MDI ("LAN1")
> || PORT 6 | 7-port +-- 1000baseT MDI ("LAN2")
> |AM572x| | Ethernet +-- 1000baseT MDI ("LAN3")
> | CPSW |MIImgmt | switch+-- 1000baseT MDI ("LAN4")
> |+-+ +--
> 1000baseT MDI ("LAN5")
> ||SPI||
> || | |
> || | +--
> 1000baseT MII ("LAN7")
> || | |
> +--+ +--+
Please use a fixed size font, otherwise this is unreadable.
>
> I have done all the configuration required for the MAC controller and
> enabled the KSZ driver in the kernel, I can see the following:
> ~$ ip link show
> 1: lo: mtu 65536 qdisc noqueue state UNKNOWN
> mode DEFAULT group default qlen 1000
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eth0: mtu 1500 qdisc mq state
> DOWN mode DEFAULT group default qlen 1000
eth0 is you master device. NO-CARRIER is bad, since that means you
cannot send/receive from from the switch. Are you using a fixed phy on
the CPSW? What does your device tree look like?
Andrew
