Re: BCM5719C/BCM5720 partially working

Thu, 23 May 2013 02:14:08 -0700 

Hi,
Having finally found the BCM5718 Programmers guide from http://www.broadcom.com/support/ethernet_nic/open_source.php I managed to get a bit further with this problem. 


The problem seems to be in the Auto polling of the mac link state. 
ifconfig shows the correct link state, but the BGE_STS_LINK bit in the 
driver is never set. Debugging the interrupt routine I found the driver 
never receives a link state changed attention, instead the last 
interrupt indicates a auto-poll error and bit 0 of register 
BGE_AUTOPOLL_STS is set.



Looking at the FreeBSD and Linux drivers I found that they don't use 
auto-polling. Also the programmers guide of the BCM5718 family(page 204) 
doesn't describe the auto-polling method anymore, as was done in the PG 
of other BCM57xx.



So I disabled Auto-polling for the BCM5718 family. This made all ports 
work on my Dell R320(BCM5720) and BCM5719. My BCM5719 still doesn't 
fully work because now i run into bge_watchdog reset's when sending big 
packet, the same issue reported by Robert Young on 5 April.



Attached is a patch that disables mac polling for the BCM5717, 5719 and 
5720. Tested on BCM5719, BCM5720 and BCM5750 C1.


I also noticed if_bge.c line 2293:
BGE_SETBIT(sc, BGE_MI_MODE, BGE_MIMODE_AUTOPOLL|10<<16);

Does any one have a clue why the (10<<16) is there? it changes the MI 
clock. But why? and why OR 0xA with the default 0xC, instead of 0x2? 
FreeBSD and Linux use the default of 0xC for the MI Clock.


Kind regards,

David

Index: sys/dev/pci/if_bge.c
===================================================================
RCS file: /cvs/src/sys/dev/pci/if_bge.c,v
retrieving revision 1.320
diff -u -r1.320 if_bge.c
--- sys/dev/pci/if_bge.c        4 Mar 2013 01:33:18 -0000       1.320
+++ sys/dev/pci/if_bge.c        21 May 2013 13:18:36 -0000
@@ -2288,7 +2288,9 @@
        /* Enable PHY auto polling (for MII/GMII only) */
        if (sc->bge_flags & BGE_PHY_FIBER_TBI) {
                CSR_WRITE_4(sc, BGE_MI_STS, BGE_MISTS_LINK);
-       } else {
+       } else if (!(BGE_ASICREV(sc->bge_chipid) == BGE_ASICREV_BCM5717 ||
+           BGE_ASICREV(sc->bge_chipid) == BGE_ASICREV_BCM5719 ||
+           BGE_ASICREV(sc->bge_chipid) == BGE_ASICREV_BCM5720)) {
                BGE_STS_SETBIT(sc, BGE_STS_AUTOPOLL);
                BGE_SETBIT(sc, BGE_MI_MODE, BGE_MIMODE_AUTOPOLL|10<<16);
                if (BGE_ASICREV(sc->bge_chipid) == BGE_ASICREV_BCM5700)
@@ -4386,18 +4388,13 @@
                        if_link_state_change(ifp);
                        ifp->if_baudrate = 0;
                }
-       /*

-	 * Discard link events for MII/GMII cards if MI auto-polling 
disabled.

-        * This should not happen since mii callouts are locked now, but
-        * we keep this check for debug.
-        */
-       } else if (BGE_STS_BIT(sc, BGE_STS_AUTOPOLL)) {
+       } else {
                /*
                 * Some broken BCM chips have BGE_STATFLAG_LINKSTATE_CHANGED bit
                 * in status word always set. Workaround this bug by reading
                 * PHY link status directly.
                 */
-               link = (CSR_READ_4(sc, BGE_MI_STS) & BGE_MISTS_LINK)?
+               link = (CSR_READ_4(sc, BGE_TX_STS) & BGE_TXSTAT_LINK_UP)?
                    BGE_STS_LINK : 0;

                if (BGE_STS_BIT(sc, BGE_STS_LINK) != link) {


David Imhoff schreef op 2013-05-10 14:12:

Hi,

I'm having problems with a 4-ports BCM5719C based PCI-E network card
and the 2-ports BCM5720 network interfaces build into a Dell R320
server. The first network port functions fine but the other ports do
not, partially(only receive) or sometimes function. Where the most
common situation is that a port only receives packets, but do not
transmit packets. Forcing packets to be send on one of these broken
interfaces, by creating a static arp entry and flood pinging that
address, always results in the following error:
 "ping: sendto: No buffer space available"
Currently i have the second port of the BCM5719C based card working,

only if a bring up all ports of the interface in the order from 0 to 
4.



I tested with two different R320 servers, two different BCM5719C 
cards,

and two different firmware versions for the BCM5719C cards. All with
the same results.

I'm sorry that I can't be more specific, but i'm available for
questions or running tests/debugging.

Kind Regards,

David

dmesg of server with BCM5719C cards:
OpenBSD 5.3-current (GENERIC.MP) #108: Tue Apr 30 11:35:41 MDT 2013
    [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4273274880 (4075MB)
avail mem = 4151812096 (3959MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xcfb9c000 (67 entries)
bios0: vendor Dell Inc. version "2.5.0" date 10/13/2008
bios0: Dell Inc. PowerEdge 1950
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5

acpi0: tables DSDT FACP APIC SPCR HPET MCFG WD__ SLIC ERST HEST BERT 
EINJ TCPA

acpi0: wakeup devices PCI0(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz, 2494.10 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE,NXE,LONG,LAHF,PERF
cpu0: 6MB 64b/line 16-way L2 cache
cpu0: apic clock running at 332MHz
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz, 2493.78 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE,NXE,LONG,LAHF,PERF
cpu1: 6MB 64b/line 16-way L2 cache
cpu2 at mainbus0: apid 1 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz, 2493.78 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE,NXE,LONG,LAHF,PERF
cpu2: 6MB 64b/line 16-way L2 cache
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz, 2493.78 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE,NXE,LONG,LAHF,PERF
cpu3: 6MB 64b/line 16-way L2 cache
ioapic0 at mainbus0: apid 4 pa 0xfec00000, version 20, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 4
acpihpet0 at acpi0: 14318179 Hz
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 4 (PEX2)
acpiprt2 at acpi0: bus 5 (UPST)
acpiprt3 at acpi0: bus 6 (DWN1)
acpiprt4 at acpi0: bus 8 (DWN2)
acpiprt5 at acpi0: bus 1 (PEX3)
acpiprt6 at acpi0: bus -1 (PE2P)
acpiprt7 at acpi0: bus 10 (PEX4)
acpiprt8 at acpi0: bus 12 (PEX6)
acpiprt9 at acpi0: bus 2 (SBEX)
acpiprt10 at acpi0: bus 14 (COMP)
acpicpu0 at acpi0: C3
acpicpu1 at acpi0: C3
acpicpu2 at acpi0: C3
acpicpu3 at acpi0: C3
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 5000X Host" rev 0x12
ppb0 at pci0 dev 2 function 0 "Intel 5000 PCIE" rev 0x12
pci1 at ppb0 bus 4
ppb1 at pci1 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01
pci2 at ppb1 bus 5
ppb2 at pci2 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01
pci3 at ppb2 bus 6
ppb3 at pci3 dev 0 function 0 "ServerWorks PCIE-PCIX" rev 0xc3
pci4 at ppb3 bus 7

bnx0 at pci4 dev 0 function 0 "Broadcom BCM5708" rev 0x12: apic 4 int 
16

ppb4 at pci2 dev 1 function 0 "Intel 6321ESB PCIE" rev 0x01
pci5 at ppb4 bus 8
ppb5 at pci1 dev 0 function 3 "Intel 6321ESB PCIE-PCIX" rev 0x01
pci6 at ppb5 bus 9
ppb6 at pci0 dev 3 function 0 "Intel 5000 PCIE" rev 0x12
pci7 at ppb6 bus 1

mfi0 at pci7 dev 0 function 0 "Symbios Logic SAS1078" rev 0x04: apic 4 
int 16

mfi0: "PERC 6/i Integrated", firmware 6.2.0-0013, 256MB cache
scsibus0 at mfi0: 64 targets
sd0 at scsibus0 targ 0 lun 0: <DELL, PERC 6/i, 1.22> SCSI3 0/direct
fixed naa.60024e80631d5a0018e2db97103eaaf7
sd0: 139392MB, 512 bytes/sector, 285474816 sectors
ppb7 at pci0 dev 4 function 0 "Intel 5000 PCIE x8" rev 0x12: msi
pci8 at ppb7 bus 10
bge0 at pci8 dev 0 function 0 "Broadcom BCM5719" rev 0x01, unknown
BCM5719 (0x5719001), APE firmware NCSI 1.0.85.0: apic 4 int 16,
address 00:10:18:f6:1d:70
brgphy0 at bge0 phy 1: BCM5719C 10/100/1000baseT PHY, rev. 0
bge1 at pci8 dev 0 function 1 "Broadcom BCM5719" rev 0x01, unknown
BCM5719 (0x5719001), APE firmware NCSI 1.0.85.0: apic 4 int 17,
address 00:10:18:f6:1d:71
brgphy1 at bge1 phy 2: BCM5719C 10/100/1000baseT PHY, rev. 0
bge2 at pci8 dev 0 function 2 "Broadcom BCM5719" rev 0x01, unknown
BCM5719 (0x5719001), APE firmware NCSI 1.0.85.0: apic 4 int 16,
address 00:10:18:f6:1d:72
brgphy2 at bge2 phy 3: BCM5719C 10/100/1000baseT PHY, rev. 0
bge3 at pci8 dev 0 function 3 "Broadcom BCM5719" rev 0x01, unknown
BCM5719 (0x5719001), APE firmware NCSI 1.0.85.0: apic 4 int 17,
address 00:10:18:f6:1d:73
brgphy3 at bge3 phy 4: BCM5719C 10/100/1000baseT PHY, rev. 0
ppb8 at pci0 dev 5 function 0 "Intel 5000 PCIE" rev 0x12
pci9 at ppb8 bus 11
ppb9 at pci0 dev 6 function 0 "Intel 5000 PCIE x8" rev 0x12: msi
pci10 at ppb9 bus 12
bge4 at pci10 dev 0 function 0 "Broadcom BCM5719" rev 0x01, unknown
BCM5719 (0x5719001), APE firmware NCSI 1.1.7.0: apic 4 int 16, address
00:10:18:f6:1d:80
brgphy4 at bge4 phy 1: BCM5719C 10/100/1000baseT PHY, rev. 0
bge5 at pci10 dev 0 function 1 "Broadcom BCM5719" rev 0x01, unknown
BCM5719 (0x5719001), APE firmware NCSI 1.1.7.0: apic 4 int 17, address
00:10:18:f6:1d:81
brgphy5 at bge5 phy 2: BCM5719C 10/100/1000baseT PHY, rev. 0
bge6 at pci10 dev 0 function 2 "Broadcom BCM5719" rev 0x01, unknown
BCM5719 (0x5719001), APE firmware NCSI 1.1.7.0: apic 4 int 16, address
00:10:18:f6:1d:82
brgphy6 at bge6 phy 3: BCM5719C 10/100/1000baseT PHY, rev. 0
bge7 at pci10 dev 0 function 3 "Broadcom BCM5719" rev 0x01, unknown
BCM5719 (0x5719001), APE firmware NCSI 1.1.7.0: apic 4 int 17, address
00:10:18:f6:1d:83
brgphy7 at bge7 phy 4: BCM5719C 10/100/1000baseT PHY, rev. 0
ppb10 at pci0 dev 7 function 0 "Intel 5000 PCIE" rev 0x12
pci11 at ppb10 bus 13
pchb1 at pci0 dev 16 function 0 "Intel 5000 Error Reporting" rev 0x12
pchb2 at pci0 dev 16 function 1 "Intel 5000 Error Reporting" rev 0x12
pchb3 at pci0 dev 16 function 2 "Intel 5000 Error Reporting" rev 0x12
pchb4 at pci0 dev 17 function 0 "Intel 5000 Reserved" rev 0x12
pchb5 at pci0 dev 19 function 0 "Intel 5000 Reserved" rev 0x12
pchb6 at pci0 dev 21 function 0 "Intel 5000 FBD" rev 0x12
pchb7 at pci0 dev 22 function 0 "Intel 5000 FBD" rev 0x12
ppb11 at pci0 dev 28 function 0 "Intel 6321ESB PCIE" rev 0x09
pci12 at ppb11 bus 2
ppb12 at pci12 dev 0 function 0 "ServerWorks PCIE-PCIX" rev 0xc3
pci13 at ppb12 bus 3

bnx1 at pci13 dev 0 function 0 "Broadcom BCM5708" rev 0x12: apic 4 int 
16
uhci0 at pci0 dev 29 function 0 "Intel 6321ESB USB" rev 0x09: apic 4 
int 21
uhci1 at pci0 dev 29 function 1 "Intel 6321ESB USB" rev 0x09: apic 4 
int 20
uhci2 at pci0 dev 29 function 2 "Intel 6321ESB USB" rev 0x09: apic 4 
int 21
uhci3 at pci0 dev 29 function 3 "Intel 6321ESB USB" rev 0x09: apic 4 
int 20
ehci0 at pci0 dev 29 function 7 "Intel 6321ESB USB" rev 0x09: apic 4 
int 21

usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb13 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xd9
pci14 at ppb13 bus 14
vga1 at pci14 dev 13 function 0 "ATI ES1000" rev 0x02
radeondrm0 at vga1: apic 4 int 19
drm0 at radeondrm0
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 31 function 0 "Intel 6321ESB LPC" rev 0x09
pciide0 at pci0 dev 31 function 2 "Intel 6321ESB SATA" rev 0x09: DMA,
channel 0 configured to native-PCI, channel 1 configured to native-PCI
pciide0: using apic 4 int 23 for native-PCI interrupt
atapiscsi0 at pciide0 channel 0 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <TSSTcorp, DVD+-RW TS-L633A, D200>
ATAPI 5/cdrom removable
cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci2: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci3: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
com1: console
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
mtrr: Pentium Pro MTRR support

uhub5 at uhub0 port 5 "Cypress Semiconductor USB2 Hub" rev 2.00/0.0b 
addr 2

uhidev0 at uhub2 port 2 configuration 1 interface 0 "Lite-On
Technology Corp. USB Multimedia Keyboard" rev 1.10/1.03 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub2 port 2 configuration 1 interface 1 "Lite-On
Technology Corp. USB Multimedia Keyboard" rev 1.10/1.03 addr 2
uhidev1: iclass 3/0, 2 report ids
uhid0 at uhidev1 reportid 1: input=2, output=0, feature=0
uhid1 at uhidev1 reportid 2: input=1, output=0, feature=0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (5baf90cf05f69c25.a) swap on sd0b dump on sd0b
bnx0: address 00:22:19:5c:dd:b2
brgphy8 at bnx0 phy 1: BCM5708C 10/100/1000baseT PHY, rev. 6
bnx1: address 00:22:19:5c:dd:b0
brgphy9 at bnx1 phy 1: BCM5708C 10/100/1000baseT PHY, rev. 6
























					Reply via email to