Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-03 Thread Måns Rullgård 
Mason  writes:

> IIUC, sender (desktop system) sends datagrams as fast as possible.
> Receiver (tango board) drops around 24% of all datagrams.
> I think this invalidates the theory that exhausting RX descriptors
> wedges RX DMA.

No, it doesn't.  The bottleneck causing packet loss could be somewhere
else.

-- 
Måns Rullgård

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-03 Thread Måns Rullgård 
Mason  writes:

> On 02/08/2017 19:31, Mason wrote:
>
>> # iperf3 -c 172.27.64.45 -u -b 950M
>> Connecting to host 172.27.64.45, port 5201
>> [  4] local 172.27.64.1 port 55533 connected to 172.27.64.45 port 5201
>> [ ID] Interval   Transfer Bandwidth   Total Datagrams
>> [  4]   0.00-1.00   sec   102 MBytes   858 Mbits/sec  13091  
>> [  4]   1.00-2.00   sec   114 MBytes   953 Mbits/sec  14541  
>
> 114 MB in 14541 packets => 7840 bytes per packet
> Is iperf3 sending jumbo frames??

It's probably sending fragmented packets.

-- 
Måns Rullgård

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-03 Thread Mason 
On 02/08/2017 22:02, Mason wrote:

> I need to run the test slightly slower, to prevent packet loss
> at the sender.

# iperf3 -c 172.27.64.45 -u -b 0 -l 1000
Connecting to host 172.27.64.45, port 5201
[  4] local 172.27.64.1 port 42607 connected to 172.27.64.45 port 5201
[ ID] Interval   Transfer Bandwidth   Total Datagrams
[  4]   0.00-1.00   sec   111 MBytes   931 Mbits/sec  116420  
[  4]   1.00-2.00   sec   111 MBytes   931 Mbits/sec  116390  
[  4]   2.00-3.00   sec   111 MBytes   930 Mbits/sec  116220  
[  4]   3.00-4.00   sec   111 MBytes   930 Mbits/sec  116310  
[  4]   4.00-5.00   sec   111 MBytes   931 Mbits/sec  116380  
[  4]   5.00-6.00   sec   111 MBytes   930 Mbits/sec  116280  
[  4]   6.00-7.00   sec   111 MBytes   931 Mbits/sec  116390  
[  4]   7.00-8.00   sec   111 MBytes   931 Mbits/sec  116370  
[  4]   8.00-9.00   sec   111 MBytes   931 Mbits/sec  116340  
[  4]   9.00-10.00  sec   111 MBytes   930 Mbits/sec  116310  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval   Transfer Bandwidth   JitterLost/Total 
Datagrams
[  4]   0.00-10.00  sec  1.08 GBytes   931 Mbits/sec  0.009 ms  278644/1163363 
(24%)  
[  4] Sent 1163363 datagrams

iperf Done.


# iperf3 -s
Accepted connection from 172.27.64.1, port 42966
[  5] local 172.27.64.45 port 5201 connected to 172.27.64.1 port 42607
[ ID] Interval   Transfer Bandwidth   JitterLost/Total 
Datagrams
[  5]   0.00-1.00   sec  81.1 MBytes   681 Mbits/sec  0.017 ms  26834/111909 
(24%)  
[  5]   1.00-2.00   sec  84.2 MBytes   706 Mbits/sec  0.019 ms  28127/116384 
(24%)  
[  5]   2.00-3.00   sec  84.2 MBytes   706 Mbits/sec  0.013 ms  27946/116204 
(24%)  
[  5]   3.00-4.00   sec  84.5 MBytes   709 Mbits/sec  0.013 ms  27674/116311 
(24%)  
[  5]   4.00-5.00   sec  84.6 MBytes   709 Mbits/sec  0.015 ms  27712/116387 
(24%)  
[  5]   5.00-6.00   sec  84.5 MBytes   709 Mbits/sec  0.010 ms  27649/116265 
(24%)  
[  5]   6.00-7.00   sec  84.3 MBytes   707 Mbits/sec  0.011 ms  27995/116382 
(24%)  
[  5]   7.00-8.00   sec  84.3 MBytes   707 Mbits/sec  0.013 ms  27972/116387 
(24%)  
[  5]   8.00-9.00   sec  84.3 MBytes   708 Mbits/sec  0.020 ms  27899/116343 
(24%)  
[  5]   9.00-10.00  sec  84.4 MBytes   708 Mbits/sec  0.014 ms  27759/116305 
(24%)  
[  5]  10.00-10.04  sec  3.25 MBytes   710 Mbits/sec  0.009 ms  1077/4486 (24%) 
 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval   Transfer Bandwidth   JitterLost/Total 
Datagrams
[  5]   0.00-10.04  sec  0.00 Bytes  0.00 bits/sec  0.009 ms  278644/1163363 
(24%)  


IIUC, sender (desktop system) sends datagrams as fast as possible.
Receiver (tango board) drops around 24% of all datagrams.
I think this invalidates the theory that exhausting RX descriptors
wedges RX DMA.

Regards.

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Mason 
On 02/08/2017 19:31, Mason wrote:

> # iperf3 -c 172.27.64.45 -u -b 950M
> Connecting to host 172.27.64.45, port 5201
> [  4] local 172.27.64.1 port 55533 connected to 172.27.64.45 port 5201
> [ ID] Interval   Transfer Bandwidth   Total Datagrams
> [  4]   0.00-1.00   sec   102 MBytes   858 Mbits/sec  13091  
> [  4]   1.00-2.00   sec   114 MBytes   953 Mbits/sec  14541  

114 MB in 14541 packets => 7840 bytes per packet
Is iperf3 sending jumbo frames??
In the nb8800 driver, RX_BUF_SIZE is only 1552,
how would it deal with jumbo frames... truncate?

> # iperf3 -c 172.27.64.45 -u -b 950M -l 800
> Connecting to host 172.27.64.45, port 5201
> [  4] local 172.27.64.1 port 35197 connected to 172.27.64.45 port 5201
> [ ID] Interval   Transfer Bandwidth   Total Datagrams
> [  4]   0.00-1.00   sec  90.6 MBytes   760 Mbits/sec  118724  
> [  4]   1.00-2.00   sec   107 MBytes   894 Mbits/sec  139718  

107 MB in 139718 packets => 766 bytes per packet
800 - 8 (UDP) - 20 (IPv4) = 772 bytes per packet
I suppose that's close enough...
772 * 139718 = 107.86 MB

I need to run the test slightly slower, to prevent
packet loss at the sender.

Perhaps -b 0 or -b 800M

Regards.

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Mason 
On 02/08/2017 18:10, Måns Rullgård wrote:

> ping -f is limited to 100 packets per second.
> Use something like iperf in UDP mode instead.

CLIENT: DESKTOP
# iperf3 -c 172.27.64.45 -u -b 950M
Connecting to host 172.27.64.45, port 5201
[  4] local 172.27.64.1 port 55533 connected to 172.27.64.45 port 5201
[ ID] Interval   Transfer Bandwidth   Total Datagrams
[  4]   0.00-1.00   sec   102 MBytes   858 Mbits/sec  13091  
[  4]   1.00-2.00   sec   114 MBytes   953 Mbits/sec  14541  
[  4]   2.00-3.00   sec   114 MBytes   953 Mbits/sec  14542  
[  4]   3.00-4.00   sec   114 MBytes   953 Mbits/sec  14541  
[  4]   4.00-5.00   sec   114 MBytes   953 Mbits/sec  14542  
[  4]   5.00-6.00   sec   114 MBytes   953 Mbits/sec  14541  
[  4]   6.00-7.00   sec   114 MBytes   953 Mbits/sec  14535  
[  4]   7.00-8.00   sec   114 MBytes   953 Mbits/sec  14536  
[  4]   8.00-9.00   sec   114 MBytes   953 Mbits/sec  14543  
[  4]   9.00-10.00  sec   114 MBytes   953 Mbits/sec  14541  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval   Transfer Bandwidth   JitterLost/Total 
Datagrams
[  4]   0.00-10.00  sec  1.10 GBytes   943 Mbits/sec  0.247 ms  132176/143788 
(92%)  
[  4] Sent 143788 datagrams

iperf Done.


SERVER: TANGO BOARD
# iperf3 -s
Accepted connection from 172.27.64.1, port 44995
[  5] local 172.27.64.45 port 5201 connected to 172.27.64.1 port 55533
[ ID] Interval   Transfer Bandwidth   JitterLost/Total 
Datagrams
[  5]   0.00-1.00   sec  7.90 MBytes  66.2 Mbits/sec  0.218 ms  11365/12376 
(92%)  
[  5]   1.00-2.00   sec  9.13 MBytes  76.6 Mbits/sec  0.230 ms  13381/14550 
(92%)  
[  5]   2.00-3.00   sec  9.12 MBytes  76.5 Mbits/sec  0.290 ms  13372/14540 
(92%)  
[  5]   3.00-4.00   sec  9.11 MBytes  76.4 Mbits/sec  0.292 ms  13369/14535 
(92%)  
[  5]   4.00-5.00   sec  9.18 MBytes  77.0 Mbits/sec  0.178 ms  13374/14549 
(92%)  
[  5]   5.00-6.00   sec  9.10 MBytes  76.3 Mbits/sec  0.228 ms  13367/14532 
(92%)  
[  5]   6.00-7.00   sec  9.26 MBytes  77.7 Mbits/sec  0.607 ms  13356/14541 
(92%)  
[  5]   7.00-8.00   sec  9.23 MBytes  77.4 Mbits/sec  0.507 ms  13364/14545 
(92%)  
[  5]   8.00-9.00   sec  9.20 MBytes  77.1 Mbits/sec  0.215 ms  13351/14528 
(92%)  
[  5]   9.00-10.00  sec  9.16 MBytes  76.9 Mbits/sec  0.188 ms  13356/14529 
(92%)  
[  5]  10.00-10.04  sec   336 KBytes  72.2 Mbits/sec  0.247 ms  521/563 (93%)  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval   Transfer Bandwidth   JitterLost/Total 
Datagrams
[  5]   0.00-10.04  sec  0.00 Bytes  0.00 bits/sec  0.247 ms  132176/143788 
(92%)  


There is some packet loss, but the board doesn't lose connectivity.

With smaller packets...

# iperf3 -c 172.27.64.45 -u -b 950M -l 800
Connecting to host 172.27.64.45, port 5201
[  4] local 172.27.64.1 port 35197 connected to 172.27.64.45 port 5201
[ ID] Interval   Transfer Bandwidth   Total Datagrams
[  4]   0.00-1.00   sec  90.6 MBytes   760 Mbits/sec  118724  
[  4]   1.00-2.00   sec   107 MBytes   894 Mbits/sec  139718  
[  4]   2.00-3.00   sec   106 MBytes   889 Mbits/sec  138918  
[  4]   3.00-4.00   sec   107 MBytes   895 Mbits/sec  139768  
[  4]   4.00-5.00   sec   106 MBytes   891 Mbits/sec  139275  
[  4]   5.00-6.00   sec   107 MBytes   895 Mbits/sec  139862  
[  4]   6.00-7.00   sec   107 MBytes   895 Mbits/sec  139825  
[  4]   7.00-8.00   sec   107 MBytes   895 Mbits/sec  139775  
[  4]   8.00-9.00   sec   107 MBytes   895 Mbits/sec  139849  
[  4]   9.00-10.00  sec   107 MBytes   895 Mbits/sec  139835  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval   Transfer Bandwidth   JitterLost/Total 
Datagrams
[  4]   0.00-10.00  sec  1.02 GBytes   880 Mbits/sec  0.009 ms  564117/1375520 
(41%)  
[  4] Sent 1375520 datagrams

iperf Done.


Accepted connection from 172.27.64.1, port 46151
[  5] local 172.27.64.45 port 5201 connected to 172.27.64.1 port 35197
[ ID] Interval   Transfer Bandwidth   JitterLost/Total 
Datagrams
[  5]   0.00-1.00   sec  60.8 MBytes   510 Mbits/sec  0.004 ms  33508/113252 
(30%)  
iperf3: OUT OF ORDER - incoming packet = 147146 and received packet = 0 AND SP 
= 147497
iperf3: OUT OF ORDER - incoming packet = 146128 and received packet = 0 AND SP 
= 147690
iperf3: OUT OF ORDER - incoming packet = 146067 and received packet = 0 AND SP 
= 147863
iperf3: OUT OF ORDER - incoming packet = 147242 and received packet = 0 AND SP 
= 148039
iperf3: OUT OF ORDER - incoming packet = 163837 and received packet = 0 AND SP 
= 164363
[  5]   1.00-2.00   sec  61.0 MBytes   511 Mbits/sec  0.198 ms  59635/139649 
(43%)  
iperf3: OUT OF ORDER - incoming packet = 286437 and received packet = 0 AND SP 
= 287226
iperf3: OUT OF ORDER - incoming packet = 302990 and received packet = 0 AND SP 
= 305710
[  5]   2.00-3.00   sec  61.5 MBytes   517 Mbits/sec  0.005 ms  58369/138944 
(42%)  
iperf3: OUT OF ORDER - incoming packet

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Måns Rullgård 
Mason  writes:

> On 02/08/2017 18:10, Måns Rullgård wrote:
>
>> Mason writes:
>> 
>>> On 02/08/2017 17:56, Måns Rullgård wrote:
>>>
 What does the tango5 do if you flood it with packets faster than the
 kernel can keep up with?  That would make it hit the end of the rx
 chain, which is apparently what makes it miserable with the current dma
 stop code.
>>>
>>> The simplest way to test this would be sending tiny packets
>>> as fast as possible, right? So ping -f on a GigE link should
>>> fit the bill?
>> 
>> ping -f is limited to 100 packets per second.  Use something like iperf
>> in UDP mode instead.
>
> ping -f can go 100 times faster than 100 pps:
>
> # ping -f -q -c 15 -s 300 172.27.64.45
> PING 172.27.64.45 (172.27.64.45) 300(328) bytes of data.
>
> --- 172.27.64.45 ping statistics ---
> 15 packets transmitted, 15 received, 0% packet loss, time 15035ms
> rtt min/avg/max/mdev = 0.065/0.084/0.537/0.014 ms, ipg/ewma 0.100/0.087 ms
>
> 150,000 packets in 15 seconds = 10,000 pps
>
> (172.27.64.45 is the tango5 board)
>
> Ergo, dealing with 10,000 packets per second does not hose RX.

ping -f goes as fast as the other end replies or 100 per second,
whichever is higher, so says the man page.

-- 
Måns Rullgård

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Mason 
On 02/08/2017 18:10, Måns Rullgård wrote:

> Mason writes:
> 
>> On 02/08/2017 17:56, Måns Rullgård wrote:
>>
>>> What does the tango5 do if you flood it with packets faster than the
>>> kernel can keep up with?  That would make it hit the end of the rx
>>> chain, which is apparently what makes it miserable with the current dma
>>> stop code.
>>
>> The simplest way to test this would be sending tiny packets
>> as fast as possible, right? So ping -f on a GigE link should
>> fit the bill?
> 
> ping -f is limited to 100 packets per second.  Use something like iperf
> in UDP mode instead.

ping -f can go 100 times faster than 100 pps:

# ping -f -q -c 15 -s 300 172.27.64.45
PING 172.27.64.45 (172.27.64.45) 300(328) bytes of data.

--- 172.27.64.45 ping statistics ---
15 packets transmitted, 15 received, 0% packet loss, time 15035ms
rtt min/avg/max/mdev = 0.065/0.084/0.537/0.014 ms, ipg/ewma 0.100/0.087 ms


150,000 packets in 15 seconds = 10,000 pps

(172.27.64.45 is the tango5 board)

Ergo, dealing with 10,000 packets per second does not hose RX.

Regards.

RE: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread David Laight 
From: Måns Rullgård
> Sent: 02 August 2017 17:10
...
> ping -f is limited to 100 packets per second.  Use something like iperf
> in UDP mode instead.

Or break a MAC driver so it just saturates the network.

You might actually need bursts of traffic - otherwise the receiver
will be continuously out of rx buffers and no dma will be active.

David

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Måns Rullgård 
Mason  writes:

> On 02/08/2017 17:56, Måns Rullgård wrote:
>
>> Mason writes:
>> 
>>> From my perspective, the older method does not work on newer chips :-)
>> 
>> It does work on tango4.
>
> Agreed.
>
>> What does the tango5 do if you flood it with packets faster than the
>> kernel can keep up with?  That would make it hit the end of the rx
>> chain, which is apparently what makes it miserable with the current dma
>> stop code.
>
> The simplest way to test this would be sending tiny packets
> as fast as possible, right? So ping -f on a GigE link should
> fit the bill?

ping -f is limited to 100 packets per second.  Use something like iperf
in UDP mode instead.

-- 
Måns Rullgård

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Mason 
On 02/08/2017 17:56, Måns Rullgård wrote:

> Mason writes:
> 
>> From my perspective, the older method does not work on newer chips :-)
> 
> It does work on tango4.

Agreed.

> What does the tango5 do if you flood it with packets faster than the
> kernel can keep up with?  That would make it hit the end of the rx
> chain, which is apparently what makes it miserable with the current dma
> stop code.

The simplest way to test this would be sending tiny packets
as fast as possible, right? So ping -f on a GigE link should
fit the bill?

Regards.

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Måns Rullgård 
Mason  writes:

> On 02/08/2017 17:36, Måns Rullgård wrote:
>
>> Mason wrote:
>> 
>>> Looking at the tango-specific integration, I note this nugget:
>>>
>>> 1.5.4 Stopping & Starting the DMA
>>>
>>> This feature has been added to allow the software to stop and start
>>> the DMA without any issues.
>>>
>>> Procedure:
>>> 1- STOP:
>>> 2- Stop RX core;
>>> 3- Set OWN bit of all descriptor of the chain to 1;
>>> 4- Stop DMA by writing dma_stop bit to 1 in  RX_DMA_Stop register
>>> 5- Wait around 100 clock cycles.
>>>
>>> The pending packets are held until the system will re-start.
>>>
>>> RE-START:
>>> 1- Clear dma_stop bit (note that if at the time of stopping the DMA,
>>> the next packet in the FIFO was an UDP packet, when clearing dma_stop,
>>> this packet will directly start being written in the DRAM since UDP
>>> packets are not controlled by the descriptor mechanism);
>>> 2- Program a new chain of descriptor;
>>> 3- Re-enable DMA (rx_ctrl register)
>>>
>>> rx_dma_stop:
>>> Software control to stop the Rx DMA.
>>> A write to this bit with "1" will gracefully stop the Rx DMA by after
>>> transferring the current packet. If more packets are pending they will
>>> be held until the software clears this bit.
>>>
>>> Hmmm, what do you think? This looks promising...
>> 
>> This is only available in the more recent Sigma versions.  Although it
>> is nicer, I didn't think it was worth the trouble to support both
>> methods since the older method should work on all chips.
>
> Well, from my perspective, the older method does not work on newer
> chips :-)

It does work on tango4.

What does the tango5 do if you flood it with packets faster than the
kernel can keep up with?  That would make it hit the end of the rx
chain, which is apparently what makes it miserable with the current dma
stop code.

-- 
Måns Rullgård

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Mason 
On 02/08/2017 17:36, Måns Rullgård wrote:

> Mason wrote:
> 
>> Looking at the tango-specific integration, I note this nugget:
>>
>> 1.5.4 Stopping & Starting the DMA
>>
>> This feature has been added to allow the software to stop and start
>> the DMA without any issues.
>>
>> Procedure:
>> 1- STOP:
>> 2- Stop RX core;
>> 3- Set OWN bit of all descriptor of the chain to 1;
>> 4- Stop DMA by writing dma_stop bit to 1 in  RX_DMA_Stop register
>> 5- Wait around 100 clock cycles.
>>
>> The pending packets are held until the system will re-start.
>>
>> RE-START:
>> 1- Clear dma_stop bit (note that if at the time of stopping the DMA,
>> the next packet in the FIFO was an UDP packet, when clearing dma_stop,
>> this packet will directly start being written in the DRAM since UDP
>> packets are not controlled by the descriptor mechanism);
>> 2- Program a new chain of descriptor;
>> 3- Re-enable DMA (rx_ctrl register)
>>
>> rx_dma_stop:
>> Software control to stop the Rx DMA.
>> A write to this bit with "1" will gracefully stop the Rx DMA by after
>> transferring the current packet. If more packets are pending they will
>> be held until the software clears this bit.
>>
>> Hmmm, what do you think? This looks promising...
> 
> This is only available in the more recent Sigma versions.  Although it
> is nicer, I didn't think it was worth the trouble to support both
> methods since the older method should work on all chips.

Well, from my perspective, the older method does not
work on newer chips :-)

The "new" method is available on the SMP8670 onward
(released Jan 2011). I might try implementing it for
"sigma,smp8734-ethernet" boards.

Regards.

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Måns Rullgård 
Mason  writes:

> On 02/08/2017 16:41, Mason wrote:
>
>> On 01/08/2017 18:32, Mason wrote:
>> 
>>> I need suspend/resume support in the nb8800 driver.
>>> On tango platforms, suspend loses all context (MMIO registers).
>>> To make the task easy, we just close the device on suspend,
>>> and open it again on resume. This requires properly resetting
>>> the HW on resume.
>>>
>>> Patch 1 moves all the HW init to nb8800_init()
>>> Patch 2 adds suspend/resume support
>> 
>> I have now confirmed that the "flow control" issue I reported
>> in another thread has nothing to do with flow control per se.
>> 
>> The problem is that nb8800_pause_config() calls nb8800_dma_stop()
>> and when it does, RX is borked.
>> 
>> On a GigE switch:
>> [   21.444268] ENTER nb8800_pause_config
>> [   21.448604] rxcr=06100a8f pause_rx=1 pause_tx=0 pause=1 asym_pause=1
>> [   21.455020] nb8800 26000.ethernet eth0: Link is Up - 1Gbps/Full - flow 
>> control rx/tx
>> 
>> In this case, pause_tx and RCR_FL match, so we skip the
>> silly dance.
>
> The documentation states:
>
> Receive Channel Control Register
> Description:
> The Receive Channel Control Register holds channel enable, mode, endian,
> DMA control, and interrupt control information.  This register can only
> be written when the Receive DMA Channel is idle - the Enable bit in it is "0".
> Register Number:  0x200
> Access:  read/write
> Reset Value: le = AMBA_LE; all other bits = 0x0
> Fields:
>
> fl:  Flow control enable.  "1" indicates flow control is enabled. 
> When flow control is enabled and a Receive FIFO overrun occurs,
> the Ethernet 10/100/1000 Subsystem will send PAUSE frames if in
> full duplex mode.  This continues until the Receive FIFO is emptied.
>
> en:  Receive DMA Channel enable.  "1" indicates that the Receive
> DMA Channel is being configured from a descriptor, or that the DMA
> operation is in progress.  The Receive DMA Channel is idle when
> this enable bit is "0".  Software sets this bit to "1" to start the
> configuration from a descriptor and DMA operation.  When the DMA
> operation is finished, this bit is automatically reset to "0".

Here's the problem.  Once started, there is no way to forcibly stop rx
dma.  It keeps going until it hits a descriptor with the end of chain
flag set.

>> Receive DMA Channel Disabling
>>
>> When the entire receive frame has been read from the Receive FIFO and
>> sent over the AMBA bus, the DMA operation ends, and the Receive DMA
>> Channel is automatically disabled.  To do this, hardware resets the
>> Enable bit in the Receive Channel Control Register to "0" after the
>> last data has been read from the Receive FIFO and sent over the AMBA
>> bus.
>> 
>> When operating in descriptor mode, upon completion of a receive frame
>> DMA operation, if the descriptor chain has not ended when a receive
>> frame DMA operation completes, the next receive frame DMA operation
>> begins.  The last descriptor in a descriptor chain is indicated by
>> having its End Of Chain- EOC, flag set to "1".  If this EOC flag is
>> "0", to begin the next receive frame DMA operation, the next
>> descriptor is automatically retrieved and used to configure the
>> Receive DMA Channel.  The Receive DMA Channel is then automatically
>> re-enabled and the next receive frame DMA operation begins.
>> 
>> In descriptor mode, an AMBA bus error can occur when reading receive
>> descriptor data.  If this happens, receive descriptor processing ends
>> and the Receive DMA Channel is turned off.  The Descriptor Error bit
>> in the Receive Status Register is set to "1".
>
> Hmmm, I guess this is what Maxime/Mans did...
>
> Looking at the tango-specific integration, I note this nugget:
>
> 1.5.4 Stopping & Starting the DMA
>
> This feature has been added to allow the software to stop and start
> the DMA without any issues.
>
> Procedure:
> 1- STOP:
> 2- Stop RX core;
> 3- Set OWN bit of all descriptor of the chain to 1;
> 4- Stop DMA by writing dma_stop bit to 1 in  RX_DMA_Stop register
> 5- Wait around 100 clock cycles.
>
> The pending packets are held until the system will re-start.
>
> RE-START:
> 1- Clear dma_stop bit (note that if at the time of stopping the DMA,
> the next packet in the FIFO was an UDP packet, when clearing dma_stop,
> this packet will directly start being written in the DRAM since UDP
> packets are not controlled by the descriptor mechanism);
> 2- Program a new chain of descriptor;
> 3- Re-enable DMA (rx_ctrl register)
>
> rx_dma_stop:
> Software control to stop the Rx DMA.
> A write to this bit with “1” will gracefully stop the Rx DMA by after
> transferring the current packet. If more packets are pending they will
> be held until the software clears this bit.
>
> Hmmm, what do you think? This looks promising...

This is only available in the more recent Sigma versions.  Although it
is nicer, I didn't think it was worth the trouble to support both
methods since the older method should work on all chips.

-- 
Måns Rullgård

Re: [RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-02 Thread Mason 
On 01/08/2017 18:32, Mason wrote:

> I need suspend/resume support in the nb8800 driver.
> On tango platforms, suspend loses all context (MMIO registers).
> To make the task easy, we just close the device on suspend,
> and open it again on resume. This requires properly resetting
> the HW on resume.
> 
> Patch 1 moves all the HW init to nb8800_init()
> Patch 2 adds suspend/resume support

I have now confirmed that the "flow control" issue I reported
in another thread has nothing to do with flow control per se.

The problem is that nb8800_pause_config() calls nb8800_dma_stop()
and when it does, RX is borked.

On a GigE switch:
[   21.444268] ENTER nb8800_pause_config
[   21.448604] rxcr=06100a8f pause_rx=1 pause_tx=0 pause=1 asym_pause=1
[   21.455020] nb8800 26000.ethernet eth0: Link is Up - 1Gbps/Full - flow 
control rx/tx

In this case, pause_tx and RCR_FL match, so we skip the
silly dance.

On a FastE switch:
[   11.611613] ENTER nb8800_pause_config
[   11.615942] rxcr=06100a8f pause_rx=1 pause_tx=1 pause=1 asym_pause=0
[   11.825535] nb8800 26000.ethernet eth0: Link is Up - 100Mbps/Full - flow 
control rx/tx

In this case pause_tx and RCR_FL don't match, so we hit
nb8800_dma_stop() and the HW block becomes deaf.

In conclusion, two issues I've been discussing:
A) RX wedged after ndo_close
B) flow control breaks on FastE switches
are in fact two symptoms of the same root issue.

Regards.

[RFC PATCH v2 0/2] nb8800 suspend/resume support

2017-08-01 Thread Mason 
Hello,

I need suspend/resume support in the nb8800 driver.
On tango platforms, suspend loses all context (MMIO registers).
To make the task easy, we just close the device on suspend,
and open it again on resume. This requires properly resetting
the HW on resume.

Patch 1 moves all the HW init to nb8800_init()
Patch 2 adds suspend/resume support

Regards.