Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Tue, Feb 23, 2016 at 08:35:04PM +0100, hiro wrote:
> downloads.kergis.com is some http server operated by OVH.
> 
> it would be easier to post a pcap from a transfer where you control
> both sides and possibly enable debugging of window sizes, timeouts,
> packet loss, etc. in tcp.c like erik did.

Both sides won't be possible (at least for me). What I plan to do is
tcpdump/snoopy the interface under NetBSD and Plan9 (from the very same 
workstation), when downloading the "test" file, and look, separately
the two pcap and then compare the processing (I might even go as far as
compare the plan9 implementation with the 4.4BSD-lite).

But for the moment, I have taken down from the shelf Stevens TCP/IP
illustrated, since my knowledge needs both to be refreshed and to be
enhanced... (This means that it will take some time.)

> 
> On 2/23/16, erik quanstrom  wrote:
> > I saw this.  I'm not looking at the pcap file
> >
> > On Feb 23, 2016 10:38 AM, hiro <23h...@gmail.com> wrote:
> >>
> >> i didn't see any out-of-window rx in the pcap. did i look the wrong way?
> >>
> >>
> >

-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[hiro]

downloads.kergis.com is some http server operated by OVH.

it would be easier to post a pcap from a transfer where you control
both sides and possibly enable debugging of window sizes, timeouts,
packet loss, etc. in tcp.c like erik did.

On 2/23/16, erik quanstrom  wrote:
> I saw this.  I'm not looking at the pcap file
>
> On Feb 23, 2016 10:38 AM, hiro <23h...@gmail.com> wrote:
>>
>> i didn't see any out-of-window rx in the pcap. did i look the wrong way?
>>
>>
>

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

I saw this.  I'm not looking at the pcap file

On Feb 23, 2016 10:38 AM, hiro <23h...@gmail.com> wrote:
>
> i didn't see any out-of-window rx in the pcap. did i look the wrong way? 
>
>

Re: [9fans] rtl8169 gbe slow

[hiro]

i didn't see any out-of-window rx in the pcap. did i look the wrong way?

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

On Tue Feb 23 09:52:28 PST 2016, 23h...@gmail.com wrote:
> which machine is in the west coast? the one your tracing on? is that
> hget or a web server on plan9? is this the same test as posted by
> david?
> 
> where do you see out-of-window rxes? what does that even mean?

this is from the same http://downloads.kergis.com/kertex/kertex_bundle.tar 
download as before.

an out-of-window rx is a rx packet that has no byte in the current window.
this can happen for legitimate reasons, but generally it does not.

- erik

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

On Tue Feb 23 09:55:58 PST 2016, kennylevin...@gmail.com wrote:
> Just in case you want a another point of reference to eliminate weirdness 
> with the specific box: http://de.kl.wtf/f/10mburandom
> 
> Linode Arch Linux box in Frankfurt, serving you with a pretty standard usage 
> of Go’s http server. Should count as a “stock linux box with a non-weird HTTP 
> server”.
> 
> if you add an “s”, you get TLS. If you add a “0”, you get 100MB. If you 
> remove a country code, it goes through Cloudflare. You’ll have to guess where 
> to insert and remove those characters yourself, though!

10x better performance than kergis.com

0.094u 0.641s 48.016rhget -v http://de.kl.wtf/f/10mburandom

from the east coast with a non-residential uplink i get this very odd little 
wiggle

 ; time hget -v http://de.kl.wtf/f/10mburandom>/dev/null
1024 10485760
908553 10485760
2502873 10485760
2517473 10485760

unfortunately i don't have time to do more than putter today.

- erik

Re: [9fans] rtl8169 gbe slow

[Kenny Lasse Hoff Levinsen]

Just in case you want a another point of reference to eliminate weirdness with 
the specific box: http://de.kl.wtf/f/10mburandom

Linode Arch Linux box in Frankfurt, serving you with a pretty standard usage of 
Go’s http server. Should count as a “stock linux box with a non-weird HTTP 
server”.

if you add an “s”, you get TLS. If you add a “0”, you get 100MB. If you remove 
a country code, it goes through Cloudflare. You’ll have to guess where to 
insert and remove those characters yourself, though!

Best regards,
Kenny Levinsen

> On 23 Feb 2016, at 18:38, erik quanstrom  wrote:
> 
> On Tue Feb 23 09:25:53 PST 2016, 23h...@gmail.com wrote:
>> in the long run the rwin seems much higher (65535) than the number of
>> bytes in flight (less than 3x1500 bytes).
>> 
>> i just noticed that the minimum latency numbers seem way low. many
>> latency samples appear at around 40ms and 100ms, but there's also
>> outliers? below 1ms. i don't get how this pcap got produced. perhaps
>> wireshark is also interpreting it wrong, or timestamps are broken...
> 
>>> 26/status:Established qin 0 qout 0 rq 0.0 srtt 1256 mdev 628 sst 65535 cwin
>>> 4517 swin 5808>>0 rwin 65535>>4 qscale 0 timer.start 10 timer.count 10
>>> rerecv 0 katimer.start 2400 katimer.count 2400
>> 
>> where did you run this?
> 
> machine on the us west coast.  clearly we are prevoking some sort of odd 
> behavior
> in this machine, but it's not clear to me what we're doing.
> 
> the only clue we have is the out-of-window rxes.  perhaps the sender is 
> scaling.
> 
> - erik
> 

Re: [9fans] rtl8169 gbe slow

[hiro]

no, i don't see any routing info in your output.

On 2/23/16, lu...@proxima.alt.za  wrote:
>> i don't get how this pcap got produced. perhaps
>> wireshark is also interpreting it wrong, or timestamps are broken...
>
> I wonder if there isn't some route flapping involved here.  Is it
> possible that the hop count is not stable?  Or the link gets
> saturated?
>
> Lucio.
>
>
>

Re: [9fans] rtl8169 gbe slow

[hiro]

which machine is in the west coast? the one your tracing on? is that
hget or a web server on plan9? is this the same test as posted by
david?

where do you see out-of-window rxes? what does that even mean?

On 2/23/16, erik quanstrom  wrote:
> On Tue Feb 23 09:25:53 PST 2016, 23h...@gmail.com wrote:
>> in the long run the rwin seems much higher (65535) than the number of
>> bytes in flight (less than 3x1500 bytes).
>>
>> i just noticed that the minimum latency numbers seem way low. many
>> latency samples appear at around 40ms and 100ms, but there's also
>> outliers? below 1ms. i don't get how this pcap got produced. perhaps
>> wireshark is also interpreting it wrong, or timestamps are broken...
>
>> > 26/status:Established qin 0 qout 0 rq 0.0 srtt 1256 mdev 628 sst 65535
>> > cwin
>> > 4517 swin 5808>>0 rwin 65535>>4 qscale 0 timer.start 10 timer.count 10
>> > rerecv 0 katimer.start 2400 katimer.count 2400
>>
>> where did you run this?
>
> machine on the us west coast.  clearly we are prevoking some sort of odd
> behavior
> in this machine, but it's not clear to me what we're doing.
>
> the only clue we have is the out-of-window rxes.  perhaps the sender is
> scaling.
>
> - erik
>
>

Re: [9fans] rtl8169 gbe slow

[lucio]

> i don't get how this pcap got produced. perhaps
> wireshark is also interpreting it wrong, or timestamps are broken...

I wonder if there isn't some route flapping involved here.  Is it
possible that the hop count is not stable?  Or the link gets
saturated?

Lucio.

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

On Tue Feb 23 09:25:53 PST 2016, 23h...@gmail.com wrote:
> in the long run the rwin seems much higher (65535) than the number of
> bytes in flight (less than 3x1500 bytes).
>
> i just noticed that the minimum latency numbers seem way low. many
> latency samples appear at around 40ms and 100ms, but there's also
> outliers? below 1ms. i don't get how this pcap got produced. perhaps
> wireshark is also interpreting it wrong, or timestamps are broken...

> > 26/status:Established qin 0 qout 0 rq 0.0 srtt 1256 mdev 628 sst 65535 cwin
> > 4517 swin 5808>>0 rwin 65535>>4 qscale 0 timer.start 10 timer.count 10
> > rerecv 0 katimer.start 2400 katimer.count 2400
> 
> where did you run this?

machine on the us west coast.  clearly we are prevoking some sort of odd 
behavior
in this machine, but it's not clear to me what we're doing.

the only clue we have is the out-of-window rxes.  perhaps the sender is scaling.

- erik

Re: [9fans] rtl8169 gbe slow

[hiro]

> 26/status:Established qin 0 qout 0 rq 0.0 srtt 1256 mdev 628 sst 65535 cwin
> 4517 swin 5808>>0 rwin 65535>>4 qscale 0 timer.start 10 timer.count 10
> rerecv 0 katimer.start 2400 katimer.count 2400

where did you run this?

Re: [9fans] rtl8169 gbe slow

[hiro]

in the long run the rwin seems much higher (65535) than the number of
bytes in flight (less than 3x1500 bytes).

i just noticed that the minimum latency numbers seem way low. many
latency samples appear at around 40ms and 100ms, but there's also
outliers? below 1ms. i don't get how this pcap got produced. perhaps
wireshark is also interpreting it wrong, or timestamps are broken...

On 2/23/16, erik quanstrom  wrote:
> On Tue Feb 23 04:39:42 PST 2016, 23h...@gmail.com wrote:
>> in any case we seem limited by congestion window, not rwin.
>
> can you explain?
>
> - erik
>
>

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

On Tue Feb 23 04:49:55 PST 2016, 23h...@gmail.com wrote:
> i just realized the http *response* packets all have their rwin set to
> 5808 only, while the other side has the former described behavior
> hovering around 65535.
> perhaps the http server does no window scaling?!

26/status:Established qin 0 qout 0 rq 0.0 srtt 1256 mdev 628 sst 65535 cwin 
4517 swin 5808>>0 rwin 65535>>4 qscale 0 timer.start 10 timer.count 10 rerecv 0 
katimer.start 2400 katimer.count 2400

yes.  but as we're sending nothing, we can ack into an empty window.

- erik

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

On Tue Feb 23 04:32:50 PST 2016, 23h...@gmail.com wrote:
> erik: I don't think nowadays we need to limit rwin unless we
> artificially want to reduce the bandwidth (e.g. in my torrent program,
> or an rsync that's running in the background and shouldn't use up the
> whole bandwidth of the slow DSL uplink).
> in the past it seems to have been used to combat memory limitations of
> the receiver. but nowadays we have enough memory, so in normal
> operation rwin should always be fully open.

we're not.  we're advertizing 65535<<4, as noted earlier.

- erik

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

On Tue Feb 23 04:39:42 PST 2016, 23h...@gmail.com wrote:
> in any case we seem limited by congestion window, not rwin.

can you explain?

- erik

Re: [9fans] rtl8169 gbe slow

[hiro]

and obviously a blocking pipe would be a good reason to reduce rwin.
the other keyword used often is "flow control".
redirecting hget output into /dev/null should be fast enough though,
so again, this doesn't matter in our case.

Re: [9fans] rtl8169 gbe slow

[hiro]

in any case we seem limited by congestion window, not rwin.

Re: [9fans] rtl8169 gbe slow

[hiro]

Can you please tell more about your setup here?
Is this capture taken only on the client?

in a seq number graph you can see that (disregarding the beginning
where it obviously first has to catch up some speed) until 5.73s the
throughput is quite ok, but then suddenly goes down to approximately
half without recovery till the end: http://176.31.253.70/tcp/

The receive window from hget seems to be *nearly* fully open, but is
changing (slightly) all the time. This I didn't expect, but I doubt
it's to be blamed for the bandwidth degradation you're seeing.

On 2/22/16, David du Colombier <0in...@gmail.com> wrote:
> I've uploaded a pcap trace for future reference.
>
> http://9legacy.org/download/pcap/kergis_plan9.pcap
>
> --
> David du Colombier
>
>

Re: [9fans] rtl8169 gbe slow

[Skip Tavakkolian]

is the local throughput what you expect to get?  if yes, can you enable
tls? i'm wondering if http traffic is being intercepted in some way
(caches).

On Mon, Feb 22, 2016 at 9:13 AM David du Colombier <0in...@gmail.com> wrote:

> I've uploaded a pcap trace for future reference.
>
> http://9legacy.org/download/pcap/kergis_plan9.pcap
>
> --
> David du Colombier
>
>

Re: [9fans] rtl8169 gbe slow

[David du Colombier]

I've uploaded a pcap trace for future reference.

http://9legacy.org/download/pcap/kergis_plan9.pcap

-- 
David du Colombier

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 04:45:12PM +, r...@hemiola.co.uk wrote:
> >Does plan9 under lguest actually use the linux
> >hardware services? Is plan9 under lguest using "its" implementation
> >except for the low level device driving i.e. the ethernet provided
> >by the Linux host?
> 
> Yes. The lguest plan9 instance has a virtio ethernet driver,
> which is a 'wire' to a tap interface on the host. Packets are routed
> at the ip level from the tap to the linux ethernet interface by
> the linux kernel in the usual way. I'm not sure why plan9 is half
> the speed in this situation, but I feel it might be a red herring,
> and that the combination of lguest/plan9 isn't terribly efficient
> at minimising the context switching that happens when packets are sent
> and received.
> 

If I'm not mistaken, on the server log, even under lguest the string is
still "Plan9/hget" so this seems to rule this out. And if the
performance (minus emulation/switching overhead) is better using,
actually, Linux TCP/IP implementation for the real connection, it will
show that this is the Plan9 implementation that has, under some
circumstances (perhaps with the size of the packets sent by some server)
an unfelicity.
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[Kenny Lasse Hoff Levinsen]

Lucio: Of course I'll make an issue, I only just noticed it and traced it to 
goexitsall. Don't worry. :)

Richard: Thanks, I'll try that. The trace of goexitsall still contain FP 
register access (XORPS and duffzero which contains MOVUPS), but maybe that 
doesn't matter if the race is fixed?

I didn't intend to hijack the thread, just pointed out that it is indeed not 
hget or webfs at fault.

Best regards,
Kenny Levinsen

On 22. feb. 2016, at 17.40, lu...@proxima.alt.za wrote:

>> Fun sidenote: more floating point code in note handlers, this time 
>> duffzero when calling os.Exit. *sigh*.
> 
> Can you please submit it as an issue, so we at least know it's there?
> 
> Lucio.
> 
> 

Re: [9fans] rtl8169 gbe slow

[rod]

>Does plan9 under lguest actually use the linux
>hardware services? Is plan9 under lguest using "its" implementation
>except for the low level device driving i.e. the ethernet provided
>by the Linux host?

Yes. The lguest plan9 instance has a virtio ethernet driver,
which is a 'wire' to a tap interface on the host. Packets are routed
at the ip level from the tap to the linux ethernet interface by
the linux kernel in the usual way. I'm not sure why plan9 is half
the speed in this situation, but I feel it might be a red herring,
and that the combination of lguest/plan9 isn't terribly efficient
at minimising the context switching that happens when packets are sent
and received.

-rod

Re: [9fans] rtl8169 gbe slow

[lucio]

> Fun sidenote: more floating point code in note handlers, this time 
> duffzero when calling os.Exit. *sigh*.

Can you please submit it as an issue, so we at least know it's there?

Lucio.

Re: [9fans] rtl8169 gbe slow

[Kenny Lasse Hoff Levinsen]

Go http client + go http server on a vps (14.2ms ping) show same issues: 182s 
to download 100MB from remote using a 9front VM, 7s local, vs. 30s and 1s for 
the linux host.

So yeah, hget/webfs has nothing to do with it.

Fun sidenote: more floating point code in note handlers, this time duffzero 
when calling os.Exit. *sigh*.

Best regards,
Kenny Levinsen

On 22. feb. 2016, at 16.46, David du Colombier <0in...@gmail.com> wrote:

>> No. Is it possible to change the string for hget?
> 
> You can change the string in /sys/src/cmd/hget.c.
> But this issue is not related to hget, since it doesn't happen when
> using hget on Linux or 9vx.
> 
> It seems related to the Plan 9 TCP stack, or how the network
> infrastructure behaves with it.
> 
> -- 
> David du Colombier
> 

Re: [9fans] rtl8169 gbe slow

[David du Colombier]

> No. Is it possible to change the string for hget?

You can change the string in /sys/src/cmd/hget.c.
But this issue is not related to hget, since it doesn't happen when
using hget on Linux or 9vx.

It seems related to the Plan 9 TCP stack, or how the network
infrastructure behaves with it.

-- 
David du Colombier

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

On Mon Feb 22 07:29:56 PST 2016, tlaro...@polynum.com wrote:
> On Mon, Feb 22, 2016 at 10:21:02AM -0500, stanley lieber wrote:
> > Have you tried setting and alternate user agent?
> > 
> 
> No. Is it possible to change the string for hget?

that's been ruled out, i think, by david.

- erik

Re: [9fans] rtl8169 gbe slow

[lucio]

> But I'm out of my depth: I'm not a TCP/IP expert.

I thought I was, but that was a long time ago.  But what you are
looking for might not be difficult to identity.  Probably traffic
holding up for reasons only snooping the link can reveal.

I hate that type of stuff!

Lucio.

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

On Mon Feb 22 05:41:31 PST 2016, 0in...@gmail.com wrote:
> This issue seems more related to Plan 9 than hget.
> 
> On Linux :
> 
> $ time hget -o /dev/null http://downloads.kergis.com/kertex/kertex_bundle.tar
> 
> real0m1.783s
> user0m0.037s
> sys 0m0.046s
> 
> On 9vx:
> 
> % time hget -o /dev/null http://downloads.kergis.com/kertex/kertex_bundle.tar
> couldn't set mtime: permission denied
> 0.00u 0.00s 1.93r hget -o /dev/null
> http://downloads.kergis.com/kertex/kertex_bundle.tar
> 
> On Plan 9:
> 
> % time hget -o /dev/null http://downloads.kergis.com/kertex/kertex_bundle.tar
> couldn't set mtime: permission denied
> 0.02u 0.01s 37.95r hget -o /dev/null
> http://downloads.kergis.com/kertex/kertex_bundle.tar
> 

i noticed this message:

tcp: trim: !inwind: seq 3582371632-3582373071 win 3582374512-3582440046 l 1440 
from 213.186.33.4

i'll need to do a detailed analysis of this to figure out what's going on here.
this could be something that tcpdump, or snoopy could make kind of obvious.

- erik

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 05:19:04PM +0200, lu...@proxima.alt.za wrote:
> > I have hence to ask the provider if there is something,
> > in their configuration, that could explain this
> 
> If you can run NetBSD at the same time as Plan 9, you could also use
> tcpdump (whatever its current impersonation) to monitor the link.
> It's been a long time since I last did that, but it may be revealing.
> 

I will have to snoopy and tcpdump as a last resort to try to have a clue.
Just looking at the numbers, it's like Plan9 only assembling packets in
order (1500 bytes every latency time), and reasking for packet n+1 after
receiving packet n, whatever packets being received in the mean time.
But I'm out of my depth: I'm not a TCP/IP expert.
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[stanley lieber]

Have you tried setting and alternate user agent?

sl

Re: [9fans] rtl8169 gbe slow

[lucio]

> I have hence to ask the provider if there is something,
> in their configuration, that could explain this

If you can run NetBSD at the same time as Plan 9, you could also use
tcpdump (whatever its current impersonation) to monitor the link.
It's been a long time since I last did that, but it may be revealing.

Lucio.

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 02:39:20PM +, r...@hemiola.co.uk wrote:
> I get quite consistent results here.
> 
> Downloading http://downloads.kergis.com/kertex/kertex_bundle.tar to /dev/null:
> 
> linux over rtl8169 & ASDL  :  5.4 seconds
> plan 9 native over rtl8169 & ASDL  : 12.4 seconds
> as above, but latest sources tcp.c : 12.4 seconds
> 
> linux on server:  1.2 seconds
> plan9 in lguest on the same server :  2.2 seconds
> 
> Oldish labs kernels in both cases.
> 

The interesting data is the difference between plan9 native and
plan9 under lguest. Does plan9 under lguest actually use the linux
hardware services? Is plan9 under lguest using "its" implementation
except for the low level device driving i.e. the ethernet provided
by the Linux host?

-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 05:20:40AM -0800, erik quanstrom wrote:
> why?  what's the evidence?
> 

If I download from vanilla Plan9 (running on bare metal) data from 
_another http server_, I have correct results.

So the problem is not with Plan9 per se, but downloading from _this_
site (and it is mine!) and under Plan9.

If there was a problem with Plan9, I will have bad speed from whatever
server under Plan9. But it is not the case.

So there is something at the intersection (&&) of Plan9 and this site. 
And it can be Plan9 identifier (client ID) triggering something on
the server side leading to throttling the connection (or my DSL provider
throttling to the server provider; but this doesn't explain why others
have also poor results connecting from Plan9 while I guess they are not
with the same DSL provider as me---SFR here---and this doesn't explain
why with the same connection, downloading under NetBSD doesn't show the
problem---unless, once more, "plan9/hget" is considered a bittorrent
client or a robot and is causing the throttling, but this would seem a
"general" rule then).

Yes, it could be something with Plan9 at some upper IP level. But before
searching the needle in the hay stack, I have to rule out a possible
throttling on the server side (that I don't own or manage; I only
lend room on it; I have hence to ask the provider if there is something,
in their configuration, that could explain this).
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[David du Colombier]

This issue seems more related to Plan 9 than hget.

On Linux :

$ time hget -o /dev/null http://downloads.kergis.com/kertex/kertex_bundle.tar

real0m1.783s
user0m0.037s
sys 0m0.046s

On 9vx:

% time hget -o /dev/null http://downloads.kergis.com/kertex/kertex_bundle.tar
couldn't set mtime: permission denied
0.00u 0.00s 1.93r hget -o /dev/null
http://downloads.kergis.com/kertex/kertex_bundle.tar

On Plan 9:

% time hget -o /dev/null http://downloads.kergis.com/kertex/kertex_bundle.tar
couldn't set mtime: permission denied
0.02u 0.01s 37.95r hget -o /dev/null
http://downloads.kergis.com/kertex/kertex_bundle.tar

-- 
David du Colombier

Re: [9fans] rtl8169 gbe slow

[lucio]

> FWIW, I have sent a request to my provider asking if Plan9/hget could
> trigger a "robot" rule leading to the throttling of the connection.

It's unlikely, still QoS may be a factor.  But different results here
in South Africa (1300 km apart, granted) suggest otherwise.  More load
related, is my gut feel.  The host or the network.

Still, you did get consistent results on your side, didn't you.  That
was the original complaint, one slow host (adapter).

Lucio.

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

why?  what's the evidence?

- erik


On Feb 22, 2016 5:02 AM, tlaro...@polynum.com wrote:
>
> On Mon, Feb 22, 2016 at 01:52:48PM +0100, Mark van Atten wrote: 
> > Same 9front under virtualbox: 
> > 
> > term% time hget -o /dev/null http://mirrors.ctan.org/macros/latex/base.zip 
> > 0.06u 0.24s 8.74r hget -o /dev/null 
> > http://mirrors.ctan.org/macros/latex/base.zip 
>
> Yes, this is the problem. It is this very address: kergis.com that is 
> causing a problem. The question is: is it because of something in the 
> Plan9 implementation; or is it a different behavior from the http server 
> depending on the OS/utility advertised? I suspect more the latter. 
> -- 
>     Thierry Laronde  
>  http://www.kergis.com/ 
>  http://www.arts-po.fr/ 
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C 
>
>

Re: [9fans] rtl8169 gbe slow

[tlaronde]

FWIW, I have sent a request to my provider asking if Plan9/hget could
trigger a "robot" rule leading to the throttling of the connection.

-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 01:52:48PM +0100, Mark van Atten wrote:
> Same 9front under virtualbox:
> 
> term% time hget -o /dev/null http://mirrors.ctan.org/macros/latex/base.zip
> 0.06u 0.24s 8.74r  hget -o /dev/null
> http://mirrors.ctan.org/macros/latex/base.zip

Yes, this is the problem. It is this very address: kergis.com that is
causing a problem. The question is: is it because of something in the
Plan9 implementation; or is it a different behavior from the http server
depending on the OS/utility advertised? I suspect more the latter.
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[Mark van Atten]

Same 9front under virtualbox:

term% time hget -o /dev/null http://mirrors.ctan.org/macros/latex/base.zip
0.06u 0.24s 8.74rhget -o /dev/null
http://mirrors.ctan.org/macros/latex/base.zip

Mark.


On 2/22/16, tlaro...@polynum.com  wrote:
> On Mon, Feb 22, 2016 at 12:17:30PM +, Richard Miller wrote:
>> > It seems that Plan9 is not at fault per se
>>
>> I think it probably is.  Here's another data point (same ADSL connection)
>> -
>
> The delicate point is: is plan9 at fault or it is the fact that it is
> advertised as Plan9 that is the source of this throttling down.
>
> Because I have tested retrieving a 10MB file from another server, under
> plan9 (http://mirrors.ctan.org/macros/latex/base.zip) and I have good
> results on par with what I have under NetBSD (same node; dualboot).
>
> So I think the ethernet layer (the rtl8169) is not at fault. Perhaps
> another IP layer is at fault (bad negociation under some circonstances
> leading to very small packets). Or the server is throttling down
> the connection, whether explicitely because it is Plan9/hget (used
> by some? as bittorrent utility, or the string used etc.) or simply
> because there is a rule that everything not recognized/authorized
> (ie, chrome, mozilla, wget, ftp, lftp) is considered a robot, and
> is throttled down...
>
>>
>> #l0: i82579: 1Gbps port 0xFE50 irq 10: 386077f0e800
>> 0.09u 0.08s 182.26r   hget
>> http://downloads.kergis.com/kertex/kertex_bundle.tar
>>
>> But on the same machine, using linux instead:
>>
>> $ time wget -o /dev/null
>> http://downloads.kergis.com/kertex/kertex_bundle.tar
>>
>> real 0m9.134s
>> user 0m0.048s
>> sys  0m0.186s
>>
>
> --
> Thierry Laronde 
>  http://www.kergis.com/
>  http://www.arts-po.fr/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>
>

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 12:17:30PM +, Richard Miller wrote:
> > It seems that Plan9 is not at fault per se
> 
> I think it probably is.  Here's another data point (same ADSL connection) -

The delicate point is: is plan9 at fault or it is the fact that it is
advertised as Plan9 that is the source of this throttling down.

Because I have tested retrieving a 10MB file from another server, under
plan9 (http://mirrors.ctan.org/macros/latex/base.zip) and I have good
results on par with what I have under NetBSD (same node; dualboot).

So I think the ethernet layer (the rtl8169) is not at fault. Perhaps 
another IP layer is at fault (bad negociation under some circonstances
leading to very small packets). Or the server is throttling down
the connection, whether explicitely because it is Plan9/hget (used
by some? as bittorrent utility, or the string used etc.) or simply
because there is a rule that everything not recognized/authorized
(ie, chrome, mozilla, wget, ftp, lftp) is considered a robot, and
is throttled down...

> 
> #l0: i82579: 1Gbps port 0xFE50 irq 10: 386077f0e800
> 0.09u 0.08s 182.26rhget 
> http://downloads.kergis.com/kertex/kertex_bundle.tar
> 
> But on the same machine, using linux instead:
> 
> $ time wget -o /dev/null http://downloads.kergis.com/kertex/kertex_bundle.tar
> 
> real  0m9.134s
> user  0m0.048s
> sys   0m0.186s
> 

-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[Mark van Atten]

I observe the same on Debian 8.3 64 bit (the machine on which I run
9front in a virtualbox, giving the result I reported earlier today):

; time wget -o /dev/null http://downloads.kergis.com/kertex/kertex_bundle.tar
0.16u 0.57s 8.39r wget -o /dev/null
http://downloads.kergis.com/kertex/kertex_bundle.tar

Mark.

Re: [9fans] rtl8169 gbe slow

[Richard Miller]

> It seems that Plan9 is not at fault per se

I think it probably is.  Here's another data point (same ADSL connection) -

#l0: i82579: 1Gbps port 0xFE50 irq 10: 386077f0e800
0.09u 0.08s 182.26r  hget 
http://downloads.kergis.com/kertex/kertex_bundle.tar

But on the same machine, using linux instead:

$ time wget -o /dev/null http://downloads.kergis.com/kertex/kertex_bundle.tar

real0m9.134s
user0m0.048s
sys 0m0.186s

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 02:00:53PM +0200, lu...@proxima.alt.za wrote:
> > It seems that Plan9 is not at fault per se, but the server I'm on has
> > not a tremendous throughput, and since it is shared, varies greatly.
> 
> It could be traffic related in a lot of ways.  Or load related.  Might
> be worth speaking to the service provider to see if they are aware of
> the wild swings?

Yes, I will contact them too!
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[lucio]

> It seems that Plan9 is not at fault per se, but the server I'm on has
> not a tremendous throughput, and since it is shared, varies greatly.

It could be traffic related in a lot of ways.  Or load related.  Might
be worth speaking to the service provider to see if they are aware of
the wild swings?

Lucio.

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 11:06:47AM +, Richard Miller wrote:
> > If someone under Plan9 could try to download with hget(1):
> 
> >From home (ADSL connection) - standard distribution on x86:
> 
> 0.15u 0.16s 183.90rhget 
> http://downloads.kergis.com/kertex/kertex_bundle.tar
> #l0: rtl8169: 1Gbps port 0xDE00 irq 10: 003018a47956
> 

Same configuration for me. Twice the speed but not tremendous...

> >From server room somewhere in Amsterdam - standard distribution on raspberry 
> >pi B+:
> 
> 1.02u 0.86s 87.53r hget 
> http://downloads.kergis.com/kertex/kertex_bundle.tar
> 

It seems that Plan9 is not at fault per se, but the server I'm on has
not a tremendous throughput, and since it is shared, varies greatly.

I will try to see if I can get a pattern by comparing the Plan9
connection with the NetBSD one (same amd64, dualboot).

Thanks to all for the data!

-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[Richard Miller]

> If someone under Plan9 could try to download with hget(1):

>From home (ADSL connection) - standard distribution on x86:

0.15u 0.16s 183.90r  hget 
http://downloads.kergis.com/kertex/kertex_bundle.tar
#l0: rtl8169: 1Gbps port 0xDE00 irq 10: 003018a47956

>From server room somewhere in Amsterdam - standard distribution on raspberry 
>pi B+:

1.02u 0.86s 87.53r   hget 
http://downloads.kergis.com/kertex/kertex_bundle.tar

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 12:15:41PM +0200, lu...@proxima.alt.za wrote:
> > 0.10u 0.22s 192.79r   hget -o kertex_bundle.tar
> > http://downloads.kergis.com/kertex/kertex_bundle.tar
> > 
> > This is under 9front under virtualbox 4.3.32.
> 
> I get, from my workstation:
> 
> 0.21u 1.34s 50.52r hget -o /tmp/kertex_bundle.tar 
> http://downloads.kergis.com/kertex/kertex_bundle.tar

Thank you. Is it plain plan9 in this case?

> 
> and
> 
> term% md5sum /tmp/kertex_bundle.tar
> fbf91d2c9fed66604d14c70e4ab84f5a  /tmp/kertex_bundle.tar
> 
> >From my Cape Town server: 
> 
> 0.26u 2.67s 714.38rhget -o /tmp/kertex_bundle.tar 
> http://downloads.kergis.com/kertex/kertex_bundle.tar
> 
> More like what you get, isn't it? Same file:

Yes, on par with what I get. But I'm not under emulation: it's plain
Plan9 on bare metal in my case...


> This one is a VMware ESXi instance with IP address 192.96.32.148 (or
> somesuch).

Thanks for the data.

So with what Mark van Atten and you have, the problem is not local (to
my workstation) and this is why I'd like to now the difference between
the VMware instance and the first with more correct download times. If
it is plain plan9 (not hosted), I'm a bit puzzled...
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[lucio]

> 0.10u 0.22s 192.79r   hget -o kertex_bundle.tar
> http://downloads.kergis.com/kertex/kertex_bundle.tar
> 
> This is under 9front under virtualbox 4.3.32.

I get, from my workstation:

0.21u 1.34s 50.52r   hget -o /tmp/kertex_bundle.tar 
http://downloads.kergis.com/kertex/kertex_bundle.tar

and

term% md5sum /tmp/kertex_bundle.tar
fbf91d2c9fed66604d14c70e4ab84f5a/tmp/kertex_bundle.tar

>From my Cape Town server: 

0.26u 2.67s 714.38r  hget -o /tmp/kertex_bundle.tar 
http://downloads.kergis.com/kertex/kertex_bundle.tar

More like what you get, isn't it? Same file:

fumble# md5sum /tmp/kertex_bundle.tar
fbf91d2c9fed66604d14c70e4ab84f5a/tmp/kertex_bundle.tar

This one is a VMware ESXi instance with IP address 192.96.32.148 (or
somesuch).

Lucio.

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Mon, Feb 22, 2016 at 10:40:18AM +0100, Mark van Atten wrote:
> Dear Thierry,
> 
> > If someone under Plan9 could try to download with hget(1):
> >
> > http://downloads.kergis.com/kertex/kertex_bundle.tar
> >
> > and give me the time (it is a 10MB file) to do so,
> 
> 0.10u 0.22s 192.79r   hget -o kertex_bundle.tar
> http://downloads.kergis.com/kertex/kertex_bundle.tar
> 
> This is under 9front under virtualbox 4.3.32.

Thank you. It's better than what I get, but it is still poor: ~53KB/s...
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[Mark van Atten]

Dear Thierry,

> If someone under Plan9 could try to download with hget(1):
>
> http://downloads.kergis.com/kertex/kertex_bundle.tar
>
> and give me the time (it is a 10MB file) to do so,

0.10u 0.22s 192.79r   hget -o kertex_bundle.tar
http://downloads.kergis.com/kertex/kertex_bundle.tar

This is under 9front under virtualbox 4.3.32.

Mark.

Re: [9fans] rtl8169 gbe slow

[tlaronde]

Actually, the sources are up-to-date.

Setting "tcp" for /net/log doesn't produce any message.

Since the problem is with one address (http://downloads.kergis.com/),
I will have to snoopy the interface to have a clue about what is going
on (is the negociation leading to this poor performance? Are the packets
arriving sparsely, meaning that the problem is "before" with the gateway
or the external server? Is "Plan9/hget" considered a bittorrent
application and the external server throttling down on purpose?).

If someone under Plan9 could try to download with hget(1):

http://downloads.kergis.com/kertex/kertex_bundle.tar

and give me the time (it is a 10MB file) to do so, I will have a clue
about whether there is something "generally" going on on the server 
with Plan9 or hget, or if the problem is local to my installation.

TIA
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Sat, Feb 20, 2016 at 08:41:01PM +0100, Mark van Atten wrote:
> > i think that david has a mirror up, and 9fs sources still works here.
> 
> http://9p.io/

Thanks, Mark! 
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Sat, Feb 20, 2016 at 11:26:58AM -0800, erik quanstrom wrote:
> > > anyway, please update your tcp.  the debugging tools that are most
> > > helpful with tcp are
> > > /net/tcp/stats
> > > /net/tcp/*/status
> > > echo tcp>/net/log && tail -f /net/log
> > 
> > To update I need to update the sources. Where are now the "updated"
> > sources? since Bell Labs site seems to be definitively down... Is there
> > somewhere the "latest" sources, at least with the TCP corrections?
> 
> i think that david has a mirror up, and 9fs sources still works here.
> 
> you can also grab the 9atom version @ 
> http://sources.9atom.org/sys/src/9/ip/tcp.c
> contact me off-list about any compile issues.  a few greps don't show any of 
> the usual suspects, but i haven't tried myself.

Thanks. I will try the Bell Labs mirror and fall back to the 9atom
version in case of problem (just to try to avoid mixing sources with the
inability finally to be able to say exactly what version I run...).
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[Mark van Atten]

> i think that david has a mirror up, and 9fs sources still works here.

http://9p.io/

Mark.

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

> > anyway, please update your tcp.  the debugging tools that are most
> > helpful with tcp are
> > /net/tcp/stats
> > /net/tcp/*/status
> > echo tcp>/net/log && tail -f /net/log
> 
> To update I need to update the sources. Where are now the "updated"
> sources? since Bell Labs site seems to be definitively down... Is there
> somewhere the "latest" sources, at least with the TCP corrections?

i think that david has a mirror up, and 9fs sources still works here.

you can also grab the 9atom version @ 
http://sources.9atom.org/sys/src/9/ip/tcp.c
contact me off-list about any compile issues.  a few greps don't show any of 
the usual suspects, but i haven't tried myself.

- erik

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

> > anyway, please update your tcp.  the debugging tools that are most
> > helpful with tcp are
> > /net/tcp/stats
> > /net/tcp/*/status
> > echo tcp>/net/log && tail -f /net/log
> 
> We have definitively not the same systems ;-) The echo tcp brings an
> error for netlog.

sorry, it's "echo tcp set>/net/log && tail -f /net/log".

chasing the performance of different sites is what lead to the tcp improvements.

- erik

Re: [9fans] rtl8169 gbe slow

[tlaronde]

> anyway, please update your tcp.  the debugging tools that are most
> helpful with tcp are
> /net/tcp/stats
> /net/tcp/*/status
> echo tcp>/net/log && tail -f /net/log

We have definitively not the same systems ;-) The echo tcp brings an
error for netlog.

But for further puzzling things (for me).

If I try to download the kertex tarball from "my" site (I just pay for
some space on a remote server; it is not my machine) the performance are
abysmal.

I tried another site, with http, in this case:

http://mirrors.ctan.org/macros/latex/base.zip

since the file for the base LaTeX is the same size as the tarball for
the whole kerTeX.

In this case, I have the following:

first ": 1024
second ": 34k
afterward: 1.7M!

So there is "something" about the "negociation" between hget and
"my" site that is not good---note: "my" site uses cookies (I don't;
this is the Apache running on the server that does; my pages don't
use cookies at all and I have no hand on that). I wonder if some
"persistent" (cookie) information could cause this misbehavior
(since I have a multiboot PC, this very same IP address is depending
on the moment whether NetBSD, Plan9 or, more rarely, Windows 8.1).

-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Sat, Feb 20, 2016 at 09:20:52AM -0800, erik quanstrom wrote:
> On Sat Feb 20 06:04:02 PST 2016, tlaro...@polynum.com wrote:
> > On Sat, Feb 20, 2016 at 02:31:54PM +0100, hiro wrote:
> > > what is the latency on WAN?
> > 
> > When using traceroute, I have 42.6ms for a roundtrip 
> > (cf. with LAN: 0.23ms).
> > 
> > But the very same machine, under NetBSD, with the very same ip address,
> > downloads the very same file from the very same external server
> > (downloads.kergis.com) in 17s, while hget(1) spends 6 minutes doing
> > it.
> > 
> > I wondered if the unsupported same chip integrated network card would
> > be a problem. But disabling it via the BIOS doesn't change anything.
> > 
> > Is there a way to trace what hget is doing/calling so that I can have a
> > clue about the bottleneck? There is no transmission errors on the
> > interface, so the problem is in the upper levels of TCP/IP.
> 
> yes.  i believe this was suggested before.  from the evidence, the best
> guess is that you are using an old kernel with an old tcp.

Does it explain the difference between LAN and WAN? ftp is TCP; and on
LAN there is no visible problem (connecting to a host running NetBSD)---
well, since the latency on LAN is the 1/300 of the latency of WAN, the 
problem could be masked...

> 
> the old tcp had abysmal performance starting at a latency of ~10ms.  this
> was due to a flawed implementation of tcp reno.  plan 9 used to commit
> the cardinal sin of tcp, and move the left edge of the window.
> 
> anyway, please update your tcp.  the debugging tools that are most
> helpful with tcp are
> /net/tcp/stats
> /net/tcp/*/status
> echo tcp>/net/log && tail -f /net/log

To update I need to update the sources. Where are now the "updated"
sources? since Bell Labs site seems to be definitively down... Is there
somewhere the "latest" sources, at least with the TCP corrections?
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

On Sat Feb 20 06:04:02 PST 2016, tlaro...@polynum.com wrote:
> On Sat, Feb 20, 2016 at 02:31:54PM +0100, hiro wrote:
> > what is the latency on WAN?
> 
> When using traceroute, I have 42.6ms for a roundtrip 
> (cf. with LAN: 0.23ms).
> 
> But the very same machine, under NetBSD, with the very same ip address,
> downloads the very same file from the very same external server
> (downloads.kergis.com) in 17s, while hget(1) spends 6 minutes doing
> it.
> 
> I wondered if the unsupported same chip integrated network card would
> be a problem. But disabling it via the BIOS doesn't change anything.
> 
> Is there a way to trace what hget is doing/calling so that I can have a
> clue about the bottleneck? There is no transmission errors on the
> interface, so the problem is in the upper levels of TCP/IP.

yes.  i believe this was suggested before.  from the evidence, the best
guess is that you are using an old kernel with an old tcp.

the old tcp had abysmal performance starting at a latency of ~10ms.  this
was due to a flawed implementation of tcp reno.  plan 9 used to commit
the cardinal sin of tcp, and move the left edge of the window.

anyway, please update your tcp.  the debugging tools that are most
helpful with tcp are
/net/tcp/stats
/net/tcp/*/status
echo tcp>/net/log && tail -f /net/log

- erik

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Sat, Feb 20, 2016 at 04:06:24PM +0200, lu...@proxima.alt.za wrote:
> > But the very same machine, under NetBSD, with the very same ip address,
> > downloads the very same file from the very same external server
> > (downloads.kergis.com) in 17s, while hget(1) spends 6 minutes doing
> > it.
> 
> Just for one more data point: dump the hget output to /dev/null.  That
> may at least exclude the disks and fossil from the equation.

I have already ruled this out: copying the file in the very same place 
from mounted ftpfs (LAN), no speed problem. Copying in /tmp served by 
ramfs, no difference for hget...
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[lucio]

> But the very same machine, under NetBSD, with the very same ip address,
> downloads the very same file from the very same external server
> (downloads.kergis.com) in 17s, while hget(1) spends 6 minutes doing
> it.

Just for one more data point: dump the hget output to /dev/null.  That
may at least exclude the disks and fossil from the equation.

Lucio.

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Sat, Feb 20, 2016 at 02:31:54PM +0100, hiro wrote:
> what is the latency on WAN?

When using traceroute, I have 42.6ms for a roundtrip 
(cf. with LAN: 0.23ms).

But the very same machine, under NetBSD, with the very same ip address,
downloads the very same file from the very same external server
(downloads.kergis.com) in 17s, while hget(1) spends 6 minutes doing
it.

I wondered if the unsupported same chip integrated network card would
be a problem. But disabling it via the BIOS doesn't change anything.

Is there a way to trace what hget is doing/calling so that I can have a
clue about the bottleneck? There is no transmission errors on the
interface, so the problem is in the upper levels of TCP/IP.
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[hiro]

what is the latency on WAN?

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Sat, Feb 20, 2016 at 12:23:29PM +0100, Kenny Lasse Hoff Levinsen wrote:
>> [rtl8169 gbe full speed on LAN; very slow on WAN]
> Is your MTU higher that 1500? That might be able to mess things up over the 
> internet.
> 

Thanks for the suggestion but no: even with -m 1500, speed is still
awful.

Best,
-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[Kenny Lasse Hoff Levinsen]

Is your MTU higher that 1500? That might be able to mess things up over the 
internet.

Best regards,
Kenny Levinsen

> On 20. feb. 2016, at 11.32, tlaro...@polynum.com wrote:
> 
> I have compared downloading a file (via ftpfs) on the LAN, and
> downloading it from the WAN.
> 
> On the LAN, I get the 10MB file in less than a 1s (this is normal since
> the node I download from has only a 100Mb ethernet).
> 
> On the WAN, it takes 6 minutes (with hget).
> 
> My conclusion is that the card device (driver) is not at fault but that
> something is wrong along the path when I get through the gateway.
> 
> Has somebody an idea about what I may do wrong or what could cause such
> a cost on the gateway.
> 
> I have set IPv6 on the gateway: no difference. I have tried to disable
> IPv6 on the ether: no difference. 
> 
> ip/traceroute to the outside server shows results that are on par
> with what I get from NetBSD on this very node where I run also
> plan9.
> 
> "Something" is obviously wrong. But "what" is less obvious (at least to
> me).
> 
> -- 
>Thierry Laronde 
> http://www.kergis.com/
> http://www.arts-po.fr/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
> 

Re: [9fans] rtl8169 gbe slow

[tlaronde]

I have compared downloading a file (via ftpfs) on the LAN, and
downloading it from the WAN.

On the LAN, I get the 10MB file in less than a 1s (this is normal since
the node I download from has only a 100Mb ethernet).

On the WAN, it takes 6 minutes (with hget).

My conclusion is that the card device (driver) is not at fault but that
something is wrong along the path when I get through the gateway.

Has somebody an idea about what I may do wrong or what could cause such
a cost on the gateway.

I have set IPv6 on the gateway: no difference. I have tried to disable
IPv6 on the ether: no difference. 

ip/traceroute to the outside server shows results that are on par
with what I get from NetBSD on this very node where I run also
plan9.

"Something" is obviously wrong. But "what" is less obvious (at least to
me).

-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

> Is /dev/irqstat a lapsus? Here are /dev/irqalloc and
> /net/ether0/ifstats:
[...]
>  42  10 ether0

well, boo.  the labs version doesn't give enough information.  i was expecting
something like
; grep ether0 /dev/irqalloc
   65.0  11 17224065 190397006374 msi-x
ether0
which gives vector.processor, bus irq, number of interrupts, total fastticks 
taken, type, and driver.
with this info, one can calculate ns/interrupt and interrupts/sec.

> TxOk: 4483
> RxOk: 7520
[..]
> xmit descr queue len: highwater 0/31 curr 0 hitmax 0

this doesn't look like very much traffic.  how did you test?  are you using
the latest tcp?  what is the ping latency?  older labs kernels did a poor job
with moderate latency, and any packet loss.  newer versions should be ok,
but i haven't tested myself.

> rtl8169: unknown mac 8168 4c00

my version of the driver handles this hardware

> i8042: fe returned to the ea command

you don't have a ps2 mouse, but the system is configured to expect one.
0xea -> set streaming.

- erik

Re: [9fans] rtl8169 gbe slow

[tlaronde]

On Thu, Feb 18, 2016 at 05:22:42AM -0800, erik quanstrom wrote:
> the 8169 driver is pretty fast.  I've measured it at more than 500mbps.
> it sounds like something else is misbehaving.  what does
> /dev/irqstat say.  I bet something is stuck.
> 

Is /dev/irqstat a lapsus? Here are /dev/irqalloc and
/net/ether0/ifstats:

  3   0 debugpt
  7   0 mathemu
  8   0 doublefault
  9   0 mathover
 14   0 fault386
 15   0 unexpected
 16   0 matherror
 32   0 clock
 33   1 kbd
 35   3 sdE (iahci)
 38   6 floppy
 39   7 lpt
 42  10 ether0
 44  12 kbdaux

ifstats:

TxOk: 4483
RxOk: 7520
TxEr: 0
RxEr: 0
MissPkt: 0
FAE: 0
Tx1Col: 0
TxMCol: 0
RxOkPh: 7482
RxOkBrd: 28
RxOkMu: 10
TxAbt: 0
TxUndrn: 0
txdu: 0
tcpf: 0
udpf: 0
ipf: 0
fovf: 0
ierrs: 0
rer: 0
rdu: 0
punlc: 0
fovw: 0
tcr: 0x2f200700
rcr: 0xe70e
multicast: 10
phy:1000 796d 001c c914 01e1 c5e1 000d 2801
4e2e 0300 3800     3000
01ee ac9c   8040 0006 4100 2100
 8c00 0040 0106 217c 8fbc 0123 
rcv descrs processed at once:   highwater 2/255 curr 1 hitmax 0
xmit descr queue len:   highwater 0/31 curr 0 hitmax 0

Note: _this_ card is a PCIe supplementary one. There is another rtl
embedded in the motherboard that Plan9 does not recognize:

rtl8169: unknown mac 8168 4c00
oui 0x732 phyno 1, macv = 0x2c00 phyv = 0x0004
#l0: rtl8169: 1Gbps port 0xC000 irq 10: e8de2701f455

I have tried by disabling the embedded ether controller, the result is
the same.

Another kernel message for what is worth (since I don't know what it
means, I don't know if it's relevant):

i8042: fe returned to the ea command

-- 
Thierry Laronde 
 http://www.kergis.com/
 http://www.arts-po.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

the standard labs driver checks the speed during auto negotiation.
so it is often wrong.

- erik


On Feb 18, 2016 3:41 PM, arisawa  wrote:
>
> hello, 
>
> rtl8169 is popular in cheap MB, so it is installed in my many MBs. 
> however, cat /dev/kmesg claims: 
> rtl8169: oui 0x732 phyno 1, macv = 0x3c00 phyv = 0x0002 
> #l0: rtl8169: 100Mbps port 0xD000 irq 10: 001fd0169891 
>
> the “100Mbps" in the message is correct or not? 
>
> I also feel rtl8169 is slow, so I replace on-board rtl8169 by intel’s card  
> if possible. 
>
>
>
> > 2016/02/18 22:22、erik quanstrom  のメール： 
> > 
> > the 8169 driver is pretty fast.  I've measured it at more than 500mbps. 
> > it sounds like something else is misbehaving.  what does 
> > /dev/irqstat say.  I bet something is stuck. 
> > 
> > - erik 
> > 
> > 
> > On Feb 18, 2016 3:30 AM, tlaro...@polynum.com wrote: 
> >> 
> >> Hello, 
> >> 
> >> I have finally managed to install plan9 on my new workstation. 
> >> 
> >> By putting back the keyboard on the PS2 connector, I have solved some 
> >> unfelicities (with the USB->legacy emulation, the keyboard switched 
> >> every other typing to UPPERCASE...). 
> >> 
> >> The mouse, still USB connected and hence "emulated" by the BIOS, 
> >> does not react very gracefully but I will see if I can play with 
> >> the acceleration and the resolution to have a better terminal. (Or 
> >> if I manage to find a long enough cable to have a COM slot back since 
> >> there is the bare connector on the motherboard; in this case I will go 
> >> back to a com mouse and will be able to probe USB for other 
> >> devices---external disks.) 
> >> 
> >> One thing is inconvenient: I have a rtl8169 gbe pci-e ether card, but 
> >> when testing the compilation of kerTeX (it has been fixed: it works for 
> >> the last release; rio to come for METAFONT), the throughput with hget is 
> >> abysmal: 30kB/s... The disk is not a fault, reacting well enough (I have 
> >> plenty of RAM and the blocks cache for fossil is set to 3000---it could 
> >> be obviously higher). 
> >> 
> >> Setting the mtu to jumbo packet does not help. 
> >> 
> >> Is there something to tune or is it simply that the chip is not well 
> >> supported? 
> >> 
> >> TIA 
> >> -- 
> >> Thierry Laronde  
> >>  http://www.kergis.com/ 
> >>  http://www.arts-po.fr/ 
> >> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C 
> >> 
> >> 
>
>
>

Re: [9fans] rtl8169 gbe slow

[arisawa]

hello,

rtl8169 is popular in cheap MB, so it is installed in my many MBs.
however, cat /dev/kmesg claims:
rtl8169: oui 0x732 phyno 1, macv = 0x3c00 phyv = 0x0002
#l0: rtl8169: 100Mbps port 0xD000 irq 10: 001fd0169891

the “100Mbps" in the message is correct or not?

I also feel rtl8169 is slow, so I replace on-board rtl8169 by intel’s card  if 
possible.



> 2016/02/18 22:22、erik quanstrom  のメール：
> 
> the 8169 driver is pretty fast.  I've measured it at more than 500mbps.
> it sounds like something else is misbehaving.  what does
> /dev/irqstat say.  I bet something is stuck.
> 
> - erik
> 
> 
> On Feb 18, 2016 3:30 AM, tlaro...@polynum.com wrote:
>> 
>> Hello, 
>> 
>> I have finally managed to install plan9 on my new workstation. 
>> 
>> By putting back the keyboard on the PS2 connector, I have solved some 
>> unfelicities (with the USB->legacy emulation, the keyboard switched 
>> every other typing to UPPERCASE...). 
>> 
>> The mouse, still USB connected and hence "emulated" by the BIOS, 
>> does not react very gracefully but I will see if I can play with 
>> the acceleration and the resolution to have a better terminal. (Or 
>> if I manage to find a long enough cable to have a COM slot back since 
>> there is the bare connector on the motherboard; in this case I will go 
>> back to a com mouse and will be able to probe USB for other 
>> devices---external disks.) 
>> 
>> One thing is inconvenient: I have a rtl8169 gbe pci-e ether card, but 
>> when testing the compilation of kerTeX (it has been fixed: it works for 
>> the last release; rio to come for METAFONT), the throughput with hget is 
>> abysmal: 30kB/s... The disk is not a fault, reacting well enough (I have 
>> plenty of RAM and the blocks cache for fossil is set to 3000---it could 
>> be obviously higher). 
>> 
>> Setting the mtu to jumbo packet does not help. 
>> 
>> Is there something to tune or is it simply that the chip is not well 
>> supported? 
>> 
>> TIA 
>> -- 
>> Thierry Laronde  
>>  http://www.kergis.com/ 
>>  http://www.arts-po.fr/ 
>> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C 
>> 
>> 

Re: [9fans] rtl8169 gbe slow

[erik quanstrom]

the 8169 driver is pretty fast.  I've measured it at more than 500mbps.
it sounds like something else is misbehaving.  what does
/dev/irqstat say.  I bet something is stuck.

- erik


On Feb 18, 2016 3:30 AM, tlaro...@polynum.com wrote:
>
> Hello, 
>
> I have finally managed to install plan9 on my new workstation. 
>
> By putting back the keyboard on the PS2 connector, I have solved some 
> unfelicities (with the USB->legacy emulation, the keyboard switched 
> every other typing to UPPERCASE...). 
>
> The mouse, still USB connected and hence "emulated" by the BIOS, 
> does not react very gracefully but I will see if I can play with 
> the acceleration and the resolution to have a better terminal. (Or 
> if I manage to find a long enough cable to have a COM slot back since 
> there is the bare connector on the motherboard; in this case I will go 
> back to a com mouse and will be able to probe USB for other 
> devices---external disks.) 
>
> One thing is inconvenient: I have a rtl8169 gbe pci-e ether card, but 
> when testing the compilation of kerTeX (it has been fixed: it works for 
> the last release; rio to come for METAFONT), the throughput with hget is 
> abysmal: 30kB/s... The disk is not a fault, reacting well enough (I have 
> plenty of RAM and the blocks cache for fossil is set to 3000---it could 
> be obviously higher). 
>
> Setting the mtu to jumbo packet does not help. 
>
> Is there something to tune or is it simply that the chip is not well 
> supported? 
>
> TIA 
> -- 
>     Thierry Laronde  
>  http://www.kergis.com/ 
>  http://www.arts-po.fr/ 
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C 
>
>