Re: APU.1D RealtekRTL8111E

[Darren Tucker]

On Mon, Nov 2, 2015 at 12:56 PM, Darren Tucker  wrote:
> Not that I have seen, but I don't know what the limiting factor is.
> iperf will push ~500Mbit/s from userspace (mtu 1500)
[...]
> I also notice dlg just made the following change to sys/dev/ic/re.c
> which will probably make a difference (this change is not on the
> device I tested):

I reran the test with dlg's change and the iperf output rate went up
to ~535Mbit/s with a couple of percent of idle cpu.  I should update
my interrupt mitigation diff and see if that helps further.

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

Re: APU.1D RealtekRTL8111E

[Theo de Raadt]

> "...push 1 Gigabit of packets..."
> 
> Here is the logic that to me doesn't make sense. It' ok for you to
> disagree with me, but think about it for 2 minutes.

oh come on daniel, your logic is all wrong.

obviously the OP has bought a full 1G of bandwidth from HE at some
datacenter that is fitted with 7" x 7" x 2" racks, and he wants to use
it all.

unfortunately, this device is the wrong fit for his needs.  he'll need
to buy more rackspace also.

Re: APU.1D RealtekRTL8111E

[Daniel Ouellet]

dmesg in the archive already.

APU.1D4

http://marc.info/?l=openbsd-misc=144531661519455=2

Works well so far and the box is somewhat hot as it is the heat sink for
the board.

But in your concern I have one question that really puzzle me about it.

"...push 1 Gigabit of packets..."

Here is the logic that to me doesn't make sense. It' ok for you to
disagree with me, but think about it for 2 minutes.

If your customer have that kind of bandwidth, their internet access cost
most likely is not small. They must have more then a handful employees
and if they actually use that 1Gb, then they must be doing serious work,
well one would hope so anyway.

So, if you are concern for the capacity of this small router, and I may
be mistaken, but as far as I know, Soekris doesn't pass 1Gb I wouldn't
assume this box would either. Disclosure, I haven't tested that far to
be right or wrong... So, assumption only...

But the logic here is, if all the points above are true, that business
or customer of yours really have the resources to do something better.

Then put a few more $ it and do it right! I am not talking 2K, but you
sure can get something more powerful under 1K for sure!

If they can't justify that, then how serious are they about their
business...

It's not like they can't afford it right?

Why go sooo cheap on what may be very critical for them???

Anyway, that's the logic, it is sure great for home, or small office, or
even some what bigger for sure.

But thinking to use this and push it to it's limit for a company that
actually would push 1Gb, well, the logic appear to be broken here to me.

And if you start to add pf, filtering, and what not to that box, what's
left...

It's ONE time expense, not like their monthly recurring cost for the
current 1Gb access is it?

Hope this help some, put thing back in perspective and give you some
food for thought...

It's not because you can get away with something that it make sense to
do it...

Good luck.

Daniel

On 11/1/15 7:51 PM, Predrag Punosevac wrote:
> I am looking to build a firewall for a customer using APU.1D
> 
> http://www.mini-box.com/APU-1D-AMD-G-Series-T40E-APU?sc=8=754
> 
> I have two major concerns. The first one is RealtekRTL8111E. Does that
> thing really work on OpenBSD and can really push 1 Gigabit of packets?
> This thread 
> 
> http://marc.info/?l=openbsd-misc=2=1=APU1D=b
> 
> indicate that it works?
> 
> My second concern is the heat. How hot does APU-1D-AMD run on OpenBSD?
> Is it too hot?  What are recommended ways to deal with the heat. Could
> anybody post a dmesg for APU1D or APU.1D4?
> 
> Personally I really like Soekris net6501 but the price difference is
> substantial.
> 
> Cheers,
> Predrag

Re: APU.1D RealtekRTL8111E

[Daniel Ouellet]

On 11/1/15 10:28 PM, Theo de Raadt wrote:
>> "...push 1 Gigabit of packets..."
>>
>> Here is the logic that to me doesn't make sense. It' ok for you to
>> disagree with me, but think about it for 2 minutes.
> 
> oh come on daniel, your logic is all wrong.
> 
> obviously the OP has bought a full 1G of bandwidth from HE at some
> datacenter that is fitted with 7" x 7" x 2" racks, and he wants to use
> it all.
> 
> unfortunately, this device is the wrong fit for his needs.  he'll need
> to buy more rackspace also.

Well your right again. If he got 7" x 7" x 2", I am totally wrong!!!

Shame on me!

He can get two and CARP them! (P>

The box is actually, 6 1/2" X 6 1/8" x 1" (:>

I just measured it...

I hope he bought power too for HE as this take a HUGE amount. May be
could run it on a small ups with no AC for it for a week before it die.

But I haven't tested that either, so I really don't know what I am
talking about other then the unit they sale for it, can only provide 24
watts max and I know it's not close to what the unit take.

But hey, if he squeeze them real good in that hole the rack will also
serve as heat sink so his concern about heat are resolved!

Anyway, back to more serious things...

Re: APU.1D RealtekRTL8111E

[Darren Tucker]

On Mon, Nov 2, 2015 at 11:51 AM, Predrag Punosevac
 wrote:
> I am looking to build a firewall for a customer using APU.1D
>
> http://www.mini-box.com/APU-1D-AMD-G-Series-T40E-APU?sc=8=754
>
> I have two major concerns. The first one is RealtekRTL8111E. Does that
> thing really work on OpenBSD

Yes.

> and can really push 1 Gigabit of packets?

Not that I have seen, but I don't know what the limiting factor is.
iperf will push ~500Mbit/s from userspace (mtu 1500) but the cpus are
maxed out, mostly on interrupts.  I have not tested routing
performance for it (my experience with ALIX leads me to believe the
routing throughput will be better) nor have I tried to tune it.

I have a WIP diff that enables hardware interrupt mitigation on those
realtek chips but I don't have any performance numbers for it.

I also notice dlg just made the following change to sys/dev/ic/re.c
which will probably make a difference (this change is not on the
device I tested):

revision 1.182
date: 2015/11/02 00:08:50;  author: dlg;  state: Exp;  lines: +31 -25;
 commitid: 3h4FftdbNE7umRAE;
later variants of these chips can support bigger rx and tx rings. this
diff expands them so devices that need more packets per interrupt can
use them.

> This thread
>
> http://marc.info/?l=openbsd-misc=2=1=APU1D=b
>
> indicate that it works?
>
> My second concern is the heat. How hot does APU-1D-AMD run on OpenBSD?
> Is it too hot?

The case is noticeably warm, even at idle.

$ uptime
 6:26PM  up 9 days, 23:49, 1 user, load averages: 0.07, 0.08, 0.08
$ sysctl hw.sensors
hw.sensors.km0.temp0=58.00 degC

running both cores flat out:
$ uptime
 6:30PM  up 9 days, 23:54, 1 user, load averages: 2.98, 1.52, 0.69
$ sysctl hw.sensors
hw.sensors.km0.temp0=72.88 degC

I haven't had any heat-related problems, but I'm not using it for
anything serious.

> What are recommended ways to deal with the heat. Could
> anybody post a dmesg for APU1D or APU.1D4?

I think it's an APU1D (2G ram):

$ dmesg
OpenBSD 5.8-current (GENERIC.MP) #1537: Tue Oct 20 09:44:09 MDT 2015
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 2098511872 (2001MB)
avail mem = 2030845952 (1936MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7e16d820 (7 entries)
bios0: vendor coreboot version "4.0" date 09/08/2014
bios0: PC Engines APU
acpi0 at bios0: rev 0
acpi0: sleep states S0 S1 S3 S4 S5
acpi0: tables DSDT FACP SPCR HPET APIC HEST SSDT SSDT SSDT
acpi0: wakeup devices AGPB(S4) HDMI(S4) PBR4(S4) PBR5(S4) PBR6(S4)
PBR7(S4) PE20(S4) PE21(S4) PE22(S4) PE23(S4) PIBR(S4) UOH1(S3)
UOH2(S3) UOH3(S3) UOH4(S3) UOH5(S3) [...]
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpihpet0 at acpi0: 14318180 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD G-T40N Processor, 1000.15 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,SSSE3,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,IBS,SKINIT,ITSC
cpu0: 32KB 64b/line 2-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 16-way L2 cache
cpu0: 8 4MB entries fully associative
cpu0: DTLB 40 4KB entries fully associative, 8 4MB entries fully associative
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 200MHz
cpu0: mwait min=64, max=64, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD G-T40N Processor, 1000.00 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,SSSE3,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,IBS,SKINIT,ITSC
cpu1: 32KB 64b/line 2-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 16-way L2 cache
cpu1: 8 4MB entries fully associative
cpu1: DTLB 40 4KB entries fully associative, 8 4MB entries fully associative
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 21, 24 pins
acpiprt0 at acpi0: bus -1 (AGPB)
acpiprt1 at acpi0: bus -1 (HDMI)
acpiprt2 at acpi0: bus 1 (PBR4)
acpiprt3 at acpi0: bus 2 (PBR5)
acpiprt4 at acpi0: bus 3 (PBR6)
acpiprt5 at acpi0: bus -1 (PBR7)
acpiprt6 at acpi0: bus 5 (PE20)
acpiprt7 at acpi0: bus -1 (PE21)
acpiprt8 at acpi0: bus -1 (PE22)
acpiprt9 at acpi0: bus -1 (PE23)
acpiprt10 at acpi0: bus 0 (PCI0)
acpiprt11 at acpi0: bus 4 (PIBR)
acpicpu0 at acpi0: !C2(0@100 io@0x841), C1(@1 halt!), PSS
acpicpu1 at acpi0: !C2(0@100 io@0x841), C1(@1 halt!), PSS
acpibtn0 at acpi0: PWRB
cpu0: 1000 MHz: speeds: 1000 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "AMD AMD64 14h Host" rev 0x00
ppb0 at pci0 dev 4 function 0 "AMD AMD64 14h PCIE" rev 0x00: msi
pci1 at ppb0 bus 1
re0 at pci1 dev 0 function 0 "Realtek 8168" rev 0x06: RTL8168E/8111E
(0x2c00), msi, address 00:0d:b9:31:30:74
rgephy0 at re0 phy 7: