Re: Bad network performance on apu2c4
Ah, I’m not using pppoe so perhaps that’s significant? I have a straight ethernet set up, em0 as uplink, em1 connected to a dumb switch, em2 connected to a dumb WiFi AP. I measured the speed using fast.com on my mobile, laptop, desktop, as well as downloading large files from different servers and CDNs. As pointed out elsewhere in this thread, that test only covers full size packets. Since my APU2 is an edge router/firewall for my home network, I pretty much only get full size TCP packets, and some low throughput UDP packets. All in the kernel, i.e. pf, since the router doesn’t really generate traffic of its own. Peter > On Nov 4, 2017, at 10:49 AM, miraculli . wrote: > > Hi, > > i´ve also an APU2 as router. > The uplink connection (16Mbit/s) is via pppoe(4) on em0 > and i couldn´t manage to messure the throughput of this interface: > - iftop doesn´t work on pppoe and shows nothing on em0. > - ifperf also calculates some strange numbers (14669317741 Gbits/sec) > when trying to connect to one of the public iperf-servers from > https://iperf.fr/iperf-servers.php > > how do you messure the performance? > > > 2017-11-04 18:24 GMT+01:00 Peter Faiman : > > On Nov 4, 2017, at 09:53, Chris Cappuccio wrote: > > > > Rupert Gallagher [r...@protonmail.com] wrote: > >> > >> You seem to say that handling larger packets is a feature of having > >> limited CPU. I disagree. > >> > > > > Rupert, I'm saying that a slower CPU can process less packets per second. > > > > The important measurement is packets-per-second. The APU has plenty of > > memory bandwidth to handle large volumes of data. For adequate CPU power, > > you have to either lower the cost of processing (make software better/more > > efficient) or you have to distribute the cost across the 4 cores of the APU2 > > (make software execution parallel). > > > >>> The same traffic level, with 1500 byte packets generates 6 times more > >>> packets per second than that traffic level with 9000 bytes packets. > >> > >> You divided 9000 by 1500 without mistakes. Congratulations. > >> > > > > The point was clearly lost on you. > > > >>> There is ongoing work to improve the network stack performance on boxes > >>> like the APU2 (which have 4 cores). You will see improvements. If you > >>> want it better today, you need a faster box. Chris > >> > >> The apu2c4 is fast enough to saturate its Intel 1Gbits/sec link. It has > >> three of those. If you connect all three to the switch, you get 3Gbps shy. > >> No need for a faster box. You rather need a faster switch, class 7 S-FTP > >> wires (better than class 6), and 2.5Gbps lan cards for clients. > > > > No, you don't need any of that. You have no idea what you are talking about. > > > > The APU requires software crafted to evenly distribute PER-PACKET PROCESSING > > cost across multiple cores. That is what is happening in OpenBSD today. It > > has > > been happening for years, and it is getting closer to becoming a reality > > with > > OpenBSD + APU2, as well as other chipsets/platforms. > > > > For a couple years now, we've had interrupts processed by one core, PF on > > another, and other parts of the kernel on a third core. But to accelerate > > packet processing alone, we need interrupts handled on multiple cores, > > PF processing handled on multiple cores. This is hard work. > > > > By the way, what I'm describing is the general-purpose OS approach towads > > this problem. If you want to turn computer hardware into routers with little > > other concern, the go-to platform is DPDK + VPP. It is something like an > > order of magnitude faster than any general purpose OS (OpenBSD, Linux) at > > packet pushing. > > > > https://www.reddit.com/r/networking/comments/6upchy/can_a_bsd_system_replicate_the_performance_of/dlvdq2e/ > > > > Chris > > Thank you for this explanation. My uplink is only 240mbit and my APU2 handles > that perfectly, so I’m not having any of these problems. But the insight into > the current state of networking was great! :) > > Peter > > > > -- > +49.179.1448024 > Karl-Kunger-Straße 68 > D - 12435 Berlin
Re: Bad network performance on apu2c4
> On Nov 4, 2017, at 13:15, Stuart Henderson wrote: > >> On 2017-11-04, Peter Faiman wrote: >> Thank you for this explanation. My uplink is only 240mbit and my APU2 >> handles that perfectly, so I’m not having any of these problems. >> But the insight into the current state of networking was great! :) > > But it doesn't handle 240Mbit/s, *unless* the packets are large. > > If somebody sends 240Mb/s of minimal sized packets at you, it won't cope. Probably. I’ve never tried, it’s a home router so I only use that kind of bandwidth for stream connections. At full download it’s got two cores at about 45%, which is consistent with what you said about core separation of tasks, and maximum speed.
Re: Bad network performance on apu2c4
> On Nov 4, 2017, at 09:53, Chris Cappuccio wrote: > > Rupert Gallagher [r...@protonmail.com] wrote: >> >> You seem to say that handling larger packets is a feature of having limited >> CPU. I disagree. >> > > Rupert, I'm saying that a slower CPU can process less packets per second. > > The important measurement is packets-per-second. The APU has plenty of > memory bandwidth to handle large volumes of data. For adequate CPU power, > you have to either lower the cost of processing (make software better/more > efficient) or you have to distribute the cost across the 4 cores of the APU2 > (make software execution parallel). > >>> The same traffic level, with 1500 byte packets generates 6 times more >>> packets per second than that traffic level with 9000 bytes packets. >> >> You divided 9000 by 1500 without mistakes. Congratulations. >> > > The point was clearly lost on you. > >>> There is ongoing work to improve the network stack performance on boxes >>> like the APU2 (which have 4 cores). You will see improvements. If you want >>> it better today, you need a faster box. Chris >> >> The apu2c4 is fast enough to saturate its Intel 1Gbits/sec link. It has >> three of those. If you connect all three to the switch, you get 3Gbps shy. >> No need for a faster box. You rather need a faster switch, class 7 S-FTP >> wires (better than class 6), and 2.5Gbps lan cards for clients. > > No, you don't need any of that. You have no idea what you are talking about. > > The APU requires software crafted to evenly distribute PER-PACKET PROCESSING > cost across multiple cores. That is what is happening in OpenBSD today. It has > been happening for years, and it is getting closer to becoming a reality with > OpenBSD + APU2, as well as other chipsets/platforms. > > For a couple years now, we've had interrupts processed by one core, PF on > another, and other parts of the kernel on a third core. But to accelerate > packet processing alone, we need interrupts handled on multiple cores, > PF processing handled on multiple cores. This is hard work. > > By the way, what I'm describing is the general-purpose OS approach towads > this problem. If you want to turn computer hardware into routers with little > other concern, the go-to platform is DPDK + VPP. It is something like an > order of magnitude faster than any general purpose OS (OpenBSD, Linux) at > packet pushing. > > https://www.reddit.com/r/networking/comments/6upchy/can_a_bsd_system_replicate_the_performance_of/dlvdq2e/ > > Chris Thank you for this explanation. My uplink is only 240mbit and my APU2 handles that perfectly, so I’m not having any of these problems. But the insight into the current state of networking was great! :) Peter
Re: Cheap 2x NIC OpenBSD device
Do you mean it runs OpenBSD by default, or you can install OpenBSD? I have a Ubiquiti UniFi and it runs Linux. The Edgerouter Lite looks like a cool little piece of hardware, good tip! > On Nov 1, 2017, at 11:36 AM, Sean Murphy wrote: > > Check out the Ubiquiti Edgerouter Lite. Sub $100 (US), three NICs, > and runs OpenBSD. > > I've used it as a router, firewall, dhcp server, you name it. Versatile > device. > > On Wed, Nov 1, 2017 at 10:27 AM, Alex Waite wrote: >> I'm deploying a server to a different data center and I don't want to expose >> the IPMI interface of the machine to their semi-trusted management network. >> So, I'm planning on putting a simple OpenBSD device in front of it, logging >> and filtering. >> >> Can someone here recommend a relatively cheap (< ~100 EUR) device that runs >> OpenBSD and has 2 NICs? >> >> ---Alex >> >
Re: Cheap 2x NIC OpenBSD device
I have an APU2 from PC Engines, which has 3 gigabit ports. I think it’s a bit above your budget of €100, but if you can’t find anything else I highly recommend it. I use one as my edge firewall and haven’t had any problems. > On Nov 1, 2017, at 07:27, Alex Waite wrote: > > I'm deploying a server to a different data center and I don't want to expose > the IPMI interface of the machine to their semi-trusted management network. > So, I'm planning on putting a simple OpenBSD device in front of it, logging > and filtering. > > Can someone here recommend a relatively cheap (< ~100 EUR) device that runs > OpenBSD and has 2 NICs? > > ---Alex >
Re: DMCA Free OpenBSD VPS Hosting, multiple payment methods
You use OpenBSD, so why are you worried about DMCA? That is, you must care about security so you’re already using aggressive blocklists, encrypted peers only, etc etc. A well configured torrent client leaks very little info. Unless laws have changed and you don’t need any proof of wrongdoing besides lots of peer to peer bandwidth to get investigated? In that case, just ignore me. I don’t torrent anymore so I could be out of date. Actually if anyone knows more about the current status quo I’d like to hear more just out of curiosity. The discussion about the Netherlands has been tremendously interesting to me. Only asking since I’ve found other OpenBSD people do tend to be more astute and well informed than me! >> On Oct 19, 2017, at 13:12, Mike wrote: >> >> On 10/19/2017 11:36 AM, Michael Hekeler wrote: >> Am Thu, 19 Oct 2017 16:32:34 +0200 >> schrieb "Christoph R. Murauer" : >> >>> To the other things spoken here (which I don't quote to keep it more >>> short). Hetzner is a German company, which is part of the EU. There >>> are not so many OpenBSD friendly hoster outside the USA and the EU. >> >> At the risk of sounding stupid, what is an "OpenBSD friendly hoster"? > > For me, that's an easy answer. > > An "OpenBSD friendly hoster" is one who knows you are running an OpenBSD > VPS, and doesn't suggest you change iptables settings when talking about > your firewall with their support team. >
Re: php-fpm and OpenBSD 6.2
> On Oct 12, 2017, at 00:39, Markus Rosjat wrote: > > Hi there, > > I can't find a php-fpm package under 6.2 but there are php-fastcgi packages. > Is this the new php-fpm naming convention starting with 6.2 or do I get this > wrong here? > > regards > > -- > Markus Rosjatfon: +49 351 8107223mail: ros...@ghweb.de > > G+H Webservice GbR Gorzolla, Herrmann > Königsbrücker Str. 70, 01099 Dresden > > http://www.ghweb.de > fon: +49 351 8107220 fax: +49 351 8107227 > > Bitte prüfen Sie, ob diese Mail wirklich ausgedruckt werden muss! Before you > print it, think about your responsibility and commitment to the ENVIRONMENT > There is no php-fpm package, fpm is built in the plain php package. There is an effort to split php into more granular packages, including a php-fpm package, but it didn’t make it into 6.2. You can read more about the repackaging effort on the ports mailing list; the thread was updated just yesterday. I believe php-fastcgi is a legacy module of some kind, and fpm is the preferred way to run php. So you just need the plain php package that comes with fpm. Peter
Re: Read sysctl from file
> On Jul 21, 2017, at 1:30 PM, Mihai Popescu wrote: > >> Also it does not fail halfway, it will report errors for each of the >> settings that cannot > be applied, > > So Peter, just to check if i got it right, you did a script who > reports errors about things people knows in advance they will generate > errors, that despite the warnings that the concept is wrong from the > start till the end. > > Nice, I remember about a Dilbert situation, but I'm too lazy to search for it. Yes, my script reports errors. Since apparently everyone knows in advance which sysctls will produce errors, it is bad for me to report them? Is that what you're saying? Well then you should submit a patch to /sbin/sysctl that stops reporting errors. After all, as you say, "people knows in advance they will generate errors.” Now THAT is a Dilbert situation. Peter
Re: Read sysctl from file
> On Jul 21, 2017, at 3:42 PM, li...@wrant.com wrote: > > Fri, 21 Jul 2017 12:33:31 -0700 Peter Faiman >> # ./sysctl -p example.conf >> Peter > > Hi Peter, ansibles, > > No guarantee systems controls stay affixed, wrapper tools comply got it? The point of sysctl -p is reloading from a file. So that you put controls in the file and load that file, exactly as happens in system startup. The whole point is to ensure consistency with system startup. True, securelevel throws a bit of a wrench in that, but this works for all other settings. > Wrap around as advised for a system operator, don't push for short cuts. It’s not a short cut. Ansible wants sysctl -p, I implemented sysctl -p exactly as Linux does it, using the OpenBSD /etc/rc code that actually applies sysctls from /etc/sysctl.conf. I never said anyone should use Ansible. I don't use it, I don't like it. But clearly this person is going to use it, so I might as well give them something that will do what they want, even if I don't agree with it. > Please, stop imposing your designs on our systems wasting precious time. I'm not imposing my designs on anyone. Someone on the mailing list needed the exact Linux behavior, so I spent 5 minutes on the train to work writing and testing a compatible tool. I already _specifically_ said I wrote a wrapper this way because it's the easiest way to be compatible without changing ANY OpenBSD code, or ANYTHING else about the OpenBSD system. In other words I deliberately chose to solve this problem in a way that imposes NOTHING on anyone else. > Kind regards, You should stop putting this at the bottom of your emails if you think it's acceptable to talk to others this way. When you send out half-baked responses that clearly demonstrate you did not bother to read what I said, you're the one wasting my time. Peter
Re: Read sysctl from file
> On Jul 21, 2017, at 12:22 PM, Theo de Raadt wrote: > >>> On Jul 21, 2017, at 3:47 AM, Stuart Henderson = >> wrote: >>> =20 >>> On 2017-07-20, BARDOU Pierre wrote: Is there a way to make sysctl re-read its conf file, or even another = >> file, like sysctl -p does on linux systems ? Supporting this option would be nice, as it is used by the sysctl = >> module of ansible. >>> =20 >>> Sounds risky. It won't reset default values thag are unspecified in >>> sysctl.conf, so you could be sitting on a configuration that appears = >> ok, >>> but will fail after a reboot. >> >> Stuart makes a good point. So does Theo, adding -p to the sysctl binary = >> when >> it doesn't currently do any file handling at all seems extreme. So I = >> wrote a >> wrapper script that emulates Linux sysctl -p. I put very simple = >> directions >> at the top of the file. Find it here: >> >> https://gist.github.com/PeterFaiman/5b67c530b0ffa009ebef904ed0678e26 >> >> Ideally these tools wouldn't use Linux-specific features. But emulating >> simple features like sysctl -p in a non-invasive way isn't too hard. > > One more point to add: > > Some setting can only be changed before securelevel. They fail afterwards. > > I'd say the entire approach is wrong, because it cannot tell them apart. > It will fail halfway. > > It was obviously written by people who don't care. True, there is no way to get around the securelevel problem without rebooting, by definition. But if this MUST be done with these workflow constraints, I think this is the "best" way to do it. Also it does not fail halfway, it will report errors for each of the settings that cannot be applied, e.g. with a config that sets kern.securelevel=0 and net.inet.udp.sendspace=9216, this happens: # ./sysctl -p example.conf sysctl: kern.securelevel: Operation not permitted net.inet.udp.sendspace: 9216 -> 9216 Peter
Re: Read sysctl from file
> On Jul 21, 2017, at 3:47 AM, Stuart Henderson wrote: > > On 2017-07-20, BARDOU Pierre wrote: >> Is there a way to make sysctl re-read its conf file, or even another file, >> like sysctl -p does on linux systems ? >> Supporting this option would be nice, as it is used by the sysctl module of >> ansible. > > Sounds risky. It won't reset default values thag are unspecified in > sysctl.conf, so you could be sitting on a configuration that appears ok, > but will fail after a reboot. Stuart makes a good point. So does Theo, adding -p to the sysctl binary when it doesn't currently do any file handling at all seems extreme. So I wrote a wrapper script that emulates Linux sysctl -p. I put very simple directions at the top of the file. Find it here: https://gist.github.com/PeterFaiman/5b67c530b0ffa009ebef904ed0678e26 Ideally these tools wouldn't use Linux-specific features. But emulating simple features like sysctl -p in a non-invasive way isn't too hard. Peter
Re: For the super paranoid
There is no way hardware supported way to do this on mainstream Intel / AMD. Yes it's possible to make a chip that could do it. No it's not reasonable, it would destroy performance without really helping that much. If you are facing an adversary powerful enough to have access to your RAM sticks, it's cheaper and more effective to self destruct. It's definitely paranoid, in the sense that it's crazy and ridiculous. If you really want to encrypt part of memory you probably could patch a kernel to reserve some ring 0 restricted registers to store keys. But the hardware just does not support running in a full encrypted way, there are certain things that must be put in main memory unencrypted for the processor to use. > On Mar 11, 2017, at 08:44, Luke Small wrote: > > Is there a way to encrypt memory and keep the key on the CPU like a > transparent partition so that if the ram cards are physically accessed, hey > can't be read? Is it reasonable?
Re: AP using AR9287 working yesterday, broken today.. How to diagnose?
>> add athn0 > > If i recall correctly, from some discussion on misc@, you cannot use a > wireless interface in a bridge ( athn0 or all, I'm not sure). But > maybe I say something wrong, search the archive. > You certainly can have a wireless device in a bridge, this is how my current hostap athn card is set up.
Re: relayd[66834]: relayd: socketpair: Too many open files
Ah yes I see those lines now, thank you. Kevin, what version of OpenBSD are you using? You mentioned this is a new project so I assume 6.0? Peter On Jan 5, 2017, at 10:08, Theo de Raadt wrote: >> Hmm. The default number of files is 128 for daemons, but it's strange you'd >> hit that JUST starting up. >> >> Can you try starting relayd with -v -d to see if it logs anything of >> interest? >> >> Can you binary search ulimits until you find the lowest it will start with? >> >> Reading the source it looks like socket pairs are created between all the >> relayd processes, i.e. n^2 * 2 ish file descriptors, which could exceed 128 >> pretty fast. Are you running with a non-default prefork setting? > > This was fixed after 6.0. > > date: 2016/11/24 21:01:18; author: reyk; state: Exp; lines: +110 -79; commitid: FkVuQgzULddApn9S; > The new fork+exec mode used too many fds in the parent process on > startup, for a short time, so we needed a rlimit hack in relayd.c. > Sync the fix from httpd: rzalamena@ has fixed proc.c and I added the > proc_flush_imsg() mechanism that makes sure that each fd is > immediately closed after forwarding it to a child process instead of > queueing it up. > > OK rzalamena@ jca@ benno@
Re: relayd[66834]: relayd: socketpair: Too many open files
Hmm. The default number of files is 128 for daemons, but it's strange you'd
hit that JUST starting up.
Can you try starting relayd with -v -d to see if it logs anything of
interest?
Can you binary search ulimits until you find the lowest it will start with?
Reading the source it looks like socket pairs are created between all the
relayd processes, i.e. n^2 * 2 ish file descriptors, which could exceed 128
pretty fast. Are you running with a non-default prefork setting?
Peter
> On Jan 5, 2017, at 09:12, Kevin wrote:
>
> Nope. I was hoping for another solution, especially given that:
>
> 1. the only thing runnings on this machine are pf and relayd
> 2. there's zero traffic going to it at present
> 3. there's only one site being load balanced
>
> it seems like it shouldn't be necessary.
>
> I'm open to it, if that's the only choice, but it strikes me as outside of
the bounds of normal operation.
>
>> On Thu, Jan 5, 2017 at 9:07 AM, Peter Faiman
wrote:
>> Have you modified your open file limits in /etc/login.conf? Especially in
the daemon section?
>>
>> Peter
>>
>> > On Jan 5, 2017, at 08:50, Kevin wrote:
>> >
>> >> On Tue, Jan 3, 2017 at 1:16 PM, Kevin wrote:
>> >>
>> >> Hey gang,
>> >>
>> >> So I'm putting a new firewall in place and have run into issues with
>> >> getting relayd to start using:
>> >>
>> >> # /etc/rc.d/relayd start
>> >>
>> >> When I try starting it like that inevitably I get:
>> >>
>> >>relayd(failed)
>> >>
>> >> checking the log files tells me:
>> >>
>> >>relayd: socketpair: Too many open files
>> >>
>> >> Having trolled through pages of SERPs, I can't find an answer; however,
in
>> >> the interest of science, if I do this:
>> >>
>> >> # ulimit -n 512
>> >> # /usr/sbin/relayd
>> >>
>> >> it starts perfectly.
>> >>
>> >> Anyone care to give me a quick strike with the clue stick, please?
>> >>
>> >> Oh yah, here's my relayd.conf
>> >>
>> >> # Example.com
>> >> # 145.176.20.136
>> >> exm_chi01="192.168.2.0"
>> >> exm_chi02="192.168.2.1"
>> >>
>> >> table{ $exm_chi01, $exm_chi02 }
>> >>
>> >> #=#
>> >> # Servers #
>> >> #=#
>> >> redirect "Example.com" {
>> >>listen on 145.176.20.162 port 80 interface vio0
>> >>pftag RELAYD-Example.com
>> >>forward to check tcp
>> >> }
>> >>
>> >>
>> >> For what it's worth, I'm using a hosts file to point example.com to my
IP
>> >> for the time being, as I can't pull the real sites down and move them
'til
>> >> this is working.
>> >>
>> >> Also of interest: pf seems to be working as advertised, as does relayd
>> >> when it's started with the ulimit cranked up.
>> >>
>> >>
>> >> Thanks,
>> >> Kevin
>> >>
>> >
>> >
>> >
>> > Unless there's word to the contrary, and as much as it's not officially
the
>> > right thing to do, it seems the only real choice for me here is to run
>> > relayd with ulimit sufficiently cranked, eh?
Re: relayd[66834]: relayd: socketpair: Too many open files
Have you modified your open file limits in /etc/login.conf? Especially in the
daemon section?
Peter
> On Jan 5, 2017, at 08:50, Kevin wrote:
>
>> On Tue, Jan 3, 2017 at 1:16 PM, Kevin wrote:
>>
>> Hey gang,
>>
>> So I'm putting a new firewall in place and have run into issues with
>> getting relayd to start using:
>>
>> # /etc/rc.d/relayd start
>>
>> When I try starting it like that inevitably I get:
>>
>>relayd(failed)
>>
>> checking the log files tells me:
>>
>>relayd: socketpair: Too many open files
>>
>> Having trolled through pages of SERPs, I can't find an answer; however, in
>> the interest of science, if I do this:
>>
>> # ulimit -n 512
>> # /usr/sbin/relayd
>>
>> it starts perfectly.
>>
>> Anyone care to give me a quick strike with the clue stick, please?
>>
>> Oh yah, here's my relayd.conf
>>
>> # Example.com
>> # 145.176.20.136
>> exm_chi01="192.168.2.0"
>> exm_chi02="192.168.2.1"
>>
>> table{ $exm_chi01, $exm_chi02 }
>>
>> #=#
>> # Servers #
>> #=#
>> redirect "Example.com" {
>>listen on 145.176.20.162 port 80 interface vio0
>>pftag RELAYD-Example.com
>>forward to check tcp
>> }
>>
>>
>> For what it's worth, I'm using a hosts file to point example.com to my IP
>> for the time being, as I can't pull the real sites down and move them 'til
>> this is working.
>>
>> Also of interest: pf seems to be working as advertised, as does relayd
>> when it's started with the ulimit cranked up.
>>
>>
>> Thanks,
>> Kevin
>>
>
>
>
> Unless there's word to the contrary, and as much as it's not officially the
> right thing to do, it seems the only real choice for me here is to run
> relayd with ulimit sufficiently cranked, eh?
