Re: [pfSense] NIC support

[Jim Thompson]

> On Oct 15, 2014, at 5:01 PM, compdoc  wrote:
> 
> > I am well-aware of Olivier’s work in this area, as are many in the FreeBSD 
> > community.
>  
> You’ve failed to disprove anything I've said, even the part about tools.


I'm not going to argue with an individual who defines terms to suit his 
position. 
>  
> > You’re still assigning fault to pfSense
>  
> Not at all.

I see.  "It crashes!", but no detail forthcoming. 

> But it would be nice if any of this pleasant banter becomes useful by pushing 
> someone to actually try this type of testing, to find out why it happens. And 
> if not, oh well...

We're not quite ready to publish the results, (because we want people to be 
able to reproduce them, and maybe put an end to this "benchmarking as a 
sport"), but yes, the testing is certainly taking place. 

> By the way, does the C2758  hardware sold by pfSense include pps performance 
> information? Has anyone with this hardware tested it? (speaking to others who 
> might be reading this)
>  
> You suggest it can operate at near 'wirespeed', or at least that the OP will 
> be very happy with a C2758 , but you’ve not proven it.

There is no proof, except that which is documented and reproducible.  We're 
doing something like science here. 

Jim

___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list

Re: [pfSense] NIC support

[compdoc]

> I am well-aware of Olivier’s work in this area, as are many in the FreeBSD 
> community.

 

You’ve failed to disprove anything I've said, even the part about tools. 

 

 

> You’re still assigning fault to pfSense

 

Not at all. But it would be nice if any of this pleasant banter becomes useful 
by pushing someone to actually try this type of testing, to find out why it 
happens. And if not, oh well...

 

By the way, does the C2758  hardware sold by pfSense include pps performance 
information? Has anyone with this hardware tested it? (speaking to others who 
might be reading this)

 

You suggest it can operate at near 'wirespeed', or at least that the OP will be 
very happy with a C2758 , but you’ve not proven it. 

 

 

___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list

Re: [pfSense] NIC support

[Jim Thompson]

> On Oct 15, 2014, at 4:06 PM, compdoc  wrote:
> 
> > There has been some testing using BSDRP, but it is not "a tool to test 
> > hardware".
>  
> I used it as a tool to benchmark my hardware. There are several examples on 
> their website of using it for just that purpose.

I am well-aware of Olivier’s work in this area, as are many in the FreeBSD 
community.
 
> >You were testing forwarding, by the look of it.   This is not all there is 
> >to routing.
>  
> The testing results I posted were pure packets per second without forwarding. 
> I also tested forwarding but did not post the results, and I mentioned that.

So this (“pure packets per second without forwarding”) reduces to just “testing 
netmap”.

> >>However, I will mention one thing: if you try to route 1.488M packets per 
> >>second through the 'generic' pfSense, it will crash after a minute or so.
> > 
> >That's an interesting result.  We've not seen it. 
>  
> These crashes happened during a forwarding test using pfSense. I disabled 
> packet filtering to try to lessen overhead, but it doesn’t seem that pfSense 
> is designed to push a great flood of very tiny packets for any length of 
> time, in one interface and out another. 
>  
> And I don’t fault it for that. For normal types of traffic, it’s a very 
> capable firewall. It would be interesting to know your results.

You’re still assigning fault to pfSense, haven’t properly documented what 
you’re seeing (thus your assertion that this is pfSense, rather than something 
in your hardware or in
the testing environment) is not well-supported) and haven’t even answered my 
questions asking for more detail.

I am also well-aware of the performance issues with pf.   We’re working on it.  
You may have missed the blog post yesterday (https://blog.pfsense.org/?p=1473 
).

Jim___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list

Re: [pfSense] NIC support

[compdoc]

> There has been some testing using BSDRP, but it is not "a tool to test 
> hardware".

 

I used it as a tool to benchmark my hardware. There are several examples on 
their website of using it for just that purpose.

 

It also a tool to build simple routers. It has very little overhead and runs on 
freebsd, which made it interesting. 

 

It is a tool.

 

 

>You were testing forwarding, by the look of it.   This is not all there is to 
>routing.

 

The testing results I posted were pure packets per second without forwarding. I 
also tested forwarding but did not post the results, and I mentioned that.

 

 

>>However, I will mention one thing: if you try to route 1.488M packets per 
>>second through the 'generic' pfSense, it will crash after a minute or so.

> 

>That's an interesting result.  We've not seen it. 

 

These crashes happened during a forwarding test using pfSense. I disabled 
packet filtering to try to lessen overhead, but it doesn’t seem that pfSense is 
designed to push a great flood of very tiny packets for any length of time, in 
one interface and out another. 

 

And I don’t fault it for that. For normal types of traffic, it’s a very capable 
firewall. It would be interesting to know your results.

 

 

 

___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list

Re: [pfSense] NIC support

[Jim Thompson]

-- Jim
> On Oct 15, 2014, at 10:06 AM, compdoc  wrote:
> 
> > When I speak of the C2758, I speak of the product sold at the pfSense store,
> > as sold by the pfSense store, not the generic pfsense release running on 
> > "some
> >brand of board@.
>  
> I was speaking of a C2758 board that was tested by someone else, and which 
> wasn’t able to reach Ethernet's maximum throughput. Clearly not all C2758 
> boards are the same. Buyer beware.
>  
> If you have tests results that prove the product you mentioned doesn’t have 
> this problem, feel free to post them. I'd love to see.
>  
>  
> > You seem confused. 
>  
> Not at all. You seem defensive.
>  
>  
> >- this list is about pfsense, not the BSDRP
>  
> Never said it was. BSDRP is a tool to test hardware.

Actually it's not.  Olivier uses it in his work at Orange. 
There has been some testing using BSDRP, but it is not "a tool to test 
hardware".

> If the hardware cannot achieve maximum throughput, then pfSense cannot 
> achieve maximum throughput.

This is a true statement but it ignores the reality that software also plays a 
part. 

> > Pkt-gen does not test routing.  What tests did you run?
>  
> Here's a clue:  BSD *Router* Project. I doubt you’ve done this sort of 
> testing, so I'm not going to spoil this learning opportunity for you...

You seem defensive.

You were testing forwarding, by the look of it.   This is not all there is to 
routing.  I will not further ecludiate because you are obviously an expert. 

While you "doubt" we "have done this sort of testing" you should look at: 
https://github.com/gvnn3/conductor

Quoting README

   [...]
A common use for Conductor is to test a network devices, such as a router or 
firewall, that is connected to multiple senders and receivers.  Each of the 
senders, receivers, and the device under test
(DUT) are a Player, and another system is designated as the Conductor.

[...]
 
This work supported by: Rubicon Communications, LLC (Netgate)
Conductor uses pkt-gen or iperf, though our preference going forward is 
pit-gen. Recent additions to pkt-gen include playback of pcap files, for more 
repeatable testing.  It's also important to be able to test multiple senders 
and receivers.  I will not further ecludiate because you are an expert. 

> However, I will mention one thing: if you try to route 1.488M packets per 
> second through the 'generic' pfSense, it will crash after a minute or so. 
> (and that's not a criticism of pfSense)

That's an interesting result.  We've not seen it. 
Which particular hardware were you using?
Which version of pfsense?
Any tunables?
What switches to pkt-gen?

>  
> >I don't see where a C2758 is tested. 
>  
> I clearly stated what I was testing and how. You seem confused. The OP was 
> asking what hardware might serve his purpose. I offered suggestions.
>  
> You're welcome to prove anything I've said was wrong - but with actual test 
> results, and without the misplaced rancor.
>  
> Also, it's better to reply to the list, and not send emails directly to me.
>  
>  
> ___
> List mailing list
> List@lists.pfsense.org
> https://lists.pfsense.org/mailman/listinfo/list
___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list

Re: [pfSense] NIC support

[compdoc]

> When I speak of the C2758, I speak of the product sold at the pfSense store, 

> as sold by the pfSense store, not the generic pfsense release running on 
> "some 

>brand of board@.

 

I was speaking of a C2758 board that was tested by someone else, and which 
wasn’t able to reach Ethernet's maximum throughput. Clearly not all C2758 
boards are the same. Buyer beware. 

 

If you have tests results that prove the product you mentioned doesn’t have 
this problem, feel free to post them. I'd love to see.

 

 

> You seem confused. 

 

Not at all. You seem defensive.

 

 

>- this list is about pfsense, not the BSDRP

 

Never said it was. BSDRP is a tool to test hardware. If the hardware cannot 
achieve maximum throughput, then pfSense cannot achieve maximum throughput.

 

 

> Pkt-gen does not test routing.  What tests did you run?

 

Here's a clue:  BSD *Router* Project. I doubt you’ve done this sort of testing, 
so I'm not going to spoil this learning opportunity for you...

 

However, I will mention one thing: if you try to route 1.488M packets per 
second through the 'generic' pfSense, it will crash after a minute or so. (and 
that's not a criticism of pfSense)

 

 

>I don't see where a C2758 is tested. 

 

I clearly stated what I was testing and how. You seem confused. The OP was 
asking what hardware might serve his purpose. I offered suggestions. 

 

You're welcome to prove anything I've said was wrong - but with actual test 
results, and without the misplaced rancor. 

 

Also, it's better to reply to the list, and not send emails directly to me.

 

 

___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list

Re: [pfSense] pfsense crash dump

[Espen Johansen]

This can be several things. Bad controller/memory on the Controller. Bad
BBU. Or simply bad drive(s).
Also check if this occurs when the controller performs BBU tests. (If the
BBU is bad then the controller switches to write thru mode and strange
things can happen).

HTH.
13. okt. 2014 19:27 skrev "Mark Loza"  følgende:

>  Does this something have to do with faulty PERC controller?
>
> On 10/14/14 1:29 AM, Mark Loza wrote:
>
> Yes, a hardware raid and pfsense is physically running on a Dell PE R515
> machine.
>
> On 10/14/14 12:49 AM, Espen Johansen wrote:
>
> Is this a RAID?
> Seen this on dells with PERC/megaraid controllers when they run the
> sceduled BBU test.
> 13. okt. 2014 18:44 skrev "Mark Loza"  følgende:
>
>>  Hi, pfsense is running fine for now. Is there any pfsense package that
>> I can perform a live test on the drive?
>>
>> On 10/14/14 12:09 AM, Aaron C. de Bruyn wrote:
>>
>> To me, it looks like a disk issue:
>>
>>  mfi0: 35354 (465709273s/0x0002/info) - Patrol Read corrected medium error 
>> on PD 02(e0x20/s2) at 1692f3e4
>> mfi0: 35355 (465709275s/0x0002/info) - Unexpected sense: PD 02(e0x20/s2) 
>> Path 539358c92146, CDB: 2f 00 16 92 f3 e5 00 10 00 00, Sense: 1/00/00
>>
>> You might want to download something like "The Ultimate Boot CD" and use the 
>> manufacturers test tools on your drive.
>>
>> -A
>>
>>
>> On Sun, Oct 12, 2014 at 11:43 PM, Mark Loza  wrote:
>>
>>> Hi,
>>>
>>> Can anyone happen to know what's of this crash dump in pfsense
>>> http://sprunge.us/CGDH ? Actually, this already happened twice, the
>>> first crash happened approximately 30 days ago and second occurred
>>> yesterday. I suspect this might be a disk issue. Thanks in advance to those
>>> who would me determine the real cause.
>>>
>>>
>>>
>>> ___
>>> List mailing list
>>> List@lists.pfsense.org
>>> https://lists.pfsense.org/mailman/listinfo/list
>>>
>>
>>
>>
>> ___
>> List mailing 
>> listList@lists.pfsense.orghttps://lists.pfsense.org/mailman/listinfo/list
>>
>>
>>
>> ___
>> List mailing list
>> List@lists.pfsense.org
>> https://lists.pfsense.org/mailman/listinfo/list
>>
>
>
> ___
> List mailing 
> listList@lists.pfsense.orghttps://lists.pfsense.org/mailman/listinfo/list
>
>
>
>
> ___
> List mailing list
> List@lists.pfsense.org
> https://lists.pfsense.org/mailman/listinfo/list
>
___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list

Re: [pfSense] NIC support

[Jim Thompson]

> On Oct 14, 2014, at 5:15 PM, compdoc  wrote:
> 
> >as close to wirespeed as possible, be happy with a C2758. ?
> > 
> >Very
>  
>  
> That C2758 has nice specs and should be able to keep up, however there seems 
> to be a throughput problem on at least one brand of board running the C2758.

When I speak of the C2758, I speak of the product sold at the pfSense store, as 
sold by the pfSense store, not the generic pfsense release running on "some 
brand of board@.

> (I think it’s more a problem with the nics than the cpu)

You seem confused. 

> I recently tested various nics and cpus to see if the systems I was building 
> could reach Gigabit Ethernet's max throughput of  1.488Mpps on one port

Please show your work.  Which pkt-gen switches are in use?

> Tests were run on AMD FM1+ and AM1 APUs, an FX-4100, and an Intel i5-2400 
> Sandy Bridge.
None of these is the system in question.  They don't even run the same cpu. 

> Tests used the BSD Router Project (BSDRP) OS, and a program named 'pkt-gen'.

- I am quite familiar with pkt-gen.  

- this list is about pfsense, not the BSDRP

> During routing tests, I found that an AMD A8-7600 Kaveri was the only cpu I 
> had that was equal in performance to the Intel i5-2400. (the routing tests 
> involved a 3rd test machine, and aren't covered in the scores below)

Pkt-gen does not test routing.  What tests did you run?

> Anyway, I hope you find this helpful...

I don't see where a C2758 is tested. 

> In these tests, I used the two fastest test machines connected to each other. 
> One sends, and one receives:
>  
> Realtek  8169sc 32-bit PCI card
> 266935 pps (283752 pkts in 1063001 usec)
> Speed: 267.19 Kpps Bandwidth: 128.25 Mbps (raw 179.55 Mbps)
>  
> Realtek RTL8111DL, Onboard
> 405708 pps (406113 pkts in 1000998 usec)
> Speed: 404.78 Kpps Bandwidth: 194.29 Mbps (raw 272.01 Mbps)
>  
> Intel pro 1000 32-bit PCI card
> 307102 pps (307586 pkts in 1001577 usec)
> Speed: 276.49 Kpps Bandwidth: 132.72 Mbps (raw 185.80 Mbps)
>  
> Intel Pro 1000, x1 PCI-e card (no heatsink)
> 1367299 pps (1453440 pkts in 1063001 usec)
> Speed: 1.36 Mpps Bandwidth: 654.85 Mbps (raw 916.79 Mbps)
>  
> Intel Pro 1000, x1 PCI-e card, server version (with heatsink)
> 1488012 pps (1490981 pkts in 1001995 usec)
> Speed: 1.49 Mpps Bandwidth: 714.23 Mbps (raw 999.92 Mbps)
>  
> Intel PRO/1000 PT, Dual Port, 4x PCI-e, Server Adapter  (with heatsink)
> 1488012 pps (1490981 pkts in 1001995 usec)
> Speed: 1.49 Mpps Bandwidth: 714.23 Mbps (raw 999.92 Mbps)
>  
>  
> ***
>  
> These tests were using the lowest TDP(watt) APUs I had.

APUs?   I thought we were talking C2758. 

> The Intel server nics were the fastest nics tested, and used the least cpu 
> time, so I used those in these tests:
>  
> AMD 5150 quad core APU @ 1.6GHz
> Intel PRO/1000 PT, Dual Port, 4x PCI-e, Server Adapter  (with heatsink)
> 1179367 pps (1180530 pkts in 1000986 usec)
> Speed: 1.17 Mpps Bandwidth: 562.85 Mbps (raw 787.99 Mbps)

AMD CPU.  NON-identified NIC. 

> AMD 5350 quad core APU @ 2GHz
> Intel PRO/1000 PT, Dual Port, 4x PCI-e, Server Adapter  (with heatsink)
> 1488106 pps (1489615 pkts in 1001014 usec)
> Speed: 1.48 Mpps Bandwidth: 709.33 Mbps (raw 993.07 Mbps)

AMD CPU.  NON-identified NIC.

> AMD 5350 quad APU @ 2GHz
> Onboard RTL8111/8168B PCI Express Gigabit Ethernet controller
> 560938 pps (561565 pkts in 1001117 usec)
> Speed: 558.35 Kpps Bandwidth: 268.01 Mbps (raw 375.21 Mbps)

AMD CPU.  NON-identified NIC.

> AMD A4-6300 dual core APU @ 3.7GHz
> Intel PRO/1000 PT, Dual Port, 4x PCI-e, Server Adapter  (with heatsink)
> 1129784 pps (1130961 pkts in 1001042 usec)
> Speed: 1.09 Mpps Bandwidth: 521.00 Mbps (raw 729.39 Mbps

AMD CPU.  NON-identified NIC. 

Now the track has been completely lost. 

Jim

___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list

Re: [pfSense] NIC support

[Chris L]

On Oct 15, 2014, at 12:59 AM, Ulrik Lunddahl  wrote:

> Will A SMB without L3 capable switches, that needs routing between 3-4 local 
> subnets (LAN, SERVERS, WIRELESS/GUEST, OTHER/DMZ) as close to wirespeed as 
> possible, be happy with a C2758. ?
>  
> Very.  
>  
> Is a dual socket Xeon a bit faster? Yes.  
> Does your application need that speed? Unlikely. 
> 
> Really depends on what you mean by "wirespeed". 
>  
> The case I always seem to run into is Clients on the LAN, moving a bulk 
> amount of data to/from NAS devices on the SERVER or DMZ subnet, that is 
> typically backup data or data that are somewhat being replicated.
>  
> I work a lot with companies dealing in media, and RAW images and/or video is 
> very huge, and devices to store it on is dead cheap.
>  
> I also work a lot with virtual environments; backup and replication of 
> virtual machines also generate huge files, which need to be transferred as 
> fast as possible.
>  
> So having a hardware router that can both handle internet access from the 
> many LAN clients, and hours of forwarding at interface speed between a few 
> other interfaces is what I would like.
>  
> Let’s say that we have a Intel Rangeley Atom 8-core C2758 box with 5 
> interfaces. (WAN, LAN, SERVERS, OPT1, OPT2)
>  
> Will it be able to handle forwarding the packets generated from copying 
> approx. 1 TB of files from LAN to SERVERS and OPT1 to OPT2, and services 50 
> computers + 50 phones with heavy internet usage.
>  
> NAT only, very few rules. ?
>  
> I ask because I have no idea how powerful the new Atom’s is.
> 

My first thoughts are:

What is the threat profile you are facing in your organization?  Why do you 
need a firewall between your users and your NAS?

I, personally, would not put pfSense in that duty.  If firewalling was not 
necessary, I’d use a layer 3 switch.  And with only 100 devices plus a few 
servers, I’d wonder why layer 2 wouldn’t suffice.

___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list

Re: [pfSense] NIC support

[Ulrik Lunddahl]

Will A SMB without L3 capable switches, that needs routing between 3-4 local 
subnets (LAN, SERVERS, WIRELESS/GUEST, OTHER/DMZ) as close to wirespeed as 
possible, be happy with a C2758. ?

Very.

Is a dual socket Xeon a bit faster? Yes.
Does your application need that speed? Unlikely.

Really depends on what you mean by "wirespeed".

The case I always seem to run into is Clients on the LAN, moving a bulk amount 
of data to/from NAS devices on the SERVER or DMZ subnet, that is typically 
backup data or data that are somewhat being replicated.

I work a lot with companies dealing in media, and RAW images and/or video is 
very huge, and devices to store it on is dead cheap.

I also work a lot with virtual environments; backup and replication of virtual 
machines also generate huge files, which need to be transferred as fast as 
possible.

So having a hardware router that can both handle internet access from the many 
LAN clients, and hours of forwarding at interface speed between a few other 
interfaces is what I would like.

Let’s say that we have a Intel Rangeley Atom 8-core C2758 box with 5 
interfaces. (WAN, LAN, SERVERS, OPT1, OPT2)

Will it be able to handle forwarding the packets generated from copying approx. 
1 TB of files from LAN to SERVERS and OPT1 to OPT2, and services 50 computers + 
50 phones with heavy internet usage.

NAT only, very few rules. ?

I ask because I have no idea how powerful the new Atom’s is.


- Ulrik Lunddahl




___
List mailing list
List@lists.pfsense.org
https://lists.pfsense.org/mailman/listinfo/list