Re: [pfSense] 10GBASE-T hardware

[Yehuda Katz]

I agree with everything my brother said except recommending the Uniquiti
EdgeSwitch.
We have seen a few instances of the EdgeSwitch locking up without any
apparent reason (once we traced it to a thermal issue, but we couldn't find
a cause for the others).
The EdgeSwitch also only has a 1 year warranty while the Netgear you
mentioned has a Lifetime Warranty (for whatever that is worth).
At (insert university name here) we were happily standardizing on Brocade
ICX switches until we hit major OSPF firmware bugs. Dell N and S series are
good, but also more expensive than that Netgear.

- Y

On Tue, Mar 27, 2018 at 8:10 PM, Moshe Katz  wrote:

> According to the specs that I found on HP's website, your HP switch does
> not support 10Gb, only 1Gb on its mini-GBIC ports. You will definitely need
> a new switch to take advantage of 10Gb.
>
> If you do get a switch that supports 10GBase-T, you should definitely
> consider the Intel X540. The vast majority of reports that I have seen say
> that it works great. (There was one report I found on a forum claiming
> performance issues, but others on the same thread said it worked fine for
> them.)
>
> There are also many dual-port SFP+ cards out there (such as the Intel X520)
> that are not too expensive and support lots of different types of SFP+
> connectors. Although Intel does not make a 10GBase-T SFP+ itself, there are
> third parties that make it. You would use one of those to connect to the
> 10GbE feed into the rack and then a regular fiber SFP (or the option listed
> below) to connect to the switch.
>
> To connect the pfSense to the switch, I would probably use a Direct-Attach
> cable (DAC) instead of fiber or Ethernet. Approved Optics
>  is a company that makes many OEM network
> connectors under contract and they also make their own versions of them at
> significantly reduced prices. Their DAC Finder
>  tool lets you order a cable that
> has SFP+ ends for different manufacturers (for example, an Intel end for
> your pfSense and an HP end for your switch). There's no need to worry about
> fiber or CAT7A Ethernet cables; just plug the cable in (taking care to make
> sure it is oriented correctly) and that's it.
>
> Since you have a limited budget, I really recommend going the
> direct-attached route. They are so much cheaper and more resilient than
> fiber, and switches with SFP+ slots are often much cheaper than switches
> with 10GbE. For example, you can get a Uniquiti EdgeSwitch with 48 Gb ports
> and 2 SFP+ ports for just around $400. These are the switches I have used
> in many of our limited-budget installations in the past (including in a
> University setting like yours seems to be from your email address) and they
> perform well. (Note that Approved Optics does not have official Ubiquiti
> cables, but many on the Ubiquiti forums report that it works with Cisco and
> other brand cables as long as they are 2 meters or shorter. In a single
> rack, that should not be an issue.)
>
>
> Moshe
>
> --
> Moshe Katz
> -- mo...@ymkatz.net
> -- +1(301)867-3732
>
> On Tue, Mar 27, 2018 at 6:41 PM, Paul Mather 
> wrote:
>
> > A 10GBASE-T port became available to us in our server rack.  The rack
> > currently has a 20-node Hadoop cluster, each node having dual Intel i350
> > 1000BASE-T NICs.  The Hadoop nodes connect to an old HP 2910al-48G
> 48-port
> > GbE switch that, in turn, connects to an old Dell R310 server running
> > pfSense that serves as the WAN gateway for the cluster.
> >
> > It appears that the choice (not ours) of RJ45 for the 10 GbE provided for
> > us in the rack will necessitate some equipment changes if we are to
> utilise
> > the 10 GbE connection.  Having done some investigation, I've decided the
> > following changes are likely needed, and I would like to solicit from the
> > list comment regarding any obvious blunders in the plan below:
> >
> > 1) I need a 10 GbE uplink capability from my switch to the pfSense
> gateway
> > and also 10GBASE-T WAN connectivity from my pfSense gateway to the
> > 10GBASE-T port in the rack.
> >
> > 2) The 10 GbE expansion options for the HP 2910al-48G are limited and I
> > couldn't actually find any 10GBASE-T solutions (IIRC).  If I went for 10
> > GbE SFP+ in the HP 2910al-48G that would mean I would also need 10 GbE
> SFP+
> > capability in my pfSense gateway---likely meaning I would need two 10 GbE
> > NICs (one SFP+ and one 10GBASE-T), which means...
> >
> > 3) It is probably cheaper (alas, we are on a budget) to buy a new switch
> > to replace the HP 2910al-48G that includes 10GBASE-T uplink capability.
> > That would let me just have a single 10 GbE card for the pfSense gateway.
> > I think the Netgear GS752TX 52-port switch would be a good candidate as
> it
> > includes two 10GBASE-T ports in addition to the 48 1000BASE-T ports.
> >
> > 4) I am considering a Chelsio NIC for the 10GBASE-T WAN/LAN connections
> > because I keep hearing these are

Re: [pfSense] 10GBASE-T hardware

[Moshe Katz]

Note: that should say CAT*6*A, not *7*. Other than that, no changes.

--
Moshe Katz
-- mo...@ymkatz.net
-- +1(301)867-3732

On Tue, Mar 27, 2018 at 8:10 PM, Moshe Katz  wrote:

> According to the specs that I found on HP's website, your HP switch does
> not support 10Gb, only 1Gb on its mini-GBIC ports. You will definitely need
> a new switch to take advantage of 10Gb.
>
> If you do get a switch that supports 10GBase-T, you should definitely
> consider the Intel X540. The vast majority of reports that I have seen say
> that it works great. (There was one report I found on a forum claiming
> performance issues, but others on the same thread said it worked fine for
> them.)
>
> There are also many dual-port SFP+ cards out there (such as the Intel
> X520) that are not too expensive and support lots of different types of
> SFP+ connectors. Although Intel does not make a 10GBase-T SFP+ itself,
> there are third parties that make it. You would use one of those to connect
> to the 10GbE feed into the rack and then a regular fiber SFP (or the option
> listed below) to connect to the switch.
>
> To connect the pfSense to the switch, I would probably use a Direct-Attach
> cable (DAC) instead of fiber or Ethernet. Approved Optics
>  is a company that makes many OEM network
> connectors under contract and they also make their own versions of them at
> significantly reduced prices. Their DAC Finder
>  tool lets you order a cable that
> has SFP+ ends for different manufacturers (for example, an Intel end for
> your pfSense and an HP end for your switch). There's no need to worry about
> fiber or CAT7A Ethernet cables; just plug the cable in (taking care to make
> sure it is oriented correctly) and that's it.
>
> Since you have a limited budget, I really recommend going the
> direct-attached route. They are so much cheaper and more resilient than
> fiber, and switches with SFP+ slots are often much cheaper than switches
> with 10GbE. For example, you can get a Uniquiti EdgeSwitch with 48 Gb ports
> and 2 SFP+ ports for just around $400. These are the switches I have used
> in many of our limited-budget installations in the past (including in a
> University setting like yours seems to be from your email address) and they
> perform well. (Note that Approved Optics does not have official Ubiquiti
> cables, but many on the Ubiquiti forums report that it works with Cisco and
> other brand cables as long as they are 2 meters or shorter. In a single
> rack, that should not be an issue.)
>
>
> Moshe
>
> --
> Moshe Katz
> -- mo...@ymkatz.net
> -- +1(301)867-3732 <(301)%20867-3732>
>
> On Tue, Mar 27, 2018 at 6:41 PM, Paul Mather 
> wrote:
>
>> A 10GBASE-T port became available to us in our server rack.  The rack
>> currently has a 20-node Hadoop cluster, each node having dual Intel i350
>> 1000BASE-T NICs.  The Hadoop nodes connect to an old HP 2910al-48G 48-port
>> GbE switch that, in turn, connects to an old Dell R310 server running
>> pfSense that serves as the WAN gateway for the cluster.
>>
>> It appears that the choice (not ours) of RJ45 for the 10 GbE provided for
>> us in the rack will necessitate some equipment changes if we are to utilise
>> the 10 GbE connection.  Having done some investigation, I've decided the
>> following changes are likely needed, and I would like to solicit from the
>> list comment regarding any obvious blunders in the plan below:
>>
>> 1) I need a 10 GbE uplink capability from my switch to the pfSense
>> gateway and also 10GBASE-T WAN connectivity from my pfSense gateway to the
>> 10GBASE-T port in the rack.
>>
>> 2) The 10 GbE expansion options for the HP 2910al-48G are limited and I
>> couldn't actually find any 10GBASE-T solutions (IIRC).  If I went for 10
>> GbE SFP+ in the HP 2910al-48G that would mean I would also need 10 GbE SFP+
>> capability in my pfSense gateway---likely meaning I would need two 10 GbE
>> NICs (one SFP+ and one 10GBASE-T), which means...
>>
>> 3) It is probably cheaper (alas, we are on a budget) to buy a new switch
>> to replace the HP 2910al-48G that includes 10GBASE-T uplink capability.
>> That would let me just have a single 10 GbE card for the pfSense gateway.
>> I think the Netgear GS752TX 52-port switch would be a good candidate as it
>> includes two 10GBASE-T ports in addition to the 48 1000BASE-T ports.
>>
>> 4) I am considering a Chelsio NIC for the 10GBASE-T WAN/LAN connections
>> because I keep hearing these are the best-supported 10 GbE cards under
>> FreeBSD.  I'd get a Chelsio T420-BT but these seem to be discontinued in
>> favour of the Chelsio T520-BT.  Are there any better choices I should be
>> considering?  Intel X540-T2??
>>
>>
>> So, as I said earlier, are there any glaring problems in the above plan?
>> (Does it seem sensible?)  Or, alternatively, is there a much better
>> solution that I've overlooked entirely?  Constructive criticism/input is
>> appreciated.
>>
>> Thanks in advanc

Re: [pfSense] 10GBASE-T hardware

[Moshe Katz]

According to the specs that I found on HP's website, your HP switch does
not support 10Gb, only 1Gb on its mini-GBIC ports. You will definitely need
a new switch to take advantage of 10Gb.

If you do get a switch that supports 10GBase-T, you should definitely
consider the Intel X540. The vast majority of reports that I have seen say
that it works great. (There was one report I found on a forum claiming
performance issues, but others on the same thread said it worked fine for
them.)

There are also many dual-port SFP+ cards out there (such as the Intel X520)
that are not too expensive and support lots of different types of SFP+
connectors. Although Intel does not make a 10GBase-T SFP+ itself, there are
third parties that make it. You would use one of those to connect to the
10GbE feed into the rack and then a regular fiber SFP (or the option listed
below) to connect to the switch.

To connect the pfSense to the switch, I would probably use a Direct-Attach
cable (DAC) instead of fiber or Ethernet. Approved Optics
 is a company that makes many OEM network
connectors under contract and they also make their own versions of them at
significantly reduced prices. Their DAC Finder
 tool lets you order a cable that
has SFP+ ends for different manufacturers (for example, an Intel end for
your pfSense and an HP end for your switch). There's no need to worry about
fiber or CAT7A Ethernet cables; just plug the cable in (taking care to make
sure it is oriented correctly) and that's it.

Since you have a limited budget, I really recommend going the
direct-attached route. They are so much cheaper and more resilient than
fiber, and switches with SFP+ slots are often much cheaper than switches
with 10GbE. For example, you can get a Uniquiti EdgeSwitch with 48 Gb ports
and 2 SFP+ ports for just around $400. These are the switches I have used
in many of our limited-budget installations in the past (including in a
University setting like yours seems to be from your email address) and they
perform well. (Note that Approved Optics does not have official Ubiquiti
cables, but many on the Ubiquiti forums report that it works with Cisco and
other brand cables as long as they are 2 meters or shorter. In a single
rack, that should not be an issue.)


Moshe

--
Moshe Katz
-- mo...@ymkatz.net
-- +1(301)867-3732

On Tue, Mar 27, 2018 at 6:41 PM, Paul Mather 
wrote:

> A 10GBASE-T port became available to us in our server rack.  The rack
> currently has a 20-node Hadoop cluster, each node having dual Intel i350
> 1000BASE-T NICs.  The Hadoop nodes connect to an old HP 2910al-48G 48-port
> GbE switch that, in turn, connects to an old Dell R310 server running
> pfSense that serves as the WAN gateway for the cluster.
>
> It appears that the choice (not ours) of RJ45 for the 10 GbE provided for
> us in the rack will necessitate some equipment changes if we are to utilise
> the 10 GbE connection.  Having done some investigation, I've decided the
> following changes are likely needed, and I would like to solicit from the
> list comment regarding any obvious blunders in the plan below:
>
> 1) I need a 10 GbE uplink capability from my switch to the pfSense gateway
> and also 10GBASE-T WAN connectivity from my pfSense gateway to the
> 10GBASE-T port in the rack.
>
> 2) The 10 GbE expansion options for the HP 2910al-48G are limited and I
> couldn't actually find any 10GBASE-T solutions (IIRC).  If I went for 10
> GbE SFP+ in the HP 2910al-48G that would mean I would also need 10 GbE SFP+
> capability in my pfSense gateway---likely meaning I would need two 10 GbE
> NICs (one SFP+ and one 10GBASE-T), which means...
>
> 3) It is probably cheaper (alas, we are on a budget) to buy a new switch
> to replace the HP 2910al-48G that includes 10GBASE-T uplink capability.
> That would let me just have a single 10 GbE card for the pfSense gateway.
> I think the Netgear GS752TX 52-port switch would be a good candidate as it
> includes two 10GBASE-T ports in addition to the 48 1000BASE-T ports.
>
> 4) I am considering a Chelsio NIC for the 10GBASE-T WAN/LAN connections
> because I keep hearing these are the best-supported 10 GbE cards under
> FreeBSD.  I'd get a Chelsio T420-BT but these seem to be discontinued in
> favour of the Chelsio T520-BT.  Are there any better choices I should be
> considering?  Intel X540-T2??
>
>
> So, as I said earlier, are there any glaring problems in the above plan?
> (Does it seem sensible?)  Or, alternatively, is there a much better
> solution that I've overlooked entirely?  Constructive criticism/input is
> appreciated.
>
> Thanks in advance.
>
> Cheers,
>
> Paul.
> ___
> pfSense mailing list
> https://lists.pfsense.org/mailman/listinfo/list
> Support the project with Gold! https://pfsense.org/gold
>
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Suppo

[pfSense] 10GBASE-T hardware

[Paul Mather]

A 10GBASE-T port became available to us in our server rack.  The rack currently 
has a 20-node Hadoop cluster, each node having dual Intel i350 1000BASE-T NICs. 
 The Hadoop nodes connect to an old HP 2910al-48G 48-port GbE switch that, in 
turn, connects to an old Dell R310 server running pfSense that serves as the 
WAN gateway for the cluster.

It appears that the choice (not ours) of RJ45 for the 10 GbE provided for us in 
the rack will necessitate some equipment changes if we are to utilise the 10 
GbE connection.  Having done some investigation, I've decided the following 
changes are likely needed, and I would like to solicit from the list comment 
regarding any obvious blunders in the plan below:

1) I need a 10 GbE uplink capability from my switch to the pfSense gateway and 
also 10GBASE-T WAN connectivity from my pfSense gateway to the 10GBASE-T port 
in the rack.

2) The 10 GbE expansion options for the HP 2910al-48G are limited and I 
couldn't actually find any 10GBASE-T solutions (IIRC).  If I went for 10 GbE 
SFP+ in the HP 2910al-48G that would mean I would also need 10 GbE SFP+ 
capability in my pfSense gateway---likely meaning I would need two 10 GbE NICs 
(one SFP+ and one 10GBASE-T), which means...

3) It is probably cheaper (alas, we are on a budget) to buy a new switch to 
replace the HP 2910al-48G that includes 10GBASE-T uplink capability.  That 
would let me just have a single 10 GbE card for the pfSense gateway.  I think 
the Netgear GS752TX 52-port switch would be a good candidate as it includes two 
10GBASE-T ports in addition to the 48 1000BASE-T ports.

4) I am considering a Chelsio NIC for the 10GBASE-T WAN/LAN connections because 
I keep hearing these are the best-supported 10 GbE cards under FreeBSD.  I'd 
get a Chelsio T420-BT but these seem to be discontinued in favour of the 
Chelsio T520-BT.  Are there any better choices I should be considering?  Intel 
X540-T2??


So, as I said earlier, are there any glaring problems in the above plan?  (Does 
it seem sensible?)  Or, alternatively, is there a much better solution that 
I've overlooked entirely?  Constructive criticism/input is appreciated.

Thanks in advance.

Cheers,

Paul.
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold