choose.
I'd like to hear more about this effort once you get it going. Which vendor you
went with, how you tuned it, and why you selected who you did. Also, how it
works.
LFoD
Date: Sun, 10 May 2015 01:17:07 +
From: jo...@iecc.com
To: nanog@nanog.org
Subject: Re: Thousands of hosts on a gigabit
Also, do you need line rate forwarding? Having 1,000 devices with 1Gb
uplinks doesn't necessarily mean that full throughput is required... the
clustering and the applications may be sporadic and bursty?
It's definitely sporadic and bursty. There's another network for high
speed traffic among
On 10/05/2015 00:33, Karl Auer wrote:
Would be interesting to see how IPv6 performed, since is one of the
things it was supposed to be able to deliver - massively scalable links
(equivalent to an IPv4 broadcast domain) via massively reduced protocol
chatter (IPv6 multicast groups vs IPv4
On 2015-05-09 11:57, Baldur Norddahl wrote:
The standard 48 port with 2 port uplink 1U switch is far from full
depth.
You put them in the back of the rack and have the small computers in
the
front. You might even turn the switches around, so the ports face
inwards
into the rack. The network
Juniper OCX1100 have 72 ports in 1U.
And you can tune Linux IPv4 neighbor:
https://ams-ix.net/technical/specifications-descriptions/config-guide#11
--
Eduardo Schoedler
Em sábado, 9 de maio de 2015, Lamar Owen lo...@pari.edu escreveu:
On 05/08/2015 02:53 PM, John Levine wrote:
...
Most
You do not mention low cost before ;)
Em sábado, 9 de maio de 2015, John Levine jo...@iecc.com escreveu:
In article
cahf3uwypqn1ns_umjz-znuk3i5ufczbu9l39b-crovg6yum...@mail.gmail.com
javascript:; you write:
Juniper OCX1100 have 72 ports in 1U.
Yeah, too bad it costs $32,000. Other than
To the OP please do tell us more about what you are doing, it sounds
very interesting.
There's a conference paper in preparation. I'll send a pointer when I can.
R's,
John
On Sat, 2015-05-09 at 17:06 -0400, Lamar Owen wrote:
The effective limit on subnet size would be of course broadcast
overhead; 1,000 nodes on a /22 would likely be painfully slow due to
broadcast overhead alone.
Would be interesting to see how IPv6 performed, since is one of the
things it
In article cahf3uwypqn1ns_umjz-znuk3i5ufczbu9l39b-crovg6yum...@mail.gmail.com
you write:
Juniper OCX1100 have 72 ports in 1U.
Yeah, too bad it costs $32,000. Other than that it'd be perfect.
R's,
John
On 05/08/2015 02:53 PM, John Levine wrote:
...
Most of the traffic will be from one node to another, with
considerably less to the outside. Physical distance shouldn't be a
problem since everything's in the same room, maybe the same rack.
What's the rule of thumb for number of hosts per
On 09/05/2015 23:33, Karl Auer wrote:
IPv4 ARP, for example, hits every on-subnet neighbour; the IPv6
equivalent uses multicast to hit only those neighbours that happen to
share the same 24 low-end L3 address bits as the desired target - a
statistically much smaller subset of on-link neighbours,
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks. Each of the
computers runs Linux and has a gigabit ethernet interface. It occurs
to me that it is unlikely that I can buy an ethernet switch with
thousands of ports, and
The standard 48 port with 2 port uplink 1U switch is far from full depth.
You put them in the back of the rack and have the small computers in the
front. You might even turn the switches around, so the ports face inwards
into the rack. The network cables would be very short and go directly from
On 05/08/2015 02:53 PM, John Levine wrote:
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks. Each of the
computers runs Linux and has a gigabit ethernet interface. It occurs
to me that it is unlikely that I can buy an ethernet
- The more switches a packet has to go through, the higher the latency, so
your response times may deteriorate if you cascade too many switches.
Legend says up to 4 is a good number, any further you risk creating a big
mess.
- The more switches you add, the higher your bandwidth utilized by
John Levine wrote:
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks. Each of the
computers runs Linux and has a gigabit ethernet interface. It occurs
to me that it is unlikely that I can buy an ethernet switch with
thousands
On Fri, May 8, 2015 at 2:53 PM, John Levine jo...@iecc.com wrote:
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks. Each of the
computers runs Linux and has a gigabit ethernet interface. It occurs
to me that it is unlikely
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks. Each of the
computers runs Linux and has a gigabit ethernet interface. It occurs
to me that it is unlikely that I can buy an ethernet switch with
thousands of ports, and even if
On Fri, May 8, 2015 at 11:53 AM, John Levine jo...@iecc.com wrote:
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks.
Very cool-ly crazy.
Each of the
computers runs Linux and has a gigabit ethernet interface. It occurs
to
chuck of a rack
itself.
Chuck
-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of John Levine
Sent: Friday, May 08, 2015 2:53 PM
To: nanog@nanog.org
Subject: Thousands of hosts on a gigabit LAN, maybe not
Some people I know (yes really) are building a system
leafs and spines.
Sk.
-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of John Levine
Sent: Friday, May 08, 2015 2:53 PM
To: nanog@nanog.org
Subject: Thousands of hosts on a gigabit LAN, maybe not
Some people I know (yes really) are building a system
* lists.na...@monmotha.net (Brandon Martin) [Fri 08 May 2015, 21:42 CEST]:
[1] Purely as an example, you can cram 3x Brocade MLX-16 chassis into
a 42U rack (with 0RU to spare). That gives you 48 slots for line cards.
You really can't. Cables need to come from the top, not from the
sides, or
to have 10,000 entries or more in its ARP table.
Agreed. :) You don't really want 10,000 entries in a routing FIB
table either, but I was seriously encouraged by the work going
on in linux 4.0 and 4.1 to improve those lookups.
One obvious way to deal with that is to put some manageable number
Linux has a (configurable) limit on the neighbor table. I know in RHEL
variants, the default has been 1024 neighbors for a while.
net.ipv4.neigh.default.gc_thresh3
net.ipv4.neigh.default.gc_thresh2
net.ipv4.neigh.default.gc_thresh1
net.ipv6.neigh.default.gc_thresh3
Forgot to mention - you might also want to check out Beowulf clusters -
there's an email list at http://www.beowulf.org/ - probably some useful
info in the list archives, maybe a good place to post your query.
Miles
Miles Fidelman wrote:
John Levine wrote:
Some people I know (yes really)
Date: Fri, 8 May 2015 18:53:03 +
From: jo...@iecc.com
To: nanog@nanog.org
Subject: Thousands of hosts on a gigabit LAN, maybe not
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks. Each of the
computers runs Linux and has
On 05/08/2015 04:17 PM, Niels Bakker wrote:
* lists.na...@monmotha.net (Brandon Martin) [Fri 08 May 2015, 21:42 CEST]:
[1] Purely as an example, you can cram 3x Brocade MLX-16 chassis into
a 42U rack (with 0RU to spare). That gives you 48 slots for line cards.
You really can't. Cables need
On 2015-05-08 13:53, John Levine wrote:
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks.
How many racks?
How many computers per rack unit? How many computers per rack?
(How are you handling power?)
How big is each computer?
On 9 May 2015, at 1:53, John Levine wrote:
What's the rule of thumb for number of hosts per switch, cascaded
switches vs. routers, and whatever else one needs to design a dense
network like this?
Most of the major switch vendors have design guides and other examples
like this available
-
From: John Levine jo...@iecc.com
Sent: 5/8/2015 2:59 PM
To: nanog@nanog.org nanog@nanog.org
Subject: Thousands of hosts on a gigabit LAN, maybe not
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks. Each of the
computers runs
On 2015-05-08 18:20, Phil Bedard wrote:
The real answer to this is being able to cram them into a single
chassis which can multiplex the network through a backplane.
Something like the HP Moonshot ARM system or the way others like
Google build high density compute with integrated Ethernet
On Fri, May 8, 2015 at 11:53 AM, John Levine jo...@iecc.com wrote:
Some people I know (yes really) are building a system that will have
several thousand little computers in some racks. Each of the
computers runs Linux and has a gigabit ethernet interface.
Though a bit off-topic I ran in to
On 2015-05-08 12:53, John Levine wrote:
What's the rule of thumb for number of hosts per switch, cascaded
switches vs. routers, and whatever else one needs to design a dense
network like this? TIA
I won't pretend to know best practices, but my inclination would be to
connect the devices to
On Fri, May 8, 2015 at 5:19 PM, Jima na...@jima.us wrote:
Dang. The more I think about this project, the more expensive it sounds.
Naw, just use WiFi. ;)
--
Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474
off topic
The first thing that came to mind was Bitcoin farm! then Ask Bitmaintech and then
I'd be more worried about the number of fans and A/C units.
/off topic
I promise, no bitcoins involved.
R's,
John
Morrow's comment about the ARMD WG notwithstanding, there might be some
useful context in https://tools.ietf.org/html/draft-karir-armd-statistics-01
Cheers,
-Benson
Christopher Morrow mailto:morrowc.li...@gmail.com
May 8, 2015 at 12:19 PM
consider the pain of also ipv6's link-local gamery.
36 matches
Mail list logo
