RE: an over-the-top data center

[Mikael Abrahamsson]

On Mon, 1 Dec 2008, Deepak Jain wrote:

3) No one cares if the server farm is blast proof (it isn't), if the 
connectivity in/out of it gets blasted (submessage: silos were meant to 
deliver one thing, datacenters aren't in the same operational model once 
they need connectivity to the outside world)


It's much easier to restore fiber connectivity in a time of crisis than it 
is to source hardware manufacturered at the other end of the world and 
have this set up properly. I do think there is value in keeping the hw 
safer than the connectivity to the outside.


I bet the military or emergency services can establish a 10km fiber 
stretch in a few hours. Replacing some telecom hw and set it up from 
scratch would probably take weeks (I'm not talking about a single router 
here).


--
Mikael Abrahamssonemail: [EMAIL PROTECTED]

RADB service outage

[Shin Yamasaki]

Hi,

It seems Merit's RADB service is not working.

Both Web and command-line accesses aren't available.  On the Web, it
only returns the following string: "Number of objects found: 1"  On the
command-line, nothing is returned.

Not only us but also other folks here in Japan are affected.

If someone from Merit sees this, please take a look at the system and
take appropriate action.

Thank you in advance,

-- 
Shin Yamasaki
Japan Network Information Center (JPNIC)

Re: an over-the-top data center

[Dragos Ruiu]

On 28-Nov-08, at 7:35 PM, Gadi Evron wrote:


On Fri, 28 Nov 2008, Howard C. Berkowitz wrote:


It seems that all these cases are more under the bottom than over  
the top.




Every couple of years there is a story about some anti virus  
company, data center, or whatever running out of an old nuclear  
bunker/military base/middle of no where. It is exciting the first  
few times.



Hey I'll defend the interest in this one. They at least have cool  
architecture.
And to all the folks debating the form of security, let me also remind  
that massive redundancy always provides even more security than one  
very, very, hard point.


cheers,
--dr



--
World Security Pros. Cutting Edge Training, Tools, and Techniques
Vancouver, Canada  March 16-20 2009  http://cansecwest.com
London, U.K. May 27/28 2009 http://eusecwest.com
pgpkey http://dragos.com/ kyxpgp

Re: an over-the-top data center

[Randy Bush]

> Despite the huge amount of "content which transcends the language 
> barrier" [tip of the hat wbn], it is worth noting that there is
> a non-trivial amount of language-/culture-specific traffic that 
> doesn't need or want to traverse globally (viz massive IXes & large
> xTTH deplyoments in otherwise 'small' countries).  Sometimes that 
> maps near to the political boundaries.

 slide 6

of course, these data are a bit long in the tooth

randy

Re: an over-the-top data center

[Joe Provo]

On Mon, Dec 01, 2008 at 08:14:20PM +0100, Kurt Erik Lindqvist wrote:
[snip]
> On 1 dec 2008, at 15.08, Patrick W. Gilmore wrote:
[snip]
> >I don't think any IXP can become a significant player on the  
> >Internet today by only attracting participants from the country in  
> >question.  The Internet is not bound by political borders.   
> >(Usually. :)

Despite the huge amount of "content which transcends the language 
barrier" [tip of the hat wbn], it is worth noting that there is
a non-trivial amount of language-/culture-specific traffic that 
doesn't need or want to traverse globally (viz massive IXes & large
xTTH deplyoments in otherwise 'small' countries).  Sometimes that 
maps near to the political boundaries.

Joe [by all means, do not take this as a SPoF endorsement]

-- 
 RSUC / GweepNet / Spunk / FnB / Usenix / SAGE

RE: an over-the-top data center

[Deepak Jain]

Apologies to the list. 

I didn't know whether to fork this into a couple of replies, or just run with 
it. I chose the latter. 

1) This datacenter is only 12,000 sq ft. (submessage: who cares?)

2) The generators are underground. A leak in their exhaust system kills 
everyone -- worse, a leak in their fuel tank or filler lines (when being filled 
from above) could do the same. Yes, you could address this with alarms 
(provided they work and are tested, etc).

3) No one cares if the server farm is blast proof (it isn't), if the 
connectivity in/out of it gets blasted (submessage: silos were meant to deliver 
one thing, datacenters aren't in the same operational model once they need 
connectivity to the outside world)

4) With all of that fog and plant life, I wonder how they critically manage 
humidity. [Or if they even do].



To the question of carrier hotels and their supposed secrecy, etc. If you need 
connectivity to multiple providers, those providers know where the buildings 
are, and presumably so do most of their employees. If 500,000 people (say the 
top 10 companies together) know where the building is, it's not a secret. **

Carrier hotels aren't meant to be more secure than the lines coming into them. 
Those lines are coming in on unsecured poles, manholes and the rest. Their most 
dramatic failure modes are pretty obvious if not well-studied. Internet 
"security" [as in resilience] is built on the concept of a point-of-view of 
connectivity with multiple failures and routing around them -- NOT sacred nodes 
that cannot fail or universal end-to-end reachability. Internet "security" [as 
in integrity] is not something that's been proven on the Internet yet [general 
case, please no banter about encryption/quantum oscillation, etc].

Lots of people have already said this is dull -- it is, it is also a nice set 
of pictures.

** Submitted without proof. This covers all the buildings that make claims 
about not having their name on the door and have loading docks with no security 
on them. (you know who you are).

Deepak

Re: an over-the-top data center

[Jean-François Mezei]

[EMAIL PROTECTED] wrote:

> The Internet can be mission critical.  (Well, not really, but it's =20
> trying.)  And for something mission critical, a single point, no =20
> matter how well reinforced, is not good enough.

It may not be "mission critical" for any one particular client, but when
you bundle all of the separate non critical applications on the net, the
 impact of downtime on the population becomes important enough to be
seen as "critical".  Think about airlines expecting passengers to
check-in via the internet more and more so that they can reduce staff at
airports.


> The exchange point should _NOT_ be mission critical.  As I explained =20
> multiple times in the thread, if that is your only vector, your design =20=
> is broken.  Period.  Care to argue otherwise?

Fair enough. However, in a particular city, you may have difficulty
finding multiple different transit providers whose fiber trunks are
truly differently routed. It is bad enough that different transit
providers may share the same dark fibre cable out of the city.

Very large cities such as New York may make it much easier to find truly
independant transit links. But for small, medium cities, it becomes
harder (especially if geography limits the number of truly separate links).

In the end, to form a truly redundant service, you probably need to have
a presence in multiple cities, each with its own carrier hotel. And at
that point, each carrier hotel need not be "mission critical" because
you can continue service from another city.

But even if you have backup, you still want the carrier hotels to be
robust.  And if you can't afford to have data centres/networks in
different cities, you do want to have a robust interconnect to the
internet. Consider the number of small/medium size ISPs whose
infrastructrure is located at the carrier hotel where the local exchange
resides. To them, the availability of services at that carrier hotel is
mission critical because their bueiness depends on it, and they can't
afford to be in multiple locations.

Re: an over-the-top data center

[Lamar Owen]

On Monday 01 December 2008 16:34:26 Steven M. Bellovin wrote:
> On Mon, 1 Dec 2008 16:03:39 -0500
> Lamar Owen <[EMAIL PROTECTED]> wrote:
> > You mean something akin to Sealand's HavenCo?  Yes, I know that's an
> > old fort, and not a ship, but a similar concept at least.

> HavenCo, which ran a datacenter on the "nation" of Sealand, is
> no longer operating there:
> http://www.theregister.co.uk/2008/11/25/havenco/

Which shows how well the concept works; which is why I mentioned it

Re: an over-the-top data center

[Jim Popovitch]

On Mon, Dec 1, 2008 at 16:34, Steven M. Bellovin <[EMAIL PROTECTED]> wrote:
> HavenCo, which ran a datacenter on the "nation" of Sealand, is
> no longer operating there:

Which is the same story for most (if not all) of these hype-driven
"bullet-proof" data centers.

I recall a .com CEO espousing the capabilities of his
datacenter-inside-an-old-bank-vault to prevent DoS attacks such as the
one that had hit Yahoo! the week before.   I must say that the
provided dinner, drinks and Hummer Limo ride, to the DC, made the
humor of the CEO more enjoyable.   Sadly a lot of older pensioners
were eating his every word.   At that time I worked for an
equipment/services reseller and I persisted quietly, as best I could,
to save some people's life savings.   I felt like a diver witnessing a
herring infused shark fest.

-Jim P.

Re: an over-the-top data center

[Martin List-Petersen]

Steven M. Bellovin wrote:


HavenCo, which ran a datacenter on the "nation" of Sealand, is
no longer operating there:
http://www.theregister.co.uk/2008/11/25/havenco/ 



--Steve Bellovin, http://www.cs.columbia.edu/~smb
  


If you do a bit more research on that one, it never got to a serious 
point. They had one 802.11b onto the platform and never got very far 
with it. No fiber and no redundancy.


However the idea was a bit of a novelty, because it's claimed to be 
sovereign territory.


Kind regards,
Martin List-Petersen

--
Airwire - Ag Nascadh Pobal an Iarthar
http://www.airwire.ie
Phone: 091-865 968 

Re: an over-the-top data center

[Steven M. Bellovin]

On Mon, 1 Dec 2008 16:03:39 -0500
Lamar Owen <[EMAIL PROTECTED]> wrote:

> On Monday 01 December 2008 13:27:30 Danny McPherson wrote:
> > On a related noted, some have professed that adapting old
> > ships into data centers would provide eco-friendly secure
> > data center solutions.  
> 
> You mean something akin to Sealand's HavenCo?  Yes, I know that's an
> old fort, and not a ship, but a similar concept at least.
> 
> 
HavenCo, which ran a datacenter on the "nation" of Sealand, is
no longer operating there:
http://www.theregister.co.uk/2008/11/25/havenco/ 


--Steve Bellovin, http://www.cs.columbia.edu/~smb

Re: an over-the-top data center

[Lamar Owen]

On Monday 01 December 2008 13:27:30 Danny McPherson wrote:
> On a related noted, some have professed that adapting old
> ships into data centers would provide eco-friendly secure
> data center solutions.  

You mean something akin to Sealand's HavenCo?  Yes, I know that's an old fort, 
and not a ship, but a similar concept at least.

RE: EIGRP question...

[Darden, Patrick S.]

My first thought for this was: route filtering.  My second thought 
was: use different AS#s.  Then I reread your question and thought 
of something far simpler.

It seems to me if you are migrating from provider A to provider B
then you should set everything up for B, then shut down the 
interface to A.  If everything works, then you are good, if not
then you bring that interface back up in a hurry!

Is this more complicated?  Are you, for example, moving to B
but planning on keeping A as a backup?

Or, perhaps your MPLS migration is from non-MPLS to an MPLS based
system?  So you are keeping A and B?

Or perhaps A and B are customers, not really providers?

Anyways, not sure of your exact situation, but to answer your
question directly:

EIGRP uses 5 metrics for weighting paths
--delay
--bandwidth
--reliability
--load
--MTU

It does not use hop count for path weighting.  This is a problem, as it would 
be your easy solution--pretty much auto-solving the condition.

1.  You can't really count on delay, and you can't fiddle with it in any easy 
manner.
2.  You could artificially limit bandwidth using your core.  I am unsure if 
this would help you; however, it would answer your stated question.
3.  Reliability--calculated dynamically.  Not helpful for you.
4.  Load--this is the load on the interface I think (0-255).  Calculated 
dynamically.  Not helpful for you.
5.  MTU--I have no idea how you could use this.  Not user configurable in this 
case, I don't think.  I have never used it myself for metrics.  It is 
theoretically used, but never seen it used (in eigrp).

Not sure if this helps.
--p


-Original Message-
From: Mike Lyon [mailto:[EMAIL PROTECTED]
Sent: Monday, December 01, 2008 2:49 PM
To: Nanog Mailing list
Subject: EIGRP question...


Howdy,

So I am working on an MPLS migration from provider "A" to provider "B"
of which both terminate into my core via customer prem routers. I have
a single EIGRP process between my core and the two customer prem
routers supplied to me by both providers, of which I don't have access
to. My question is, I would like to take the routes that come in from
the neighbor "A" router and apply some kind of metrics to them so they
are not preferred over the routes learned by the provider "B" router.

Is this possible or would I need to be running different EIGRP
processes between the two customer prem routers and then play around
with some redistribution? I am hoping this isn't the case because I
don't have access to those CPE routers and redistribution is a nasty
thing...

Thanks in advance for any enlightnment.

Cheers,
Mike

Re: EIGRP question...

[Jeff Ambern]

How about setting the bandwidth of the link to provider B higher.  Or
increasing the delay of the link to provider A?  Either of these should work
for you.  


On 12/1/08 2:49 PM, "Mike Lyon" <[EMAIL PROTECTED]> wrote:

> Howdy,
> 
> So I am working on an MPLS migration from provider "A" to provider "B"
> of which both terminate into my core via customer prem routers. I have
> a single EIGRP process between my core and the two customer prem
> routers supplied to me by both providers, of which I don't have access
> to. My question is, I would like to take the routes that come in from
> the neighbor "A" router and apply some kind of metrics to them so they
> are not preferred over the routes learned by the provider "B" router.
> 
> Is this possible or would I need to be running different EIGRP
> processes between the two customer prem routers and then play around
> with some redistribution? I am hoping this isn't the case because I
> don't have access to those CPE routers and redistribution is a nasty
> thing...
> 
> Thanks in advance for any enlightnment.
> 
> Cheers,
> Mike
> 

Jeff

EIGRP question...

[Mike Lyon]

Howdy,

So I am working on an MPLS migration from provider "A" to provider "B"
of which both terminate into my core via customer prem routers. I have
a single EIGRP process between my core and the two customer prem
routers supplied to me by both providers, of which I don't have access
to. My question is, I would like to take the routes that come in from
the neighbor "A" router and apply some kind of metrics to them so they
are not preferred over the routes learned by the provider "B" router.

Is this possible or would I need to be running different EIGRP
processes between the two customer prem routers and then play around
with some redistribution? I am hoping this isn't the case because I
don't have access to those CPE routers and redistribution is a nasty
thing...

Thanks in advance for any enlightnment.

Cheers,
Mike

Re: an over-the-top data center

[Kurt Erik Lindqvist]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

- -BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Patrick,

On 1 dec 2008, at 02.33, Patrick W. Gilmore wrote:


On Nov 28, 2008, at 4:04 PM, Jean-François Mezei wrote:

The thing about a carrier hotel is that it cannot be a secret  
location

since you need to allow various carriers and ISPs to have physical
access to the building so they can install/manage their
servers/routers/switches.

The advantage of this swedish data centre is that even if its  
location
is well known, it is pretty hard to harm the building. You can't  
run a

truck full of explosives into it for instance.


Unfortunately, you also cannot run your own fiber there, colo  
equipment there, visit it for any reason, etc.


for the non-Stockholm locations that is not true. As a matter of fact,  
you will have to get your own fibers to Netnod there. As for co-lo of  
equipment, not as easy as in a neutral co-location. As for visits, why  
would you need to?


As for fibers, Stockholm has a fiber monopoloy run by the city of  
Stockholm. So you would have to buy fibers from that monopoloy in any  
case.


I was going to say 'this probably hinders customers adoption at  
NetNod', but I know for a fact the "probably" is superfluous.



That is your judgement. We have seen the largest growth for a long  
time in the last year.


Best regards,

- - - kurtis -



- -BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkk0MeMACgkQAFdZ6xrc/t7REACfThTzW+3+mvA0ttvViTTVmMfv
qgUAmwQyiuAaB/+vTD9wMtqCq7PDhw0F
=ycFe
- -END PGP SIGNATURE-
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkk0PIgACgkQAFdZ6xrc/t727wCgvi0zOw4ivBe7AG98hb+DqoGI
qicAn0WKn/yUoqYLln2yP7GuxM16NHzT
=7Njx
-END PGP SIGNATURE-

Re: an over-the-top data center

[Kurt Erik Lindqvist]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

- -BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


On 1 dec 2008, at 15.08, Patrick W. Gilmore wrote:


On Dec 1, 2008, at 4:58 AM, Måns Nilsson wrote:

--On söndag, söndag 30 nov 2008 23.05.01 -0500 "Patrick W. Gilmore"
<[EMAIL PROTECTED]> wrote:


In Sweden, the reason to not choose NetNod (and to go with the  
smaller

exchangepoints) is price and only price. No swedish ISP I know of has
stated that the fact that the Stokab fibre is bought by the IXP and  
not the

ISP is a problem per se. Some might have a better wholesale deal than
NetNod has but that is still just about price.


I don't think any IXP can become a significant player on the  
Internet today by only attracting participants from the country in  
question.  The Internet is not bound by political borders.   
(Usually. :)



I am not trying to defend myself here, everyone is entitled to their  
opinion on which IX model works better than another, but it might be  
worth pointing something out in the history of Netnod. Because of the  
fiber monopoly in Stockholm, that pre-dates the estblishment of any  
neutral co-lo, the Swedish operators built their own datacenters.  
Therefor, when NEtnod was established, there simply was no single  
point where the operators could have established the switches. This  
was *one* of the reasons the bunkers where chosen.


Best regards,

- - - kurtis -



- -BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkk0M9kACgkQAFdZ6xrc/t4oHgCgq1JRMxde9eWYchUyQvQgnITY
PnAAn1K6C5Lird6GWKuPqRSEFfKinjU9
=SA80
- -END PGP SIGNATURE-
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkk0N4wACgkQAFdZ6xrc/t6OfgCgitw9i+PsfM76nc1UqxAfHNbj
PJUAn3jjtA2xQlH/r4LqsXr1KU+N3VVZ
=3QNe
-END PGP SIGNATURE-

Re: an over-the-top data center

[Lyndon Nerenberg]

Not if the ship is literally encased in concrete at the shore.   
Which solves all your other problems as well.


But that's not a ship, it's a building.

There are even examples of actual free-floating ships which have  
been stable for a decade or more.


And many counter-examples.

--lyndon

Re: an over-the-top data center

[Seth Mattinen]

Patrick W. Gilmore wrote:

On Dec 1, 2008, at 2:19 PM, Lyndon Nerenberg wrote:

On 1-Dec-08, at 10:27 AM, Danny McPherson wrote:


On a related noted, some have professed that adapting old
ships into data centers would provide eco-friendly secure
data center solutions.


Your data connection to shore is going to be tenuous at best. One good 
blow strong enough to make you drag anchor and you kiss goodbye your 
fibre trunk connection. Putting that back in service is a bit more 
than a four hour splice job.


Not if the ship is literally encased in concrete at the shore.  Which 
solves all your other problems as well.


There are even examples of actual free-floating ships which have been 
stable for a decade or more.  See the floating casinos in Louisiana, 
which have been hit by hurricanes, and are still attached to shore by 
electricity, bits, and physically.




The same ones that were moved inland and deposited on top of someone's 
house? Hardly a good example of stable.


http://www.katrina.noaa.gov/helicopter/images/katrina-biloxi-miss-grand-casino2-2005.jpg

~Seth

Re: an over-the-top data center

[Patrick W. Gilmore]

On Dec 1, 2008, at 2:19 PM, Lyndon Nerenberg wrote:

On 1-Dec-08, at 10:27 AM, Danny McPherson wrote:


On a related noted, some have professed that adapting old
ships into data centers would provide eco-friendly secure
data center solutions.


Your data connection to shore is going to be tenuous at best. One  
good blow strong enough to make you drag anchor and you kiss goodbye  
your fibre trunk connection. Putting that back in service is a bit  
more than a four hour splice job.


Not if the ship is literally encased in concrete at the shore.  Which  
solves all your other problems as well.


There are even examples of actual free-floating ships which have been  
stable for a decade or more.  See the floating casinos in Louisiana,  
which have been hit by hurricanes, and are still attached to shore by  
electricity, bits, and physically.


--
TTFN,
patrick

Re: an over-the-top data center

[Patrick W. Gilmore]

On Dec 1, 2008, at 2:05 PM, Jean-François Mezei wrote:

Patrick W. Gilmore wrote:


End of day, an IXP is not some magical thing.  It is an ethernet
switch allowing multiple networks to exchange traffic more easily  
than

direct interconnection - and that is all it should be.  It should not
be mission critical.  Treating it as such raises the cost, and
therefore barrier to entry, which lowers its value.


Exchange points are often located in the same building as a carrier
hotel which houses infrastructure for many ISPs and many transit  
providers.


If you consider the internet is used only by teenage males to learn
about female anatomy (pictures and movies), then your statement is
acceptable. But with the Internet now used for serious applications,  
the

focus point of a carrier hotel and exchange becomes much more mission
critical.

Ane because it is a focus point, it becomes much harder to have
redundancy in the buildings (to provide for disaster tolerance). So  
the

natural avenue is to strenghten/re-inforce your one central building.


It is not.

The Internet can be mission critical.  (Well, not really, but it's  
trying.)  And for something mission critical, a single point, no  
matter how well reinforced, is not good enough.


The exchange point should _NOT_ be mission critical.  As I explained  
multiple times in the thread, if that is your only vector, your design  
is broken.  Period.  Care to argue otherwise?


And if the IXP is not your only vector, if your redundancy is greater  
than any single building however deeply it is buried, then that IXP /  
building / vector is not mission critical.  Treating it at such raises  
its price, which raises its barrier of entry, which lowers its utility.


Unless you think only NORAD-approved networks should peer?

--
TTFN,
patrick

Re: an over-the-top data center

[Lyndon Nerenberg]

On 1-Dec-08, at 10:27 AM, Danny McPherson wrote:


On a related noted, some have professed that adapting old
ships into data centers would provide eco-friendly secure
data center solutions.


Your data connection to shore is going to be tenuous at best. One good  
blow strong enough to make you drag anchor and you kiss goodbye your  
fibre trunk connection. Putting that back in service is a bit more  
than a four hour splice job.


An alternative would be to run a microwave link to shore, but I'm not  
sure I would want to bet the farm on the mechanics necessary to keep  
the dish aligned.


And what do you do when it's time to haul out and paint the bottom?!?

Then there is the matter of power. It wouldn't be very hard to DOS the  
entire operation by taking out the fuel barges.


I suppose you could permanently tie up to a pier, but at that point  
you're just a building with a leaky basement. I don't see how anyone  
could claim this is more secure than a purpose-built data centre. (And  
even at anchor, how do you stop someone from taking you out with  
something as simple as a drill?)


--lyndon (mailing via Wimax from S/V Bandido I, at the dock in  
Vancouver :-)

Re: an over-the-top data center

[Jean-François Mezei]

Patrick W. Gilmore wrote:

> End of day, an IXP is not some magical thing.  It is an ethernet  
> switch allowing multiple networks to exchange traffic more easily than  
> direct interconnection - and that is all it should be.  It should not  
> be mission critical.  Treating it as such raises the cost, and  
> therefore barrier to entry, which lowers its value.

Exchange points are often located in the same building as a carrier
hotel which houses infrastructure for many ISPs and many transit providers.

If you consider the internet is used only by teenage males to learn
about female anatomy (pictures and movies), then your statement is
acceptable. But with the Internet now used for serious applications, the
focus point of a carrier hotel and exchange becomes much more mission
critical.

Ane because it is a focus point, it becomes much harder to have
redundancy in the buildings (to provide for disaster tolerance). So the
natural avenue is to strenghten/re-inforce your one central building.

But availability s measured by the weakest link. You can have a bunker
data centre like the one shown in this thread, but if, at the end of the
day, all of a city's fibre links to the rest of the world follow the
same railway track right of way to exit the city (and cross the same
bridges) , then you still have a weak spot and central points of failure.

Re: an over-the-top data center

[Måns Nilsson]

--On måndag, måndag 1 dec 2008 11.53.58 -0500 "Patrick W. Gilmore"
<[EMAIL PROTECTED]> wrote:

> On Dec 1, 2008, at 11:06 AM, Måns Nilsson wrote:
> 
>>> End of day, an IXP is not some magical thing.  It is an ethernet  
>>> switch
>>> allowing multiple networks to exchange traffic more easily than  
>>> direct
>>> interconnection - and that is all it should be.  It should not be  
>>> mission
>>> critical.  Treating it as such raises the cost, and therefore  
>>> barrier to
>>> entry, which lowers its value.

Yes. I do not disagree. The alternates that popped up and made Netnod
switch to Ethernet from SRP were most welcome. SRR mode on that ring was
not funny, btw. 

> Of course knowing where the fiber is does not stop the backhoes.  It was
> obvious you were being silly, so I ignored it. 

Ok. Indeed. 

> By that logic, providers
> should not check any fiber path they purchase because it will not stop
> the backhoes.  I suspect most providers will continue to buy from
> multiple providers, check the paths themselves, ensure grooming onto a
> single path is not a problem, and several other perfectly valid
> operational best practices which are impossible at NetNod.

Netnod with the help of Stokab can guarantee that the two paths to switches
A and B are diverse. It is a normal requirement one can make (at a cost,
but that is to be expected) when sourcing Stokab fibre. They know where
their stuff is and understand the importance of getting it properly
separated. Other providers in Sweden are similar. I have no reason not to
trust them, having seen the inside of several large calls for tender on
dispersed path plants, with fibre paths well documented. That the path of
the last mile to the cave is hidden and secret is a very small problem. 
 
> OTOH: My paragraph above yours is a serious consideration, which you have
> blithely ignored.

Not so anymore, if I've understood correctly. 

Drop this dead horse? 
-- 
Måns NilssonM A C H I N A

Hello, GORRY-O!!  I'm a GENIUS from HARVARD!!


pgpyNJvCNkD4y.pgp
Description: PGP signature

Re: an over-the-top data center

[Danny McPherson]

On Nov 28, 2008, at 6:34 AM, Steven M. Bellovin wrote:


http://royal.pingdom.com/2008/11/14/the-worlds-most-super-designed-data-center-fit-for-a-james-bond-villain/
(No, I don't know if it's real or not.)


I recall visiting something of this sort a couple
years back..

On a related noted, some have professed that adapting old
ships into data centers would provide eco-friendly secure
data center solutions.  I wonder if "pirates" were listed
anywhere in their business plan...

-danny

Re: an over-the-top data center

[Jeremy Jackson]

On Sun, 2008-11-30 at 23:05 -0500, Patrick W. Gilmore wrote:

> Now compare that to forcing every single participant to use unknown  
> fiber paths into an unknown facility.  When are these fibers groomed,  
> and onto which unknown paths?  Which fiber maintenance schedules might  
> impact me without my knowledge?  Which construction projects elsewhere  
> in the city might take me down and there's no way for me to even  
> predict that?  Etc., etc.
> 
> I would prefer to take my chances with the known quantity,  
> thankyouverymuch.  Feel free to do with your network as you please.
> 

I wonder if there is a solution, in general to diverse physical
routing... if you buy from multiple carriers, they might very well share
the same fibre condo, or the same dark fibre vendor.  if you buy
diversity from one vendor, with only handwaving as the guarantee, you
end up with Bell Canada's CO fire a couple years ago, that took down
things which were *supposed* to be redundant.

What are people's experience with knowing the physical routing?  NetNod
may be over-the-top secrecy wise, but are *any* carriers/facility
providers any more "free" with information about the details of where
their infrastructure is that supports the services you are buying?

It seems the general practice is to claim everything is on a
need-to-know basis, with the unspoken/unwritten caveat that nobody's
needs will ever be considered valid?

-- 
Jeremy Jackson
Coplanar Networks
(519)489-4903
http://www.coplanar.net
[EMAIL PROTECTED]

Re: an over-the-top data center

[Patrick W. Gilmore]

On Dec 1, 2008, at 11:06 AM, Måns Nilsson wrote:

End of day, an IXP is not some magical thing.  It is an ethernet  
switch
allowing multiple networks to exchange traffic more easily than  
direct
interconnection - and that is all it should be.  It should not be  
mission
critical.  Treating it as such raises the cost, and therefore  
barrier to

entry, which lowers its value.


You did not answer my question on usability of fiber based on amount  
of

knowledge about where it is.


Of course knowing where the fiber is does not stop the backhoes.  It  
was obvious you were being silly, so I ignored it.  By that logic,  
providers should not check any fiber path they purchase because it  
will not stop the backhoes.  I suspect most providers will continue to  
buy from multiple providers, check the paths themselves, ensure  
grooming onto a single path is not a problem, and several other  
perfectly valid operational best practices which are impossible at  
NetNod.


OTOH: My paragraph above yours is a serious consideration, which you  
have blithely ignored.



As I said before, feel free to use what you please, where you please.   
Your network, your decision.  I frequently do things which would not  
be considered best practices in certain instances, but that does not  
make them valid for everyone everywhere, and I would not argue such.


--
TTFN,
patrick

Re: an over-the-top data center

[Måns Nilsson]

--On måndag, måndag 1 dec 2008 09.08.09 -0500 "Patrick W. Gilmore"
<[EMAIL PROTECTED]> wrote:

> I don't think any IXP can become a significant player on the Internet
> today by only attracting participants from the country in question.  The
> Internet is not bound by political borders.  (Usually. :)

There is a significant amount of traffic being exchanged between swedish
operators on Netnod. It might be the case that the broadband penetration in
Sweden justifies the establishment of local IXPen. 

This is however irrelevant to the discussion at hand -- or did you think
about some kind of issue with connectivity from Stockholm and abroad? At
least 3-4 providers sell connectivity into Stockholm on own fiber paths. Is
Netnod useless to you because you are not one of them? 
 
> As for the blasting of tunnels and national security angle, this is an
> IXP, not nuclear missile launch control.  It should not be your only
> vector to get bits from point A to B.  And if it is, then you have a
> larger problem than worrying about the facility withstanding physical
> attack.

It is an optimisation, a very well engineered one. 
 
> And no, attaching to multiple NetNod nodes is not a solution, since only
> Stockholm has a large number of participants.

Probably true for international clients. Less so for Swedish ISPen. 

> End of day, an IXP is not some magical thing.  It is an ethernet switch
> allowing multiple networks to exchange traffic more easily than direct
> interconnection - and that is all it should be.  It should not be mission
> critical.  Treating it as such raises the cost, and therefore barrier to
> entry, which lowers its value.

You did not answer my question on usability of fiber based on amount of
knowledge about where it is. 

-- 
Måns NilssonM A C H I N A

There's a little picture of ED MCMAHON doing BAD THINGS to JOAN RIVERS
in a $200,000 MALIBU BEACH HOUSE!!


pgpnhYBTVrhyn.pgp
Description: PGP signature

Re: an over-the-top data center

[Randy Bush]

hint: your continued ad homina do not help your argument

> By your logic, every IXP which has any participants is a good model and
> cannot be improved.

the criterion you set was success, not perfection.  netnod is quite
successful.

is this discussion successful?  i think not.  good bye and good night.

randy

Re: an over-the-top data center

[Patrick W. Gilmore]

On Dec 1, 2008, at 9:30 AM, Randy Bush wrote:


some go to sweden for the weather.  some go for netnode.  netnode does
not go to them.  and yes, netnod is bunkered up the ying yang.  qed.


By your logic, every IXP which has any participants is a good model  
and cannot be improved.  An obvious logical fallacy.  One could assume  
this means you have no clue what you are talking about, but I will  
give you the benefit of the doubt.


IOW: You are only interested in your word count.  QED.

--
TTFN,
patrick

Re: an over-the-top data center

[Randy Bush]

some go to sweden for the weather.  some go for netnode.  netnode does
not go to them.  and yes, netnod is bunkered up the ying yang.  qed.

randy

Re: an over-the-top data center

[Patrick W. Gilmore]

On Dec 1, 2008, at 9:12 AM, Randy Bush wrote:


I don't think any IXP can become a significant player on the Internet
today by only attracting participants from the country in question.


netnod is very successful.  i guess they must operate from more than
sweden, then, eh?


NetNod is successful.  Very is a matter of opinion.  As for "operate  
from more than sweden", that is trivial to confirm by looking at their  
member list.


So now that we have agreed, did you have a point, or just want to run  
up your word count?




engineers judge by results, not word count.


Wow, Randy, we are in agreement again.


To be clear, are you suggesting IXPs consider hiding their switches,  
forcing you to use a single fiber providers, not allowing anyone to  
know the path, etc.?  I want to be sure I understand what you mean,  
since "engineers" like to make serious points, not flippant sound bites.


--
TTFN,
patrick

Re: an over-the-top data center

[Randy Bush]

> I don't think any IXP can become a significant player on the Internet
> today by only attracting participants from the country in question.

netnod is very successful.  i guess they must operate from more than
sweden, then, eh?

engineers judge by results, not word count.

randy

Re: an over-the-top data center

[Patrick W. Gilmore]

On Dec 1, 2008, at 4:58 AM, Måns Nilsson wrote:

--On söndag, söndag 30 nov 2008 23.05.01 -0500 "Patrick W. Gilmore"
<[EMAIL PROTECTED]> wrote:



In Sweden, the reason to not choose NetNod (and to go with the smaller
exchangepoints) is price and only price. No swedish ISP I know of has
stated that the fact that the Stokab fibre is bought by the IXP and  
not the

ISP is a problem per se. Some might have a better wholesale deal than
NetNod has but that is still just about price.


I don't think any IXP can become a significant player on the Internet  
today by only attracting participants from the country in question.   
The Internet is not bound by political borders.  (Usually. :)



Now compare that to forcing every single participant to use unknown  
fiber
paths into an unknown facility.  When are these fibers groomed, and  
onto
which unknown paths?  Which fiber maintenance schedules might  
impact me
without my knowledge?  Which construction projects elsewhere in the  
city
might take me down and there's no way for me to even predict that?   
Etc.,

etc.


The fiber paths into these facilities are national security issues.  
Expect
them to be guarded accordingly (as in running them in specially  
blasted
tunnels 30-60 meters down in the ground for the last aggregated path  
to the
facility). I have not experienced more unpredictability nor more  
outages

because Netnod buys the cable than when the ISP does. Same cable. And
Stokab does indeed know where the cables are.


I'm glad to hear the fibers seem to be stable.  Past performance is no  
guarantee of future profits and all that, but it is good to know care  
has been taken in the past.


As for the blasting of tunnels and national security angle, this is an  
IXP, not nuclear missile launch control.  It should not be your only  
vector to get bits from point A to B.  And if it is, then you have a  
larger problem than worrying about the facility withstanding physical  
attack.


And no, attaching to multiple NetNod nodes is not a solution, since  
only Stockholm has a large number of participants.



End of day, an IXP is not some magical thing.  It is an ethernet  
switch allowing multiple networks to exchange traffic more easily than  
direct interconnection - and that is all it should be.  It should not  
be mission critical.  Treating it as such raises the cost, and  
therefore barrier to entry, which lowers its value.


--
TTFN,
patrick

Re: an over-the-top data center

[Måns Nilsson]

--On söndag, söndag 30 nov 2008 23.05.01 -0500 "Patrick W. Gilmore"
<[EMAIL PROTECTED]> wrote:

> On Nov 30, 2008, at 10:50 PM, Niels Bakker wrote:

>>> I was going to say 'this probably hinders customers adoption at  
>>> NetNod', but I know for a fact the "probably" is superfluous.

> I didn't say it would stop everyone.  Of course some people will not be
> deterred, but some absolutely have.

In Sweden, the reason to not choose NetNod (and to go with the smaller
exchangepoints) is price and only price. No swedish ISP I know of has
stated that the fact that the Stokab fibre is bought by the IXP and not the
ISP is a problem per se. Some might have a better wholesale deal than
NetNod has but that is still just about price.  

The alternative IPXen were started for two reasons, 

1. Price. At the time the first one got going NetNod was running OC48 SRP
as its fabric. (Anyone remember that technology?). The price of SRP
technology was simply too high for many small players, and required Cisco
gear, etc. 

2. Convenience and reduced marginal cost, ie. #1 again. Since the first
alternative (SOL-IX) was and is distributed, really small ASes could join
for the price of a patch cable and an interface.

> Now compare that to forcing every single participant to use unknown fiber
> paths into an unknown facility.  When are these fibers groomed, and onto
> which unknown paths?  Which fiber maintenance schedules might impact me
> without my knowledge?  Which construction projects elsewhere in the city
> might take me down and there's no way for me to even predict that?  Etc.,
> etc.

The fiber paths into these facilities are national security issues. Expect
them to be guarded accordingly (as in running them in specially blasted
tunnels 30-60 meters down in the ground for the last aggregated path to the
facility). I have not experienced more unpredictability nor more outages
because Netnod buys the cable than when the ISP does. Same cable. And
Stokab does indeed know where the cables are. 

> I would prefer to take my chances with the known quantity,
> thankyouverymuch.  Feel free to do with your network as you please.

Just because you know where the cable is the backhoes won´t find it? 
-- 
Måns NilssonM A C H I N A

I'll eat ANYTHING that's BRIGHT BLUE!!


pgpWf7B0SJBNi.pgp
Description: PGP signature