Re: facebook spying on us?
Data Center Knowledge posted about 20 minutes of very poorly shot video of Prineville. They're Open Compute servers in 'triplet' racks. [...] Their power supply (also open) runs across 2 legs of a 277/480 3-phase feed, which is usually what the substation supplies to your PDUs, which step it down further to 120/208. It also takes -48, and each pair of triplets has a 48V float string that will run the 180 servers for about 45 seconds. It's a nice setup. I plan to steal it. :-) That's what they want you to do - check out the specs on http://opencompute.org/ -- Simon.
RE: facebook spying on us?
That comment about wholesale prices is not actually quite true here in the northwest where avoiding BPA actually sometimes results in cheaper power (ie grant, douglas and chelan counties whoes PUDs own their own dams and are obligated to service their customer and as non-profits actually sell to retail users at near the wholesale grid rates since they have nearly zero cost). Because pacificorp is a private utility they are actually only able to get the leftovers of the hydro from the northwest, BPA must sell to public utilities first (even if it is LA) so there are effectively two prices here in the northwest for wholesale and that is why pacificorp, portland general and puget sound energy all have far far higher rates than the public utilities, even the public utilities with no generation of their own. I was pretty surprised about facebook's choice as well, almost an identical climate can be found along the columbia river in the same places that the very cheapest power is located. They must have some other factors than just weather significantly contribute to the costs to justify not going for the cheapest power. John From: Joel jaeggli [joe...@bogus.com] Sent: Friday, September 30, 2011 3:48 PM To: Steven G. Huter Cc: nanog@nanog.org Subject: Re: facebook spying on us? On 9/30/11 15:19 , Steven G. Huter wrote: I can't tell you the kind of servers, but I can say that I was recently in Prineville, OR, where FB is building a data center (and a second data center). I was used to the ol data centers - you know, where there's raised floors, cabinets, cool air, a guard and a few guys around with some screens? But this was massive. I was amazed at the size - a few city blocks long and a city block wide, with a transformer and power line the size of a small city. I wonder if the Feds were involved. the bonneville power administration. hey joelja this August 2011 article in the Economist outlines some relevant info about the prineville, oregon FB datacenter. http://www.economist.com/node/21525237 ambient cooling is important just like power is important, by sonic.net gets ~240days of ambient in santa rosa so it's feasible wholesale market prices a driven by availability from the largest producer. so you'll pay market price as benchmarked at the bonnevilla transmission yard just as is much of california and az the refence price is at palo verde az. there's only one coal plan in oregon and it's 600MW of generating capacity in boardman that's portland general electric. we've got a 20MW interuptable contract with siliconvalley power precisely becuase it's vanishingly close to the wholesale rate compared to PGEs pricing structure so if you ever wonder why the DCs are in sunnyvale and santa clara but not mountainview, there's a good reason. steve
Re: facebook spying on us?
On Thu, Sep 29, 2011 at 8:13 AM, Glen Kent glen.k...@gmail.com wrote: I also wonder about the kind of servers facebook must be having to be able to manage millions of TCP connections that must be terminating there. For what it's worth, with some kernel tuning you can maintain 500k - 1MM persistent connections on a mid-range Linux box. Providers of mobile push-notification services seem to be the ones most actively pushing these limits publicly. Urban Airship has posted some information on how they maintain 500k connections on EC2 m1.large instances: http://urbanairship.com/blog/2010/08/24/c500k-in-action-at-urban-airship/ http://urbanairship.com/blog/2010/09/29/linux-kernel-tuning-for-c500k/ WhatsApp claim to be able to maintain 1MM connections on single machine, although details are thin: http://blog.whatsapp.com/index.php/2011/09/one-million/ http://news.ycombinator.com/item?id=3028547 (discussion) -n
Re: facebook spying on us?
- Original Message - From: Barry Jones bejo...@semprautilities.com I can't tell you the kind of servers, but I can say that I was recently in Prineville, OR, where FB is building a data center (and a second data center). I was used to the ol data centers - you know, where there's raised floors, cabinets, cool air, a guard and a few guys around with some screens? Data Center Knowledge posted about 20 minutes of very poorly shot video of Prineville. They're Open Compute servers in 'triplet' racks. But this was massive. I was amazed at the size - a few city blocks long and a city block wide, with a transformer and power line the size of a small city. I wonder if the Feds were involved. Their power supply (also open) runs across 2 legs of a 277/480 3-phase feed, which is usually what the substation supplies to your PDUs, which step it down further to 120/208. It also takes -48, and each pair of triplets has a 48V float string that will run the 180 servers for about 45 seconds. It's a nice setup. I plan to steal it. :-) (The substation input voltage is very often 13k2 to 13k8, though it can be even higher.) Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
Re: facebook spying on us?
On 9/30/11 14:59 , Jones, Barry wrote: I can't tell you the kind of servers, but I can say that I was recently in Prineville, OR, where FB is building a data center (and a second data center). I was used to the ol data centers - you know, where there's raised floors, cabinets, cool air, a guard and a few guys around with some screens? But this was massive. I was amazed at the size - a few city blocks long and a city block wide, with a transformer and power line the size of a small city. I wonder if the Feds were involved. the bonneville power administration.
Re: facebook spying on us?
I can't tell you the kind of servers, but I can say that I was recently in Prineville, OR, where FB is building a data center (and a second data center). I was used to the ol data centers - you know, where there's raised floors, cabinets, cool air, a guard and a few guys around with some screens? But this was massive. I was amazed at the size - a few city blocks long and a city block wide, with a transformer and power line the size of a small city. I wonder if the Feds were involved. the bonneville power administration. hey joelja this August 2011 article in the Economist outlines some relevant info about the prineville, oregon FB datacenter. http://www.economist.com/node/21525237 steve
Re: facebook spying on us?
Steven G. Huter wrote: this August 2011 article in the Economist outlines some relevant info about the prineville, oregon FB datacenter. http://www.economist.com/node/21525237 steve Informative article...It's the climate, stupid. Got a laugh out of: The server racks are nearly silent, and their internal fans whirr almost imperceptibly. The only exceptions are network switches which, Facebook staff notes, are perversely designed by even the biggest firms to vent air out of their sides. As a result, they run loud and hot-and are openly sworn at.
Re: facebook spying on us?
On 9/30/11 15:19 , Steven G. Huter wrote: I can't tell you the kind of servers, but I can say that I was recently in Prineville, OR, where FB is building a data center (and a second data center). I was used to the ol data centers - you know, where there's raised floors, cabinets, cool air, a guard and a few guys around with some screens? But this was massive. I was amazed at the size - a few city blocks long and a city block wide, with a transformer and power line the size of a small city. I wonder if the Feds were involved. the bonneville power administration. hey joelja this August 2011 article in the Economist outlines some relevant info about the prineville, oregon FB datacenter. http://www.economist.com/node/21525237 ambient cooling is important just like power is important, by sonic.net gets ~240days of ambient in santa rosa so it's feasible wholesale market prices a driven by availability from the largest producer. so you'll pay market price as benchmarked at the bonnevilla transmission yard just as is much of california and az the refence price is at palo verde az. there's only one coal plan in oregon and it's 600MW of generating capacity in boardman that's portland general electric. we've got a 20MW interuptable contract with siliconvalley power precisely becuase it's vanishingly close to the wholesale rate compared to PGEs pricing structure so if you ever wonder why the DCs are in sunnyvale and santa clara but not mountainview, there's a good reason. steve
Re: facebook spying on us?
On 9/30/11 3:41 PM, Michael Painter wrote: Steven G. Huter wrote: this August 2011 article in the Economist outlines some relevant info about the prineville, oregon FB datacenter. http://www.economist.com/node/21525237 steve Informative article...It's the climate, stupid. Got a laugh out of: The server racks are nearly silent, and their internal fans whirr almost imperceptibly. The only exceptions are network switches which, Facebook staff notes, are perversely designed by even the biggest firms to vent air out of their sides. As a result, they run loud and hot-and are openly sworn at. Which says to me that FB staff has no clue how chassis switches are constructed, or they don't like switches with vertically oriented line cards. ~Seth
Re: facebook spying on us?
It was a relative comparison, and it's off the shelf network gear. -Callahan On Fri, Sep 30, 2011 at 3:58 PM, Seth Mattinen se...@rollernet.us wrote: On 9/30/11 3:41 PM, Michael Painter wrote: Steven G. Huter wrote: this August 2011 article in the Economist outlines some relevant info about the prineville, oregon FB datacenter. http://www.economist.com/node/21525237 steve Informative article...It's the climate, stupid. Got a laugh out of: The server racks are nearly silent, and their internal fans whirr almost imperceptibly. The only exceptions are network switches which, Facebook staff notes, are perversely designed by even the biggest firms to vent air out of their sides. As a result, they run loud and hot-and are openly sworn at. Which says to me that FB staff has no clue how chassis switches are constructed, or they don't like switches with vertically oriented line cards. ~Seth
Re: facebook spying on us?
On 9/30/11 15:58 , Seth Mattinen wrote: On 9/30/11 3:41 PM, Michael Painter wrote: Steven G. Huter wrote: this August 2011 article in the Economist outlines some relevant info about the prineville, oregon FB datacenter. http://www.economist.com/node/21525237 steve Informative article...It's the climate, stupid. Got a laugh out of: The server racks are nearly silent, and their internal fans whirr almost imperceptibly. The only exceptions are network switches which, Facebook staff notes, are perversely designed by even the biggest firms to vent air out of their sides. As a result, they run loud and hot-and are openly sworn at. Which says to me that FB staff has no clue how chassis switches are constructed, or they don't like switches with vertically oriented line cards. nobody puts a chassis switch at the top of a rack... there are several 1u tors orderable with either ftb or btf airflow but, it is a design consideration. ~Seth
Re: facebook spying on us?
did you start your browser before looking at your connection list? However, you're on a window's box, so it wouldn't surprise me if they helpfully started ie for you If you didn't start the browser you use to go to facebook (and its not ie), its fairly interesting. On Sep 29, 2011, at 6:13 AM, Glen Kent wrote: Hi, I see that i have multiple TCP sessions established with facebook. They come up even after i reboot my laptop and dont login to facebook! D:\Documents and Settings\gkentnetstat -a | more Active Connections Proto Local Address Foreign AddressState TCPgkent:3974www-10-02-snc5.facebook.com:http ESTABLISHED TCPgkent:3977www-11-05-prn1.facebook.com:http ESTABLISHED TCPgkent:3665 a184-84-111-139.deploy.akamaitechnologies.com:http ESTABLISHED [clipped] Any idea why these connections are established (with facebook and akamaitechnologies) and how i can kill them? Since my laptop has several connections open with facebook, what kind of information is flowing there? I also wonder about the kind of servers facebook must be having to be able to manage millions of TCP connections that must be terminating there. Glen
Re: facebook spying on us?
Use 'netstat -ao' to see which process(es) they are associated with. Then use a sniffer to see what actual traffic they carry. Jason On Thu, Sep 29, 2011 at 9:13 AM, Glen Kent glen.k...@gmail.com wrote: Hi, I see that i have multiple TCP sessions established with facebook. They come up even after i reboot my laptop and dont login to facebook! D:\Documents and Settings\gkentnetstat -a | more Active Connections Proto Local Address Foreign Address State TCP gkent:3974 www-10-02-snc5.facebook.com:http ESTABLISHED TCP gkent:3977 www-11-05-prn1.facebook.com:http ESTABLISHED TCP gkent:3665 a184-84-111-139.deploy.akamaitechnologies.com:http ESTABLISHED [clipped] Any idea why these connections are established (with facebook and akamaitechnologies) and how i can kill them? Since my laptop has several connections open with facebook, what kind of information is flowing there? I also wonder about the kind of servers facebook must be having to be able to manage millions of TCP connections that must be terminating there. Glen
Re: facebook spying on us?
( Being this is a Windows box) Want to scare yourself silly? . Power off the PC; . Plug it a switch; . Mirror the PC port into a Unix box running Wireshark; . Boot the PC Enjoy all the info leakages from all the apps you installed over the years. - Alain Hebertaheb...@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.netFax: 514-990-9443 On 09/29/11 09:19, Eric Clark wrote: did you start your browser before looking at your connection list? However, you're on a window's box, so it wouldn't surprise me if they helpfully started ie for you If you didn't start the browser you use to go to facebook (and its not ie), its fairly interesting. On Sep 29, 2011, at 6:13 AM, Glen Kent wrote: Hi, I see that i have multiple TCP sessions established with facebook. They come up even after i reboot my laptop and dont login to facebook! D:\Documents and Settings\gkentnetstat -a | more Active Connections Proto Local Address Foreign AddressState TCPgkent:3974www-10-02-snc5.facebook.com:http ESTABLISHED TCPgkent:3977www-11-05-prn1.facebook.com:http ESTABLISHED TCPgkent:3665 a184-84-111-139.deploy.akamaitechnologies.com:http ESTABLISHED [clipped] Any idea why these connections are established (with facebook and akamaitechnologies) and how i can kill them? Since my laptop has several connections open with facebook, what kind of information is flowing there? I also wonder about the kind of servers facebook must be having to be able to manage millions of TCP connections that must be terminating there. Glen
RE: facebook spying on us?
At least on a win 7 box, netstat -b gives the process that initiated the connection. Likely opened due to a link or something from some other web page. -Original Message- From: Patrick Muldoon [mailto:doon.b...@inoc.net] Sent: Thursday, September 29, 2011 9:25 AM To: Glen Kent Cc: nanog@nanog.org Subject: Re: facebook spying on us? On Sep 29, 2011, at 9:13 AM, Glen Kent wrote: Hi, I see that i have multiple TCP sessions established with facebook. They come up even after i reboot my laptop and dont login to facebook! D:\Documents and Settings\gkentnetstat -a | more Active Connections Proto Local Address Foreign AddressState TCPgkent:3974www-10-02-snc5.facebook.com:http ESTABLISHED TCPgkent:3977www-11-05-prn1.facebook.com:http ESTABLISHED TCPgkent:3665 a184-84-111-139.deploy.akamaitechnologies.com:http ESTABLISHED [clipped] Any idea why these connections are established (with facebook and akamaitechnologies) and how i can kill them? Since my laptop has several connections open with facebook, what kind of information is flowing there? Use a sniffer like wireshark, and see what the traffic is? Are you using a chat program that supports facebook chat? Or perhaps a game or an application that uses facebook for something? Really it could be anything as there are lots of applications that have grown up around the Facebook Eco system.. Also are you browsing the web? There are facebook like buttons and the such all over the web. So you don't even need to be logged in or have visited yet after the reboot. I also wonder about the kind of servers facebook must be having to be able to manage millions of TCP connections that must be terminating there. Lots of them. There is video of their new DC floating around that shows them.. http://www.datacenterknowledge.com/archives/2011/04/18/video-inside-face books-server-room/ -Patrick -- Patrick Muldoon Network/Software Engineer INOC (http://www.inoc.net) PGPKEY (http://www.inoc.net/~doon) Key ID: 0x370D752C Base 8 is just like base 10, if you are missing two fingers. - Tom Lehrer
Re: facebook spying on us?
Install Ghostery on your browsers and you'll see even more connections pages want to make behind the scenes to tracking sites etc. It's not just javascript. Greg On Sep 29, 2011, at 8:57 AM, valdis.kletni...@vt.edu wrote: On Thu, 29 Sep 2011 18:43:49 +0530, Glen Kent said: Any idea why these connections are established (with facebook and akamaitechnologies) and how i can kill them? Since my laptop has several connections open with facebook, what kind of information is flowing there? Probably you visited other pages that have links to Facebook on them. Try installing NoScript or similar in your browser and don't allow Facebook javascript, and see if these connections evaporate. Akamai is a content-caching service, just means somebody paid to have their content be (hopefully) nearer to you network-wise. I also wonder about the kind of servers facebook must be having to be able to manage millions of TCP connections that must be terminating there. Two words: Big Honkin' Load Balancers. OK, maybe more than two words. ;)
Re: facebook spying on us?
On Thu, Sep 29, 2011 at 06:43:49PM +0530, Glen Kent wrote: :Hi, : :I see that i have multiple TCP sessions established with facebook. :They come up even after i reboot my laptop and dont login to facebook! : :D:\Documents and Settings\gkentnetstat -a | more : :Active Connections : : Proto Local Address Foreign AddressState : TCPgkent:3974www-10-02-snc5.facebook.com:http ESTABLISHED : TCPgkent:3977www-11-05-prn1.facebook.com:http ESTABLISHED : TCPgkent:3665 :a184-84-111-139.deploy.akamaitechnologies.com:http ESTABLISHED : :[clipped] : :Any idea why these connections are established (with facebook and :akamaitechnologies) and how i can kill them? Since my laptop has :several connections open with facebook, what kind of information is :flowing there? : :I also wonder about the kind of servers facebook must be having to be :able to manage millions of TCP connections that must be terminating :there. : :Glen : For the more paranoid open source users, I have found using the xxxterm web browser to help quite a bit. You can read about it at http://www.xxxterm.org
Re: facebook spying on us?
Well what's making the connection? It looks like unencrypted http, if your social security number and last known addresses are streaming by you should be able to see them. It's a bit of a jump to say that FB (not that I'm particularly fond of them) is spying on you from a single netstat command. You probably clicked login with facebook for some site and it's just autologging you in or overzealous prefetching. Either way, I think we can all stop making tinfoil hats now... 2011/9/29 Glen Kent glen.k...@gmail.com Hi, I see that i have multiple TCP sessions established with facebook. They come up even after i reboot my laptop and dont login to facebook! D:\Documents and Settings\gkentnetstat -a | more Active Connections Proto Local Address Foreign AddressState TCPgkent:3974www-10-02-snc5.facebook.com:http ESTABLISHED TCPgkent:3977www-11-05-prn1.facebook.com:http ESTABLISHED TCPgkent:3665 a184-84-111-139.deploy.akamaitechnologies.com:http ESTABLISHED [clipped] Any idea why these connections are established (with facebook and akamaitechnologies) and how i can kill them? Since my laptop has several connections open with facebook, what kind of information is flowing there? I also wonder about the kind of servers facebook must be having to be able to manage millions of TCP connections that must be terminating there. Glen
