[sorry about the dupes - website posting is acting up...] > I'm not sure I totally agree.
People not agreeing! On the INTERNET! Say it ain't so. ;^) > On Tue, Jun 24, 2008 at 2:07 PM, Jim Davis <[EMAIL PROTECTED] > com> wrote: > > USENet is actually pretty bandwidth inefficient beast. "Your" > server is peered > > with any number of other servers. Periodically your server asks all > its peers > > "got anything new?" - if so they send everything. > > The number of peers for a news farm is pretty small. The number of > peers for a P2P network is comparatively very large. My > understanding > is that most news goes through a small number of central hubs > now-a-days and it's very unlikely that TWC (for example) is peering > with 50 other nodes, but probably only 2-3 news major provider hubs. > These are also likely the same hub companies their users are going to > start using once they no longer have a local news source, effectively > converting traffic that was local to the ISP network into public > internet traffic. Sorta kinda? ;^) The "major" usenet hubs are simply not run by ISPs but rather by dedicated USENet providers - most ISPs don't care to support that kind of retention. In that sense a major peer (like Giganews or SuperNews) tends to contribute to overall traffic since access by customers is not "local". (Although many ISPs - notably Comcast - do not provide their own infrastructure. Comcast simply gives each customer a low-bandwidth GigaNews account so I have to assume that they have some dedicated hi-capacity in line with them as well.) > > In specific however each peer may choose to accept "everything", > only some > > groups or only certain messages (just new ones, or ones younger than > a > > certain age for example). In the end however if you've got a > network of a > > million peers then you've got a million copies of the message. > > There are almost certainly not a million news peers on the internet. > Nowhere close. No - silly number. But there are a lot. Remember that each site might peer with only a few, but that number exponentially grows as you fan out. This site keeps tracks of the "top 1000" servers out there and tracks nearly 5,000 (it's an opt-in list so there are a lot more than 5,000): http://news.anthologeek.net/ Like anything else there are a few giants, a lot of mediums and innumerable tiny guys. There are probably a million servers (or much more) all together but only a small fraction of them (perhaps 50) may account for 98% of the traffic. There are also a huge number of non-peering servers (Exchange discussions are actually USENet for example - most of them are private but they can peer if they like). Some groups are privately sponsored but public: both Microsoft and Adobe provide a large number of branded groups that are peered to most large hosts. All those little guys still peer with anybody else however and still make copies of all the groups they subscribe to - they don't copy as much (because they almost definately aren't subscribing to the entire set of groups) but they still copy. > > Since the servers have vastly different speed and storage profiles > you get huge > > diffences in "retention" - a huge server might be able to store a > year's worth of > > data while an average server only a month and a tiny server only a > day or two. > > True. > > > Compared to peer-to-peer which leaves the actual data on the "home" > servers > > until requested and, until then, just sends metadata USENet isn't as > resource > > friendly. Even a large binary message is shared amongst ALL peers > even if > > NOBODY ever requests it - if it's posted, it's transfered (again and > again and again). > > True, and these news propogation connections are (in theory) between > two ISPs on a very high bandwidth connenction. Home based data > transfers also go over this high speed connection, but at a the speed > of the slowest user's home connection, usually the upspeed max of > that > user divided by the number of upload fragments, which is very, very, > very slow in comparison. Absolutely true. I think I implied this later... but I probably should have put it it. *snipped a bunch of stuff we agree on... you know, the boring stuff.* > > So a 700 Meg movie downloaded from peer-to-peer would be 700 meg. A > 700 Meg > > movie on USENet is 750 meg converted to text, plus, say 160 meg of > PAR files > > plus maybe a 30 meg sample file. And, again - these are all copied > to every peer > > regardless of demand. A 700 Meg movie on a peer-to-peer network > that nobody > > ever requests uses essentially zero bandwidth. > > Okay, let's accept those numbers and do some math. Given 1000 > clients > downloading the same 700 meg file two ways. Either P2P clients or > via > News. Let's say there are 100 news servers, each feeding 10 people. > > 1) First, P2P - 1000 clients on a 700mb file at a 1:1 UL/DL ratio > - 700,000mb over the public internet > - 700,000mb over local ISP network > > 2) Next Usenet - 100 news servers transferring 780 > (file+encoding+pars) a max of 99 times > - 77,200mb over the public internet > - 780,000mb over local ISP lines > > 3) Last, consider my numbers are wrong and it's 10 news servers > feeding 100 people each > - 7,020mb on public internet > - 780,000mb over local ISP lines > > Even the worst usenet case (2) leaves 622,800 spare mb of internet > bandwidth for those messages that get downloaded and never read off a > server. That's 89%. If 89% of the messages on a news server were > never read, P2P would only just break even with usenet. But the peer-to-peer network - seeing that the file has become "popular" will dynamically optimize itself. In this case it means that somebody that's downloading the file faster will become a SOURCE of the file for others. In this way the flexibility of the network is used to provide a potentially better experience. It also assumes some things: Saying "local ISP" is misleading: if the P2P file is avaiable on a local peer it will be used. It doesn't HAVE to "go out to the Internet" - for popular files on large ISPs it becomes more and likely that you're only using local ISP resources. You're also assuming that all USENet traffic is "local" when most of it isn't - the largest providers (especially for binaries) aren't local ISPs but dedicated service providers. If you download that file from NewsGuy you're downloading it from a server in Texas no matter where you are for example. The traffic, in either case, is going to be a broad mixture of "local" and "outside" lines, I think. > > USENet is also limited geographically: you connect to whatever > server you > > connect to. A closer server (even a peer) might be faster, but you > don't have > > access. > > Only if you are using a public server and not a local ISP server. For any binaries (and binaries is what drive USENet's bandwidth) it's almost impossible to use ISP provided servers. Comcast provides a GigaNews account with 1 Gig/month limit - but you're still hitting GigaNews. TimeWarner's servers are so badly managed that most binary groups have less than a day's retention. Southwest Bell is even worse than Time Warner (I bought my father a rolling SuperNews account because Southwest bell was so very awful). In short nearly all bandband providers (in the US at least) either farm out their service to a dedicated USENet host or provide such a poor service that it's not worth it. I'm sure there are exceptions, of course, but I've never been lucky enough to find them. > > Peer-to-peer attempts to optimize the connection by downloading > files from > > the closest/fastest peer that hosts them and downloading using > multiple hosts. > > Once you have the file you also become a new node, a new host, for > that file > > thus allowing the system to optimize even further. Once I download > something > > from USENet it's mine and nobody else can see it. > > True, but all that P2P traffic is still public internet traffic. > Usenet traffic is primarily local ISP traffic. For all the reasons above I think that's a mistaken assumption. > I'd say more like Akami than the general web. Again, news servers > don't peer with "everyone", just certain peers they have agreements > with. At the ISP I worked at just after college, we actually only > peered with one provider. But that provider peered with another and another and others and so on - it doesn't matter how many peers a single host has, it matters how many peers are in the network. Even tho' your host only peered with one host a post to "comp.lang.javascript" ends up on MANY more hosts than just those two. Each host that it ends up on copied it from another. > > This is simplistic of course - there are all sorts of tricks. > On-the-wire > > compression helps text-only USEnet more than peer-to-peer and much > > of USENet traffic is still run over the relatively open backbone > connections - > > it's the last mile connections that are truly getting clogged and > these are > > exactly what's leveraged most by peer-to-peer. > > Yup. Plus, usenet doesn't saturate your upstream like P2P Oh - you don't have to sell me on USENet - I'm one of the originals. ;^) I absolutely HATE P2P personally. One of the biggest issues, for me, is the total lack of moderation. USEnet, even the most chaotic binary groups, are still "mob moderated" - troublemakers are run out, bad files are noted, deliberate attempts at sabotage (uploading viruses as files for example) are smacked down hard. There's also a certain flair amongst regular posters - a pride. You know who does the best work, who provides the highest quality, etc. The person that always provides samples, technical details, uses the most open technology, etc. You are part of a community. It's may be honor among thieves, but it's still honor. P2P (unless it's changed radically since I tried it last) is a haystack full of needles - and they hurt! Files have no descriptions, no sense. Finding something is a crap shoot at very best even with all they have and quality is amazingly variable. Yes P2P is "easier" than USENet, but not so much so that I can understand how it's moved to such prominence. > > But in the end I'm still a little suprised that USENet doesn't have > a stronger showing. > > Heh - longer reply than I meant, but I am still not surprised. ;) I guess that little rant at the end there is why I'm so suprised... P2P sucks! Why is it so popular (I ask rhetorically) when something as elegant and peopled as USENet exists? Actually that's a great idea... it should be simple enough to have a usenet group that accepts requests where those requests are automatically converted to behind-the-scenes P2P requests and the resulting files uploaded to another group. That'd be cool. ;^) Jim Davis ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;203748912;27390454;j Archive: http://www.houseoffusion.com/groups/CF-Community/message.cfm/messageid:262638 Subscription: http://www.houseoffusion.com/groups/CF-Community/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.5
