That was a very thoughtful analysis, Serguei. Thank you.
Peter -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Serguei Osokine Sent: Tuesday, November 28, 2006 1:16 PM To: theory and practice of decentralized computer networks Subject: RE: [p2p-hackers] Do people stop studying eDonkey/Overnet ? On Tuesday, November 28, 2006 Tien Tuan Anh wrote: > With eDonkey got shutdown in September, is it illegal now to study > or do experiment of this network ? Are there still studies going on? > Or did people stop looking at it already ? eDonkey was not exactly shut down in September. Works just fine even now. I use it all the time (with eMule open source client). The original network creators might have abandoned it (or not - I'm not sure), but the network itself was virtually unaffected by this. My eMule client routinely shows 10M+ simultaneous users, though there might be some duplicates there, and the actual usage can be less. Not sure about why people do not research it, really. Maybe it is seen as too simple to merit the researchers' attention. I mean, trying to figure out what really happens in a BitTorrent cloud is a complicated task. No one really knows what's going on there, and the algorithms BitTorrent uses cannot be easily translated into the description of the network behaviour. With eDonkey, the situation is seemingly different. You find the content on a server (semi-centralized search), and then start downloading it from all the found sources. Seems easy and simple, which is why people might be not interested. However, personally I find what is really going on there very curious. And never mind their tit-for-tat approach, which probably will attract reseaschers, if their attitude towards BitTorrent is any guide. As a matter of fact, I'm not particularly interested in it. What seems really fascinating in eDonkey to me is that it is a case study in how the totally moronic design decisions lead to a creation of an incredibly popular network - and here I'm as surprised why no one investigates this as you are. So for starters, most of the content on the eDonkey network is invisible. I don't know how did they achieve that (thought that the global search of all servers should be really a global serarch of all servers), but if you'll start using it, you'll see that people request your files many times more frequently when these files are still being downloaded. As soon as the download is finished, the frequency of requests for this file drops dramatically; people still ask for it, but much less often. I think that they might have some analog of Gnutella's download mesh that lets current downloaders exchange information about each other in real time. You can see this easily after starting to download something: even if the global network search returns one or two locations for this file, immediately after you start downloading, this number grows by an order of magnitude (say, to 10 or 20), and most of the newly found locations do not have a full copy - they are current downloaders, just as you are. As a result, most of the content on the network is effectively "black" - it is present there, but is much less available than it could have been otherwise, and every download is performed from many fewer sources than it really could. Also, there are multiple bugs in code that often cause the uploader client to sit idle without sending anything out for hours, even though when someone finally is able to start downloading, it reports waiting all this time for the uploader (that was idle!) to become available. This also decreases the average download speed - the overall resulting underutilization of the upload network bandwidth should be quite sizable. As if this all would not be enough, there is also a system of "download queues", which is severely limiting the single source data transfer speed. The average download rate from one source is maybe 50-100 bytes per second (when you count in the waiting), which is why you have to have either lots of sources (which is not easy for rare movies), or download multiple movies at once. And the problem is that when you're shutting down your client, you are losing your place in waiting queues, so next time you start it - even though of course your interrupted downloads will be automatically resumed - you'll have to begin from the end of all lines again, and it takes at least several hours to get into a decent position even when you are in many lines at once. The situation is really very interesting - despite having this design, which looks like the most moronic design that one could possibly come up with, this P2P network is very popular. Last time I compared the data (in March of 2006), it had more simultaneous users than any other P2P network - above 3M, which was about 10% above its closest rival Gnutella, (as reported by Slyck). One of the reasons for such apparent popularity is a direct consequence of the eDonkey network disadvantages. Since it takes hours to really start downloading something, people have to run their clients in 24/7 mode - otherwise their already low download rate drops even more as they lose their place in waiting queues after shutdown. Besides, rare content downloads (for which it is a network of choice) are made even longer by the fact that much of the content is "invisible" and cannot be found by the search, as mentioned above. Not sure whether it is a bug or a feature, but the end result is the same - users have to spend even more time online, since their downloads are slower than they could be with other designs. All this increased online time does several things: first, it inflates the simultaneous usage number - so probably the total number of people using eDonkey network is not as high as one might imagine, and it might be even lower than the total number of Gnutella users. But this is just a cosmetic and "bragging" effect, so to speak. Other effects are more serious: Since more users are online simultaneously, the network has more content. In terms of the number of titles, I mean - the sheer increase in the data volume would be unnoticed, because it is exactly matched by the increased download demands of extra clients. This is, of course, migh be partially offset by the content invisibility. However, due to many clients running 24/7, this content tends to be more stable: in other networks it is a big problem that you often cannot complete the download because all sources (or the only source) go off-line. This problem is less pronounced in eDonkey network, though it is achieved by the fairly draconian measures, as you can see. But on the balance, users seem to find this mixed bag of advantages and disadvantages attractive (though not as many users as one might be led to believe by comparing its simultaneous usage number to other P2P nets). Of course, if you want to quickly download a single song, this P2P system is about the last place where you'd go. It is evidently targeted at people downloading tens of large files simultaneously, which is the only way to achieve something resembling a decent total download speed. This client is to be run constantly, and even though every single file takes forever to download, when you have tens of them being downloaded at once, every so often a random file from you list is finally done. Naturally, you cannot predict in advance how long is it going to take - the movie dowload can take a couple of days, or it can take a month. So yeah, eDonkey network is interesting all right. If you are thinking whether to research it or not, do not hesitate. It definitely seems "underresearched". And since it apparently manages to turn its design flaws and bugs into advantages, I cannot even start imagining what would be the net effect of possible optimizations on the overall network health and user experience. Of course, the optimizations might make it better, but then they might also kill it - who knows? :-) Disclaimer: these are all my personal observations - not a result of any systematic research. All the data above applies to the stuff that I'm using the network for (relatively rare movies), and the network behaviour can be very different for the average user's content (whatever it is). Which I think makes the proper research even more interesting. Best wishes - S.Osokine. 28 Nov 2006. -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Tien Tuan Anh Dinh Sent: Tuesday, November 28, 2006 5:59 AM To: theory and practice of decentralized computer networks Subject: [p2p-hackers] Do people stop studying eDonkey/Overnet ? Hi all, In doing a survey of p2p file-sharing, i was little suprise on the modest number of papers about eDonkey/Overnet compared to Bittorrent, Kazaa, Gnutella; while many believed that eDonkey/Overnet has the largest number of users (in Europe) compared to other file-sharing netowrks. With eDonkey got shutdown in September, is it illegal now to study or do experiment of this network ? Are there still studies going on ? Or did people stop looking at it already ? Regards, Anh _______________________________________________ p2p-hackers mailing list [email protected] http://lists.zooko.com/mailman/listinfo/p2p-hackers _______________________________________________ p2p-hackers mailing list [email protected] http://lists.zooko.com/mailman/listinfo/p2p-hackers _______________________________________________ p2p-hackers mailing list [email protected] http://lists.zooko.com/mailman/listinfo/p2p-hackers
