Arnaud,
On Dec 3, 2008, at 4:45 PM, Arnaud Legout wrote:
Hi,
You are right that you cannot obtain the best of each world.
However, it is plain wrong to claim that our results are equivalent
to what would have been obtained
with simulations, or that our results do not bring any new
significant insight compared to previous works.
I hope that a detailed reading of the paper will convince you. If
this is not the case, I would be pleased to
discuss specific concerns.
As we explain in section 3.2, the results we obtained would have
been hard, if not impossible, to obtain with simulations.
We show that the dynamics of the packets and of BitTorrent
algorithms have a major impact on the inter-ISP traffic savings.
In particular, we found that an initial seed insufficiently
provisioned may increase the inter-ISP traffic in case of locality.
Also, arguing that it is equivalent to run simulations than
controlled experiments makes me feel going back ten years
ago. And yes, running a real BitTorrent client is one of the major
difference compared to a simulation, but I don't
believe this difference can be discarded as a minor one.
If you have specific concerns on the methodology, I would be pleased
to discuss them.
Running in the wild experiments is important, but it just gives one
part of the picture. The other part can only be obtained
running controlled experiments, and varying well chosen parameters.
For instance, even if both Ono and P4P papers significantly improved
the comprehension of P2P locality,
they do not answer all questions (and never claim to do so).
However, there are still some fundamental problems to
explore, as explained in the introduction of our paper.
You are right, that a control experiment like yours can be very
helpful in getting a better understanding of what is in fact going on.
I'm talking about Marshall's sentence from first post in this thread
"This would be a very encouraging reduction of inter-ISP traffic if it
could be approached in practice by what we are doing here". I think
that if those results could be reproduced it would be great. I'm just
highly doubtful that they can.
While simulations are very important in learning how complex and
distributed systems work, in many cases they aren't very good at
predicting how they would work.
Few examples from your work:
- the content size is 100MB
- the upload speed is 20kB/s or 100kB/s
- you look at two different topologies: full mesh and two-tier topology
- very small inter ISP links ("We vary the inter-ISP link capacity
from 40kB/s to 2 000kB/s.")
Where did those values/configurations come from? Do you have any kind
of proof that this is how torrent network actually looks like? Why is
it 20kB/s and not 200kB/s or 1037kB/s? Why 100MB and not 257MB or
2057MB? I have never heard of inter ISP link with 2000kB/s capacity
(not mentioning the 40kB/s). I have heard about 10Gbit/s ones though.
I cannot see them anywhere in your work...
Few things that in my opinion are significant and are not taken into
consideration:
- other web activities (like web browsing, gaming, ftp, youtube,...)
of the users
- setting arbitrary max download and max upload speeds
- downloading many torrents at the same time
- vast part of real-world users are hidden behind a NAT
- users with very big symmetric links (e.g. 100Mbit) that are seeding
much more than downloading. Yes, there are such users...
- and many, many more.
Honestly, there are hundreds of factors, many of them very complex. Do
you really want to argue that the simplified model you presented is
good enough to reason about bittorent behavior?
I want one thing to be clear: I really like your paper, as a study
work that can help us understand the problem better. But with all due
respect, I simply cannot agree that this is a valid bittorrent model.
Hence, I don't see why those results would give us knowledge about how
the deployed ALTO system would perform in the real-world.
Regards,
Maciek Wojciechowski
_______________________________________________
alto mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/alto
