That's correct, though Pastry hasn't used a neighbour set for years
now (it's been out since at least version 1.3). We haven't simulated
PAST (our DHT application) on very large networks, choosing to focus
instead on our Scribe multicast system. Each simulated Pastry node
takes a few kilobytes of overhead, so if your DHT stores on the order
of a megabyte per node it should still be possible to simulate
thousands of nodes. Make sure to account for the overhead of
replicas when doing the calculation.
We have several different storage back-ends for PAST, so if you could
(for example) replace your actual data objects with small place-
holder objects a memory-based back-end could still simulate a large
network, depending on the amount of RAM available. Ultimately though
it's true the disk performance and memory requirements will make it
impractical to simulate DHTs with lots of keys (or large keys) on a
single machine.
hope that helps,
James
On 13 Mar, 2007, at 10:32 PM, John Casey wrote:
Hi Jeff, it sounds like you simulate the routing core, neighbour
and leafsets? based on a virtual network topology? Anyhow, the
biggest problem/limit that I've had with other DHT simulators is
the actual database size and memory use coming from the storage
modules. ie. each simulated node had its own database. Moreover,
because simulated node data are stored in different files it feels
like HDD seek latency goes up as well. Does free pastry do anything
to get around this sort of situation? what sort of node population
would be viable on 4GB machine ?
On 3/13/07, Jeff Hoye <[EMAIL PROTECTED] > wrote:
...> And by the way, how long would it take to run a simulation
consisting of
> 100K nodes ?
It depends what you want to do, for some tasks, I recall that it
can run at real-time with 100K nodes. It's the memory that
ultimately leads to the limitation.
Let us know if you have more questions!
Thanks!
-Jeff
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
