I have a rule of thumb that any system will suffer scaling problems when you increase its size to 10 times its previous size.
The largest Tahoe-LAFS grid to date was the allmydata.com grid, which had more than 100 storage server processes (nodes) on it. Therefore I assume that Tahoe-LAFS will have scaling problems when it hit 1000 storage server processes. :-) I think the allmydata.com grid had about 1000 simultaneous clients. I'm not sure if I think Tahoe-LAFS in its current form would handle 10,000 simultaneous clients. :-) On Sun, Jul 25, 2010 at 9:12 PM, Brian Warner <[email protected]> wrote: > > Probably not very well. Yes, currently it tries to establish a "full > mesh" of connections: every client keeps a TCP connection open to every > server (and usually each node behaves as both a client and a server). Nitpick: it is really a "bi-clique" topology—every client connects to every server, but servers don't connect to each other and clients don't connect to each other ("full mesh" would be every node connecting to every other node). Actually, that's what I wish it were (#1086, #1136), but currently every server acts as a client which means that it is a bi-clique topology combined with all-servers-connect-to-all-servers. The distinction is relevant here, because if you have 100 storage servers and 1000 storage clients then with the current topology each server will hold open 1099 connections and each client will hold open 100 connections, whereas with a full mesh topology each client would hold open 1099 connections. Regards, Zooko http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1086# servers should attempt to open connections to clients http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1136# don't run a storage client if you don't need one _______________________________________________ tahoe-dev mailing list [email protected] http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
