On Jan 8, 2009, at 10:24 AM, Shawn Willden wrote: > It's still 30-40 seconds per image, even if you can fill the pipe > completely.
Yes, there is some interesting network application engineering left to be done if you want smooth browsing of big files stored on tahoe. Your idea about storing some of the shares locally is interesting, plus of course we might want to do pre-fetching or thumbnails. Hm, if you use Windows, you might be interested in how the tahoe-w32- client does these things. Or even if you don't! > I know that par2, which uses Reed-Solomon, is terribly slow par2 is terribly slow. I don't know why, but it isn't just because of the computational cost of Reed-Solomon. I benchmarked par2 vs. zfec when I first released zfec: http://allmydata.org/trac/zfec/browser/zfec/README.txt """ On my Athlon 64 2.4 GHz workstation (running Linux), the "zfec" command-line tool encoded a 160 MB file with m=100, k=94 (about 6% redundancy) in 3.9 seconds, where the "par2" tool encoded the file with about 6% redundancy in 27 seconds. zfec encoded the same file with m=12, k=6 (100% redundancy) in 4.1 seconds, where par2 encoded it with about 100% redundancy in 7 minutes and 56 seconds. """ > I just used the darcs binaries available in Ubuntu 8.10. The > package version is 2.0.2-2ubuntu, so I would think I did use darcs- > v2, and it still took tens of minutes. Good, you already have darcs-v2, then if you were looking at the wiki page today you would see that it says to use the "--lazy" option and to fetch from http://allmydata.org/source/tahoe/trunk-hashedformat instead of from http://allmydata.org/source/tahoe/trunk- darcs1format . Using darcs-v2 executable, --lazy, and fetching from the -hashedformat repository will mean that it takes only a couple of minutes instead of tens of minutes. Regards, Zooko --- Tahoe, the Least-Authority Filesystem -- http://allmydata.org store your data: $10/month -- http://allmydata.com/?tracking=zsig _______________________________________________ tahoe-dev mailing list [email protected] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
