On Thursday, 2010-03-18, at 7:37 , Jody Harris wrote: > This raised it's ugly head in last night's backup. Fortunately, the symptom > showed up on the very last upload, so I found the timing data on the first > file in the list.
> I will attempt to attach a PDF of the timings page to this message. If there > is a ticket for this, or a ticket this should be attached to, let me know. Thanks for the bug report! This is #983. I guess I should have Cc:'ed you on that ticket since your earlier report was part of the process of finding the bug. This is actually my highest Tahoe-LAFS-hacking priority right now, and the way I've been going about it is to write up this "StringChain" package which I hope could be used to fix this problem. That's pretty much done (footnote 1), and the next step is for me to write a patch and off it to Brian for foolscap #149. (I'm not sure that StringChain is the right way to fix foolscap #149 and I suspect Brian is even less sure, but it is worth a try.) I'm hoping to get a fixed foolscap into Ubuntu Lucid so that Lucid users don't have this problem. In the meantime, the work-around is not to use mutable files and to keep your mutable directories nice and small. Immutable files perform much better than mutable files do, for this reason (#983) as well as a few other reasons. They also have the handy property that they are immutable, so you can be sure that the file contents have not changed. Another problem with mutable files is that they break the FTP and SFTP frontends (#680, #941). Could you remind me why you use mutable files in your backup scripts? I remember the first time you posted to this list about why you are using mutable files I thought "Huh, I hadn't thought of that.". But now I don't remember your reason. Regards, Zooko http://tahoe-lafs.org/trac/tahoe-lafs/ticket/680# Fix for mutable files with FTP http://tahoe-lafs.org/trac/tahoe-lafs/ticket/941# SFTP frontend fails when listing a directory containing a mutable file, because it relies on node.get_size() to be an integer http://tahoe-lafs.org/trac/tahoe-lafs/ticket/983# high CPU load on storage servers when uploading large mutable file http://foolscap.lothar.com/trac/ticket/149# O(n**2) CPU/malloc during receive of large strings footnote 1: benchmarks of StringChain vs. a few other data structures N is bytes processed; results are in nanoseconds per byte; on my MacBook Pro with 2.33 GHz Intel Core 2 Duo. To run the benchmarks yourself just get the StringChain package (http://tahoe-lafs.org/trac/stringchain ) and run the same command. The important thing to note is that StringChain is scalable (speed doesn't drop off as N increases) as well as fast. The task named "_alternate_str" is the one that most closely approximates what foolscap needs to do. The implementation named "Stringy" is the on e that most closely approximates what foolscap currently does. $ python -OOu -c 'from stringchain.bench import bench; bench.slow_bench()' impl: StringChain task: _accumulate_then_one_gulp N: 10000, ns/op: best: 2.79, 2th-best: 3.39, ave: 3.90, 2th-worst: 4.70, worst: 4.82 (of 5) N: 50000, ns/op: best: 2.76, 2th-best: 2.78, ave: 2.95, 2th-worst: 2.94, worst: 3.36 (of 5) N: 100000, ns/op: best: 2.33, 2th-best: 2.34, ave: 2.40, 2th-worst: 2.45, worst: 2.54 (of 5) N: 500000, ns/op: best: 2.03, 2th-best: 2.10, ave: 2.26, 2th-worst: 2.28, worst: 2.70 (of 5) N: 1000000, ns/op: best: 1.98, 2th-best: 2.01, ave: 2.30, 2th-worst: 2.45, worst: 2.68 (of 5) N: 5000000, ns/op: best: 2.11, 2th-best: 2.14, ave: 2.18, 2th-worst: 2.20, worst: 2.25 (of 5) task: _many_gulps_str N: 10000, ns/op: best: 5.20, 2th-best: 5.51, ave: 6.60, 2th-worst: 6.10, worst: 10.61 (of 5) N: 50000, ns/op: best: 3.24, 2th-best: 3.24, ave: 3.42, 2th-worst: 3.42, worst: 3.88 (of 5) N: 100000, ns/op: best: 2.87, 2th-best: 2.88, ave: 2.96, 2th-worst: 2.93, worst: 3.22 (of 5) N: 500000, ns/op: best: 2.80, 2th-best: 2.84, ave: 2.90, 2th-worst: 2.95, worst: 3.00 (of 5) N: 1000000, ns/op: best: 2.70, 2th-best: 2.70, ave: 2.77, 2th-worst: 2.84, worst: 2.88 (of 5) N: 5000000, ns/op: best: 2.70, 2th-best: 2.71, ave: 2.72, 2th-worst: 2.74, worst: 2.76 (of 5) task: _alternate_str N: 10000, ns/op: best: 6.79, 2th-best: 7.10, ave: 7.84, 2th-worst: 8.61, worst: 8.80 (of 5) N: 50000, ns/op: best: 4.74, 2th-best: 4.78, ave: 4.83, 2th-worst: 4.88, worst: 4.90 (of 5) N: 100000, ns/op: best: 4.34, 2th-best: 4.35, ave: 4.50, 2th-worst: 4.53, worst: 4.78 (of 5) N: 500000, ns/op: best: 3.98, 2th-best: 4.12, ave: 4.25, 2th-worst: 4.36, worst: 4.50 (of 5) N: 1000000, ns/op: best: 3.96, 2th-best: 4.00, ave: 4.07, 2th-worst: 4.02, worst: 4.36 (of 5) N: 5000000, ns/op: best: 3.96, 2th-best: 3.97, ave: 4.07, 2th-worst: 4.18, worst: 4.19 (of 5) impl: StringIOy task: _accumulate_then_one_gulp N: 10000, ns/op: best: 4.10, 2th-best: 4.29, ave: 4.84, 2th-worst: 5.29, worst: 5.51 (of 5) N: 50000, ns/op: best: 5.06, 2th-best: 5.60, ave: 5.75, 2th-worst: 6.14, worst: 6.32 (of 5) N: 100000, ns/op: best: 4.51, 2th-best: 4.52, ave: 4.60, 2th-worst: 4.67, worst: 4.69 (of 5) N: 500000, ns/op: best: 3.80, 2th-best: 3.90, ave: 4.70, 2th-worst: 5.10, worst: 6.71 (of 5) N: 1000000, ns/op: best: 4.19, 2th-best: 4.29, ave: 4.69, 2th-worst: 5.08, worst: 5.49 (of 5) N: 5000000, ns/op: best: 5.27, 2th-best: 5.30, ave: 5.57, 2th-worst: 5.89, worst: 6.00 (of 5) task: _many_gulps_str N: 10000, ns/op: best: 4.70, 2th-best: 5.51, ave: 5.70, 2th-worst: 5.70, worst: 6.91 (of 5) N: 50000, ns/op: best: 24.28, 2th-best: 24.34, ave: 27.87, 2th-worst: 25.36, worst: 40.90 (of 5) N: 100000, ns/op: best: 38.64, 2th-best: 38.84, ave: 39.03, 2th-worst: 39.22, worst: 39.38 (of 5) N: 500000, ns/op: best: 158.61, 2th-best: 159.18, ave: 162.47, 2th-worst: 165.73, worst: 168.40 (of 5) N: 1000000, ns/op: best: 455.90, 2th-best: 459.57, ave: 471.79, 2th-worst: 488.72, worst: 490.74 (of 5) N: 5000000, ns/op: best: 2730.39, 2th-best: 2734.04, ave: 2771.03, 2th-worst: 2734.04, worst: 2848.67 (of 3) task: _alternate_str N: 10000, ns/op: best: 6.99, 2th-best: 7.08, ave: 8.10, 2th-worst: 7.70, worst: 11.11 (of 5) N: 50000, ns/op: best: 8.34, 2th-best: 8.34, ave: 8.52, 2th-worst: 8.38, worst: 9.18 (of 5) N: 100000, ns/op: best: 22.37, 2th-best: 22.68, ave: 22.82, 2th-worst: 22.90, worst: 23.44 (of 5) N: 500000, ns/op: best: 74.92, 2th-best: 75.96, ave: 76.61, 2th-worst: 76.87, worst: 78.45 (of 5) N: 1000000, ns/op: best: 135.24, 2th-best: 136.14, ave: 139.87, 2th-worst: 137.46, worst: 153.85 (of 5) N: 5000000, ns/op: best: 1000.30, 2th-best: 1003.59, ave: 1076.12, 2th-worst: 1003.59, worst: 1224.47 (of 3) impl: Stringy task: _accumulate_then_one_gulp N: 10000, ns/op: best: 2.38, 2th-best: 2.69, ave: 3.01, 2th-worst: 3.00, worst: 4.20 (of 5) N: 50000, ns/op: best: 15.86, 2th-best: 16.16, ave: 18.65, 2th-worst: 16.34, worst: 28.66 (of 5) N: 100000, ns/op: best: 26.60, 2th-best: 26.80, ave: 30.51, 2th-worst: 34.00, worst: 35.07 (of 5) N: 500000, ns/op: best: 102.15, 2th-best: 102.16, ave: 103.20, 2th-worst: 103.29, worst: 105.36 (of 5) N: 1000000, ns/op: best: 266.58, 2th-best: 266.73, ave: 280.54, 2th-worst: 286.67, worst: 311.59 (of 5) N: 5000000, ns/op: best: 1543.46, 2th-best: 1554.51, ave: 1553.37, 2th-worst: 1554.51, worst: 1562.16 (of 3) task: _many_gulps_str N: 10000, ns/op: best: 2.60, 2th-best: 2.91, ave: 3.15, 2th-worst: 3.50, worst: 3.72 (of 5) N: 50000, ns/op: best: 15.06, 2th-best: 15.08, ave: 15.34, 2th-worst: 15.60, worst: 15.86 (of 5) N: 100000, ns/op: best: 24.20, 2th-best: 24.21, ave: 25.17, 2th-worst: 24.88, worst: 27.80 (of 5) N: 500000, ns/op: best: 98.93, 2th-best: 101.46, ave: 101.85, 2th-worst: 102.31, worst: 105.05 (of 5) N: 1000000, ns/op: best: 264.17, 2th-best: 266.87, ave: 268.33, 2th-worst: 270.90, worst: 272.28 (of 5) N: 5000000, ns/op: best: 1556.56, 1th-best: 1556.56, ave: 1586.65, 1th-worst: 1616.73, worst: 1616.73 (of 2) task: _alternate_str N: 10000, ns/op: best: 4.29, 2th-best: 4.70, ave: 5.48, 2th-worst: 6.01, worst: 6.39 (of 5) N: 50000, ns/op: best: 5.16, 2th-best: 5.20, ave: 5.38, 2th-worst: 5.44, worst: 5.68 (of 5) N: 100000, ns/op: best: 20.53, 2th-best: 20.65, ave: 20.77, 2th-worst: 20.71, worst: 21.30 (of 5) N: 500000, ns/op: best: 81.43, 2th-best: 81.60, ave: 82.35, 2th-worst: 82.54, worst: 84.52 (of 5) N: 1000000, ns/op: best: 143.76, 2th-best: 146.12, ave: 146.30, 2th-worst: 146.19, worst: 149.30 (of 5) N: 5000000, ns/op: best: 917.51, 2th-best: 925.97, ave: 923.90, 2th-worst: 925.97, worst: 928.21 (of 3) _______________________________________________ tahoe-dev mailing list [email protected] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
