On Mon, Sep 09, 2002 at 10:16:18PM +0100, Matthew Toseland wrote: > I have some free time, and would like to spend it being paid to work for > the Freenet Project, which I am informed has the funds. Previously > prominent developers - oskar and tavin - have been paid $2500 for > approximately 2 months work on freenet. I would like to use the same > deal, and IMHO I am the only person available at the moment who has worked > on the code sufficiently AND has the time for it to be practicable. > I can work at least 35 hours per week, for most of the foreseeable > future. In particular, I can work from immediately to 2 months later. 40 hours per week, for 8 weeks, for $2500 = $312.50/week = $62.50/weekday... > > There are a number of major unresolved issues with freenet at the > moment, but the one that has the greatest impact is probably the > DataStoreBug. The problem with this, apart from the fact that I don't > know fs/ very well, is that bugfixing is indeterminate. There are > however some things that can be done in this direction that are > reasonably determinate. For example, in order of desparation (reading > the docs/source first would be a good idea, fixing them up where > necessary would also be useful): > a Removing all the red-black trees, and converting them to skiplists. > Not as trivial as it sounds, because the nodes are mutilated. This may > not solve the bug, but it should simplify the code substantially for > little cost (I have heard that the worst case cost is worse with > skiplists... the consensus is that it's not bad enough to be a big > problem though). 1-3 days $200 > b Trying to prove that there is a problem with red-black trees, by making > a tester that compares it with a skiplist (therefore faster than > reverifying), more thoroughly. The current tests are based on special > cases found while debugging the redblack tree. 1-2 days $100
$300 for converting red-black trees to skiplists and testing red-black trees. > c Reverse engineering the accounting structures, and documenting them. 1-5 days // $300, but combine with the fsck > d Find conditions likely to cause a DSB. Some people have reported > getting it regularly. 1-5 days $0, it's public knowledge. > e Making the fsck etc tools work reliably, at least to detect > corruption. 5 days $300+$300 for documenting the accounting structures. Total $700. > f And to fix it, by rebuilding it. Not necessarily on the critical path. > g band-aid: do a read-only check (this may take a while) when starting > the node, if it's broken, try to regenerate it, if that fails, reset > the store. Should be optional, and a clear warning in the log at NORMAL. 2 days, if above done right $200 > h log all changes to the accounting tree (assuming migrating to > skiplists etc hasn't fixed it), in a parsable format. Include a 1-3 days //$100 > backtrace. Export the initial state of the tree. Checkpoint every 1-2 days //$100 > timeperiod, either internally (more work), or by shutting it down > and fsck'ing it externally. This gives us a period during which it <1 day // $100, but combine > corrupted. Then build up the tree, step by step, just as the node > would have, fsck'ing after each completed change (or in a binary > search, if this is computationally prohibitive, which I doubt), and 2 days //$200 > find the exact instant of corruption. Then look at the backtrace, and $500 for finding the exact instant of corruption, or proving that this is not the area where corruption occurs. $500 for actually fixing the bug. > fix the bug. This can also be converted into a regression test: run a $500 for regression test. > node, checkpoint it every <moderate period of time> and fsck, keep the > old state and log all changes since the clean old state along with > backtraces, and email the list with a stack trace of the instant of > corruption if/when we recorrupt. I am going to call this 'the nuclear $500 > option', and it requires a working accounting structures fsck and a > good idea of what conditions will cause corruption; it's also probably > best to try converting to skiplists first. > > Not knowing the code intimately, I can only give very rough estimates > for the effort involved in each stage. How detailed do I need to make my > proposal? Need input. One possibility, for a purely results-oriented scheme, would be: $300 for removing RBTree and testing it more thorougly. $700 for a working fsck. $500 for finding the location of the instant of corruption. $500 for fixing the corruption, or establishing that it isn't being corrupted in the tree. $500 for automated DSB regression tester, using the tracing machinery set up in finding it. The disadvantage being that I could be completely wrong about the time requirement of any or each task. Another possibility is "give me the money, pay by the hour, and I'll make my best effort on the above, roughly in order, possibly moving the fsck rebuild command to a lower priority". -- Matthew Toseland [EMAIL PROTECTED] [EMAIL PROTECTED] Freenet/Coldstore open source hacker. Looking for $coding (I'm cheap)
msg03944/pgp00000.pgp
Description: PGP signature
