On October 07, 2003 04:27 pm, Martin Stone Davis wrote: > Ed Tomlinson wrote: > > On October 07, 2003 02:52 pm, Toad wrote: > > > > [...SNIP...] > > > >>The basic problem we have been having is that the stable branch does not > >>work. If the stable branch worked, I would be able to commit disruptive > >>changes to the unstable branch without exploding the list. As it is, I > >>should have tested the changes on a local test network before committing > >>them (not like anyone else here does). The solution is to debug the > >>unstable branch, and get it good enough to merge. If people want to set > >>up an independant network that uses prehistoric builds, that's fine, but > >>if everyone uses them, I don't see how we can possibly move forward, for > >>the reasons I just outlined. > > > > In this I agree with toad. It is going to be a very hard slog to get new > > code tested and debugged in a developer only network... Old builds were > > replaced because they were buggy. Going back is not going to fix those > > bugs. Instead we will have people complaining that freenet eats to much > > cpu and has other (long fixed) bugs. > > > > What we have not is not working well. IMHO we need to put efforts into > > fixing this. This means freenet will have problems for the next couple > > of days (or maybe weeks, but I hope not). I can live with it. > > > > Hopfully we will be able use more small test nets to verify basic > > functionality before realeasing into the wild. This implies we need some > > sort of testing script that can be executed automaticly on a test > > network. I think testing of this type would lead to a much more stable > > network. > > > > Before you say but thats a developer network - its not. I would start > > from a standard set of datastores and do a common set of operation on the > > nodes measuring results. For instance: > > > > do inserts work? > > are they faster/slower than the last test (what is the trend)? > > do retrieves of known data work? > > what is the performance trends? > > is FEC working? > > and a bugs are found we add tests to try and catch them before we > > (re)release code with problems. > > > > This would imply that new code would get tested before being placed in a > > snapshot. > > > > Comments? > > That sounds right to me. The main objective should be to fix existing > bugs and to reduce the chance of releasing a highly buggy version "into > the wild" in the future. > > I think Toad (or someone) has mentioned a "smoke test suite" before as > an objective. That sounds to me like what you're suggesting. > > Having never tried such a thing, my question is: how would you run the > test to see that, for example, inserts are faster/slower than the last > test? Doesn't that mean you have to run a large private network of > computers to see how it works? Or instead, can you get away with > running a very small network (maybe even on the same machine) where you > build in artificial delays to simulate the "real-life" delays in a > larger network?
No you have 10 or so data stores backed up. You restore them and start 10 nodes on one box. It will be slow. You run the same tests on the same stores each time and record the results. Process should be something like rebuild from CVS restore DS reboot start nodes start scripts record results evaluate results if they are ok release new snapshot This would need ONE faily hefty box or a few older ones. Ed _______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
