> Sadly, the issue is not related to build system complexity. That's what I said. It's a packaging issue (eg automated test and deploy). The build system in place is not particularly amenable to making the packaging process easier. The fact that people are having to do it by hand is IMHO a design flaw in the build and packaging process.
> When a new release happens, it relies on a number of volunteers who > build and test the source on their respective platforms. We generally > try to ensure that we have good build and test coverage before making a > release, which all takes time. For RedHat, for example, a complete set > of builds for a new release can take nearly a week to complete. Whilst > we could automate portions of this build process for all of the systems > we have buildbots for, we can't automate the testing process and this > stage remains important, if only to pick up bad builds. Can't is a pretty big word. Haven't is more accurate. I think that can be done, although it will require some changes in how development is done. What it means is that some thought has to be put into three things: a test framework, unit tests, and (since we're dealing with kernel code running in supervisor mode) an overall deadman monitor for the test machine to determine if the machine has hung based on some thing the test did. A good test framework that can be incorporated into the build system is a positive argument for looking at how the build is done. The second item (unit tests) requires the discipline to generate them with each patch. That can be hard, but it's by no means impossible. If a test can be done by hand, then it's not impossible to automate it. You need to have a way to create a known test scenario, simulate a sequence of events, and know the expected outcomes. Functional testing is relatively straightforward; load testing more difficult in that it requires multiple drivers to get realistic tests. The third item is well understood (easy to borrow from the embedded systems world). We use a 2514 as a console serial heartbeat monitor from a build system master controller to verify during the testing phase for other tools. The STONITH code from 'cluster' is also helpful to construct such things. If you're interested in using our test harness, let's talk. _______________________________________________ OpenAFS-info mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-info
