Hi, after fixing some issues we have set up a complete GNOME tinderbox compilation for 2.14 moduleset. You can see it here http://tinderbox.fisterra.org/all_trees.panel.html We want to point out that we did the minimum changes required to the tools we use, in order to have a first release.
Once this test has been completed we gathered some conclusions we want to share with you. I'll try to list the changes we made and some proposals for changes in the tools. Comments will be highly appreciated. * Client changes * tinderbox - build_shellscript: This script usually iterates over a list of trees passed as arguments. We think it is not useful for the whole GNOME compilation though. The reason is that the command that launches the tinderbox loop would be "tainted" because it should include something like "--tree :cairo:libxml2:libslt:.....:" up to more than 100 modules. The second reason is that all the modules will send their results to the same web page. We modified it in order to iterate over all the modules of GNOME, considering that each module is a tree. We get the list of the modules with the 'jhbuild list' command, that gives us the list of modules sorted in the right order. - buildcf: There is no need for the TreeData.pm file in the client because we will not use any information that could be stored there because jhbuild takes all data from the moduleset files. This would be a new buildcf_jhbuild version. - Sometimes, the build logs show weird symbols. Encoding issue? - Despite the current error parsing rules are quite good there are still some false positives, mainly related to compiler options (-fno-exception) or file names that include words like error, exception and so on. What about adding a gnome_parser to handle this process properly? - Neither the client nor the server are moduleset-dependent, changing the moduleset is as easy as change the value of a parameter. - Currently it is not possible no begin the build loop at a specified module (because we do a buildone over the whole list). Maybe it could be done just skipping all the modules into the build_shellscript loop until the specified module name was found. It would be also necessary to add a new parameter to the build_shellscript to pass this module name. - the build loop is not stopped even a basic library fails. You can argue that if we haven't got glib we have nothing. Well, we think that the tinderbox loop should not stop because it's probable that you have installed into your jhbuild environment an older version of the library that failed. This will allow the other modules to be compiled. In our humble opinion the use of 'jhbuild buildone' isolates the errors within each module. * jhbuild - added support for independent phase execution for the buildone command. This allows you to call for example jhbuild buildone -f gtk+ for only executing the configure phase for the gtk+ module. As I said before we do not like our current implementation because it lacks some design work. One important question for taking into account is that the available STATES are different per module, for example a module packaged into a tarball (without CVS access) will not have a STATE_CHECKOUT but STATE_DOWNLOAD or STATE_UNPACK. We think it would be nice to have this feature even for other projects. - the use of "--no-interact" parameter is mandatory. This prevents the compilation wait for an user input. * Server changes * tinderbox - TreeData.pm: this file is mandatory for the server. We modified it in order to add dynamically the list of jhbuild modules to the VC_TREE variable that stores all the available modules. It is also unnecessary to have there information about branches or repository roots, the server does only need the module names. For getting the list of modules, we call 'jhbuild list' as in the client. There is an issue here, we must ensure that both clients and server are using the same moduleset. Maybe adding something like "moduleset: gnome-2.14" to the header of the log files would help the server to discard client build logs that does not belong to the moduleset that is being built. Another possibility is to configure the server to receive compilation logs from clients that build modules of different modulesets. This will imply more work because the URL generation system should take into account the moduleset for creating the URLs. - Another interesting feature would be adding links to the phases inside the full log in the summary page. - The default outputs of the tinderbox are not quite good for a long set of modules, it would be nice to generate another kind of view. - We had some problems with the gtk+ module due to the '+' symbol. The tinderbox server removed it when creating the required files for showing the status pages. We fixed it adding the '+' symbol to the set of allowed symbols, but we do not know if it will break builds in another file systems different than ext3. Anyway IMHO there are some renames spread over the tinderbox code that should be managed in an unique location (for example the tinderbox still generates some URLs with "gtk" instead of "gtk+"). We have still some little bugs to fix and the system lacks some features that are a must for a real installation, like for example security. What are the next steps? I think the following: * client packaging: for distributing client compilation * integration with measurement tools: lcov, memcheck (valgrind)... * integration with GNOME Bonsai (ViewCVS after the SVN migration?) * integration with GNOME Bugzilla * security concerns: remote administration of the server, client authentication... How much can it take? Client packaging should be not difficult, we could start with a limited number of package types, .deb and .tgz for example. We already have a tinderbox integrated with some measurement tools here at Igalia, so it won't take also a lot of time. The last two points, integration with remote Bonsai and Bugzilla, should give us not a lot of problems (maybe Mike Taylor could give us more details). And finally the security concerns. It's a risky task IMHO. I saw some options for doing email authentication but no idea about how many time will it take. Anybody has some experience related to this? >From here we think that we have enough info to decide if this solution is suitable for gnome continuous integration. Is there any other info that should be added or answered to all the requirements Luis sent? Comments? Opinions? Regards. _______________________________________________ gnome-love mailing list [email protected] http://mail.gnome.org/mailman/listinfo/gnome-love
