Holger Freyther said: > > I understand that in the beginning infrastructure will be solely operated > by Nokia, would it make sense to setup a Jenkins outside of Nokia?
I am not sure. The only immediate benefits I can see from doing that is (1) removes one blocker for non-Nokia entities to add test machines with their own configurations, (2) the full build logs could immediately be made public. The disadvantage would be disruption on the existing codereview/CI setup for all developers. The build logs issue is tracked by https://bugreports.qt.nokia.com/browse/QTQAINFRA-139 and should be solvable with a less drastic solution than an entire switchover of the CI tool. > Is there > a list of what Pulse/Scripts do that Jenkins is not capable of doing right > now? No, not right now. It would make sense to create this. In our setup, the responsibilities of Pulse/Jenkins/whatever else at that layer consists of "merely": - schedule arbitrary test scripts over a pool of machines - gather and store build logs, test results - reading the contents of some git repositories - run arbitrary pre/post build commands for a few things (e.g. automatic reboot of virtual machines after each test run) I expect these requirements are simple enough to be satisfied by many tools. The actual test logic and test machine setups are not implemented in Pulse at all, they are almost all public and could be immediately used outside of Nokia's current Pulse setup. - http://qt.gitorious.org/qt/qtqa <- the scripts used for testing Qt5 in CI - http://qt.gitorious.org/qtqa/testconfig <- glue between CI tool (e.g. Pulse) and the above scripts - http://qt.gitorious.org/qtqa/sysadmin <- metadata for configuration of test machines There is some documentation at http://wiki.qt-project.org/index.php/Public_Autotest_Infrastructure which should be expanded. > I assume the Qt Project starts off while still using your internal > and non-free pulse system while an alternative is built? > Yes, as far as the CI system is concerned, the Qt project going live meant nothing other than the firewall rules blocking access to codereview.qt-project.org from outside of Nokia being dropped. > > If they are generic enough to be usable for e.g. "any" DirectFB backend, > > then it may be appropriate to check them into the source tree, if they > > are fairly stable. If they're likely to be unstable, or they are very > > large, we don't have any good solution at the moment :( > > The question is a general one. How do we have custom results? In this > specific case the DirectFB software image decoders generate different > results to Qt, but if I would build a SoC platform and I decide to run > Qt tests and my hardware is giving a slightly different result i would > want to override some results? Any idea? > > In case of image results it would make sense to look for 'foo.png' > in images/foo.png, images/QPA-DirectFB/foo.png, > images/QPA-DirectFB/HW/foo.png, That would be fine, we've used that approach for testdata sometimes in the past. However it's hard to decide how much detail needs to go into the identification of the testdata. For example, maybe changing the version of system libraries such as freetype could change the font rendering slightly, so would you then include the freetype version number in the string as well? > maybe even have qrc:/ look into an external path? > Not sure about that. _______________________________________________ Qt5-feedback mailing list [email protected] http://lists.qt.nokia.com/mailman/listinfo/qt5-feedback
