I understand your explanation about builds. I know builds are stable and do work fairly well on a variety of platforms.
But when I talk about CI, I don't talk specifically about rebuilds, but I am in the same problematic as you: I think about testing. Build bots have a wrong name in that they are not restricted to buids, but my intent was to evaluate the possibilities of CI software in terms of testing. Of course it needs an automated backend, because the build (I should say the _task_) is machine-controlled. But as long as some tests are available and are automated (make check is our backend here), the CI soft can gather the test results, publish them, warn/blame devs who broke the devs, and also keep track of the results over time and give us statistics. So indeed I was not exactly focused on build, but rather on testing, given the current status of libs. Now, I agree with you on another point: the backend (make check or whatever more specific app) and the actual TEST CASES are the important topic to work on. And there's nothing much I can give back about that, all of you will be much more experienced. I have no experience in graphical framework testing. So in the meantime, I just tried to see what we could get from a CI software. You've just demotivated me so I may give up with this task anyway. I will think and have a look at what could help in automating gui interaction. Hope I will eventually bring something you can find useful. BTW, I can try to work on the tests themselves as well, but I don't know where to start. Is there any test-oriented roadmap anywhere? On Sat, Feb 19, 2011 at 2:10 PM, Carsten Haitzler <ras...@rasterman.com> wrote: > On Mon, 14 Feb 2011 12:17:03 +0100 Lionel Orry <lionel.o...@gmail.com> said: > > just fyi - CI is one of the lesser worries we have. so let's not make this > more > that it is - builds for us are stable and well tested. i rebuild efl between > 1-4 times a day. sometimes much more. between the developers we have little > issue with rebuilds. > > now let's get to the core of this - REBUILDS arent a problem. i rebuild efl > and > then some in about 6 mins on my desktop/laptop. i dont use distcc - nothing > beyond the single cpu there. some smart Makefile to allow parallel builds > between libs. so this isn't an issue. making sure things build is the least of > our issues. > > what we need to track is BUGS. when someone introduces a bug - the longer it > is > not found, the harder it is to fix later. this means we arent really about > rebuilds. we are about TESTING every change that we can. we have SOME test > suites right now - expedite is an automated one for evas, we have some for > eina > as well ans a bit for ecore - but they are mostly very thing and done test a > lot. elementary has a test, but its interactive. > > what we need to do is work on fleshing out tests where they are mostly good > (expedite for example) so they test more or everything - and can automate the > test. then for others create tests ANd find ways to automate them - > increasingly we will need to find a way to automate gui interaction as thats a > huge amount of what we do - and then verifying that the results of the > interaction (logical and display) are "right". > > this here has nothing to do with build bots, hosts, jenkins etc. but requires > building and improving other infra and tools. we should be doing that long > before we care about the infra to run those tests. > > so... who is volunteering to work on the tests? :) > >> On Sat, Feb 12, 2011 at 2:32 AM, Ravenlock <ravenl...@ravenlock.us> wrote: >> > >> > Can FreeBSD users play too, with the majority of ya'll running linux? >> > >> >> In the case of Buildbot and Jenkins, the concept of master/slave is >> central and should definitely be considered seriously. That means not >> only a build server, but distributed build over a build farm of >> computers with different OSes. >> >> I know Jenkins better so I'll talk a little bit about that. >> >> Say the master is a Linux box. Fine, it runs the java container and >> Jenkins application as the server machine, and it can also be used as >> a build machine by following a task to try to build a library or >> application, or generate doxygen, or get code coverage / unit testing >> / whatever a script or a plugin can do for us. >> >> But this master can also trigger a build on a distant machine, should >> it be running any Unix (in that case, ssh connection between the 2 >> machines is the easiest and painless way, with a dedicated >> 'hudson_slave' user for example for security) or even Windows >> (commands issued via JLNP or a windows service). >> >> One of the machines could be running a 'BSD OS. >> >> Going further: we only want one machine because the other ones are >> owned by the devs, they bite and don't want a distant robot to execute >> commands and take up their CPU. >> >> Fine again. Let's virtualize. Jenkins is able to connect, via its >> plugins, to various virtual machine interfaces (vmware, virtualbox). >> QEmu is easy to automate so no plugin needed. So we can simply run >> different virtual machines on the server, on demand, when a build is >> triggered on them. It is transparent from the Jenkins interface, the >> virtual machines appear like other physical nodes. >> >> So, many possibilities. FreeBSD users will have their playground too, >> don't worry. :) >> >> Lionel >> >> ------------------------------------------------------------------------------ >> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: >> Pinpoint memory and threading errors before they happen. >> Find and fix more than 250 security defects in the development cycle. >> Locate bottlenecks in serial and parallel code that limit performance. >> http://p.sf.net/sfu/intel-dev2devfeb >> _______________________________________________ >> enlightenment-devel mailing list >> enlightenment-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel >> > > > -- > ------------- Codito, ergo sum - "I code, therefore I am" -------------- > The Rasterman (Carsten Haitzler) ras...@rasterman.com > > ------------------------------------------------------------------------------ The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb _______________________________________________ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
