I guess I should also comment on this one, but it's more a FYI type of posting.
2014-10-31 6:02 GMT+01:00 singh.janmejay <[email protected]>: > Im unsure if I have understood you correctly, but you seem to be thinking > of CI as a way of writing tests(like a testing framework, eg. google-test, > make-check etc). > > But actually CI is a process(not a tool/framework). I guess this is the > best source of information about it: > http://en.wikipedia.org/wiki/Continuous_integration > This sounds much like what we do with rsyslog for quite a while. > > So having a CI process in simple terms means that the integration-branch > (which happens to be master in our case) and all the other long-lived > branches such as feature-branches that live for a few days/weeks, should be > monitored by an automated build process. > > they are on a daily basis. > This build process would trigger 'make check' for every new commit (people > often use git-push based post-update hook to notify build-server). This > build would be reported on a CI dashboard, which tells people if the build > was green or red. > > Given the fact that the testbench runs 30 to 60 minutes currently (and growing), I don't know if it is a good idea to run it for each commit. So far, daily worked well for me. Note that I usually [for more complex things] run either the complete testbench or at least a subset that handles the modules I modified before doing a commit. As a side-note, the testbench has caught numerous problems in the past. > CI servers also go one step further and support output-artifact integration > so developers can see which tests failed, why they failed etc. > > This is done automatically in the current system, as part of the result email notification. I would appreciate if someone would create whatever is needed to tie in our current system into the github process. TBH, I don't see enough additional value compared to the current system to spent time on it. What we don't have is these unit tests. Well... a handful exists, but I stopped that route as it didn't really work out. One problem is that rsyslog is very modular. So if you want to test any function that really does work, you need to link in and activate a lot of the rsyslog runtime. In true isolation, only tiny functions can be tested. But as I said previously: I would be very happy if someone comes up an actually implements this, or at least provides the necessary templates to do so. If nobody comes up, I'll probably try myself to see how I could integrate current unit tests. Rainer So no changes are required to the way rsyslog codebase is right now. We > have a directory full of tests, we have a command that returns non-zero > exit code for failing tests and this setup is sufficient for us to setup an > automated build process. > > We should check if travis can run 'make check' for us. > > > On Thu, Oct 30, 2014 at 8:38 PM, Rainer Gerhards <[email protected] > > > wrote: > > > 2014-10-30 13:35 GMT+01:00 singh.janmejay <[email protected]>: > > > > > +1 for testsuite and CI. > > > > > > > I tried to get a quick glimpse at CI, but not only rsyslog doc is full of > > acronyms and references that talk only for those in the field ;) > > > > Can someone tell me in quick words how in CI a test to do this looks > like: > > > > - spin up a rsyslogd listener instance "LST" > > - spin up a rsyslogd sender instance "SND" > > - make "SND" send a couple of thousand messages to LST > > - stop both instances *when work is finished* > > - count if LST did receive all messages and did so in correct format > > > > This is one of the basic tests in the current rsyslog testbench. > > > > I would appreciate if a control file for CI could be provided, so that I > > can judge the effort that's to be made if I were to convert to that > system > > (it sounds interesting). Is there some volunteer who would migrate the > > current testbench and educate me ... in the way James has done this for > the > > rsyslog-doc project? > > > > Thanks, > > Rainer > > > > > > > +1 for time based releases. > > > > > > It'll be valuable to have adhoc minor/micro releases. Release feature > by > > > feature or few features/fixes at a time type of thing. > > > > > > -- > > > Regards, > > > Janmejay > > > > > > PS: Please blame the typos in this mail on my phone's uncivilized soft > > > keyboard sporting it's not-so-smart-assist technology. > > > > > > On Oct 30, 2014 6:01 PM, "Thomas D." <[email protected]> wrote: > > > > > > > Hi, > > > > > > > > +1 for an always working "master" branch. > > > > > > > > Do your work in feature branches. Only merge when you think the > changes > > > > are ready. Don't merge when you think "I am ready, but this needs > > > testing". > > > > > > > > Regarding the testing problem in general: I would stop adding new > > > > features for a while. Spend more time in improving code quality. > > > > Try to find/create a working test suite. There will always be the > > > > problem that nobody will test your stuff. So you need a way to write > > > > tests. Yes, creating a test suite will take some time. But in the end > > it > > > > will improve the software quality and boost your development. > > > > Remember that you can use CI for free with GitHub. Every pull request > > > > could be automatically tested for you... > > > > > > > > > > > > +1 for a time-based release approach. > > > > > > > > > > > > PS: Debian Jessie will freeze at 23:59 UTC on the 5th of November > 2014. > > > > Just a reminder if you want to see some of the recent changes in > > > Jessie... > > > > > > > > > > > > -Thomas > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > > > > DON'T LIKE THAT. > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > > DON'T LIKE THAT. > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > > > > -- > Regards, > Janmejay > http://codehunk.wordpress.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
