2014-11-05 17:14 GMT+01:00 Thomas D. <[email protected]>: > Hi, > > sorry for my late response. I was very busy the last days. > > np at all, I guess we all know that
> On 2014-10-30 15:08, Rainer Gerhards wrote: > > rsyslog has a test suite for many years now, and I am continuously > > expanding it. But it unfortunately covers only a subset of the code > pathes. > > Some of this is fixable, but e.g. config option interaction grows > > exponentially, so you can never write (or run) sufficient tests for that. > > Every user should be able to run the test suite. > > That's possible today. You just need to setup the environment. I don't say any magic, but in order to run e.g. the mysql tests, you need to have - mysql server up and running - test-specific user credentials and permissions - test-specfic databases if you have that, no problem at all. The test suite must be able to run in a sandbox. > > That's not possible at the moment. Have a look at mysql, PHP, json-c, > salt... this should be possible for rsyslog, too. > I really don't understand that part. If by sandbox you mean that the testbench must install everything it needs, then it must setup a complete environment, including databases etc. That also means it must query which distro it runs on. I've installed VMs for these environments the past days, and 100% scripting this is for sure another big project to do. Also, with all that setup, the testbench probably runs 90 minutes or so per platform - and growing. > Also, I am more talking about something like a unit testing framework. > This should allow easy module testing. > I tied to set this up, got stuck. I sent a mail asking for help, so far nobody came up offering it ;) > I am familiar with testing web applications, C# or C++ code on Windows. > I never tried something similar on Linux yet, but this should be nearly > the same. Let's see if I find the time to create a small demo so that > you can get an idea (but don't expect anything before December). > That would be great, whenever it is. > > Regarding the last sentence: This *is* testable. Testing all option combinations grows exponentionally, no way around this. If you have 2 binary options, you need 2^2 test, if you have 20, you need 2^20 for *full* coverage. If you can do full coverage with fewer tests, you have solved the P=NP problem. It's easy to reduce e.g. the scheduling problem to this config testing problem. Poor RSA... ;) > Maybe it will require a > rewrite of some components if your architecture never thought about > being testable but this shouldn't be the problem. > > > > Still, it would be very well to add a couple of hundred test cases, but > > than the problem given above kicks in. Right now, I usually try to add a > > test for any problem that was seen in practice. > > > > The testbench is invoked via "make check". It is run automatically every > > day around noon CET and I receive notifications of these results. > > The most important thing about CI everybody has to understand: Every > commit has to be tested. There are no exceptions. Tests must run > automatically or people will start to ignore/bypass.. > yeah, give me the resources and we can make them sweat. I don't have the money for them. If Travis offers them, I just need to stop developing for a couple of weeks - except... > > I can also help you to set up everything on GitHub. I will do that in my > rsyslog clone. If you like it, you can move everything to the main > repository. > ... you do it - that would be awesome! > > But as said before, don't expect anything before December. > > wouldn't be any quicker if I begin to learn the systems involved... Thanks, Rainer > > -Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
