On the move right now but yes a CI system is what we need - I recommend TeamCity and am happy to maintain Debian, Ubuntu, SmartOS boxes. Possible Solaris but need to check. We do this for our platforms at the moment. A fuller reply once I'm home.
Sent from my iPhone > On 21 Feb 2014, at 17:52, Rainer Gerhards <[email protected]> wrote: > > Let me hijack this thread at least a little... > > First: the issue of multiple DB engines and such as real pain in the b... > for me. I'd like to have as many automated tests as possible, but all those > plugins that need "elaborate" setup usually fall short as it is pretty hard > to do (for me and on every dev machine). Remember that for some plugins I > don't even have the slightest idea of how the backend works. And for sure > not even remotely how I could conduct an automated test, which requires > > a) getting to a known initial state > b) shuffeling data at a test run > c) extracting the results and check if they are ok > d) probably re-setting things to look untouched again (ok, somewhat > redundant with a). > > Keeping the ES example, I know barely how I can do a, c, and d manually so > that I can conduct manual tests (and after some weeks this even requires > review of notes, as I really don't care/use ES in day-to-day operations). > Doing this in an automated way is way above my head. Now think about the > many other plugins, and those that hopefully will be written. I am not that > super-guru, and I think for many folks it's hard to be really good at all > of these tools (congrats if you are!). > > Disabling a test if the feature is not enabled might help an end-"user", > but it doesn't solve the core problem that I don't detect regressions right > in time. > > So... now to the hijacking. I have a dream. As a community, I think we > would be able to create a machine where we could run nightly builds, > nightly *testbench* runs (on a much more elaborate testbench including all > the backend system), night doc generation and upload ... and so on. > > This would need to be a group effort as we would need to drag in all the > experts that could help craft these tests. Also, we would need some folks > who would take care of handling the server itself, I mean the usual admin > things like keeping it secure, creating user accounts for other > collaborators and so on. I know we have many site reliability engineers > (and similar positions) on our list, so they could chime in here. Such a > machine would probably also enable me to find bugs quicker. Right now, for > things like ES that I don't work on regularly, the main obstacle to start a > bughunt is to setup the env, remember how get around with it and so it. If > there is a ready machine with clear procedure, I could hunt bugs much more > efficient. But quite frankly this is a too large task for us small Adiscon > to handle. We simply have no funding for all that talent required. > Regarding the platform, I'd assume that a virtual server somewhere in the > cloud could be used (hopefully a sponsored one). And as I am right now in > "whish mode", it would help immensely to have access to maintained build > environments for Solaris, SmartOS, AIX, *BSD and so on, together with some > helping hands I could ask if I don't find may way around easily enough. > > OK, that was the dream. Does anybody think we could realize at least a bit > of it in practice? ;) > > Thank, > Rainer > > > On Fri, Feb 21, 2014 at 4:54 PM, Radu Gheorghe <[email protected]>wrote: > >> Hello increasingly-active-rsyslog-list! >> >> I think we need to decide how to move forward with the testbench, >> whenever people find the time to work on it. The problem came from >> comments of this issue: >> https://github.com/rsyslog/rsyslog/issues/10 >> >> (check out the comments after the issue was closed) >> >> Basically, the general decision point is to see how we should handle >> rsyslog's integrations with various data stores. Whether that's >> Elasticsearch, MongoDB, MySQL, etc. >> >> So far, dealing with files was done by... wait for it... writing to >> actual files. This gets more complicated with DBs because you need to >> have the DB there. Installing it might take time and tons of >> dependencies. >> >> There are a couple of solutions mentioned in the GitHub issue, but I'm >> throwing a new one after a bit of thinking. Let's assume that the >> likes of Elasticsearch are already there, and make a list of >> "requirements" so that one can see what is assumed. >> >> The point is, if you don't have ES installed, you probably don't care >> about ES tests. So all tests about ES could, for instance, try and do >> a: >> >> curl localhost:9200 >> >> And if there's an error, or the version is too old, skip the test and >> print a warning. >> >> What do you think? Other suggestions? >> >> Best regards, >> Radu >> >> P.S. The point of starting this thread is, as you might see from >> GitHub, that I've worked on a few omelasticsearch tests, and I'd like >> to see what we need in order to get them into the main rsyslog >> testbench. Blindly assuming ES is there is a bit too much, as Rainer >> pointed out. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
