On Fri, Feb 21, 2014 at 7:40 PM, Sphonic <[email protected]> wrote:
> On the move right now but yes a CI system is what we need - I recommend > TeamCity and am happy to maintain Debian, Ubuntu, SmartOS boxes. Possible > Solaris but need to check. We do this for our platforms at the moment. A > fuller reply once I'm home. > > That would be really great. Looking forward to the full info :-) Rainer > Sent from my iPhone > > > On 21 Feb 2014, at 17:52, Rainer Gerhards <[email protected]> > wrote: > > > > Let me hijack this thread at least a little... > > > > First: the issue of multiple DB engines and such as real pain in the b... > > for me. I'd like to have as many automated tests as possible, but all > those > > plugins that need "elaborate" setup usually fall short as it is pretty > hard > > to do (for me and on every dev machine). Remember that for some plugins I > > don't even have the slightest idea of how the backend works. And for sure > > not even remotely how I could conduct an automated test, which requires > > > > a) getting to a known initial state > > b) shuffeling data at a test run > > c) extracting the results and check if they are ok > > d) probably re-setting things to look untouched again (ok, somewhat > > redundant with a). > > > > Keeping the ES example, I know barely how I can do a, c, and d manually > so > > that I can conduct manual tests (and after some weeks this even requires > > review of notes, as I really don't care/use ES in day-to-day operations). > > Doing this in an automated way is way above my head. Now think about the > > many other plugins, and those that hopefully will be written. I am not > that > > super-guru, and I think for many folks it's hard to be really good at all > > of these tools (congrats if you are!). > > > > Disabling a test if the feature is not enabled might help an end-"user", > > but it doesn't solve the core problem that I don't detect regressions > right > > in time. > > > > So... now to the hijacking. I have a dream. As a community, I think we > > would be able to create a machine where we could run nightly builds, > > nightly *testbench* runs (on a much more elaborate testbench including > all > > the backend system), night doc generation and upload ... and so on. > > > > This would need to be a group effort as we would need to drag in all the > > experts that could help craft these tests. Also, we would need some folks > > who would take care of handling the server itself, I mean the usual admin > > things like keeping it secure, creating user accounts for other > > collaborators and so on. I know we have many site reliability engineers > > (and similar positions) on our list, so they could chime in here. Such a > > machine would probably also enable me to find bugs quicker. Right now, > for > > things like ES that I don't work on regularly, the main obstacle to > start a > > bughunt is to setup the env, remember how get around with it and so it. > If > > there is a ready machine with clear procedure, I could hunt bugs much > more > > efficient. But quite frankly this is a too large task for us small > Adiscon > > to handle. We simply have no funding for all that talent required. > > Regarding the platform, I'd assume that a virtual server somewhere in the > > cloud could be used (hopefully a sponsored one). And as I am right now in > > "whish mode", it would help immensely to have access to maintained build > > environments for Solaris, SmartOS, AIX, *BSD and so on, together with > some > > helping hands I could ask if I don't find may way around easily enough. > > > > OK, that was the dream. Does anybody think we could realize at least a > bit > > of it in practice? ;) > > > > Thank, > > Rainer > > > > > > On Fri, Feb 21, 2014 at 4:54 PM, Radu Gheorghe <[email protected] > >wrote: > > > >> Hello increasingly-active-rsyslog-list! > >> > >> I think we need to decide how to move forward with the testbench, > >> whenever people find the time to work on it. The problem came from > >> comments of this issue: > >> https://github.com/rsyslog/rsyslog/issues/10 > >> > >> (check out the comments after the issue was closed) > >> > >> Basically, the general decision point is to see how we should handle > >> rsyslog's integrations with various data stores. Whether that's > >> Elasticsearch, MongoDB, MySQL, etc. > >> > >> So far, dealing with files was done by... wait for it... writing to > >> actual files. This gets more complicated with DBs because you need to > >> have the DB there. Installing it might take time and tons of > >> dependencies. > >> > >> There are a couple of solutions mentioned in the GitHub issue, but I'm > >> throwing a new one after a bit of thinking. Let's assume that the > >> likes of Elasticsearch are already there, and make a list of > >> "requirements" so that one can see what is assumed. > >> > >> The point is, if you don't have ES installed, you probably don't care > >> about ES tests. So all tests about ES could, for instance, try and do > >> a: > >> > >> curl localhost:9200 > >> > >> And if there's an error, or the version is too old, skip the test and > >> print a warning. > >> > >> What do you think? Other suggestions? > >> > >> Best regards, > >> Radu > >> > >> P.S. The point of starting this thread is, as you might see from > >> GitHub, that I've worked on a few omelasticsearch tests, and I'd like > >> to see what we need in order to get them into the main rsyslog > >> testbench. Blindly assuming ES is there is a bit too much, as Rainer > >> pointed out. > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >> DON'T LIKE THAT. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
