Let me hijack this thread at least a little... First: the issue of multiple DB engines and such as real pain in the b... for me. I'd like to have as many automated tests as possible, but all those plugins that need "elaborate" setup usually fall short as it is pretty hard to do (for me and on every dev machine). Remember that for some plugins I don't even have the slightest idea of how the backend works. And for sure not even remotely how I could conduct an automated test, which requires
a) getting to a known initial state b) shuffeling data at a test run c) extracting the results and check if they are ok d) probably re-setting things to look untouched again (ok, somewhat redundant with a). Keeping the ES example, I know barely how I can do a, c, and d manually so that I can conduct manual tests (and after some weeks this even requires review of notes, as I really don't care/use ES in day-to-day operations). Doing this in an automated way is way above my head. Now think about the many other plugins, and those that hopefully will be written. I am not that super-guru, and I think for many folks it's hard to be really good at all of these tools (congrats if you are!). Disabling a test if the feature is not enabled might help an end-"user", but it doesn't solve the core problem that I don't detect regressions right in time. So... now to the hijacking. I have a dream. As a community, I think we would be able to create a machine where we could run nightly builds, nightly *testbench* runs (on a much more elaborate testbench including all the backend system), night doc generation and upload ... and so on. This would need to be a group effort as we would need to drag in all the experts that could help craft these tests. Also, we would need some folks who would take care of handling the server itself, I mean the usual admin things like keeping it secure, creating user accounts for other collaborators and so on. I know we have many site reliability engineers (and similar positions) on our list, so they could chime in here. Such a machine would probably also enable me to find bugs quicker. Right now, for things like ES that I don't work on regularly, the main obstacle to start a bughunt is to setup the env, remember how get around with it and so it. If there is a ready machine with clear procedure, I could hunt bugs much more efficient. But quite frankly this is a too large task for us small Adiscon to handle. We simply have no funding for all that talent required. Regarding the platform, I'd assume that a virtual server somewhere in the cloud could be used (hopefully a sponsored one). And as I am right now in "whish mode", it would help immensely to have access to maintained build environments for Solaris, SmartOS, AIX, *BSD and so on, together with some helping hands I could ask if I don't find may way around easily enough. OK, that was the dream. Does anybody think we could realize at least a bit of it in practice? ;) Thank, Rainer On Fri, Feb 21, 2014 at 4:54 PM, Radu Gheorghe <[email protected]>wrote: > Hello increasingly-active-rsyslog-list! > > I think we need to decide how to move forward with the testbench, > whenever people find the time to work on it. The problem came from > comments of this issue: > https://github.com/rsyslog/rsyslog/issues/10 > > (check out the comments after the issue was closed) > > Basically, the general decision point is to see how we should handle > rsyslog's integrations with various data stores. Whether that's > Elasticsearch, MongoDB, MySQL, etc. > > So far, dealing with files was done by... wait for it... writing to > actual files. This gets more complicated with DBs because you need to > have the DB there. Installing it might take time and tons of > dependencies. > > There are a couple of solutions mentioned in the GitHub issue, but I'm > throwing a new one after a bit of thinking. Let's assume that the > likes of Elasticsearch are already there, and make a list of > "requirements" so that one can see what is assumed. > > The point is, if you don't have ES installed, you probably don't care > about ES tests. So all tests about ES could, for instance, try and do > a: > > curl localhost:9200 > > And if there's an error, or the version is too old, skip the test and > print a warning. > > What do you think? Other suggestions? > > Best regards, > Radu > > P.S. The point of starting this thread is, as you might see from > GitHub, that I've worked on a few omelasticsearch tests, and I'd like > to see what we need in order to get them into the main rsyslog > testbench. Blindly assuming ES is there is a bit too much, as Rainer > pointed out. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
