On Fri, 2009-02-27 at 18:48 +0100, Luis Fernando Muñoz Mejías wrote: > Rainer, > > Good and bad news... > > > > That sounds really great. Before you start coding or preparing > > > anything, let me check how well our DBs perform, because it's not > > > yet clear if they'll be able to cope with the high insertion rate we > > > expect. If we don't go for the Oracle database this work doesn't > > > make sense. I bet we'll want the Oracle, anyways. > > > > Sounds fair. > > Good news: I did my tests and, for many tasks I need to do, Oracle is > our way to go. So, I'm willing to write the module, with your > guidance/advise. > so far this sounds good ;)
> As I said, I need **excellent** performance. I definitely need batch > operations, the ability to prepare the statements given as arguments on > the configuration file, and not to commit entries one by one, but after > a number of entries are ready or (better) after some not so small > time. According to the advise I got from experts around here, I'll have > to use Oracle Call Interface for this module, I don't know if there are > any licensing issues. I can't comment on the licensing issue, I simply don't know what Oracle demands. On thing to do it is let the output module handle the "combination work" together. The output module is called one per message, however, it does not mean the output must directly write them to the database. It may buffer them until the batch is large enough. But this currently needs to be implemented on the output module basis. Obviously, that will not make coding simpler. If we find a sponsor for the necessary non-trivial extension of the core engine, the output module's task may become much easier. If things go well, such a sponsor may show up... > > It seems I'll have to review how rsyslog's queing modules work... I would suggest not to move into them - but, of course, if you like to... Lol, this is the non-trival task I talked about, there are numerous subtleties and, of course, they are weakly documented (but the inline doc is quite good). > > > > For this evaluation, I already have a timestamp formatter that fits > > > into Oracle, something that can be used with the property replacer, > > > like %timereported:::date-oracle%. > > > The bad news is that this timestamp formatter works perfectly on > interactive sessions (sqlplus) but not on non-interactive ones, f.i, in > Python scripts. You need to call Oracle's to_timestamp(string, format), > and by bloating your code with this ugly function the rfc-3339 formatter > is good enough. So I won't submit this one. > Sounds fair ;) Do you have a time frame for your project? (and maybe a rough overview of the "big picture" - I am always soooo curios ;)) Rainer > Cheers. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
