On Thu, Jan 23, 2014 at 6:46 PM, David Lang <[email protected]> wrote: > so what exactly is being proposed? > > It sounds as if we are talking about omprog, but that also captures stderr > of the program that's executed, and watches that stderr for specific > keywords. > > I think yes, that's bascially the idea.
> what keywords are you talking about, and what actions will be taken? how > would these actions differ from the program just stalling reading of the > pipe or exiting with an error code? > > don't know yet - this needs to evolve/be specified. Currently working on a python test script and test integration. > This is not going to be able to support batch mode, some number of logs > are going to be lost in any case because we will dequeue messages (consider > them delivered) when we put them in the pipe to the program, any feedback > we get from stderr can only affect logs that haven't been sent to the pipe > yet. > yeah, it's not totally reliable. It's the same scenario we have with omfwd in plain tcp syslog mode. Judging from that, it's still good for many applications. If that's a problem, we could even go down (via config option) to do a half-duplex mode, where only one message is posted at one time, and reply awaited. Obviously much slower, but there always is a price for reliability. > what other restrictions are there? > I don't know yet. Quite honestly, I don't try to design this fully through. This time, I'd prefer to do some thing, see how they work out and what hurts, refactor and begin a new cycle. In this mode, I hope to have something workable (even without real feedback, much like UDP) within a day or two. That would probably be enough to see the utility and if it is being used. A full design requires more time - time I don't have and I don't know if it would be well spent. You may consider this effort of a simple kind of "omprog" evolution but from the marketing PoV this seems not to be very appealing... So let's do what everyone does and exaggregate the terms. Still, I think it's technical far superior, and we could have a fairly working SOLR script (with batches) very quickly... Rainer > > David Lang > > > On Thu, 23 Jan 2014, Radu Gheorghe wrote: > > Or, maybe it's easy to add imthrift&omthrift? >> >> 2014/1/23 Radu Gheorghe <[email protected]> >> >> +1. Let's make what we already have (and maybe add some stuff with >>> minimum >>> effort) easy to use. Let's see what the adoption is. Let's see people >>> complain about performance, if that's ever going to happen and then we >>> can >>> "do it right" according to demand. >>> >>> 2014/1/23 Boylan, James <[email protected]> >>> >>> I agree with Rainer. >>> >>>> >>>> While 'doing it right' from from the start and building from the ground >>>> up perspective is a fantastic goal, the reality of trying to fit that in >>>> with an entirely new direction is difficult at best. When you have a >>>> mounting list of items that need to be worked on already, sometimes >>>> taking >>>> that fast to implement first step to just show how much better something >>>> can be is critical to get stakeholders to back investing the kind of >>>> time >>>> required to make a major overhaul to something. >>>> >>>> -- James >>>> >>>> -----Original Message----- >>>> From: [email protected] [mailto: >>>> [email protected]] On Behalf Of Rainer Gerhards >>>> On Thu, Jan 23, 2014 at 9:09 AM, David Lang <[email protected]> wrote: >>>> >>>> On Thu, 23 Jan 2014, Rainer Gerhards wrote: >>>>> >>>>> On Thu, Jan 23, 2014 at 8:00 AM, David Lang <[email protected]> wrote: >>>>> >>>>>> >>>>>> On Thu, 23 Jan 2014, Radu Gheorghe wrote: >>>>>> >>>>>>> >>>>>>> 2014/1/22 Rainer Gerhards <[email protected]> >>>>>>> >>>>>>> >>>>>>>> On Wed, Jan 22, 2014 at 3:17 PM, Radu Gheorghe < >>>>>>>> [email protected] >>>>>>>> >>>>>>>> >>>>> I think it all depends on how much work we are going to have to do. >>>>> >>>>> why do we need to do anything? can't omprog with a JSON template >>>>> already do what's needed? >>>>> >>>>> >>>> IMHO quite some things, but it lacks some others (like a feedback >>>> capability, see below). My interest is to extend it in a way that mostly >>>> solves these issues. >>>> >>>> >>>> >>>>> as long as we stick to pipes, we aren't going to be able to get per >>>>> message success/failure messages (the buffering in the pipe will >>>>> prevent that from working well) >>>>> >>>>> >>>> That's right, but we get pretty close to "around when it begins to >>>> fail". >>>> Per message is hard in any case. Think e.g. TCP forwarding: we never >>>> know >>>> exactly when the connection broke, so this is also approximate. >>>> >>>> >>>> I guess I'm not fully understanding what's being proposed for the >>>>> short term. >>>>> >>>>> >>>> Let me coin it into a non-technical statement: I try to do the minimal >>>> thing necessary to make it easier for people to connect to destinations >>>> for >>>> which rsyslog does not yet have any native plugins. Among the tech >>>> issues, >>>> this probably means make folks aware that they actually *can do that* >>>> (as >>>> you say, to a large extent even with what we have today, but that seems >>>> to >>>> be too well-hidden). Once this is done, I'd like to sit back, relax, >>>> gain >>>> some experience and then see how to proceed further. >>>> >>> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
