Well, it isn't really prioritization that I am worried about. For example, if I have a developer running something in DEBUG, that is actually quite high priority for him.
With TCP, there is no way to separate them at the system OS tcp buffer level without opening several concurrent TCP connections, basically one connection for each ActionQueue. TCP has only one output buffer per connection while SCTP can have several parallel streams operating over a single socket connection. Maybe I will hack on it when I get time. If I have sent a bunch of DEBUG level messages, they are in the TCP/IP output buffer. If I want to send a USER level message, it gets put in the queue after the DEBUG messages I have already "sent". With SCTP, the streams are handled in round-robin fashion. If the packet is flagged for "out of order delivery", one could potentially send a higher priority packet NOW despite a lot of lower priority messages being sent. The problem with prioritization is that higher priority traffic can starve "lower" priority traffic of service. You can already easily accomplish the prioritization with the exsting Linux ToS bits. By default Linux has four classes of traffic with the pfifo_fast qdisc. If you mark the packets accordingly, you can already get higher priority service for certain traffic (marking ToS 0 for EMERGENCY, for example). The problem with that is a program spewing a bunch of USER messages can block DEBUG messages if USER is placed at a higher priority. What I had in mind was basically a separate worker thread for each log level. This prevents any of them blocking the others. George > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Rainer Gerhards > Sent: Monday, April 26, 2010 12:00 AM > To: rsyslog-users > Subject: Re: [rsyslog] Syslog over SCTP? > > This is interesting, but unfortunately I lack time to look deeper into > it. > > I just wonder if another - maybe more flexible - approach would be to > add a > priority queue queueing mode. In that, we would enqueue messages based > on > priority and process higher priority messages before lower priority > ones. > That would work with all transports (and all other outout plugins) and > solved > the need you have. > > Am I right here? > > Rainer > > > -----Original Message----- > > From: [email protected] [mailto:rsyslog- > > [email protected]] On Behalf Of George Bonser > > Sent: Sunday, April 25, 2010 4:30 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] Syslog over SCTP? > > > > What I had in mind was something like an SCTP stream for each log > level > > or each facility, haven't decided which would be most useful. That > > prevents head-of-line blocking as happens in TCP. So, for example, > if > > a > > buffer is full of DEBUG level messages, an ALERT level message would > go > > immediately as it is in a different stream. A separate stream could > be > > added for RELP messages (or something like RELP or maybe an extension > > of > > it) and all of these streams would be within a single SCTP > connection. > > Or maybe you can send RELP-like messages back across each stream in > the > > opposite direction but having an "out of band" control channel sounds > > like an interesting idea and something fairly easy to do with SCTP. > > > > I would also think this lends itself well to threading of the > different > > streams with a different thread handling different log levels (or > > facilities but I think dividing among log levels is probably easier). > > On an SMP system, a remote host sending a lot of DEGUG level messages > > while it is being tested would bog down one thread but other messages > > would be processed separately without network head-of-line blocking > or > > CPU blocking (disk contention would be a different story, though ;) > > > > > > > > > -----Original Message----- > > > From: [email protected] [mailto:rsyslog- > > > [email protected]] On Behalf Of George Bonser > > > Sent: Saturday, April 24, 2010 11:20 AM > > > To: rsyslog-users > > > Subject: [rsyslog] Syslog over SCTP? > > > > > > Has anyone experimented with using SCTP for logging to a remote > host? > > > It seems it might have some advantages over TCP and UDP for that > > > purpose. > > > > > > http://www.ibm.com/developerworks/linux/library/l-sctp/ > > > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
