How does LinkedIn handle syslog? Is Kafka at LinkedIN dedicated to application logs & metrics and syslog messages (e.g. network, OS logs) is a separate flow and datastore?
On Mon, Apr 23, 2012 at 11:26 AM, Joel Koshy <jjkosh...@gmail.com> wrote: > This is an interesting use-case and similar to using the log4j appender for > application logs. What you describe sounds reasonable: i.e., have a > producer process on the syslog server to send syslog messages to your Kafka > brokers. > > >> 2) How about tailing a file to a central logging like you can do with >> scribe/flume agents? >> > > You could use tail and pipe (or a named pipe) to a console-based producer. > I don't know enough about syslog.d to tell if this would be too much of a > hack or not, but I think it would be more reliable than using tail+pipes: > you could configure your local machines to direct syslog messages to the > remote server and you could write a simple component that listens on the > syslog socket and redirects incoming messages to a Kafka producer. > > Joel
