Hi Corbin, Chukwa is intended for your use case, but it's not perfect in term of functionality. There are two possible solutions to refine Chukwa to do what you need.
First solution, extend UTF8 File Tailing Adaptor to delete the processed file, and use that to tail the apache log file. This method should be fairly easy to implement. Second solution, write an adaptor to receive log from named pipe, and configure Apache to pipe the log to a perl script which writes to the named pipe. This method is great for reducing disk load for the webserver. It is more tricky to get it right. Hope this helps. Regards, Eric On 1/27/10 10:18 AM, "Corbin Hoenes" <cor...@tynt.com> wrote: > I've noticed some chatter about log rotation and FileTailingAdaptor: > http://issues.apache.org/jira/browse/CHUKWA-421 > > Is there some kind of documentation on how it deals with say a directory of > files that get rotated. At first glance it looks like it keeps an adaptor > open forever and there doesn't appear to be a way to know that a file has been > processed by the adaptor so we can't really move files out, etc... Right now > we are piping apache access logs through rotatelogs and then using chukwa to > tail these files. Is there a better way? > > >
