Howdy. It's a good question, and maybe we should have a Chukwa-vs-the-competition page on the website.
Chukwa is designed to help collect system logs, and store them in HDFS. Scribe is designed as more of a syslog++. We went to some trouble to make sure our metadata and output formats were MapReduce friendly, whereas I think they leave that much more up to you. My understanding is that Scribe exposes a Thrift service, only. So if you want to use Scribe, you pretty much have to go all-in and modify your app and deployment to send logs via Thrift. Chukwa can just read whatever files or data sources are lying around. We can pretty easily listen for incoming Thrift RPC calls -- I believe Jerome is currently coding that up. Chukwa comes with visualization and processing tools, whereas Scribe doesn't. Scribe might be slightly lower overhead -- I believe it's C++ and Chukwa is Java. Chukwa isn't too big, either, so the absolute difference might not be significant. --Ari On Wed, Dec 23, 2009 at 3:06 PM, Corbin Hoenes <cor...@tynt.com> wrote: > Does anyone know of a comparison between Chukwa and Scribe( > http://sameerparwani.com/posts/facebook-scribe-server-documentation-and-tutorials > ) > I am interested in why someone might choose Chukwa vs. Scribe and their > pros/cons. > > > -- Ari Rabkin asrab...@gmail.com UC Berkeley Computer Science Department
