Dear,
I'm currently building a hadoop cluster, and I was more than happy to
see native support of HDFS in syslog, unfortunately I can't get it
running.
My hadoop cluster works on top of debian squeeze + backports and I
rebuilt the rsyslog package from the debian backports (rsyslog-5.8.9)
with omhdfs support.
Installation went fine, and everything launched without error. But
when a piece of log has to be written in the hadoop cluster, I get the
following error :
Exception in thread "Thread-3" java.io.IOException: Not supported
at
org.apache.hadoop.fs.ChecksumFileSystem.append(ChecksumFileSystem.java:290)
at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:650)
Call to
org.apache.hadoop.conf.FileSystem::append((Lorg/apache/hadoop/fs/Path;)Lorg/apache/hadoop/fs/FSDataOutputStream;)
failed!
So, not a syslog issue, but rather an hadoop issue. But when I looked
at hadoop code (stable version, 1.0.1), I got a bit surprised :
287 /** {@inheritDoc} */
288 public FSDataOutputStream append(Path f, int bufferSize,
289 Progressable progress) throws IOException {
290 throw new IOException("Not supported");
291 }
This method has not been implemented yet... So my question is, how did
you get syslog working with hadoop? Is there any Hadoop setting I have
to change to use another set of methods than this 'append'?
Any help with this would be greatly appreciated
Thanks!
Alexis
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
