I remember that you need a pretty fresh version of hadoop. The append method is vital for syslog logging, so I suggest you look at which version supports it.
HTH Rainer > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Alexis Lelion > Sent: Monday, May 14, 2012 1:31 PM > To: [email protected] > Subject: [rsyslog] Append issues with omhdfs module > > Dear, > > I'm currently building a hadoop cluster, and I was more than happy to > see native support of HDFS in syslog, unfortunately I can't get it > running. > My hadoop cluster works on top of debian squeeze + backports and I > rebuilt the rsyslog package from the debian backports (rsyslog-5.8.9) > with omhdfs support. > Installation went fine, and everything launched without error. But > when a piece of log has to be written in the hadoop cluster, I get the > following error : > Exception in thread "Thread-3" java.io.IOException: Not supported > at > org.apache.hadoop.fs.ChecksumFileSystem.append(ChecksumFileSystem.java: > 290) > at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:650) > Call to > org.apache.hadoop.conf.FileSystem::append((Lorg/apache/hadoop/fs/Path;) > Lorg/apache/hadoop/fs/FSDataOutputStream;) > failed! > > So, not a syslog issue, but rather an hadoop issue. But when I looked > at hadoop code (stable version, 1.0.1), I got a bit surprised : > 287 /** {@inheritDoc} */ > 288 public FSDataOutputStream append(Path f, int bufferSize, > 289 Progressable progress) throws IOException { > 290 throw new IOException("Not supported"); > 291 } > > This method has not been implemented yet... So my question is, how did > you get syslog working with hadoop? Is there any Hadoop setting I have > to change to use another set of methods than this 'append'? > Any help with this would be greatly appreciated > Thanks! > > Alexis > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards
