Well, the version I'm using is from February 2012 so released after your HDFS support announcement :-) Hence why I was thinking about a secret compile command or so, rather than a version issue. But I'll investigate on Hadoop side, maybe this method was causing issues and has been removed recently Thanks
Alexis On Mon, May 14, 2012 at 2:02 PM, Rainer Gerhards <[email protected]> wrote: > I remember that you need a pretty fresh version of hadoop. The append method > is vital for syslog logging, so I suggest you look at which version supports > it. > > HTH > Rainer > >> -----Original Message----- >> From: [email protected] [mailto:rsyslog- >> [email protected]] On Behalf Of Alexis Lelion >> Sent: Monday, May 14, 2012 1:31 PM >> To: [email protected] >> Subject: [rsyslog] Append issues with omhdfs module >> >> Dear, >> >> I'm currently building a hadoop cluster, and I was more than happy to >> see native support of HDFS in syslog, unfortunately I can't get it >> running. >> My hadoop cluster works on top of debian squeeze + backports and I >> rebuilt the rsyslog package from the debian backports (rsyslog-5.8.9) >> with omhdfs support. >> Installation went fine, and everything launched without error. But >> when a piece of log has to be written in the hadoop cluster, I get the >> following error : >> Exception in thread "Thread-3" java.io.IOException: Not supported >> at >> org.apache.hadoop.fs.ChecksumFileSystem.append(ChecksumFileSystem.java: >> 290) >> at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:650) >> Call to >> org.apache.hadoop.conf.FileSystem::append((Lorg/apache/hadoop/fs/Path;) >> Lorg/apache/hadoop/fs/FSDataOutputStream;) >> failed! >> >> So, not a syslog issue, but rather an hadoop issue. But when I looked >> at hadoop code (stable version, 1.0.1), I got a bit surprised : >> 287 /** {@inheritDoc} */ >> 288 public FSDataOutputStream append(Path f, int bufferSize, >> 289 Progressable progress) throws IOException { >> 290 throw new IOException("Not supported"); >> 291 } >> >> This method has not been implemented yet... So my question is, how did >> you get syslog working with hadoop? Is there any Hadoop setting I have >> to change to use another set of methods than this 'append'? >> Any help with this would be greatly appreciated >> Thanks! >> >> Alexis >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards
