As far as I read this: https://issues.apache.org/jira/browse/HDFS-265
append seems to be there... Rainer > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Alexis Lelion > Sent: Monday, May 14, 2012 5:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] Append issues with omhdfs module > > Well, the version I'm using is from February 2012 so released after > your HDFS support announcement :-) > Hence why I was thinking about a secret compile command or so, rather > than a version issue. > But I'll investigate on Hadoop side, maybe this method was causing > issues and has been removed recently > Thanks > > Alexis > > > On Mon, May 14, 2012 at 2:02 PM, Rainer Gerhards > <[email protected]> wrote: > > I remember that you need a pretty fresh version of hadoop. The append > method > > is vital for syslog logging, so I suggest you look at which version supports > > it. > > > > HTH > > Rainer > > > >> -----Original Message----- > >> From: [email protected] [mailto:rsyslog- > >> [email protected]] On Behalf Of Alexis Lelion > >> Sent: Monday, May 14, 2012 1:31 PM > >> To: [email protected] > >> Subject: [rsyslog] Append issues with omhdfs module > >> > >> Dear, > >> > >> I'm currently building a hadoop cluster, and I was more than happy to > >> see native support of HDFS in syslog, unfortunately I can't get it > >> running. > >> My hadoop cluster works on top of debian squeeze + backports and I > >> rebuilt the rsyslog package from the debian backports (rsyslog-5.8.9) > >> with omhdfs support. > >> Installation went fine, and everything launched without error. But > >> when a piece of log has to be written in the hadoop cluster, I get the > >> following error : > >> Exception in thread "Thread-3" java.io.IOException: Not supported > >> at > >> > org.apache.hadoop.fs.ChecksumFileSystem.append(ChecksumFileSystem.java: > >> 290) > >> at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:650) > >> Call to > >> org.apache.hadoop.conf.FileSystem::append((Lorg/apache/hadoop/fs/Path;) > >> Lorg/apache/hadoop/fs/FSDataOutputStream;) > >> failed! > >> > >> So, not a syslog issue, but rather an hadoop issue. But when I looked > >> at hadoop code (stable version, 1.0.1), I got a bit surprised : > >> 287 /** {@inheritDoc} */ > >> 288 public FSDataOutputStream append(Path f, int bufferSize, > >> 289 Progressable progress) throws IOException { > >> 290 throw new IOException("Not supported"); > >> 291 } > >> > >> This method has not been implemented yet... So my question is, how did > >> you get syslog working with hadoop? Is there any Hadoop setting I have > >> to change to use another set of methods than this 'append'? > >> Any help with this would be greatly appreciated > >> Thanks! > >> > >> Alexis > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards
