My HDFS knowledge is unfortunately rather limited. I just know what was required to write the module. Most importantly, I do not know the exact setup and config files for HDFS. But... [inline]
> -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Piavlo > Sent: Thursday, April 28, 2011 1:27 AM > To: rsyslog-users > Subject: [rsyslog] problems with using omhdfs module > > Hi, > > I'm tryin to send logs from rsyslog server to remote hdfs cluster > The rsyslog server is compiled with omhdfs support with following CDH3 > rpms installed > > root@dev1 ~]# rpm -qa | grep hadoop > hadoop-0.20-libhdfs-0.20.2+923.21-1 > hadoop-0.20-source-0.20.2+923.21-1 > hadoop-0.20-native-0.20.2+923.21-1 > hadoop-0.20-0.20.2+923.21-1 > [root@dev1 ~]# > > trying the following > ---------------- > $ModLoad omhdfs > > $OMHDFSHost hadoop-master1.internal > $OMHDFSPort 8020 > $OMHDFSFileName /var/log/logfile > local3.info :omhdfs: > ---------------- > > i get the following error > > 2011-04-27T23:19:06.335954+00:00 dev1 rsyslogd-2179: omhdfs: failed to > open /var/log/logfile - retrying later [try > http://www.rsyslog.com/e/2179 ] > > I fail to understand the meaning of $OMHDFSFileName directive > does this refers to file which will/should be created withing the hdfs > filesystem? ... this I can say for sure: yes, it is. There needs to be a way to specify the actual file name to write to. I *guess* that the open failure may be related to a permission problem. > > In core-site.xml I have > <property> > <name>fs.default.name</name> > <value>hdfs://hadoop-master1a.internal:8020</value> > </property> > so I've set the following > $OMHDFSHost hadoop-master1.internal > $OMHDFSPort 8020 Not konwing exactly what the Hadoop config does, this at least looks plausible. Does that help? Rainer > > What am I doing wrong? > > Thanks > Alex > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
