It looks like Hbase does not have write permission to /hbase on HDFS. This may be due to / is not writable by hbase, and /hbase can not be created. You might want to manually create /hbase and change the ownership to hbase user.
Regards, Eric On 6/1/11 10:32 AM, "Preetam Patil" <[email protected]> wrote: Don't know if it matters--I'm using CDH3 on Ubuntu 10.04. My collector.log attached now. Thanks, -preetam On Wed, Jun 1, 2011 at 10:33 PM, Preetam Patil <[email protected]> wrote: > Hi Eric and DKN, > > I am facing a similar (maybe a noob) problem: chukwa collector is able > to write to a sequence file, but when I enabl eHBaseWriter, the > HBaseTable entries are not added. I am not sure whether it's a problem > with my HBase setup (it's in standalone mode) or Chukwa Demux parsers. > I get two errors: "ERROR main HBaseWriter - Hbase schema mismatch with > demux parser" and "INFO main HConnectionManager$TableServers - > getMaster at tempt 6 of 10 failed; retrying after sleep of 4000". > Can't figure out where the problem lies. > > Attaching the collector.log to this mail and below is my > chukwa-collector-conf.xml: > > Thanks, > -preetam > > <?xml version="1.0"?> > <!-- > Licensed to the Apache Software Foundation (ASF) under one or more > contributor license agreements. See the NOTICE file distributed with > this work for additional information regarding copyright ownership. > The ASF licenses this file to You under the Apache License, Version 2.0 > (the "License"); you may not use this file except in compliance with > the License. You may obtain a copy of the License at > > http://www.apache.org/licenses/LICENSE-2.0 > > Unless required by applicable law or agreed to in writing, software > distributed under the License is distributed on an "AS IS" BASIS, > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > See the License for the specific language governing permissions and > limitations under the License. > --> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > <!-- Put site-specific property overrides in this file. --> > > <configuration> > > <property> > <name>chukwaCollector.writerClass</name> > > <value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value> > </property> > > <!-- peetam: replaced by HBaseWriter pipeline below > <property> > <name>chukwaCollector.pipeline</name> > > <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value> > </property> > --> > > <!-- LocalWriter parameters > <property> > <name>chukwaCollector.localOutputDir</name> > <value>/tmp/chukwa/dataSink/</value> > <description>Chukwa local data sink directory, see > LocalWriter.java</description> > </property> > > <property> > <name>chukwaCollector.writerClass</name> > > <value>org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalWriter</value> > <description>Local chukwa writer, see LocalWriter.java</description> > </property> > --> > > <!-- When writing to HBase, uncomment the following parameters. If > you're running > HBase in distributed mode, you'll also need to copy your > hbase-site.xml file with > your hbase.zookeeper.quorum setting to the conf/ dir. --> > > <property> > <name>chukwaCollector.pipeline</name> > > <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.hbase.HBaseWriter</value> > </property> > > <property> > <name>hbase.demux.package</name> > <value>org.apache.hadoop.chukwa.extraction.demux.processor</value> > <description>Demux parser class package, HBaseWriter uses this > package name to validate HBase for annotated demux parser > classes.</description> > </property> > > <property> > <name>hbase.writer.verify.schema</name> > <value>true</value> > <description>Verify HBase Table schema with demux parser schema, log > warning if there are mismatch between hbase schema and demux parsers. > </description> > </property> > > <property> > <name>hbase.writer.halt.on.schema.mismatch</name> > <value>false</value> > <description>If this option is set to true, and HBase table schema > is mismatched with demux parser, collector will shut down itself. > </description> > </property> > > <property> > <name>writer.hdfs.filesystem</name> > <value>hdfs://localhost:8020/</value> > <description>HDFS to dump to</description> > </property> > > <property> > <name>chukwaCollector.outputDir</name> > <value>/chukwa/logs/</value> > <description>Chukwa data sink directory</description> > </property> > > <property> > <name>chukwaCollector.rotateInterval</name> > <value>300000</value> > <description>Chukwa rotate interval (ms)</description> > </property> > > <property> > <name>chukwaCollector.http.port</name> > <value>8080</value> > <description>The HTTP port number the collector will listen > on</description> > </property> > </configuration> > > > On Wed, Jun 1, 2011 at 10:11 PM, Eric Yang <[email protected]> wrote: >> From the current configuration, it is setup to write to Hbase only, it does >> not write to HDFS. If you want the data to be written to HDFS as well, use >> this line in chukwa-collector-conf.xml. >> >> <property> >> <name>chukwaCollector.pipeline</name> >> <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter, >> org.apache.hadoop.chukwa.datacollection.writer.hbase.HBaseWriter, >> org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value> >> </property> >> >> Make sure that hadoop-site.xml and hbase-site.xml are in the chukwa >> collector class path. This can be done by copying hadoop-site.xml and >> hbase-site.xml to CHUKWA_CONF_DIR. >> >> For HICC listening to localhost issue, this is a bug in Chukwa. Feel free >> to file a jira. Before compile, modify web/hicc/WEB-INF/jetty.xml and set >> jetty.host ip address to 0.0.0.0. Without this setup, jetty would choose to >> bind to first available IP address, it could randomly bind to localhost or >> eth0 depending on the OS. >> >> Regards, >> Eric >> >
