Hi Brock, could you please provide me the sample configuration?
Thanks, Khadar On Thu, Jul 26, 2012 at 7:23 PM, Brock Noland <[email protected]> wrote: > If you want to use a bucketed paths your events need to have a "timestamp" > header which tail source does not appear to add. > > Brock > > On Thu, Jul 26, 2012 at 6:32 AM, khadar basha <[email protected]> wrote: > >> >> Hi >> >> we are using flume-1.1.0-cdh4.0.1 . we have the following configurations >> in flume-conf.properties. >> >> agent1Test1.sources = seqGenSrc >> agent1Test1.channels = memoryChannel >> agent1Test1.sinks = loggerSink >> >> # For each one of the sources, the type is defined >> agent1Test1.sources.seqGenSrc.type = exec >> agent1Test1.sources.seqGenSrc.command= tail -f /tomcat/logs/catalina.out >> >> >> # The channel can be defined as follows. >> agent1Test1.sources.seqGenSrc.channels = memoryChannel >> >> # Each sink's type must be defined >> agent1Test1.sinks.loggerSink.type = hdfs >> agent1Test1.sinks.loggerSink.hdfs.path = >> hdfs://hdfshost:port/data/%Y/%m/%{host}/mylogs >> >> agent1Test1.sinks.loggerSink.hdfs.fileType = DataStream >> >> #Specify the channel the sink should use >> agent1Test1.sinks.loggerSink.channel = memoryChannel >> >> # Each channel's type is defined. >> agent1Test1.channels.memoryChannel.type = memory >> >> # Other config values specific to each type of channel(sink or source) >> # can be defined as well >> # In this case, it specifies the capacity of the memory channel >> agent1Test1.channels.memoryChannel.capacity = 1000 >> >> >> I am getting the following exception. Anybody facing this issue. >> >> >> *2012-07-26 13:07:52,002 ERROR hdfs.HDFSEventSink: process failed* >> *java.lang.NumberFormatException: null* >> * at java.lang.Long.parseLong(Long.java:375)* >> * at java.lang.Long.valueOf(Long.java:525)* >> * at >> org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:220) >> * >> * at >> org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:310) >> * >> * at >> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:402)* >> * at >> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) >> * >> * at >> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)* >> * at java.lang.Thread.run(Thread.java:662)* >> *2012-07-26 13:07:52,003 ERROR flume.SinkRunner: Unable to deliver >> event. Exception follows.* >> *org.apache.flume.EventDeliveryException: >> java.lang.NumberFormatException: null* >> * at >> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:469)* >> * at >> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) >> * >> * at >> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)* >> * at java.lang.Thread.run(Thread.java:662)* >> *Caused by: java.lang.NumberFormatException: null* >> * at java.lang.Long.parseLong(Long.java:375)* >> * at java.lang.Long.valueOf(Long.java:525)* >> * at >> org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:220) >> * >> * at >> org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:310) >> * >> * at >> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:402)* >> * ... 3 more* >> >> -- >> Thanks, >> Khadar >> >> > > > -- > Apache MRUnit - Unit testing MapReduce - > http://incubator.apache.org/mrunit/ > -- Thanks, Khadar
