Hi Brock. flume-1.1.0 doesn't support this escape sequence options?
-Khadar On Thu, Jul 26, 2012 at 10:34 PM, Brock Noland <[email protected]> wrote: > The flume user doc here http://flume.apache.org/releases/1.2.0.html has > many examples. > > Brock > > > On Thu, Jul 26, 2012 at 11:37 AM, khadar basha <[email protected]>wrote: > >> Hi Brock, >> >> could you please provide me the sample configuration? >> >> Thanks, >> Khadar >> >> >> On Thu, Jul 26, 2012 at 7:23 PM, Brock Noland <[email protected]> wrote: >> >>> If you want to use a bucketed paths your events need to have a >>> "timestamp" header which tail source does not appear to add. >>> >>> Brock >>> >>> On Thu, Jul 26, 2012 at 6:32 AM, khadar basha <[email protected]>wrote: >>> >>>> >>>> Hi >>>> >>>> we are using flume-1.1.0-cdh4.0.1 . we have the following >>>> configurations in flume-conf.properties. >>>> >>>> agent1Test1.sources = seqGenSrc >>>> agent1Test1.channels = memoryChannel >>>> agent1Test1.sinks = loggerSink >>>> >>>> # For each one of the sources, the type is defined >>>> agent1Test1.sources.seqGenSrc.type = exec >>>> agent1Test1.sources.seqGenSrc.command= tail -f /tomcat/logs/catalina.out >>>> >>>> >>>> # The channel can be defined as follows. >>>> agent1Test1.sources.seqGenSrc.channels = memoryChannel >>>> >>>> # Each sink's type must be defined >>>> agent1Test1.sinks.loggerSink.type = hdfs >>>> agent1Test1.sinks.loggerSink.hdfs.path = >>>> hdfs://hdfshost:port/data/%Y/%m/%{host}/mylogs >>>> >>>> agent1Test1.sinks.loggerSink.hdfs.fileType = DataStream >>>> >>>> #Specify the channel the sink should use >>>> agent1Test1.sinks.loggerSink.channel = memoryChannel >>>> >>>> # Each channel's type is defined. >>>> agent1Test1.channels.memoryChannel.type = memory >>>> >>>> # Other config values specific to each type of channel(sink or source) >>>> # can be defined as well >>>> # In this case, it specifies the capacity of the memory channel >>>> agent1Test1.channels.memoryChannel.capacity = 1000 >>>> >>>> >>>> I am getting the following exception. Anybody facing this issue. >>>> >>>> >>>> *2012-07-26 13:07:52,002 ERROR hdfs.HDFSEventSink: process failed* >>>> *java.lang.NumberFormatException: null* >>>> * at java.lang.Long.parseLong(Long.java:375)* >>>> * at java.lang.Long.valueOf(Long.java:525)* >>>> * at >>>> org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:220) >>>> * >>>> * at >>>> org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:310) >>>> * >>>> * at >>>> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:402) >>>> * >>>> * at >>>> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) >>>> * >>>> * at >>>> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)* >>>> * at java.lang.Thread.run(Thread.java:662)* >>>> *2012-07-26 13:07:52,003 ERROR flume.SinkRunner: Unable to deliver >>>> event. Exception follows.* >>>> *org.apache.flume.EventDeliveryException: >>>> java.lang.NumberFormatException: null* >>>> * at >>>> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:469) >>>> * >>>> * at >>>> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) >>>> * >>>> * at >>>> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)* >>>> * at java.lang.Thread.run(Thread.java:662)* >>>> *Caused by: java.lang.NumberFormatException: null* >>>> * at java.lang.Long.parseLong(Long.java:375)* >>>> * at java.lang.Long.valueOf(Long.java:525)* >>>> * at >>>> org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:220) >>>> * >>>> * at >>>> org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:310) >>>> * >>>> * at >>>> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:402) >>>> * >>>> * ... 3 more* >>>> >>>> -- >>>> Thanks, >>>> Khadar >>>> >>>> >>> >>> >>> -- >>> Apache MRUnit - Unit testing MapReduce - >>> http://incubator.apache.org/mrunit/ >>> >> >> >> >> -- >> Thanks, >> Khadar >> >> > > > -- > Apache MRUnit - Unit testing MapReduce - > http://incubator.apache.org/mrunit/ > -- Thanks, Khadar
