[
https://issues.apache.org/jira/browse/FLUME-1357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410317#comment-13410317
]
Sebastien N. commented on FLUME-1357:
-------------------------------------
MARK come back in v1.3.0
agent.sources.src.type = syslogtcp
agent.sources.src.port = 5140
agent.sources.src.channels =c1
agent.channels.c1.type = memory
agent.channels.c1.capacity = 100000
agent.channels.c1.transactionCapacity = 1000
agent.sinks.s.type = hdfs
agent.sinks.s.hdfs.path = hdfs://mycluster:8020/user/flume/%{host}/%Y-%m-%d/
agent.sinks.s.hdfs.filePrefix = SyslogTCP
agent.sinks.s.hdfs.fileType = DataStream
agent.sinks.s.hdfs.rollInterval = 3600
agent.sinks.s.hdfs.rollSize = 0
agent.sinks.s.hdfs.rollCount = 0
agent.sinks.s.hdfs.batchSize = 1
agent.sinks.s.hdfs.txnEventMax = 100
agent.sinks.s.serialize= org.apache.flume.serialization.BodyTextEventSerializer
agent.sinks.s.serializer.compressionCodec = snappy
agent.sinks.s.channel = c1
agent.sources.src.interceptors = ts host
agent.sources.src.interceptors.ts.type =
org.apache.flume.interceptor.TimestampInterceptor$Builder
agent.sources.src.interceptors.host.type =
org.apache.flume.interceptor.HostInterceptor$Builder
agent.sources.src.interceptors.host.useIP = false
agent.sources.src.interceptors.host.preserveExisting = true
> Strange host directory when using %{host}
> -----------------------------------------
>
> Key: FLUME-1357
> URL: https://issues.apache.org/jira/browse/FLUME-1357
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.1.0, v1.3.0
> Environment: - CentOS6.2
> - HDFS 2.0.0-cdh4.0.0
> Reporter: Sebastien N.
> Priority: Minor
> Attachments: MARK-1.png, MARK-2.png
>
>
> Hi
> When i use:
> agent.sources.src.type = syslogtcp
> agent.sources.src.port = 5140
> agent.sources.src.host = 0.0.0.0
> agent.sinks.s.type = hdfs
> agent.sinks.s.hdfs.path = hdfs://mycluster:8020/user/flume/%{host}/
> agent.sinks.s.serialize=
> org.apache.flume.serialization.BodyTextEventSerializer
> i have my list of rsyslog host and a special host named "MARK".
> This directory contains files and the content of files is "--".
> I think this bug generate the following error when using
> agent.sinks.s.hdfs.path = hdfs://mycluster:8020/user/flume/%{host}/%Y-%m-%d.
> ERROR hdfs.HDFSEventSink: process failed
> java.lang.NumberFormatException: null
> at java.lang.Long.parseLong(Long.java:404)
> at java.lang.Long.valueOf(Long.java:540)
> at
> org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:220)
> at
> org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:310)
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:402)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> The headder of this event doesn't seem correct...
> More informations:
> [root@mymachine:~]#hdfs dfs -copyToLocal
> /user/flume/MARK/SyslogTCP.1341818888517
> [root@mymachine:~]#cat SyslogTCP.1341818888517
> --
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
