Hi,
I am trying process the flume event body using the Morphline interceptor,
grok pattern, and write it to a file using file_roll sink (started with a
simple setup for testing purpose, later will be using HDFS sink).
The source and the sink start successfully but the output file remains
empty. Below are the details :
Source is tailing a sample file containing the word "sample". I append more
words to the file, still dont see any output in the sink destination file.
flume.conf file:
---------------------------------------------------------------------------
agent.sources = seqGenSrc
agent.channels = memoryChannel
agent.sinks = loggerSink
agent.sources.seqGenSrc.type = exec
agent.sources.seqGenSrc.command = tail -F /home/sandesh/sample.txt
agent.sources.seqGenSrc.interceptors = morphlineinterceptor
agent.sources.seqGenSrc.interceptors.morphlineinterceptor.type =
org.apache.flume.sink.solr.morphline.MorphlineInterceptor$Builder
agent.sources.seqGenSrc.interceptors.morphlineinterceptor.morphlineFile =
/usr/local/lib/flume-ng/conf/morphline.conf
agent.sources.seqGenSrc.interceptors.morphlineinterceptor.morphlineId =
morphline1
agent.sources.seqGenSrc.channels = memoryChannel
agent.sinks.loggerSink.type = file_roll
agent.sinks.loggerSink.sink.directory = /home/sandesh/
agent.sinks.loggerSink.channel = memoryChannel
agent.channels.memoryChannel.type = memory
agent.channels.memoryChannel.capacity = 1000
--------------------------------------------------------------------------------
morphline.conf file:
--------------------------------------------------------------------------------
morphlines : [
{
id : morphline1
importCommands : ["com.cloudera.**"]
commands : [
#Reading the line from Flume event:
{
readLine {
charset : UTF-8
}
}
{
#Regex - pattern matching and transformation will be taken care
by below config
grok {
dictionaryFiles :
[/home/sandesh/cdk/cdk-morphlines/cdk-morphlines-core/src/test/resources/grok-dictionaries]
expressions : {
message : """%{WORD:word}"""
}
}
}
]
}
]
------------------------------------------------------------------------------
Am I missing something obvious? Please help me if yes..
Regards,
Sandesh
