[
https://issues.apache.org/jira/browse/FLUME-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Liu updated FLUME-3254:
----------------------------
Description:
Hello,
I am using flume 1.8.0 for ubuntu 16.04. I met an question when using the
agent, taildir-memory-fileroll. I programed for appending data into the file
monitored by taildir *every 2 second*．Then at this speed the agent worked
abnormally. Although the json file generated by taildir always updated and
recorded the newest position, the file generated by rollfile didn't have any
more data until I killed the bin which is appending data into the file
monitored by taildir. When I changed the file appending interval to be *3
second*. Everything worked normally. the following is my agent configuration
file.
_#a1 taildir-----> fileroll_
_a1.sources = src1_
_a1.sinks = sink1_
_a1.channels = chan1_
_a1.sources.src1.type = TAILDIR_
_a1.sources.src1.positionFile =
/home/EFM/Downloads/apache-flume-1.8.0-bin/TailDir/position.log_
_a1.sources.src1.filegroups =f1_
_a1.sources.src1.filegroups.f1 =
/home/EFM/Downloads/apache-flume-1.8.0-bin/TailDir/log2.txt_
_a1.sources.src1.writePosInterval=500_
_a1.channels.chan1.type=memory_
_a1.channels.chan1.capacity=100000_
_a1.channels.chan1.transactionCapacity=10000_
_a1.sinks.sink1.channel = chan1_
_a1.sinks.sink1.type = file_roll_
_a1.sinks.sink1.sink.directory =
/home/EFM/Downloads/apache-flume-1.8.0-bin/FileRoll_
_a1.sinks.sink1.sink.rollInterval=0_
_a1.sources.src1.channels = chan1_
_a1.sinks.sink1.channel = chan1_
Would you like give me some suggestions for that questions?
thanks.
was:
Hello,
I am using flume 1.8.0 for ubuntu 16.04. I met an question when using the
agent, taildir-memory-fileroll. I programed for appending data into the file
monitored by taildir *every 2 second*．Then at this speed the agent worked
abnormally. Although the json file generated by taildir always updated and
recorded the newest position, the file generated by rollfile didn't have any
more data until I killed the bin which is appending data into the file
monitored by taildir. When I changed the file appending speed to be *3 second
each time*. Everything worked normally. the following is my agent configuration
file.
_#a1 taildir-----> fileroll_
_a1.sources = src1_
_a1.sinks = sink1_
_a1.channels = chan1_
_a1.sources.src1.type = TAILDIR_
_a1.sources.src1.positionFile =
/home/EFM/Downloads/apache-flume-1.8.0-bin/TailDir/position.log_
_a1.sources.src1.filegroups =f1_
_a1.sources.src1.filegroups.f1 =
/home/EFM/Downloads/apache-flume-1.8.0-bin/TailDir/log2.txt_
_a1.sources.src1.writePosInterval=500_
_a1.channels.chan1.type=memory_
_a1.channels.chan1.capacity=100000_
_a1.channels.chan1.transactionCapacity=10000_
_a1.sinks.sink1.channel = chan1_
_a1.sinks.sink1.type = file_roll_
_a1.sinks.sink1.sink.directory =
/home/EFM/Downloads/apache-flume-1.8.0-bin/FileRoll_
_a1.sinks.sink1.sink.rollInterval=0_
_a1.sources.src1.channels = chan1_
_a1.sinks.sink1.channel = chan1_
Would you like give me some suggestions for that questions?
thanks.
> taildir source works abnormally when the monitored file has too fast append
> speed．
> ----------------------------------------------------------------------------------
>
> Key: FLUME-3254
> URL: https://issues.apache.org/jira/browse/FLUME-3254
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: 1.8.0
> Environment: flume 1.8.0 virtual-machine-ubuntu16.04 desktop
> Reporter: Alan Liu
> Priority: Major
> Fix For: notrack
>
>
> Hello,
> I am using flume 1.8.0 for ubuntu 16.04. I met an question when using the
> agent, taildir-memory-fileroll. I programed for appending data into the file
> monitored by taildir *every 2 second*．Then at this speed the agent worked
> abnormally. Although the json file generated by taildir always updated and
> recorded the newest position, the file generated by rollfile didn't have any
> more data until I killed the bin which is appending data into the file
> monitored by taildir. When I changed the file appending interval to be *3
> second*. Everything worked normally. the following is my agent configuration
> file.
> _#a1 taildir-----> fileroll_
> _a1.sources = src1_
> _a1.sinks = sink1_
> _a1.channels = chan1_
> _a1.sources.src1.type = TAILDIR_
> _a1.sources.src1.positionFile =
> /home/EFM/Downloads/apache-flume-1.8.0-bin/TailDir/position.log_
> _a1.sources.src1.filegroups =f1_
> _a1.sources.src1.filegroups.f1 =
> /home/EFM/Downloads/apache-flume-1.8.0-bin/TailDir/log2.txt_
> _a1.sources.src1.writePosInterval=500_
> _a1.channels.chan1.type=memory_
> _a1.channels.chan1.capacity=100000_
> _a1.channels.chan1.transactionCapacity=10000_
> _a1.sinks.sink1.channel = chan1_
> _a1.sinks.sink1.type = file_roll_
> _a1.sinks.sink1.sink.directory =
> /home/EFM/Downloads/apache-flume-1.8.0-bin/FileRoll_
> _a1.sinks.sink1.sink.rollInterval=0_
> _a1.sources.src1.channels = chan1_
> _a1.sinks.sink1.channel = chan1_
> Would you like give me some suggestions for that questions?
> thanks.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
