Arpit Gupta created FLUME-2808:
----------------------------------
Summary: TailDirSource fails to process the sources if one of the
directories configured for file groups does not exist
Key: FLUME-2808
URL: https://issues.apache.org/jira/browse/FLUME-2808
Project: Flume
Issue Type: Bug
Affects Versions: v1.7.0
Reporter: Arpit Gupta
Configured TailDirSource with multiple file groups. Some of the file group
directories do not exist on all nodes on the cluster.
One the nodes where the file group directory did not exist TailDirSource failed
to run with exception
{code}
05 Oct 2015 20:39:56,433 ERROR [lifecycleSupervisor-1-0]
(org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:253) -
Unable to start PollableSourceRunner: { source:Taildir source: { positionFile:
/dir/flume/taildir_position.json, skipToEnd: false, byteOffsetHeader: false,
idleTimeout: 120000, writePosInterval: 3000 } counterGroup:{ name:null
counters:{} } } - Exception follows.
java.lang.IllegalStateException: Directory does not exist: /dir/log/hive
at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at
org.apache.flume.source.taildir.ReliableTaildirEventReader.<init>(ReliableTaildirEventReader.java:86)
at
org.apache.flume.source.taildir.ReliableTaildirEventReader.<init>(ReliableTaildirEventReader.java:55)
at
org.apache.flume.source.taildir.ReliableTaildirEventReader$Builder.build(ReliableTaildirEventReader.java:343)
at
org.apache.flume.source.taildir.TaildirSource.start(TaildirSource.java:91)
at
org.apache.flume.source.PollableSourceRunner.start(PollableSourceRunner.java:74)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
I built flume from sha: bd80c5e67deda4ec146f19fc92ff0c3bff61a982
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
