[
https://issues.apache.org/jira/browse/FLUME-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249708#comment-13249708
]
[email protected] commented on FLUME-1110:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4681/
-----------------------------------------------------------
Review request for Flume and Arvind Prabhakar.
Summary
-------
The sink runner's stop method first calls stop() to underlying sink and then
shuts down the PollingRunner thread. If that thread is in middle of process, it
leads to race conditions in the sink's process() and stop().
Rather than making all sinks to handle concurrently process() and stop(), its
safer to shutdown the runner thread first and then stop the sink.
This addresses bug FLUME-1110.
https://issues.apache.org/jira/browse/FLUME-1110
Diffs
-----
flume-ng-core/src/main/java/org/apache/flume/SinkRunner.java e73c09b
Diff: https://reviews.apache.org/r/4681/diff
Testing
-------
full regression test run
Thanks,
Prasad
> HDFS Sink throws IllegalStateException when flume-daemon shuts down
> -------------------------------------------------------------------
>
> Key: FLUME-1110
> URL: https://issues.apache.org/jira/browse/FLUME-1110
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.1.0
> Reporter: Prasad Mujumdar
> Assignee: Prasad Mujumdar
> Fix For: v1.2.0
>
>
> When using HDFS sink, if you shutdown the daemon (sudo
> /etc/init.d/flume-ng-node stop), then an IllegalStateException is shown in
> the logs (/var/log/flume-ng/flume.log).
> 2012-04-06 10:44:19,912 ERROR hdfs.HDFSEventSink: Error calling
> org.apache.flume.sink.hdfs.HDFSEventSink$4@32091738
> java.lang.IllegalStateException: Shutdown in progress
> at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:39)
> at java.lang.Runtime.addShutdownHook(Runtime.java:192)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1607)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1579)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:228)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:183)
> at org.apache.flume.sink.hdfs.BucketWriter.renameBucket(BucketWriter.java:196)
> at org.apache.flume.sink.hdfs.BucketWriter.close(BucketWriter.java:122)
> at org.apache.flume.sink.hdfs.HDFSEventSink$4.call(HDFSEventSink.java:440)
> at org.apache.flume.sink.hdfs.HDFSEventSink$4.call(HDFSEventSink.java:436)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2012-04-06 10:44:19,927 INFO source.SyslogTcpSource: Syslog TCP Source
> stopping...
> 2012-04-06 10:44:19,927 INFO source.SyslogTcpSource: Metrics:{ name:null
> counters:{events.success=11002} }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
