[
https://issues.apache.org/jira/browse/FLUME-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249789#comment-13249789
]
[email protected] commented on FLUME-1110:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4681/#review6789
-----------------------------------------------------------
I think the change makes sense, but I am not sure if it solves the problem from
the JIRA? From what I can tell about the error, it looks like HDFS is trying to
add a shutdown hook after the shutdown has started.
- Brock
On 2012-04-09 06:54:55, Prasad Mujumdar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4681/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-09 06:54:55)
bq.
bq.
bq. Review request for Flume and Arvind Prabhakar.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The sink runner's stop method first calls stop() to underlying sink and
then shuts down the PollingRunner thread. If that thread is in middle of
process, it leads to race conditions in the sink's process() and stop().
bq. Rather than making all sinks to handle concurrently process() and stop(),
its safer to shutdown the runner thread first and then stop the sink.
bq.
bq.
bq. This addresses bug FLUME-1110.
bq. https://issues.apache.org/jira/browse/FLUME-1110
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-core/src/main/java/org/apache/flume/SinkRunner.java e73c09b
bq.
bq. Diff: https://reviews.apache.org/r/4681/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. full regression test run
bq.
bq.
bq. Thanks,
bq.
bq. Prasad
bq.
bq.
> HDFS Sink throws IllegalStateException when flume-daemon shuts down
> -------------------------------------------------------------------
>
> Key: FLUME-1110
> URL: https://issues.apache.org/jira/browse/FLUME-1110
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.1.0
> Reporter: Prasad Mujumdar
> Assignee: Prasad Mujumdar
> Fix For: v1.2.0
>
>
> When using HDFS sink, if you shutdown the daemon (sudo
> /etc/init.d/flume-ng-node stop), then an IllegalStateException is shown in
> the logs (/var/log/flume-ng/flume.log).
> 2012-04-06 10:44:19,912 ERROR hdfs.HDFSEventSink: Error calling
> org.apache.flume.sink.hdfs.HDFSEventSink$4@32091738
> java.lang.IllegalStateException: Shutdown in progress
> at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:39)
> at java.lang.Runtime.addShutdownHook(Runtime.java:192)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1607)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1579)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:228)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:183)
> at org.apache.flume.sink.hdfs.BucketWriter.renameBucket(BucketWriter.java:196)
> at org.apache.flume.sink.hdfs.BucketWriter.close(BucketWriter.java:122)
> at org.apache.flume.sink.hdfs.HDFSEventSink$4.call(HDFSEventSink.java:440)
> at org.apache.flume.sink.hdfs.HDFSEventSink$4.call(HDFSEventSink.java:436)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2012-04-06 10:44:19,927 INFO source.SyslogTcpSource: Syslog TCP Source
> stopping...
> 2012-04-06 10:44:19,927 INFO source.SyslogTcpSource: Metrics:{ name:null
> counters:{events.success=11002} }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
