[
https://issues.apache.org/jira/browse/STORM-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stig Rohde Døssing updated STORM-2184:
--------------------------------------
Description:
As far as I know spout methods are not called concurrently by Storm. When the
new Kafka spout closes or deactivates, the closing thread calls
KafkaConsumer.wakeup. If there are uncommitted offsets and the spout is in
manual commit mode, it then tries to commit offsets. Since there are no other
threads concurrently interacting with the spout, the call to
KafkaConsumer.commitSync will end up throwing WakeupException. This is because
the behavior of wakeup is to cause the next blocking call to throw
WakeupException if there are no other threads blocking on the consumer when
wakeup is called.
The call to wakeup should be removed, since it does nothing if there are no
offsets to commit, and causes an uncaught exception if there are.
was:
If a thread is blocking in one of a number of calls to KafkaConsumer and wakeup
is called, the thread will throw WakeupException. If there is no such thread,
the next call to one of the blocking methods will throw WakeupException instead.
The spout currently calls wakeup, then attempts to commit offsets to Kafka when
shutting down. If there was no other thread to wake up (e.g. the async loop
thread was doing something other than hanging in KakfaConsumer.poll), the call
to KafkaConsumer.commitSync from the closing thread will throw WakeupException.
The thread running KafkaSpout.close shouldn't be interrupting itself, and
should ignore WakeupExceptions when committing offsets.
> New Kafka spout will fail to commit offsets on shutdown if there are offsets
> to commit
> --------------------------------------------------------------------------------------
>
> Key: STORM-2184
> URL: https://issues.apache.org/jira/browse/STORM-2184
> Project: Apache Storm
> Issue Type: Bug
> Reporter: Stig Rohde Døssing
> Assignee: Stig Rohde Døssing
> Priority: Trivial
>
> As far as I know spout methods are not called concurrently by Storm. When the
> new Kafka spout closes or deactivates, the closing thread calls
> KafkaConsumer.wakeup. If there are uncommitted offsets and the spout is in
> manual commit mode, it then tries to commit offsets. Since there are no other
> threads concurrently interacting with the spout, the call to
> KafkaConsumer.commitSync will end up throwing WakeupException. This is
> because the behavior of wakeup is to cause the next blocking call to throw
> WakeupException if there are no other threads blocking on the consumer when
> wakeup is called.
> The call to wakeup should be removed, since it does nothing if there are no
> offsets to commit, and causes an uncaught exception if there are.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
