GitHub user slava92 opened a pull request:
https://github.com/apache/storm/pull/1542
STORM-1946: initialize lastHeartbeatTimestamp before starting heartbeat task
When storm stars a large number of ShellBolt-s that consume a lot of CPU
time to initialize, it creates a lot of contention between processes for CPU
resource. That leads to BoltHeartbeatTimerTask being fired up after 1 second
delay before setHeartbeat() assigns initial value to lastHeartbeatTimestamp
variable.
As a result when BoltHeartbeatTimeTask fires up for the first time,
getLastHeartbeat() returns value of 0. This in turn leads bolt to die with
"subprocess heartbeat timeout" message.
The fix is to place setHeartBeat() before BoltHeartbeatTimerTask is
created. The patch for this is attached.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/slava92/storm master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1542.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1542
----
commit d38c40026ab5dac266986896424e6c25a440e512
Author: Slava Andreyev <[email protected]>
Date: 2016-07-06T17:14:09Z
initialize lastHeartbeatTimestamp before starting heartbeat task
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
