[
https://issues.apache.org/jira/browse/STORM-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745154#comment-16745154
]
Stig Rohde Døssing commented on STORM-2359:
-------------------------------------------
Thanks, responded there.
Just to give an update on where I am with this, I took another look at whether
we can avoid all the extra tracking of anchors in the critical path. It turns
out we can, if the JCTools queues are updated to allow another thread to peek
at the queue contents. I have a local branch of JCTools that seems like it can
do this, so I'll try suggesting adding this method to the JCTools guys.
If we can peek at the queue contents, we can do resets for all queued tuples
without adding any extra logic to the critical path. The timeout resetter
thread can just look at the queue contents, without having to involve the
critical path code in maintaining a ConcurrentHashMap of anchors.
Resetting timeouts for queued tuples will let users reset timeouts that have
been delivered to a bolt's exeute() without risking queued tuples timing out
(the method is effectively useless right now).
As a convenience, we can add a component-level configuration that will do
automatic timeout reset for tuples that have been delivered to execute() but
not acked or failed, using the same technique (a ConcurrentHashMap) as I
proposed above for the worker.
This solution would be much better than what I suggested before, because it
means users can choose to enable the expensive tracking only for specific
problem bolts, and only if they don't want to/can't manually call
collector.resetTimeout. Resetting for tuples that are queued will be pretty
cheap, because it doesn't add any slowdown to the critical path, and only
produces reset tuples when processing is actually slow.
> Revising Message Timeouts
> -------------------------
>
> Key: STORM-2359
> URL: https://issues.apache.org/jira/browse/STORM-2359
> Project: Apache Storm
> Issue Type: Sub-task
> Components: storm-core
> Affects Versions: 2.0.0
> Reporter: Roshan Naik
> Assignee: Stig Rohde Døssing
> Priority: Major
> Attachments: STORM-2359-with-auto-reset.ods, STORM-2359.ods
>
>
> A revised strategy for message timeouts is proposed here.
> Design Doc:
>
> https://docs.google.com/document/d/1am1kO7Wmf17U_Vz5_uyBB2OuSsc4TZQWRvbRhX52n5w/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
