[
https://issues.apache.org/jira/browse/STORM-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rick Kellogg updated STORM-157:
-------------------------------
Component/s: storm-core
> ack/fail should not require keeping input values in scope.
> ----------------------------------------------------------
>
> Key: STORM-157
> URL: https://issues.apache.org/jira/browse/STORM-157
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: James Xu
> Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/752
> ack/fail takes a Tuple, but it appears the values are not needed to ack. If
> we aggregate many things locally before we commit, we keep refs to many
> Tuples. We think this could be keeping more in memory than we need and
> pushing some topologies to the breaking point.
> We are not 100% that this is the issue, but it would be good to have
> something like, getToken on Tuple. And ack/fail might take that Token which
> is an opaque object that includes the minimal refs to ack.
> ----------
> nathanmarz: +1
> This should be pretty easy to implement. Tuples already have a MessageID
> which is the primary object used for acking. There's also the "ack val" (the
> xors of tuples anchored onto this tuple) which should be moved into the
> MessageID. Then we can use the MessageID as the token, and update the APIs to
> accept the MessageID for acking and MessageID for anchoring as part of
> execute. IOutputCollector should be changed to only accept MessageID for
> acking/failing/anchoring, and then OutputCollector can add the convenience
> methods for accepting Tuple acking/anchoring/failing.
> ----------
> jmlogan: We had this similar issue when we had very large tuples...at time
> time I "worked around it" by using Reflection to get ahold of the internal
> list storing the values, and clearing it.
> I ended up mitigating this problem for good by having smaller tuples, and
> passing the large payloads through Redis.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
