Robert Joseph Evans created STORM-2422:
------------------------------------------

             Summary: Serialized Trident topology size does not grow linerarly
                 Key: STORM-2422
                 URL: https://issues.apache.org/jira/browse/STORM-2422
             Project: Apache Storm
          Issue Type: Bug
          Components: trident
            Reporter: Robert Joseph Evans
            Assignee: Robert Joseph Evans


Many of the Trident bolts contain a serialized version of the entire trident 
graph.  This means that as the size of the graph grows, and the number of bolts 
grows with it, that the serialized topology grows quadraticly.

The bolt only uses a small portion of that graph so we can filter out most of 
that.

As an extreme example a topology with 1000 Bolts (4000 nodes in the graph) can 
grow to be over 1GB in size.  If we trim out the parts of the graph that are 
not needed the serialized topology is < 40MB in size.

This is a backwards compatible change, because we are only stripping out data 
that is never touched by the bolt anyways.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Reply via email to