[ 
https://issues.apache.org/jira/browse/STORM-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim resolved STORM-2422.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 2.0.0

Thanks [~revans2], I merged into master.

I'm following your comment on PR and just only merged into master, but we can 
port back this anytime.

> Serialized Trident topology size does not grow linerarly
> --------------------------------------------------------
>
>                 Key: STORM-2422
>                 URL: https://issues.apache.org/jira/browse/STORM-2422
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: trident
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>             Fix For: 2.0.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> Many of the Trident bolts contain a serialized version of the entire trident 
> graph.  This means that as the size of the graph grows, and the number of 
> bolts grows with it, that the serialized topology grows quadraticly.
> The bolt only uses a small portion of that graph so we can filter out most of 
> that.
> As an extreme example a topology with 1000 Bolts (4000 nodes in the graph) 
> can grow to be over 1GB in size.  If we trim out the parts of the graph that 
> are not needed the serialized topology is < 40MB in size.
> This is a backwards compatible change, because we are only stripping out data 
> that is never touched by the bolt anyways.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Reply via email to