[
https://issues.apache.org/jira/browse/STORM-339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Noll updated STORM-339:
-------------------------------
Description:
Without any ackers enabled, fast component will continuously leak memory and
causing OOM problems when target component is slow. The OOM problem can be
reproduced by running this fast-slow-topology:
https://github.com/Gvain/storm-perf-test/tree/fast-slow-topology
with command:
{code}
$ storm jar storm_perf_test-1.0.0-SNAPSHOT-jar-with-dependencies.jar
com.yahoo.storm.perftest.Main --spout 1 --bolt 1 --workers 2 --testTime 600
--messageSize 6400
{code}
And the worker childopts with {{-Xms2g -Xmx2g -Xmn512m ...}}.
At the same time, the executed count of target component is far behind from the
emitted count of source component. I guess it could be that netty client is
buffering too much messages in its message_queue as target component sends back
OK/Failure Response too slowly.
was:
Without any ackers enabled, fast component will continuously leak memory and
causing OOM problems when target component is slow. The OOM problem can be
reproduced by running this fast-slow-topology
https://github.com/Gvain/storm-perf-test/tree/fast-slow-topology
with command:
storm jar storm_perf_test-1.0.0-SNAPSHOT-jar-with-dependencies.jar
com.yahoo.storm.perftest.Main --spout 1 --bolt 1 --workers 2 --testTime 600
--messageSize 6400
And the worker childopts with -Xms2g -Xmx2g -Xmn512m ...
At the same time, the executed count of target component is far behind from the
emitted count of source component.
I guess it could be that netty client is buffering too much messages in its
mesage_queue as target component sends back OK/Failure Response too slowly.
> Severe memory leak to OOM when ackers disabled
> ----------------------------------------------
>
> Key: STORM-339
> URL: https://issues.apache.org/jira/browse/STORM-339
> Project: Apache Storm (Incubating)
> Issue Type: Bug
> Affects Versions: 0.9.2-incubating
> Reporter: Jiahong Li
> Priority: Critical
>
> Without any ackers enabled, fast component will continuously leak memory and
> causing OOM problems when target component is slow. The OOM problem can be
> reproduced by running this fast-slow-topology:
> https://github.com/Gvain/storm-perf-test/tree/fast-slow-topology
> with command:
> {code}
> $ storm jar storm_perf_test-1.0.0-SNAPSHOT-jar-with-dependencies.jar
> com.yahoo.storm.perftest.Main --spout 1 --bolt 1 --workers 2 --testTime 600
> --messageSize 6400
> {code}
> And the worker childopts with {{-Xms2g -Xmx2g -Xmn512m ...}}.
> At the same time, the executed count of target component is far behind from
> the emitted count of source component. I guess it could be that netty client
> is buffering too much messages in its message_queue as target component sends
> back OK/Failure Response too slowly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
