[
https://issues.apache.org/jira/browse/IGNITE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Kasnacheev updated IGNITE-12751:
-------------------------------------
Attachment: word-count-reproducer.zip
> callAsync(jobs, rdc) performance degrades quadratically as jobs.size() grows
> ----------------------------------------------------------------------------
>
> Key: IGNITE-12751
> URL: https://issues.apache.org/jira/browse/IGNITE-12751
> Project: Ignite
> Issue Type: Bug
> Components: compute
> Affects Versions: 2.7.6
> Reporter: Ilya Kasnacheev
> Priority: Major
> Attachments: word-count-reproducer.zip
>
>
> Please consider attached reproducer and linked report.
> {{compute.callAsync(jobs, reducer);
> Result [res=33, tookMs=81, jobs=5] //warm up
> Result [res=99, tookMs=21, jobs=15]
> Result [res=330, tookMs=22, jobs=50]
> Result [res=990, tookMs=57, jobs=150]
> Result [res=3300, tookMs=146, jobs=500]
> Result [res=9900, tookMs=231, jobs=1500]
> Result [res=33000, tookMs=840, jobs=5000]
> Result [res=99000, tookMs=6965, jobs=15000]
> Result [res=330000, tookMs=118394, jobs=50000]}}
> As soon jobs.size() grows past 5000, performance begins to degrade
> quadratically.
> I don't expect that it will be completely linear, but I would assume that it
> should stay linear-ish until size() hits at least 100000, given that we see
> clusters of 100 nodes and it's not unthinkable to expect 1000 jobs to be run
> on each node. 5000 jobs (which still give OK performance) / 100 nodes is just
> 50 jobs per node, which becomes limiting factor.
> Linked question also mentions OOM event, which may be caused of intermediate
> storage of (N^2) data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
