[
https://issues.apache.org/jira/browse/GEODE-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721889#comment-16721889
]
ASF subversion and git services commented on GEODE-6148:
--------------------------------------------------------
Commit f0a8721c129a3214be0f89b0fe4041435776c8a3 in geode-benchmarks's branch
refs/heads/develop from [~balesh2]
[ https://gitbox.apache.org/repos/asf?p=geode-benchmarks.git;h=f0a8721 ]
GEODE-6148: improve performance of Prepopulate (#25)
Prepopulating the region used to occur by means of puts from every
server for every key in the specified range. This resulted in
unnecessarily conflicting puts.
To improve performance of the setup, PrePopulateRegion.run() now divides
the key range as evenly as possible between all servers in the cluster,
prepares the data in a map, then puts the map to the region using
putAll() in threads.
Also:
* add redundancy to Partitioned Regions
* increase JVM Memory
The changes result in an observable improvement in performance.
Signed-off-by: Dan Smith <[email protected]>
> Improve the efficiency of PrePopulateRegion and increase the key range
> ----------------------------------------------------------------------
>
> Key: GEODE-6148
> URL: https://issues.apache.org/jira/browse/GEODE-6148
> Project: Geode
> Issue Type: Improvement
> Components: benchmarks
> Reporter: Dan Smith
> Assignee: Helena Bales
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We are currently running with a keyRange of 1000, which may be skewing our
> results. We want to run with a much larger key range, which means
> prepopulating more data.
> The PrePopulateRegion task could be much faster if we take some of these
> steps:
> * Stop duplicating work - right now each server will put the entire key
> range, resulting in duplicate puts of the same keys
> * use multiple threads - each server populating the region could be using
> multiple threads to populate it faster. One option would be to use a
> parallelStream, there may be better choicse.
> * Use putAlls - putAlls in batches require fewer round trips than puts.
> Acceptance:
> * All benchmarks can finish their prepopulation step in 30 seconds or less on
> reasonable hardware.
> * We set a fixed key range that is somewhere between 100 thousand and 100
> million, whatever is most reasonable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
