[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965729#comment-15965729
]
Taras Ledkov edited comment on IGNITE-3018 at 4/12/17 12:25 PM:
Patch without balancer: [PR1779|https://github.com/apache/ignite/pull/1779]
Branch {{ignite-3018}}
Waits for TC
[results|http://195.239.208.174/project.html?projectId=IgniteTests=projectOverview_IgniteTests=pull%2F1779%2Fhead]
was (Author: tledkov-gridgain):
Patch without balancer: [PR1779|https://github.com/apache/ignite/pull/1779]
Branch {{ignite-3018}}
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Yakov Zhdanov
> Labels: important
> Fix For: 2.0
>
> Attachments: 003.png, 004.png, 008.png, 016.png, 064.png, 100.png,
> 128.png, 200.png, 256.png, 400.png, 600.png, balanced.003.png,
> balanced.004.png, balanced.008.png, balanced.016.png, balanced.064.png,
> balanced.100.png, balanced.128.png, balanced.200.png, balanced.256.png,
> balanced.400.png, balanced.600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965729#comment-15965729
]
Taras Ledkov edited comment on IGNITE-3018 at 4/12/17 12:24 PM:
Patch without balancer: [PR1779|https://github.com/apache/ignite/pull/1779]
Branch {{ignite-3018}}
was (Author: tledkov-gridgain):
Patch without balancer: [PR1779|https://github.com/apache/ignite/pull/1779]
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Yakov Zhdanov
> Labels: important
> Fix For: 2.0
>
> Attachments: 003.png, 004.png, 008.png, 016.png, 064.png, 100.png,
> 128.png, 200.png, 256.png, 400.png, 600.png, balanced.003.png,
> balanced.004.png, balanced.008.png, balanced.016.png, balanced.064.png,
> balanced.100.png, balanced.128.png, balanced.200.png, balanced.256.png,
> balanced.400.png, balanced.600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964431#comment-15964431
]
Taras Ledkov edited comment on IGNITE-3018 at 4/11/17 6:36 PM:
---
Tables below contains average count of partitions to rebalance (new assignment
contains a node that doesn't exist in the previous assignment)
(2 backups, 1024 partitions)
|| Nodes || Fair (total) || Fair (parts on stable nodes)
||Rendezvous (total)||Rendezvous (parts on stable nodes)||
| *3* | 988.3|397.2 | 729.3 | 459.3 |
| *4* | 790.7|563.1 | 638.3 | 392.3 |
| *6* | 408.3|243.8 | 447.0 | 257.9 |
| *8* | 322.8|195.8 | 380.2 | 235.7 |
| *10* | 270.8|156.2 | 314.1 | 194.7 |
| *64* | 47.2 |31.2 | 135.1 | 149.9 |
(2 backups, 10240 partitions)
|| Nodes || Fair (total) || Fair (parts on stable nodes)
||Rendezvous (total)||Rendezvous (parts on stable nodes)||
| *100* | 324.7| 184.4| 570.5|425.1|
| *200* | 157.6| 96.6 | 769.1|692.6|
| *300* | 114.7| 62.3 | 935.7|285.9|
| *400* | 79.2 | 44.4 | 336.3|318.4|
| *500* | 63.0 | 34.9 | 277.3|256.7|
| *600* | 54.0 | 30.1 | 175.8|142.7|
was (Author: tledkov-gridgain):
Tables below contains average count of partitions to rebalance (new assignment
contains a node that doesn't exist in the previous assignment)
(2 backups, 1024 partitions)
|| Nodes || Fair (total) || Fair (parts on stable nodes)
||Rendezvous (total)||Rendezvous (parts on stable nodes)||
| *3* | 988.3|397.2 | 729.3 | 459.3 |
| *4* | 790.7|563.1 | 638.3 | 392.3 |
| *6* | 408.3|243.8 | 447.0 | 257.9 |
| *8* | 322.8|195.8 | 380.2 | 235.7 |
| *10* | 270.8|156.2 | 314.1 | 194.7 |
| *64* | 47.2 |31.2 | 135.1 | 149.9 |
(2 backups, 10240 partitions)
|| Nodes || Fair (total) || Fair (parts on stable nodes)
||Rendezvous (total)||Rendezvous (parts on stable nodes)||
| *100* | 324.7| 184.4| 570.5|425.1|
| *200* | 157.6| 96.6 | 547.2|692.6|
| *300* | 114.7| 62.3 | 935.7|285.9|
| *400* | 79.2 | 44.4 | 336.3|318.4|
| *500* | 63.0 | 34.9 | 277.3|256.7|
| *600* | 54.0 | 30.1 | 175.8|142.7|
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Yakov Zhdanov
> Labels: important
> Fix For: 2.0
>
> Attachments: 003.png, 004.png, 008.png, 016.png, 064.png, 100.png,
> 128.png, 200.png, 256.png, 400.png, 600.png, balanced.003.png,
> balanced.004.png, balanced.008.png, balanced.016.png, balanced.064.png,
> balanced.100.png, balanced.128.png, balanced.200.png, balanced.256.png,
> balanced.400.png, balanced.600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964431#comment-15964431
]
Taras Ledkov edited comment on IGNITE-3018 at 4/11/17 6:33 PM:
---
Tables below contains average count of partitions to rebalance (new assignment
contains a node that doesn't exist in the previous assignment)
(2 backups, 1024 partitions)
|| Nodes || Fair (total) || Fair (parts on stable nodes)
||Rendezvous (total)||Rendezvous (parts on stable nodes)||
| *3* | 988.3|397.2 | 729.3 | 459.3 |
| *4* | 790.7|563.1 | 638.3 | 392.3 |
| *6* | 408.3|243.8 | 447.0 | 257.9 |
| *8* | 322.8|195.8 | 380.2 | 235.7 |
| *10* | 270.8|156.2 | 314.1 | 194.7 |
| *64* | 47.2 |31.2 | 135.1 | 149.9 |
(2 backups, 10240 partitions)
|| Nodes || Fair (total) || Fair (parts on stable nodes)
||Rendezvous (total)||Rendezvous (parts on stable nodes)||
| *100* | 324.7| 184.4| 570.5|425.1|
| *200* | 157.6| 96.6 | 547.2|692.6|
| *300* | 114.7| 62.3 | 935.7|285.9|
| *400* | 79.2 | 44.4 | 336.3|318.4|
| *500* | 63.0 | 34.9 | 277.3|256.7|
| *600* | 54.0 | 30.1 | 175.8|142.7|
was (Author: tledkov-gridgain):
Tables below contains average count of partitions to rebalance (new assignment
contains a node that doesn't exist in the previous assignment)
(2 backups, 1024 partitions)
|| Nodes || Fair ||Rendezvous (balanced)||
| *3* | 988.3 | 729.3 |
| *4* | 790.7 | 638.3 |
| *6* | 408.3 | 447.0 |
| *8* | 322.8 | 380.2 |
| *10* | 270.8 | 314.1 |
| *64* | 47.2 | 135.1 |
(2 backups, 10240 partitions)
|| Nodes || Fair ||Rendezvous (balanced)||
| *100* | 324.7| 570.5|
| *200* | 157.6| 547.2|
| *300* | 114.7| 935.7|
| *400* | 79.2 | 336.3|
| *500* | 63.0 | 277.3|
| *600* | 54.0 | 175.8|
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Yakov Zhdanov
> Labels: important
> Fix For: 2.0
>
> Attachments: 003.png, 004.png, 008.png, 016.png, 064.png, 100.png,
> 128.png, 200.png, 256.png, 400.png, 600.png, balanced.003.png,
> balanced.004.png, balanced.008.png, balanced.016.png, balanced.064.png,
> balanced.100.png, balanced.128.png, balanced.200.png, balanced.256.png,
> balanced.400.png, balanced.600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964431#comment-15964431
]
Taras Ledkov edited comment on IGNITE-3018 at 4/11/17 2:19 PM:
---
Tables below contains average count of partitions to rebalance (new assignment
contains a node that doesn't exist in the previous assignment)
(2 backups, 1024 partitions)
|| Nodes || Fair ||Rendezvous (balanced)||
| *3* | 988.3 | 729.3 |
| *4* | 790.7 | 638.3 |
| *6* | 408.3 | 447.0 |
| *8* | 322.8 | 380.2 |
| *10* | 270.8 | 314.1 |
| *64* | 47.2 | 135.1 |
(2 backups, 10240 partitions)
|| Nodes || Fair ||Rendezvous (balanced)||
| *100* | 324.7| 570.5|
| *200* | 157.6| 547.2|
| *300* | 114.7| 935.7|
| *400* | 79.2 | 336.3|
| *500* | 63.0 | 277.3|
| *600* | 54.0 | 175.8|
was (Author: tledkov-gridgain):
Tables below contains average count of partitions to rebalance (new assignment
contains a node that doesn't exist in the previous assignment)
(2 backups, 1024 partitions)
|| Nodes || Fair ||Rendezvous ||
| *3* | 988.3 | 729.3 |
| *4* | 790.7 | 638.3 |
| *6* | 408.3 | 447.0 |
| *8* | 322.8 | 380.2 |
| *10* | 270.8 | 314.1 |
| *64* | 47.2 | 135.1 |
(2 backups, 10240 partitions)
|| Nodes || Fair ||Rendezvous ||
| *100* | 324.7| 570.5|
| *200* | 157.6| 547.2|
| *300* | 114.7| 935.7|
| *400* | 79.2 | 336.3|
| *500* | 63.0 | 277.3|
| *600* | 54.0 | 175.8|
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Yakov Zhdanov
> Labels: important
> Fix For: 2.0
>
> Attachments: 003.png, 004.png, 008.png, 016.png, 064.png, 100.png,
> 128.png, 200.png, 256.png, 400.png, 600.png, balanced.003.png,
> balanced.004.png, balanced.008.png, balanced.016.png, balanced.064.png,
> balanced.100.png, balanced.128.png, balanced.200.png, balanced.256.png,
> balanced.400.png, balanced.600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963015#comment-15963015
]
Taras Ledkov edited comment on IGNITE-3018 at 4/10/17 7:23 PM:
---
Average count of primary partitions to migrate when one node leaves (test does
about 100 topology changes, AF configured with 1K partitions).
NB: For pure-Rendezvous AF this value is 0.0.
For FairAF this value <1.0
|| Nodes || Parts to migrate ||
| 2 | 13.8 |
| 3 | 13.4 |
| 4 | 18.1 |
| 6 | 17.5 |
| 8 | 16.8 |
| 10 | 19.4 |
| 64 | 26.3 |
| 100 | 6.1 |
| 200 | 3.8 |
| 300 | 3.9 |
| 400 | 2.7 |
| 500 | 0.6 |
| 600 | 1.8 |
was (Author: tledkov-gridgain):
Average count of primary partitions to migrate when one node leaves (test does
about 100 topology changes).
NB: For pure-Rendezvous AF this value is 0.0.
For FairAF this value <1.0
|| Nodes || Parts to migrate ||
| 2 | 13.8 |
| 3 | 13.4 |
| 4 | 18.1 |
| 6 | 17.5 |
| 8 | 16.8 |
| 10 | 19.4 |
| 64 | 26.3 |
| 100 | 6.1 |
| 200 | 3.8 |
| 300 | 3.9 |
| 400 | 2.7 |
| 500 | 0.6 |
| 600 | 1.8 |
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Yakov Zhdanov
> Labels: important
> Fix For: 2.0
>
> Attachments: 003.png, 004.png, 008.png, 016.png, 064.png, 100.png,
> 128.png, 200.png, 256.png, 400.png, 600.png, balanced.003.png,
> balanced.004.png, balanced.008.png, balanced.016.png, balanced.064.png,
> balanced.100.png, balanced.128.png, balanced.200.png, balanced.256.png,
> balanced.400.png, balanced.600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943316#comment-15943316
]
Taras Ledkov edited comment on IGNITE-3018 at 3/28/17 11:54 AM:
TC
[results|http://195.239.208.174/project.html?projectId=IgniteTests=projectOverview_IgniteTests=pull%2F1647%2Fhead]
Tests results are OK with me. I don't see significant difference of the tests
results between the master branch and the patch.
was (Author: tledkov-gridgain):
Waits for TC
[results|http://195.239.208.174/project.html?projectId=IgniteTests=projectOverview_IgniteTests=pull%2F1647%2Fhead]
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
> Attachments: 003.png, 004.png, 008.png, 016.png, 064.png, 100.png,
> 128.png, 200.png, 256.png, 400.png, 600.png, balanced.003.png,
> balanced.004.png, balanced.008.png, balanced.016.png, balanced.064.png,
> balanced.100.png, balanced.128.png, balanced.200.png, balanced.256.png,
> balanced.400.png, balanced.600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15926181#comment-15926181
]
Taras Ledkov edited comment on IGNITE-3018 at 3/27/17 12:09 PM:
Core review with partitions balancer:
[IGNT-CR-137|http://reviews.ignite.apache.org/ignite/review/IGNT-CR-137]
was (Author: tledkov-gridgain):
Core review with partitions balancer:
[IGNT-CR-116|http://reviews.ignite.apache.org/ignite/review/IGNT-CR-116]
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
> Attachments: 003.png, 004.png, 008.png, 016.png, 064.png, 100.png,
> 128.png, 200.png, 256.png, 400.png, 600.png, balanced.003.png,
> balanced.004.png, balanced.008.png, balanced.016.png, balanced.064.png,
> balanced.100.png, balanced.128.png, balanced.200.png, balanced.256.png,
> balanced.400.png, balanced.600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878234#comment-15878234
]
Taras Ledkov edited comment on IGNITE-3018 at 2/22/17 1:49 PM:
---
Chi^2 test:
|| Nodes || Fair || Rendezvous (old)|| FastRendezvous ||
|*5* | 0.04 | 0.003618| 0.002531 |
|*64*| 0.00 | 0.063599| 0.077637 |
|*100* | 0.001740 | 0.113892| 0.115036 |
|*128* | 0.00 | 0.121094| 0.125732 |
|*200* | 0.004028 | 0.203918| 0.216888 |
|*256* | 0.00 | 0.248535| 0.269531 |
|*300* | 0.020813 | 0.280594| 0.287460 |
|*400* | 0.037598 | 0.361847| 0.396179 |
|*500* | 0.010895 | 0.501083| 0.504898 |
|*600* | 0.071167 | 0.589584| 0.617050 |
was (Author: tledkov-gridgain):
Chi^2 test:
|| Nodes || Fair || Rendezvous (old)|| FastRendezvous ||
|*5* | 0.002531 | 0.003618| 0.002531 |
|*64*| 0.077637 | 0.063599| 0.077637 |
|*100* | 0.115036 | 0.113892| 0.115036 |
|*128* | 0.125732 | 0.121094| 0.125732 |
|*200* | 0.216888 | 0.203918| 0.216888 |
|*256* | 0.269531 | 0.248535| 0.269531 |
|*300* | 0.287460 | 0.280594| 0.287460 |
|*400* | 0.396179 | 0.361847| 0.396179 |
|*500* | 0.504898 | 0.501083| 0.504898 |
|*600* | 0.617050 | 0.589584| 0.617050 |
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Yakov Zhdanov
> Fix For: 2.0
>
> Attachments: 003.png, 064.png, 100.png, 128.png, 200.png, 300.png,
> 400.png, 500.png, 600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-3018) Cache affinity calculation is slow with large nodes number
[
https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871804#comment-15871804
]
Taras Ledkov edited comment on IGNITE-3018 at 2/17/17 1:04 PM:
---
[~yzhdanov], [~sboikov] , please review the
[patch|https://github.com/apache/ignite/pull/684].
was (Author: tledkov-gridgain):
[~yzhdanov], please review the
[patch|https://github.com/apache/ignite/pull/684].
> Cache affinity calculation is slow with large nodes number
> --
>
> Key: IGNITE-3018
> URL: https://issues.apache.org/jira/browse/IGNITE-3018
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Semen Boikov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
> Attachments: 003.png, 064.png, 100.png, 128.png, 200.png, 300.png,
> 400.png, 500.png, 600.png
>
>
> With large number of cache server nodes (> 200) RendezvousAffinityFunction
> and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of
> milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and
> nodes list sorting. As optimization we can try to cache {partion, node} MD5
> hash or try another hash function. Also several minor optimizations are
> possible (avoid unncecessary allocations, only one thread local 'get', etc).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
