[
https://issues.apache.org/jira/browse/IGNITE-20503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856710#comment-17856710
]
Andrey Mashenkov commented on IGNITE-20503:
-------------------------------------------
I'd suggest using more effective bitmap implementations like RoaringBitmap or
Tree-encoded bitmap or else.
Typical query execution flow is running fragments on data nodes and fetch the
data to a single node.
As I understand, mostly used execution targets will be OneOf (of a single node)
and Partitioned (a bunch of very small node sets, comparing to cluster size).
In most cases, I'd expect a BitMap will contains either a single 1-bit or have
high clustering factor (i.e. how probable 1-bit will be followed by another
1-bit). In these cases, suggested implementations are very effective in both
aspects: memory and performance.
> Sql. Support big clusters by mapping service
> --------------------------------------------
>
> Key: IGNITE-20503
> URL: https://issues.apache.org/jira/browse/IGNITE-20503
> Project: Ignite
> Issue Type: Improvement
> Components: sql
> Reporter: Konstantin Orlov
> Assignee: Andrey Mashenkov
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> After refactoring made in IGNITE-20502, only clusters up to 64 nodes
> supported by MappingService.
> Let's implement one more ExecutionTargetFactory to support clusters of
> arbitrary size. See SmallClusterFactory and other classes in the same package
> for example.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
