[
https://issues.apache.org/jira/browse/KYLIN-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822173#comment-15822173
]
Daniel Lemire commented on KYLIN-2386:
--------------------------------------
We would be willing to expose static functions that compute the cardinality and
the serialized size directly within the RoaringBitmap library without needing
any memory allocation whatsoever, just data access.
The code is already there, it is just private:
https://github.com/RoaringBitmap/RoaringBitmap/blob/master/src/main/java/org/roaringbitmap/buffer/ImmutableRoaringArray.java#L131-L147
If you are interested, just ping me with the desired function signatures.
> Revert KYLIN-2349 and KYLIN-2353
> --------------------------------
>
> Key: KYLIN-2386
> URL: https://issues.apache.org/jira/browse/KYLIN-2386
> Project: Kylin
> Issue Type: Task
> Components: Metadata
> Affects Versions: v2.0.0
> Reporter: Dayue Gao
> Assignee: Dayue Gao
> Fix For: v2.0.0
>
>
> In KYLIN-2349 and KYLIN-2353, we changed the storage format of BitmapCounter
> for better performance. In the new format, cardinality and serialized size
> are recorded in the header part. This enables us to retrieve those
> information without deserialize the whole data.
> However, cardinality and serialized size can be quickly calculated just from
> the header of [roaring
> format|https://github.com/RoaringBitmap/RoaringFormatSpec/]. Performance
> tests show that we could achieve the same performance gain without the format
> change. The benefits are
> * there is no need for user to rebuild existing cube to get better performance
> * there is no need for developer to maintain two formats and deal with
> compatibility issues
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
