[
https://issues.apache.org/jira/browse/KYLIN-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dayue Gao updated KYLIN-2386:
-----------------------------
Description:
In KYLIN-2349 and KYLIN-2353, we optimized performance of BitmapCounter by
changing its storage format. Cardinality and serialized size are recorded in
the header of the new format, enables us to retrieve those information without
deserialize the data.
In fact, cardinality and serialized size can be quickly calculated just from
the header of [roaring
format|https://github.com/RoaringBitmap/RoaringFormatSpec/]. Performance tests
show that we could achieve the same performance boost without the format change
of BitmapCounter. The benefits are
* there is no need for user to rebuild existing cube to get better performance
* there is no need for developer to maintain two formats and deal with
compatibility issues
was:
In KYLIN-2349 and KYLIN-2353, we optimized performance of BitmapCounter by
changing its storage format. Cardinality and serialized size are recorded in
the header of the new format, enables us to retrieve those information without
deserialize the data.
In fact, cardinality and serialized size can be quickly calculated just from
the header of [roaring
format|https://github.com/RoaringBitmap/RoaringFormatSpec/]. Performance tests
show that we could achieve the same performance boost without the format change
of BitmapCounter. The benefits are
* no need to rebuild existing cube to get better performance
* no need to maintain two formats and deal with compatibility issues
> Revert KYLIN-2349 and KYLIN-2353
> --------------------------------
>
> Key: KYLIN-2386
> URL: https://issues.apache.org/jira/browse/KYLIN-2386
> Project: Kylin
> Issue Type: Task
> Components: Metadata
> Affects Versions: v2.0.0
> Reporter: Dayue Gao
> Assignee: Dayue Gao
>
> In KYLIN-2349 and KYLIN-2353, we optimized performance of BitmapCounter by
> changing its storage format. Cardinality and serialized size are recorded in
> the header of the new format, enables us to retrieve those information
> without deserialize the data.
> In fact, cardinality and serialized size can be quickly calculated just from
> the header of [roaring
> format|https://github.com/RoaringBitmap/RoaringFormatSpec/]. Performance
> tests show that we could achieve the same performance boost without the
> format change of BitmapCounter. The benefits are
> * there is no need for user to rebuild existing cube to get better performance
> * there is no need for developer to maintain two formats and deal with
> compatibility issues
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
