Github user simafengyun commented on the issue:
https://github.com/apache/incubator-carbondata/pull/627
You mentioned the below,
>>>>>>>>>>>>>>>>>>>>>>>>>>
But using this search in setFilterdIndexToBitSet is not always correct.
Because here the data may not be sorted always, Only for the first column of
dimensions is sorted naturally because of mdk order.
>>>>>>>>>>>>>>>>>>>>>>>>>>
but i don't think so.
the order you mentioned is the logical level order(MDK).
As I know, for dimension column, it has physical order in chunk level.
for dimension data which has dictionary encode, the dictionary data will
sorted in blocklet level and keep the order in chunk on the physical disk.
So after the one chunk dimension data read, it will keep the order, so I
think it is fit for the binary search.
if I was wrong, please feel free to tell me, thanks
At 2017-03-07 17:44:28, "Ravindra Pesala" <[email protected]> wrote:
@simafengyun Thanks for working on it.
Range binary search is impressive. we can use same search in other places
as well.
But using this search in setFilterdIndexToBitSet is not always correct.
Because here the data may not be sorted always, Only for the first column of
dimensions is sorted naturally because of mdk order.
And if the data is sorted explicitly(it means it has inverted index) then
it goes to another method setFilterdIndexToBitSetWithColumnIndex. So here we
need extra checks to do binary search in the method setFilterdIndexToBitSet ,
that is like whether the column is naturally sorted or not.This information we
may need get from store.
Please use this Range binary search in
'setFilterdIndexToBitSetWithColumnIndex' as well.
â
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
