[
https://issues.apache.org/jira/browse/HIVE-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705698#comment-14705698
]
Daniel Dai commented on HIVE-10289:
-----------------------------------
The benefit of BinarySortableSerde is realized by doing a prefix HBase scan.
PartitionKeyComparator is to handle the case if there are additional conditions
beside the prefix scan, which includes comparison on minor key (non-prefix
part), operators other than comparison. BinarySortableSerde only offer byte
comparison from the beginning, not from the middle (minor key). It is possible
to do additional optimization to do binary comparison in the middle by skipping
the major fields, but this is not what BinarySortableSerde currently can do.
> Support filter on non-first partition key and non-string partition key
> ----------------------------------------------------------------------
>
> Key: HIVE-10289
> URL: https://issues.apache.org/jira/browse/HIVE-10289
> Project: Hive
> Issue Type: Sub-task
> Components: HBase Metastore, Metastore
> Affects Versions: hbase-metastore-branch
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Attachments: HIVE-10289.1.patch, HIVE-10289.2.patch
>
>
> Currently, partition filtering only handles the first partition key and the
> type for this partition key must be string. In order to break this
> limitation, several improvements are required:
> 1. Change serialization format for partition key. Currently partition keys
> are serialized into delimited string, which sorted on string order not with
> regard to the actual type of the partition key. We use BinarySortableSerDe
> for this purpose.
> 2. For filter condition not on the initial partition keys, push it into HBase
> RowFilter. RowFilter will deserialize the partition key and evaluate the
> filter condition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
