[
https://issues.apache.org/jira/browse/HIVE-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697356#comment-14697356
]
Alan Gates commented on HIVE-10289:
-----------------------------------
I'm not done reviewing the patch yet, but I have one bigger question: it looks
like we're doing the comparison in String format (in
PartitionKeyComparator.compareTo we convert the byte[] value passed in into a
String and the values passed in the protobuf are string). Why pay the cost of
the string conversion? Why not leave it in byte[] and use bytes in the
protobuf? This seems like it would be faster since this filter will be applied
to every row in the scan range.
[~sershe], I think the value of ObjectInspectors over OrderedBytes is that
there's guaranteed to be an ObjectInspector for every Hive type, whereas there
are some Hive types not covered by OrderedBytes (e.g. Date, Timestamp).
> Support filter on non-first partition key and non-string partition key
> ----------------------------------------------------------------------
>
> Key: HIVE-10289
> URL: https://issues.apache.org/jira/browse/HIVE-10289
> Project: Hive
> Issue Type: Sub-task
> Components: HBase Metastore, Metastore
> Affects Versions: hbase-metastore-branch
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Attachments: HIVE-10289.1.patch, HIVE-10289.2.patch
>
>
> Currently, partition filtering only handles the first partition key and the
> type for this partition key must be string. In order to break this
> limitation, several improvements are required:
> 1. Change serialization format for partition key. Currently partition keys
> are serialized into delimited string, which sorted on string order not with
> regard to the actual type of the partition key. We use BinarySortableSerDe
> for this purpose.
> 2. For filter condition not on the initial partition keys, push it into HBase
> RowFilter. RowFilter will deserialize the partition key and evaluate the
> filter condition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
