[
https://issues.apache.org/jira/browse/HIVE-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705713#comment-14705713
]
Alan Gates commented on HIVE-10289:
-----------------------------------
Ok, thanks. That makes sense. Then my only other comment is that in
PKC.compareTo, there' s a lot of setup work being done at the beginning of the
function:
{code}
byte[] bytes = Arrays.copyOfRange(value, offset, offset + length);
LOG.info("Get key " + new String(bytes));
BinarySortableSerDe serDe = new BinarySortableSerDe();
Properties props = new Properties();
props.setProperty(serdeConstants.LIST_COLUMNS, "dbName,tableName," + names);
props.setProperty(serdeConstants.LIST_COLUMN_TYPES, "string,string," + types);
{code}
Given that this method will be called for every row in the scan range, could we
do this setup once?
Also, this method uses info level logging, which seems like it will flood the
debug logs. And given that this method will be in the inner loop of the scan
we should probably gate any logging statements that construct objects with {{if
(LOG.debugIsEnabled())}}
> Support filter on non-first partition key and non-string partition key
> ----------------------------------------------------------------------
>
> Key: HIVE-10289
> URL: https://issues.apache.org/jira/browse/HIVE-10289
> Project: Hive
> Issue Type: Sub-task
> Components: HBase Metastore, Metastore
> Affects Versions: hbase-metastore-branch
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Attachments: HIVE-10289.1.patch, HIVE-10289.2.patch
>
>
> Currently, partition filtering only handles the first partition key and the
> type for this partition key must be string. In order to break this
> limitation, several improvements are required:
> 1. Change serialization format for partition key. Currently partition keys
> are serialized into delimited string, which sorted on string order not with
> regard to the actual type of the partition key. We use BinarySortableSerDe
> for this purpose.
> 2. For filter condition not on the initial partition keys, push it into HBase
> RowFilter. RowFilter will deserialize the partition key and evaluate the
> filter condition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
