[
https://issues.apache.org/jira/browse/HIVE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-6052:
-----------------------------------
Release Note: JDO pushdown for integers in metastore didn't work correctly
for some legacy data in partition columns in Hive 0.12. In 0.13,
hive.metastore.integral.jdo.pushdown setting is added to enable it, and it's
disabled by default. Enabling it improves metastore perf for integral columns,
especially if there's a large number of partitions. However, it doesn't work
correctly with integral values that are not normalized (e.g. have leading
zeroes, like 0012). If metastore direct SQL is enabled and works, this
optimization is also irrelevant. (was: JDO pushdown for integers in metastore
didn't work correctly in Hive 0.12. In 0.13,
hive.metastore.integral.jdo.pushdown setting is added to enable it, and it's
disabled by default. Enabling it improves metastore perf for integral columns,
especially if there's a large number of partitions. However, it doesn't work
correctly with integral values that are not normalized (e.g. have leading
zeroes, like 0012). If metastore direct SQL is enabled and works, this
optimization is also irrelevant.)
> metastore JDO filter pushdown for integers may produce unexpected results
> with non-normalized integer columns
> -------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-6052
> URL: https://issues.apache.org/jira/browse/HIVE-6052
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.12.0, 0.13.0
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Fix For: 0.13.0
>
> Attachments: HIVE-6052.01.patch, HIVE-6052.02.patch, HIVE-6052.patch
>
>
> If integer partition columns have values stores in non-canonical form, for
> example with leading zeroes, the integer filter doesn't work. That is because
> JDO pushdown uses substrings to compare for equality, and SQL pushdown is
> intentionally crippled to do the same to produce same results.
> Probably, since both SQL pushdown and integers pushdown are just perf
> optimizations, we can remove it for JDO (or make configurable and disable by
> default), and uncripple SQL.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
