[jira] [Updated] (HIVE-15061) Metastore types are sometimes case sensitive
[ https://issues.apache.org/jira/browse/HIVE-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-15061: --- Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 Status: Resolved (was: Patch Available) Committed to 2.2.0 and 2.1.1. Thanks [~mohitsabharwal] for reviewing the patch. > Metastore types are sometimes case sensitive > > > Key: HIVE-15061 > URL: https://issues.apache.org/jira/browse/HIVE-15061 > Project: Hive > Issue Type: Bug > Components: API >Affects Versions: 1.1.0 >Reporter: Thomas Tauber-Marshall >Assignee: Chaoyu Tang > Fix For: 2.2.0, 2.1.1 > > Attachments: HIVE-15061.1.patch, HIVE-15061.1.patch, HIVE-15061.patch > > > Impala recently encountered an issue with the metastore > ([IMPALA-4260|https://issues.cloudera.org/browse/IMPALA-4260] ) where column > stats would get dropped when adding a column to a table. > The reason seems to be that Hive does a case sensitive check on the column > stats types during an "alter table" and expects the types to be all lower > case. This case sensitive check doesn't appear to happen when the stats are > set in the first place. > We're solving this on the Impala end by storing types in the metastore as all > lower case, but Hive's behavior here is very confusing. It should either > always be case sensitive, so that you can't create column stats with types > that Hive considers invalid, or it should never be case sensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15061) Metastore types are sometimes case sensitive
[ https://issues.apache.org/jira/browse/HIVE-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-15061: --- Attachment: (was: HIVE-15061.1.patch) > Metastore types are sometimes case sensitive > > > Key: HIVE-15061 > URL: https://issues.apache.org/jira/browse/HIVE-15061 > Project: Hive > Issue Type: Bug > Components: API >Affects Versions: 1.1.0 >Reporter: Thomas Tauber-Marshall >Assignee: Chaoyu Tang > Attachments: HIVE-15061.1.patch, HIVE-15061.1.patch, HIVE-15061.patch > > > Impala recently encountered an issue with the metastore > ([IMPALA-4260|https://issues.cloudera.org/browse/IMPALA-4260] ) where column > stats would get dropped when adding a column to a table. > The reason seems to be that Hive does a case sensitive check on the column > stats types during an "alter table" and expects the types to be all lower > case. This case sensitive check doesn't appear to happen when the stats are > set in the first place. > We're solving this on the Impala end by storing types in the metastore as all > lower case, but Hive's behavior here is very confusing. It should either > always be case sensitive, so that you can't create column stats with types > that Hive considers invalid, or it should never be case sensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15061) Metastore types are sometimes case sensitive
[ https://issues.apache.org/jira/browse/HIVE-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-15061: --- Attachment: HIVE-15061.1.patch > Metastore types are sometimes case sensitive > > > Key: HIVE-15061 > URL: https://issues.apache.org/jira/browse/HIVE-15061 > Project: Hive > Issue Type: Bug > Components: API >Affects Versions: 1.1.0 >Reporter: Thomas Tauber-Marshall >Assignee: Chaoyu Tang > Attachments: HIVE-15061.1.patch, HIVE-15061.1.patch, HIVE-15061.patch > > > Impala recently encountered an issue with the metastore > ([IMPALA-4260|https://issues.cloudera.org/browse/IMPALA-4260] ) where column > stats would get dropped when adding a column to a table. > The reason seems to be that Hive does a case sensitive check on the column > stats types during an "alter table" and expects the types to be all lower > case. This case sensitive check doesn't appear to happen when the stats are > set in the first place. > We're solving this on the Impala end by storing types in the metastore as all > lower case, but Hive's behavior here is very confusing. It should either > always be case sensitive, so that you can't create column stats with types > that Hive considers invalid, or it should never be case sensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15061) Metastore types are sometimes case sensitive
[ https://issues.apache.org/jira/browse/HIVE-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-15061: --- Attachment: HIVE-15061.1.patch Reattached patch to trigger the precommit test > Metastore types are sometimes case sensitive > > > Key: HIVE-15061 > URL: https://issues.apache.org/jira/browse/HIVE-15061 > Project: Hive > Issue Type: Bug > Components: API >Affects Versions: 1.1.0 >Reporter: Thomas Tauber-Marshall >Assignee: Chaoyu Tang > Attachments: HIVE-15061.1.patch, HIVE-15061.1.patch, HIVE-15061.patch > > > Impala recently encountered an issue with the metastore > ([IMPALA-4260|https://issues.cloudera.org/browse/IMPALA-4260] ) where column > stats would get dropped when adding a column to a table. > The reason seems to be that Hive does a case sensitive check on the column > stats types during an "alter table" and expects the types to be all lower > case. This case sensitive check doesn't appear to happen when the stats are > set in the first place. > We're solving this on the Impala end by storing types in the metastore as all > lower case, but Hive's behavior here is very confusing. It should either > always be case sensitive, so that you can't create column stats with types > that Hive considers invalid, or it should never be case sensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15061) Metastore types are sometimes case sensitive
[ https://issues.apache.org/jira/browse/HIVE-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-15061: --- Attachment: HIVE-15061.1.patch > Metastore types are sometimes case sensitive > > > Key: HIVE-15061 > URL: https://issues.apache.org/jira/browse/HIVE-15061 > Project: Hive > Issue Type: Bug > Components: API >Affects Versions: 1.1.0 >Reporter: Thomas Tauber-Marshall >Assignee: Chaoyu Tang > Attachments: HIVE-15061.1.patch, HIVE-15061.patch > > > Impala recently encountered an issue with the metastore > ([IMPALA-4260|https://issues.cloudera.org/browse/IMPALA-4260] ) where column > stats would get dropped when adding a column to a table. > The reason seems to be that Hive does a case sensitive check on the column > stats types during an "alter table" and expects the types to be all lower > case. This case sensitive check doesn't appear to happen when the stats are > set in the first place. > We're solving this on the Impala end by storing types in the metastore as all > lower case, but Hive's behavior here is very confusing. It should either > always be case sensitive, so that you can't create column stats with types > that Hive considers invalid, or it should never be case sensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15061) Metastore types are sometimes case sensitive
[ https://issues.apache.org/jira/browse/HIVE-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-15061: --- Status: Patch Available (was: Open) [~aihuaxu], [~mohitsabharwal], could you review the patch since you have done something related. Thanks > Metastore types are sometimes case sensitive > > > Key: HIVE-15061 > URL: https://issues.apache.org/jira/browse/HIVE-15061 > Project: Hive > Issue Type: Bug > Components: API >Affects Versions: 1.1.0 >Reporter: Thomas Tauber-Marshall >Assignee: Chaoyu Tang > Attachments: HIVE-15061.patch > > > Impala recently encountered an issue with the metastore > ([IMPALA-4260|https://issues.cloudera.org/browse/IMPALA-4260] ) where column > stats would get dropped when adding a column to a table. > The reason seems to be that Hive does a case sensitive check on the column > stats types during an "alter table" and expects the types to be all lower > case. This case sensitive check doesn't appear to happen when the stats are > set in the first place. > We're solving this on the Impala end by storing types in the metastore as all > lower case, but Hive's behavior here is very confusing. It should either > always be case sensitive, so that you can't create column stats with types > that Hive considers invalid, or it should never be case sensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15061) Metastore types are sometimes case sensitive
[ https://issues.apache.org/jira/browse/HIVE-15061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoyu Tang updated HIVE-15061: --- Attachment: HIVE-15061.patch This patch is to fix the following issues: 1. alterTableUpdateTableColumnStats and updatePartColumnStatsForAlterColumns did not ignore the case when they compare the columns to determine if their stats should be updated or deleted 2. HMS alterTable allows the uppercase column name and type to be populated to HMS backend DB 3. update_table_column_statistics/update_partition_column_statistics allow the uppercase column type to be populated to HMS backend DB > Metastore types are sometimes case sensitive > > > Key: HIVE-15061 > URL: https://issues.apache.org/jira/browse/HIVE-15061 > Project: Hive > Issue Type: Bug > Components: API >Affects Versions: 1.1.0 >Reporter: Thomas Tauber-Marshall >Assignee: Chaoyu Tang > Attachments: HIVE-15061.patch > > > Impala recently encountered an issue with the metastore > ([IMPALA-4260|https://issues.cloudera.org/browse/IMPALA-4260] ) where column > stats would get dropped when adding a column to a table. > The reason seems to be that Hive does a case sensitive check on the column > stats types during an "alter table" and expects the types to be all lower > case. This case sensitive check doesn't appear to happen when the stats are > set in the first place. > We're solving this on the Impala end by storing types in the metastore as all > lower case, but Hive's behavior here is very confusing. It should either > always be case sensitive, so that you can't create column stats with types > that Hive considers invalid, or it should never be case sensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
