[jira] [Commented] (HIVE-3404) UDF to obtain the quarter of an year if a date or timestamp is given .
[
https://issues.apache.org/jira/browse/HIVE-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440127#comment-13440127
]
Deepti Antony commented on HIVE-3404:
-
We can add a UDF which will return the quarter in which a particular date lies.
Case 1 : hive SELECT QUARTER ('2011-02-18')FROM TABLE_NAME ;
1
Case 2: hive SELECT * FROM TABLE_NAME WHERE QUARTER (date) = 2;
date|store_id|unit|value|price
2010-10-12 |11001|3|5.00|15.00
2011-02-18 |12001|4|6.00|24.00
2000-04-19 |11002|6|1.00|6.00
2012-05-21 |11002|8|1.00|8.00
2004-12-31|11003|2|3.00|6.00
RESULT:
2000-04-19 |11002|6|1.00|6.00
2012-05-21 |11002|8|1.00|8.00
UDF to obtain the quarter of an year if a date or timestamp is given .
--
Key: HIVE-3404
URL: https://issues.apache.org/jira/browse/HIVE-3404
Project: Hive
Issue Type: New Feature
Components: UDF
Reporter: Sanam Naz
Hive current releases lacks a function which returns the quarter of an year
if a date or timestamp is given .The function QUARTER(date) would return the
quarter from a date / timestamp .This can be used in HiveQL.This will be
useful for different domains like retail ,finance etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3378) UDF to obtain the numeric day of an year from date or timestamp in HIVE.
[ https://issues.apache.org/jira/browse/HIVE-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepti Antony updated HIVE-3378: Status: Patch Available (was: Open) Patch for the UDF to find numeric day of year has been attached for review. UDF to obtain the numeric day of an year from date or timestamp in HIVE. Key: HIVE-3378 URL: https://issues.apache.org/jira/browse/HIVE-3378 Project: Hive Issue Type: New Feature Components: UDF Affects Versions: 0.9.0, 0.8.1 Reporter: Deepti Antony Attachments: HIVE-3378.1.patch.txt Hive current releases lacks a function which returns the numeric day of an year if a date or timestamp is given .The function DAYOFYEAR(date) would return the numeric day from a date / timestamp or which would be useful while using HiveQL.DAYOFYEAR can be used to compare data with respect to number of days till the given date.It can be used in different domains. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3240) Fix non-deterministic results in newline.q and timestamp_lazy.q
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440165#comment-13440165 ] Carl Steinbach commented on HIVE-3240: -- +1. Will commit if tests pass. Fix non-deterministic results in newline.q and timestamp_lazy.q --- Key: HIVE-3240 URL: https://issues.apache.org/jira/browse/HIVE-3240 Project: Hive Issue Type: Bug Components: Tests Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Attachments: HIVE-3240.1.patch.txt, HIVE-3240.2.patch.txt, Windows_alter1.q.out.patch.txt newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-3072 started by Gang Tim Liu. Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3072: --- Attachment: HIVE-3072.patch.5 Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3072: --- Status: Patch Available (was: In Progress) Patch is ready for review. It's in both jira and phabricator. Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #114
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/ -- [...truncated 10116 lines...] [echo] Project: odbc [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/odbc/src/conf does not exist. ivy-resolve-test: [echo] Project: odbc ivy-retrieve-test: [echo] Project: odbc compile-test: [echo] Project: odbc create-dirs: [echo] Project: serde [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/serde/src/test/resources does not exist. init: [echo] Project: serde ivy-init-settings: [echo] Project: serde ivy-resolve: [echo] Project: serde [ivy:resolve] :: loading settings :: file = https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml [ivy:report] Processing https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-serde-default.xml to https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/artifact/hive/build/ivy/report/org.apache.hive-hive-serde-default.html ivy-retrieve: [echo] Project: serde dynamic-serde: compile: [echo] Project: serde ivy-resolve-test: [echo] Project: serde ivy-retrieve-test: [echo] Project: serde compile-test: [echo] Project: serde [javac] Compiling 26 source files to https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/artifact/hive/build/serde/test/classes [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. create-dirs: [echo] Project: service [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/service/src/test/resources does not exist. init: [echo] Project: service ivy-init-settings: [echo] Project: service ivy-resolve: [echo] Project: service [ivy:resolve] :: loading settings :: file = https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml [ivy:report] Processing https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml to https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/artifact/hive/build/ivy/report/org.apache.hive-hive-service-default.html ivy-retrieve: [echo] Project: service compile: [echo] Project: service ivy-resolve-test: [echo] Project: service ivy-retrieve-test: [echo] Project: service compile-test: [echo] Project: service [javac] Compiling 2 source files to https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/artifact/hive/build/service/test/classes test: [echo] Project: hive test-shims: [echo] Project: hive test-conditions: [echo] Project: shims gen-test: [echo] Project: shims create-dirs: [echo] Project: shims [copy] Warning: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/test/resources does not exist. init: [echo] Project: shims ivy-init-settings: [echo] Project: shims ivy-resolve: [echo] Project: shims [ivy:resolve] :: loading settings :: file = https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml [ivy:report] Processing https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml to https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/artifact/hive/build/ivy/report/org.apache.hive-hive-shims-default.html ivy-retrieve: [echo] Project: shims compile: [echo] Project: shims [echo] Building shims 0.20 build_shims: [echo] Project: shims [echo] Compiling https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/common/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java against hadoop 0.20.2 (https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/114/artifact/hive/build/hadoopcore/hadoop-0.20.2) ivy-init-settings: [echo] Project: shims ivy-resolve-hadoop-shim: [echo] Project: shims [ivy:resolve] :: loading settings :: file = https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml ivy-retrieve-hadoop-shim: [echo] Project: shims [echo] Building shims 0.20S build_shims: [echo] Project: shims [echo] Compiling
Hive-trunk-h0.21 - Build # 1622 - Still Failing
Changes for Build #1619 [cws] HIVE-3242. Fix cascade_dbdrop.q when building hive on hadoop0.23 (Zhenxiao Luo via cws) Changes for Build #1620 Changes for Build #1621 Changes for Build #1622 No tests ran. The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1622) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1622/ to view the results.
[jira] [Updated] (HIVE-3086) Skewed Join Optimization
[ https://issues.apache.org/jira/browse/HIVE-3086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nadeem Moidu updated HIVE-3086: --- Assignee: (was: Nadeem Moidu) Skewed Join Optimization Key: HIVE-3086 URL: https://issues.apache.org/jira/browse/HIVE-3086 Project: Hive Issue Type: New Feature Reporter: Nadeem Moidu During a join operation, if one of the columns has a skewed key, it can cause that particular reducer to become the bottleneck. The following feature will address it: https://cwiki.apache.org/confluence/display/Hive/Skewed+Join+Optimization -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440461#comment-13440461 ] Namit Jain commented on HIVE-3072: -- some minor comments. Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3072: - Status: Open (was: Patch Available) I had some minor comments on the patch. Otherwise, it looks good to me. @Carl, do you have any additional comments ? Otherwise, I will start testing once Tim has addressed the new comments. Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3171) Bucketed sort merge join doesn't work when multiple files exist for small alias
[
https://issues.apache.org/jira/browse/HIVE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440508#comment-13440508
]
Carl Steinbach commented on HIVE-3171:
--
+1. Will commit if tests pass.
Bucketed sort merge join doesn't work when multiple files exist for small
alias
---
Key: HIVE-3171
URL: https://issues.apache.org/jira/browse/HIVE-3171
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Joey Echeverria
Assignee: Navis
Labels: bucketing, joins, partitioning
Executing a query with the MAPJOIN hint and the bucketed sort merge join
optimizations enabled:
{noformat}
set hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
set hive.optimize.bucketmapjoin = true;
set hive.optimize.bucketmapjoin.sortedmerge = true;
{noformat}
works fine with partitioned tables if there is only one partition in the
table. However, if you add a second partition, Hive attempts to do a regular
map-side join which can fail because the tables are too large. Hive ought to
be able to still do the bucketed sort merge join with partitions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3056) Ability to bulk update location field in Db/Table/Partition records
[ https://issues.apache.org/jira/browse/HIVE-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shreepadma Venugopalan updated HIVE-3056: - Status: Patch Available (was: Open) Ability to bulk update location field in Db/Table/Partition records --- Key: HIVE-3056 URL: https://issues.apache.org/jira/browse/HIVE-3056 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Shreepadma Venugopalan Attachments: HIVE-3056.2.patch.txt, HIVE-3056.3.patch.txt, HIVE-3056.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3056) Ability to bulk update location field in Db/Table/Partition records
[ https://issues.apache.org/jira/browse/HIVE-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shreepadma Venugopalan updated HIVE-3056: - Attachment: HIVE-3056.3.patch.txt Ability to bulk update location field in Db/Table/Partition records --- Key: HIVE-3056 URL: https://issues.apache.org/jira/browse/HIVE-3056 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Shreepadma Venugopalan Attachments: HIVE-3056.2.patch.txt, HIVE-3056.3.patch.txt, HIVE-3056.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-3056: Ability to bulk update location field in Db/Table/Partition records
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6650/ --- (Updated Aug. 23, 2012, 8:19 p.m.) Review request for hive and Carl Steinbach. Changes --- Updated patch to remove TestHiveMetaTool.launch file with hard coded paths. The new patch contains TestHiveMetaTool.launchtemplate file with relative paths. Description --- This patch implement hive metatool which, * lets admins perform a HA upgrade by patching the location of the NN in Hive's metastore * allows JDOQL to be executed against the metastore. Additionally this patch also upgrades DN to 3.0 This addresses bug HIVE-3056. https://issues.apache.org/jira/browse/HIVE-3056 Diffs (updated) - bin/ext/metatool.sh PRE-CREATION bin/metatool PRE-CREATION build.xml 6712af9 conf/hive-default.xml.template cc1565f eclipse-templates/TestHiveMetaTool.launchtemplate PRE-CREATION metastore/ivy.xml 3011d2f metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 6f4716c metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java PRE-CREATION metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java PRE-CREATION Diff: https://reviews.apache.org/r/6650/diff/ Testing --- Manual testing for various metatool options. Thanks, Shreepadma Venugopalan
[jira] [Commented] (HIVE-3056) Ability to bulk update location field in Db/Table/Partition records
[ https://issues.apache.org/jira/browse/HIVE-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440616#comment-13440616 ] Shreepadma Venugopalan commented on HIVE-3056: -- Updated patch is available both on this page as well as https://reviews.apache.org/r/6650/diff/ Ability to bulk update location field in Db/Table/Partition records --- Key: HIVE-3056 URL: https://issues.apache.org/jira/browse/HIVE-3056 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Shreepadma Venugopalan Attachments: HIVE-3056.2.patch.txt, HIVE-3056.3.patch.txt, HIVE-3056.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3171) Bucketed sort merge join doesn't work when multiple files exist for small alias
[
https://issues.apache.org/jira/browse/HIVE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440629#comment-13440629
]
Carl Steinbach commented on HIVE-3171:
--
@Navis: Can you please attach the most recent version of the patch to this
ticket? Thanks.
Bucketed sort merge join doesn't work when multiple files exist for small
alias
---
Key: HIVE-3171
URL: https://issues.apache.org/jira/browse/HIVE-3171
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Joey Echeverria
Assignee: Navis
Labels: bucketing, joins, partitioning
Executing a query with the MAPJOIN hint and the bucketed sort merge join
optimizations enabled:
{noformat}
set hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
set hive.optimize.bucketmapjoin = true;
set hive.optimize.bucketmapjoin.sortedmerge = true;
{noformat}
works fine with partitioned tables if there is only one partition in the
table. However, if you add a second partition, Hive attempts to do a regular
map-side join which can fail because the tables are too large. Hive ought to
be able to still do the bucketed sort merge join with partitions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440644#comment-13440644 ] Carl Steinbach commented on HIVE-3072: -- @Namit: I'll make another pass through the patch later today. One thing I'd like to request is that we add an internal configuration property that disables the new DDL by default. We can remove this once the rest of the DML changes get committed, but in the meantime I don't think it makes sense to make the DDL visible to users. So to recap, I'm proposing the following: * Add a configuration property named hive.internal.ddl.list.bucketing.enable and set the default value to false. * Add a comment in HiveConf explaining that this will be removed once the rest of the DML changes are committed. * Do *not* add this property to hive-default.xml.template since we don't want users messing with it. * Throw an error if the user tries to use the DDL with hive.internal.ddl.list.bucketing.enable set to false. Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #114
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/114/ -- [...truncated 36554 lines...] [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/jenkins/hive_2012-08-23_13-52-38_997_8536776089834202555/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/114/artifact/hive/build/service/tmp/hive_job_log_jenkins_201208231352_501873261.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Copying file: https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] PREHOOK: query: load data local inpath 'https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Output: default@testhivedrivertable [junit] Copying data from https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/jenkins/hive_2012-08-23_13-52-43_321_6539443442351992831/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/jenkins/hive_2012-08-23_13-52-43_321_6539443442351992831/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/114/artifact/hive/build/service/tmp/hive_job_log_jenkins_201208231352_2015633741.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/114/artifact/hive/build/service/tmp/hive_job_log_jenkins_201208231352_1381510832.txt [junit] Hive history file=https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/114/artifact/hive/build/service/tmp/hive_job_log_jenkins_201208231352_1125384816.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit]
[jira] [Commented] (HIVE-3323) Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440666#comment-13440666
]
Feng Peng commented on HIVE-3323:
-
Bump.
Is it possible some committer to take a look at this patch? A few of our later
patches depend on it and it'd be great if we can get some feedback. Thanks!
Enable enum to string conversions
-
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-775) Hive to support Set type
[ https://issues.apache.org/jira/browse/HIVE-775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440684#comment-13440684 ] Feng Peng commented on HIVE-775: Finished first version of the patch that covers the read path. Tested using our thrift data with sets and it worked fine. We'd like to use the complex thrift struct added in https://issues.apache.org/jira/browse/HIVE-3323 for the unit tests. It'd be nice if someone take a look at https://issues.apache.org/jira/browse/HIVE-3323 first. Thanks! Hive to support Set type -- Key: HIVE-775 URL: https://issues.apache.org/jira/browse/HIVE-775 Project: Hive Issue Type: Bug Affects Versions: 0.5.0 Reporter: Zheng Shao Thrift supports a type set which maps to java.util.Set. Hive should support this as well. We can either add set to array, struct, map, or we can just treat set as an array which will be easier to do. This is blocking Hive from doing SELECT * on some tables defined by thrift, because thrift supports set right now. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HIVE-3323:
--
Summary: ThriftSerde: Enable enum to string conversions (was: Enable enum
to string conversions)
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440713#comment-13440713
]
Jakob Homan commented on HIVE-3323:
---
One comment I have:
{noformat}
+CONVERT_ENUM_TO_STRING(hive.data.convert.enum.to.string, false),
{noformat}
since AvroSerde already does this and doesn't provide an option not to, can we
change option name to be thrift specific?
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440719#comment-13440719
]
Dmitriy V. Ryaboy commented on HIVE-3323:
-
Jakob, that sounds like something to fix in AvroSerde.
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440722#comment-13440722
]
Dmitriy V. Ryaboy commented on HIVE-3323:
-
Sorry, let me try a less snide reply :)
It doesn't make sense to me to have a situation where you are working with
Thrift and Avro, and you want to not convert from Thrift but you do want to
convert from Avro. What you really want is establish how Hive does serde type
conversions, not how it does them for individual encodings of data.
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440724#comment-13440724
]
Jakob Homan commented on HIVE-3323:
---
Right, but AvroSerde already does this conversion is what I'm saying. There's
never been an option not to do the conversion.
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440739#comment-13440739
]
Dmitriy V. Ryaboy commented on HIVE-3323:
-
How about if we (as in, me/feng/travis) make a patch for AvroSerde to respect
this property?
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440741#comment-13440741
]
Jakob Homan commented on HIVE-3323:
---
Is the current Thrift behavior worth keeping around? Maybe just convert it to
just do the string conversion? I can't come up with a use case where I would
want the struct Thrift provides.
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440748#comment-13440748
]
Dmitriy V. Ryaboy commented on HIVE-3323:
-
We are thinking of a situation where you are going through HCat but consuming
in something that speaks structs, your own service.
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-3056: Ability to bulk update location field in Db/Table/Partition records
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6650/ --- (Updated Aug. 23, 2012, 11:49 p.m.) Review request for hive and Carl Steinbach. Description (updated) --- This patch implement hive metatool which, * lets admins perform a HA upgrade by patching the location of the NN in Hive's metastore * allows JDOQL to be executed against the metastore. This addresses bug HIVE-3056. https://issues.apache.org/jira/browse/HIVE-3056 Diffs - bin/ext/metatool.sh PRE-CREATION bin/metatool PRE-CREATION build.xml 6712af9 conf/hive-default.xml.template cc1565f eclipse-templates/TestHiveMetaTool.launchtemplate PRE-CREATION metastore/ivy.xml 3011d2f metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 6f4716c metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java PRE-CREATION metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java PRE-CREATION Diff: https://reviews.apache.org/r/6650/diff/ Testing (updated) --- A new JUnit test - TestHiveMetaTool - has been added to test the various metatool options. Thanks, Shreepadma Venugopalan
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440821#comment-13440821
]
Jakob Homan commented on HIVE-3323:
---
OK, but that sounds like a less frequently useful use case than converting to
strings. Should we make the default behavior convert-to-string and add
convert-to-struct as an option for thrift and avro?
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440838#comment-13440838
]
Dmitriy V. Ryaboy commented on HIVE-3323:
-
I am ok with making the default true. Don't think that would break anything.
Feng?
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3366) Upgrade Hive's Avro dependency to version 1.7
[ https://issues.apache.org/jira/browse/HIVE-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-3366. -- Resolution: Duplicate Upgrade Hive's Avro dependency to version 1.7 - Key: HIVE-3366 URL: https://issues.apache.org/jira/browse/HIVE-3366 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3367) Upgrade Hive's Avro dependency to version 1.7
[ https://issues.apache.org/jira/browse/HIVE-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-3367. -- Resolution: Duplicate Upgrade Hive's Avro dependency to version 1.7 - Key: HIVE-3367 URL: https://issues.apache.org/jira/browse/HIVE-3367 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach a -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3368) Upgrade Hive's Avro dependency to version 1.7
[ https://issues.apache.org/jira/browse/HIVE-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-3368. -- Resolution: Duplicate Upgrade Hive's Avro dependency to version 1.7 - Key: HIVE-3368 URL: https://issues.apache.org/jira/browse/HIVE-3368 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions
[
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440844#comment-13440844
]
Feng Peng commented on HIVE-3323:
-
I think it is fine. The current behavior for thrift is false and that's why we
set it to false in the current patch. But I don't think people are using SerDe
to read complex thrift data, otherwise they would have complained about this
(and other problems we are having right now) already.
ThriftSerde: Enable enum to string conversions
--
Key: HIVE-3323
URL: https://issues.apache.org/jira/browse/HIVE-3323
Project: Hive
Issue Type: Improvement
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3323_enum_to_string.1.patch,
HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch,
HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
When using serde-reported schemas with the ThriftDeserializer, Enum fields
are presented as {{structvalue:int}}
Many users expect to work with the string values, which is both easier and
more meaningful as the string value communicates what is represented.
Hive should provide a mechanism to optionally convert enum values to strings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-3072 started by Gang Tim Liu. Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5, HIVE-3072.patch.6 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440845#comment-13440845 ] Gang Tim Liu commented on HIVE-3072: Patch is available on both jara and phabricator. Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5, HIVE-3072.patch.6 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3072: --- Attachment: HIVE-3072.patch.6 Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5, HIVE-3072.patch.6 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3171) Bucketed sort merge join doesn't work when multiple files exist for small alias
[
https://issues.apache.org/jira/browse/HIVE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-3171:
Attachment: HIVE-3171.1.patch.txt
Passed all tests
Bucketed sort merge join doesn't work when multiple files exist for small
alias
---
Key: HIVE-3171
URL: https://issues.apache.org/jira/browse/HIVE-3171
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Joey Echeverria
Assignee: Navis
Labels: bucketing, joins, partitioning
Attachments: HIVE-3171.1.patch.txt
Executing a query with the MAPJOIN hint and the bucketed sort merge join
optimizations enabled:
{noformat}
set hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
set hive.optimize.bucketmapjoin = true;
set hive.optimize.bucketmapjoin.sortedmerge = true;
{noformat}
works fine with partitioned tables if there is only one partition in the
table. However, if you add a second partition, Hive attempts to do a regular
map-side join which can fail because the tables are too large. Hive ought to
be able to still do the bucketed sort merge join with partitions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1173) Partition pruner cancels pruning if non-deterministic function present in filtering expression only in joins is present in query
[
https://issues.apache.org/jira/browse/HIVE-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-1173:
Status: Patch Available (was: Open)
Passed all tests
Partition pruner cancels pruning if non-deterministic function present in
filtering expression only in joins is present in query
Key: HIVE-1173
URL: https://issues.apache.org/jira/browse/HIVE-1173
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.4.1, 0.4.0, 0.10.0
Reporter: Vladimir Klimontovich
Assignee: Navis
Brief description:
case 1) non-deterministic present in partition condition, joins are present
in query = partition pruner doesn't do filtering of partitions based on
condition
case 2) non-deterministic present in partition condition, joins aren't
present in query = partition pruner do filtering of partitions based on
condition
It's quite illogical when pruning depends on presence of joins in query.
Example:
Let's consider following sequence of hive queries:
1) Create non-deterministic function:
create temporary function UDF2 as 'UDF2';
{{
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.hive.ql.udf.UDFType;
@UDFType(deterministic=false)
public class UDF2 extends UDF {
public String evaluate(String val) {
return val;
}
}
}}
2) Create tables
CREATE TABLE Main (
a STRING,
b INT
)
PARTITIONED BY(part STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '10'
STORED AS TEXTFILE;
ALTER TABLE Main ADD PARTITION (part=part1) LOCATION
/hive-join-test/part1/;
ALTER TABLE Main ADD PARTITION (part=part2) LOCATION
/hive-join-test/part2/;
CREATE TABLE Joined (
a STRING,
f STRING
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '10'
STORED AS TEXTFILE
LOCATION '/hive-join-test/join/';
3) Run first query:
select
m.a,
m.b
from Main m
where
part UDF2('part0') AND part = 'part1';
The pruner will work for this query:
mapred.input.dir=hdfs://localhost:9000/hive-join-test/part1
4) Run second query (with join):
select
m.a,
j.a,
m.b
from Main m
join Joined j on
j.a=m.a
where
part UDF2('part0') AND part = 'part1';
Pruner doesn't work:
mapred.input.dir=hdfs://localhost:9000/hive-join-test/part1,hdfs://localhost:9000/hive-join-test/part2,hdfs://localhost:9000/hive-join-test/join
5) Also lets try to run query with MAPJOIN hint
select /*+MAPJOIN(j)*/
m.a,
j.a,
m.b
from Main m
join Joined j on
j.a=m.a
where
part UDF2('part0') AND part = 'part1';
The result is the same, pruner doesn't work:
mapred.input.dir=hdfs://localhost:9000/hive-join-test/part1,hdfs://localhost:9000/hive-join-test/part2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3171) Bucketed sort merge join doesn't work when multiple files exist for small alias
[
https://issues.apache.org/jira/browse/HIVE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440869#comment-13440869
]
Carl Steinbach commented on HIVE-3171:
--
@Navis: Since the tests passed can you please commit this yourself? Thanks.
Bucketed sort merge join doesn't work when multiple files exist for small
alias
---
Key: HIVE-3171
URL: https://issues.apache.org/jira/browse/HIVE-3171
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Joey Echeverria
Assignee: Navis
Labels: bucketing, joins, partitioning
Attachments: HIVE-3171.1.patch.txt
Executing a query with the MAPJOIN hint and the bucketed sort merge join
optimizations enabled:
{noformat}
set hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
set hive.optimize.bucketmapjoin = true;
set hive.optimize.bucketmapjoin.sortedmerge = true;
{noformat}
works fine with partitioned tables if there is only one partition in the
table. However, if you add a second partition, Hive attempts to do a regular
map-side join which can fail because the tables are too large. Hive ought to
be able to still do the bucketed sort merge join with partitions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440875#comment-13440875 ] Carl Steinbach commented on HIVE-3072: -- @Tim: Can you please add the configuration property to disable this DDL? Thanks. Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5, HIVE-3072.patch.6 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive CLI wiki -- draft ready to go
It's been a month without further comments, so I'm planning to swap out the
current CLI wikidoc next week, replacing it with this draft:
https://cwiki.apache.org/confluence/display/Hive/Revised+LanguageManual+Cli
Thanks to Sushanth Sowmyan for reviewing the draft. Additional reviews are
welcome.
I'll make the swap on Thursday, August 30th.
– Lefty Leverenz
-- Forwarded message --
From: Lefty Leverenz le...@hortonworks.com
Date: Tue, Jul 24, 2012 at 7:01 PM
Subject: Re: Hive CLI wikidoc review
To: dev@hive.apache.org
Okay, we can get rid of the xdoc but what about this information from it?
The cli when invoked without the -i option will attempt to load
HIVE_HOME/bin/.hiverc and $HOME/.hiverc as initialization files.
Is it still useful, or should I delete it from the draft?
– Lefty
On Tue, Jul 24, 2012 at 6:44 PM, Edward Capriolo edlinuxg...@gmail.comwrote:
The xdoc should be removed its dead and out of sync.
On Tue, Jul 24, 2012 at 7:21 PM, Lefty Leverenz le...@hortonworks.com
wrote:
The Hive CLI
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Cli
wikidoc
differs slightly from the xdocs
versionhttp://hive.apache.org/docs/r0.9.0/language_manual/cli.html
(source-controlled
xml file) and they both need updating, so I merged them into a single
wikidoc for review by this community. You can find the draft here:
Revised
LanguageManual CLI
https://cwiki.apache.org/confluence/display/Hive/Revised+LanguageManual+Cli
.
After the wikidoc has been reviewed and fixed, I'll apply the same
changes
to the xdocs version. Eventually only one copy should remain, but
that's a
separate issue. I'll be revising more wikidocs as time permits; the CLI
doc is first because diverging copies can cause trouble.
Besides a simple merge, this draft includes technical changes based on
Thejas Nair's review comments as well as some editorial changes. Please
let me know (or edit the draft) if you find any errors or want any
changes.
If more information is available, please provide it.
Q1: In the Hive Resources section is the syntax for DELETE FILE | JAR
|
ARCHIVE correct?
DELETE { FILE[S] | JAR[S] | ARCHIVE[S] } [filepath1 filepath2 ..]
Q2: If DELETE does not require a filepath, what's the result – deleting
*
all* files | jars | archives?
– Lefty Leverenz, *technical writer on a mission to improve Hive docs*
[jira] [Commented] (HIVE-3068) Add ability to export table metadata as JSON on table drop
[ https://issues.apache.org/jira/browse/HIVE-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440877#comment-13440877 ] Edward Capriolo commented on HIVE-3068: --- Andrew. It is ok this patch took a little longer then it should have to get reviewed. I applied arc diff https://reviews.facebook.net/D4341 You are missing the changes to the HiveConf.java and to the hive-site.xml.template make these changes and I will review. Add ability to export table metadata as JSON on table drop -- Key: HIVE-3068 URL: https://issues.apache.org/jira/browse/HIVE-3068 Project: Hive Issue Type: New Feature Components: Metastore, Serializers/Deserializers Reporter: Andrew Chalfant Assignee: Andrew Chalfant Priority: Minor Labels: features, newbie Attachments: HIVE-3068.2.patch.txt Original Estimate: 24h Remaining Estimate: 24h When a table is dropped, the contents go to the users trash but the metadata is lost. It would be super neat to be able to save the metadata as well so that tables could be trivially re-instantiated via thrift. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3072) Hive List Bucketing - DDL support
[ https://issues.apache.org/jira/browse/HIVE-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440881#comment-13440881 ] Gang Tim Liu commented on HIVE-3072: @Carl, yes , will code after dinner. Thanks a lot Hive List Bucketing - DDL support - Key: HIVE-3072 URL: https://issues.apache.org/jira/browse/HIVE-3072 Project: Hive Issue Type: New Feature Components: SQL Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3072.patch, HIVE-3072.patch.1, HIVE-3072.patch.2, HIVE-3072.patch.3, HIVE-3072.patch.4, HIVE-3072.patch.5, HIVE-3072.patch.6 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DDL change for the feature. It's for both single skewed column and multiple columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3171) Bucketed sort merge join doesn't work when multiple files exist for small alias
[
https://issues.apache.org/jira/browse/HIVE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440886#comment-13440886
]
Navis commented on HIVE-3171:
-
@Carl: I've not yet received account for ASF, and is it ok to commit by myself?
(I thought that the author and the committer should be different)
Bucketed sort merge join doesn't work when multiple files exist for small
alias
---
Key: HIVE-3171
URL: https://issues.apache.org/jira/browse/HIVE-3171
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Joey Echeverria
Assignee: Navis
Labels: bucketing, joins, partitioning
Attachments: HIVE-3171.1.patch.txt
Executing a query with the MAPJOIN hint and the bucketed sort merge join
optimizations enabled:
{noformat}
set hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
set hive.optimize.bucketmapjoin = true;
set hive.optimize.bucketmapjoin.sortedmerge = true;
{noformat}
works fine with partitioned tables if there is only one partition in the
table. However, if you add a second partition, Hive attempts to do a regular
map-side join which can fail because the tables are too large. Hive ought to
be able to still do the bucketed sort merge join with partitions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3365) Upgrade Hive's Avro dependency to version 1.7
[ https://issues.apache.org/jira/browse/HIVE-3365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shreepadma Venugopalan reassigned HIVE-3365: Assignee: Shreepadma Venugopalan (was: Carl Steinbach) Upgrade Hive's Avro dependency to version 1.7 - Key: HIVE-3365 URL: https://issues.apache.org/jira/browse/HIVE-3365 Project: Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Assignee: Shreepadma Venugopalan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3171) Bucketed sort merge join doesn't work when multiple files exist for small alias
[
https://issues.apache.org/jira/browse/HIVE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440920#comment-13440920
]
Namit Jain commented on HIVE-3171:
--
@Carl, @Navis, we mostly don't commit our patches. There have been a few
exceptions, but I think
we should try to stick to this policy: dont commit our own patches.
Also, can you hold off for a few hours - I wanted to take a pass at this patch.
Should be done today itself.
Bucketed sort merge join doesn't work when multiple files exist for small
alias
---
Key: HIVE-3171
URL: https://issues.apache.org/jira/browse/HIVE-3171
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Joey Echeverria
Assignee: Navis
Labels: bucketing, joins, partitioning
Attachments: HIVE-3171.1.patch.txt
Executing a query with the MAPJOIN hint and the bucketed sort merge join
optimizations enabled:
{noformat}
set hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
set hive.optimize.bucketmapjoin = true;
set hive.optimize.bucketmapjoin.sortedmerge = true;
{noformat}
works fine with partitioned tables if there is only one partition in the
table. However, if you add a second partition, Hive attempts to do a regular
map-side join which can fail because the tables are too large. Hive ought to
be able to still do the bucketed sort merge join with partitions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3406) Yet better error message in CLI on invalid column name
Navis created HIVE-3406:
---
Summary: Yet better error message in CLI on invalid column name
Key: HIVE-3406
URL: https://issues.apache.org/jira/browse/HIVE-3406
Project: Hive
Issue Type: Improvement
Components: Diagnosability
Affects Versions: 0.10.0
Reporter: Navis
Assignee: Navis
Priority: Trivial
HIVE-2256 appended column names to error message for invalid column reference,
but it's not alias by which a column can be referenced. For example query in
clustern4.q (negative)
{code}
SELECT x.key as k1, x.value FROM SRC x CLUSTER BY key;
{code}
makes exception with error message,
{code}
FAILED: SemanticException [Error 10004]: Line 2:50 Invalid table alias or
column reference 'key': (possible column names are: _col0, _col1)
{code}
But replacing 'key' with '_col0' or '_col1' does not make this query work. The
error message should be,
{code}
FAILED: SemanticException [Error 10004]: Line 2:50 Invalid table alias or
column reference 'key': (possible column names are: k1, x.value)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3406) Yet better error message in CLI on invalid column name
[
https://issues.apache.org/jira/browse/HIVE-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-3406:
Status: Patch Available (was: Open)
https://reviews.facebook.net/D4869
Yet better error message in CLI on invalid column name
--
Key: HIVE-3406
URL: https://issues.apache.org/jira/browse/HIVE-3406
Project: Hive
Issue Type: Improvement
Components: Diagnosability
Affects Versions: 0.10.0
Reporter: Navis
Assignee: Navis
Priority: Trivial
HIVE-2256 appended column names to error message for invalid column
reference, but it's not alias by which a column can be referenced. For
example query in clustern4.q (negative)
{code}
SELECT x.key as k1, x.value FROM SRC x CLUSTER BY key;
{code}
makes exception with error message,
{code}
FAILED: SemanticException [Error 10004]: Line 2:50 Invalid table alias or
column reference 'key': (possible column names are: _col0, _col1)
{code}
But replacing 'key' with '_col0' or '_col1' does not make this query work.
The error message should be,
{code}
FAILED: SemanticException [Error 10004]: Line 2:50 Invalid table alias or
column reference 'key': (possible column names are: k1, x.value)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3406) Yet better error message in CLI on invalid column name
[
https://issues.apache.org/jira/browse/HIVE-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-3406:
Attachment: HIVE-3406.1.patch.txt
Yet better error message in CLI on invalid column name
--
Key: HIVE-3406
URL: https://issues.apache.org/jira/browse/HIVE-3406
Project: Hive
Issue Type: Improvement
Components: Diagnosability
Affects Versions: 0.10.0
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-3406.1.patch.txt
HIVE-2256 appended column names to error message for invalid column
reference, but it's not alias by which a column can be referenced. For
example query in clustern4.q (negative)
{code}
SELECT x.key as k1, x.value FROM SRC x CLUSTER BY key;
{code}
makes exception with error message,
{code}
FAILED: SemanticException [Error 10004]: Line 2:50 Invalid table alias or
column reference 'key': (possible column names are: _col0, _col1)
{code}
But replacing 'key' with '_col0' or '_col1' does not make this query work.
The error message should be,
{code}
FAILED: SemanticException [Error 10004]: Line 2:50 Invalid table alias or
column reference 'key': (possible column names are: k1, x.value)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3171) Bucketed sort merge join doesn't work when multiple files exist for small alias
[
https://issues.apache.org/jira/browse/HIVE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain updated HIVE-3171:
-
Status: Open (was: Patch Available)
some small comments on phabricator
Bucketed sort merge join doesn't work when multiple files exist for small
alias
---
Key: HIVE-3171
URL: https://issues.apache.org/jira/browse/HIVE-3171
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Joey Echeverria
Assignee: Navis
Labels: bucketing, joins, partitioning
Attachments: HIVE-3171.1.patch.txt
Executing a query with the MAPJOIN hint and the bucketed sort merge join
optimizations enabled:
{noformat}
set hive.input.format=org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
set hive.optimize.bucketmapjoin = true;
set hive.optimize.bucketmapjoin.sortedmerge = true;
{noformat}
works fine with partitioned tables if there is only one partition in the
table. However, if you add a second partition, Hive attempts to do a regular
map-side join which can fail because the tables are too large. Hive ought to
be able to still do the bucketed sort merge join with partitions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3226) ColumnPruner is not working on LateralView
[
https://issues.apache.org/jira/browse/HIVE-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440969#comment-13440969
]
Namit Jain commented on HIVE-3226:
--
This one slipped. Running tests again
ColumnPruner is not working on LateralView
--
Key: HIVE-3226
URL: https://issues.apache.org/jira/browse/HIVE-3226
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: Navis
Assignee: Navis
Attachments: HIVE-3226.1.patch.txt, HIVE-3226.2.patch.txt
Column pruning is not applied to LVJ and SEL operator, which makes exceptions
at various stages. For example,
{noformat}
drop table array_valued_src;
create table array_valued_src (key string, value arraystring);
insert overwrite table array_valued_src select key, array(value) from src;
select sum(val) from (select a.key as key, b.value as array_val from src a
join array_valued_src b on a.key=b.key) i lateral view explode (array_val) c
as val;
... 9 more
Caused by: java.lang.RuntimeException: Reduce operator initialization failed
at
org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:157)
... 14 more
Caused by: java.lang.RuntimeException: cannot find field _col0 from [0:_col5]
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:345)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:143)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:896)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:922)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
at
org.apache.hadoop.hive.ql.exec.JoinOperator.initializeOp(JoinOperator.java:62)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at
org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:150)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3406) Yet better error message in CLI on invalid column name
[
https://issues.apache.org/jira/browse/HIVE-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440971#comment-13440971
]
Namit Jain commented on HIVE-3406:
--
+1
Yet better error message in CLI on invalid column name
--
Key: HIVE-3406
URL: https://issues.apache.org/jira/browse/HIVE-3406
Project: Hive
Issue Type: Improvement
Components: Diagnosability
Affects Versions: 0.10.0
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-3406.1.patch.txt
HIVE-2256 appended column names to error message for invalid column
reference, but it's not alias by which a column can be referenced. For
example query in clustern4.q (negative)
{code}
SELECT x.key as k1, x.value FROM SRC x CLUSTER BY key;
{code}
makes exception with error message,
{code}
FAILED: SemanticException [Error 10004]: Line 2:50 Invalid table alias or
column reference 'key': (possible column names are: _col0, _col1)
{code}
But replacing 'key' with '_col0' or '_col1' does not make this query work.
The error message should be,
{code}
FAILED: SemanticException [Error 10004]: Line 2:50 Invalid table alias or
column reference 'key': (possible column names are: k1, x.value)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
