[jira] [Updated] (HIVE-3694) Generate test jars and publish them to Maven
[ https://issues.apache.org/jira/browse/HIVE-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3694: -- Attachment: D6843.2.patch mbautin updated the revision [jira] [HIVE-3694] Generate hive-exec-test jar and publish it to Maven locally. Reviewers: ashutoshc, JIRA Proper error handling in add_dependency_to_pom.py REVISION DETAIL https://reviews.facebook.net/D6843 AFFECTED FILES .gitignore bin/build/add_dependency_to_pom.py build-common.xml build.xml metastore/build.xml ql/build.xml To: ashutoshc, JIRA, mbautin Generate test jars and publish them to Maven Key: HIVE-3694 URL: https://issues.apache.org/jira/browse/HIVE-3694 Project: Hive Issue Type: Improvement Components: Build Infrastructure Reporter: Mikhail Bautin Priority: Minor Attachments: D6843.1.patch, D6843.2.patch It should be possible to generate Hive test jars and publish them to Maven so that other projects that rely on Hive or extend it could reuse its test library. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501780#comment-13501780 ] Shengsheng Huang commented on HIVE-3384: Mikhail Bautin, thank you very much for your work. I failed to login Phabricator because of some weird authentication problem and still working on that. Really appreciate what you've done. I'd very much like to see this to be checked into trunk as soon. HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Reporter: Weidong Bian Priority: Minor Attachments: D6873.1.patch, HIVE-3384.2.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3729) Error in groupSetExpression rule in Hive grammar
Harish Butani created HIVE-3729: --- Summary: Error in groupSetExpression rule in Hive grammar Key: HIVE-3729 URL: https://issues.apache.org/jira/browse/HIVE-3729 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Environment: All Reporter: Harish Butani Priority: Minor Fix For: 0.10.0 Here is the error: Hive.g:1902:38: reference to rewrite element groupByExpression without reference on left of - -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3729) Error in groupSetExpression rule in Hive grammar
[ https://issues.apache.org/jira/browse/HIVE-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-3729: Attachment: HIVE-3729.1.patch.txt Error in groupSetExpression rule in Hive grammar Key: HIVE-3729 URL: https://issues.apache.org/jira/browse/HIVE-3729 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Environment: All Reporter: Harish Butani Priority: Minor Fix For: 0.10.0 Attachments: HIVE-3729.1.patch.txt Original Estimate: 5m Remaining Estimate: 5m Here is the error: Hive.g:1902:38: reference to rewrite element groupByExpression without reference on left of - -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #205
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/205/ -- [...truncated 10533 lines...] compile-test: [echo] Project: serde [javac] Compiling 26 source files to /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/serde/test/classes [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. create-dirs: [echo] Project: service [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/service/src/test/resources does not exist. init: [echo] Project: service ivy-init-settings: [echo] Project: service ivy-resolve: [echo] Project: service [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml [ivy:report] Processing /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml to /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-service-default.html ivy-retrieve: [echo] Project: service compile: [echo] Project: service ivy-resolve-test: [echo] Project: service ivy-retrieve-test: [echo] Project: service compile-test: [echo] Project: service [javac] Compiling 2 source files to /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/service/test/classes test: [echo] Project: hive test-shims: [echo] Project: hive test-conditions: [echo] Project: shims gen-test: [echo] Project: shims create-dirs: [echo] Project: shims [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/test/resources does not exist. init: [echo] Project: shims ivy-init-settings: [echo] Project: shims ivy-resolve: [echo] Project: shims [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml [ivy:report] Processing /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml to /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-shims-default.html ivy-retrieve: [echo] Project: shims compile: [echo] Project: shims [echo] Building shims 0.20 build_shims: [echo] Project: shims [echo] Compiling /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java against hadoop 0.20.2 (/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-0.20.2) ivy-init-settings: [echo] Project: shims ivy-resolve-hadoop-shim: [echo] Project: shims [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml ivy-retrieve-hadoop-shim: [echo] Project: shims [echo] Building shims 0.20S build_shims: [echo] Project: shims [echo] Compiling /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common-secure/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20S/java against hadoop 1.0.0 (/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-1.0.0) ivy-init-settings: [echo] Project: shims ivy-resolve-hadoop-shim: [echo] Project: shims [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml ivy-retrieve-hadoop-shim: [echo] Project: shims [echo] Building shims 0.23 build_shims: [echo] Project: shims [echo] Compiling /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common-secure/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.23/java against hadoop 0.23.3 (/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-0.23.3) ivy-init-settings: [echo] Project:
[jira] [Updated] (HIVE-3729) Error in groupSetExpression rule in Hive grammar
[ https://issues.apache.org/jira/browse/HIVE-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-3729: Attachment: (was: HIVE-3729.1.patch.txt) Error in groupSetExpression rule in Hive grammar Key: HIVE-3729 URL: https://issues.apache.org/jira/browse/HIVE-3729 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Environment: All Reporter: Harish Butani Priority: Minor Fix For: 0.10.0 Original Estimate: 5m Remaining Estimate: 5m Here is the error: Hive.g:1902:38: reference to rewrite element groupByExpression without reference on left of - -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3691) TestDynamicSerDe failed with IBM JDK
[ https://issues.apache.org/jira/browse/HIVE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502124#comment-13502124 ] Renata Ghisloti Duarte de Souza commented on HIVE-3691: --- Bing Li, I think both are ok in this case. If it works in IBM and Sun Java I am happy with either solution. TestDynamicSerDe failed with IBM JDK Key: HIVE-3691 URL: https://issues.apache.org/jira/browse/HIVE-3691 Project: Hive Issue Type: Bug Affects Versions: 0.7.1, 0.8.0, 0.9.0 Environment: ant-1.8.2, IBM JDK 1.6 Reporter: Bing Li Assignee: Bing Li Priority: Minor Attachments: HIVE-3691.1.patch-trunk.txt, HIVE-3691.1.patch.txt the order of the output in the gloden file are different from JDKs. the root cause of this is the implementation of HashMap in JDK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3384: - Status: Patch Available (was: Open) Submitting patch for Jenkins testing. HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Reporter: Weidong Bian Priority: Minor Attachments: D6873.1.patch, HIVE-3384.2.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
JIRA permissions for Hive
Hello, For some reason I don't seem to have permissions to assign issues, change issue priority, change issue status, cancel patch, etc. on Hive issues. Can someone add me to the relevant group on JIRA to allow these actions? My JIRA profile is: https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mikhail Thanks a lot! Mikhail
[jira] [Updated] (HIVE-3729) Error in groupSetExpression rule in Hive grammar
[ https://issues.apache.org/jira/browse/HIVE-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-3729: Attachment: HIVE-3729.1.patch.txt Error in groupSetExpression rule in Hive grammar Key: HIVE-3729 URL: https://issues.apache.org/jira/browse/HIVE-3729 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Environment: All Reporter: Harish Butani Priority: Minor Fix For: 0.10.0 Attachments: HIVE-3729.1.patch.txt Original Estimate: 5m Remaining Estimate: 5m Here is the error: Hive.g:1902:38: reference to rewrite element groupByExpression without reference on left of - -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3731) Ant target to create a Debian package
Mikhail Bautin created HIVE-3731: Summary: Ant target to create a Debian package Key: HIVE-3731 URL: https://issues.apache.org/jira/browse/HIVE-3731 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor We need an Ant target to generate a Debian package with Hive binary distribution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502201#comment-13502201 ] Chris Drome commented on HIVE-3384: --- Thanks for picking this up and integrating my changes. I'd also like to see a 0.9 patch get committed as well, so I will prepare one based on your trunk patch. HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Reporter: Weidong Bian Priority: Minor Attachments: D6873.1.patch, HIVE-3384.2.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: JIRA permissions for Hive
Hey Mikhail, Added you to the list of contributor. You should be able to perform those actions now. Ashutosh On Wed, Nov 21, 2012 at 10:06 AM, Mikhail Bautin bautin.mailing.li...@gmail.com wrote: Hello, For some reason I don't seem to have permissions to assign issues, change issue priority, change issue status, cancel patch, etc. on Hive issues. Can someone add me to the relevant group on JIRA to allow these actions? My JIRA profile is: https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mikhail Thanks a lot! Mikhail
[jira] [Commented] (HIVE-3722) Create index fails on CLI using remote metastore
[ https://issues.apache.org/jira/browse/HIVE-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502227#comment-13502227 ] Ashutosh Chauhan commented on HIVE-3722: Yeah, looks like adding null check is the only possible solution here. +1 Kevin / Namit, feel free to commit. Create index fails on CLI using remote metastore Key: HIVE-3722 URL: https://issues.apache.org/jira/browse/HIVE-3722 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3722.1.patch.txt If the CLI uses a remote metastore and the user attempts to create an index without a comment, it will fail with a NPE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3731) Ant target to create a Debian package
[ https://issues.apache.org/jira/browse/HIVE-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3731: -- Attachment: D6879.1.patch mbautin requested code review of [jira] [HIVE-3731] Ant target to create a Debian package. Reviewers: njain, heyongqiang, raghotham, cwsteinbach, ashutoshc, JIRA Adding an Ant target to create a Debian package from the bin tarball. Also adding a skip.javadoc setting for faster builds. TEST PLAN ant -Dskip.javadoc=true deb REVISION DETAIL https://reviews.facebook.net/D6879 AFFECTED FILES build.xml deb/control ivy.xml ivy/libraries.properties To: JIRA Ant target to create a Debian package - Key: HIVE-3731 URL: https://issues.apache.org/jira/browse/HIVE-3731 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor Attachments: D6879.1.patch We need an Ant target to generate a Debian package with Hive binary distribution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3731) Ant target to create a Debian package
[ https://issues.apache.org/jira/browse/HIVE-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3731: - Status: Patch Available (was: Open) Ant target to create a Debian package - Key: HIVE-3731 URL: https://issues.apache.org/jira/browse/HIVE-3731 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Attachments: D6879.1.patch We need an Ant target to generate a Debian package with Hive binary distribution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502330#comment-13502330 ] Chris Drome commented on HIVE-3384: --- Just wondering if anyone following this thread has experienced consistent success building and running with JDK7. Unless I upgrade DataNucleus I get the following types of test failures: java.lang.VerifyError: Expecting a stackmap frame at branch target 18 in method org.apache.hadoop.hive.metastore.model.MDatabase.jdoReplaceDetachedState()V at offset 4 Upgrading DataNucleus helps me get around some issues, but many tests that previously succeeded now fail. Are others experiencing the same or is there something I've missed. HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Reporter: Weidong Bian Priority: Minor Attachments: D6873.1.patch, HIVE-3384.2.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-3384: -- Attachment: D6873-0.9.1.patch Attached a patch for branch 0.9. HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Reporter: Weidong Bian Priority: Minor Attachments: D6873-0.9.1.patch, D6873.1.patch, HIVE-3384.2.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3630) udf_substr.q fails when using JDK7
[ https://issues.apache.org/jira/browse/HIVE-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-3630: -- Attachment: HIVE-3630-trunk.patch HIVE-3630-0.9.patch Attached branch 0.9 and trunk patches. udf_substr.q fails when using JDK7 -- Key: HIVE-3630 URL: https://issues.apache.org/jira/browse/HIVE-3630 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-3630-0.9.patch, HIVE-3630-trunk.patch Internal error: Cannot find ConstantObjectInspector for BINARY This exception has two causes. JDK7 iterators do not return values in the same order as JDK6, which selects a different implementation of this UDF when the first argument is null. With JDK7 this happens to be the binary version. The binary version is not implemented properly which ultimately causes the exception when the method is called. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1810 - Still Failing
Changes for Build #1767 [kevinwilfong] HIVE-3276. optimize union sub-queries. (njain via kevinwilfong) Changes for Build #1768 Changes for Build #1769 Changes for Build #1770 [namit] HIVE-3570 Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr (Satadru Pan via namit) [namit] HIVE-3554 Hive List Bucketing - Query logic (Gang Tim Liu via namit) [cws] HIVE-3563. Drop database cascade fails when there are indexes on any tables (Prasad Mujumdar via cws) Changes for Build #1771 [kevinwilfong] HIVE-3640. Reducer allocation is incorrect if enforce bucketing and mapred.reduce.tasks are both set. (Vighnesh Avadhani via kevinwilfong) Changes for Build #1772 Changes for Build #1773 Changes for Build #1774 Changes for Build #1775 [namit] HIVE-3673 Sort merge join not used when join columns have different names (Kevin Wilfong via namit) Changes for Build #1776 [kevinwilfong] HIVE-3627. eclipse misses library: javolution-@javolution-version@.jar. (Gang Tim Liu via kevinwilfong) Changes for Build #1777 [kevinwilfong] HIVE-3524. Storing certain Exception objects thrown in HiveMetaStore.java in MetaStoreEndFunctionContext. (Maheshwaran Srinivasan via kevinwilfong) [cws] HIVE-1977. DESCRIBE TABLE syntax doesn't support specifying a database qualified table name (Zhenxiao Luo via cws) [cws] HIVE-3674. Test case TestParse broken after recent checkin (Sambavi Muthukrishnan via cws) Changes for Build #1778 [cws] HIVE-1362. Column level scalar valued statistics on Tables and Partitions (Shreepadma Venugopalan via cws) Changes for Build #1779 Changes for Build #1780 [kevinwilfong] HIVE-3686. Fix compile errors introduced by the interaction of HIVE-1362 and HIVE-3524. (Shreepadma Venugopalan via kevinwilfong) Changes for Build #1781 [namit] HIVE-3687 smb_mapjoin_13.q is nondeterministic (Kevin Wilfong via namit) Changes for Build #1782 [hashutosh] HIVE-2715: Upgrade Thrift dependency to 0.9.0 (Ashutosh Chauhan) Changes for Build #1783 [kevinwilfong] HIVE-3654. block relative path access in hive. (njain via kevinwilfong) [hashutosh] HIVE-3658 : Unable to generate the Hbase related unit tests using velocity templates on Windows (Kanna Karanam via Ashutosh Chauhan) [hashutosh] HIVE-3661 : Remove the Windows specific = related swizzle path changes from Proxy FileSystems (Kanna Karanam via Ashutosh Chauhan) [hashutosh] HIVE-3480 : Resource leak: Fix the file handle leaks in Symbolic Symlink related input formats. (Kanna Karanam via Ashutosh Chauhan) Changes for Build #1784 [kevinwilfong] HIVE-3675. NaN does not work correctly for round(n). (njain via kevinwilfong) [cws] HIVE-3651. bucketmapjoin?.q tests fail with hadoop 0.23 (Prasad Mujumdar via cws) Changes for Build #1785 [namit] HIVE-3613 Implement grouping_id function (Ian Gorbachev via namit) [namit] HIVE-3692 Update parallel test documentation (Ivan Gorbachev via namit) [namit] HIVE-3649 Hive List Bucketing - enhance DDL to specify list bucketing table (Gang Tim Liu via namit) Changes for Build #1786 [namit] HIVE-3696 Revert HIVE-3483 which causes performance regression (Gang Tim Liu via namit) Changes for Build #1787 [kevinwilfong] HIVE-3621. Make prompt in Hive CLI configurable. (Jingwei Lu via kevinwilfong) [kevinwilfong] HIVE-3695. TestParse breaks due to HIVE-3675. (njain via kevinwilfong) Changes for Build #1788 [kevinwilfong] HIVE-3557. Access to external URLs in hivetest.py. (Ivan Gorbachev via kevinwilfong) Changes for Build #1789 [hashutosh] HIVE-3662 : TestHiveServer: testScratchDirShouldClearWhileStartup is failing on Windows (Kanna Karanam via Ashutosh Chauhan) [hashutosh] HIVE-3659 : TestHiveHistory::testQueryloglocParentDirNotExist Test fails on Windows because of some resource leaks in ZK (Kanna Karanam via Ashutosh Chauhan) [hashutosh] HIVE-3663 Unable to display the MR Job file path on Windows in case of MR job failures. (Kanna Karanam via Ashutosh Chauhan) Changes for Build #1790 Changes for Build #1791 Changes for Build #1792 Changes for Build #1793 [hashutosh] HIVE-3704 : name of some metastore scripts are not per convention (Ashutosh Chauhan) Changes for Build #1794 [hashutosh] HIVE-3243 : ignore white space between entries of hive/hbase table mapping (Shengsheng Huang via Ashutosh Chauhan) [hashutosh] HIVE-3215 : JobDebugger should use RunningJob.getTrackingURL (Bhushan Mandhani via Ashutosh Chauhan) Changes for Build #1795 [cws] HIVE-3437. 0.23 compatibility: fix unit tests when building against 0.23 (Chris Drome via cws) [hashutosh] HIVE-3626 : RetryingHMSHandler should wrap JDOException inside MetaException (Bhushan Mandhani via Ashutosh Chauhan) [hashutosh] HIVE-3560 : Hive always prints a warning message when using remote metastore (Travis Crawford via Ashutosh Chauhan) Changes for Build #1796 Changes for Build #1797 [hashutosh] HIVE-3664 : Avoid to create a symlink for hive-contrib.jar file in
[jira] [Commented] (HIVE-3630) udf_substr.q fails when using JDK7
[ https://issues.apache.org/jira/browse/HIVE-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502378#comment-13502378 ] Chris Drome commented on HIVE-3630: --- Branch 0.9: https://reviews.facebook.net/D6885 Trunk: https://reviews.facebook.net/D6891 udf_substr.q fails when using JDK7 -- Key: HIVE-3630 URL: https://issues.apache.org/jira/browse/HIVE-3630 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-3630-0.9.patch, HIVE-3630-trunk.patch Internal error: Cannot find ConstantObjectInspector for BINARY This exception has two causes. JDK7 iterators do not return values in the same order as JDK6, which selects a different implementation of this UDF when the first argument is null. With JDK7 this happens to be the binary version. The binary version is not implemented properly which ultimately causes the exception when the method is called. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3732) Multiple aggregates in query fail the job
Jeremy Hanna created HIVE-3732:
--
Summary: Multiple aggregates in query fail the job
Key: HIVE-3732
URL: https://issues.apache.org/jira/browse/HIVE-3732
Project: Hive
Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Jeremy Hanna
Attachments: hive_output.txt
To reproduce:
Create the hive table:
{code}CREATE EXTERNAL TABLE Birthdays (id int, name string, birthday timestamp)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ';{code}
Insert the data (from file):
{quote}
1 A 2012-10-15 08:01:00
2 B 2012-10-15 08:02:00
3 C 2012-10-15 08:03:00
4 D 2012-10-15 08:04:00
5 E 2012-10-15 08:05:00
6 F 2012-10-15 08:06:00
7 G 2012-10-15 08:07:00
8 H 2012-10-15 08:08:00
9 I 2012-10-15 08:09:00
{quote}
The first 3 queries work, the last query errors out:
{code}
SELECT * FROM Birthdays;
SELECT MIN(Birthday) AS MinBirthday FROM Birthdays;
SELECT MAX(Birthday) AS MaxBirthday FROM Birthdays;
SELECT MIN(Birthday) AS MinBirthday, MAX(Birthday) AS MaxBirthday FROM
Birthdays;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3732) Multiple aggregates in query fail the job
[
https://issues.apache.org/jira/browse/HIVE-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Hanna updated HIVE-3732:
---
Attachment: hive_output.txt
Attaching the error from the failed reduce task log.
Multiple aggregates in query fail the job
-
Key: HIVE-3732
URL: https://issues.apache.org/jira/browse/HIVE-3732
Project: Hive
Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Jeremy Hanna
Attachments: hive_output.txt
To reproduce:
Create the hive table:
{code}CREATE EXTERNAL TABLE Birthdays (id int, name string, birthday
timestamp) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ';{code}
Insert the data (from file):
{quote}
1 A 2012-10-15 08:01:00
2 B 2012-10-15 08:02:00
3 C 2012-10-15 08:03:00
4 D 2012-10-15 08:04:00
5 E 2012-10-15 08:05:00
6 F 2012-10-15 08:06:00
7 G 2012-10-15 08:07:00
8 H 2012-10-15 08:08:00
9 I 2012-10-15 08:09:00
{quote}
The first 3 queries work, the last query errors out:
{code}
SELECT * FROM Birthdays;
SELECT MIN(Birthday) AS MinBirthday FROM Birthdays;
SELECT MAX(Birthday) AS MaxBirthday FROM Birthdays;
SELECT MIN(Birthday) AS MinBirthday, MAX(Birthday) AS MaxBirthday FROM
Birthdays;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3631) script_pipe.q fails when using JDK7
[ https://issues.apache.org/jira/browse/HIVE-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502407#comment-13502407 ] Chris Drome commented on HIVE-3631: --- Branch 0.9: https://reviews.facebook.net/D6897 Trunk: https://reviews.facebook.net/D6909 script_pipe.q fails when using JDK7 --- Key: HIVE-3631 URL: https://issues.apache.org/jira/browse/HIVE-3631 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Assignee: Chris Drome Hive Runtime Error while closing operators: Hit error while closing .. The MR job fails on this test. Unfortunately, the exception is not all that helpful. I tracked this down to a class which attempts to close a stream that is already closed. Broken pipe exceptions are caught and not propagated further, but stream closed exception are not caught. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3631) script_pipe.q fails when using JDK7
[ https://issues.apache.org/jira/browse/HIVE-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-3631: -- Attachment: HIVE-3631-trunk.patch HIVE-3631-0.9.patch Attached branch 0.9 and trunk patches. script_pipe.q fails when using JDK7 --- Key: HIVE-3631 URL: https://issues.apache.org/jira/browse/HIVE-3631 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Assignee: Chris Drome Attachments: HIVE-3631-0.9.patch, HIVE-3631-trunk.patch Hive Runtime Error while closing operators: Hit error while closing .. The MR job fails on this test. Unfortunately, the exception is not all that helpful. I tracked this down to a class which attempts to close a stream that is already closed. Broken pipe exceptions are caught and not propagated further, but stream closed exception are not caught. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3726) History file closed in finalize method
[ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-3726: - Attachment: HIVE-3726.2-r1411423.patch History file closed in finalize method -- Key: HIVE-3726 URL: https://issues.apache.org/jira/browse/HIVE-3726 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-3726.2-r1411423.patch, HIVE-3736.1-r1411423.patch TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3726) History file closed in finalize method
[ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502415#comment-13502415 ] Gunther Hagleitner commented on HIVE-3726: -- Updated patch uses a new SessionState per task (parallel execution), so they don't step on one another. History file closed in finalize method -- Key: HIVE-3726 URL: https://issues.apache.org/jira/browse/HIVE-3726 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-3726.2-r1411423.patch, HIVE-3736.1-r1411423.patch TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3726) History file closed in finalize method
[ https://issues.apache.org/jira/browse/HIVE-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-3726: - Status: Patch Available (was: Open) History file closed in finalize method -- Key: HIVE-3726 URL: https://issues.apache.org/jira/browse/HIVE-3726 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-3726.2-r1411423.patch, HIVE-3736.1-r1411423.patch TestCliNegative fails intermittently because it's up to the garbage collector to close History files. This is only a problem if you deal with a lot of SessionState objects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502440#comment-13502440 ] Shreepadma Venugopalan commented on HIVE-3678: -- @Ashutosh: My answers are inline. We can add two more column in M*ColumnStatistics table of type BigDecimal: BigDecimalLowValue and BigDecimalHighValue. But is BigDecimal type supported consistently across different DBs? Agreed, BigDecimal is not consistently supported across DBs. Hence we can't add a BigDecimal column consistently across DB vendors and versions easily. We can have these two columns of type Double, but then we loose precision. Yes, we can store BigDecimal and Long as Double but we will lose precision. We can store as plain strings in column of type varchar. The maximum number of digits after the decimal point in a BigDecimal number is unlimited for all practical purpose. If we stored it in a varchar, it could result in truncation of some digits following the decimal point in some cases, but this seems to be the only practical solution. We can store in json format in column of type varchar. The maximum number of digits in a BigDecimal number after the decimal point is unlimited for all practical purposes (Java allows nearly 2 billion digits after the decimal point). At this time, we collect MIN, MAX column values for numeric columns. If we stored BigDecimal value, we may exceed the varchar size limit and as a result truncate the JSON blob. This would result in a malformed JSON object. Additionally we will also lose some of the column statistics. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3733) Improve Hive's logic for conditional merge
Pradeep Kamath created HIVE-3733: Summary: Improve Hive's logic for conditional merge Key: HIVE-3733 URL: https://issues.apache.org/jira/browse/HIVE-3733 Project: Hive Issue Type: Improvement Reporter: Pradeep Kamath If the config hive.merge.mapfiles is set to true and hive.merge.mapredfiles is set to false then when hive encounters a FileSinkOperator when generating map reduce tasks, it will look at the entire job to see if it has a reducer, if it does it will not merge. Instead it should be check if the FileSinkOperator is a child of the reducer. This means that outputs generated in the mapper will be merged, and outputs generated in the reducer will not be, the intended effect of setting those configs. Simple repro: set hive.merge.mapfiles=true; set hive.merge.mapredfiles=false; EXPLAIN FROM input_table INSERT OVERWRITE TABLE output_table1 SELECT key, COUNT(*) group by key INSERT OVERWRITE TABLE output_table2 SELECT *; The output should contain a Conditional Operator, Mapred Stages, and Move tasks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3688) Various tests failing in TestNegativeCliDriver, TestParseNegative, TestParse when using JDK7
[ https://issues.apache.org/jira/browse/HIVE-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Drome updated HIVE-3688: -- Attachment: HIVE-3688-trunk.patch HIVE-3688-0.9.patch Attached branc 0.9 and trunk patches. Various tests failing in TestNegativeCliDriver, TestParseNegative, TestParse when using JDK7 Key: HIVE-3688 URL: https://issues.apache.org/jira/browse/HIVE-3688 Project: Hive Issue Type: Bug Components: Query Processor, Tests Affects Versions: 0.10.0, 0.9.1 Reporter: Chris Drome Attachments: HIVE-3688-0.9.patch, HIVE-3688-trunk.patch The following tests are failing when using JDK7. TestNegativeCliDriver: case_sensitivity.q cast1.q groupby1.q groupby2.q groupby3.q groupby4.q groupby5.q groupby6.q input1.q input2.q input20.q input3.q input4.q input5.q input6.q input7.q input8.q input9.q input_part1.q input_testsequencefile.q input_testxpath.q input_testxpath2.q join1.q join2.q join3.q join4.q join5.q join6.q join7.q join8.q sample1.q sample2.q sample3.q sample4.q sample5.q sample6.q sample7.q subq.q udf1.q udf4.q udf6.q udf_case.q udf_when.q union.q TestParseNegative: invalid_function_param2.q TestNegativeCliDriver: fs_default_name1.q.out_0.23_1.7 fs_default_name2.q.out_0.23_1.7 invalid_cast_from_binary_1.q.out_0.23_1.7 invalid_cast_from_binary_2.q.out_0.23_1.7 invalid_cast_from_binary_3.q.out_0.23_1.7 invalid_cast_from_binary_4.q.out_0.23_1.7 invalid_cast_from_binary_5.q.out_0.23_1.7 invalid_cast_from_binary_6.q.out_0.23_1.7 wrong_column_type.q.out_0.23_1.7 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2983) Hive ant targets for publishing maven artifacts can be simplified
[
https://issues.apache.org/jira/browse/HIVE-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502530#comment-13502530
]
Ashutosh Chauhan commented on HIVE-2983:
Patch looks good. Couple of questions:
* I don't see a reason to publish ant-tasks jar, we can take that out of the
list.
* I don't get why you have special handled exec?
Hive ant targets for publishing maven artifacts can be simplified
-
Key: HIVE-2983
URL: https://issues.apache.org/jira/browse/HIVE-2983
Project: Hive
Issue Type: Improvement
Reporter: Travis Crawford
Assignee: Travis Crawford
Priority: Minor
Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2983.D2961.1.patch
Hive has a few ant tasks related to publishing maven artifacts. As not all
sub projects publish artifacts the {{iterate}} macro that simplifies other
tasks cannot be used in this context.
Hive already uses the {{for}} task from ant-contrib, which works great here.
{{build.xml}} can be simplified by using the for task when preparing maven
artifacts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3734) Static partition DML create duplicate files and records
Gang Tim Liu created HIVE-3734: -- Summary: Static partition DML create duplicate files and records Key: HIVE-3734 URL: https://issues.apache.org/jira/browse/HIVE-3734 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Gang Tim Liu Static DML create duplicate files and record. Given the attached test case, hive will return 2 records: 484 val_484 484 val_484 but srcpart returns one record: 484 val_484 If you look at file system, DML generates duplicate file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3734) Static partition DML create duplicate files and records
[ https://issues.apache.org/jira/browse/HIVE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3734: --- Description: Static DML create duplicate files and record. Given the attached test case, hive will return 2 records: 484 val_484 484 val_484 but srcpart returns one record: 484 val_484 If you look at file system, DML generates duplicate file with the same content: -rw-r--r-- 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 00_0 -rwxr-xr-x 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 01_0 was: Static DML create duplicate files and record. Given the attached test case, hive will return 2 records: 484 val_484 484 val_484 but srcpart returns one record: 484 val_484 If you look at file system, DML generates duplicate file. Static partition DML create duplicate files and records --- Key: HIVE-3734 URL: https://issues.apache.org/jira/browse/HIVE-3734 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Gang Tim Liu Static DML create duplicate files and record. Given the attached test case, hive will return 2 records: 484 val_484 484 val_484 but srcpart returns one record: 484 val_484 If you look at file system, DML generates duplicate file with the same content: -rw-r--r-- 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 00_0 -rwxr-xr-x 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 01_0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3734) Static partition DML create duplicate files and records
[ https://issues.apache.org/jira/browse/HIVE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3734: --- Description: Static DML create duplicate files and record. Given the following test case, hive will return 2 records: 484 val_484 484 val_484 but srcpart returns one record: 484 val_484 If you look at file system, DML generates duplicate file with the same content: -rw-r--r-- 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 00_0 -rwxr-xr-x 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 01_0 Test Case === set hive.mapred.supports.subdirectories=true; set hive.exec.dynamic.partition=true; set hive.exec.dynamic.partition.mode=nonstrict; set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat; set hive.merge.mapfiles=false; set hive.merge.mapredfiles=false; set mapred.input.dir.recursive=true; create table testtable (key String, value String) partitioned by (ds String, hr String) ; -- list bucketing DML explain extended insert overwrite table testtable partition (ds='2008-04-08', hr='11') select key, value from srcpart where ds='2008-04-08'; insert overwrite table testtable partition (ds='2008-04-08', hr='11') select key, value from srcpart where ds='2008-04-08'; -- check DML result desc formatted testtable partition (ds='2008-04-08', hr='11'); select count(1) from srcpart where ds='2008-04-08'; select count(1) from testtable where ds='2008-04-08'; select key, value from srcpart where ds='2008-04-08' and hr='11' and key = 484; set hive.optimize.listbucketing=true; explain extended select key, value from testtable where ds='2008-04-08' and hr='11' and key = 484; select key, value from testtable where ds='2008-04-08' and hr='11' and key = 484; === was: Static DML create duplicate files and record. Given the attached test case, hive will return 2 records: 484 val_484 484 val_484 but srcpart returns one record: 484 val_484 If you look at file system, DML generates duplicate file with the same content: -rw-r--r-- 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 00_0 -rwxr-xr-x 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 01_0 Static partition DML create duplicate files and records --- Key: HIVE-3734 URL: https://issues.apache.org/jira/browse/HIVE-3734 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Gang Tim Liu Static DML create duplicate files and record. Given the following test case, hive will return 2 records: 484 val_484 484 val_484 but srcpart returns one record: 484 val_484 If you look at file system, DML generates duplicate file with the same content: -rw-r--r-- 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 00_0 -rwxr-xr-x 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 01_0 Test Case === set hive.mapred.supports.subdirectories=true; set hive.exec.dynamic.partition=true; set hive.exec.dynamic.partition.mode=nonstrict; set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat; set hive.merge.mapfiles=false; set hive.merge.mapredfiles=false; set mapred.input.dir.recursive=true; create table testtable (key String, value String) partitioned by (ds String, hr String) ; -- list bucketing DML explain extended insert overwrite table testtable partition (ds='2008-04-08', hr='11') select key, value from srcpart where ds='2008-04-08'; insert overwrite table testtable partition (ds='2008-04-08', hr='11') select key, value from srcpart where ds='2008-04-08'; -- check DML result desc formatted testtable partition (ds='2008-04-08', hr='11'); select count(1) from srcpart where ds='2008-04-08'; select count(1) from testtable where ds='2008-04-08'; select key, value from srcpart where ds='2008-04-08' and hr='11' and key = 484; set hive.optimize.listbucketing=true; explain extended select key, value from testtable where ds='2008-04-08' and hr='11' and key = 484; select key, value from testtable where ds='2008-04-08' and hr='11' and key = 484; === -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3736) hive unit test case build failure.
Ashish Singh created HIVE-3736: -- Summary: hive unit test case build failure. Key: HIVE-3736 URL: https://issues.apache.org/jira/browse/HIVE-3736 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.10.0 Environment: hive unit test case build failure. Reporter: Ashish Singh Fix For: 0.10.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3736) hive unit test case build failure.
[ https://issues.apache.org/jira/browse/HIVE-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singh updated HIVE-3736: --- Description: Hive depend on hbase-test jar. since classifier is not defined in ivysettings.xml, hence hive is downloading hbase-version.jar as hbase-test.jar. This causes the junit test failures. hive unit test case build failure. -- Key: HIVE-3736 URL: https://issues.apache.org/jira/browse/HIVE-3736 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.10.0 Environment: hive unit test case build failure. Reporter: Ashish Singh Fix For: 0.10.0 Hive depend on hbase-test jar. since classifier is not defined in ivysettings.xml, hence hive is downloading hbase-version.jar as hbase-test.jar. This causes the junit test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3736) hive unit test case build failure.
[ https://issues.apache.org/jira/browse/HIVE-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singh reassigned HIVE-3736: -- Assignee: Ashish Singh hive unit test case build failure. -- Key: HIVE-3736 URL: https://issues.apache.org/jira/browse/HIVE-3736 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.10.0 Environment: hive unit test case build failure. Reporter: Ashish Singh Assignee: Ashish Singh Fix For: 0.10.0 Hive depend on hbase-test jar. since classifier is not defined in ivysettings.xml, hence hive is downloading hbase-version.jar as hbase-test-version.jar. This causes the junit test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3735) PTest doesn't work due to hive snapshot version upgrade to 11
[ https://issues.apache.org/jira/browse/HIVE-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3735: --- Status: Patch Available (was: Open) patch is available: https://reviews.facebook.net/D6951 PTest doesn't work due to hive snapshot version upgrade to 11 - Key: HIVE-3735 URL: https://issues.apache.org/jira/browse/HIVE-3735 Project: Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.10.0 Reporter: Gang Tim Liu Priority: Critical PTest fails. Error :: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :: [ivy:resolve] :: org.apache.hive#hive-builtins;0.11.0-SNAPSHOT: not found [ivy:resolve] :: -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3736) hive unit test case build failure.
[ https://issues.apache.org/jira/browse/HIVE-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singh updated HIVE-3736: --- Attachment: HIVE-3736.patch uploading the patch. hive unit test case build failure. -- Key: HIVE-3736 URL: https://issues.apache.org/jira/browse/HIVE-3736 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.10.0 Environment: hive unit test case build failure. Reporter: Ashish Singh Assignee: Ashish Singh Fix For: 0.10.0 Attachments: HIVE-3736.patch Hive depend on hbase-test jar. since classifier is not defined in ivysettings.xml, hence hive is downloading hbase-version.jar as hbase-test-version.jar. This causes the junit test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3735) PTest doesn't work due to hive snapshot version upgrade to 11
[ https://issues.apache.org/jira/browse/HIVE-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3735: --- Attachment: HIVE-3735.patch.1 PTest doesn't work due to hive snapshot version upgrade to 11 - Key: HIVE-3735 URL: https://issues.apache.org/jira/browse/HIVE-3735 Project: Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.10.0 Reporter: Gang Tim Liu Priority: Critical Attachments: HIVE-3735.patch.1 PTest fails. Error :: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :: [ivy:resolve] :: org.apache.hive#hive-builtins;0.11.0-SNAPSHOT: not found [ivy:resolve] :: -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3722) Create index fails on CLI using remote metastore
[ https://issues.apache.org/jira/browse/HIVE-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3722: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Kevin Create index fails on CLI using remote metastore Key: HIVE-3722 URL: https://issues.apache.org/jira/browse/HIVE-3722 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Fix For: 0.10.0 Attachments: HIVE-3722.1.patch.txt If the CLI uses a remote metastore and the user attempts to create an index without a comment, it will fail with a NPE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3735) PTest doesn't work due to hive snapshot version upgrade to 11
[ https://issues.apache.org/jira/browse/HIVE-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502571#comment-13502571 ] Namit Jain commented on HIVE-3735: -- +1 PTest doesn't work due to hive snapshot version upgrade to 11 - Key: HIVE-3735 URL: https://issues.apache.org/jira/browse/HIVE-3735 Project: Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.10.0 Reporter: Gang Tim Liu Priority: Critical Attachments: HIVE-3735.patch.1 PTest fails. Error :: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :: [ivy:resolve] :: org.apache.hive#hive-builtins;0.11.0-SNAPSHOT: not found [ivy:resolve] :: -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3735) PTest doesn't work due to hive snapshot version upgrade to 11
[ https://issues.apache.org/jira/browse/HIVE-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3735: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Tim PTest doesn't work due to hive snapshot version upgrade to 11 - Key: HIVE-3735 URL: https://issues.apache.org/jira/browse/HIVE-3735 Project: Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.10.0 Reporter: Gang Tim Liu Priority: Critical Fix For: 0.10.0 Attachments: HIVE-3735.patch.1 PTest fails. Error :: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :: [ivy:resolve] :: org.apache.hive#hive-builtins;0.11.0-SNAPSHOT: not found [ivy:resolve] :: -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3733) Improve Hive's logic for conditional merge
[ https://issues.apache.org/jira/browse/HIVE-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-3733: Assignee: Pradeep Kamath Improve Hive's logic for conditional merge -- Key: HIVE-3733 URL: https://issues.apache.org/jira/browse/HIVE-3733 Project: Hive Issue Type: Improvement Reporter: Pradeep Kamath Assignee: Pradeep Kamath If the config hive.merge.mapfiles is set to true and hive.merge.mapredfiles is set to false then when hive encounters a FileSinkOperator when generating map reduce tasks, it will look at the entire job to see if it has a reducer, if it does it will not merge. Instead it should be check if the FileSinkOperator is a child of the reducer. This means that outputs generated in the mapper will be merged, and outputs generated in the reducer will not be, the intended effect of setting those configs. Simple repro: set hive.merge.mapfiles=true; set hive.merge.mapredfiles=false; EXPLAIN FROM input_table INSERT OVERWRITE TABLE output_table1 SELECT key, COUNT(*) group by key INSERT OVERWRITE TABLE output_table2 SELECT *; The output should contain a Conditional Operator, Mapred Stages, and Move tasks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3735) PTest doesn't work due to hive snapshot version upgrade to 11
[ https://issues.apache.org/jira/browse/HIVE-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502575#comment-13502575 ] Gang Tim Liu commented on HIVE-3735: Thank you Namit :) Sent from my iPhone PTest doesn't work due to hive snapshot version upgrade to 11 - Key: HIVE-3735 URL: https://issues.apache.org/jira/browse/HIVE-3735 Project: Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.10.0 Reporter: Gang Tim Liu Priority: Critical Fix For: 0.10.0 Attachments: HIVE-3735.patch.1 PTest fails. Error :: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :: [ivy:resolve] :: org.apache.hive#hive-builtins;0.11.0-SNAPSHOT: not found [ivy:resolve] :: -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3703) Hive Query Explain Plan JSON not being created properly
[ https://issues.apache.org/jira/browse/HIVE-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502578#comment-13502578 ] Namit Jain commented on HIVE-3703: -- +1 Hive Query Explain Plan JSON not being created properly --- Key: HIVE-3703 URL: https://issues.apache.org/jira/browse/HIVE-3703 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Mayank Garg Assignee: Mayank Garg Priority: Minor Attachments: HIVE-3703.2.patch.txt Original Estimate: 12h Remaining Estimate: 12h There is an option to generate a JSON query plan for the hive query, however, the JSON being created is invalid and json_decoders are unable to decode it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3646) Add 'IGNORE PROTECTION' predicate for dropping partitions
[ https://issues.apache.org/jira/browse/HIVE-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3646: - Status: Open (was: Patch Available) comments on phabricator Add 'IGNORE PROTECTION' predicate for dropping partitions - Key: HIVE-3646 URL: https://issues.apache.org/jira/browse/HIVE-3646 Project: Hive Issue Type: New Feature Components: CLI Affects Versions: 0.9.0 Reporter: Andrew Chalfant Assignee: Andrew Chalfant Priority: Minor Attachments: HIVE-3646.1.patch.txt Original Estimate: 1m Remaining Estimate: 1m There are cases where it is desirable to move partitions between clusters. Having to undo protection and then re-protect tables in order to delete partitions from a source are multi-step and can leave us in a failed open state where partition and table metadata is dirty. By implementing an 'rm -rf'-like functionality, we can perform these operations atomically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3646) Add 'IGNORE PROTECTION' predicate for dropping partitions
[ https://issues.apache.org/jira/browse/HIVE-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502580#comment-13502580 ] Namit Jain commented on HIVE-3646: -- From the code, it looks like, it doesn't matter if the tables/partitions are hdfs or non-hdfs. They will be dropped either way. https://cwiki.apache.org/Hive/languagemanual-ddl.html Can you document the behavior above ? Add 'IGNORE PROTECTION' predicate for dropping partitions - Key: HIVE-3646 URL: https://issues.apache.org/jira/browse/HIVE-3646 Project: Hive Issue Type: New Feature Components: CLI Affects Versions: 0.9.0 Reporter: Andrew Chalfant Assignee: Andrew Chalfant Priority: Minor Attachments: HIVE-3646.1.patch.txt Original Estimate: 1m Remaining Estimate: 1m There are cases where it is desirable to move partitions between clusters. Having to undo protection and then re-protect tables in order to delete partitions from a source are multi-step and can leave us in a failed open state where partition and table metadata is dirty. By implementing an 'rm -rf'-like functionality, we can perform these operations atomically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502582#comment-13502582 ] Mikhail Bautin commented on HIVE-3384: -- [~cdrome]: This type of error sometimes happens when loading class files compiled with Java 6 into a Java 7 JVM. The -XX:-UseSplitVerifier JVM option helps even though some people say it is unsafe. HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Reporter: Weidong Bian Priority: Minor Attachments: D6873-0.9.1.patch, D6873.1.patch, HIVE-3384.2.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3729) Error in groupSetExpression rule in Hive grammar
[ https://issues.apache.org/jira/browse/HIVE-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-3729: Status: Patch Available (was: Open) Error in groupSetExpression rule in Hive grammar Key: HIVE-3729 URL: https://issues.apache.org/jira/browse/HIVE-3729 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Environment: All Reporter: Harish Butani Priority: Minor Fix For: 0.10.0 Attachments: HIVE-3729.1.patch.txt Original Estimate: 5m Remaining Estimate: 5m Here is the error: Hive.g:1902:38: reference to rewrite element groupByExpression without reference on left of - -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Phabricator login problem: GitHub Auth Failed
Hi, I am having the same issue as was posted here http://mail-archives.apache.org/mod_mbox/hive-dev/201209.mbox/%3C62A90015252F9340B077D64464E2CD612BE24F%40EXMB102.corp.netflix.com%3E Don't see any response to the previous post. Reposting. Regards, Harish Butani.
[jira] [Updated] (HIVE-3685) TestCliDriver (script_pipe.q) failed with IBM JDK
[ https://issues.apache.org/jira/browse/HIVE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li updated HIVE-3685: -- Status: Open (was: Patch Available) TestCliDriver (script_pipe.q) failed with IBM JDK - Key: HIVE-3685 URL: https://issues.apache.org/jira/browse/HIVE-3685 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0, 0.8.0, 0.7.1 Environment: ant-1.8.2 IBM JDK 1.6 Reporter: Bing Li Assignee: Bing Li Fix For: 0.9.0, 0.8.0 Attachments: HIVE-3685.1.patch-trunk.txt 1 failed: TestCliDriver (script_pipe.q) [junit] Begin query: script_pipe.q [junit] java.io.IOException: No such file or directory [junit] at java.io.FileOutputStream.writeBytes(Native Method) [junit] at java.io.FileOutputStream.write(FileOutputStream.java:293) [junit] at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:76) [junit] at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:134) [junit] at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:135) [junit] at java.io.DataOutputStream.flush(DataOutputStream.java:117) [junit] at org.apache.hadoop.hive.ql.exec.TextRecordWriter.close(TextRecordWriter.java:48) [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:365) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] Ended Job = job_local_0001 with errors [junit] Error during job, obtaining debugging information... [junit] Exception: Client Execution failed with error code = 9 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit]
[jira] [Updated] (HIVE-3691) TestDynamicSerDe failed with IBM JDK
[ https://issues.apache.org/jira/browse/HIVE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li updated HIVE-3691: -- Status: Patch Available (was: Open) replace HashMap with LinkedHashMap in the test case to avoid different order of the output from different JDKs TestDynamicSerDe failed with IBM JDK Key: HIVE-3691 URL: https://issues.apache.org/jira/browse/HIVE-3691 Project: Hive Issue Type: Bug Affects Versions: 0.9.0, 0.8.0, 0.7.1 Environment: ant-1.8.2, IBM JDK 1.6 Reporter: Bing Li Assignee: Bing Li Priority: Minor Attachments: HIVE-3691.1.patch-trunk.txt, HIVE-3691.1.patch.txt the order of the output in the gloden file are different from JDKs. the root cause of this is the implementation of HashMap in JDK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3685) TestCliDriver (script_pipe.q) failed with IBM JDK
[ https://issues.apache.org/jira/browse/HIVE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li updated HIVE-3685: -- Status: Patch Available (was: Open) the patch would resolve this failure on IBM JDK and won't affect the results on Sun's TestCliDriver (script_pipe.q) failed with IBM JDK - Key: HIVE-3685 URL: https://issues.apache.org/jira/browse/HIVE-3685 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0, 0.8.0, 0.7.1 Environment: ant-1.8.2 IBM JDK 1.6 Reporter: Bing Li Assignee: Bing Li Fix For: 0.9.0, 0.8.0 Attachments: HIVE-3685.1.patch-trunk.txt 1 failed: TestCliDriver (script_pipe.q) [junit] Begin query: script_pipe.q [junit] java.io.IOException: No such file or directory [junit] at java.io.FileOutputStream.writeBytes(Native Method) [junit] at java.io.FileOutputStream.write(FileOutputStream.java:293) [junit] at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:76) [junit] at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:134) [junit] at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:135) [junit] at java.io.DataOutputStream.flush(DataOutputStream.java:117) [junit] at org.apache.hadoop.hive.ql.exec.TextRecordWriter.close(TextRecordWriter.java:48) [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:365) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] Ended Job = job_local_0001 with errors [junit] Error during job, obtaining debugging information... [junit] Exception: Client Execution failed with error code = 9 [junit] See build/ql/tmp/hive.log, or
[jira] [Updated] (HIVE-3721) ALTER TABLE ADD PARTS should check for valid partition spec and throw a SemanticException if part spec is not valid
[ https://issues.apache.org/jira/browse/HIVE-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3721: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Pamela ALTER TABLE ADD PARTS should check for valid partition spec and throw a SemanticException if part spec is not valid --- Key: HIVE-3721 URL: https://issues.apache.org/jira/browse/HIVE-3721 Project: Hive Issue Type: Task Reporter: Pamela Vagata Assignee: Pamela Vagata Priority: Minor Fix For: 0.10.0 Attachments: HIVE-3721.1.patch.txt, HIVE-3721.2.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3729) Error in groupSetExpression rule in Hive grammar
[ https://issues.apache.org/jira/browse/HIVE-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain reassigned HIVE-3729: Assignee: Harish Butani Error in groupSetExpression rule in Hive grammar Key: HIVE-3729 URL: https://issues.apache.org/jira/browse/HIVE-3729 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Environment: All Reporter: Harish Butani Assignee: Harish Butani Priority: Minor Fix For: 0.10.0 Attachments: HIVE-3729.1.patch.txt Original Estimate: 5m Remaining Estimate: 5m Here is the error: Hive.g:1902:38: reference to rewrite element groupByExpression without reference on left of - -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502613#comment-13502613 ] Namit Jain commented on HIVE-2935: -- I haven't looked at the patch either, and I agree with Ashutosh, that additive portions (like beeline) can be checked in. However, would it be possible for you to break this patch, and extract the changes that you have made to the current hive code. That presents a huge risk, and should be reviewed very carefully. Or, you can take the other approach, which is to check in the new additive isolated components first (which will not be used), and then have the code which touches the current hive code in a patch. If someone is not using the hive server, what are the changes that this patch brings in ? That definitely needs to be reviewed very thoroughly. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3737) show partitions command do not accpet dbname
Sujesh Chirackkal created HIVE-3737: --- Summary: show partitions command do not accpet dbname Key: HIVE-3737 URL: https://issues.apache.org/jira/browse/HIVE-3737 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.8.1 Reporter: Sujesh Chirackkal Priority: Minor show partitions dbname.tablename throws error mimatched input . expecting EOF near dbname' Summary; command accpets only table name as input. Similar issue was reported for describe command also. (Hive-3589) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3737) show partitions command do not accept dbname
[ https://issues.apache.org/jira/browse/HIVE-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujesh Chirackkal updated HIVE-3737: Summary: show partitions command do not accept dbname (was: show partitions command do not accpet dbname) show partitions command do not accept dbname Key: HIVE-3737 URL: https://issues.apache.org/jira/browse/HIVE-3737 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.8.1 Reporter: Sujesh Chirackkal Priority: Minor show partitions dbname.tablename throws error mimatched input . expecting EOF near dbname' Summary; command accpets only table name as input. Similar issue was reported for describe command also. (Hive-3589) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3738) Bugs exists in SEMI JOIN
Yingzhong Xu created HIVE-3738:
--
Summary: Bugs exists in SEMI JOIN
Key: HIVE-3738
URL: https://issues.apache.org/jira/browse/HIVE-3738
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: JDK1.6
Reporter: Yingzhong Xu
I am using the version 0.9.0 and my tables are the same with TPC-H benchmark:
Here is a simple query(works correctly):
*Q1*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN(
SELECT O_CUSTKEY FROM ORDERS WHERE unix_timestamp(O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
) tempTable ON tempTable.O_CUSTKEY=CUSTOMER.C_CUSTKEY
{quote}
it means inserting the key of customers who has orders since 1995-12-31 into
another table.
But if I write the query like this:
*Q2*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN ORDERS
ON CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY
AND unix_timestamp(ORDERS.O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
{quote}
I will get exception from Hive:
{quote}
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:1566)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.pushJoinFilters(SemanticAnalyzer.java:5254)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6754)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7531)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:336)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:909)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
{quote}
Also,If I write the query like this:
*Q3*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN ORDERS
ON CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY
WHERE unix_timestamp(ORDERS.O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
{quote}
Then this query can be executed(wondering the right hand of SEMI JOIN can be
referenced in WHERE clause now?), but the result is wrong(comparing to Q1, Q1's
result is the same with mysql).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support
[ https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3073: --- Attachment: HIVE-3073.patch.18 Hive List Bucketing - DML support -- Key: HIVE-3073 URL: https://issues.apache.org/jira/browse/HIVE-3073 Project: Hive Issue Type: New Feature Components: SQL Affects Versions: 0.10.0 Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3073.patch.12, HIVE-3073.patch.13, HIVE-3073.patch.15, HIVE-3073.patch.18 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DML change for the feature: 1. single skewed column 2. manual load data -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support
[ https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3073: --- Status: Patch Available (was: Open) patch is available. Hive List Bucketing - DML support -- Key: HIVE-3073 URL: https://issues.apache.org/jira/browse/HIVE-3073 Project: Hive Issue Type: New Feature Components: SQL Affects Versions: 0.10.0 Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3073.patch.12, HIVE-3073.patch.13, HIVE-3073.patch.15, HIVE-3073.patch.18 If a hive table column has skewed keys, query performance on non-skewed key is always impacted. Hive List Bucketing feature will address it: https://cwiki.apache.org/Hive/listbucketing.html This jira issue will track DML change for the feature: 1. single skewed column 2. manual load data -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3739) Hive auto convert join result a java.lang.InstantiationException: org.antlr.runtime.CommonToken
fantasy created HIVE-3739: - Summary: Hive auto convert join result a java.lang.InstantiationException: org.antlr.runtime.CommonToken Key: HIVE-3739 URL: https://issues.apache.org/jira/browse/HIVE-3739 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.9.0 Environment: hive.auto.convert.join=true Reporter: fantasy After I set hive.auto.convert.join=true. Any HiveQL with a join executed in hive result a error as this: - java.lang.InstantiationException: org.antlr.runtime.CommonToken Continuing ... java.lang.RuntimeException: failed to evaluate: unbound=Class.new(); Continuing ... java.lang.InstantiationException: org.antlr.runtime.CommonToken Continuing ... java.lang.RuntimeException: failed to evaluate: unbound=Class.new(); Continuing ... java.lang.InstantiationException: org.antlr.runtime.CommonToken Continuing ... java.lang.RuntimeException: failed to evaluate: unbound=Class.new(); Continuing ... java.lang.InstantiationException: org.antlr.runtime.CommonToken Continuing ... java.lang.RuntimeException: failed to evaluate: unbound=Class.new(); Continuing ... --- can anyone tell why? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3739) Hive auto convert join result error: java.lang.InstantiationException: org.antlr.runtime.CommonToken
[ https://issues.apache.org/jira/browse/HIVE-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fantasy updated HIVE-3739: -- Summary: Hive auto convert join result error: java.lang.InstantiationException: org.antlr.runtime.CommonToken (was: Hive auto convert join result a java.lang.InstantiationException: org.antlr.runtime.CommonToken) Hive auto convert join result error: java.lang.InstantiationException: org.antlr.runtime.CommonToken Key: HIVE-3739 URL: https://issues.apache.org/jira/browse/HIVE-3739 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.9.0 Environment: hive.auto.convert.join=true Reporter: fantasy After I set hive.auto.convert.join=true. Any HiveQL with a join executed in hive result a error as this: - java.lang.InstantiationException: org.antlr.runtime.CommonToken Continuing ... java.lang.RuntimeException: failed to evaluate: unbound=Class.new(); Continuing ... java.lang.InstantiationException: org.antlr.runtime.CommonToken Continuing ... java.lang.RuntimeException: failed to evaluate: unbound=Class.new(); Continuing ... java.lang.InstantiationException: org.antlr.runtime.CommonToken Continuing ... java.lang.RuntimeException: failed to evaluate: unbound=Class.new(); Continuing ... java.lang.InstantiationException: org.antlr.runtime.CommonToken Continuing ... java.lang.RuntimeException: failed to evaluate: unbound=Class.new(); Continuing ... --- can anyone tell why? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3738) Bugs exists in SEMI JOIN
[
https://issues.apache.org/jira/browse/HIVE-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yingzhong Xu updated HIVE-3738:
---
Attachment: DDL
The DDL helps you create related tables
Bugs exists in SEMI JOIN
Key: HIVE-3738
URL: https://issues.apache.org/jira/browse/HIVE-3738
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: JDK1.6
Reporter: Yingzhong Xu
Labels: Semijoin
Attachments: DDL
I am using the version 0.9.0 and my tables are the same with TPC-H benchmark:
Here is a simple query(works correctly):
*Q1*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN(
SELECT O_CUSTKEY FROM ORDERS WHERE unix_timestamp(O_ORDERDATE,
'-MM-dd') unix_timestamp('1995-12-31','-MM-dd')
) tempTable ON tempTable.O_CUSTKEY=CUSTOMER.C_CUSTKEY
{quote}
it means inserting the key of customers who has orders since 1995-12-31 into
another table.
But if I write the query like this:
*Q2*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN ORDERS
ON CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY
AND unix_timestamp(ORDERS.O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
{quote}
I will get exception from Hive:
{quote}
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:1566)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.pushJoinFilters(SemanticAnalyzer.java:5254)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6754)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7531)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:336)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:909)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
{quote}
Also,If I write the query like this:
*Q3*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN ORDERS
ON CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY
WHERE unix_timestamp(ORDERS.O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
{quote}
Then this query can be executed(wondering the right hand of SEMI JOIN can be
referenced in WHERE clause now?), but the result is wrong(comparing to Q1,
Q1's result is the same with mysql).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3738) Bugs exists in SEMI JOIN
[
https://issues.apache.org/jira/browse/HIVE-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yingzhong Xu updated HIVE-3738:
---
Description:
I am using the version 0.9.0 and my tables are the same with TPC-H benchmark:
Here is a simple query(works correctly):
*Q1*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN(
SELECT O_CUSTKEY FROM ORDERS WHERE unix_timestamp(O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
) tempTable ON tempTable.O_CUSTKEY=CUSTOMER.C_CUSTKEY
{quote}
it means inserting the key of customers who has orders since 1995-12-31 into
another table.
But if I write the query like this:
*Q2*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN ORDERS
ON CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY
AND unix_timestamp(ORDERS.O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
{quote}
I will get exception from Hive:
{quote}
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:1566)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.pushJoinFilters(SemanticAnalyzer.java:5254)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6754)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7531)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:336)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:909)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
{quote}
Also,If I write the query like this:
*Q3*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN ORDERS
ON CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY
WHERE unix_timestamp(ORDERS.O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
{quote}
Then this query can be executed(wondering the right hand of SEMI JOIN can be
referenced in WHERE clause now?), but the result is wrong(comparing to *Q1*,
*Q1*'s result is the same with mysql).
was:
I am using the version 0.9.0 and my tables are the same with TPC-H benchmark:
Here is a simple query(works correctly):
*Q1*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN(
SELECT O_CUSTKEY FROM ORDERS WHERE unix_timestamp(O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
) tempTable ON tempTable.O_CUSTKEY=CUSTOMER.C_CUSTKEY
{quote}
it means inserting the key of customers who has orders since 1995-12-31 into
another table.
But if I write the query like this:
*Q2*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN ORDERS
ON CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY
AND unix_timestamp(ORDERS.O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
{quote}
I will get exception from Hive:
{quote}
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:1566)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.pushJoinFilters(SemanticAnalyzer.java:5254)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6754)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7531)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:336)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:909)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
at
[jira] [Updated] (HIVE-3738) Bugs exist in SEMI JOIN
[
https://issues.apache.org/jira/browse/HIVE-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yingzhong Xu updated HIVE-3738:
---
Summary: Bugs exist in SEMI JOIN (was: Bugs exists in SEMI JOIN)
Bugs exist in SEMI JOIN
---
Key: HIVE-3738
URL: https://issues.apache.org/jira/browse/HIVE-3738
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: JDK1.6
Reporter: Yingzhong Xu
Labels: Semijoin
Attachments: DDL
I am using the version 0.9.0 and my tables are the same with TPC-H benchmark:
Here is a simple query(works correctly):
*Q1*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN(
SELECT O_CUSTKEY FROM ORDERS WHERE unix_timestamp(O_ORDERDATE,
'-MM-dd') unix_timestamp('1995-12-31','-MM-dd')
) tempTable ON tempTable.O_CUSTKEY=CUSTOMER.C_CUSTKEY
{quote}
it means inserting the key of customers who has orders since 1995-12-31 into
another table.
But if I write the query like this:
*Q2*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN ORDERS
ON CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY
AND unix_timestamp(ORDERS.O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
{quote}
I will get exception from Hive:
{quote}
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:1566)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.pushJoinFilters(SemanticAnalyzer.java:5254)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6754)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7531)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:336)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:909)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
{quote}
Also,If I write the query like this:
*Q3*
{quote}
INSERT OVERWRITE TABLE customer_orders_statistics
SELECT C_CUSTKEY FROM CUSTOMER
LEFT SEMI JOIN ORDERS
ON CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY
WHERE unix_timestamp(ORDERS.O_ORDERDATE, '-MM-dd')
unix_timestamp('1995-12-31','-MM-dd')
{quote}
Then this query can be executed(wondering the right hand of SEMI JOIN can be
referenced in WHERE clause now?), but the result is wrong(comparing to *Q1*,
*Q1*'s result is the same with mysql).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
