[jira] Commented: (HIVE-787) Hive Freeway - support near-realtime data processing
[ https://issues.apache.org/jira/browse/HIVE-787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12926192#action_12926192 ] Jeff Hammerbacher commented on HIVE-787: More details on the Data Freeway implementation at Facebook: http://vimeo.com/15337985 Hive Freeway - support near-realtime data processing Key: HIVE-787 URL: https://issues.apache.org/jira/browse/HIVE-787 Project: Hive Issue Type: New Feature Reporter: Zheng Shao Most people are using Hive for daily (or at most hourly) data processing. We want to explore what are the obstacles for using Hive for 15 minutes, 5 minutes or even 1 minute data processing intervals, and remove these obstacles. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1657) join results are displayed wrongly for some complex joins using select *
[ https://issues.apache.org/jira/browse/HIVE-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amareshwari Sriramadasu updated HIVE-1657: -- Attachment: patch-1657.txt Here is a patch to add the output columns for select plan in the order of aliases specified in the query. join results are displayed wrongly for some complex joins using select * Key: HIVE-1657 URL: https://issues.apache.org/jira/browse/HIVE-1657 Project: Hive Issue Type: Bug Reporter: Amareshwari Sriramadasu Attachments: patch-1657.txt Noticed that the output of the join is displayed wrongly for queries involving more than one table and mixture of left and outer joins, with different join conditions. For ex: SELECT * from T1 a RIGHT OUTER JOIN T2 b ON (a.value=b.value) LEFT OUTER JOIN T3 c ON (b.key=c.key); displays the table T2 first, then T1 and T3. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1657) join results are displayed wrongly for some complex joins using select *
[ https://issues.apache.org/jira/browse/HIVE-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amareshwari Sriramadasu updated HIVE-1657: -- Assignee: Amareshwari Sriramadasu Status: Patch Available (was: Open) join results are displayed wrongly for some complex joins using select * Key: HIVE-1657 URL: https://issues.apache.org/jira/browse/HIVE-1657 Project: Hive Issue Type: Bug Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Attachments: patch-1657.txt Noticed that the output of the join is displayed wrongly for queries involving more than one table and mixture of left and outer joins, with different join conditions. For ex: SELECT * from T1 a RIGHT OUTER JOIN T2 b ON (a.value=b.value) LEFT OUTER JOIN T3 c ON (b.key=c.key); displays the table T2 first, then T1 and T3. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1759) Many important broken links on Hive web page
[ https://issues.apache.org/jira/browse/HIVE-1759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12926399#action_12926399 ] John Sichi commented on HIVE-1759: -- I committed a fix for the change log links (per Carl's suggestion, just use JIRA release notes alone and omit CHANGES.txt). I also changed the nav bar trail to be Apache Hive instead of Apache Hadoop Hive, but kept the Hadoop logo. I'll leave the resolution of the documentation links to Ed since that's still under discussion. Once we fix that, we can do another push of the site. Many important broken links on Hive web page Key: HIVE-1759 URL: https://issues.apache.org/jira/browse/HIVE-1759 Project: Hive Issue Type: Bug Components: Documentation Reporter: Jeff Hammerbacher Assignee: Edward Capriolo The change log links are broken, perhaps because of the move to a TLP, and the Jira issue log links all point to the 0.5 issue log. Also, all of the documentation links are broken. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1646) Hive 0.5 Build Crashing
[ https://issues.apache.org/jira/browse/HIVE-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1646: - Fix Version/s: (was: 0.5.1) Hive 0.5 Build Crashing --- Key: HIVE-1646 URL: https://issues.apache.org/jira/browse/HIVE-1646 Project: Hive Issue Type: Bug Components: Testing Infrastructure Affects Versions: 0.5.0 Environment: SLES 10 SP2, SLES 11, RHEL 5.4, ANT 1.8.1 and ANT 1.7.1, SUN JDK 1.6.14 Reporter: Stephen Watt I've tried this on a variety of configurations. Operating Systems SLES 10 SP2, SLES 11, RHEL 5.4 on a variety of machines using both ANT 1.8.1 and ANT 1.7.1 and SUN JDK 1.6.14. I've tried building this by going to the Hive Release page and download hive-0.5.0-src and using that. I've tried building by obtaining the branch tag release using svn checkout http://svn.apache.org/repos/asf/hadoop/hive/tags/release-0.5.0/ hive-0.5.0-dev. Always the same thing: When I run the Hive 0.5 build it runs for just under 2 hours and then crashes with the following message (tail end of ant.log): - - - [junit] diff /home/hive/hive-0.5.0-build/hive-0.5.0-dev/build/ql/test/logs/negative/wrong_distinct2.q.out /home/hive/hive-0.5.0-build/hive-0.5.0-dev/ql/src/test/results/compiler/errors/wrong_distinct2.q.out [junit] Done query: wrong_distinct2.q [junit] Tests run: 31, Failures: 0, Errors: 0, Time elapsed: 90.974 sec [junit] Running org.apache.hadoop.hive.ql.tool.TestLineageInfo [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.525 sec BUILD FAILED /home/hive/hive-0.5.0-build/hive-0.5.0-dev/build.xml:151: The following error occurred while executing this line: /home/hive/hive-0.5.0-build/hive-0.5.0-dev/build.xml:91: The following error occurred while executing this line: /home/hive/hive-0.5.0-build/hive-0.5.0-dev/build-common.xml:327: Tests failed! Total time: 94 minutes 43 seconds - - - My build script is very simplistic : #!/bin/sh # Set Build Dependencies set PATH=$PATH:/home/hive/Java-Versions/jdk1.6.0_14/bin/ export JAVA_HOME=/home/hive/Java-Versions/jdk1.6.0_14 export BUILD_DIR=/home/hive/hive-0.5.0-build export ANT_HOME=$BUILD_DIR/apache-ant-1.8.1 export HIVE_INSTALL=$BUILD_DIR/hive-0.5.0-dev/ export PATH=$PATH:$ANT_HOME/bin # Run Build and Unit Test cd $HIVE_INSTALL ant clean test tar -logfile ant.log -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1686) XMLEncoder failing to serialize classes containing Enums for non-SUN JREs
[ https://issues.apache.org/jira/browse/HIVE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1686: - Fix Version/s: (was: 0.5.1) XMLEncoder failing to serialize classes containing Enums for non-SUN JREs - Key: HIVE-1686 URL: https://issues.apache.org/jira/browse/HIVE-1686 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.5.0 Environment: SLES 10 SP2, IBM Java 1.6 SR8 Reporter: Stephen Watt Priority: Minor Attachments: HIVE-1686.patch If one is using Hive 0.5 with IBM Java 1.6 certain Hive Queries will fail in the Hive CLI, such as SELECT Count(1) from TABLE with the error failed to write expression: GenericUDAFEvaluator$Mode=Class.new(). This is due to the fact that XMLEncoder in the JRE's beans.jar is not able to serialize Classes with Enums without an explicitly having an EnumPersistenceDelegate assigned to each class that needs to be serialized. This was an issue in SUN JDK 1.5 but not 1.6 and is still an issue in IBM Java 1.6. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1760) Mismatched open/commit transaction calls in case of connection retry
[
https://issues.apache.org/jira/browse/HIVE-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12926500#action_12926500
]
Namit Jain commented on HIVE-1760:
--
+1
running tests now
Mismatched open/commit transaction calls in case of connection retry
Key: HIVE-1760
URL: https://issues.apache.org/jira/browse/HIVE-1760
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.7.0
Reporter: Paul Yang
Assignee: Paul Yang
Attachments: HIVE-1760.1.patch
Consider the create table function (parts removed for simplicity):
{code}
private void create_table_core(final RawStore ms, final Table tbl)
throws AlreadyExistsException, MetaException, InvalidObjectException {
Path tblPath = null;
boolean success = false, madeDir = false;
try {
ms.openTransaction();
// get_table checks whether database exists, it should be moved here
if (is_table_exists(tbl.getDbName(), tbl.getTableName())) {
throw new AlreadyExistsException(Table + tbl.getTableName()
+ already exists);
}
ms.createTable(tbl);
success = ms.commitTransaction();
} finally {
if (!success) {
ms.rollbackTransaction();
if (madeDir) {
wh.deleteDir(tblPath, true);
}
}
}
}
{code}
A potential openTransaction() / commitTransaction() mismatch can occur if the
is_table_exits() method call experiences a connection failure.
Since get_table() in is_table_exists() uses executeWithRetry(), the
transaction will be rolled back and get_table() will be called again if the
is a connection problem. However, this rollback and retry will reset the
global openTransactionCalls counter back to 0, effectively canceling out the
openTransaction() call.
Then later in the method when commitTransaction() is called, Hive will throw
an error similar to the following:
Caused by: java.lang.RuntimeException: commitTransaction was called but
openTransactionCalls = 0. This probably indicates that there are unbalanced
calls to openTransaction/commitTransaction
A similar problem exists with create_type_core()
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1612) Cannot build hive for hadoop 0.21.0
[ https://issues.apache.org/jira/browse/HIVE-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12926518#action_12926518 ] AJ Pahl commented on HIVE-1612: --- As per Daisuke's instructions I was able to complile trunk-1000628 against Hadoop 0.21.0. Unfortunately, execution isn't exactly like Singo's but is still contains a few exceptions: -bash-3.2$ ./hive Exception in thread main java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at org.apache.hadoop.util.RunJar.main(RunJar.java:185) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:252) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) ... 3 more Also starting hiveserver... Starting Hive Thrift Server Exception in thread main java.lang.NoClassDefFoundError: org/apache/thrift/protocol/TProtocolFactory at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at org.apache.hadoop.util.RunJar.main(RunJar.java:185) Caused by: java.lang.ClassNotFoundException: org.apache.thrift.protocol.TProtocolFactory at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:252) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) ... 3 more I've had success with compiling previous releases of 0.6.0 and 0.7.0 (prior to 0.21.0) so I'm pretty sure my environment is correct for testing this. (ie: Classpath etc) Daisuke, keep up the good work. I think this JIRA will be renamed DAISUKE-1612 if you can get it worked out. :) Cannot build hive for hadoop 0.21.0 --- Key: HIVE-1612 URL: https://issues.apache.org/jira/browse/HIVE-1612 Project: Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: AJ Pahl Attachments: HIVE-1612.patch Current trunk for 0.7.0 does not support building HIVE against the Hadoop 0.21.0 release. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1746) Support for using ALTER to set IDXPROPERTIES
[ https://issues.apache.org/jira/browse/HIVE-1746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marquis Wang updated HIVE-1746: --- Attachment: 1746.prelim.patch Preliminary patch. I think I need to add an alter_index function to the HiveMetaStoreClient, which I think requires editing the thrift files. I'm not sure if that is the correct way to go about that... is there a better way to allow us to change the properties on an existing index? If that is correct, how do I generate the new ThriftHiveMetaStoreClient.java? Support for using ALTER to set IDXPROPERTIES Key: HIVE-1746 URL: https://issues.apache.org/jira/browse/HIVE-1746 Project: Hive Issue Type: Improvement Reporter: Marquis Wang Assignee: Marquis Wang Attachments: 1746.prelim.patch Hive-1498 has support for IDXPROPERTIES on index creation, so now we want to support ALTERing those properties. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1501) when generating reentrant INSERT for index rebuild, quote identifiers using backticks
[ https://issues.apache.org/jira/browse/HIVE-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Skye Berghel updated HIVE-1501: --- Status: Patch Available (was: Open) when generating reentrant INSERT for index rebuild, quote identifiers using backticks - Key: HIVE-1501 URL: https://issues.apache.org/jira/browse/HIVE-1501 Project: Hive Issue Type: Bug Components: Indexing Affects Versions: 0.7.0 Reporter: John Sichi Assignee: Skye Berghel Fix For: 0.7.0 Attachments: 1501.patch, 1501_new_tests.patch, 1501_with_tests.patch Yongqiang, you mentioned that you weren't able to do this due to SORT BY not accepting them. The SORT BY is gone now as of HIVE-1494 (and SORT BY needs to be fixed anyway). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
