[jira] [Commented] (HIVE-2015) Eliminate bogus Datanucleus.Plugin Bundle ERROR log messages
[
https://issues.apache.org/jira/browse/HIVE-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13011155#comment-13011155
]
Andy Jefferson commented on HIVE-2015:
--
or just use recent DataNucleus (3.0Mx) which, by default, omits checks on OSGi
dependencies.
PS. if having such issues with third party software i'd expect people to go to
that third-party software and register an issue there to be able to turn
something off etc, rather than rely on that projects developers to just happen
across issues like this in a web trawl.
Eliminate bogus Datanucleus.Plugin Bundle ERROR log messages
Key: HIVE-2015
URL: https://issues.apache.org/jira/browse/HIVE-2015
Project: Hive
Issue Type: Bug
Components: Diagnosability, Metastore
Reporter: Carl Steinbach
Every time I start up the Hive CLI with logging enabled I'm treated to the
following ERROR log messages courtesy of DataNucleus:
{code}
DEBUG metastore.ObjectStore: datanucleus.plugin.pluginRegistryBundleCheck =
LOG
ERROR DataNucleus.Plugin: Bundle org.eclipse.jdt.core requires
org.eclipse.core.resources but it cannot be resolved.
ERROR DataNucleus.Plugin: Bundle org.eclipse.jdt.core requires
org.eclipse.core.runtime but it cannot be resolved.
ERROR DataNucleus.Plugin: Bundle org.eclipse.jdt.core requires
org.eclipse.text but it cannot be resolved.
{code}
Here's where this comes from:
* The bin/hive scripts cause Hive to inherit Hadoop's classpath.
* Hadoop's classpath includes $HADOOP_HOME/lib/core-3.1.1.jar, an Eclipse
library.
* core-3.1.1.jar includes a plugin.xml file defining an OSGI plugin
* At startup, Datanucleus scans the classpath looking for OSGI plugins, and
will attempt to initialize any that it finds, including the Eclipse OSGI
plugins located in core-3.1.1.jar
* Initialization of the OSGI plugin in core-3.1.1.jar fails because of
unresolved dependencies.
* We see an ERROR message telling us that Datanucleus failed to initialize a
plugin that we don't care about in the first place.
I can think of two options for solving this problem:
# Rewrite the scripts in $HIVE_HOME/bin so that they don't inherit ALL of
Hadoop's CLASSPATH.
# Replace DataNucleus's NOnManagedPluginRegistry with our own implementation
that does nothing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2076) Provide Metastore upgrade scripts and default schemas for PostgreSQL
[ https://issues.apache.org/jira/browse/HIVE-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13011182#comment-13011182 ] Yuanjun Li commented on HIVE-2076: -- This patch can be applied to HIVE-0.7 branch too. Can the fix version set [0.7.0,0.8.0] be better? Provide Metastore upgrade scripts and default schemas for PostgreSQL Key: HIVE-2076 URL: https://issues.apache.org/jira/browse/HIVE-2076 Project: Hive Issue Type: Task Components: Metastore Reporter: Carl Steinbach Assignee: Yuanjun Li Fix For: 0.8.0 Attachments: HIVE-2011.postgres.1.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.7.0-h0.20 #53
See https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/53/ -- [...truncated 26900 lines...] [junit] Loading data to table default.srcbucket2 [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/srcbucket23.txt' INTO TABLE srcbucket2 [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@srcbucket2 [junit] OK [junit] Copying file: https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' INTO TABLE src [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.src [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' INTO TABLE src [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src [junit] OK [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv3.txt' INTO TABLE src1 [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv3.txt [junit] Copying file: https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv3.txt [junit] Loading data to table default.src1 [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv3.txt' INTO TABLE src1 [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src1 [junit] OK [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.seq' INTO TABLE src_sequencefile [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.seq [junit] Copying file: https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.seq [junit] Loading data to table default.src_sequencefile [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.seq' INTO TABLE src_sequencefile [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src_sequencefile [junit] OK [junit] Copying file: https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/complex.seq [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/complex.seq' INTO TABLE src_thrift [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/complex.seq [junit] Loading data to table default.src_thrift [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/complex.seq' INTO TABLE src_thrift [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src_thrift [junit] OK [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/json.txt' INTO TABLE src_json [junit] PREHOOK: type: LOAD [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/json.txt [junit] Copying file: https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/json.txt [junit] Loading data to table default.src_json [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/json.txt' INTO TABLE src_json [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src_json [junit] OK [junit] diff https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/ql/test/logs/negative/wrong_distinct1.q.out https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/ql/src/test/results/compiler/errors/wrong_distinct1.q.out [junit] Done query: wrong_distinct1.q [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/ql/tmp/hive_job_log_hudson_201103251210_2092483662.txt [junit] Begin query: wrong_distinct2.q [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/ql/tmp/hive_job_log_hudson_201103251210_1076247552.txt [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://hudson.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' OVERWRITE INTO TABLE srcpart PARTITION (ds='2008-04-08',hr='11') [junit] PREHOOK: type: LOAD [junit] Copying data from
Build failed in Jenkins: Hive-trunk-h0.20 #642
See https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/642/ -- [...truncated 3 lines...] [junit] OK [junit] PREHOOK: query: select count(1) as cnt from testhivedrivertable [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-25_12-14-32_149_5296886701504161620/-mr-1 [junit] Total MapReduce jobs = 1 [junit] Launching Job 1 out of 1 [junit] Number of reduce tasks determined at compile time: 1 [junit] In order to change the average load for a reducer (in bytes): [junit] set hive.exec.reducers.bytes.per.reducer=number [junit] In order to limit the maximum number of reducers: [junit] set hive.exec.reducers.max=number [junit] In order to set a constant number of reducers: [junit] set mapred.reduce.tasks=number [junit] Job running in-process (local Hadoop) [junit] Hadoop job information for null: number of mappers: 0; number of reducers: 0 [junit] 2011-03-25 12:14:35,165 null map = 100%, reduce = 100% [junit] Ended Job = job_local_0001 [junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-25_12-14-32_149_5296886701504161620/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201103251214_383685430.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] PREHOOK: type: LOAD [junit] PREHOOK: Output: default@testhivedrivertable [junit] Copying data from https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.testhivedrivertable [junit] POSTHOOK: query: load data local inpath 'https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/data/files/kv1.txt' into table testhivedrivertable [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: select * from testhivedrivertable limit 10 [junit] PREHOOK: type: QUERY [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: file:/tmp/hudson/hive_2011-03-25_12-14-36_712_4202806813542269420/-mr-1 [junit] POSTHOOK: query: select * from testhivedrivertable limit 10 [junit] POSTHOOK: type: QUERY [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: file:/tmp/hudson/hive_2011-03-25_12-14-36_712_4202806813542269420/-mr-1 [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] PREHOOK: Input: default@testhivedrivertable [junit] PREHOOK: Output: default@testhivedrivertable [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] Hive history file=https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/service/tmp/hive_job_log_hudson_201103251214_1043212577.txt [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE [junit] POSTHOOK: query: drop table testhivedrivertable [junit] POSTHOOK: type: DROPTABLE [junit] OK [junit] PREHOOK: query: create table testhivedrivertable (num int) [junit] PREHOOK: type: CREATETABLE [junit] POSTHOOK: query: create table testhivedrivertable (num int) [junit] POSTHOOK: type: CREATETABLE [junit] POSTHOOK: Output: default@testhivedrivertable [junit] OK [junit] PREHOOK: query: drop table testhivedrivertable [junit] PREHOOK: type: DROPTABLE
[jira] [Commented] (HIVE-1803) Implement bitmap indexing in Hive
[ https://issues.apache.org/jira/browse/HIVE-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13011380#comment-13011380 ] John Sichi commented on HIVE-1803: -- Right, without row-level skipping, the main use case is AND/OR for block filtering. I'd suggest we get this committed without row-level skipping, and then create a followup for that. Besides AND/OR, having the bitmap index build/access code committed will be useful for others working on related issues such as automatic usage in the WHERE clause. Implement bitmap indexing in Hive - Key: HIVE-1803 URL: https://issues.apache.org/jira/browse/HIVE-1803 Project: Hive Issue Type: New Feature Components: Indexing Reporter: Marquis Wang Assignee: Marquis Wang Attachments: HIVE-1803.1.patch, HIVE-1803.2.patch, HIVE-1803.3.patch, HIVE-1803.4.patch, HIVE-1803.5.patch, HIVE-1803.6.patch, HIVE-1803.7.patch, JavaEWAH_20110304.zip, bitmap_index_1.png, bitmap_index_2.png, javaewah.jar, javaewah.jar Implement bitmap index handler to complement compact indexing. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Hive 0.7.0 Release Candidate 1
+1, ship it! JVS On Mar 20, 2011, at 2:32 AM, Carl Steinbach wrote: Hive 0.7.0 Release Candidate 1 is available here: http://people.apache.org/~cws/hive-0.7.0-candidate-1 We need 3 +1 votes from Hive PMC members in order to release. Please vote.
Re: Review Request: HIVE-2050. batch processing partition pruning process
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/522/ --- (Updated 2011-03-25 13:50:22.615065) Review request for hive. Changes --- The previous patch is too large due to thrift-generated files. This is a Java-only patch by removing all thrift-generated files. Summary --- Introducing a new metastore API to retrieve a list of partitions in batch. Diffs (updated) - trunk/metastore/if/hive_metastore.thrift 108 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 108 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 108 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 108 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 108 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 108 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 108 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartExprEvalUtils.java 108 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java 108 Diff: https://reviews.apache.org/r/522/diff Testing --- Thanks, Ning
[jira] [Updated] (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1434: - Status: Open (was: Patch Available) Some other things which need to be addressed: * Apache headers are missing on many new files * all commented-out code should be removed * new classes (e.g. CassandraStorageHandler) should have Javadoc (and for ones that have it, like CassandraQTestUtil, eliminate copy-and-paste evidence) * there is a file in the patch with the name cassandra-handler/src/test/results/cassandra_queries; I don't think it's supposed to be there (there should only be the .q.out file) For the HBase handler, there's a wiki page; it would be good to have one here too. Also, for HBase, we originally had some bugs with joins against tables with different schemas (and for joining HBase vs non-HBase tables), so you probably want to add some tests for those similar to the ones in hbase_queries.q and hbase_joins.q. Cassandra Storage Handler - Key: HIVE-1434 URL: https://issues.apache.org/jira/browse/HIVE-1434 Project: Hive Issue Type: New Feature Affects Versions: 0.7.0 Reporter: Edward Capriolo Assignee: Edward Capriolo Attachments: cas-handle.tar.gz, cass_handler.diff, hive-1434-1.txt, hive-1434-2-patch.txt, hive-1434-2011-02-26.patch.txt, hive-1434-2011-03-07.patch.txt, hive-1434-2011-03-07.patch.txt, hive-1434-2011-03-14.patch.txt, hive-1434-3-patch.txt, hive-1434-4-patch.txt, hive-1434-5.patch.txt, hive-1434.2011-02-27.diff.txt, hive-cassandra.2011-02-25.txt, hive.diff Add a cassandra storage handler. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Hive 0.7.0 Release Candidate 1
+1. Thanks Carl! On Mar 20, 2011, at 2:32 AM, Carl Steinbach wrote: Hive 0.7.0 Release Candidate 1 is available here: http://people.apache.org/~cws/hive-0.7.0-candidate-1 We need 3 +1 votes from Hive PMC members in order to release. Please vote.
Re: [VOTE] Hive 0.7.0 Release Candidate 1
+1 Thanks Carl! Ashish On Mar 25, 2011, at 3:08 PM, Ning Zhang wrote: +1. Thanks Carl! On Mar 20, 2011, at 2:32 AM, Carl Steinbach wrote: Hive 0.7.0 Release Candidate 1 is available here: http://people.apache.org/~cws/hive-0.7.0-candidate-1 We need 3 +1 votes from Hive PMC members in order to release. Please vote.
Re: [VOTE] Hive 0.7.0 Release Candidate 1
+1 - go ahead On 3/25/11 12:53 PM, John Sichi jsi...@fb.com wrote: +1, ship it! JVS On Mar 20, 2011, at 2:32 AM, Carl Steinbach wrote: Hive 0.7.0 Release Candidate 1 is available here: http://people.apache.org/~cws/hive-0.7.0-candidate-1 We need 3 +1 votes from Hive PMC members in order to release. Please vote.
[jira] [Commented] (HIVE-1095) Hive in Maven
[
https://issues.apache.org/jira/browse/HIVE-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13011514#comment-13011514
]
Giridharan Kesavan commented on HIVE-1095:
--
I got the same error as Amareshwari mentioned in the previous comment. But I
fixed locally by changing the dependency of make-pom target. ie made the
make-pom target to depend on check-ivy instead of ivy-init.
{code:xml}build-common.xml
- target name=make-pom if=ivy.present depends=ivy-init, jar
+ target name=make-pom if=ivy.present depends=check-ivy, jar
{code}
This seem to work fine..
About snapshot versioning:
I think we cannot publish snapshots to the staging/release repository.
Snapshots can only go the the snapshots repo.
We need to fix the version string so that we can publish to the staging/release
repo.
Hive in Maven
-
Key: HIVE-1095
URL: https://issues.apache.org/jira/browse/HIVE-1095
Project: Hive
Issue Type: Task
Components: Build Infrastructure
Affects Versions: 0.6.0
Reporter: Gerrit Jansen van Vuuren
Priority: Minor
Attachments: HIVE-1095-trunk.patch, HIVE-1095.v2.PATCH,
HIVE-1095.v3.PATCH, HIVE-1095.v4.PATCH, hiveReleasedToMaven.tar.gz
Getting hive into maven main repositories
Documentation on how to do this is on:
http://maven.apache.org/guides/mini/guide-central-repository-upload.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2050) batch processing partition pruning process
[ https://issues.apache.org/jira/browse/HIVE-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13011518#comment-13011518 ] Namit Jain commented on HIVE-2050: -- Based on an offline review, this may increase memory, we need to return the partition names periodically to put a memory bound batch processing partition pruning process -- Key: HIVE-2050 URL: https://issues.apache.org/jira/browse/HIVE-2050 Project: Hive Issue Type: Sub-task Reporter: Ning Zhang Assignee: Ning Zhang Attachments: HIVE-2050.patch For partition predicates that cannot be pushed down to JDO filtering (HIVE-2049), we should fall back to the old approach of listing all partition names first and use Hive's expression evaluation engine to select the correct partitions. Then the partition pruner should hand Hive a list of partition names and return a list of Partition Object (this should be added to the Hive API). A possible optimization is that the the partition pruner should give Hive a set of ranges of partition names (say [ts=01, ts=11], [ts=20, ts=24]), and the JDO query should be formulated as range queries. Range queries are possible because the first step list all partition names in sorted order. It's easy to come up with a range and it is guaranteed that the JDO range query results should be equivalent to the query with a list of partition names. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2050) batch processing partition pruning process
[ https://issues.apache.org/jira/browse/HIVE-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2050: - Status: Open (was: Patch Available) batch processing partition pruning process -- Key: HIVE-2050 URL: https://issues.apache.org/jira/browse/HIVE-2050 Project: Hive Issue Type: Sub-task Reporter: Ning Zhang Assignee: Ning Zhang Attachments: HIVE-2050.patch For partition predicates that cannot be pushed down to JDO filtering (HIVE-2049), we should fall back to the old approach of listing all partition names first and use Hive's expression evaluation engine to select the correct partitions. Then the partition pruner should hand Hive a list of partition names and return a list of Partition Object (this should be added to the Hive API). A possible optimization is that the the partition pruner should give Hive a set of ranges of partition names (say [ts=01, ts=11], [ts=20, ts=24]), and the JDO query should be formulated as range queries. Range queries are possible because the first step list all partition names in sorted order. It's easy to come up with a range and it is guaranteed that the JDO range query results should be equivalent to the query with a list of partition names. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1095) Hive in Maven
[ https://issues.apache.org/jira/browse/HIVE-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13011521#comment-13011521 ] Giridharan Kesavan commented on HIVE-1095: -- someone with hive committer access should try this to see if we are able to publish to the nexus staging repo. ant make-maven -Dversion=0.8.0 ant maven-publish -Dversion=0.8.0 Hive in Maven - Key: HIVE-1095 URL: https://issues.apache.org/jira/browse/HIVE-1095 Project: Hive Issue Type: Task Components: Build Infrastructure Affects Versions: 0.6.0 Reporter: Gerrit Jansen van Vuuren Priority: Minor Attachments: HIVE-1095-trunk.patch, HIVE-1095.v2.PATCH, HIVE-1095.v3.PATCH, HIVE-1095.v4.PATCH, hiveReleasedToMaven.tar.gz Getting hive into maven main repositories Documentation on how to do this is on: http://maven.apache.org/guides/mini/guide-central-repository-upload.html -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1095) Hive in Maven
[ https://issues.apache.org/jira/browse/HIVE-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13011527#comment-13011527 ] Carl Steinbach commented on HIVE-1095: -- @Giridharan: 0.8.0 is the current trunk development branch. We haven't voted to release this version yet, so we shouldn't publish anything other than 0.8.0 SNAPSHOTs to the snapshot repository. If you want to publish artifacts to the release repo then we need to backport this patch to branch-0.7. Hive in Maven - Key: HIVE-1095 URL: https://issues.apache.org/jira/browse/HIVE-1095 Project: Hive Issue Type: Task Components: Build Infrastructure Affects Versions: 0.6.0 Reporter: Gerrit Jansen van Vuuren Priority: Minor Attachments: HIVE-1095-trunk.patch, HIVE-1095.v2.PATCH, HIVE-1095.v3.PATCH, HIVE-1095.v4.PATCH, hiveReleasedToMaven.tar.gz Getting hive into maven main repositories Documentation on how to do this is on: http://maven.apache.org/guides/mini/guide-central-repository-upload.html -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1988) Make the delegation token issued by the MetaStore owned by the right user
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HIVE-1988: -- Fix Version/s: 0.8.0 Status: Patch Available (was: Open) Make the delegation token issued by the MetaStore owned by the right user - Key: HIVE-1988 URL: https://issues.apache.org/jira/browse/HIVE-1988 Project: Hive Issue Type: Bug Components: Metastore, Security, Server Infrastructure Affects Versions: 0.7.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.8.0 Attachments: hive-1988-3.patch, hive-1988.patch The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner. Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1988) Make the delegation token issued by the MetaStore owned by the right user
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HIVE-1988: -- Attachment: hive-1988-3.patch https://reviews.apache.org/r/528/ is the reviewboard page. Make the delegation token issued by the MetaStore owned by the right user - Key: HIVE-1988 URL: https://issues.apache.org/jira/browse/HIVE-1988 Project: Hive Issue Type: Bug Components: Metastore, Security, Server Infrastructure Affects Versions: 0.7.0 Reporter: Devaraj Das Fix For: 0.8.0 Attachments: hive-1988-3.patch, hive-1988.patch The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner. Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-1988) Make the delegation token issued by the MetaStore owned by the right user
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das reassigned HIVE-1988: - Assignee: Devaraj Das Make the delegation token issued by the MetaStore owned by the right user - Key: HIVE-1988 URL: https://issues.apache.org/jira/browse/HIVE-1988 Project: Hive Issue Type: Bug Components: Metastore, Security, Server Infrastructure Affects Versions: 0.7.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.8.0 Attachments: hive-1988-3.patch, hive-1988.patch The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner. Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2077) Allow HBaseStorageHandler to work with hbase 0.90.1
Allow HBaseStorageHandler to work with hbase 0.90.1 --- Key: HIVE-2077 URL: https://issues.apache.org/jira/browse/HIVE-2077 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.7.0 Reporter: Ted Yu Fix For: 0.8.0 Currently HBase handler works with hbase 0.89 We should make it work with 0.90.1 as well. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2077) Allow HBaseStorageHandler to work with hbase 0.90.1
[ https://issues.apache.org/jira/browse/HIVE-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HIVE-2077: - Description: Currently HBase handler works with hbase 0.89 We should make it work with 0.90.1 and utilize new features of 0.90.1 was: Currently HBase handler works with hbase 0.89 We should make it work with 0.90.1 as well. Allow HBaseStorageHandler to work with hbase 0.90.1 --- Key: HIVE-2077 URL: https://issues.apache.org/jira/browse/HIVE-2077 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.7.0 Reporter: Ted Yu Fix For: 0.8.0 Currently HBase handler works with hbase 0.89 We should make it work with 0.90.1 and utilize new features of 0.90.1 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
