[jira] [Updated] (HIVE-2214) CommandNeedRetryException.java is missing ASF header
[ https://issues.apache.org/jira/browse/HIVE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2214: --- Attachment: hive-2214.patch I think this should be applied to 0.8 branch too before release because of compliance reason. CommandNeedRetryException.java is missing ASF header Key: HIVE-2214 URL: https://issues.apache.org/jira/browse/HIVE-2214 Project: Hive Issue Type: Bug Reporter: John Sichi Assignee: Siying Dong Attachments: hive-2214.patch Please add one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2547) Tiny bug in init-hive-dfs.sh
[ https://issues.apache.org/jira/browse/HIVE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142948#comment-13142948 ] Stephen Boesch commented on HIVE-2547: -- I am a simple user, not even a contributor. Would a contributor or committer pls review this and decide whether to make the requested change on line 73 of init-hive-dfs.sh (or not) and then close this bug. thx! Tiny bug in init-hive-dfs.sh - Key: HIVE-2547 URL: https://issues.apache.org/jira/browse/HIVE-2547 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.8.0 Environment: ubuntu / general linux Reporter: Stephen Boesch Priority: Minor Labels: initialization Fix For: 0.8.0 Original Estimate: 5m Remaining Estimate: 5m init-hive-dfs.sh seems to have a small typo on line 73 in which it requests $HADOOP instead of $HADOOP_EXEC -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2547) Tiny bug in init-hive-dfs.sh
[
https://issues.apache.org/jira/browse/HIVE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Boesch updated HIVE-2547:
-
Status: Patch Available (was: Open)
I'm trying to submit a proposed patch, but there is no place to attach a file.
So here is the patch itself
Index: bin/init-hive-dfs.sh
===
--- bin/init-hive-dfs.sh(Revision 1196932)
+++ bin/init-hive-dfs.sh(Arbeitskopie)
@@ -69,7 +69,7 @@
fi
HADOOP_EXEC=$HADOOP_HOME/bin/hadoop
-if [ ! -f ${HADOOP} ]; then
+if [ ! -f ${HADOOP_EXEC} ]; then
echo Cannot find hadoop installation: \$HADOOP_HOME must be set or hadoop
must be in the path;
exit 4;
fi
Tiny bug in init-hive-dfs.sh
-
Key: HIVE-2547
URL: https://issues.apache.org/jira/browse/HIVE-2547
Project: Hive
Issue Type: Bug
Components: CLI
Affects Versions: 0.8.0
Environment: ubuntu / general linux
Reporter: Stephen Boesch
Priority: Minor
Labels: initialization
Fix For: 0.8.0
Original Estimate: 5m
Remaining Estimate: 5m
init-hive-dfs.sh seems to have a small typo on line 73 in which it requests
$HADOOP instead of $HADOOP_EXEC
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HIVE-2544) Nullpointer on registering udfs.
[ https://issues.apache.org/jira/browse/HIVE-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-2544 started by Bennie Schut. Nullpointer on registering udfs. Key: HIVE-2544 URL: https://issues.apache.org/jira/browse/HIVE-2544 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Bennie Schut Assignee: Bennie Schut Attachments: HIVE-2544.1.patch.txt Currently the Function registry can throw NullPointers when multiple threads are trying to register the same function. The normal put() will replace the existing registered function object even if it's exactly the same function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: Log related Check style Comments fixes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1978/ --- (Updated 2011-11-03 14:59:09.059591) Review request for hive, Carl Steinbach and John Sichi. Changes --- Patch become stale so rebased the patch Summary --- Fixed Log related Check style Comments like including root exception and adding messages. This addresses bug HIVE-2178. https://issues.apache.org/jira/browse/HIVE-2178 Diffs (updated) - trunk/jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveConnection.java 1197152 trunk/jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveDataSource.java 1197152 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1197152 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java 1197152 trunk/serde/src/java/org/apache/hadoop/hive/serde2/dynamic_type/SimpleCharStream.java 1197152 trunk/serde/src/java/org/apache/hadoop/hive/serde2/thrift/TBinarySortableProtocol.java 1197152 Diff: https://reviews.apache.org/r/1978/diff Testing --- Ran all unit tests Thanks, chinna
[jira] [Updated] (HIVE-2017) Driver.execute() should maintaining SessionState in case of runtime errors
[
https://issues.apache.org/jira/browse/HIVE-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinna Rao Lalam updated HIVE-2017:
---
Attachment: HIVE-2017.4.patch
Driver.execute() should maintaining SessionState in case of runtime errors
--
Key: HIVE-2017
URL: https://issues.apache.org/jira/browse/HIVE-2017
Project: Hive
Issue Type: Bug
Reporter: Ning Zhang
Assignee: Chinna Rao Lalam
Attachments: HIVE-2017.1.patch, HIVE-2017.2.patch, HIVE-2017.3.patch,
HIVE-2017.4.patch
Here's a snippet from Driver.execute():
{code}
// TODO: This error messaging is not very informative. Fix that.
errorMessage = FAILED: Execution Error, return code + exitVal
+ from
+ tsk.getClass().getName();
SQLState = 08S01;
console.printError(errorMessage);
if (running.size() != 0) {
taskCleanup();
}
return 9;
{code}
I simply returned in case of runtime errors without maintaining SessionState.
It could cause resource leak mentioned in HIVE-1959.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2017) Driver.execute() should maintaining SessionState in case of runtime errors
[
https://issues.apache.org/jira/browse/HIVE-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143263#comment-13143263
]
jirapos...@reviews.apache.org commented on HIVE-2017:
-
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1609/
---
(Updated 2011-11-03 16:01:16.713162)
Review request for hive, John Sichi and Ning Zhang.
Changes
---
Patch becomes stale so rebased
Summary
---
while processing two parallel tasks if one of the task fails the
Driver.taskCleanup() will call system.exit() this will shutdown the jvm so it
is replaced with the logic to stop the remaining tasks.. Here need to cleanup 2
kinds of tasks one is non-mr tasks and mr tasks. For stopping non-mr tasks
used the thread.interrupt() because every non-mr task will be executed as a
thread and for mr tasks maintained a variable called jobKillUri's this variable
will track the spawned job kill uri, in taskCleanup() if it is a mr task using
this variable kill the job.
This addresses bug HIVE-2017.
https://issues.apache.org/jira/browse/HIVE-2017
Diffs (updated)
-
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1197183
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1197183
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java
1197183
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1197183
trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java
1197183
trunk/ql/src/test/queries/clientnegative/alter_exit.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/alter_exit.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1609/diff
Testing
---
Added test case for this scenario. Ran existing test cases
Thanks,
chinna
Driver.execute() should maintaining SessionState in case of runtime errors
--
Key: HIVE-2017
URL: https://issues.apache.org/jira/browse/HIVE-2017
Project: Hive
Issue Type: Bug
Reporter: Ning Zhang
Assignee: Chinna Rao Lalam
Attachments: HIVE-2017.1.patch, HIVE-2017.2.patch, HIVE-2017.3.patch,
HIVE-2017.4.patch
Here's a snippet from Driver.execute():
{code}
// TODO: This error messaging is not very informative. Fix that.
errorMessage = FAILED: Execution Error, return code + exitVal
+ from
+ tsk.getClass().getName();
SQLState = 08S01;
console.printError(errorMessage);
if (running.size() != 0) {
taskCleanup();
}
return 9;
{code}
I simply returned in case of runtime errors without maintaining SessionState.
It could cause resource leak mentioned in HIVE-1959.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2465) Primitive Data Types returning null if the data is out of range of the data type.
[ https://issues.apache.org/jira/browse/HIVE-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2465: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Chinna Primitive Data Types returning null if the data is out of range of the data type. - Key: HIVE-2465 URL: https://issues.apache.org/jira/browse/HIVE-2465 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Environment: Hadoop 0.20.1, Hive0.9.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5) Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-2465.1.patch, HIVE-2465.2.patch, HIVE-2465.patch Primitive Data Types returning null if the input data is out of range of the data type. In this case it is better to log the message with the proper message and actual data then user get to know some data is missing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2533) test load_fs.q failing
[ https://issues.apache.org/jira/browse/HIVE-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain resolved HIVE-2533. -- Resolution: Not A Problem Thanks John, that was the problem test load_fs.q failing -- Key: HIVE-2533 URL: https://issues.apache.org/jira/browse/HIVE-2533 Project: Hive Issue Type: Bug Reporter: Namit Jain -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-11) better error code from Hive describe command
[ https://issues.apache.org/jira/browse/HIVE-11?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-11: --- Status: Open (was: Patch Available) Existing negative test cases external1.q and external2.q are now failing; they need .q.out updates. better error code from Hive describe command Key: HIVE-11 URL: https://issues.apache.org/jira/browse/HIVE-11 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.6.0, 0.5.0, 0.4.1, 0.4.0, 0.3.0, 0.8.0, 0.9.0 Reporter: Joydeep Sen Sarma Assignee: Ashutosh Chauhan Fix For: 0.9.0 Attachments: hive-11.patch, hive-11_1.patch cryptic, non-informative error message hive describe hive1_scribeloadertest FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask in this case the table was missing. better say that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2548) How to submit documentation fixes
[ https://issues.apache.org/jira/browse/HIVE-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143444#comment-13143444 ] John Sichi commented on HIVE-2548: -- Create an account in the wiki, and let me know the username; then I'll grant you edit privileges and you can fix it. How to submit documentation fixes - Key: HIVE-2548 URL: https://issues.apache.org/jira/browse/HIVE-2548 Project: Hive Issue Type: Bug Components: Documentation Affects Versions: 0.7.1 Environment: general linux Reporter: Stephen Boesch Assignee: Stephen Boesch Priority: Minor Fix For: 0.8.0 Original Estimate: 1h Remaining Estimate: 1h I am walking through the developer's guide and tutorial and finding issues: e.g. broken links. Is there a way to try out updates to the docs and submit patches? Here is the first example on https://cwiki.apache.org/Hive/tutorial.html The following examples highlight some salient features of the system. A detailed set of query test cases can be found at Hive Query Test Cases and the corresponding results can be found at Query Test Case Results. The first link is listed as http://svn.apache.org/viewvc/hadoop/hive/trunk/ql/src/test/queries/clientpositive/ Second link is http://svn.apache.org/viewvc/hadoop/hive/trunk/ql/src/test/results/clientpositive/ Both links are 404's -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2548) How to submit documentation fixes
[ https://issues.apache.org/jira/browse/HIVE-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi reassigned HIVE-2548: Assignee: Stephen Boesch How to submit documentation fixes - Key: HIVE-2548 URL: https://issues.apache.org/jira/browse/HIVE-2548 Project: Hive Issue Type: Bug Components: Documentation Affects Versions: 0.7.1 Environment: general linux Reporter: Stephen Boesch Assignee: Stephen Boesch Priority: Minor Fix For: 0.8.0 Original Estimate: 1h Remaining Estimate: 1h I am walking through the developer's guide and tutorial and finding issues: e.g. broken links. Is there a way to try out updates to the docs and submit patches? Here is the first example on https://cwiki.apache.org/Hive/tutorial.html The following examples highlight some salient features of the system. A detailed set of query test cases can be found at Hive Query Test Cases and the corresponding results can be found at Query Test Case Results. The first link is listed as http://svn.apache.org/viewvc/hadoop/hive/trunk/ql/src/test/queries/clientpositive/ Second link is http://svn.apache.org/viewvc/hadoop/hive/trunk/ql/src/test/results/clientpositive/ Both links are 404's -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2547) Tiny bug in init-hive-dfs.sh
[ https://issues.apache.org/jira/browse/HIVE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi reassigned HIVE-2547: Assignee: Stephen Boesch Tiny bug in init-hive-dfs.sh - Key: HIVE-2547 URL: https://issues.apache.org/jira/browse/HIVE-2547 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.8.0 Environment: ubuntu / general linux Reporter: Stephen Boesch Assignee: Stephen Boesch Priority: Minor Labels: initialization Fix For: 0.8.0 Original Estimate: 5m Remaining Estimate: 5m init-hive-dfs.sh seems to have a small typo on line 73 in which it requests $HADOOP instead of $HADOOP_EXEC -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2547) Tiny bug in init-hive-dfs.sh
[ https://issues.apache.org/jira/browse/HIVE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143447#comment-13143447 ] John Sichi commented on HIVE-2547: -- Here are the instructions for submitting patches: https://cwiki.apache.org/confluence/display/Hive/PhabricatorCodeReview (Also, Attach Files is now under More Actions above in JIRA.) Tiny bug in init-hive-dfs.sh - Key: HIVE-2547 URL: https://issues.apache.org/jira/browse/HIVE-2547 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.8.0 Environment: ubuntu / general linux Reporter: Stephen Boesch Assignee: Stephen Boesch Priority: Minor Labels: initialization Fix For: 0.8.0 Original Estimate: 5m Remaining Estimate: 5m init-hive-dfs.sh seems to have a small typo on line 73 in which it requests $HADOOP instead of $HADOOP_EXEC -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2214) CommandNeedRetryException.java is missing ASF header
[ https://issues.apache.org/jira/browse/HIVE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143451#comment-13143451 ] John Sichi commented on HIVE-2214: -- +1, will commit. CommandNeedRetryException.java is missing ASF header Key: HIVE-2214 URL: https://issues.apache.org/jira/browse/HIVE-2214 Project: Hive Issue Type: Bug Affects Versions: 0.8.0 Reporter: John Sichi Assignee: Siying Dong Fix For: 0.8.0 Attachments: hive-2214.patch Please add one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2548) How to submit documentation fixes
[ https://issues.apache.org/jira/browse/HIVE-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143453#comment-13143453 ] Stephen Boesch commented on HIVE-2548: -- ok done: javadba. thx! 2011/11/3 John Sichi (Commented) (JIRA) j...@apache.org How to submit documentation fixes - Key: HIVE-2548 URL: https://issues.apache.org/jira/browse/HIVE-2548 Project: Hive Issue Type: Bug Components: Documentation Affects Versions: 0.7.1 Environment: general linux Reporter: Stephen Boesch Assignee: Stephen Boesch Priority: Minor Fix For: 0.8.0 Original Estimate: 1h Remaining Estimate: 1h I am walking through the developer's guide and tutorial and finding issues: e.g. broken links. Is there a way to try out updates to the docs and submit patches? Here is the first example on https://cwiki.apache.org/Hive/tutorial.html The following examples highlight some salient features of the system. A detailed set of query test cases can be found at Hive Query Test Cases and the corresponding results can be found at Query Test Case Results. The first link is listed as http://svn.apache.org/viewvc/hadoop/hive/trunk/ql/src/test/queries/clientpositive/ Second link is http://svn.apache.org/viewvc/hadoop/hive/trunk/ql/src/test/results/clientpositive/ Both links are 404's -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2278) Support archiving for multiple partitions if the table is partitioned by multiple columns
[ https://issues.apache.org/jira/browse/HIVE-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-2278: --- Attachment: HIVE-2278.12.patch fixed some .q.out diffs Support archiving for multiple partitions if the table is partitioned by multiple columns - Key: HIVE-2278 URL: https://issues.apache.org/jira/browse/HIVE-2278 Project: Hive Issue Type: New Feature Reporter: Namit Jain Assignee: Marcin Kurczych Attachments: HIVE-2278.10.patch, HIVE-2278.11.patch, HIVE-2278.12.patch, HIVE-2278.2.patch, HIVE-2278.3.patch, HIVE-2278.4.patch, HIVE-2278.5.patch, HIVE-2278.5.patch, HIVE-2278.6.patch, HIVE-2278.6.patch, HIVE-2278.7.patch, HIVE-2278.8.patch, HIVE-2278.9.patch, archive_corrupt.rc, hive.2278.1.patch If a table is partitioned by ds,hr it should be possible to archive all the files in ds to reduce the number of files -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1496) enhance CREATE INDEX to support immediate index build
[ https://issues.apache.org/jira/browse/HIVE-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143475#comment-13143475 ] John Sichi commented on HIVE-1496: -- Oof. That sounds too painful+brittle. What if we instead prepare a reentrant CREATE TABLE AS SELECT statement (rather than an INSERT statement), but then splice out the CREATE TABLE part (leaving just the load part). Still very hacky, but maybe not too much. enhance CREATE INDEX to support immediate index build - Key: HIVE-1496 URL: https://issues.apache.org/jira/browse/HIVE-1496 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.7.0, 0.8.0 Reporter: John Sichi Assignee: Syed S. Albiz Attachments: hive-1496.patch Currently we only support WITH DEFERRED REBUILD. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2472) Metastore statistics are not being updated for CTAS queries.
[ https://issues.apache.org/jira/browse/HIVE-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143477#comment-13143477 ] jirapos...@reviews.apache.org commented on HIVE-2472: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2583/ --- (Updated 2011-11-03 19:46:23.842574) Review request for Ning Zhang and Kevin Wilfong. Summary --- Now table stats will be collected for CTAS queries. This addresses bug HIVE-2472. https://issues.apache.org/jira/browse/HIVE-2472 Diffs - trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/StatsTask.java 1196269 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java 1196269 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1196269 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1196269 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 1196269 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 1196269 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 1196269 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/StatsWork.java 1196269 trunk/ql/src/test/results/clientpositive/ctas.q.out 1196269 trunk/ql/src/test/results/clientpositive/database.q.out 1196269 trunk/ql/src/test/results/clientpositive/merge3.q.out 1196269 trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out 1196269 trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1196269 Diff: https://reviews.apache.org/r/2583/diff Testing --- run ant tests with overwrite option, changes to out files are part of the diff Thanks, Robert Metastore statistics are not being updated for CTAS queries. Key: HIVE-2472 URL: https://issues.apache.org/jira/browse/HIVE-2472 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Robert Surówka Attachments: HIVE-2472.1.patch.txt, HIVE-2472.2.patch We need to add a Statistics task at the end of a CTAS query in order to update the metastore statistics for the table being created. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2472) Metastore statistics are not being updated for CTAS queries.
[ https://issues.apache.org/jira/browse/HIVE-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Surówka updated HIVE-2472: - Attachment: HIVE-2472.2.patch Metastore statistics are not being updated for CTAS queries. Key: HIVE-2472 URL: https://issues.apache.org/jira/browse/HIVE-2472 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Robert Surówka Attachments: HIVE-2472.1.patch.txt, HIVE-2472.2.patch We need to add a Statistics task at the end of a CTAS query in order to update the metastore statistics for the table being created. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2472) Metastore statistics are not being updated for CTAS queries.
[ https://issues.apache.org/jira/browse/HIVE-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143478#comment-13143478 ] Robert Surówka commented on HIVE-2472: -- The diff for HIVE-2472.2.patch can be found at https://reviews.apache.org/r/2583/diff/#index_header (last version). Metastore statistics are not being updated for CTAS queries. Key: HIVE-2472 URL: https://issues.apache.org/jira/browse/HIVE-2472 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Robert Surówka Attachments: HIVE-2472.1.patch.txt, HIVE-2472.2.patch We need to add a Statistics task at the end of a CTAS query in order to update the metastore statistics for the table being created. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.0-SNAPSHOT-h0.21 - Build # 79 - Failure
Changes for Build #79 1 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:47) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:7852) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:154) at junit.framework.TestCase.runBare(TestCase.java:127) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) The Apache Jenkins build system has built Hive-0.8.0-SNAPSHOT-h0.21 (build #79) Status: Failure Check console output at https://builds.apache.org/job/Hive-0.8.0-SNAPSHOT-h0.21/79/ to view the results.
[jira] [Assigned] (HIVE-2467) HA Support for Metastore Server
[ https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi reassigned HIVE-2467: Assignee: Thomas Weise HA Support for Metastore Server Key: HIVE-2467 URL: https://issues.apache.org/jira/browse/HIVE-2467 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.8.0, 0.9.0 Reporter: Thomas Weise Assignee: Thomas Weise Fix For: 0.9.0 Attachments: HIVE-2467.patch We require HA deployment for metastore server for HCatalog: * Multiple server instances run behind VIP * Database provides HA Metastore server instances will need to be able to share any state required for VIP outside RDBMS. As of Hive 0.8 affected conversational state that needs to support VIP/HA setup is limited to current delegation tokens. Is this correct? We are planning to use ZooKeeper to share current delegation tokens and master keys between nodes of the VIP. ZK is already (optionally) used by Hive for concurrency control. Access to ZK would be limited on the network level or in the future, when ZooKeeper supports security, through Kerberos, similar to NN access. Currently Hive taps into Hadoop core security delegation token support through extension of org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManagerTokenIdent A solution could amend the Hive specific extension to support: * Pluggable delegation token and master key store (ZooKeeper as alternative for in-memory AbstractDelegationTokenSecretManager) * Delegation token retrieval from token store when not found in memory (wrap/extend retrievePassword(...)) * Cancellation of token in token store * Purging of expired tokens from token store http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2214) CommandNeedRetryException.java is missing ASF header
[ https://issues.apache.org/jira/browse/HIVE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143594#comment-13143594 ] Hudson commented on HIVE-2214: -- Integrated in Hive-trunk-h0.21 #1056 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1056/]) HIVE-2214. CommandNeedRetryException.java is missing ASF header (Ashutosh Chauhan via jvs) jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1197276 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/CommandNeedRetryException.java CommandNeedRetryException.java is missing ASF header Key: HIVE-2214 URL: https://issues.apache.org/jira/browse/HIVE-2214 Project: Hive Issue Type: Bug Affects Versions: 0.8.0 Reporter: John Sichi Assignee: Ashutosh Chauhan Fix For: 0.8.0 Attachments: hive-2214.patch Please add one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2465) Primitive Data Types returning null if the data is out of range of the data type.
[ https://issues.apache.org/jira/browse/HIVE-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143595#comment-13143595 ] Hudson commented on HIVE-2465: -- Integrated in Hive-trunk-h0.21 #1056 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1056/]) HIVE-2465 Primitive Data Types returning null if the data is out of range of the data type. (Ashutosh Chauhan via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1197227 Files : * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBoolean.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyByte.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyDouble.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFloat.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyInteger.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyLong.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyPrimitive.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyShort.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java * /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java Primitive Data Types returning null if the data is out of range of the data type. - Key: HIVE-2465 URL: https://issues.apache.org/jira/browse/HIVE-2465 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Environment: Hadoop 0.20.1, Hive0.9.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5) Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-2465.1.patch, HIVE-2465.2.patch, HIVE-2465.patch Primitive Data Types returning null if the input data is out of range of the data type. In this case it is better to log the message with the proper message and actual data then user get to know some data is missing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2549) Support standard cross join syntax
Support standard cross join syntax
--
Key: HIVE-2549
URL: https://issues.apache.org/jira/browse/HIVE-2549
Project: Hive
Issue Type: Bug
Reporter: David Phillips
Hive should support standard (ANSI) cross join syntax:
{code}
SELECT a.*, b.*
FROM a
CROSS JOIN b
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2366) Metastore upgrade scripts for HIVE-2246 do not migrate indexes nor rename the old COLUMNS table
[ https://issues.apache.org/jira/browse/HIVE-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143630#comment-13143630 ] Paul Yang commented on HIVE-2366: - I regenerated the patch based on trunk. Looks good to me as well. Metastore upgrade scripts for HIVE-2246 do not migrate indexes nor rename the old COLUMNS table --- Key: HIVE-2366 URL: https://issues.apache.org/jira/browse/HIVE-2366 Project: Hive Issue Type: Bug Components: Metastore Reporter: Sohan Jain Assignee: Sohan Jain Attachments: HIVE-2366.1.patch, HIVE-2366.2.patch The upgrade scripts for the hive metastore in HIVE-2246 do not upgrade the indexes. They also need to rename the old COLUMNS table after migration so that old clients will not accidentally access the COLUMNS table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: Use sorted nature of compact indexes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2605/ --- (Updated 2011-11-04 00:42:51.180303) Review request for hive, Yongqiang He, Ning Zhang, and namit jain. Changes --- As Namit suggested, I merged HiveBinarySearchRecordReader and HiveContextAwareRecordReader into a single class, as it seemed the binary search is closely tied to the context. I also realized while doing this that I could update BucketizedHiveRecordReader to support binary search as well. I updated the tests, reran the tests, and also ran tests to verify BucketizedHiveRecordReader was not broken. Summary --- The CompactIndexHandler determines if the reentrant query it creates is a candidate for using the fact the index is sorted (it has an appropriate number of non-partition conditions, and the query plan is of the form expected). It sets the input format to HiveSortedInputFormat, and marks the FilterOperator for the non-partition condition. The HiveSortedInputFormat is extends HiveInputFormat, so its splits consist of data from a single file, and its record reader is HiveBinarySearchRecordReader. HiveBinarySearchRecordReader starts by assuming it is performing a binary search. It sets the appropriate flags in IOContext, which acts as the means of communication between the FilterOperators and the record reader. The non-partition FilterOperator is responsible for executing a comparison between the value in the row and column of interest and the constant. It also provides the type of the generic UDF. It sets this data in the IOContext. As long as the binary search continues the FilterOperators do not forward rows to the operators below them. The record reader uses the comparison and the type of the generic UDF to execute a binary search on the underlying RCFile until it finds the block of interest, or determines that if any block is of interest it is the last one. The search then proceeds linearly from the beginning of the identified block. If ever in the binary search a problem occurs, like the comparison fails for some reason, a linear search begins from the beginning of the data which has yet to be eliminated. Regardless of whether or not a binary search is performed, the record reader attempts to end the linear search as soon as it can based on the comparison and the type of the generic UDF. This addresses bug HIVE-2535. https://issues.apache.org/jira/browse/HIVE-2535 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1183507 trunk/conf/hive-default.xml 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/IOContext.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFileRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseCompare.java 1183507 trunk/ql/src/test/org/apache/hadoop/hive/ql/hooks/VerifyHiveSortedInputFormatUsedHook.java PRE-CREATION trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestHiveBinarySearchRecordReader.java PRE-CREATION trunk/ql/src/test/queries/clientpositive/index_compact_binary_search.q PRE-CREATION trunk/ql/src/test/results/clientpositive/index_compact_binary_search.q.out PRE-CREATION Diff: https://reviews.apache.org/r/2605/diff Testing --- I added a test to verify the functionality of the HiveBinarySearchRecordReader. I also added a .q file to test that this returns the correct results when the underlying index is stored in an RCFile and when it is stored in as a text file, with all of the supported operators. I ran the .q files to verify they still pass. I ran some queries to verify there was a CPU benefit to doing this. I saw as much as a 45% reduction in the total CPU used by the map reduce job to scan the index, for a large data set. Thanks, Kevin
[jira] [Commented] (HIVE-2535) Use sorted nature of compact indexes
[ https://issues.apache.org/jira/browse/HIVE-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143656#comment-13143656 ] jirapos...@reviews.apache.org commented on HIVE-2535: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2605/ --- (Updated 2011-11-04 00:42:51.180303) Review request for hive, Yongqiang He, Ning Zhang, and namit jain. Changes --- As Namit suggested, I merged HiveBinarySearchRecordReader and HiveContextAwareRecordReader into a single class, as it seemed the binary search is closely tied to the context. I also realized while doing this that I could update BucketizedHiveRecordReader to support binary search as well. I updated the tests, reran the tests, and also ran tests to verify BucketizedHiveRecordReader was not broken. Summary --- The CompactIndexHandler determines if the reentrant query it creates is a candidate for using the fact the index is sorted (it has an appropriate number of non-partition conditions, and the query plan is of the form expected). It sets the input format to HiveSortedInputFormat, and marks the FilterOperator for the non-partition condition. The HiveSortedInputFormat is extends HiveInputFormat, so its splits consist of data from a single file, and its record reader is HiveBinarySearchRecordReader. HiveBinarySearchRecordReader starts by assuming it is performing a binary search. It sets the appropriate flags in IOContext, which acts as the means of communication between the FilterOperators and the record reader. The non-partition FilterOperator is responsible for executing a comparison between the value in the row and column of interest and the constant. It also provides the type of the generic UDF. It sets this data in the IOContext. As long as the binary search continues the FilterOperators do not forward rows to the operators below them. The record reader uses the comparison and the type of the generic UDF to execute a binary search on the underlying RCFile until it finds the block of interest, or determines that if any block is of interest it is the last one. The search then proceeds linearly from the beginning of the identified block. If ever in the binary search a problem occurs, like the comparison fails for some reason, a linear search begins from the beginning of the data which has yet to be eliminated. Regardless of whether or not a binary search is performed, the record reader attempts to end the linear search as soon as it can based on the comparison and the type of the generic UDF. This addresses bug HIVE-2535. https://issues.apache.org/jira/browse/HIVE-2535 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1183507 trunk/conf/hive-default.xml 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/IOContext.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFileRecordReader.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 1183507 trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseCompare.java 1183507 trunk/ql/src/test/org/apache/hadoop/hive/ql/hooks/VerifyHiveSortedInputFormatUsedHook.java PRE-CREATION trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestHiveBinarySearchRecordReader.java PRE-CREATION trunk/ql/src/test/queries/clientpositive/index_compact_binary_search.q PRE-CREATION trunk/ql/src/test/results/clientpositive/index_compact_binary_search.q.out PRE-CREATION Diff: https://reviews.apache.org/r/2605/diff Testing --- I added a test to verify the functionality of the HiveBinarySearchRecordReader. I also added a .q file to test that this returns the correct results when the underlying index is stored in an RCFile and when it is stored in as a text file, with all of the
[jira] [Updated] (HIVE-2535) Use sorted nature of compact indexes
[ https://issues.apache.org/jira/browse/HIVE-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2535: Attachment: HIVE-2535.3.patch.txt Use sorted nature of compact indexes Key: HIVE-2535 URL: https://issues.apache.org/jira/browse/HIVE-2535 Project: Hive Issue Type: Improvement Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2535.1.patch.txt, HIVE-2535.2.patch.txt, HIVE-2535.3.patch.txt Compact indexes are sorted based on the indexed columns, but we are not using this fact when we access the index. To start with, if the index is stored as an RC file, and if the predicate being used to access the index consists of only one non-partition condition using one of the operators ,=,,=,= we could use a binary search (if necessary) to find the block to begin scanning for unfiltered rows, and we could use the result of comparing the value in the column with the constant (this is necessarily the form of a predicate which is optimized using an index) to determine when we have found all the rows which will be unfiltered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2214) CommandNeedRetryException.java is missing ASF header
[ https://issues.apache.org/jira/browse/HIVE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143660#comment-13143660 ] Hudson commented on HIVE-2214: -- Integrated in Hive-0.8.0-SNAPSHOT-h0.21 #80 (See [https://builds.apache.org/job/Hive-0.8.0-SNAPSHOT-h0.21/80/]) HIVE-2214. CommandNeedRetryException.java is missing ASF header (Ashutosh Chauhan via jvs) jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1197278 Files : * /hive/branches/branch-0.8/ql/src/java/org/apache/hadoop/hive/ql/CommandNeedRetryException.java CommandNeedRetryException.java is missing ASF header Key: HIVE-2214 URL: https://issues.apache.org/jira/browse/HIVE-2214 Project: Hive Issue Type: Bug Affects Versions: 0.8.0 Reporter: John Sichi Assignee: Ashutosh Chauhan Fix For: 0.8.0 Attachments: hive-2214.patch Please add one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.0-SNAPSHOT-h0.21 - Build # 80 - Fixed
Changes for Build #79 Changes for Build #80 [jvs] HIVE-2214. CommandNeedRetryException.java is missing ASF header (Ashutosh Chauhan via jvs) All tests passed The Apache Jenkins build system has built Hive-0.8.0-SNAPSHOT-h0.21 (build #80) Status: Fixed Check console output at https://builds.apache.org/job/Hive-0.8.0-SNAPSHOT-h0.21/80/ to view the results.
[jira] [Updated] (HIVE-2467) HA Support for Metastore Server
[ https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated HIVE-2467: --- Component/s: (was: Metastore) Server Infrastructure Security HA Support for Metastore Server Key: HIVE-2467 URL: https://issues.apache.org/jira/browse/HIVE-2467 Project: Hive Issue Type: Improvement Components: Security, Server Infrastructure Affects Versions: 0.8.0, 0.9.0 Reporter: Thomas Weise Assignee: Thomas Weise Fix For: 0.9.0 Attachments: HIVE-2467.patch We require HA deployment for metastore server for HCatalog: * Multiple server instances run behind VIP * Database provides HA Metastore server instances will need to be able to share any state required for VIP outside RDBMS. As of Hive 0.8 affected conversational state that needs to support VIP/HA setup is limited to current delegation tokens. Is this correct? We are planning to use ZooKeeper to share current delegation tokens and master keys between nodes of the VIP. ZK is already (optionally) used by Hive for concurrency control. Access to ZK would be limited on the network level or in the future, when ZooKeeper supports security, through Kerberos, similar to NN access. Currently Hive taps into Hadoop core security delegation token support through extension of org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManagerTokenIdent A solution could amend the Hive specific extension to support: * Pluggable delegation token and master key store (ZooKeeper as alternative for in-memory AbstractDelegationTokenSecretManager) * Delegation token retrieval from token store when not found in memory (wrap/extend retrievePassword(...)) * Cancellation of token in token store * Purging of expired tokens from token store http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2467) HA Support for Metastore Server
[ https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143667#comment-13143667 ] Carl Steinbach commented on HIVE-2467: -- @Thomas: Please submit a review request on reviewboard https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-ReviewProcess HA Support for Metastore Server Key: HIVE-2467 URL: https://issues.apache.org/jira/browse/HIVE-2467 Project: Hive Issue Type: Improvement Components: Metastore, Security, Server Infrastructure Affects Versions: 0.8.0, 0.9.0 Reporter: Thomas Weise Assignee: Thomas Weise Fix For: 0.9.0 Attachments: HIVE-2467.patch We require HA deployment for metastore server for HCatalog: * Multiple server instances run behind VIP * Database provides HA Metastore server instances will need to be able to share any state required for VIP outside RDBMS. As of Hive 0.8 affected conversational state that needs to support VIP/HA setup is limited to current delegation tokens. Is this correct? We are planning to use ZooKeeper to share current delegation tokens and master keys between nodes of the VIP. ZK is already (optionally) used by Hive for concurrency control. Access to ZK would be limited on the network level or in the future, when ZooKeeper supports security, through Kerberos, similar to NN access. Currently Hive taps into Hadoop core security delegation token support through extension of org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManagerTokenIdent A solution could amend the Hive specific extension to support: * Pluggable delegation token and master key store (ZooKeeper as alternative for in-memory AbstractDelegationTokenSecretManager) * Delegation token retrieval from token store when not found in memory (wrap/extend retrievePassword(...)) * Cancellation of token in token store * Purging of expired tokens from token store http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2550) Provide jira_base_url for improved arc commit workflow
Provide jira_base_url for improved arc commit workflow -- Key: HIVE-2550 URL: https://issues.apache.org/jira/browse/HIVE-2550 Project: Hive Issue Type: Bug Reporter: Marek Sapota -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2550) Provide jira_base_url for improved arc commit workflow
[ https://issues.apache.org/jira/browse/HIVE-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2550: -- Attachment: D249.1.patch mareksapotafb requested code review of HIVE-2550 [jira] Provide jira_base_url for improved arc commit workflow. Reviewers: JIRA Provide jira_base_url for D243. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D249 AFFECTED FILES .arcconfig MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/465/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Provide jira_base_url for improved arc commit workflow -- Key: HIVE-2550 URL: https://issues.apache.org/jira/browse/HIVE-2550 Project: Hive Issue Type: Bug Reporter: Marek Sapota Attachments: D249.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2550) Provide jira_base_url for improved arc commit workflow
[ https://issues.apache.org/jira/browse/HIVE-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143699#comment-13143699 ] Phabricator commented on HIVE-2550: --- mareksapotafb has added reviewers to the revision HIVE-2550 [jira] Provide jira_base_url for improved arc commit workflow. Added Reviewers: jsichi REVISION DETAIL https://reviews.facebook.net/D249 Provide jira_base_url for improved arc commit workflow -- Key: HIVE-2550 URL: https://issues.apache.org/jira/browse/HIVE-2550 Project: Hive Issue Type: Bug Reporter: Marek Sapota Attachments: D249.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2467) HA Support for Metastore Server
[ https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143698#comment-13143698 ] jirapos...@reviews.apache.org commented on HIVE-2467: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2721/ --- Review request for Carl Steinbach, Ashutosh Chauhan and Devaraj Das. Summary --- https://issues.apache.org/jira/browse/HIVE-2467 This addresses bug HIVE-2467. https://issues.apache.org/jira/browse/HIVE-2467 Diffs - trunk/shims/ivy.xml 1196916 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1196916 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java PRE-CREATION trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java PRE-CREATION trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java PRE-CREATION trunk/shims/src/0.20S/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java PRE-CREATION trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java 1196916 Diff: https://reviews.apache.org/r/2721/diff Testing --- unit test added, ant clean package test - passed Thanks, Thomas HA Support for Metastore Server Key: HIVE-2467 URL: https://issues.apache.org/jira/browse/HIVE-2467 Project: Hive Issue Type: Improvement Components: Metastore, Security, Server Infrastructure Affects Versions: 0.8.0, 0.9.0 Reporter: Thomas Weise Assignee: Thomas Weise Fix For: 0.9.0 Attachments: HIVE-2467.patch We require HA deployment for metastore server for HCatalog: * Multiple server instances run behind VIP * Database provides HA Metastore server instances will need to be able to share any state required for VIP outside RDBMS. As of Hive 0.8 affected conversational state that needs to support VIP/HA setup is limited to current delegation tokens. Is this correct? We are planning to use ZooKeeper to share current delegation tokens and master keys between nodes of the VIP. ZK is already (optionally) used by Hive for concurrency control. Access to ZK would be limited on the network level or in the future, when ZooKeeper supports security, through Kerberos, similar to NN access. Currently Hive taps into Hadoop core security delegation token support through extension of org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManagerTokenIdent A solution could amend the Hive specific extension to support: * Pluggable delegation token and master key store (ZooKeeper as alternative for in-memory AbstractDelegationTokenSecretManager) * Delegation token retrieval from token store when not found in memory (wrap/extend retrievePassword(...)) * Cancellation of token in token store * Purging of expired tokens from token store http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2467) HA Support for Metastore Server
[ https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13143702#comment-13143702 ] jirapos...@reviews.apache.org commented on HIVE-2467: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2721/ --- (Updated 2011-11-04 02:34:21.317393) Review request for Carl Steinbach, Ashutosh Chauhan and Devaraj Das. Summary (updated) --- https://issues.apache.org/jira/browse/HIVE-2467 This addresses bug HIVE-2467. https://issues.apache.org/jira/browse/HIVE-2467 Diffs - trunk/shims/ivy.xml 1196916 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1196916 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java PRE-CREATION trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java PRE-CREATION trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java PRE-CREATION trunk/shims/src/0.20S/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java PRE-CREATION trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java 1196916 Diff: https://reviews.apache.org/r/2721/diff Testing --- unit test added, ant clean package test - passed Thanks, Thomas HA Support for Metastore Server Key: HIVE-2467 URL: https://issues.apache.org/jira/browse/HIVE-2467 Project: Hive Issue Type: Improvement Components: Metastore, Security, Server Infrastructure Affects Versions: 0.8.0, 0.9.0 Reporter: Thomas Weise Assignee: Thomas Weise Fix For: 0.9.0 Attachments: HIVE-2467.patch We require HA deployment for metastore server for HCatalog: * Multiple server instances run behind VIP * Database provides HA Metastore server instances will need to be able to share any state required for VIP outside RDBMS. As of Hive 0.8 affected conversational state that needs to support VIP/HA setup is limited to current delegation tokens. Is this correct? We are planning to use ZooKeeper to share current delegation tokens and master keys between nodes of the VIP. ZK is already (optionally) used by Hive for concurrency control. Access to ZK would be limited on the network level or in the future, when ZooKeeper supports security, through Kerberos, similar to NN access. Currently Hive taps into Hadoop core security delegation token support through extension of org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManagerTokenIdent A solution could amend the Hive specific extension to support: * Pluggable delegation token and master key store (ZooKeeper as alternative for in-memory AbstractDelegationTokenSecretManager) * Delegation token retrieval from token store when not found in memory (wrap/extend retrievePassword(...)) * Cancellation of token in token store * Purging of expired tokens from token store http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
