[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-2249:
---
Fix Version/s: 0.9.0
When creating constant expression for numbers, try to infer type from another
comparison operand, instead of trying to use integer first, and then long and
double
--
Key: HIVE-2249
URL: https://issues.apache.org/jira/browse/HIVE-2249
Project: Hive
Issue Type: Improvement
Reporter: Siying Dong
Assignee: Zhiqiu Kong
Fix For: 0.9.0
Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt,
HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch,
HIVE-2249.D1383.4.patch
The current code to build constant expression for numbers, here is the code:
try {
v = Double.valueOf(expr.getText());
v = Long.valueOf(expr.getText());
v = Integer.valueOf(expr.getText());
} catch (NumberFormatException e) {
// do nothing here, we will throw an exception in the following block
}
if (v == null) {
throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
.getMsg(expr));
}
return new ExprNodeConstantDesc(v);
The for the case that WHERE BIG_INT_COLUMN = 0, or WHERE DOUBLE_COLUMN
= 0, we always have to do a type conversion when comparing, which is
unnecessary if it is slightly smarter to choose type when creating the
constant expression. We can simply walk one level up the tree, find another
comparison party and use the same type with that one if it is possible. For
user's wrong query like 'INT_COLUMN=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2686) Test ppr_pushdown.q is failing on trunk
[
https://issues.apache.org/jira/browse/HIVE-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13197874#comment-13197874
]
Ashutosh Chauhan commented on HIVE-2686:
In the meanwhile HIVE-2249 got committed which introduced order-by in
ppr_pushdown.q which makes this test to pass. I think that just mask this
problem and its better to fix the ordering issue for union queries.
Test ppr_pushdown.q is failing on trunk
---
Key: HIVE-2686
URL: https://issues.apache.org/jira/browse/HIVE-2686
Project: Hive
Issue Type: Test
Components: Testing Infrastructure
Affects Versions: 0.9.0
Reporter: Ashutosh Chauhan
Assignee: Navis
Priority: Minor
Fix For: 0.9.0
Attachments: HIVE-2686.D1215.1.patch, HIVE-2686.D1215.2.patch
{noformat}
$ ant test -Dtestcase=TestCliDriver -Dqfile=ppr_pushdown.q
[junit] See build/ql/tmp/hive.log, or try ant test ...
-Dtest.silent=false to get more logs.
[junit] at junit.framework.Assert.fail(Assert.java:50)
[junit] at
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_pushdown(TestCliDriver.java:129)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at junit.framework.TestCase.runTest(TestCase.java:168)
[junit] at junit.framework.TestCase.runBare(TestCase.java:134)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
[junit] at junit.framework.TestResult.run(TestResult.java:113)
[junit] at junit.framework.TestCase.run(TestCase.java:124)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
[junit] at junit.framework.TestSuite.run(TestSuite.java:238)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
[junit] 178d177
[junit] 12341234
[junit] 179a179
[junit] 12341234
[junit] 196d195
[junit] 12241224
[junit] 197a197
[junit] 12241224
[junit] 214d213
[junit] Exception: Client execution results failed with error code = 1
[junit] See build/ql/tmp/hive.log, or try ant test ...
-Dtest.silent=false to get more logs.
[junit] 12141214
[junit] 215a215
[junit] 12141214
[junit] 232d231
[junit] 12.412.4
[junit] 233a233
[junit] 12.412.4
[junit] 250d249
[junit] 12+412+4
[junit] 251a251
[junit] 12+412+4
[junit] 268d267
[junit] 12:412:4
[junit] 269a269
[junit] 12:412:4
[junit] 286d285
[junit] 12%412%4
[junit] 287a287
[junit] 12%412%4
[junit] 304d303
[junit] 12*412*4
[junit] 305a305
[junit] 12*412*4
[junit] Cleaning up TestCliDriver
[junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 137.556 sec
[junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2771) Add support for filter pushdown for key ranges in hbase for keys of type string
[
https://issues.apache.org/jira/browse/HIVE-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-2771:
---
Status: Patch Available (was: Open)
{noformat}
BUILD SUCCESSFUL
Total time: 269 minutes 29 seconds
{noformat}
All the tests passes with this patch. This is ready for review.
Add support for filter pushdown for key ranges in hbase for keys of type
string
---
Key: HIVE-2771
URL: https://issues.apache.org/jira/browse/HIVE-2771
Project: Hive
Issue Type: Sub-task
Components: HBase Handler
Affects Versions: 0.8.0, 0.7.1, 0.7.0, 0.8.1
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: HIVE-2771.D1551.1.patch
This is a subtask of HIVE-1643
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2372) java.io.IOException: error=7, Argument list too long
[
https://issues.apache.org/jira/browse/HIVE-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Tryuber updated HIVE-2372:
-
Attachment: HIVE-2372.1.patch.txt
Patch, 1st version
java.io.IOException: error=7, Argument list too long
Key: HIVE-2372
URL: https://issues.apache.org/jira/browse/HIVE-2372
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Sergey Tryuber
Priority: Critical
Attachments: HIVE-2372.1.patch.txt
I execute a huge query on a table with a lot of 2-level partitions. There is
a perl reducer in my query. Maps worked ok, but every reducer fails with the
following exception:
2011-08-11 04:58:29,865 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator:
Executing [/usr/bin/perl, reducer.pl, my_argument]
2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator:
tablename=null
2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator:
partname=null
2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator:
alias=null
2011-08-11 04:58:29,935 FATAL ExecReducer:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=0)
{key:{reducesinkkey0:129390185139228,reducesinkkey1:8AF163CA6F},value:{_col0:8AF163CA6F,_col1:2011-07-27
22:48:52,_col2:129390185139228,_col3:2006,_col4:4100,_col5:10017388=6,_col6:1063,_col7:NULL,_col8:address.com,_col9:NULL,_col10:NULL},alias:0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:256)
at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:468)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:416)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Cannot
initialize ScriptOperator
at
org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:320)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:247)
... 7 more
Caused by: java.io.IOException: Cannot run program /usr/bin/perl:
java.io.IOException: error=7, Argument list too long
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at
org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:279)
... 15 more
Caused by: java.io.IOException: java.io.IOException: error=7, Argument list
too long
at java.lang.UNIXProcess.init(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 16 more
It seems to me, I found the cause. ScriptOperator.java puts a lot of configs
as environment variables to the child reduce process. One of variables is
mapred.input.dir, which in my case more than 150KB. There are a huge amount
of input directories in this variable. In short, the problem is that Linux
(up to 2.6.23 kernel version) limits summary size of environment variables
for child processes to 132KB. This problem could be solved by upgrading the
kernel. But strings limitations still be 132KB per string in environment
variable. So such huge variable doesn't work even on my home computer
(2.6.32). You can read more information on
(http://www.kernel.org/doc/man-pages/online/pages/man2/execve.2.html).
For now all our work has been stopped because of this problem and I can't
find the solution. The only solution, which seems to me more reasonable is to
get rid of this variable in reducers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2372) java.io.IOException: error=7, Argument list too long
[
https://issues.apache.org/jira/browse/HIVE-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13197895#comment-13197895
]
Sergey Tryuber commented on HIVE-2372:
--
I've attached a patch (as an attachment, not by submit patch, as described on
wiki HowToContribute). When I cloned trunk and run tests without any changes,
for about 5 hours, there was several test errors((( Build and testing with my
changes showed the same errors count. So, please, review this patch and make
remarks.
java.io.IOException: error=7, Argument list too long
Key: HIVE-2372
URL: https://issues.apache.org/jira/browse/HIVE-2372
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Sergey Tryuber
Priority: Critical
Attachments: HIVE-2372.1.patch.txt
I execute a huge query on a table with a lot of 2-level partitions. There is
a perl reducer in my query. Maps worked ok, but every reducer fails with the
following exception:
2011-08-11 04:58:29,865 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator:
Executing [/usr/bin/perl, reducer.pl, my_argument]
2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator:
tablename=null
2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator:
partname=null
2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator:
alias=null
2011-08-11 04:58:29,935 FATAL ExecReducer:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row (tag=0)
{key:{reducesinkkey0:129390185139228,reducesinkkey1:8AF163CA6F},value:{_col0:8AF163CA6F,_col1:2011-07-27
22:48:52,_col2:129390185139228,_col3:2006,_col4:4100,_col5:10017388=6,_col6:1063,_col7:NULL,_col8:address.com,_col9:NULL,_col10:NULL},alias:0}
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:256)
at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:468)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:416)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Cannot
initialize ScriptOperator
at
org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:320)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:247)
... 7 more
Caused by: java.io.IOException: Cannot run program /usr/bin/perl:
java.io.IOException: error=7, Argument list too long
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at
org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:279)
... 15 more
Caused by: java.io.IOException: java.io.IOException: error=7, Argument list
too long
at java.lang.UNIXProcess.init(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 16 more
It seems to me, I found the cause. ScriptOperator.java puts a lot of configs
as environment variables to the child reduce process. One of variables is
mapred.input.dir, which in my case more than 150KB. There are a huge amount
of input directories in this variable. In short, the problem is that Linux
(up to 2.6.23 kernel version) limits summary size of environment variables
for child processes to 132KB. This problem could be solved by upgrading the
kernel. But strings limitations still be 132KB per string in environment
variable. So such huge variable doesn't work even on my home computer
(2.6.32). You can read more information on
(http://www.kernel.org/doc/man-pages/online/pages/man2/execve.2.html).
For now all our work has been stopped because of this problem and I can't
find the solution. The only solution, which seems to me more reasonable is to
get rid of this variable in reducers.
--
This
[jira] [Created] (HIVE-2772) make union31.q deterministic
make union31.q deterministic Key: HIVE-2772 URL: https://issues.apache.org/jira/browse/HIVE-2772 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2772.D1557.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2772) make union31.q deterministic
[ https://issues.apache.org/jira/browse/HIVE-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2772: -- Attachment: HIVE-2772.D1557.1.patch njain requested code review of HIVE-2772 [jira] make union31.q deterministic. Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2772 HIVE-2772 TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1557 AFFECTED FILES ql/src/test/results/clientpositive/union31.q.out ql/src/test/queries/clientpositive/union31.q MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3273/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. make union31.q deterministic Key: HIVE-2772 URL: https://issues.apache.org/jira/browse/HIVE-2772 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2772.D1557.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2762: - Status: Open (was: Patch Available) Alter Table Partition Concatenate Fails On Certain Characters - Key: HIVE-2762 URL: https://issues.apache.org/jira/browse/HIVE-2762 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2762.1.patch.txt, HIVE-2762.D1533.1.patch, HIVE-2762.D1533.2.patch Alter table partition concatenate creates a Java URI object for the location of a partition. If the partition name contains certain characters, such as } or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13197965#comment-13197965 ] Phabricator commented on HIVE-2762: --- njain has commented on the revision HIVE-2762 [jira] Alter Table Partition Concatenate Fails On Certain Characters. INLINE COMMENTS ql/src/test/queries/clientpositive/escape2.q:47 Why are you running the same command multiple times ? REVISION DETAIL https://reviews.facebook.net/D1533 Alter Table Partition Concatenate Fails On Certain Characters - Key: HIVE-2762 URL: https://issues.apache.org/jira/browse/HIVE-2762 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2762.1.patch.txt, HIVE-2762.D1533.1.patch, HIVE-2762.D1533.2.patch Alter table partition concatenate creates a Java URI object for the location of a partition. If the partition name contains certain characters, such as } or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2758) Metastore is caching too aggressively
[
https://issues.apache.org/jira/browse/HIVE-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Wilfong updated HIVE-2758:
Resolution: Fixed
Fix Version/s: 0.9.0
Status: Resolved (was: Patch Available)
Metastore is caching too aggressively
-
Key: HIVE-2758
URL: https://issues.apache.org/jira/browse/HIVE-2758
Project: Hive
Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
Fix For: 0.9.0
Attachments: HIVE-2758.D1491.1.patch, HIVE-2758.D1491.2.patch
The metastore is caching values, like table names and locations too
aggressively, leading to inconsistencies across Hive clients and metastore
Thrift servers.
For example, open two Hive clients, in each call
DESCRIBE FORMATTED table_foo;
Then in one of those clients, execute
ALTER TABLE table_foo RENAME TO table_bar;
Then in both clients call
DESCRIBE FORMATTED table_bar;
In the client that executed the alter command, the location is correct,
however, in the other Hive client, it will still show the original location
of table_foo.
A similar experiment can be done using metastore Thrift servers, substituting
get_table for DESCRIBE FORMATTED and alter_table for ALTER TABLE ... RENAME
TO.
On the Thrift server you can see that the one which did not execute the alter
command, not only returns the wrong location, despite calling
get_table('table_bar') it will return a table that still has the name
table_foo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2772) make union31.q deterministic
[ https://issues.apache.org/jira/browse/HIVE-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13197994#comment-13197994 ] Phabricator commented on HIVE-2772: --- ashutoshc has accepted the revision HIVE-2772 [jira] make union31.q deterministic. +1 REVISION DETAIL https://reviews.facebook.net/D1557 make union31.q deterministic Key: HIVE-2772 URL: https://issues.apache.org/jira/browse/HIVE-2772 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2772.D1557.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2758) Metastore is caching too aggressively
[
https://issues.apache.org/jira/browse/HIVE-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13197997#comment-13197997
]
Phabricator commented on HIVE-2758:
---
kevinwilfong has commented on the revision HIVE-2758 [jira] Metastore is
caching too aggressively.
Thanks, Carl. I got it.
REVISION DETAIL
https://reviews.facebook.net/D1491
Metastore is caching too aggressively
-
Key: HIVE-2758
URL: https://issues.apache.org/jira/browse/HIVE-2758
Project: Hive
Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
Fix For: 0.9.0
Attachments: HIVE-2758.D1491.1.patch, HIVE-2758.D1491.2.patch
The metastore is caching values, like table names and locations too
aggressively, leading to inconsistencies across Hive clients and metastore
Thrift servers.
For example, open two Hive clients, in each call
DESCRIBE FORMATTED table_foo;
Then in one of those clients, execute
ALTER TABLE table_foo RENAME TO table_bar;
Then in both clients call
DESCRIBE FORMATTED table_bar;
In the client that executed the alter command, the location is correct,
however, in the other Hive client, it will still show the original location
of table_foo.
A similar experiment can be done using metastore Thrift servers, substituting
get_table for DESCRIBE FORMATTED and alter_table for ALTER TABLE ... RENAME
TO.
On the Thrift server you can see that the one which did not execute the alter
command, not only returns the wrong location, despite calling
get_table('table_bar') it will return a table that still has the name
table_foo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2635) wrong class loader used for external jars
[ https://issues.apache.org/jira/browse/HIVE-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2635: - Status: Patch Available (was: Open) wrong class loader used for external jars - Key: HIVE-2635 URL: https://issues.apache.org/jira/browse/HIVE-2635 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2635.D687.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2635) wrong class loader used for external jars
[ https://issues.apache.org/jira/browse/HIVE-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2635: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) wrong class loader used for external jars - Key: HIVE-2635 URL: https://issues.apache.org/jira/browse/HIVE-2635 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2635.D687.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2769) union with a multi-table insert is not working
[ https://issues.apache.org/jira/browse/HIVE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2769: - Status: Patch Available (was: Open) union with a multi-table insert is not working -- Key: HIVE-2769 URL: https://issues.apache.org/jira/browse/HIVE-2769 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2769.D1545.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2769) union with a multi-table insert is not working
[ https://issues.apache.org/jira/browse/HIVE-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2769: --- Fix Version/s: 0.9.0 union with a multi-table insert is not working -- Key: HIVE-2769 URL: https://issues.apache.org/jira/browse/HIVE-2769 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.9.0 Attachments: HIVE-2769.D1545.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2773) HiveStorageHandler.configureTableJobProperites() should let the handler know wether it is configuration for input or output
HiveStorageHandler.configureTableJobProperites() should let the handler know wether it is configuration for input or output --- Key: HIVE-2773 URL: https://issues.apache.org/jira/browse/HIVE-2773 Project: Hive Issue Type: Improvement Reporter: Francis Liu HiveStorageHandler.configureTableJobProperties() is called to allow the storage handler to setup any properties that the underlying inputformat/outputformat/serde may need. But the handler implementation does not know whether it is being called for configuring input or output. This makes it a problem for handlers which sets an external state. In the case of HCatalog's HBase storageHandler, whenever a write needs to be configured we create a write transaction which needs to be committed or aborted later on. In this case configuring for both input and output each time configureTableJobProperties() is called would not be desirable. This has become an issue since HCatalog is dropping storageDrivers for SerDe and StorageHandler (see HCATALOG-237). My proposal is to replace configureTableJobProperties() with two methods: configureInputJobProperties() configureOutputJobProperties() Each method will have the same signature. I cursory look at the code and I believe changes should be straighforward also given that we are not really changing anything just splitting responsibility. If the community is fine with this approach I will go ahead and create a aptch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2772) make union31.q deterministic
[ https://issues.apache.org/jira/browse/HIVE-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2772: - Status: Patch Available (was: Open) make union31.q deterministic Key: HIVE-2772 URL: https://issues.apache.org/jira/browse/HIVE-2772 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2772.D1557.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2772) make union31.q deterministic
[ https://issues.apache.org/jira/browse/HIVE-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2772: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed by Yongqiang make union31.q deterministic Key: HIVE-2772 URL: https://issues.apache.org/jira/browse/HIVE-2772 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-2772.D1557.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2762: Attachment: HIVE-2762.2.patch.txt Alter Table Partition Concatenate Fails On Certain Characters - Key: HIVE-2762 URL: https://issues.apache.org/jira/browse/HIVE-2762 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2762.1.patch.txt, HIVE-2762.2.patch.txt, HIVE-2762.D1533.1.patch, HIVE-2762.D1533.2.patch, HIVE-2762.D1533.3.patch Alter table partition concatenate creates a Java URI object for the location of a partition. If the partition name contains certain characters, such as } or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-2762: Status: Patch Available (was: Open) Alter Table Partition Concatenate Fails On Certain Characters - Key: HIVE-2762 URL: https://issues.apache.org/jira/browse/HIVE-2762 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2762.1.patch.txt, HIVE-2762.2.patch.txt, HIVE-2762.D1533.1.patch, HIVE-2762.D1533.2.patch, HIVE-2762.D1533.3.patch Alter table partition concatenate creates a Java URI object for the location of a partition. If the partition name contains certain characters, such as } or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198129#comment-13198129 ] Phabricator commented on HIVE-2762: --- kevinwilfong has commented on the revision HIVE-2762 [jira] Alter Table Partition Concatenate Fails On Certain Characters. If you open the patch in an editor that supports displaying those unprintable characters, you can see them in the file. REVISION DETAIL https://reviews.facebook.net/D1533 Alter Table Partition Concatenate Fails On Certain Characters - Key: HIVE-2762 URL: https://issues.apache.org/jira/browse/HIVE-2762 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2762.1.patch.txt, HIVE-2762.2.patch.txt, HIVE-2762.D1533.1.patch, HIVE-2762.D1533.2.patch, HIVE-2762.D1533.3.patch Alter table partition concatenate creates a Java URI object for the location of a partition. If the partition name contains certain characters, such as } or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2772) make union31.q deterministic
[ https://issues.apache.org/jira/browse/HIVE-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2772: --- Fix Version/s: 0.9.0 make union31.q deterministic Key: HIVE-2772 URL: https://issues.apache.org/jira/browse/HIVE-2772 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.9.0 Attachments: HIVE-2772.D1557.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198175#comment-13198175 ] Namit Jain commented on HIVE-2612: -- https://cwiki.apache.org/confluence/display/Hive/Hive+across+Multiple+Data+Centers+(Physical+Clusters) Added a new document which explains some of the thinking and the design. Please comment support hive table/partitions coexistes in more than one clusters - Key: HIVE-2612 URL: https://issues.apache.org/jira/browse/HIVE-2612 Project: Hive Issue Type: New Feature Components: Metastore Reporter: He Yongqiang Assignee: Namit Jain Attachments: HIVE-2612.1.patch 1) add cluster object into hive metastore 2) each partition/table has a creation cluster and a list of living clusters, and also data location in each cluster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
multi cluster suppoort in hive
I added a new document: https://cwiki.apache.org/confluence/display/Hive/Hive+across+Multiple+Data+Centers+(Physical+Clusters) for https://issues.apache.org/jira/browse/HIVE-2612 Please comment. Thanks, -namit
Hive-trunk-h0.21 - Build # 1237 - Still Failing
Changes for Build #1193 [hashutosh] HIVE-2705: SemanticAnalyzer twice swallows an exception it shouldn't (jghoman via hashutosh) Changes for Build #1194 Changes for Build #1195 [hashutosh] HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan) [hashutosh] HIVE-2682: Clean-up logs (Rajat Goel via Ashutosh Chauhan) Changes for Build #1196 [amareshwari] HIVE-2629. Make a single Hive binary work with both 0.20.x and 0.23.0. (Thomas Weise via amareshwari) Changes for Build #1197 Changes for Build #1198 [namit] HIVE-2504 Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory (Chinna Rao Lalam via namit) [namit] HIVE-2695 Add PRINTF() Udf (Zhenxiao Luo via namit) Changes for Build #1199 Changes for Build #1200 Changes for Build #1201 Changes for Build #1202 Changes for Build #1203 Changes for Build #1204 [cws] HIVE-2719. Revert HIVE-2589 (He Yongqiang via cws) Changes for Build #1205 Changes for Build #1207 [namit] HIVE-2718 NPE in union followed by join (He Yongqiang via namit) Changes for Build #1208 Changes for Build #1209 Changes for Build #1210 [namit] HIVE-2674 get_partitions_ps throws TApplicationException if table doesn't exist (Kevin Wilfong via namit) Changes for Build #1211 [cws] HIVE-2203. Extend concat_ws() UDF to support arrays of strings (Zhenxiao Luo via cws) [cws] HIVE-2279. Implement sort(array) UDF (Zhenxiao Luo via cws) Changes for Build #1212 [hashutosh] HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) Changes for Build #1213 Changes for Build #1214 Changes for Build #1215 Changes for Build #1216 [cws] HIVE-2724. Remove unused lib/log4j-1.2.15.jar (Andrew Bayer via cws) Changes for Build #1217 Changes for Build #1218 Changes for Build #1219 [hashutosh] HIVE-2665 : Support for metastore service specific HADOOP_OPTS environment setting (thw via hashutosh) Changes for Build #1220 [namit] HIVE-2727 add a testcase for partitioned view on union and base tables have index (He Yongqiang via namit) Changes for Build #1221 [hashutosh] HIVE-2746 : Metastore client doesn't log properly in case of connection failure to server (hashutosh) [cws] HIVE-2698 [jira] Enable Hadoop-1.0.0 in Hive (Enis Söztutar via Carl Steinbach) Summary: third version of the patch Hadoop-1.0.0 is recently released, which is AFAIK, API compatible to the 0.20S release. Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: cwsteinbach, enis Differential Revision: https://reviews.facebook.net/D1389 Changes for Build #1222 [namit] HIVE-2750 Hive multi group by single reducer optimization causes invalid column reference error (Kevin Wilfong via namit) Changes for Build #1223 Changes for Build #1224 [cws] HIVE-2734 [jira] Fix some nondeterministic test output (Zhenxiao Luo via Carl Steinbach) Summary: HIVE-2734: Fix some nondeterministic test output Many Hive query tests lack an ORDER BY clause, and consequently the ordering of the rows in the result set is nondeterministic: groupby1_limit input11_limit input1_limit input_lazyserde join18_multi_distinct join_1to1 join_casesensitive join_filters join_nulls merge3 rcfile_columnar rcfile_lazydecompress rcfile_union sample10 udf_sentences union24 columnarserde_create_shortcut combine1 global_limit Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach CC: zhenxiao, cwsteinbach Differential Revision: https://reviews.facebook.net/D1449 [namit] HIVE-2754 NPE in union with lateral view (Yongqiang He via namit) Changes for Build #1225 Changes for Build #1226 Changes for Build #1227 [namit] HIVE-2755 union follwowed by union_subq does not work if the subquery union has reducers (He Yongqiang via namit) Changes for Build #1228 Changes for Build #1229 [hashutosh] HIVE-2735: PlanUtils.configureTableJobPropertiesForStorageHandler() is not called for partitioned table (sushanth via ashutosh) Changes for Build #1230 [cws] HIVE-2760 [jira] TestCliDriver should log elapsed time Summary: HIVE-2760. TestCliDriver should log elapsed time Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc CC: ashutoshc, cwsteinbach Differential Revision: https://reviews.facebook.net/D1503 [cws] HIVE-2662 [jira] Add Ant configuration property for dumping classpath of tests Summary: HIVE-2662. Add Ant configuration property for dumping classpath of tests Test Plan: EMPTY Reviewers: JIRA, jsichi, ashutoshc Reviewed By: ashutoshc CC: ashutoshc Differential Revision: https://reviews.facebook.net/D903 Changes for Build #1231 [hashutosh] HIVE_2645: Hive Web Server startup messages logs incorrect path it is searching for WAR (Edward Capriolo via Ashutosh Chauhan) Changes for Build #1232 Changes for Build #1233 [sdong] HIVE-2249 When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use
[jira] [Updated] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2612: -- Attachment: HIVE-2612.D1569.1.patch njain requested code review of HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters. Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2612 HIVE-2612 Initial version - not for review 1) add cluster object into hive metastore 2) each partition/table has a creation cluster and a list of living clusters, and also data location in each cluster TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1569 AFFECTED FILES metastore/src/model/package.jdo metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MCluster.java metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java metastore/if/hive_metastore.thrift common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ql/src/java/org/apache/hadoop/hive/ql/plan/ClusterDDLDesc.java ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3291/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. support hive table/partitions coexistes in more than one clusters - Key: HIVE-2612 URL: https://issues.apache.org/jira/browse/HIVE-2612 Project: Hive Issue Type: New Feature Components: Metastore Reporter: He Yongqiang Assignee: Namit Jain Attachments: HIVE-2612.1.patch, HIVE-2612.D1569.1.patch 1) add cluster object into hive metastore 2) each partition/table has a creation cluster and a list of living clusters, and also data location in each cluster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters
[ https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198342#comment-13198342 ] Namit Jain commented on HIVE-2762: -- +1 Running tests Alter Table Partition Concatenate Fails On Certain Characters - Key: HIVE-2762 URL: https://issues.apache.org/jira/browse/HIVE-2762 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-2762.1.patch.txt, HIVE-2762.2.patch.txt, HIVE-2762.D1533.1.patch, HIVE-2762.D1533.2.patch, HIVE-2762.D1533.3.patch Alter table partition concatenate creates a Java URI object for the location of a partition. If the partition name contains certain characters, such as } or space ' ', the object constructor fails, causing the query to fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2774) Message when trying to create a view from a view should give name of offending view
Message when trying to create a view from a view should give name of offending
view
---
Key: HIVE-2774
URL: https://issues.apache.org/jira/browse/HIVE-2774
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.7.1
Reporter: Jakob Homan
Assignee: Jakob Homan
Fix For: 0.9.0
Attachments: HIVE-2774.patch
Currently when trying to create a view that contains another view (a prohibited
operation) the exception is:
{noformat}FAILED: Error in semantic analysis: Invalid Table Alias Referencing
view from foreign databases is not supported.{noformat}
'Twould be better if this message included the view that caused the error so
that when building views from lots of tables, one doesn't need to hunt to
figure out which one was the actual view.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2774) Message when trying to create a view from a view should give name of offending view
[
https://issues.apache.org/jira/browse/HIVE-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HIVE-2774:
--
Attachment: HIVE-2774.patch
Quick patch that adds the offending table name to the message. Not easily
testable and easily reasonable, so no tests.
Message when trying to create a view from a view should give name of
offending view
---
Key: HIVE-2774
URL: https://issues.apache.org/jira/browse/HIVE-2774
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.7.1
Reporter: Jakob Homan
Assignee: Jakob Homan
Fix For: 0.9.0
Attachments: HIVE-2774.patch
Currently when trying to create a view that contains another view (a
prohibited operation) the exception is:
{noformat}FAILED: Error in semantic analysis: Invalid Table Alias Referencing
view from foreign databases is not supported.{noformat}
'Twould be better if this message included the view that caused the error so
that when building views from lots of tables, one doesn't need to hunt to
figure out which one was the actual view.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2772) make union31.q deterministic
[ https://issues.apache.org/jira/browse/HIVE-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198383#comment-13198383 ] Hudson commented on HIVE-2772: -- Integrated in Hive-trunk-h0.21 #1238 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1238/]) HIVE-2772 [jira] make union31.q deterministic (Namit Jain via Yongqiang He) Summary: https://issues.apache.org/jira/browse/HIVE-2772 HIVE-2772 Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc CC: ashutoshc Differential Revision: https://reviews.facebook.net/D1557 heyongqiang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1239286 Files : * /hive/trunk/ql/src/test/queries/clientpositive/union31.q * /hive/trunk/ql/src/test/results/clientpositive/union31.q.out make union31.q deterministic Key: HIVE-2772 URL: https://issues.apache.org/jira/browse/HIVE-2772 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.9.0 Attachments: HIVE-2772.D1557.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2758) Metastore is caching too aggressively
[
https://issues.apache.org/jira/browse/HIVE-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198384#comment-13198384
]
Hudson commented on HIVE-2758:
--
Integrated in Hive-trunk-h0.21 #1238 (See
[https://builds.apache.org/job/Hive-trunk-h0.21/1238/])
HIVE-2758 Metastore is caching too aggressively (Kevin Wilfong reviewed by
Carl Steinbach)
kevinwilfong :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1239232
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
*
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
Metastore is caching too aggressively
-
Key: HIVE-2758
URL: https://issues.apache.org/jira/browse/HIVE-2758
Project: Hive
Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
Fix For: 0.9.0
Attachments: HIVE-2758.D1491.1.patch, HIVE-2758.D1491.2.patch
The metastore is caching values, like table names and locations too
aggressively, leading to inconsistencies across Hive clients and metastore
Thrift servers.
For example, open two Hive clients, in each call
DESCRIBE FORMATTED table_foo;
Then in one of those clients, execute
ALTER TABLE table_foo RENAME TO table_bar;
Then in both clients call
DESCRIBE FORMATTED table_bar;
In the client that executed the alter command, the location is correct,
however, in the other Hive client, it will still show the original location
of table_foo.
A similar experiment can be done using metastore Thrift servers, substituting
get_table for DESCRIBE FORMATTED and alter_table for ALTER TABLE ... RENAME
TO.
On the Thrift server you can see that the one which did not execute the alter
command, not only returns the wrong location, despite calling
get_table('table_bar') it will return a table that still has the name
table_foo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2774) Message when trying to create a view from a view should give name of offending view
[
https://issues.apache.org/jira/browse/HIVE-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HIVE-2774:
--
Status: Patch Available (was: Open)
Message when trying to create a view from a view should give name of
offending view
---
Key: HIVE-2774
URL: https://issues.apache.org/jira/browse/HIVE-2774
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.7.1
Reporter: Jakob Homan
Assignee: Jakob Homan
Fix For: 0.9.0
Attachments: HIVE-2774.patch
Currently when trying to create a view that contains another view (a
prohibited operation) the exception is:
{noformat}FAILED: Error in semantic analysis: Invalid Table Alias Referencing
view from foreign databases is not supported.{noformat}
'Twould be better if this message included the view that caused the error so
that when building views from lots of tables, one doesn't need to hunt to
figure out which one was the actual view.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2611) Make index table output of create index command if index is table based
[ https://issues.apache.org/jira/browse/HIVE-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2611: - Component/s: Indexing Fix Version/s: 0.9.0 Make index table output of create index command if index is table based --- Key: HIVE-2611 URL: https://issues.apache.org/jira/browse/HIVE-2611 Project: Hive Issue Type: Improvement Components: Indexing Reporter: Kevin Wilfong Assignee: Kevin Wilfong Fix For: 0.9.0 Attachments: HIVE-2611.1.patch.txt, HIVE-2611.D705.1.patch, HIVE-2611.D705.2.patch, HIVE-2611.D705.3.patch If an index is table based, when that index is created a table is created to contain that index. This should be listed in the output of the command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2635) wrong class loader used for external jars
[ https://issues.apache.org/jira/browse/HIVE-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2635: --- Fix Version/s: 0.9.0 wrong class loader used for external jars - Key: HIVE-2635 URL: https://issues.apache.org/jira/browse/HIVE-2635 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.9.0 Attachments: HIVE-2635.D687.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2775) allow the number of files to be a multiple of bucketed table
allow the number of files to be a multiple of bucketed table Key: HIVE-2775 URL: https://issues.apache.org/jira/browse/HIVE-2775 Project: Hive Issue Type: New Feature Components: Metastore Reporter: xiaoyu wang Currently, hive bucketed table requires the number of files to match the bucket number in order to for correct sampling. This is very restrictive. e.g. we can only populate the table using a fix number of reducer, which can be a bottleneck. The idea is to introduce this physical bucket and logical bucket concept. physical bucket is the number of files and logical bucket is the number of bucket stored in meda-data for bucketed table. By allowing physical bucket to be a multiple of logical bucket, we can do correct sampling as well as scaling up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2775) allow the number of files to be a multiple of bucketed table
[
https://issues.apache.org/jira/browse/HIVE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198542#comment-13198542
]
xiaoyu wang commented on HIVE-2775:
---
{code}
index d0ff67e..bcddc5b 100644
@@ -349,7 +349,25 @@ public class Partition implements Serializable {
* we are just storing it as a property of the table as a short term measure.
*/
public int getBucketCount() {
-return table.getNumBuckets();
+ int logicalBucketNumber = table.getNumBuckets();
+ String pathPattern = this.getPartitionPath().toString() + /*;
+ try {
+ FileSystem fs =
FileSystem.get(this.table.getDataLocation(),Hive.get().getConf());
+ FileStatus srcs[] = fs.globStatus(new Path(pathPattern));
+ int physicalBucketNumber = srcs.length;
+ if ((physicalBucketNumber/logicalBucketNumber) * logicalBucketNumber
== physicalBucketNumber){
+ return physicalBucketNumber;
+ } else {
+ throw new RuntimeException(Cannot get bucket count for table
+ this.table.getTableName() +
+ logical bucket is + logicalBucketNumber + physical
bucket number is + physicalBucketNumber);
+ }
+ }catch (Exception e)
+ {
+ throw new RuntimeException(Cannot get bucket count for table +
this.table.getTableName(), e) ;
+ }
+
+
+//return table.getNumBuckets();
/*
* TODO: Keeping this code around for later use when we will support
* sampling on tables which are not created with CLUSTERED INTO clause
{code}
allow the number of files to be a multiple of bucketed table
Key: HIVE-2775
URL: https://issues.apache.org/jira/browse/HIVE-2775
Project: Hive
Issue Type: New Feature
Components: Metastore
Reporter: xiaoyu wang
Currently, hive bucketed table requires the number of files to match the
bucket number in order to for correct sampling. This is very restrictive.
e.g. we can only populate the table using a fix number of reducer, which can
be a bottleneck.
The idea is to introduce this physical bucket and logical bucket concept.
physical bucket is the number of files and logical bucket is the number
of bucket stored in meda-data for bucketed table. By allowing physical
bucket to be a multiple of logical bucket, we can do correct sampling as
well as scaling up.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
