[jira] [Resolved] (HIVE-46) derbyclient.jar and derbytools.jar should be included in hive/lib
[ https://issues.apache.org/jira/browse/HIVE-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-46. -- Resolution: Fixed derby.jar is now present in lib/ dir. No more NPE. derbyclient.jar and derbytools.jar should be included in hive/lib - Key: HIVE-46 URL: https://issues.apache.org/jira/browse/HIVE-46 Project: Hive Issue Type: Bug Components: Metastore Environment: Cent OS 5 x86-64 JDK 1.6 Reporter: Edward Capriolo Priority: Minor Attempting to use the Derby ClientDriver will cause a null pointer exception because derbyclient.jar and derbytools.jar are not in lib. hive show tables; 08/10/02 15:17:12 INFO hive.metastore: Trying to connect to metastore with URI jdbc:derby://nyhadoop1:1527/metastore_db FAILED: Error in semantic analysis: java.lang.NullPointerException 08/10/02 15:17:12 ERROR ql.Driver: FAILED: Error in semantic analysis: java.lang.NullPointerException derbyclient.jar and derbytools.jar should probably be put in the hive/lib directory. Also this type of exception could happen often with different JDBC libraries the exception should be trapped better -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1461) Clean up references to 'hive.metastore.local'
[ https://issues.apache.org/jira/browse/HIVE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-1461: --- Attachment: hive-1461.patch Patch for adding hive.metastore.local in HiveConf Clean up references to 'hive.metastore.local' - Key: HIVE-1461 URL: https://issues.apache.org/jira/browse/HIVE-1461 Project: Hive Issue Type: Bug Affects Versions: 0.7.0 Reporter: Paul Yang Priority: Minor Attachments: hive-1461.patch 'hive.metastore.local' should not be referred directly as a string. Instead, a HiveConf.ConfVar entry should be created and used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1461) Clean up references to 'hive.metastore.local'
[ https://issues.apache.org/jira/browse/HIVE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-1461: --- Assignee: Ashutosh Chauhan Affects Version/s: (was: 0.7.0) Status: Patch Available (was: Open) Clean up references to 'hive.metastore.local' - Key: HIVE-1461 URL: https://issues.apache.org/jira/browse/HIVE-1461 Project: Hive Issue Type: Bug Reporter: Paul Yang Assignee: Ashutosh Chauhan Priority: Minor Attachments: hive-1461.patch 'hive.metastore.local' should not be referred directly as a string. Instead, a HiveConf.ConfVar entry should be created and used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135093#comment-13135093 ] Jonathan Chang commented on HIVE-2509: -- Done. Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Chang updated HIVE-2509: - Attachment: D15.4.patch Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2372) java.io.IOException: error=7, Argument list too long
[ https://issues.apache.org/jira/browse/HIVE-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135111#comment-13135111 ] Edward Capriolo commented on HIVE-2372: --- I am surprised no one has ran into this sooner. I had assumed the limit was much shorter. Streaming is 'a neat hack' but with the UDF/UDAF frameworks I have never been convinced it is needed. Most often I see use it improperly to make hacky map side joins etc. Maybe your real problem is having to many input files and you should merge your input first, or use bucketing instead of two level partitioning. java.io.IOException: error=7, Argument list too long Key: HIVE-2372 URL: https://issues.apache.org/jira/browse/HIVE-2372 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Sergey Tryuber Priority: Critical I execute a huge query on a table with a lot of 2-level partitions. There is a perl reducer in my query. Maps worked ok, but every reducer fails with the following exception: 2011-08-11 04:58:29,865 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: Executing [/usr/bin/perl, reducer.pl, my_argument] 2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: tablename=null 2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: partname=null 2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: alias=null 2011-08-11 04:58:29,935 FATAL ExecReducer: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {key:{reducesinkkey0:129390185139228,reducesinkkey1:8AF163CA6F},value:{_col0:8AF163CA6F,_col1:2011-07-27 22:48:52,_col2:129390185139228,_col3:2006,_col4:4100,_col5:10017388=6,_col6:1063,_col7:NULL,_col8:address.com,_col9:NULL,_col10:NULL},alias:0} at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:256) at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:468) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:416) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Cannot initialize ScriptOperator at org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:320) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744) at org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:247) ... 7 more Caused by: java.io.IOException: Cannot run program /usr/bin/perl: java.io.IOException: error=7, Argument list too long at java.lang.ProcessBuilder.start(ProcessBuilder.java:460) at org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:279) ... 15 more Caused by: java.io.IOException: java.io.IOException: error=7, Argument list too long at java.lang.UNIXProcess.init(UNIXProcess.java:148) at java.lang.ProcessImpl.start(ProcessImpl.java:65) at java.lang.ProcessBuilder.start(ProcessBuilder.java:453) ... 16 more It seems to me, I found the cause. ScriptOperator.java puts a lot of configs as environment variables to the child reduce process. One of variables is mapred.input.dir, which in my case more than 150KB. There are a huge amount of input directories in this variable. In short, the problem is that Linux (up to 2.6.23 kernel version) limits summary size of environment variables for child processes to 132KB. This problem could be solved by upgrading the kernel. But strings limitations still be 132KB per string in environment variable. So such huge variable doesn't work even on my home computer (2.6.32). You can read more information on (http://www.kernel.org/doc/man-pages/online/pages/man2/execve.2.html). For now all our work has been stopped because of this problem and I can't find the solution. The only solution, which seems to me more reasonable is to
[jira] [Commented] (HIVE-1461) Clean up references to 'hive.metastore.local'
[ https://issues.apache.org/jira/browse/HIVE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135221#comment-13135221 ] Namit Jain commented on HIVE-1461: -- +1 Clean up references to 'hive.metastore.local' - Key: HIVE-1461 URL: https://issues.apache.org/jira/browse/HIVE-1461 Project: Hive Issue Type: Bug Reporter: Paul Yang Assignee: Ashutosh Chauhan Priority: Minor Attachments: hive-1461.patch 'hive.metastore.local' should not be referred directly as a string. Instead, a HiveConf.ConfVar entry should be created and used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2525) Introduce proper error messages, when explain fails on commands that otherwise could be succesful (commands that are not analyzed by Semantic Analyzer)
[ https://issues.apache.org/jira/browse/HIVE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2525: - Status: Open (was: Patch Available) Introduce proper error messages, when explain fails on commands that otherwise could be succesful (commands that are not analyzed by Semantic Analyzer) --- Key: HIVE-2525 URL: https://issues.apache.org/jira/browse/HIVE-2525 Project: Hive Issue Type: Improvement Reporter: Robert Surówka Assignee: Robert Surówka Priority: Trivial Attachments: HIVE-2525.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2525) Introduce proper error messages, when explain fails on commands that otherwise could be succesful (commands that are not analyzed by Semantic Analyzer)
[ https://issues.apache.org/jira/browse/HIVE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135227#comment-13135227 ] Namit Jain commented on HIVE-2525: -- hive explain dfs -ls PATH; FAILED: Parse Error: line 1:8 cannot recognize input near 'dfs' '-' 'ls' in statement The above error is a legitimate parse error. If we want to fix it with a better error message, we should catch the parser error and throw a beter message (the way you have put it in CliDriver). The approach you have taken is difficult to maintain in the long run. Introduce proper error messages, when explain fails on commands that otherwise could be succesful (commands that are not analyzed by Semantic Analyzer) --- Key: HIVE-2525 URL: https://issues.apache.org/jira/browse/HIVE-2525 Project: Hive Issue Type: Improvement Reporter: Robert Surówka Assignee: Robert Surówka Priority: Trivial Attachments: HIVE-2525.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2253: - Status: Open (was: Patch Available) Merge failing of join tree in exceptional case -- Key: HIVE-2253 URL: https://issues.apache.org/jira/browse/HIVE-2253 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Environment: hadoop 0.20.2, hive 0.7.0 Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2253-0.8.0.patch In some very exceptional cases, SemanticAnayzer fails to merge join tree. Example is below. create table a (val1 int, val2 int) create table b (val1 int, val2 int) create table c (val1 int, val2 int) create table d (val1 int, val2 int) create table e (val1 int, val2 int) 1. all same(single) join key -- one MR, good select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val1=e.val1 2. two join keys -- expected to have two MR, but resulted to three MR select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val2=e.val2 3. by changing the join order, we could attain two MR as first-expectation. select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135237#comment-13135237 ] Namit Jain commented on HIVE-2253: -- @Navis, your code changes look good. Thanks for catching this. For testing, the convention we have adopted is to have a test file for such a case. Look at the *.q files in ql/src/test/queries/clientpositive Add a new .q file (say, mergejoins.q) Add a explain plan in the .q file to check the number of map-reduce jobs etc. You can run the test by executing: ant test -Dtestcase=TestCliDriver -Dqfile=mergejoins.q Verify that the output file mergejoins.q.out is correct, and check it in ql/src/results/clientpositive Merge failing of join tree in exceptional case -- Key: HIVE-2253 URL: https://issues.apache.org/jira/browse/HIVE-2253 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Environment: hadoop 0.20.2, hive 0.7.0 Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2253-0.8.0.patch In some very exceptional cases, SemanticAnayzer fails to merge join tree. Example is below. create table a (val1 int, val2 int) create table b (val1 int, val2 int) create table c (val1 int, val2 int) create table d (val1 int, val2 int) create table e (val1 int, val2 int) 1. all same(single) join key -- one MR, good select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val1=e.val1 2. two join keys -- expected to have two MR, but resulted to three MR select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val2=e.val2 3. by changing the join order, we could attain two MR as first-expectation. select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2522) HIVE-2446 bug (next one) - If constructor of ClientStatsPublisher throws runtime exception it will be propagated to HadoopJobExecHelper's progress method and beyond, where
[ https://issues.apache.org/jira/browse/HIVE-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated HIVE-2522: - Resolution: Fixed Fix Version/s: 0.9.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Robert! HIVE-2446 bug (next one) - If constructor of ClientStatsPublisher throws runtime exception it will be propagated to HadoopJobExecHelper's progress method and beyond, whereas it shouldn't -- Key: HIVE-2522 URL: https://issues.apache.org/jira/browse/HIVE-2522 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Fix For: 0.9.0 Attachments: HIVE-2522.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2465) Primitive Data Types returning null if the data is out of range of the data type.
[ https://issues.apache.org/jira/browse/HIVE-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135238#comment-13135238 ] Namit Jain commented on HIVE-2465: -- The changes look good to me. However, it might be better to use LOG.warn or LOG.debug instead of LOG.info - this might generate a lot of logs. Primitive Data Types returning null if the data is out of range of the data type. - Key: HIVE-2465 URL: https://issues.apache.org/jira/browse/HIVE-2465 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Environment: Hadoop 0.20.1, Hive0.9.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5) Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-2465.1.patch, HIVE-2465.patch Primitive Data Types returning null if the input data is out of range of the data type. In this case it is better to log the message with the proper message and actual data then user get to know some data is missing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2465) Primitive Data Types returning null if the data is out of range of the data type.
[ https://issues.apache.org/jira/browse/HIVE-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-2465: - Status: Open (was: Patch Available) Primitive Data Types returning null if the data is out of range of the data type. - Key: HIVE-2465 URL: https://issues.apache.org/jira/browse/HIVE-2465 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Environment: Hadoop 0.20.1, Hive0.9.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5) Reporter: Chinna Rao Lalam Assignee: Chinna Rao Lalam Attachments: HIVE-2465.1.patch, HIVE-2465.patch Primitive Data Types returning null if the input data is out of range of the data type. In this case it is better to log the message with the proper message and actual data then user get to know some data is missing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1850) alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135242#comment-13135242 ] Namit Jain commented on HIVE-1850: -- +1 alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?) -- Key: HIVE-1850 URL: https://issues.apache.org/jira/browse/HIVE-1850 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.7.0 Environment: Trunk build from a few days ago, but seen once before with older version as well. Reporter: Terje Marthinussen Assignee: Amareshwari Sriramadasu Attachments: patch-1850-2.txt, patch-1850.txt {code} create table aa ( test STRING ) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' WITH SERDEPROPERTIES (input.regex = [^\\](.*), output.format.string = $1s); {code} This will fail. Great! {code} create table aa ( test STRING ) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' WITH SERDEPROPERTIES (input.regex = (.*), output.format.string = $1s); {code} Works, no problem there. {code} alter table aa set serdeproperties (input.regex = [^\\](.*), output.format.string = $1s); {code} Wups... I can set that without any problems! {code} alter table aa set serdeproperties (input.regex = (.*), output.format.string = $1s); FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7 [^\](.*) ^) java.util.regex.PatternSyntaxException: Unclosed character class near index 7 [^\](.*) ^ at java.util.regex.Pattern.error(Pattern.java:1713) at java.util.regex.Pattern.clazz(Pattern.java:2254) at java.util.regex.Pattern.sequence(Pattern.java:1818) at java.util.regex.Pattern.expr(Pattern.java:1752) at java.util.regex.Pattern.compile(Pattern.java:1460) at java.util.regex.Pattern.init(Pattern.java:1133) at java.util.regex.Pattern.compile(Pattern.java:847) at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484) at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558) at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) {code} After this, all further commands on the table fails, including drop table :) 1. The alter table command should probably check the regexp just like the create table command does 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135251#comment-13135251 ] Ashutosh Chauhan commented on HIVE-2510: Oh. Since RB is not posting back on jira (earlier it used to). I have left couple of comments on RB. Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name if the given db_name doesn't exist - Key: HIVE-2510 URL: https://issues.apache.org/jira/browse/HIVE-2510 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Trivial Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, HIVE-2510.4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2501) When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed).
[ https://issues.apache.org/jira/browse/HIVE-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Surówka updated HIVE-2501: - Attachment: HIVE-2501.2.patch Thank you for the response. The current diff at: https://reviews.apache.org/r/2361/diff/#index_header I added the appropriate test. About the previous .q test, I believe that new database wouldn't be created for each .q file, unless also tables like src and src_part are recreated for each .q file - but now it is immaterial anyway. When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed). --- Key: HIVE-2501 URL: https://issues.apache.org/jira/browse/HIVE-2501 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Minor Attachments: HIVE-2501.1.patch, HIVE-2501.2.patch This bug manifested to me, when first thing I did after starting Hive, was to call use db_name; but then calling show tables; was still showing tables from the default database (and I had to call use again to actually change database). This bug might have manifested only due to a specific Hive deployment I am using (I didn't investigate this issue that deeply). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2501) When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed).
[ https://issues.apache.org/jira/browse/HIVE-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Surówka updated HIVE-2501: - Status: Patch Available (was: Open) When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed). --- Key: HIVE-2501 URL: https://issues.apache.org/jira/browse/HIVE-2501 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Minor Attachments: HIVE-2501.1.patch, HIVE-2501.2.patch This bug manifested to me, when first thing I did after starting Hive, was to call use db_name; but then calling show tables; was still showing tables from the default database (and I had to call use again to actually change database). This bug might have manifested only due to a specific Hive deployment I am using (I didn't investigate this issue that deeply). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive runtime error while run TestCliDriver auto_join21.q
Hi, Guys I met an error which is similar like described in HIVE-1478, but not exactly the same when run auto_join21.q in TestCliDriver. Do you have some ideas on this? == Re-Produce == Hive: 0.7.1 ANT: 1.8.2 Hadoop: 0.20.2 command: ant test -Dtestcase=TestCliDriver -Dqfile=auto_join21.q [junit] 11/10/25 10:03:57 INFO persistence.HashMapWrapper: maximum memory: 1048576000 [junit] 11/10/25 10:03:57 INFO persistence.HashMapWrapper: maximum memory: 1048576000 [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: Initialization Done 1 MAPJOIN [junit] 11/10/25 10:03:57 INFO exec.HashTableDummyOperator: Initialization Done 5 HASHTABLEDUMMY [junit] 11/10/25 10:03:57 INFO exec.MapOperator: Processing path pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src/kv1.txt [junit] 11/10/25 10:03:57 INFO exec.MapOperator: Processing alias src1 for file pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: *** Load from HashTable File: input : pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src/kv1.txt [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: Load back 1 hashtable file from tmp file uri:file:/tmp/root/hive_2011-10-25_10-03-53_617_6340495404173422880/-local-10003/HashTable-Stage-5/MapJoin-1--.hashtable [junit] 11/10/25 10:03:57 WARN lazybinary.LazyBinaryStruct: Extra bytes detected at the end of the row! Ignoring similar problems. [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: Load back 1 hashtable file from tmp file uri:file:/tmp/root/hive_2011-10-25_10-03-53_617_6340495404173422880/-local-10003/HashTable-Stage-5/MapJoin-2--.hashtable [junit] 11/10/25 10:03:57 WARN lazybinary.LazyBinaryStruct: Extra bytes detected at the end of the row! Ignoring similar problems. [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 forwarding 1 rows [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 forwarding 1 rows [junit] 11/10/25 10:03:57 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {key:238,value:val_238} [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:546) [junit] at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) [junit] at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) [junit] at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) [junit] Caused by: java.lang.ClassCastException: org.apache.hadoop.io.Text incompatible with org.apache.hadoop.io.BooleanWritable [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.joinObjectsRightOuterJoin(CommonJoinOperator.java:489) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.joinObjects(CommonJoinOperator.java:639) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:681) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:685) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:685) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:845) [junit] at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:264) [junit] at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) [junit] at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744) [junit] at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:78) [junit] at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) [junit] at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744) [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:527) [junit] ... 5 more [junit] [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 finished. closing... [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 forwarded 1 rows [junit] 11/10/25 10:03:57 INFO exec.MapOperator: DESERIALIZE_ERRORS:0 [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 finished. closing... [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 forwarded 1 rows [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 Close done [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 Close done [junit] 11/10/25 10:03:57 INFO exec.HashTableDummyOperator: 4 finished. closing... [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 Close done [junit] 11/10/25 10:03:57 INFO
[jira] [Commented] (HIVE-2512) After HIVE-2145, Hive disallow any use of function in cluster-by clause
[ https://issues.apache.org/jira/browse/HIVE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135312#comment-13135312 ] Hudson commented on HIVE-2512: -- Integrated in Hive-trunk-h0.21 #1032 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1032/]) HIVE-2512. After HIVE-2145, Hive disallow any use of function in cluster-by clause (Chinna Rao Lalam via Ning Zhang) nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188473 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java * /hive/trunk/ql/src/test/queries/clientnegative/orderby_function.q * /hive/trunk/ql/src/test/results/clientnegative/orderby_function.q.out After HIVE-2145, Hive disallow any use of function in cluster-by clause --- Key: HIVE-2512 URL: https://issues.apache.org/jira/browse/HIVE-2512 Project: Hive Issue Type: Bug Reporter: Ning Zhang Assignee: Chinna Rao Lalam Attachments: HIVE-2512.1.patch, HIVE-2512.patch After HIVE-2145, the following query returns a semantic analysis error: FROM src SELECT * cluster by rand(); FAILED: Error in semantic analysis: functions are not supported in order by Looking back at HIVE-2145, it's clear that the patch is more restrictive than necessary. Chinna, are you able to work on it? Please let me know if you don't have cycles to do it now. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2519) Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135314#comment-13135314 ] Hudson commented on HIVE-2519: -- Integrated in Hive-trunk-h0.21 #1032 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1032/]) HIVE-2519 Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema (Ning Zhang via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188450 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java * /hive/trunk/ql/src/test/queries/clientnegative/dyn_part4.q * /hive/trunk/ql/src/test/results/clientnegative/dyn_part4.q.out Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema Key: HIVE-2519 URL: https://issues.apache.org/jira/browse/HIVE-2519 Project: Hive Issue Type: Bug Reporter: Ning Zhang Assignee: Ning Zhang Fix For: 0.9.0 Attachments: HIVE-2519.3.patch, HIVE-2519.patch Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to: insert overwrite ... partition (p2=..., p1); which will create the wrong HDFS directory structure such as /.../p2=.../p1= This is contradictory to the metastore's assumption of the HDFS directory structure. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1567) increase hive.mapjoin.maxsize to 10 million
[ https://issues.apache.org/jira/browse/HIVE-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135315#comment-13135315 ] Hudson commented on HIVE-1567: -- Integrated in Hive-trunk-h0.21 #1032 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1032/]) HIVE-1567. Remove hive.mapjoin.maxsize - it was not being used (Asuhtosh Chauhan via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188449 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractMapJoinOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java increase hive.mapjoin.maxsize to 10 million --- Key: HIVE-1567 URL: https://issues.apache.org/jira/browse/HIVE-1567 Project: Hive Issue Type: Improvement Reporter: He Yongqiang Assignee: Ashutosh Chauhan Attachments: hive-1567.patch i saw in a very wide table, hive can process 1million rows in less than one minute (select all columns). setting the hive.mapjoin.maxsize to 100k is kind of too restrictive. Let's increase this to 10 million. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2497) partition pruning prune some right partition under specific conditions
[ https://issues.apache.org/jira/browse/HIVE-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135313#comment-13135313 ] Hudson commented on HIVE-2497: -- Integrated in Hive-trunk-h0.21 #1032 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1032/]) HIVE-2497 partition pruning prune some right partition under specific conditions (binlijin via namit) - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188507 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/ExprProcFactory.java * /hive/trunk/ql/src/test/queries/clientpositive/ppr_allchildsarenull.q * /hive/trunk/ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out partition pruning prune some right partition under specific conditions --- Key: HIVE-2497 URL: https://issues.apache.org/jira/browse/HIVE-2497 Project: Hive Issue Type: Bug Affects Versions: 0.7.1 Reporter: binlijin Assignee: binlijin Labels: patch Fix For: 0.9.0 Attachments: hive-2497.2.patch, hive-2497.patch create table src3(key string, value string) partitioned by (pt string) row format delimited fields terminated by ','; ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='2011091100') ; ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='2011091200') ; ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='2011091300') ; explain extended select user_id from ( select cast(key as int) as user_id ,case when (value like 'aaa%' or value like 'vvv%') then 1 else 0 end as tag_student from src3 ) sub where sub.tag_student 0; STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias - Map Operator Tree: sub:src3 TableScan alias: src3 Filter Operator isSamplingPred: false predicate: expr: (CASE WHEN (((value like 'aaa%') or (value like 'vvv%'))) THEN (1) ELSE (0) END 0) type: boolean Select Operator expressions: expr: UDFToInteger(key) type: int expr: CASE WHEN (((value like 'aaa%') or (value like 'vvv%'))) THEN (1) ELSE (0) END type: int outputColumnNames: _col0, _col1 Filter Operator isSamplingPred: false predicate: expr: (_col1 0) type: boolean Select Operator expressions: expr: _col0 type: int outputColumnNames: _col0 File Output Operator compressed: false GlobalTableId: 0 directory: hdfs://localhost:54310/tmp/hive-tianzhao/hive_2011-10-11_19-26-12_894_9085644225727185586/-ext-10001 NumFilesPerFileSink: 1 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0 columns.types int serialization.format 1 TotalFiles: 1 MultiFileSpray: false Needs Tagging: false Stage: Stage-0 Fetch Operator limit: -1 if we set hive.optimize.ppd=false; STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias - Map Operator Tree: sub:src3 TableScan alias: src3 Select Operator expressions: expr: UDFToInteger(key) type: int expr: CASE WHEN (((value like 'aaa%') or (value like 'vvv%'))) THEN (1) ELSE (0) END type: int outputColumnNames: _col0, _col1 Filter Operator isSamplingPred: false predicate: expr: (_col1 0) type: boolean Select Operator expressions: expr: _col0 type: int outputColumnNames: _col0 File Output Operator compressed: false
[jira] [Commented] (HIVE-2145) NPE during parsing order-by expression
[ https://issues.apache.org/jira/browse/HIVE-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135316#comment-13135316 ] Hudson commented on HIVE-2145: -- Integrated in Hive-trunk-h0.21 #1032 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1032/]) HIVE-2512. After HIVE-2145, Hive disallow any use of function in cluster-by clause (Chinna Rao Lalam via Ning Zhang) nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188473 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java * /hive/trunk/ql/src/test/queries/clientnegative/orderby_function.q * /hive/trunk/ql/src/test/results/clientnegative/orderby_function.q.out NPE during parsing order-by expression -- Key: HIVE-2145 URL: https://issues.apache.org/jira/browse/HIVE-2145 Project: Hive Issue Type: Bug Reporter: Ning Zhang Assignee: Chinna Rao Lalam Fix For: 0.9.0 Attachments: HIVE-2145.patch The following query throws NPE, where it should have throw parsing exception. hive select key, count(1) cnt from src group by key order by count(1) limit 10; select key, count(1) cnt from src group by key order by count(1) limit 10; FAILED: Hive Internal Error: java.lang.NullPointerException(null) java.lang.NullPointerException at org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:153) at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:640) at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:761) at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:88) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:125) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:102) at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:156) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:6830) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:6788) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genReduceSinkPlan(SemanticAnalyzer.java:4303) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:5461) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6022) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6607) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:790) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:209) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:286) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:514) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) We should still allow functions in the order by clause (as in MySQL) but fix the NPE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi resolved HIVE-2509. -- Resolution: Fixed Fix Version/s: 0.9.0 Release Note: This change adds parser support for typed integer literal variants (tinyint, smallint, and bigint). Hadoop Flags: Reviewed Committed to trunk. Thanks Jonathan! Can you update the wiki with the new syntax? Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Fix For: 0.9.0 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135338#comment-13135338 ] Phabricator commented on HIVE-2509: --- jsichi has accepted the revision [jira] [HIVE-2509] Literal bigint. REVISION DETAIL https://reviews.facebook.net/D15 Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Fix For: 0.9.0 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135339#comment-13135339 ] Phabricator commented on HIVE-2509: --- jsichi has accepted the revision [jira] [HIVE-2509] Literal bigint. REVISION DETAIL https://reviews.facebook.net/D15 Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Fix For: 0.9.0 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2525) Introduce proper error messages, when explain fails on commands that otherwise could be succesful (commands that are not analyzed by Semantic Analyzer)
[ https://issues.apache.org/jira/browse/HIVE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135341#comment-13135341 ] Robert Surówka commented on HIVE-2525: -- I think such an error is incorrect since dfs -ls PATH (without explain) would never throw such an error on it's own (and execution of dfs... doesn't go to semantic analysis), so I see no reason for explain of dfs to go to semantic analysis and throw semantic analysis type error. Furthermore having check for incorrect explains at the beginning saves a lot of time, otherwise all the analysis and various class instances would be need to be created only to find error that we can easily see from the very beginning. And if someone would like to add or remove a command that doesn't go to Driver to be compiled, he will need to add or remove an if in CliDriver anyway so it shouldn't be hard for him to notice that he also needs to update that list. Finally if that suggestion of fix will pass I can add appropriate tests, checking if explains errors of non-sematic-analisys commands are informative, so it will be easier to see if something is wrong after subsequent changes. The bad thing is that after this change, even for valid explain query we will check contains on that list, but fortunately since it is short it doesn't cost too much time. Introduce proper error messages, when explain fails on commands that otherwise could be succesful (commands that are not analyzed by Semantic Analyzer) --- Key: HIVE-2525 URL: https://issues.apache.org/jira/browse/HIVE-2525 Project: Hive Issue Type: Improvement Reporter: Robert Surówka Assignee: Robert Surówka Priority: Trivial Attachments: HIVE-2525.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135347#comment-13135347 ] Robert Surówka commented on HIVE-2510: -- Ok, thank you for this advice, I will try to make sure to do that in the future. Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name if the given db_name doesn't exist - Key: HIVE-2510 URL: https://issues.apache.org/jira/browse/HIVE-2510 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Trivial Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, HIVE-2510.4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135349#comment-13135349 ] jirapos...@reviews.apache.org commented on HIVE-2510: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2426/ --- (Updated 2011-10-25 19:25:02.433521) Review request for Yongqiang He, Ning Zhang and Kevin Wilfong. Summary --- Now when creating table in specific database, if that database doesn't exist Hive will return an error informing about that, instead of throwing Null Pointer Exception. Null Pointer happens during semantic analysis, so it seems reasonable to return an error before the code that would throw null pointer (Warehouse.getDatabasePath()) executes. This addresses bug HIVE-2510. https://issues.apache.org/jira/browse/HIVE-2510 Diffs - trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1187647 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1187647 trunk/ql/src/test/queries/clientnegative/create_table_failure1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/create_table_failure2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/create_table_failure3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/create_table_failure4.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/input45.q PRE-CREATION trunk/ql/src/test/results/clientnegative/create_table_failure1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/create_table_failure2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/create_table_failure3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/create_table_failure4.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/input45.q.out PRE-CREATION Diff: https://reviews.apache.org/r/2426/diff Testing --- Run some sample queries and it seemed to work. Thanks, Robert Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name if the given db_name doesn't exist - Key: HIVE-2510 URL: https://issues.apache.org/jira/browse/HIVE-2510 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Trivial Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, HIVE-2510.4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2501) When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed).
[ https://issues.apache.org/jira/browse/HIVE-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135352#comment-13135352 ] John Sichi commented on HIVE-2501: -- +1. Will commit when tests pass. When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed). --- Key: HIVE-2501 URL: https://issues.apache.org/jira/browse/HIVE-2501 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Minor Attachments: HIVE-2501.1.patch, HIVE-2501.2.patch This bug manifested to me, when first thing I did after starting Hive, was to call use db_name; but then calling show tables; was still showing tables from the default database (and I had to call use again to actually change database). This bug might have manifested only due to a specific Hive deployment I am using (I didn't investigate this issue that deeply). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2368) Slow dropping of partitions caused by full listing of storage descriptors
[ https://issues.apache.org/jira/browse/HIVE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-2368: Summary: Slow dropping of partitions caused by full listing of storage descriptors (was: Determining whether a Column Descriptor is unused may take too long) Slow dropping of partitions caused by full listing of storage descriptors - Key: HIVE-2368 URL: https://issues.apache.org/jira/browse/HIVE-2368 Project: Hive Issue Type: Bug Components: Metastore Reporter: Sohan Jain Attachments: HIVE-2368.1.patch To determine if a column descriptor is unused, we call listStorageDescriptorsWithCD(), which may return a big list of SDs. This can severely slow down dropping partitions. We can add a maximum number of SDs to return, and just ask for 1 SD, since we are just doing an existential check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135382#comment-13135382 ] Phabricator commented on HIVE-2509: --- jonchang has committed the revision [jira] [HIVE-2509] Literal bigint. REVISION DETAIL https://reviews.facebook.net/D15 Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Fix For: 0.9.0 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2515) Make Constant OIs work with UDTFs.
[ https://issues.apache.org/jira/browse/HIVE-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135381#comment-13135381 ] Phabricator commented on HIVE-2515: --- jonchang has committed the revision [jira] [HIVE-2515] Make Constant OIs work with UDTFs.. REVISION DETAIL https://reviews.facebook.net/D33 Make Constant OIs work with UDTFs. -- Key: HIVE-2515 URL: https://issues.apache.org/jira/browse/HIVE-2515 Project: Hive Issue Type: Improvement Reporter: Jonathan Chang Assignee: Jonathan Chang Priority: Minor Fix For: 0.9.0 Attachments: D33.1.patch, D33.1.patch, D33.2.patch, D33.2.patch, D33.2.patch, D33.3.patch, D33.3.patch, D33.3.patch, junit-noframes.html UDTFs are the last shoe to drop for constant OIs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135385#comment-13135385 ] Phabricator commented on HIVE-2509: --- jonchang has committed the revision [jira] [HIVE-2509] Literal bigint. REVISION DETAIL https://reviews.facebook.net/D15 Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Fix For: 0.9.0 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2368) Slow dropping of partitions caused by full listing of storage descriptors
[ https://issues.apache.org/jira/browse/HIVE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135387#comment-13135387 ] Paul Yang commented on HIVE-2368: - Committed. Thanks Sohan! Slow dropping of partitions caused by full listing of storage descriptors - Key: HIVE-2368 URL: https://issues.apache.org/jira/browse/HIVE-2368 Project: Hive Issue Type: Bug Components: Metastore Reporter: Sohan Jain Assignee: Sohan Jain Fix For: 0.9.0 Attachments: HIVE-2368.1.patch To determine if a column descriptor is unused, we call listStorageDescriptorsWithCD(), which may return a big list of SDs. This can severely slow down dropping partitions. We can add a maximum number of SDs to return, and just ask for 1 SD, since we are just doing an existential check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2368) Slow dropping of partitions caused by full listing of storage descriptors
[ https://issues.apache.org/jira/browse/HIVE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang resolved HIVE-2368. - Resolution: Fixed Fix Version/s: 0.9.0 Slow dropping of partitions caused by full listing of storage descriptors - Key: HIVE-2368 URL: https://issues.apache.org/jira/browse/HIVE-2368 Project: Hive Issue Type: Bug Components: Metastore Reporter: Sohan Jain Assignee: Sohan Jain Fix For: 0.9.0 Attachments: HIVE-2368.1.patch To determine if a column descriptor is unused, we call listStorageDescriptorsWithCD(), which may return a big list of SDs. This can severely slow down dropping partitions. We can add a maximum number of SDs to return, and just ask for 1 SD, since we are just doing an existential check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135398#comment-13135398 ] Jonathan Chang commented on HIVE-2509: -- How do I get edit rights to the wiki? Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Fix For: 0.9.0 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Hive runtime error while run TestCliDriver auto_join21.q
Bing, I tried ant test -Dtestcase=TestCliDriver -Dqfile=auto_join21.q on trunk and it succeeded. Is it feasible for you to upgrade? Thanks, Ashutosh On Tue, Oct 25, 2011 at 11:27, Bing Li lib...@yahoo.com.cn wrote: Hi, Guys I met an error which is similar like described in HIVE-1478, but not exactly the same when run auto_join21.q in TestCliDriver. Do you have some ideas on this? == Re-Produce == Hive: 0.7.1 ANT: 1.8.2 Hadoop: 0.20.2 command: ant test -Dtestcase=TestCliDriver -Dqfile=auto_join21.q [junit] 11/10/25 10:03:57 INFO persistence.HashMapWrapper: maximum memory: 1048576000 [junit] 11/10/25 10:03:57 INFO persistence.HashMapWrapper: maximum memory: 1048576000 [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: Initialization Done 1 MAPJOIN [junit] 11/10/25 10:03:57 INFO exec.HashTableDummyOperator: Initialization Done 5 HASHTABLEDUMMY [junit] 11/10/25 10:03:57 INFO exec.MapOperator: Processing path pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src/kv1.txt [junit] 11/10/25 10:03:57 INFO exec.MapOperator: Processing alias src1 for file pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: *** Load from HashTable File: input : pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src/kv1.txt [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator:Load back 1 hashtable file from tmp file uri:file:/tmp/root/hive_2011-10-25_10-03-53_617_6340495404173422880/-local-10003/HashTable-Stage-5/MapJoin-1--.hashtable [junit] 11/10/25 10:03:57 WARN lazybinary.LazyBinaryStruct: Extra bytes detected at the end of the row! Ignoring similar problems. [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator:Load back 1 hashtable file from tmp file uri:file:/tmp/root/hive_2011-10-25_10-03-53_617_6340495404173422880/-local-10003/HashTable-Stage-5/MapJoin-2--.hashtable [junit] 11/10/25 10:03:57 WARN lazybinary.LazyBinaryStruct: Extra bytes detected at the end of the row! Ignoring similar problems. [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 forwarding 1 rows [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 forwarding 1 rows [junit] 11/10/25 10:03:57 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {key:238,value:val_238} [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:546) [junit] at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) [junit] at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) [junit] at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) [junit] Caused by: java.lang.ClassCastException: org.apache.hadoop.io.Text incompatible with org.apache.hadoop.io.BooleanWritable [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.joinObjectsRightOuterJoin(CommonJoinOperator.java:489) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.joinObjects(CommonJoinOperator.java:639) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:681) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:685) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:685) [junit] at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:845) [junit] at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:264) [junit] at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) [junit] at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744) [junit] at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:78) [junit] at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) [junit] at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744) [junit] at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:527) [junit] ... 5 more [junit] [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 finished. closing... [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 forwarded 1 rows [junit] 11/10/25 10:03:57 INFO exec.MapOperator: DESERIALIZE_ERRORS:0 [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 finished. closing... [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 forwarded 1 rows [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0
[jira] [Commented] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135411#comment-13135411 ] John Sichi commented on HIVE-2509: -- Try now...I just granted you write access. Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Fix For: 0.9.0 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135455#comment-13135455 ] jirapos...@reviews.apache.org commented on HIVE-2510: - bq. On 2011-10-25 17:24:41, Ashutosh Chauhan wrote: bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java, lines 6923-6926 bq. https://reviews.apache.org/r/2426/diff/4/?file=52450#file52450line6923 bq. bq. Not sure if we need this. Checking of existing table already happens in analyzeCreateTable() around line 7847. So, this check at this point is redundant. bq. bq. Robert Surówka wrote: bq. Without this check a null pointer exception was being thrown (for table created as select). (Here I check for existing database, not table) - Robert --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2426/#review2823 --- On 2011-10-25 19:25:02, Robert Surówka wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2426/ bq. --- bq. bq. (Updated 2011-10-25 19:25:02) bq. bq. bq. Review request for Yongqiang He, Ning Zhang and Kevin Wilfong. bq. bq. bq. Summary bq. --- bq. bq. Now when creating table in specific database, if that database doesn't exist Hive will return an error informing about that, instead of throwing Null Pointer Exception. Null Pointer happens during semantic analysis, so it seems reasonable to return an error before the code that would throw null pointer (Warehouse.getDatabasePath()) executes. bq. bq. bq. This addresses bug HIVE-2510. bq. https://issues.apache.org/jira/browse/HIVE-2510 bq. bq. bq. Diffs bq. - bq. bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1187647 bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1187647 bq.trunk/ql/src/test/queries/clientnegative/create_table_failure1.q PRE-CREATION bq.trunk/ql/src/test/queries/clientnegative/create_table_failure2.q PRE-CREATION bq.trunk/ql/src/test/queries/clientnegative/create_table_failure3.q PRE-CREATION bq.trunk/ql/src/test/queries/clientnegative/create_table_failure4.q PRE-CREATION bq.trunk/ql/src/test/queries/clientpositive/input45.q PRE-CREATION bq.trunk/ql/src/test/results/clientnegative/create_table_failure1.q.out PRE-CREATION bq.trunk/ql/src/test/results/clientnegative/create_table_failure2.q.out PRE-CREATION bq.trunk/ql/src/test/results/clientnegative/create_table_failure3.q.out PRE-CREATION bq.trunk/ql/src/test/results/clientnegative/create_table_failure4.q.out PRE-CREATION bq.trunk/ql/src/test/results/clientpositive/input45.q.out PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/2426/diff bq. bq. bq. Testing bq. --- bq. bq. Run some sample queries and it seemed to work. bq. bq. bq. Thanks, bq. bq. Robert bq. bq. Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name if the given db_name doesn't exist - Key: HIVE-2510 URL: https://issues.apache.org/jira/browse/HIVE-2510 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Trivial Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, HIVE-2510.4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Surówka updated HIVE-2510: - Attachment: HIVE-2510.5.patch Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name if the given db_name doesn't exist - Key: HIVE-2510 URL: https://issues.apache.org/jira/browse/HIVE-2510 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Trivial Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, HIVE-2510.4.patch, HIVE-2510.5.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist
[ https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135495#comment-13135495 ] jirapos...@reviews.apache.org commented on HIVE-2510: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2426/ --- (Updated 2011-10-25 22:24:42.563999) Review request for Yongqiang He, Ning Zhang and Kevin Wilfong. Changes --- Remove is_database_exist() method from HiveMetaStore, now check whether database exist is done directly with use of getDatabase(). Summary --- Now when creating table in specific database, if that database doesn't exist Hive will return an error informing about that, instead of throwing Null Pointer Exception. Null Pointer happens during semantic analysis, so it seems reasonable to return an error before the code that would throw null pointer (Warehouse.getDatabasePath()) executes. This addresses bug HIVE-2510. https://issues.apache.org/jira/browse/HIVE-2510 Diffs (updated) - trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1188369 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1188369 trunk/ql/src/test/queries/clientnegative/create_table_failure1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/create_table_failure2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/create_table_failure3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/create_table_failure4.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/input45.q PRE-CREATION trunk/ql/src/test/results/clientnegative/create_table_failure1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/create_table_failure2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/create_table_failure3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/create_table_failure4.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/input45.q.out PRE-CREATION Diff: https://reviews.apache.org/r/2426/diff Testing --- Run some sample queries and it seemed to work. Thanks, Robert Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name if the given db_name doesn't exist - Key: HIVE-2510 URL: https://issues.apache.org/jira/browse/HIVE-2510 Project: Hive Issue Type: Bug Reporter: Robert Surówka Assignee: Robert Surówka Priority: Trivial Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, HIVE-2510.4.patch, HIVE-2510.5.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-trunk-h0.21 - Build # 1033 - Failure
Changes for Build #1032 [namit] HIVE-2497 partition pruning prune some right partition under specific conditions (binlijin via namit) - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - [nzhang] HIVE-2512. After HIVE-2145, Hive disallow any use of function in cluster-by clause (Chinna Rao Lalam via Ning Zhang) [namit] HIVE-2519 Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema (Ning Zhang via namit) [namit] HIVE-1567. Remove hive.mapjoin.maxsize - it was not being used (Asuhtosh Chauhan via namit) Changes for Build #1033 [nzhang] HIVE-2446. bug (next one) - If constructor of ClientStatsPublisher throws runtime exception it will be propagated to HadoopJobExecHelper progress method and beyond, whereas it shouldnot 6 tests failed. FAILED: org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe(TestNegativeMinimrCliDriver.java:115) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) FAILED: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherOneStat Error Message: null Stack Trace: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.Utilities.prepareWithRetry(Utilities.java:2191) at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsAggregator.cleanUp(JDBCStatsAggregator.java:215) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.setUp(TestStatsPublisherEnhanced.java:60) at junit.framework.TestCase.runBare(TestCase.java:132) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) FAILED: org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisher Error Message: null Stack Trace: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.Utilities.prepareWithRetry(Utilities.java:2191) at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsAggregator.cleanUp(JDBCStatsAggregator.java:215) at org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.setUp(TestStatsPublisherEnhanced.java:60) at junit.framework.TestCase.runBare(TestCase.java:132) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at
[jira] [Commented] (HIVE-2446) Introduction of client statistics publishers possibility
[ https://issues.apache.org/jira/browse/HIVE-2446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135498#comment-13135498 ] Hudson commented on HIVE-2446: -- Integrated in Hive-trunk-h0.21 #1033 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1033/]) HIVE-2446. bug (next one) - If constructor of ClientStatsPublisher throws runtime exception it will be propagated to HadoopJobExecHelper progress method and beyond, whereas it shouldnot nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188796 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java Introduction of client statistics publishers possibility Key: HIVE-2446 URL: https://issues.apache.org/jira/browse/HIVE-2446 Project: Hive Issue Type: Improvement Components: Clients, Statistics Reporter: Robert Surówka Assignee: Robert Surówka Priority: Minor Fix For: 0.8.0 Attachments: HIVE-2446.1.patch, HIVE-2446.1.patch, HIVE-2446.2.patch, HIVE-2446.2.patch, HIVE-2446.3.patch, HIVE-2446.3.patch Original Estimate: 1h Remaining Estimate: 1h The purpose of this change is to allow publication or storage of counters while the job is running. Introduced two new variables to hive-default.xml and HiveConf.java: hive.client.stats.publishers and hive.client.stats.counters. First one specifies classes names, whose instances will be executed by HadoopJobExecHelper.java (similarly as hooks are) in its method progress(ExecDriverTaskHandle): MapRedStats. Second one specifies list of counters that any client stat publishers should publish or stored. Details regarding format of this list is up to a specific deployment (it is up to client stats publishers to parse it), yet it is required to use display names of counter groups and counters. Added interface ClientStatsPublishers in org.apache.hadoop.hive.ql.stats package, that must be implemented by all stats publishers. Added code to progress(ExecDriverTaskHandle): MapRedStats from HadoopJobExecHelper.java that puts counters' values to a Java map and then executes registered client stats publishers giving them that map and running job id. Added two new methods to HadoopJobExecHelper: extractAllCounterValues(Counters) and getClientStatsPublishers() that are used by code from previous sentence. Made cosmetic changes in two other classes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Hive-0.8.0-SNAPSHOT-h0.21 - Build # 70 - Failure
Changes for Build #70 1 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample6 Error Message: Timeout occurred. Please note the time in the report does not reflect the time until the timeout. Stack Trace: junit.framework.AssertionFailedError: Timeout occurred. Please note the time in the report does not reflect the time until the timeout. at net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259) at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268) at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:324) at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244) The Apache Jenkins build system has built Hive-0.8.0-SNAPSHOT-h0.21 (build #70) Status: Failure Check console output at https://builds.apache.org/job/Hive-0.8.0-SNAPSHOT-h0.21/70/ to view the results.
[jira] [Created] (HIVE-2527) Consecutive string literals should be combined into a single string literal.
Consecutive string literals should be combined into a single string literal. Key: HIVE-2527 URL: https://issues.apache.org/jira/browse/HIVE-2527 Project: Hive Issue Type: Improvement Reporter: Jonathan Chang Assignee: Jonathan Chang Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2527) Consecutive string literals should be combined into a single string literal.
[ https://issues.apache.org/jira/browse/HIVE-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Chang updated HIVE-2527: - Description: C, Python, etc. all support this magical feature. Consecutive string literals should be combined into a single string literal. Key: HIVE-2527 URL: https://issues.apache.org/jira/browse/HIVE-2527 Project: Hive Issue Type: Improvement Reporter: Jonathan Chang Assignee: Jonathan Chang Priority: Minor C, Python, etc. all support this magical feature. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2509) Literal bigint
[ https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135624#comment-13135624 ] Hudson commented on HIVE-2509: -- Integrated in Hive-trunk-h0.21 #1034 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1034/]) HIVE-2509. Literal bigint (Jonathan Chang via jvs) jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188858 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java * /hive/trunk/ql/src/test/queries/clientpositive/literal_ints.q * /hive/trunk/ql/src/test/results/clientpositive/literal_ints.q.out Literal bigint -- Key: HIVE-2509 URL: https://issues.apache.org/jira/browse/HIVE-2509 Project: Hive Issue Type: New Feature Reporter: Jonathan Chang Assignee: Jonathan Chang Fix For: 0.9.0 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch Typing bigints is a pain because you have to write CAST(0 AS BIGINT). We should let people type 0L instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2360) create dynamic partition if and only if intermediate source has files
[ https://issues.apache.org/jira/browse/HIVE-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135732#comment-13135732 ] Hudson commented on HIVE-2360: -- Integrated in Hive-trunk-h0.21 #1035 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1035/]) HIVE-2360 create dynamic partition if and only if intermediate source has files (Franklin Hu, Kaushik Lakshminarayanan via Siying Dong) sdong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188989 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java create dynamic partition if and only if intermediate source has files - Key: HIVE-2360 URL: https://issues.apache.org/jira/browse/HIVE-2360 Project: Hive Issue Type: Bug Reporter: Franklin Hu Priority: Minor Attachments: hive-2360.1.patch, hive-2360.2.patch There are some conditions under which a partition description is created due to insert overwriting a table using dynamic partitioning for partitions that that are empty (have no files). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira