[jira] [Resolved] (HIVE-46) derbyclient.jar and derbytools.jar should be included in hive/lib

2011-10-25 Thread Ashutosh Chauhan (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-46.
--

Resolution: Fixed

derby.jar is now present in lib/ dir. No more NPE.

 derbyclient.jar and derbytools.jar should be included in hive/lib
 -

 Key: HIVE-46
 URL: https://issues.apache.org/jira/browse/HIVE-46
 Project: Hive
  Issue Type: Bug
  Components: Metastore
 Environment: Cent OS 5 x86-64 JDK 1.6
Reporter: Edward Capriolo
Priority: Minor

 Attempting to use the Derby ClientDriver will cause a null pointer exception 
 because derbyclient.jar and derbytools.jar are not in lib.
 hive show tables;
 08/10/02 15:17:12 INFO hive.metastore: Trying to connect to metastore
 with URI jdbc:derby://nyhadoop1:1527/metastore_db
 FAILED: Error in semantic analysis: java.lang.NullPointerException
 08/10/02 15:17:12 ERROR ql.Driver: FAILED: Error in semantic analysis:
 java.lang.NullPointerException
 derbyclient.jar and derbytools.jar should probably be put in the hive/lib 
 directory. Also this type of exception could happen often with different JDBC 
 libraries the exception should be trapped better

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-1461) Clean up references to 'hive.metastore.local'

2011-10-25 Thread Ashutosh Chauhan (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-1461:
---

Attachment: hive-1461.patch

Patch for adding hive.metastore.local in HiveConf

 Clean up references to 'hive.metastore.local'
 -

 Key: HIVE-1461
 URL: https://issues.apache.org/jira/browse/HIVE-1461
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Paul Yang
Priority: Minor
 Attachments: hive-1461.patch


 'hive.metastore.local' should not be referred directly as a string. Instead, 
 a HiveConf.ConfVar entry should be created and used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-1461) Clean up references to 'hive.metastore.local'

2011-10-25 Thread Ashutosh Chauhan (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-1461:
---

 Assignee: Ashutosh Chauhan
Affects Version/s: (was: 0.7.0)
   Status: Patch Available  (was: Open)

 Clean up references to 'hive.metastore.local'
 -

 Key: HIVE-1461
 URL: https://issues.apache.org/jira/browse/HIVE-1461
 Project: Hive
  Issue Type: Bug
Reporter: Paul Yang
Assignee: Ashutosh Chauhan
Priority: Minor
 Attachments: hive-1461.patch


 'hive.metastore.local' should not be referred directly as a string. Instead, 
 a HiveConf.ConfVar entry should be created and used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2509) Literal bigint

2011-10-25 Thread Jonathan Chang (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135093#comment-13135093
 ] 

Jonathan Chang commented on HIVE-2509:
--

Done.

 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2509) Literal bigint

2011-10-25 Thread Jonathan Chang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Chang updated HIVE-2509:
-

Attachment: D15.4.patch

 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2372) java.io.IOException: error=7, Argument list too long

2011-10-25 Thread Edward Capriolo (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135111#comment-13135111
 ] 

Edward Capriolo commented on HIVE-2372:
---

I am surprised no one has ran into this sooner. I had assumed the limit was 
much shorter. Streaming is 'a neat hack' but with the UDF/UDAF frameworks I 
have never been convinced it is needed. Most often I see use it improperly to 
make hacky map side joins etc. Maybe your real problem is having to many input 
files and you should merge your input first, or use bucketing instead of two 
level partitioning.   

 java.io.IOException: error=7, Argument list too long
 

 Key: HIVE-2372
 URL: https://issues.apache.org/jira/browse/HIVE-2372
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Sergey Tryuber
Priority: Critical

 I execute a huge query on a table with a lot of 2-level partitions. There is 
 a perl reducer in my query. Maps worked ok, but every reducer fails with the 
 following exception:
 2011-08-11 04:58:29,865 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: 
 Executing [/usr/bin/perl, reducer.pl, my_argument]
 2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: 
 tablename=null
 2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: 
 partname=null
 2011-08-11 04:58:29,866 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: 
 alias=null
 2011-08-11 04:58:29,935 FATAL ExecReducer: 
 org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
 processing row (tag=0) 
 {key:{reducesinkkey0:129390185139228,reducesinkkey1:8AF163CA6F},value:{_col0:8AF163CA6F,_col1:2011-07-27
  
 22:48:52,_col2:129390185139228,_col3:2006,_col4:4100,_col5:10017388=6,_col6:1063,_col7:NULL,_col8:address.com,_col9:NULL,_col10:NULL},alias:0}
   at 
 org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:256)
   at 
 org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:468)
   at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:416)
   at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:396)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
   at org.apache.hadoop.mapred.Child.main(Child.java:262)
 Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Cannot 
 initialize ScriptOperator
   at 
 org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:320)
   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
   at 
 org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
   at 
 org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
   at 
 org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:247)
   ... 7 more
 Caused by: java.io.IOException: Cannot run program /usr/bin/perl: 
 java.io.IOException: error=7, Argument list too long
   at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
   at 
 org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:279)
   ... 15 more
 Caused by: java.io.IOException: java.io.IOException: error=7, Argument list 
 too long
   at java.lang.UNIXProcess.init(UNIXProcess.java:148)
   at java.lang.ProcessImpl.start(ProcessImpl.java:65)
   at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
   ... 16 more
 It seems to me, I found the cause. ScriptOperator.java puts a lot of configs 
 as environment variables to the child reduce process. One of variables is 
 mapred.input.dir, which in my case more than 150KB. There are a huge amount 
 of input directories in this variable. In short, the problem is that Linux 
 (up to 2.6.23 kernel version) limits summary size of environment variables 
 for child processes to 132KB. This problem could be solved by upgrading the 
 kernel. But strings limitations still be 132KB per string in environment 
 variable. So such huge variable doesn't work even on my home computer 
 (2.6.32). You can read more information on 
 (http://www.kernel.org/doc/man-pages/online/pages/man2/execve.2.html).
 For now all our work has been stopped because of this problem and I can't 
 find the solution. The only solution, which seems to me more reasonable is to 
 

[jira] [Commented] (HIVE-1461) Clean up references to 'hive.metastore.local'

2011-10-25 Thread Namit Jain (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135221#comment-13135221
 ] 

Namit Jain commented on HIVE-1461:
--

+1

 Clean up references to 'hive.metastore.local'
 -

 Key: HIVE-1461
 URL: https://issues.apache.org/jira/browse/HIVE-1461
 Project: Hive
  Issue Type: Bug
Reporter: Paul Yang
Assignee: Ashutosh Chauhan
Priority: Minor
 Attachments: hive-1461.patch


 'hive.metastore.local' should not be referred directly as a string. Instead, 
 a HiveConf.ConfVar entry should be created and used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2525) Introduce proper error messages, when explain fails on commands that otherwise could be succesful (commands that are not analyzed by Semantic Analyzer)

2011-10-25 Thread Namit Jain (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2525:
-

Status: Open  (was: Patch Available)

 Introduce proper error messages, when explain fails on commands that 
 otherwise could be succesful (commands that are not analyzed by Semantic 
 Analyzer)
 ---

 Key: HIVE-2525
 URL: https://issues.apache.org/jira/browse/HIVE-2525
 Project: Hive
  Issue Type: Improvement
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Trivial
 Attachments: HIVE-2525.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2525) Introduce proper error messages, when explain fails on commands that otherwise could be succesful (commands that are not analyzed by Semantic Analyzer)

2011-10-25 Thread Namit Jain (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135227#comment-13135227
 ] 

Namit Jain commented on HIVE-2525:
--

hive explain dfs -ls PATH;
FAILED: Parse Error: line 1:8 cannot recognize input near 'dfs' '-' 'ls' in 
statement


The above error is a legitimate parse error.
If we want to fix it with a better error message, we should catch the parser 
error and throw
a beter message (the way you have put it in CliDriver). The approach you have 
taken is difficult
to maintain in the long run. 

 Introduce proper error messages, when explain fails on commands that 
 otherwise could be succesful (commands that are not analyzed by Semantic 
 Analyzer)
 ---

 Key: HIVE-2525
 URL: https://issues.apache.org/jira/browse/HIVE-2525
 Project: Hive
  Issue Type: Improvement
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Trivial
 Attachments: HIVE-2525.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case

2011-10-25 Thread Namit Jain (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2253:
-

Status: Open  (was: Patch Available)

 Merge failing of join tree in exceptional case
 --

 Key: HIVE-2253
 URL: https://issues.apache.org/jira/browse/HIVE-2253
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.7.0
 Environment: hadoop 0.20.2, hive 0.7.0
Reporter: Navis
Assignee: Navis
Priority: Minor
 Attachments: HIVE-2253-0.8.0.patch


 In some very exceptional cases, SemanticAnayzer fails to merge join tree. 
 Example is below.
 create table a (val1 int, val2 int)
 create table b (val1 int, val2 int)
 create table c (val1 int, val2 int)
 create table d (val1 int, val2 int)
 create table e (val1 int, val2 int)
 1. all same(single) join key -- one MR, good
 select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on 
 a.val1=d.val1 join e on a.val1=e.val1
 2. two join keys -- expected to have two MR, but resulted to three MR
 select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on 
 a.val1=d.val1 join e on a.val2=e.val2
 3. by changing the join order, we could attain two MR as first-expectation.
 select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on 
 a.val1=d.val1 join b on a.val1=b.val1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2253) Merge failing of join tree in exceptional case

2011-10-25 Thread Namit Jain (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135237#comment-13135237
 ] 

Namit Jain commented on HIVE-2253:
--

@Navis, your code changes look good.
Thanks for catching this.

For testing, the convention we have adopted is to have a test file for such a 
case.
Look at the *.q files in ql/src/test/queries/clientpositive

Add a new .q file (say, mergejoins.q)
Add a explain plan in the .q file to check the number of map-reduce jobs etc.

You can run the test by executing:

ant test -Dtestcase=TestCliDriver -Dqfile=mergejoins.q 

Verify that the output file mergejoins.q.out is correct, and check it in
ql/src/results/clientpositive




 Merge failing of join tree in exceptional case
 --

 Key: HIVE-2253
 URL: https://issues.apache.org/jira/browse/HIVE-2253
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.7.0
 Environment: hadoop 0.20.2, hive 0.7.0
Reporter: Navis
Assignee: Navis
Priority: Minor
 Attachments: HIVE-2253-0.8.0.patch


 In some very exceptional cases, SemanticAnayzer fails to merge join tree. 
 Example is below.
 create table a (val1 int, val2 int)
 create table b (val1 int, val2 int)
 create table c (val1 int, val2 int)
 create table d (val1 int, val2 int)
 create table e (val1 int, val2 int)
 1. all same(single) join key -- one MR, good
 select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on 
 a.val1=d.val1 join e on a.val1=e.val1
 2. two join keys -- expected to have two MR, but resulted to three MR
 select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on 
 a.val1=d.val1 join e on a.val2=e.val2
 3. by changing the join order, we could attain two MR as first-expectation.
 select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on 
 a.val1=d.val1 join b on a.val1=b.val1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2522) HIVE-2446 bug (next one) - If constructor of ClientStatsPublisher throws runtime exception it will be propagated to HadoopJobExecHelper's progress method and beyond, where

2011-10-25 Thread Ning Zhang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ning Zhang updated HIVE-2522:
-

   Resolution: Fixed
Fix Version/s: 0.9.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Robert!

 HIVE-2446 bug (next one) - If constructor of ClientStatsPublisher throws 
 runtime exception it will be propagated to HadoopJobExecHelper's progress 
 method and beyond, whereas it shouldn't
 --

 Key: HIVE-2522
 URL: https://issues.apache.org/jira/browse/HIVE-2522
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
 Fix For: 0.9.0

 Attachments: HIVE-2522.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2465) Primitive Data Types returning null if the data is out of range of the data type.

2011-10-25 Thread Namit Jain (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135238#comment-13135238
 ] 

Namit Jain commented on HIVE-2465:
--

The changes look good to me.
However, it might be better to use LOG.warn or LOG.debug instead of LOG.info - 
this might generate a lot of logs.

 Primitive Data Types returning null if the data is out of range of the data 
 type.
 -

 Key: HIVE-2465
 URL: https://issues.apache.org/jira/browse/HIVE-2465
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.9.0
 Environment: Hadoop 0.20.1, Hive0.9.0 and SUSE Linux Enterprise 
 Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
 Attachments: HIVE-2465.1.patch, HIVE-2465.patch


 Primitive Data Types returning null if the input data is out of range of the 
 data type. In this case it is better to log the message with the proper 
 message and actual data then user get to know some data is missing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2465) Primitive Data Types returning null if the data is out of range of the data type.

2011-10-25 Thread Namit Jain (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2465:
-

Status: Open  (was: Patch Available)

 Primitive Data Types returning null if the data is out of range of the data 
 type.
 -

 Key: HIVE-2465
 URL: https://issues.apache.org/jira/browse/HIVE-2465
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.9.0
 Environment: Hadoop 0.20.1, Hive0.9.0 and SUSE Linux Enterprise 
 Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
 Attachments: HIVE-2465.1.patch, HIVE-2465.patch


 Primitive Data Types returning null if the input data is out of range of the 
 data type. In this case it is better to log the message with the proper 
 message and actual data then user get to know some data is missing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-1850) alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)

2011-10-25 Thread Namit Jain (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135242#comment-13135242
 ] 

Namit Jain commented on HIVE-1850:
--

+1



 alter table set serdeproperties bypasses regexps checks (leaves table in a 
 non-recoverable state?)
 --

 Key: HIVE-1850
 URL: https://issues.apache.org/jira/browse/HIVE-1850
 Project: Hive
  Issue Type: Bug
  Components: Serializers/Deserializers
Affects Versions: 0.7.0
 Environment: Trunk build from a few days ago, but seen once before 
 with older version as well.
Reporter: Terje Marthinussen
Assignee: Amareshwari Sriramadasu
 Attachments: patch-1850-2.txt, patch-1850.txt


 {code}
 create table aa ( test STRING )
   ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
   WITH SERDEPROPERTIES (input.regex = [^\\](.*), output.format.string = 
 $1s);
 {code}
 This will fail. Great!
 {code}
 create table aa ( test STRING )
   ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
   WITH SERDEPROPERTIES (input.regex = (.*), output.format.string = 
 $1s);
 {code}
 Works, no problem there.
 {code}
 alter table aa set serdeproperties (input.regex = [^\\](.*), 
 output.format.string = $1s);
 {code}
 Wups... I can set that without any problems!
 {code}
 alter table aa set serdeproperties (input.regex = (.*), 
 output.format.string = $1s);
 FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed 
 character class near index 7
 [^\](.*)
^)
 java.util.regex.PatternSyntaxException: Unclosed character class near index 7
 [^\](.*)
^
   at java.util.regex.Pattern.error(Pattern.java:1713)
   at java.util.regex.Pattern.clazz(Pattern.java:2254)
   at java.util.regex.Pattern.sequence(Pattern.java:1818)
   at java.util.regex.Pattern.expr(Pattern.java:1752)
   at java.util.regex.Pattern.compile(Pattern.java:1460)
   at java.util.regex.Pattern.init(Pattern.java:1133)
   at java.util.regex.Pattern.compile(Pattern.java:847)
   at 
 org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
   at 
 org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
   at 
 org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
   at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
   at 
 org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
   at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
   at 
 org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
   at 
 org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
   at 
 org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
 {code}
 After this, all further commands on the table fails, including drop table :)
 1. The alter table command should probably check the regexp just like the 
 create table command does
 2. Even though the regexp is bad, it should be possible to do things like set 
 the regexp again or drop the table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist

2011-10-25 Thread Ashutosh Chauhan (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135251#comment-13135251
 ] 

Ashutosh Chauhan commented on HIVE-2510:


Oh. Since RB is not posting back on jira (earlier it used to). I have left 
couple of comments on RB. 

 Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name 
  if the given db_name doesn't exist
 -

 Key: HIVE-2510
 URL: https://issues.apache.org/jira/browse/HIVE-2510
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Trivial
 Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, 
 HIVE-2510.4.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2501) When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed).

2011-10-25 Thread Updated

 [ 
https://issues.apache.org/jira/browse/HIVE-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Surówka updated HIVE-2501:
-

Attachment: HIVE-2501.2.patch

Thank you for the response.

The current diff at: https://reviews.apache.org/r/2361/diff/#index_header

I added the appropriate test.

About the previous .q test, I believe that new database wouldn't be created for 
each .q file, unless also tables like src and src_part are recreated for each 
.q file - but now it is immaterial anyway.   

 When new instance of Hive (class) is created, the current database is reset 
 to default (current database shouldn't be changed).
 ---

 Key: HIVE-2501
 URL: https://issues.apache.org/jira/browse/HIVE-2501
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Minor
 Attachments: HIVE-2501.1.patch, HIVE-2501.2.patch


 This bug manifested to me, when first thing I did after starting Hive, was to 
 call use db_name; but then calling show tables; was still showing tables 
 from the default database (and I had to call use again to actually change 
 database). This bug might have manifested only due to a specific Hive 
 deployment I am using (I didn't investigate this issue that deeply). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2501) When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed).

2011-10-25 Thread Updated

 [ 
https://issues.apache.org/jira/browse/HIVE-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Surówka updated HIVE-2501:
-

Status: Patch Available  (was: Open)

 When new instance of Hive (class) is created, the current database is reset 
 to default (current database shouldn't be changed).
 ---

 Key: HIVE-2501
 URL: https://issues.apache.org/jira/browse/HIVE-2501
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Minor
 Attachments: HIVE-2501.1.patch, HIVE-2501.2.patch


 This bug manifested to me, when first thing I did after starting Hive, was to 
 call use db_name; but then calling show tables; was still showing tables 
 from the default database (and I had to call use again to actually change 
 database). This bug might have manifested only due to a specific Hive 
 deployment I am using (I didn't investigate this issue that deeply). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive runtime error while run TestCliDriver auto_join21.q

2011-10-25 Thread Bing Li
Hi, Guys
I met an error which is similar like described in HIVE-1478, but not exactly 
the same when run auto_join21.q in TestCliDriver.
Do you have some ideas on this?

== Re-Produce ==
Hive: 0.7.1
ANT: 1.8.2
Hadoop: 0.20.2
command: ant test -Dtestcase=TestCliDriver -Dqfile=auto_join21.q

    [junit] 11/10/25 10:03:57 INFO persistence.HashMapWrapper: maximum memory: 
1048576000
    [junit] 11/10/25 10:03:57 INFO persistence.HashMapWrapper: maximum memory: 
1048576000
    [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: Initialization Done 1 
MAPJOIN
    [junit] 11/10/25 10:03:57 INFO exec.HashTableDummyOperator: Initialization 
Done 5 HASHTABLEDUMMY
    [junit] 11/10/25 10:03:57 INFO exec.MapOperator: Processing path 
pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src/kv1.txt
    [junit] 11/10/25 10:03:57 INFO exec.MapOperator: Processing alias src1 for 
file 
pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src
    [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: *** Load from 
HashTable File: input : 
pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src/kv1.txt
    [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator:    Load back 1 
hashtable file from tmp file 
uri:file:/tmp/root/hive_2011-10-25_10-03-53_617_6340495404173422880/-local-10003/HashTable-Stage-5/MapJoin-1--.hashtable
    [junit] 11/10/25 10:03:57 WARN lazybinary.LazyBinaryStruct: Extra bytes 
detected at the end of the row! Ignoring similar problems.
    [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator:    Load back 1 
hashtable file from tmp file 
uri:file:/tmp/root/hive_2011-10-25_10-03-53_617_6340495404173422880/-local-10003/HashTable-Stage-5/MapJoin-2--.hashtable
    [junit] 11/10/25 10:03:57 WARN lazybinary.LazyBinaryStruct: Extra bytes 
detected at the end of the row! Ignoring similar problems.
    [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 forwarding 1 rows
    [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 forwarding 1 rows
    [junit] 11/10/25 10:03:57 FATAL ExecMapper: 
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
processing row {key:238,value:val_238}
    [junit] at 
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:546)
    [junit] at 
org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
    [junit] at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    [junit] at 
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
    [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    [junit] at 
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
    [junit] Caused by: java.lang.ClassCastException: org.apache.hadoop.io.Text 
incompatible with org.apache.hadoop.io.BooleanWritable
    [junit] at 
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.joinObjectsRightOuterJoin(CommonJoinOperator.java:489)
    [junit] at 
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.joinObjects(CommonJoinOperator.java:639)
    [junit] at 
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:681)
    [junit] at 
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:685)
    [junit] at 
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:685)
    [junit] at 
org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:845)
    [junit] at 
org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:264)
    [junit] at 
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
    [junit] at 
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
    [junit] at 
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:78)
    [junit] at 
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
    [junit] at 
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
    [junit] at 
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:527)
    [junit] ... 5 more
    [junit]
    [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 finished. closing...
    [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 forwarded 1 rows
    [junit] 11/10/25 10:03:57 INFO exec.MapOperator: DESERIALIZE_ERRORS:0
    [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 finished. 
closing...
    [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 forwarded 1 rows
    [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 Close done
    [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 Close done
    [junit] 11/10/25 10:03:57 INFO exec.HashTableDummyOperator: 4 finished. 
closing...
    [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 Close done
    [junit] 11/10/25 10:03:57 INFO 

[jira] [Commented] (HIVE-2512) After HIVE-2145, Hive disallow any use of function in cluster-by clause

2011-10-25 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135312#comment-13135312
 ] 

Hudson commented on HIVE-2512:
--

Integrated in Hive-trunk-h0.21 #1032 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1032/])
HIVE-2512. After HIVE-2145, Hive disallow any use of function in cluster-by 
clause (Chinna Rao Lalam via Ning Zhang)

nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188473
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
* /hive/trunk/ql/src/test/queries/clientnegative/orderby_function.q
* /hive/trunk/ql/src/test/results/clientnegative/orderby_function.q.out


 After HIVE-2145, Hive disallow any use of function in cluster-by clause
 ---

 Key: HIVE-2512
 URL: https://issues.apache.org/jira/browse/HIVE-2512
 Project: Hive
  Issue Type: Bug
Reporter: Ning Zhang
Assignee: Chinna Rao Lalam
 Attachments: HIVE-2512.1.patch, HIVE-2512.patch


 After HIVE-2145, the following query returns a semantic analysis error: 
 FROM src SELECT * cluster by rand();
 FAILED: Error in semantic analysis: functions are not supported in order by
 Looking back at HIVE-2145, it's clear that the patch is more restrictive than 
 necessary. 
 Chinna, are you able to work on it? Please let me know if you don't have 
 cycles to do it now. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2519) Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema

2011-10-25 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135314#comment-13135314
 ] 

Hudson commented on HIVE-2519:
--

Integrated in Hive-trunk-h0.21 #1032 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1032/])
HIVE-2519 Dynamic partition insert should enforce the order of the
partition spec is the same as the one in schema (Ning Zhang via namit)

namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188450
Files : 
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
* /hive/trunk/ql/src/test/queries/clientnegative/dyn_part4.q
* /hive/trunk/ql/src/test/results/clientnegative/dyn_part4.q.out


 Dynamic partition insert should enforce the order of the partition spec is 
 the same as the one in schema
 

 Key: HIVE-2519
 URL: https://issues.apache.org/jira/browse/HIVE-2519
 Project: Hive
  Issue Type: Bug
Reporter: Ning Zhang
Assignee: Ning Zhang
 Fix For: 0.9.0

 Attachments: HIVE-2519.3.patch, HIVE-2519.patch


 Suppose the table schema is (a string, b string) partitioned by (p1 string, 
 p2 string), a dynamic partition insert is allowed to:
 insert overwrite ... partition (p2=..., p1);
 which will create the wrong HDFS directory structure such as 
 /.../p2=.../p1= This is contradictory to the metastore's assumption of 
 the HDFS directory structure. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-1567) increase hive.mapjoin.maxsize to 10 million

2011-10-25 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135315#comment-13135315
 ] 

Hudson commented on HIVE-1567:
--

Integrated in Hive-trunk-h0.21 #1032 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1032/])
HIVE-1567. Remove hive.mapjoin.maxsize - it was not being used
(Asuhtosh Chauhan via namit)

namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188449
Files : 
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/conf/hive-default.xml
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractMapJoinOperator.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java


 increase hive.mapjoin.maxsize to 10 million
 ---

 Key: HIVE-1567
 URL: https://issues.apache.org/jira/browse/HIVE-1567
 Project: Hive
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: Ashutosh Chauhan
 Attachments: hive-1567.patch


 i saw in a very wide table, hive can process 1million rows in less than one 
 minute (select all columns).
 setting the hive.mapjoin.maxsize to 100k is kind of too restrictive. Let's 
 increase this to 10 million.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2497) partition pruning prune some right partition under specific conditions

2011-10-25 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135313#comment-13135313
 ] 

Hudson commented on HIVE-2497:
--

Integrated in Hive-trunk-h0.21 #1032 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1032/])
HIVE-2497 partition pruning prune some right partition under specific 
conditions
(binlijin via namit)

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188507
Files : 
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/ExprProcFactory.java
* /hive/trunk/ql/src/test/queries/clientpositive/ppr_allchildsarenull.q
* /hive/trunk/ql/src/test/results/clientpositive/ppr_allchildsarenull.q.out


 partition pruning  prune some right partition under specific conditions
 ---

 Key: HIVE-2497
 URL: https://issues.apache.org/jira/browse/HIVE-2497
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.7.1
Reporter: binlijin
Assignee: binlijin
  Labels: patch
 Fix For: 0.9.0

 Attachments: hive-2497.2.patch, hive-2497.patch


 create table src3(key string, value string) partitioned by (pt string)
 row format delimited fields terminated by ',';
 ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='2011091100') ;
 ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='2011091200') ;
 ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='2011091300') ;
 explain extended
 select user_id 
 from
  (
select 
 cast(key as int) as user_id
 ,case when (value like 'aaa%' or value like 'vvv%')
 then 1
 else 0  end as tag_student
from src3
  ) sub
 where sub.tag_student  0;
 STAGE DEPENDENCIES:
   Stage-1 is a root stage
   Stage-0 is a root stage
 STAGE PLANS:
   Stage: Stage-1
 Map Reduce
   Alias - Map Operator Tree:
 sub:src3 
   TableScan
 alias: src3
 Filter Operator
   isSamplingPred: false
   predicate:
   expr: (CASE WHEN (((value like 'aaa%') or (value like 
 'vvv%'))) THEN (1) ELSE (0) END  0)
   type: boolean
   Select Operator
 expressions:
   expr: UDFToInteger(key)
   type: int
   expr: CASE WHEN (((value like 'aaa%') or (value like 
 'vvv%'))) THEN (1) ELSE (0) END
   type: int
 outputColumnNames: _col0, _col1
 Filter Operator
   isSamplingPred: false
   predicate:
   expr: (_col1  0)
   type: boolean
   Select Operator
 expressions:
   expr: _col0
   type: int
 outputColumnNames: _col0
 File Output Operator
   compressed: false
   GlobalTableId: 0
   directory: 
 hdfs://localhost:54310/tmp/hive-tianzhao/hive_2011-10-11_19-26-12_894_9085644225727185586/-ext-10001
   NumFilesPerFileSink: 1
   table:
   input format: 
 org.apache.hadoop.mapred.TextInputFormat
   output format: 
 org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
   properties:
 columns _col0
 columns.types int
 serialization.format 1
   TotalFiles: 1
   MultiFileSpray: false
   Needs Tagging: false
   Stage: Stage-0
 Fetch Operator
   limit: -1
 if we set hive.optimize.ppd=false;
 STAGE DEPENDENCIES:
   Stage-1 is a root stage
   Stage-0 is a root stage
 STAGE PLANS:
   Stage: Stage-1
 Map Reduce
   Alias - Map Operator Tree:
 sub:src3 
   TableScan
 alias: src3
 Select Operator
   expressions:
 expr: UDFToInteger(key)
 type: int
 expr: CASE WHEN (((value like 'aaa%') or (value like 
 'vvv%'))) THEN (1) ELSE (0) END
 type: int
   outputColumnNames: _col0, _col1
   Filter Operator
 isSamplingPred: false
 predicate:
 expr: (_col1  0)
 type: boolean
 Select Operator
   expressions:
 expr: _col0
 type: int
   outputColumnNames: _col0
   File Output Operator
 compressed: false
 

[jira] [Commented] (HIVE-2145) NPE during parsing order-by expression

2011-10-25 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135316#comment-13135316
 ] 

Hudson commented on HIVE-2145:
--

Integrated in Hive-trunk-h0.21 #1032 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1032/])
HIVE-2512. After HIVE-2145, Hive disallow any use of function in cluster-by 
clause (Chinna Rao Lalam via Ning Zhang)

nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188473
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
* /hive/trunk/ql/src/test/queries/clientnegative/orderby_function.q
* /hive/trunk/ql/src/test/results/clientnegative/orderby_function.q.out


 NPE during parsing order-by expression
 --

 Key: HIVE-2145
 URL: https://issues.apache.org/jira/browse/HIVE-2145
 Project: Hive
  Issue Type: Bug
Reporter: Ning Zhang
Assignee: Chinna Rao Lalam
 Fix For: 0.9.0

 Attachments: HIVE-2145.patch


 The following query throws NPE, where it should have throw parsing exception. 
 hive select key, count(1) cnt from src group by key order by count(1) limit 
 10;
 select key, count(1) cnt from src group by key order by count(1) limit 10;
 FAILED: Hive Internal Error: java.lang.NullPointerException(null)
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:153)
   at 
 org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:640)
   at 
 org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:761)
   at 
 org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
   at 
 org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:88)
   at 
 org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:125)
   at 
 org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:102)
   at 
 org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:156)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:6830)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:6788)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genReduceSinkPlan(SemanticAnalyzer.java:4303)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:5461)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6022)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6607)
   at 
 org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:790)
   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:209)
   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:286)
   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:514)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
 We should still allow functions in the order by clause (as in MySQL) but fix 
 the NPE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HIVE-2509) Literal bigint

2011-10-25 Thread John Sichi (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi resolved HIVE-2509.
--

   Resolution: Fixed
Fix Version/s: 0.9.0
 Release Note: This change adds parser support for typed integer literal 
variants (tinyint, smallint, and bigint).
 Hadoop Flags: Reviewed

Committed to trunk.  Thanks Jonathan!

Can you update the wiki with the new syntax?


 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Fix For: 0.9.0

 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2509) Literal bigint

2011-10-25 Thread Phabricator (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135338#comment-13135338
 ] 

Phabricator commented on HIVE-2509:
---

jsichi has accepted the revision [jira] [HIVE-2509] Literal bigint.

REVISION DETAIL
  https://reviews.facebook.net/D15


 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Fix For: 0.9.0

 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2509) Literal bigint

2011-10-25 Thread Phabricator (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135339#comment-13135339
 ] 

Phabricator commented on HIVE-2509:
---

jsichi has accepted the revision [jira] [HIVE-2509] Literal bigint.

REVISION DETAIL
  https://reviews.facebook.net/D15


 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Fix For: 0.9.0

 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2525) Introduce proper error messages, when explain fails on commands that otherwise could be succesful (commands that are not analyzed by Semantic Analyzer)

2011-10-25 Thread Commented

[ 
https://issues.apache.org/jira/browse/HIVE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135341#comment-13135341
 ] 

Robert Surówka commented on HIVE-2525:
--

I think such an error is incorrect since dfs -ls PATH (without explain) 
would never throw such an error on it's own (and execution of dfs... doesn't 
go to semantic analysis), so I see no reason for explain of dfs to go to 
semantic analysis and throw semantic analysis type error. 

Furthermore having check for incorrect explains at the beginning saves a lot of 
time, otherwise all the analysis and various class instances would be need to 
be created only to find error that we can easily see from the very beginning. 
And if someone would like to add or remove a command that doesn't go to Driver 
to be compiled, he will need to add or remove an if in CliDriver anyway so it 
shouldn't be hard for him to notice that he also needs to update that list. 
Finally if that suggestion of fix will pass I can add appropriate tests, 
checking if explains errors of non-sematic-analisys commands are informative, 
so it will be easier to see if something is wrong after subsequent changes. 

The bad thing is that after this change, even for valid explain query we will 
check contains on that list, but fortunately since it is short it doesn't 
cost too much time. 

 Introduce proper error messages, when explain fails on commands that 
 otherwise could be succesful (commands that are not analyzed by Semantic 
 Analyzer)
 ---

 Key: HIVE-2525
 URL: https://issues.apache.org/jira/browse/HIVE-2525
 Project: Hive
  Issue Type: Improvement
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Trivial
 Attachments: HIVE-2525.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist

2011-10-25 Thread Commented

[ 
https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135347#comment-13135347
 ] 

Robert Surówka commented on HIVE-2510:
--

Ok, thank you for this advice, I will try to make sure to do that in the 
future. 

 Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name 
  if the given db_name doesn't exist
 -

 Key: HIVE-2510
 URL: https://issues.apache.org/jira/browse/HIVE-2510
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Trivial
 Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, 
 HIVE-2510.4.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist

2011-10-25 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135349#comment-13135349
 ] 

jirapos...@reviews.apache.org commented on HIVE-2510:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2426/
---

(Updated 2011-10-25 19:25:02.433521)


Review request for Yongqiang He, Ning Zhang and Kevin Wilfong.


Summary
---

Now when creating table in specific database, if that database doesn't exist 
Hive will return an error informing about that, instead of throwing Null 
Pointer Exception. Null Pointer happens during semantic analysis, so it seems 
reasonable to return an error before the code that would throw null pointer 
(Warehouse.getDatabasePath()) executes.


This addresses bug HIVE-2510.
https://issues.apache.org/jira/browse/HIVE-2510


Diffs
-

  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
1187647 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
1187647 
  trunk/ql/src/test/queries/clientnegative/create_table_failure1.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/create_table_failure2.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/create_table_failure3.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/create_table_failure4.q PRE-CREATION 
  trunk/ql/src/test/queries/clientpositive/input45.q PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/create_table_failure1.q.out 
PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/create_table_failure2.q.out 
PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/create_table_failure3.q.out 
PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/create_table_failure4.q.out 
PRE-CREATION 
  trunk/ql/src/test/results/clientpositive/input45.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/2426/diff


Testing
---

Run some sample queries and it seemed to work. 


Thanks,

Robert



 Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name 
  if the given db_name doesn't exist
 -

 Key: HIVE-2510
 URL: https://issues.apache.org/jira/browse/HIVE-2510
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Trivial
 Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, 
 HIVE-2510.4.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2501) When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed).

2011-10-25 Thread John Sichi (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135352#comment-13135352
 ] 

John Sichi commented on HIVE-2501:
--

+1.  Will commit when tests pass.

 When new instance of Hive (class) is created, the current database is reset 
 to default (current database shouldn't be changed).
 ---

 Key: HIVE-2501
 URL: https://issues.apache.org/jira/browse/HIVE-2501
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Minor
 Attachments: HIVE-2501.1.patch, HIVE-2501.2.patch


 This bug manifested to me, when first thing I did after starting Hive, was to 
 call use db_name; but then calling show tables; was still showing tables 
 from the default database (and I had to call use again to actually change 
 database). This bug might have manifested only due to a specific Hive 
 deployment I am using (I didn't investigate this issue that deeply). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2368) Slow dropping of partitions caused by full listing of storage descriptors

2011-10-25 Thread Paul Yang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-2368:


Summary: Slow dropping of partitions caused by full listing of storage 
descriptors  (was: Determining whether a Column Descriptor is unused may take 
too long)

 Slow dropping of partitions caused by full listing of storage descriptors
 -

 Key: HIVE-2368
 URL: https://issues.apache.org/jira/browse/HIVE-2368
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Sohan Jain
 Attachments: HIVE-2368.1.patch


 To determine if a column descriptor is unused, we call 
 listStorageDescriptorsWithCD(), which may return a big list of SDs.  This can 
 severely slow down dropping partitions.
 We can add a maximum number of SDs to return, and just ask for 1 SD, since we 
 are just doing an existential check.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2509) Literal bigint

2011-10-25 Thread Phabricator (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135382#comment-13135382
 ] 

Phabricator commented on HIVE-2509:
---

jonchang has committed the revision [jira] [HIVE-2509] Literal bigint.

REVISION DETAIL
  https://reviews.facebook.net/D15


 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Fix For: 0.9.0

 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2515) Make Constant OIs work with UDTFs.

2011-10-25 Thread Phabricator (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135381#comment-13135381
 ] 

Phabricator commented on HIVE-2515:
---

jonchang has committed the revision [jira] [HIVE-2515] Make Constant OIs work 
with UDTFs..

REVISION DETAIL
  https://reviews.facebook.net/D33


 Make Constant OIs work with UDTFs.
 --

 Key: HIVE-2515
 URL: https://issues.apache.org/jira/browse/HIVE-2515
 Project: Hive
  Issue Type: Improvement
Reporter: Jonathan Chang
Assignee: Jonathan Chang
Priority: Minor
 Fix For: 0.9.0

 Attachments: D33.1.patch, D33.1.patch, D33.2.patch, D33.2.patch, 
 D33.2.patch, D33.3.patch, D33.3.patch, D33.3.patch, junit-noframes.html


 UDTFs are the last shoe to drop for constant OIs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2509) Literal bigint

2011-10-25 Thread Phabricator (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135385#comment-13135385
 ] 

Phabricator commented on HIVE-2509:
---

jonchang has committed the revision [jira] [HIVE-2509] Literal bigint.

REVISION DETAIL
  https://reviews.facebook.net/D15


 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Fix For: 0.9.0

 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2368) Slow dropping of partitions caused by full listing of storage descriptors

2011-10-25 Thread Paul Yang (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135387#comment-13135387
 ] 

Paul Yang commented on HIVE-2368:
-

Committed. Thanks Sohan!

 Slow dropping of partitions caused by full listing of storage descriptors
 -

 Key: HIVE-2368
 URL: https://issues.apache.org/jira/browse/HIVE-2368
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Sohan Jain
Assignee: Sohan Jain
 Fix For: 0.9.0

 Attachments: HIVE-2368.1.patch


 To determine if a column descriptor is unused, we call 
 listStorageDescriptorsWithCD(), which may return a big list of SDs.  This can 
 severely slow down dropping partitions.
 We can add a maximum number of SDs to return, and just ask for 1 SD, since we 
 are just doing an existential check.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HIVE-2368) Slow dropping of partitions caused by full listing of storage descriptors

2011-10-25 Thread Paul Yang (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang resolved HIVE-2368.
-

   Resolution: Fixed
Fix Version/s: 0.9.0

 Slow dropping of partitions caused by full listing of storage descriptors
 -

 Key: HIVE-2368
 URL: https://issues.apache.org/jira/browse/HIVE-2368
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Sohan Jain
Assignee: Sohan Jain
 Fix For: 0.9.0

 Attachments: HIVE-2368.1.patch


 To determine if a column descriptor is unused, we call 
 listStorageDescriptorsWithCD(), which may return a big list of SDs.  This can 
 severely slow down dropping partitions.
 We can add a maximum number of SDs to return, and just ask for 1 SD, since we 
 are just doing an existential check.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2509) Literal bigint

2011-10-25 Thread Jonathan Chang (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135398#comment-13135398
 ] 

Jonathan Chang commented on HIVE-2509:
--

How do I get edit rights to the wiki?

 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Fix For: 0.9.0

 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: Hive runtime error while run TestCliDriver auto_join21.q

2011-10-25 Thread Ashutosh Chauhan
Bing,

I tried ant test -Dtestcase=TestCliDriver -Dqfile=auto_join21.q on trunk and
it succeeded. Is it feasible for you to upgrade?

Thanks,
Ashutosh

On Tue, Oct 25, 2011 at 11:27, Bing Li lib...@yahoo.com.cn wrote:

 Hi, Guys
 I met an error which is similar like described in HIVE-1478, but not
 exactly the same when run auto_join21.q in TestCliDriver.
 Do you have some ideas on this?

 == Re-Produce ==
 Hive: 0.7.1
 ANT: 1.8.2
 Hadoop: 0.20.2
 command: ant test -Dtestcase=TestCliDriver -Dqfile=auto_join21.q

 [junit] 11/10/25 10:03:57 INFO persistence.HashMapWrapper: maximum
 memory: 1048576000
 [junit] 11/10/25 10:03:57 INFO persistence.HashMapWrapper: maximum
 memory: 1048576000
 [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: Initialization
 Done 1 MAPJOIN
 [junit] 11/10/25 10:03:57 INFO exec.HashTableDummyOperator:
 Initialization Done 5 HASHTABLEDUMMY
 [junit] 11/10/25 10:03:57 INFO exec.MapOperator: Processing path
 pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src/kv1.txt
 [junit] 11/10/25 10:03:57 INFO exec.MapOperator: Processing alias src1
 for file
 pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src
 [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator: *** Load from
 HashTable File: input :
 pfile:/root/libing/N20111024_1337/hive-0.7.1-ibm/src/build/ql/test/data/warehouse/src/kv1.txt
 [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator:Load back 1
 hashtable file from tmp file
 uri:file:/tmp/root/hive_2011-10-25_10-03-53_617_6340495404173422880/-local-10003/HashTable-Stage-5/MapJoin-1--.hashtable
 [junit] 11/10/25 10:03:57 WARN lazybinary.LazyBinaryStruct: Extra bytes
 detected at the end of the row! Ignoring similar problems.
 [junit] 11/10/25 10:03:57 INFO exec.MapJoinOperator:Load back 1
 hashtable file from tmp file
 uri:file:/tmp/root/hive_2011-10-25_10-03-53_617_6340495404173422880/-local-10003/HashTable-Stage-5/MapJoin-2--.hashtable
 [junit] 11/10/25 10:03:57 WARN lazybinary.LazyBinaryStruct: Extra bytes
 detected at the end of the row! Ignoring similar problems.
 [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 forwarding 1 rows
 [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 forwarding 1
 rows
 [junit] 11/10/25 10:03:57 FATAL ExecMapper:
 org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
 processing row {key:238,value:val_238}
 [junit] at
 org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:546)
 [junit] at
 org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
 [junit] at
 org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
 [junit] at
 org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
 [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
 [junit] at
 org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
 [junit] Caused by: java.lang.ClassCastException:
 org.apache.hadoop.io.Text incompatible with
 org.apache.hadoop.io.BooleanWritable
 [junit] at
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.joinObjectsRightOuterJoin(CommonJoinOperator.java:489)
 [junit] at
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.joinObjects(CommonJoinOperator.java:639)
 [junit] at
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:681)
 [junit] at
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:685)
 [junit] at
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:685)
 [junit] at
 org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:845)
 [junit] at
 org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:264)
 [junit] at
 org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
 [junit] at
 org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
 [junit] at
 org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:78)
 [junit] at
 org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
 [junit] at
 org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
 [junit] at
 org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:527)
 [junit] ... 5 more
 [junit]
 [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 finished.
 closing...
 [junit] 11/10/25 10:03:57 INFO exec.MapOperator: 12 forwarded 1 rows
 [junit] 11/10/25 10:03:57 INFO exec.MapOperator: DESERIALIZE_ERRORS:0
 [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 finished.
 closing...
 [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 forwarded 1
 rows
 [junit] 11/10/25 10:03:57 INFO exec.TableScanOperator: 0 

[jira] [Commented] (HIVE-2509) Literal bigint

2011-10-25 Thread John Sichi (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135411#comment-13135411
 ] 

John Sichi commented on HIVE-2509:
--

Try now...I just granted you write access.


 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Fix For: 0.9.0

 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist

2011-10-25 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135455#comment-13135455
 ] 

jirapos...@reviews.apache.org commented on HIVE-2510:
-



bq.  On 2011-10-25 17:24:41, Ashutosh Chauhan wrote:
bq.   trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java, 
lines 6923-6926
bq.   https://reviews.apache.org/r/2426/diff/4/?file=52450#file52450line6923
bq.  
bq.   Not sure if we need this. Checking of existing table already happens 
in analyzeCreateTable() around line 7847. So, this check at this point is 
redundant.
bq.  
bq.  Robert Surówka wrote:
bq.  Without this check a null pointer exception was being thrown (for 
table created as select).

(Here I check for existing database, not table)


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2426/#review2823
---


On 2011-10-25 19:25:02, Robert Surówka wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2426/
bq.  ---
bq.  
bq.  (Updated 2011-10-25 19:25:02)
bq.  
bq.  
bq.  Review request for Yongqiang He, Ning Zhang and Kevin Wilfong.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Now when creating table in specific database, if that database doesn't 
exist Hive will return an error informing about that, instead of throwing Null 
Pointer Exception. Null Pointer happens during semantic analysis, so it seems 
reasonable to return an error before the code that would throw null pointer 
(Warehouse.getDatabasePath()) executes.
bq.  
bq.  
bq.  This addresses bug HIVE-2510.
bq.  https://issues.apache.org/jira/browse/HIVE-2510
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
1187647 
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
1187647 
bq.trunk/ql/src/test/queries/clientnegative/create_table_failure1.q 
PRE-CREATION 
bq.trunk/ql/src/test/queries/clientnegative/create_table_failure2.q 
PRE-CREATION 
bq.trunk/ql/src/test/queries/clientnegative/create_table_failure3.q 
PRE-CREATION 
bq.trunk/ql/src/test/queries/clientnegative/create_table_failure4.q 
PRE-CREATION 
bq.trunk/ql/src/test/queries/clientpositive/input45.q PRE-CREATION 
bq.trunk/ql/src/test/results/clientnegative/create_table_failure1.q.out 
PRE-CREATION 
bq.trunk/ql/src/test/results/clientnegative/create_table_failure2.q.out 
PRE-CREATION 
bq.trunk/ql/src/test/results/clientnegative/create_table_failure3.q.out 
PRE-CREATION 
bq.trunk/ql/src/test/results/clientnegative/create_table_failure4.q.out 
PRE-CREATION 
bq.trunk/ql/src/test/results/clientpositive/input45.q.out PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/2426/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Run some sample queries and it seemed to work. 
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Robert
bq.  
bq.



 Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name 
  if the given db_name doesn't exist
 -

 Key: HIVE-2510
 URL: https://issues.apache.org/jira/browse/HIVE-2510
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Trivial
 Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, 
 HIVE-2510.4.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist

2011-10-25 Thread Updated

 [ 
https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Surówka updated HIVE-2510:
-

Attachment: HIVE-2510.5.patch

 Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name 
  if the given db_name doesn't exist
 -

 Key: HIVE-2510
 URL: https://issues.apache.org/jira/browse/HIVE-2510
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Trivial
 Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, 
 HIVE-2510.4.patch, HIVE-2510.5.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2510) Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name .... if the given db_name doesn't exist

2011-10-25 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135495#comment-13135495
 ] 

jirapos...@reviews.apache.org commented on HIVE-2510:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2426/
---

(Updated 2011-10-25 22:24:42.563999)


Review request for Yongqiang He, Ning Zhang and Kevin Wilfong.


Changes
---

Remove is_database_exist() method from HiveMetaStore, now check whether 
database exist is done directly with use of getDatabase().


Summary
---

Now when creating table in specific database, if that database doesn't exist 
Hive will return an error informing about that, instead of throwing Null 
Pointer Exception. Null Pointer happens during semantic analysis, so it seems 
reasonable to return an error before the code that would throw null pointer 
(Warehouse.getDatabasePath()) executes.


This addresses bug HIVE-2510.
https://issues.apache.org/jira/browse/HIVE-2510


Diffs (updated)
-

  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
1188369 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
1188369 
  trunk/ql/src/test/queries/clientnegative/create_table_failure1.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/create_table_failure2.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/create_table_failure3.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/create_table_failure4.q PRE-CREATION 
  trunk/ql/src/test/queries/clientpositive/input45.q PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/create_table_failure1.q.out 
PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/create_table_failure2.q.out 
PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/create_table_failure3.q.out 
PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/create_table_failure4.q.out 
PRE-CREATION 
  trunk/ql/src/test/results/clientpositive/input45.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/2426/diff


Testing
---

Run some sample queries and it seemed to work. 


Thanks,

Robert



 Hive throws Null Pointer Exception upon CREATE TABLE db_name.table_name 
  if the given db_name doesn't exist
 -

 Key: HIVE-2510
 URL: https://issues.apache.org/jira/browse/HIVE-2510
 Project: Hive
  Issue Type: Bug
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Trivial
 Attachments: HIVE-2510.1.patch, HIVE-2510.2.patch, HIVE-2510.3.patch, 
 HIVE-2510.4.patch, HIVE-2510.5.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive-trunk-h0.21 - Build # 1033 - Failure

2011-10-25 Thread Apache Jenkins Server
Changes for Build #1032
[namit] HIVE-2497 partition pruning prune some right partition under specific 
conditions
(binlijin via namit)

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

[nzhang] HIVE-2512. After HIVE-2145, Hive disallow any use of function in 
cluster-by clause (Chinna Rao Lalam via Ning Zhang)

[namit] HIVE-2519 Dynamic partition insert should enforce the order of the
partition spec is the same as the one in schema (Ning Zhang via namit)

[namit] HIVE-1567. Remove hive.mapjoin.maxsize - it was not being used
(Asuhtosh Chauhan via namit)


Changes for Build #1033
[nzhang] HIVE-2446. bug (next one) - If constructor of ClientStatsPublisher 
throws runtime exception it will be propagated to HadoopJobExecHelper progress 
method and beyond, whereas it shouldnot




6 tests failed.
FAILED:  
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe

Error Message:
Unexpected exception See build/ql/tmp/hive.log, or try ant test ... 
-Dtest.silent=false to get more logs.

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception
See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get 
more logs.
at junit.framework.Assert.fail(Assert.java:50)
at 
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe(TestNegativeMinimrCliDriver.java:115)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)


FAILED:  
org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisherOneStat

Error Message:
null

Stack Trace:
java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.Utilities.prepareWithRetry(Utilities.java:2191)
at 
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsAggregator.cleanUp(JDBCStatsAggregator.java:215)
at 
org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.setUp(TestStatsPublisherEnhanced.java:60)
at junit.framework.TestCase.runBare(TestCase.java:132)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)


FAILED:  
org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.testStatsPublisher

Error Message:
null

Stack Trace:
java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.Utilities.prepareWithRetry(Utilities.java:2191)
at 
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsAggregator.cleanUp(JDBCStatsAggregator.java:215)
at 
org.apache.hadoop.hive.ql.exec.TestStatsPublisherEnhanced.setUp(TestStatsPublisherEnhanced.java:60)
at junit.framework.TestCase.runBare(TestCase.java:132)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at 

[jira] [Commented] (HIVE-2446) Introduction of client statistics publishers possibility

2011-10-25 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135498#comment-13135498
 ] 

Hudson commented on HIVE-2446:
--

Integrated in Hive-trunk-h0.21 #1033 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1033/])
HIVE-2446. bug (next one) - If constructor of ClientStatsPublisher throws 
runtime exception it will be propagated to HadoopJobExecHelper progress method 
and beyond, whereas it shouldnot

nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188796
Files : 
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java


 Introduction of client statistics publishers possibility
 

 Key: HIVE-2446
 URL: https://issues.apache.org/jira/browse/HIVE-2446
 Project: Hive
  Issue Type: Improvement
  Components: Clients, Statistics
Reporter: Robert Surówka
Assignee: Robert Surówka
Priority: Minor
 Fix For: 0.8.0

 Attachments: HIVE-2446.1.patch, HIVE-2446.1.patch, HIVE-2446.2.patch, 
 HIVE-2446.2.patch, HIVE-2446.3.patch, HIVE-2446.3.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

 The purpose of this change is to allow publication or storage of counters 
 while the job is running.
 Introduced two new variables to hive-default.xml and HiveConf.java: 
 hive.client.stats.publishers and hive.client.stats.counters. First one 
 specifies classes names, whose instances will be executed by 
 HadoopJobExecHelper.java (similarly as hooks are) in its method 
 progress(ExecDriverTaskHandle): MapRedStats. Second one specifies list of 
 counters that any client stat publishers should publish or stored. Details 
 regarding format of this list is up to a specific deployment (it is up to 
 client stats publishers to parse it), yet it is required to use display names 
 of counter groups and counters.
 Added interface ClientStatsPublishers in org.apache.hadoop.hive.ql.stats 
 package, that must be implemented by all stats publishers.
 Added code to progress(ExecDriverTaskHandle): MapRedStats from 
 HadoopJobExecHelper.java that puts counters' values to a Java map and then 
 executes registered client stats publishers giving them that map and running 
 job id. Added two new methods to HadoopJobExecHelper: 
 extractAllCounterValues(Counters) and getClientStatsPublishers() that are 
 used by code from previous sentence.
 Made cosmetic changes in two other classes
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive-0.8.0-SNAPSHOT-h0.21 - Build # 70 - Failure

2011-10-25 Thread Apache Jenkins Server
Changes for Build #70



1 tests failed.
REGRESSION:  org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample6

Error Message:
Timeout occurred. Please note the time in the report does not reflect the time 
until the timeout.

Stack Trace:
junit.framework.AssertionFailedError: Timeout occurred. Please note the time in 
the report does not reflect the time until the timeout.
at 
net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259)
at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268)
at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:324)
at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244)




The Apache Jenkins build system has built Hive-0.8.0-SNAPSHOT-h0.21 (build #70)

Status: Failure

Check console output at 
https://builds.apache.org/job/Hive-0.8.0-SNAPSHOT-h0.21/70/ to view the results.


[jira] [Created] (HIVE-2527) Consecutive string literals should be combined into a single string literal.

2011-10-25 Thread Jonathan Chang (Created) (JIRA)
Consecutive string literals should be combined into a single string literal.


 Key: HIVE-2527
 URL: https://issues.apache.org/jira/browse/HIVE-2527
 Project: Hive
  Issue Type: Improvement
Reporter: Jonathan Chang
Assignee: Jonathan Chang
Priority: Minor




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2527) Consecutive string literals should be combined into a single string literal.

2011-10-25 Thread Jonathan Chang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Chang updated HIVE-2527:
-

Description: C, Python, etc. all support this magical feature.

 Consecutive string literals should be combined into a single string literal.
 

 Key: HIVE-2527
 URL: https://issues.apache.org/jira/browse/HIVE-2527
 Project: Hive
  Issue Type: Improvement
Reporter: Jonathan Chang
Assignee: Jonathan Chang
Priority: Minor

 C, Python, etc. all support this magical feature.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2509) Literal bigint

2011-10-25 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135624#comment-13135624
 ] 

Hudson commented on HIVE-2509:
--

Integrated in Hive-trunk-h0.21 #1034 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1034/])
HIVE-2509. Literal bigint
(Jonathan Chang via jvs)

jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188858
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
* /hive/trunk/ql/src/test/queries/clientpositive/literal_ints.q
* /hive/trunk/ql/src/test/results/clientpositive/literal_ints.q.out


 Literal bigint
 --

 Key: HIVE-2509
 URL: https://issues.apache.org/jira/browse/HIVE-2509
 Project: Hive
  Issue Type: New Feature
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Fix For: 0.9.0

 Attachments: D15.1.patch, D15.1.patch, D15.2.patch, D15.2.patch, 
 D15.3.patch, D15.3.patch, D15.4.patch, D15.4.patch, D15.4.patch


 Typing bigints is a pain because you have to write CAST(0 AS BIGINT).  We 
 should let people type 0L instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2360) create dynamic partition if and only if intermediate source has files

2011-10-25 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135732#comment-13135732
 ] 

Hudson commented on HIVE-2360:
--

Integrated in Hive-trunk-h0.21 #1035 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1035/])
HIVE-2360 create dynamic partition if and only if intermediate source has 
files (Franklin Hu, Kaushik Lakshminarayanan via Siying Dong)

sdong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1188989
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java


 create dynamic partition if and only if intermediate source has files
 -

 Key: HIVE-2360
 URL: https://issues.apache.org/jira/browse/HIVE-2360
 Project: Hive
  Issue Type: Bug
Reporter: Franklin Hu
Priority: Minor
 Attachments: hive-2360.1.patch, hive-2360.2.patch


 There are some conditions under which a partition description is created due 
 to insert overwriting a table using dynamic partitioning for partitions that 
 that are empty (have no files).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira