date:20131012

Jitendra Nath Pandey created HIVE-5525:
--

 Summary: Vectorized query failing for partitioned tables.
 Key: HIVE-5525
 URL: https://issues.apache.org/jira/browse/HIVE-5525
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey


{code}
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getInputColumnIndex(VectorizationContext.java:154)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorBinaryComparisonFilterExpression(VectorizationContext.java:1163)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:436)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:274)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.init(VectorFilterOperator.java:50)
{code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5525) Vectorized query failing for partitioned tables.


 [ 
https://issues.apache.org/jira/browse/HIVE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jitendra Nath Pandey updated HIVE-5525:
---

Status: Patch Available  (was: Open)

 Vectorized query failing for partitioned tables.
 

 Key: HIVE-5525
 URL: https://issues.apache.org/jira/browse/HIVE-5525
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
 Attachments: HIVE-5525.1.patch


 {code}
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getInputColumnIndex(VectorizationContext.java:154)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorBinaryComparisonFilterExpression(VectorizationContext.java:1163)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:436)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:274)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.init(VectorFilterOperator.java:50)
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5525) Vectorized query failing for partitioned tables.


 [ 
https://issues.apache.org/jira/browse/HIVE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jitendra Nath Pandey updated HIVE-5525:
---

Attachment: HIVE-5525.1.patch

 Vectorized query failing for partitioned tables.
 

 Key: HIVE-5525
 URL: https://issues.apache.org/jira/browse/HIVE-5525
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
 Attachments: HIVE-5525.1.patch


 {code}
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getInputColumnIndex(VectorizationContext.java:154)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorBinaryComparisonFilterExpression(VectorizationContext.java:1163)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:436)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:274)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.init(VectorFilterOperator.java:50)
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5494) Vectorization throws exception with nested UDF.


[ 
https://issues.apache.org/jira/browse/HIVE-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793289#comment-13793289
 ] 

Hudson commented on HIVE-5494:
--

FAILURE: Integrated in Hive-trunk-h0.21 #2394 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2394/])
HIVE-5494 : Vectorization throws exception with nested UDF. (Jitendra Nath 
Pandey via Ashutosh Chauhan) (hashutosh: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531391)
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/optimizer/physical
* 
/hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/optimizer/physical/TestVectorizer.java
* /hive/trunk/ql/src/test/queries/clientpositive/vectorization_nested_udf.q
* /hive/trunk/ql/src/test/results/clientpositive/vectorization_nested_udf.q.out


 Vectorization throws exception with nested UDF.
 ---

 Key: HIVE-5494
 URL: https://issues.apache.org/jira/browse/HIVE-5494
 Project: Hive
  Issue Type: Sub-task
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
 Fix For: 0.13.0

 Attachments: HIVE-5494.1.patch, HIVE-5494.2.patch


 {code}
 Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Udf: 
 GenericUDFAbs, is not supported
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:465)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:274)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getAggregatorExpression(VectorizationContext.java:1512)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.init(VectorGroupByOperator.java:133)
 ... 41 more
 FAILED: RuntimeException java.lang.reflect.InvocationTargetException
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5525) Vectorized query failing for partitioned tables.


[ 
https://issues.apache.org/jira/browse/HIVE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793307#comment-13793307
 ] 

Hive QA commented on HIVE-5525:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12608142/HIVE-5525.1.patch

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 4395 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_groupby2
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build//testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build//console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

 Vectorized query failing for partitioned tables.
 

 Key: HIVE-5525
 URL: https://issues.apache.org/jira/browse/HIVE-5525
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
 Attachments: HIVE-5525.1.patch


 {code}
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getInputColumnIndex(VectorizationContext.java:154)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorBinaryComparisonFilterExpression(VectorizationContext.java:1163)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:436)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:274)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.init(VectorFilterOperator.java:50)
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (HIVE-5526) NPE in ConstantVectorExpression.evaluate(vrg)

Remus Rusanu created HIVE-5526:
--

 Summary: NPE in ConstantVectorExpression.evaluate(vrg)
 Key: HIVE-5526
 URL: https://issues.apache.org/jira/browse/HIVE-5526
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Remus Rusanu
Assignee: Remus Rusanu
Priority: Minor


A query like select * from alltypesorc where 1=1 limit 10; triggers NPW at 
execution in ConstantVectorExpression.evaluate(vrg). this.type is null, is 
transient and doe snot make it across the compile/execution serialization.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5526) NPE in ConstantVectorExpression.evaluate(vrg)


 [ 
https://issues.apache.org/jira/browse/HIVE-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remus Rusanu updated HIVE-5526:
---

Status: Patch Available  (was: Open)

 NPE in ConstantVectorExpression.evaluate(vrg)
 -

 Key: HIVE-5526
 URL: https://issues.apache.org/jira/browse/HIVE-5526
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Remus Rusanu
Assignee: Remus Rusanu
Priority: Minor
 Attachments: HIVE-5526.patch


 A query like select * from alltypesorc where 1=1 limit 10; triggers NPW at 
 execution in ConstantVectorExpression.evaluate(vrg). this.type is null, is 
 transient and doe snot make it across the compile/execution serialization.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5526) NPE in ConstantVectorExpression.evaluate(vrg)


 [ 
https://issues.apache.org/jira/browse/HIVE-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remus Rusanu updated HIVE-5526:
---

Attachment: HIVE-5526.patch

Fix by removing the 'transient'

 NPE in ConstantVectorExpression.evaluate(vrg)
 -

 Key: HIVE-5526
 URL: https://issues.apache.org/jira/browse/HIVE-5526
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Remus Rusanu
Assignee: Remus Rusanu
Priority: Minor
 Attachments: HIVE-5526.patch


 A query like select * from alltypesorc where 1=1 limit 10; triggers NPW at 
 execution in ConstantVectorExpression.evaluate(vrg). this.type is null, is 
 transient and doe snot make it across the compile/execution serialization.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-4943) An explode function that includes the item's position in the array

2013-10-12 Thread Niko Stahl (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793360#comment-13793360
 ] 

Niko Stahl commented on HIVE-4943:
--

Hi Yin, 

Sorry I'm new to this process. Is any further action required on my part? 

Thanks,
Niko

 An explode function that includes the item's position in the array
 --

 Key: HIVE-4943
 URL: https://issues.apache.org/jira/browse/HIVE-4943
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor
Affects Versions: 0.11.0
Reporter: Niko Stahl
  Labels: patch
 Attachments: HIVE-4943.1.patch, HIVE-4943.2.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 A function that explodes an array and includes an output column with the 
 position of each item in the original array.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (HIVE-5527) Use of localtime Calendar in vectorized Timestamp arithmetic results in data corruption (depends on localtime)

Remus Rusanu created HIVE-5527:
--

 Summary: Use of localtime Calendar in vectorized Timestamp 
arithmetic results in data corruption (depends on localtime)
 Key: HIVE-5527
 URL: https://issues.apache.org/jira/browse/HIVE-5527
 Project: Hive
  Issue Type: Bug
Reporter: Remus Rusanu






--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5526) NPE in ConstantVectorExpression.evaluate(vrg)


[ 
https://issues.apache.org/jira/browse/HIVE-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793364#comment-13793364
 ] 

Hive QA commented on HIVE-5526:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12608149/HIVE-5526.patch

{color:green}SUCCESS:{color} +1 4394 tests passed

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1112/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1112/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

 NPE in ConstantVectorExpression.evaluate(vrg)
 -

 Key: HIVE-5526
 URL: https://issues.apache.org/jira/browse/HIVE-5526
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Remus Rusanu
Assignee: Remus Rusanu
Priority: Minor
 Attachments: HIVE-5526.patch


 A query like select * from alltypesorc where 1=1 limit 10; triggers NPW at 
 execution in ConstantVectorExpression.evaluate(vrg). this.type is null, is 
 transient and doe snot make it across the compile/execution serialization.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: [VOTE] Apache Hive 0.12.0 Release Candidate 1

2013-10-12 Thread Edward Capriolo

Historically we have never put a 3 day limit on voting a release. Normally
people tend to try it out and kick the tires. I am ok with this, in some
apache projects releases go +1 from multiple parties very fast and they
have bugs. A 7 day vote seems slow but it might be more realistic.


On Fri, Oct 11, 2013 at 11:32 PM, Thejas Nair the...@hortonworks.comwrote:

 Hoping to see more activity on this thread!

 I have run the unit tests and they passed with the exception of one
 test in TestThriftHttpCLIService, which fails only when run in
 sequence with TestThriftBinaryCLIService. It passes when run in
 parallel/standalone. That is not a regression, see  HIVE-5474 for
 details. None of the runs on builds.apache.org have completed because
 of some environment issues.


 I would like to clarify that you don't have to be a committer or PMC
 member to give feedback on the release candidate. So please try it out
 and give feedback on the mailing list.

 Somethings you can try out and use to give feedback -
 - run some queries against the release candidate (local mode or
 against a cluster)
 - run some or all unit tests
 - verify the md5 checksums

 % md5sum -c hive-X.Y.Z.tar.gz.md5
 hive-X.Y.Z.tar.gz: OK

 % md5sum -c hive-X.Y.Z-bin.tar.gz.md5
 hive-X.Y.Z-bin.tar.gz: OK


 - verify the PGP signature
 wget http://www.apache.org/dist/hive/KEYS
 gpg --import KEYS  (if necessarily)
 gpg --verify hive-X.Y.Z.tar.gz.asc hive-X.Y.Z.tar.gz
 gpg --verify hive-X.Y.Z-bin.tar.gz.asc hive-X.Y.Z-bin.tar.gz




 On Wed, Oct 9, 2013 at 7:50 PM, Thejas Nair the...@hortonworks.com
 wrote:
  (Fixing subject of the email!)
 
  On Wed, Oct 9, 2013 at 7:31 PM, Thejas Nair the...@hortonworks.com
 wrote:
  Apache Hive 0.12.0 Release Candidate 1 is available here:
 
  http://people.apache.org/~thejas/hive-0.12.0-rc1/
 
  Maven artifacts are available here:
 
  https://repository.apache.org/content/repositories/orgapachehive-157/
 
  Source tag for RC1 is at:
  http://svn.apache.org/viewvc/hive/tags/release-0.12.0-rc1/
 
  Voting will conclude in 72 hours.
 
  Hive PMC Members: Please test and vote.
 
  This RC includes the following jira fixes to address some of the
  concerns raised about RC0 -
  HIVE-5497 : Hive trunk broken against hadoop 0.20.2
  HIVE-5493 : duplicate jars with different versions for guava,
 commons-logging
  HIVE-5489 : NOTICE copyright dates are out of date, README needs update
  HIVE-5488 : some files are missing apache license headers
 
  Thanks to all everybody who made this new release possible. Thanks to
  everybody who reviewed RC0.
 
  Thanks
  Thejas

 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender immediately
 and delete it from your system. Thank You.

Re: [VOTE] Apache Hive 0.12.0 Release Candidate 1

2013-10-12 Thread Edward Capriolo

Has anyone noticed a strange Execution log location? The file is created
but the name used to be hive.log
hive create table c as select * from a limit 1;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=number
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=number
In order to set a constant number of reducers:
  set mapred.reduce.tasks=number
*Execution log at: /tmp/edward/.log*



On Fri, Oct 11, 2013 at 11:32 PM, Thejas Nair the...@hortonworks.comwrote:

 Hoping to see more activity on this thread!

 I have run the unit tests and they passed with the exception of one
 test in TestThriftHttpCLIService, which fails only when run in
 sequence with TestThriftBinaryCLIService. It passes when run in
 parallel/standalone. That is not a regression, see  HIVE-5474 for
 details. None of the runs on builds.apache.org have completed because
 of some environment issues.


 I would like to clarify that you don't have to be a committer or PMC
 member to give feedback on the release candidate. So please try it out
 and give feedback on the mailing list.

 Somethings you can try out and use to give feedback -
 - run some queries against the release candidate (local mode or
 against a cluster)
 - run some or all unit tests
 - verify the md5 checksums

 % md5sum -c hive-X.Y.Z.tar.gz.md5
 hive-X.Y.Z.tar.gz: OK

 % md5sum -c hive-X.Y.Z-bin.tar.gz.md5
 hive-X.Y.Z-bin.tar.gz: OK


 - verify the PGP signature
 wget http://www.apache.org/dist/hive/KEYS
 gpg --import KEYS  (if necessarily)
 gpg --verify hive-X.Y.Z.tar.gz.asc hive-X.Y.Z.tar.gz
 gpg --verify hive-X.Y.Z-bin.tar.gz.asc hive-X.Y.Z-bin.tar.gz




 On Wed, Oct 9, 2013 at 7:50 PM, Thejas Nair the...@hortonworks.com
 wrote:
  (Fixing subject of the email!)
 
  On Wed, Oct 9, 2013 at 7:31 PM, Thejas Nair the...@hortonworks.com
 wrote:
  Apache Hive 0.12.0 Release Candidate 1 is available here:
 
  http://people.apache.org/~thejas/hive-0.12.0-rc1/
 
  Maven artifacts are available here:
 
  https://repository.apache.org/content/repositories/orgapachehive-157/
 
  Source tag for RC1 is at:
  http://svn.apache.org/viewvc/hive/tags/release-0.12.0-rc1/
 
  Voting will conclude in 72 hours.
 
  Hive PMC Members: Please test and vote.
 
  This RC includes the following jira fixes to address some of the
  concerns raised about RC0 -
  HIVE-5497 : Hive trunk broken against hadoop 0.20.2
  HIVE-5493 : duplicate jars with different versions for guava,
 commons-logging
  HIVE-5489 : NOTICE copyright dates are out of date, README needs update
  HIVE-5488 : some files are missing apache license headers
 
  Thanks to all everybody who made this new release possible. Thanks to
  everybody who reviewed RC0.
 
  Thanks
  Thejas

 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender immediately
 and delete it from your system. Thank You.

Re: [VOTE] Apache Hive 0.12.0 Release Candidate 1

2013-10-12 Thread Edward Capriolo

I ran a list of queries in local mode, join, select, limit , group, rank,
they all worked as expected.

I am +1
It would be nice if we can fix the .log thing and one other (if we do
another rc)

There is one long standing bug that I feel I have fixed before.

configuration
property
  namehive.hwi.war.file/name
  valuelib/hive-hwi-0.12.0.war/value
  descriptionThis sets the path to the HWI war file, relative to
${HIVE_HOME}. /description
/property
/configuration


This should be

/lib/hive-hwi-0.12.0.war

Other then that hive_web interface does work out of the box.


On Fri, Oct 11, 2013 at 11:32 PM, Thejas Nair the...@hortonworks.comwrote:

 Hoping to see more activity on this thread!

 I have run the unit tests and they passed with the exception of one
 test in TestThriftHttpCLIService, which fails only when run in
 sequence with TestThriftBinaryCLIService. It passes when run in
 parallel/standalone. That is not a regression, see  HIVE-5474 for
 details. None of the runs on builds.apache.org have completed because
 of some environment issues.


 I would like to clarify that you don't have to be a committer or PMC
 member to give feedback on the release candidate. So please try it out
 and give feedback on the mailing list.

 Somethings you can try out and use to give feedback -
 - run some queries against the release candidate (local mode or
 against a cluster)
 - run some or all unit tests
 - verify the md5 checksums

 % md5sum -c hive-X.Y.Z.tar.gz.md5
 hive-X.Y.Z.tar.gz: OK

 % md5sum -c hive-X.Y.Z-bin.tar.gz.md5
 hive-X.Y.Z-bin.tar.gz: OK


 - verify the PGP signature
 wget http://www.apache.org/dist/hive/KEYS
 gpg --import KEYS  (if necessarily)
 gpg --verify hive-X.Y.Z.tar.gz.asc hive-X.Y.Z.tar.gz
 gpg --verify hive-X.Y.Z-bin.tar.gz.asc hive-X.Y.Z-bin.tar.gz




 On Wed, Oct 9, 2013 at 7:50 PM, Thejas Nair the...@hortonworks.com
 wrote:
  (Fixing subject of the email!)
 
  On Wed, Oct 9, 2013 at 7:31 PM, Thejas Nair the...@hortonworks.com
 wrote:
  Apache Hive 0.12.0 Release Candidate 1 is available here:
 
  http://people.apache.org/~thejas/hive-0.12.0-rc1/
 
  Maven artifacts are available here:
 
  https://repository.apache.org/content/repositories/orgapachehive-157/
 
  Source tag for RC1 is at:
  http://svn.apache.org/viewvc/hive/tags/release-0.12.0-rc1/
 
  Voting will conclude in 72 hours.
 
  Hive PMC Members: Please test and vote.
 
  This RC includes the following jira fixes to address some of the
  concerns raised about RC0 -
  HIVE-5497 : Hive trunk broken against hadoop 0.20.2
  HIVE-5493 : duplicate jars with different versions for guava,
 commons-logging
  HIVE-5489 : NOTICE copyright dates are out of date, README needs update
  HIVE-5488 : some files are missing apache license headers
 
  Thanks to all everybody who made this new release possible. Thanks to
  everybody who reviewed RC0.
 
  Thanks
  Thejas

 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender immediately
 and delete it from your system. Thank You.

[jira] [Commented] (HIVE-4943) An explode function that includes the item's position in the array

2013-10-12 Thread Edward Capriolo (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793375#comment-13793375
 ] 

Edward Capriolo commented on HIVE-4943:
---

+1. Let me re-upload the patch after it retests I will commit.

 An explode function that includes the item's position in the array
 --

 Key: HIVE-4943
 URL: https://issues.apache.org/jira/browse/HIVE-4943
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor
Affects Versions: 0.11.0
Reporter: Niko Stahl
  Labels: patch
 Attachments: HIVE-4943.1.patch, HIVE-4943.2.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 A function that explodes an array and includes an output column with the 
 position of each item in the original array.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-4943) An explode function that includes the item's position in the array

2013-10-12 Thread Edward Capriolo (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo updated HIVE-4943:
--

Attachment: HIVE-4943.3.patch

 An explode function that includes the item's position in the array
 --

 Key: HIVE-4943
 URL: https://issues.apache.org/jira/browse/HIVE-4943
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor
Affects Versions: 0.11.0
Reporter: Niko Stahl
  Labels: patch
 Attachments: HIVE-4943.1.patch, HIVE-4943.2.patch, HIVE-4943.3.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 A function that explodes an array and includes an output column with the 
 position of each item in the original array.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-4943) An explode function that includes the item's position in the array

2013-10-12 Thread Niko Stahl (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793378#comment-13793378
 ] 

Niko Stahl commented on HIVE-4943:
--

Sounds good, thanks!

 An explode function that includes the item's position in the array
 --

 Key: HIVE-4943
 URL: https://issues.apache.org/jira/browse/HIVE-4943
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor
Affects Versions: 0.11.0
Reporter: Niko Stahl
  Labels: patch
 Attachments: HIVE-4943.1.patch, HIVE-4943.2.patch, HIVE-4943.3.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 A function that explodes an array and includes an output column with the 
 position of each item in the original array.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5252) Add ql syntax for inline java code creation


 [ 
https://issues.apache.org/jira/browse/HIVE-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-5252:
---

   Resolution: Fixed
Fix Version/s: 0.13.0
   Status: Resolved  (was: Patch Available)

Thank you very much for your contribution Edward! I have committed this to 
trunk.

 Add ql syntax for inline java code creation
 ---

 Key: HIVE-5252
 URL: https://issues.apache.org/jira/browse/HIVE-5252
 Project: Hive
  Issue Type: Sub-task
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Fix For: 0.13.0

 Attachments: HIVE-5252.1.patch.txt, HIVE-5252.2.patch.txt


 Something to the effect of compile 'my code here' using 'groovycompiler'.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5513) Set the short version directly via build script


 [ 
https://issues.apache.org/jira/browse/HIVE-5513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-5513:
---

   Resolution: Fixed
Fix Version/s: 0.13.0
   Status: Resolved  (was: Patch Available)

Thank you very much for your contribution Prasad! I have committed this to 
trunk.

 Set the short version directly via build script
 ---

 Key: HIVE-5513
 URL: https://issues.apache.org/jira/browse/HIVE-5513
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure, Diagnosability
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Priority: Minor
 Fix For: 0.13.0

 Attachments: HIVE-5513.1.patch


 This is a followup to HIVE-5484. The short version should be configurable 
 directly from build script.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5526) NPE in ConstantVectorExpression.evaluate(vrg)


[ 
https://issues.apache.org/jira/browse/HIVE-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793390#comment-13793390
 ] 

Brock Noland commented on HIVE-5526:


+1

 NPE in ConstantVectorExpression.evaluate(vrg)
 -

 Key: HIVE-5526
 URL: https://issues.apache.org/jira/browse/HIVE-5526
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Remus Rusanu
Assignee: Remus Rusanu
Priority: Minor
 Attachments: HIVE-5526.patch


 A query like select * from alltypesorc where 1=1 limit 10; triggers NPW at 
 execution in ConstantVectorExpression.evaluate(vrg). this.type is null, is 
 transient and doe snot make it across the compile/execution serialization.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5252) Add ql syntax for inline java code creation


[ 
https://issues.apache.org/jira/browse/HIVE-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793394#comment-13793394
 ] 

Hudson commented on HIVE-5252:
--

FAILURE: Integrated in Hive-trunk-h0.21 #2395 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2395/])
HIVE-5252 - Add ql syntax for inline java code creation (Edward Capriolo via 
Brock Noland) (brock: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531549)
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/ivy/libraries.properties
* /hive/trunk/ql/ivy.xml
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/CommandProcessorFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/CompileProcessor.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/HiveCommand.java
* /hive/trunk/ql/src/test/queries/clientnegative/compile_processor.q
* /hive/trunk/ql/src/test/queries/clientpositive/compile_processor.q
* /hive/trunk/ql/src/test/results/clientnegative/compile_processor.q.out
* /hive/trunk/ql/src/test/results/clientpositive/compile_processor.q.out


 Add ql syntax for inline java code creation
 ---

 Key: HIVE-5252
 URL: https://issues.apache.org/jira/browse/HIVE-5252
 Project: Hive
  Issue Type: Sub-task
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Fix For: 0.13.0

 Attachments: HIVE-5252.1.patch.txt, HIVE-5252.2.patch.txt


 Something to the effect of compile 'my code here' using 'groovycompiler'.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5513) Set the short version directly via build script


[ 
https://issues.apache.org/jira/browse/HIVE-5513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793393#comment-13793393
 ] 

Hudson commented on HIVE-5513:
--

FAILURE: Integrated in Hive-trunk-h0.21 #2395 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/2395/])
HIVE-5513 - Set the short version directly via build script (Prasad Mujumdar 
via Brock Noland) (brock: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531550)
* /hive/trunk/build.properties
* /hive/trunk/common/build.xml


 Set the short version directly via build script
 ---

 Key: HIVE-5513
 URL: https://issues.apache.org/jira/browse/HIVE-5513
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure, Diagnosability
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Priority: Minor
 Fix For: 0.13.0

 Attachments: HIVE-5513.1.patch


 This is a followup to HIVE-5484. The short version should be configurable 
 directly from build script.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5423) Speed up testing of scalar UDFS


[ 
https://issues.apache.org/jira/browse/HIVE-5423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793396#comment-13793396
 ] 

Brock Noland commented on HIVE-5423:


+1

Looks good! Nice work! 


 Speed up testing of scalar UDFS
 ---

 Key: HIVE-5423
 URL: https://issues.apache.org/jira/browse/HIVE-5423
 Project: Hive
  Issue Type: Improvement
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Attachments: HIVE-5423.1.patch.txt, HIVE-5423.5.patch.txt, 
 HIVE-5423.6.patch.txt, HIVE-5423.patch.txt






--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5512) metastore filter pushdown should support between


 [ 
https://issues.apache.org/jira/browse/HIVE-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-5512:
---

   Resolution: Fixed
Fix Version/s: 0.13.0
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Sergey!

 metastore filter pushdown should support between
 --

 Key: HIVE-5512
 URL: https://issues.apache.org/jira/browse/HIVE-5512
 Project: Hive
  Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Fix For: 0.13.0

 Attachments: HIVE-5512.01.patch


 Currently, metastore filter pushdown supports compare operators, and and 
 or. Between is just = and =, so it should be easy to add thru changes 
 to Filter.g or even client-side modification in partition pruner.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5496) hcat -e drop database if exists fails on authorizing non-existent null db


[ 
https://issues.apache.org/jira/browse/HIVE-5496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793398#comment-13793398
 ] 

Ashutosh Chauhan commented on HIVE-5496:


+1

 hcat -e drop database if exists fails on authorizing non-existent null db
 ---

 Key: HIVE-5496
 URL: https://issues.apache.org/jira/browse/HIVE-5496
 Project: Hive
  Issue Type: Bug
  Components: Authorization, HCatalog
Affects Versions: 0.12.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
 Attachments: HIVE-5496.patch


 When running a drop database if exists call on hcat commandline, it fails 
 authorization with a NPE because it tries to authorize access to a null 
 database. This should be changed to not call authorize if the db for the 
 DropDatabaseDesc is null.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5485) SBAP errors on null partition being passed into partition level authorization


[ 
https://issues.apache.org/jira/browse/HIVE-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793400#comment-13793400
 ] 

Ashutosh Chauhan commented on HIVE-5485:


+1

 SBAP errors on null partition being passed into partition level authorization
 -

 Key: HIVE-5485
 URL: https://issues.apache.org/jira/browse/HIVE-5485
 Project: Hive
  Issue Type: Bug
  Components: Authorization
Affects Versions: 0.12.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
 Attachments: HIVE-5485.patch


 SBAP causes an NPE when null is passed in as a partition for partition-level 
 or column-level authorization.
 Personally, in my opinion, this is not a SBAP bug, but incorrect usage of 
 AuthorizationProviders - one should not be calling the column-level authorize 
 (given that column-level is more basic than partition-level) function and 
 pass in a null as the partition value. However, that happens on code 
 introduced by HIVE-1887, and unless we rewrite that (and possibly a whole 
 bunch more(will need evaluation)), we have to accommodate that null and 
 appropriately attempt to fall back to table-level authorization in that case.
 The offending code section is in Driver.java:685
 {code}
  678 // if we reach here, it means it needs to do a table 
 authorization
  679 // check, and the table authorization may already happened 
 because of other
  680 // partitions
  681 if (tbl != null  
 !tableAuthChecked.contains(tbl.getTableName()) 
  682 !(tableUsePartLevelAuth.get(tbl.getTableName()) == 
 Boolean.TRUE)) {
  683   ListString cols = tab2Cols.get(tbl);
  684   if (cols != null  cols.size()  0) {
  685 ss.getAuthorizer().authorize(tbl, null, cols,
  686 op.getInputRequiredPrivileges(), null);
  687   } else {
  688 ss.getAuthorizer().authorize(tbl, 
 op.getInputRequiredPrivileges(),
  689 null);
  690   }
  691   tableAuthChecked.add(tbl.getTableName());
  692 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-664) optimize UDF split

2013-10-12 Thread Teddy Choi (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793399#comment-13793399
]

Teddy Choi commented on HIVE-664:
-

mapreduce_stack_trace_hadoop20.q passed on my computer. The test doesn't call
split(), so it is not related with this patch.

optimize UDF split
--

Key: HIVE-664
URL: https://issues.apache.org/jira/browse/HIVE-664
Project: Hive
Issue Type: Bug
Components: UDF
Reporter: Namit Jain
Assignee: Teddy Choi
Labels: optimization
Attachments: HIVE-664.1.patch.txt, HIVE-664.2.patch.txt,
HIVE-664.3.patch.txt

Min Zhou added a comment - 21/Jul/09 07:34 AM
It's very useful for us .
some comments:
1. Can you implement it directly with Text ? Avoiding string decoding and
encoding would be faster. Of course that trick may lead to another problem,
as String.split uses a regular expression for splitting.
2. getDisplayString() always return a string in lowercase.
[ Show » ]
Min Zhou added a comment - 21/Jul/09 07:34 AM It's very useful for us . some
comments:
1. Can you implement it directly with Text ? Avoiding string decoding and
encoding would be faster. Of course that trick may lead to another problem,
as String.split uses a regular expression for splitting.
2. getDisplayString() always return a string in lowercase.
[ Permlink | « Hide ]
Namit Jain added a comment - 21/Jul/09 09:22 AM
Committed. Thanks Emil
[ Show » ]
Namit Jain added a comment - 21/Jul/09 09:22 AM Committed. Thanks Emil
[ Permlink | « Hide ]
Emil Ibrishimov added a comment - 21/Jul/09 10:48 AM
There are some easy (compromise) ways to optimize split:
1. Check if the regex argument actually contains some regex specific
characters and if it doesn't, do a straightforward split without converting
to strings.
2. Assume some default value for the second argument (for example -
split(str) to be equivalent to split(str, ' ') and optimize for this value
3. Have two separate split functions - one that does regex and one that
splits around plain text.
I think that 1 is a good choice and can be done rather quickly.
[ Show » ]
Emil Ibrishimov added a comment - 21/Jul/09 10:48 AM There are some easy
(compromise) ways to optimize split: 1. Check if the regex argument actually
contains some regex specific characters and if it doesn't, do a
straightforward split without converting to strings. 2. Assume some default
value for the second argument (for example - split(str) to be equivalent to
split(str, ' ') and optimize for this value 3. Have two separate split
functions - one that does regex and one that splits around plain text. I
think that 1 is a good choice and can be done rather quickly.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: [VOTE] Apache Hive 0.12.0 Release Candidate 1

2013-10-12 Thread Brock Noland

I have noticed the .log thing for some time. In fact I have PTest2
code to handle this:

https://github.com/apache/hive/blob/trunk/testutils/ptest2/src/main/resources/batch-exec.vm#L77

AFAICT it only occurs in local mode. I have actually fixed this in the
next patch for the maven branch.

Brock

On Sat, Oct 12, 2013 at 10:32 AM, Edward Capriolo edlinuxg...@gmail.com wrote:
I ran a list of queries in local mode, join, select, limit , group, rank,
they all worked as expected.

I am +1
It would be nice if we can fix the .log thing and one other (if we do
another rc)

There is one long standing bug that I feel I have fixed before.

configuration
property
namehive.hwi.war.file/name
valuelib/hive-hwi-0.12.0.war/value
descriptionThis sets the path to the HWI war file, relative to
${HIVE_HOME}. /description
/property
/configuration

This should be

/lib/hive-hwi-0.12.0.war

Other then that hive_web interface does work out of the box.

On Fri, Oct 11, 2013 at 11:32 PM, Thejas Nair the...@hortonworks.comwrote:

Hoping to see more activity on this thread!

I have run the unit tests and they passed with the exception of one
test in TestThriftHttpCLIService, which fails only when run in
sequence with TestThriftBinaryCLIService. It passes when run in
parallel/standalone. That is not a regression, see HIVE-5474 for
details. None of the runs on builds.apache.org have completed because
of some environment issues.

I would like to clarify that you don't have to be a committer or PMC
member to give feedback on the release candidate. So please try it out
and give feedback on the mailing list.

Somethings you can try out and use to give feedback -
- run some queries against the release candidate (local mode or
against a cluster)
- run some or all unit tests
- verify the md5 checksums

% md5sum -c hive-X.Y.Z.tar.gz.md5
hive-X.Y.Z.tar.gz: OK

% md5sum -c hive-X.Y.Z-bin.tar.gz.md5
hive-X.Y.Z-bin.tar.gz: OK

- verify the PGP signature
wget http://www.apache.org/dist/hive/KEYS
gpg --import KEYS (if necessarily)
gpg --verify hive-X.Y.Z.tar.gz.asc hive-X.Y.Z.tar.gz
gpg --verify hive-X.Y.Z-bin.tar.gz.asc hive-X.Y.Z-bin.tar.gz

On Wed, Oct 9, 2013 at 7:50 PM, Thejas Nair the...@hortonworks.com
wrote:
(Fixing subject of the email!)

On Wed, Oct 9, 2013 at 7:31 PM, Thejas Nair the...@hortonworks.com
wrote:
Apache Hive 0.12.0 Release Candidate 1 is available here:

http://people.apache.org/~thejas/hive-0.12.0-rc1/

Maven artifacts are available here:

https://repository.apache.org/content/repositories/orgapachehive-157/

Source tag for RC1 is at:
http://svn.apache.org/viewvc/hive/tags/release-0.12.0-rc1/

Voting will conclude in 72 hours.

Hive PMC Members: Please test and vote.

This RC includes the following jira fixes to address some of the
concerns raised about RC0 -
HIVE-5497 : Hive trunk broken against hadoop 0.20.2
HIVE-5493 : duplicate jars with different versions for guava,
commons-logging
HIVE-5489 : NOTICE copyright dates are out of date, README needs update
HIVE-5488 : some files are missing apache license headers

Thanks to all everybody who made this new release possible. Thanks to
everybody who reviewed RC0.

Thanks
Thejas

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.

--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org

[jira] [Commented] (HIVE-5479) SBAP restricts hcat -e 'show databases'


[ 
https://issues.apache.org/jira/browse/HIVE-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793406#comment-13793406
 ] 

Ashutosh Chauhan commented on HIVE-5479:


+1

 SBAP restricts hcat -e 'show databases'
 ---

 Key: HIVE-5479
 URL: https://issues.apache.org/jira/browse/HIVE-5479
 Project: Hive
  Issue Type: Bug
  Components: Authorization, HCatalog
Affects Versions: 0.12.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
 Attachments: HIVE-5479.patch


 During testing for 0.12, it was found that if someone tries to use the SBAP 
 as a client-side authorization provider, and runs hcat -e show databases;, 
 SBAP denies permission to the user.
 Looking at SBAP code, why it does so is self-evident from this section:
 {code}
   @Override
   public void authorize(Privilege[] readRequiredPriv, Privilege[] 
 writeRequiredPriv)
   throws HiveException, AuthorizationException {
 // Currently not used in hive code-base, but intended to authorize actions
 // that are directly user-level. As there's no storage based aspect to 
 this,
 // we can follow one of two routes:
 // a) We can allow by default - that way, this call stays out of the way
 // b) We can deny by default - that way, no privileges are authorized that
 // is not understood and explicitly allowed.
 // Both approaches have merit, but given that things like grants and 
 revokes
 // that are user-level do not make sense from the context of 
 storage-permission
 // based auth, denying seems to be more canonical here.
 throw new 
 AuthorizationException(StorageBasedAuthorizationProvider.class.getName() +
  does not allow user-level authorization);
   }
 {code}
 Thus, this deny-by-default behaviour affects the show databases call from 
 hcat cli, which uses user-level privileges to determine if a user can perform 
 that.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5335) Milestone 4: Most tests pass under maven


 [ 
https://issues.apache.org/jira/browse/HIVE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-5335:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 Milestone 4: Most tests pass under maven
 

 Key: HIVE-5335
 URL: https://issues.apache.org/jira/browse/HIVE-5335
 Project: Hive
  Issue Type: Sub-task
Reporter: Brock Noland
Assignee: Brock Noland
 Attachments: HIVE-5335-no-qfiles.patch, HIVE-5335.patch


 NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5527) Use of localtime Calendar in vectorized Timestamp arithmetic results in data corruption (depends on localtime)


 [ 
https://issues.apache.org/jira/browse/HIVE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remus Rusanu updated HIVE-5527:
---

Description: I did not yet identify the root cause, but the 
vectorization_regress.q returns different results depending on the local 
timezone settings

 Use of localtime Calendar in vectorized Timestamp arithmetic results in data 
 corruption (depends on localtime)
 --

 Key: HIVE-5527
 URL: https://issues.apache.org/jira/browse/HIVE-5527
 Project: Hive
  Issue Type: Bug
Reporter: Remus Rusanu

 I did not yet identify the root cause, but the vectorization_regress.q 
 returns different results depending on the local timezone settings



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: [VOTE] Apache Hive 0.12.0 Release Candidate 1

2013-10-12 Thread Ashutosh Chauhan

Downloaded the tar ball and tested on 1-node pseudo distributed 1.1.2
hadoop cluster.
Tested basic ddl like create / alter / drop
Tested various queries like select, join, group by, over , limit, order etc.
Everything worked as expected.

On Sat, Oct 12, 2013 at 9:33 AM, Brock Noland br...@cloudera.com wrote:

I have noticed the .log thing for some time. In fact I have PTest2
code to handle this:

https://github.com/apache/hive/blob/trunk/testutils/ptest2/src/main/resources/batch-exec.vm#L77

AFAICT it only occurs in local mode. I have actually fixed this in the
next patch for the maven branch.

Brock

On Sat, Oct 12, 2013 at 10:32 AM, Edward Capriolo edlinuxg...@gmail.com
wrote:
I ran a list of queries in local mode, join, select, limit , group, rank,
they all worked as expected.

I am +1
It would be nice if we can fix the .log thing and one other (if we do
another rc)

There is one long standing bug that I feel I have fixed before.

configuration
property
namehive.hwi.war.file/name
valuelib/hive-hwi-0.12.0.war/value
descriptionThis sets the path to the HWI war file, relative to
${HIVE_HOME}. /description
/property
/configuration

This should be

/lib/hive-hwi-0.12.0.war

Other then that hive_web interface does work out of the box.

On Fri, Oct 11, 2013 at 11:32 PM, Thejas Nair the...@hortonworks.com
wrote:

Hoping to see more activity on this thread!

I would like to clarify that you don't have to be a committer or PMC
member to give feedback on the release candidate. So please try it out
and give feedback on the mailing list.

Somethings you can try out and use to give feedback -
- run some queries against the release candidate (local mode or
against a cluster)
- run some or all unit tests
- verify the md5 checksums

% md5sum -c hive-X.Y.Z.tar.gz.md5
hive-X.Y.Z.tar.gz: OK

% md5sum -c hive-X.Y.Z-bin.tar.gz.md5
hive-X.Y.Z-bin.tar.gz: OK

On Wed, Oct 9, 2013 at 7:50 PM, Thejas Nair the...@hortonworks.com
wrote:
(Fixing subject of the email!)

On Wed, Oct 9, 2013 at 7:31 PM, Thejas Nair the...@hortonworks.com
wrote:
Apache Hive 0.12.0 Release Candidate 1 is available here:

http://people.apache.org/~thejas/hive-0.12.0-rc1/

Maven artifacts are available here:

https://repository.apache.org/content/repositories/orgapachehive-157/

Source tag for RC1 is at:
http://svn.apache.org/viewvc/hive/tags/release-0.12.0-rc1/

Voting will conclude in 72 hours.

Hive PMC Members: Please test and vote.

This RC includes the following jira fixes to address some of the
concerns raised about RC0 -
HIVE-5497 : Hive trunk broken against hadoop 0.20.2
HIVE-5493 : duplicate jars with different versions for guava,
commons-logging
HIVE-5489 : NOTICE copyright dates are out of date, README needs
update
HIVE-5488 : some files are missing apache license headers

Thanks to all everybody who made this new release possible. Thanks to
everybody who reviewed RC0.

Thanks
Thejas

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or
entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the
reader
of this message is not the intended recipient, you are hereby notified
that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender
immediately
and delete it from your system. Thank You.

--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org

[jira] [Commented] (HIVE-4943) An explode function that includes the item's position in the array


[ 
https://issues.apache.org/jira/browse/HIVE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793424#comment-13793424
 ] 

Hive QA commented on HIVE-4943:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12608151/HIVE-4943.3.patch

{color:green}SUCCESS:{color} +1 4397 tests passed

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1113/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1113/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

 An explode function that includes the item's position in the array
 --

 Key: HIVE-4943
 URL: https://issues.apache.org/jira/browse/HIVE-4943
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor
Affects Versions: 0.11.0
Reporter: Niko Stahl
  Labels: patch
 Attachments: HIVE-4943.1.patch, HIVE-4943.2.patch, HIVE-4943.3.patch

   Original Estimate: 8h
  Remaining Estimate: 8h

 A function that explodes an array and includes an output column with the 
 position of each item in the original array.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: [VOTE] Apache Hive 0.12.0 Release Candidate 1

2013-10-12 Thread Brock Noland

+1 (non-binding)

Downloaded tar, verified sigs and hashes.
Tested queries such as create, alter, drop, select, join (via mapjoin
and reduce side), order by.
Verified tag and src directory of tar.

I noticed the RELEASE NOTES are different from the tarball and the
tag.  I don't think we should block the release for this minor issue.

--- hive-0.12.0/src/RELEASE_NOTES.txt 2013-10-09 20:30:39.0 -0500
+++ release-0.12.0-rc1/RELEASE_NOTES.txt 2013-10-12 12:52:14.720002162 -0500
@@ -320,7 +320,13 @@
 * [HIVE-5429] - HiveVarcharWritable length not reset when value is changed
 * [HIVE-5431] - PassthroughOutputFormat SH changes causes
IllegalArgumentException
 * [HIVE-5433] - Fix varchar unit tests to work with hadoop-2.1.1
+* [HIVE-5476] - Authorization-provider tests fail in sequential run
 * [HIVE-5477] - maven-publish fails because it can't find
hive-metastore-0.12.0.pom
+* [HIVE-5488] - some files are missing apache license headers
+* [HIVE-5489] - NOTICE copyright dates are out of date, README needs update
+* [HIVE-5493] - duplicate jars with different versions for guava,
commons-logging
+* [HIVE-5497] - Hive trunk broken against hadoop 0.20.2
+




On Sat, Oct 12, 2013 at 11:59 AM, Ashutosh Chauhan hashut...@apache.org wrote:
 Downloaded the tar ball and tested on 1-node pseudo distributed 1.1.2
 hadoop cluster.
 Tested basic ddl like create / alter / drop
 Tested various queries like select, join, group by, over , limit, order etc.
 Everything worked as expected.

 +1


 On Sat, Oct 12, 2013 at 9:33 AM, Brock Noland br...@cloudera.com wrote:

 I have noticed the .log thing for some time. In fact I have PTest2
 code to handle this:


 https://github.com/apache/hive/blob/trunk/testutils/ptest2/src/main/resources/batch-exec.vm#L77

 AFAICT it only occurs in local mode. I have actually fixed this in the
 next patch for the maven branch.

 Brock

 On Sat, Oct 12, 2013 at 10:32 AM, Edward Capriolo edlinuxg...@gmail.com
 wrote:
  I ran a list of queries in local mode, join, select, limit , group, rank,
  they all worked as expected.
 
  I am +1
  It would be nice if we can fix the .log thing and one other (if we do
  another rc)
 
  There is one long standing bug that I feel I have fixed before.
 
  configuration
  property
namehive.hwi.war.file/name
valuelib/hive-hwi-0.12.0.war/value
descriptionThis sets the path to the HWI war file, relative to
  ${HIVE_HOME}. /description
  /property
  /configuration
 
 
  This should be
 
  /lib/hive-hwi-0.12.0.war
 
  Other then that hive_web interface does work out of the box.
 
 
  On Fri, Oct 11, 2013 at 11:32 PM, Thejas Nair the...@hortonworks.com
 wrote:
 
  Hoping to see more activity on this thread!
 
  I have run the unit tests and they passed with the exception of one
  test in TestThriftHttpCLIService, which fails only when run in
  sequence with TestThriftBinaryCLIService. It passes when run in
  parallel/standalone. That is not a regression, see  HIVE-5474 for
  details. None of the runs on builds.apache.org have completed because
  of some environment issues.
 
 
  I would like to clarify that you don't have to be a committer or PMC
  member to give feedback on the release candidate. So please try it out
  and give feedback on the mailing list.
 
  Somethings you can try out and use to give feedback -
  - run some queries against the release candidate (local mode or
  against a cluster)
  - run some or all unit tests
  - verify the md5 checksums
 
  % md5sum -c hive-X.Y.Z.tar.gz.md5
  hive-X.Y.Z.tar.gz: OK
 
  % md5sum -c hive-X.Y.Z-bin.tar.gz.md5
  hive-X.Y.Z-bin.tar.gz: OK
 
 
  - verify the PGP signature
  wget http://www.apache.org/dist/hive/KEYS
  gpg --import KEYS  (if necessarily)
  gpg --verify hive-X.Y.Z.tar.gz.asc hive-X.Y.Z.tar.gz
  gpg --verify hive-X.Y.Z-bin.tar.gz.asc hive-X.Y.Z-bin.tar.gz
 
 
 
 
  On Wed, Oct 9, 2013 at 7:50 PM, Thejas Nair the...@hortonworks.com
  wrote:
   (Fixing subject of the email!)
  
   On Wed, Oct 9, 2013 at 7:31 PM, Thejas Nair the...@hortonworks.com
  wrote:
   Apache Hive 0.12.0 Release Candidate 1 is available here:
  
   http://people.apache.org/~thejas/hive-0.12.0-rc1/
  
   Maven artifacts are available here:
  
  
 https://repository.apache.org/content/repositories/orgapachehive-157/
  
   Source tag for RC1 is at:
   http://svn.apache.org/viewvc/hive/tags/release-0.12.0-rc1/
  
   Voting will conclude in 72 hours.
  
   Hive PMC Members: Please test and vote.
  
   This RC includes the following jira fixes to address some of the
   concerns raised about RC0 -
   HIVE-5497 : Hive trunk broken against hadoop 0.20.2
   HIVE-5493 : duplicate jars with different versions for guava,
  commons-logging
   HIVE-5489 : NOTICE copyright dates are out of date, README needs
 update
   HIVE-5488 : some files are missing apache license headers
  
   Thanks to all everybody who made this new release possible. Thanks to
   everybody who reviewed RC0.
  
   Thanks
   Thejas

[jira] [Updated] (HIVE-5277) HBase handler skips rows with null valued first cells when only row key is selected

2013-10-12 Thread Teddy Choi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-5277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-5277:
-

Attachment: HIVE-5277.2.patch.txt

The first patch returned only the row key column when all columns were 
selected. The second patch fixed this error. It passed all failed tests and 
hbase_null_cell.q test.

 HBase handler skips rows with null valued first cells when only row key is 
 selected
 ---

 Key: HIVE-5277
 URL: https://issues.apache.org/jira/browse/HIVE-5277
 Project: Hive
  Issue Type: Bug
  Components: HBase Handler
Affects Versions: 0.11.0, 0.11.1, 0.12.0, 0.13.0
Reporter: Teddy Choi
Assignee: Teddy Choi
 Attachments: HIVE-5277.1.patch.txt, HIVE-5277.2.patch.txt


 HBaseStorageHandler skips rows with null valued first cells when only row key 
 is selected.
 {noformat}
 SELECT key, col1, col2 FROM hbase_table;
 key1  cell1   cell2 
 key2  NULLcell3
 SELECT COUNT(key) FROM hbase_table;
 1
 {noformat}
 HiveHBaseTableInputFormat.getRecordReader makes first cell selected to avoid 
 skipping rows. But when the first cell is null, HBase skips that row.
 http://hbase.apache.org/book/perf.reading.html 12.9.6. Optimal Loading of Row 
 Keys describes how to deal with this problem.
 I tried to find an existing issue, but I couldn't. If you find a same issue, 
 please make this issue duplicated.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5520) Use factory methods to instantiate HiveDecimal instead of constructors

2013-10-12 Thread Xuefu Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793435#comment-13793435
 ] 

Xuefu Zhang commented on HIVE-5520:
---

Hive current error handling is to set NULL value in case of errors, such as 
divide-by-zero, data underflow or overflow, etc. Instead of letting almost 
every caller to do something like this:
{code}
try {
  HiveDecimal d = new HiveDecimal(...);
catch (NumberFormatException e) {
  return null;
}
{code}
I think it's simpler and cleaner just to do this:
{code}
  return HiveDecimal.create(...);
{code}

Another reason to returning null is that HiveDecimal doesn't have a NULL 
instance, yet null value is expected in input/output for cases such error or 
miss value. Relying caller to catch a runtime exception and propagate null 
seems requiring due diligence. I saw many cases this is omitted or missed.

Since Hive puts null for error conditions, it seems natural to return null from 
multiple() or divide() because that's what the callers will do anyway.

Let me know if you have further thoughts. Thanks.

 Use factory methods to instantiate HiveDecimal instead of constructors
 --

 Key: HIVE-5520
 URL: https://issues.apache.org/jira/browse/HIVE-5520
 Project: Hive
  Issue Type: Improvement
  Components: Types
Affects Versions: 0.11.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: 0.13.0

 Attachments: HIVE-5520.1.patch, HIVE-5520.patch


 Currently HiveDecimal class provided a bunch of constructors that  
 unfortunately also throws a runtime exception. For example,
 {code}
  public HiveDecimal(BigInteger unscaled, int scale) {
 bd = this.normalize(new BigDecimal(unscaled, scale), MAX_PRECISION, 
 false);
 if (bd == null) {
  throw new NumberFormatException(Assignment would result in truncation);
}
 {code}
 As a result, it's hard for the caller to detect error occurrences and the 
 error handling is also complicated. In many cases, the error handling is 
 omitted or missed. For instance,
 {code}
  HiveDecimalWritable result = new 
 HiveDecimalWritable(HiveDecimal.ZERO);
 try {
   result.set(aggregation.sum.divide(new 
 HiveDecimal(aggregation.count)));
 } catch (NumberFormatException e) {
   result = null;
 }
 {code} 
 Throwing runtime exception while expecting caller to catch seems 
 anti-pattern. In the case of constructor, factory class or methods seem more 
 appropriate. With such a change, the apis are cleaner, and the error handling 
 is simplified.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5245) hive create table as select(CTAS) can not work(not support) with join on operator

2013-10-12 Thread Yin Huai (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793439#comment-13793439
 ] 

Yin Huai commented on HIVE-5245:


I meant you can try hive trunk and see if the error also exist. If the error 
also exist, we need to find a way to reproduce it.

 hive create table as select(CTAS) can not work(not support) with join on 
 operator
 -

 Key: HIVE-5245
 URL: https://issues.apache.org/jira/browse/HIVE-5245
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 0.11.0
Reporter: jeff little
  Labels: CTAS, hive
   Original Estimate: 96h
  Remaining Estimate: 96h

 hello everyone, recently i came across one hive problem as below:
 hive (test) create table test_09 as
 select a.* from test_01 a
 join test_02 b
 on (a.id=b.id);
 Automatically selecting local only mode for query
 Total MapReduce jobs = 2
 setting HADOOP_USER_NAMEhadoop
 13/09/09 17:22:36 WARN conf.Configuration: 
 file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a
  attempt to override final parameter: mapred.system.dir;  Ignoring.
 13/09/09 17:22:36 WARN conf.Configuration: 
 file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10008/jobconf.xml:a
  attempt to override final parameter: mapred.local.dir;  Ignoring.
 Execution log at: /tmp/hadoop/.log
 2013-09-09 05:22:36 Starting to launch local task to process map join;
   maximum memory = 932118528
 2013-09-09 05:22:37 Processing rows:4   Hashtable size: 4 
   Memory usage:   113068056   rate:   0.121
 2013-09-09 05:22:37 Dump the hashtable into file: 
 file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable
 2013-09-09 05:22:37 Upload 1 File to: 
 file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10005/HashTable-Stage-6/MapJoin-mapfile90--.hashtable
  File size: 788
 2013-09-09 05:22:37 End of local task; Time Taken: 0.444 sec.
 Execution completed successfully
 Mapred Local Task Succeeded . Convert the Join into MapJoin
 Mapred Local Task Succeeded . Convert the Join into MapJoin
 Launching Job 1 out of 2
 Number of reduce tasks is set to 0 since there's no reduce operator
 13/09/09 17:22:38 WARN conf.Configuration: 
 file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a
  attempt to override final parameter: mapred.system.dir;  Ignoring.
 13/09/09 17:22:38 WARN conf.Configuration: 
 file:/tmp/hadoop/hive_2013-09-09_17-22-34_848_1629553341892012305/-local-10009/jobconf.xml:a
  attempt to override final parameter: mapred.local.dir;  Ignoring.
 Execution log at: /tmp/hadoop/.log
 Job running in-process (local Hadoop)
 Hadoop job information for null: number of mappers: 0; number of reducers: 0
 2013-09-09 17:22:41,807 null map = 0%,  reduce = 0%
 2013-09-09 17:22:44,814 null map = 100%,  reduce = 0%
 Ended Job = job_local_0001
 Execution completed successfully
 Mapred Local Task Succeeded . Convert the Join into MapJoin
 Stage-7 is filtered out by condition resolver.
 OK
 Time taken: 13.138 seconds
 hive (test) select * from test_09;
 FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'test_09'
 hive (test)
 Problem:
 I can't get the created table, namely this CTAS is nonavailable, and this 
 table is not created by this hql sentence at all.who can explain for 
 me.Thanks.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5512) metastore filter pushdown should support between


[ 
https://issues.apache.org/jira/browse/HIVE-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793445#comment-13793445
 ] 

Hudson commented on HIVE-5512:
--

FAILURE: Integrated in Hive-trunk-hadoop2-ptest #137 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/137/])
HIVE-5512 : metastore filter pushdown should support between (Sergey Shelukhin 
via Ashutosh Chauhan) (hashutosh: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531555)
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/parser/Filter.g
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java
* 
/hive/trunk/ql/src/test/org/apache/hadoop/hive/metastore/TestMetastoreExpr.java
* /hive/trunk/ql/src/test/queries/clientpositive/filter_numeric.q
* /hive/trunk/ql/src/test/results/clientpositive/filter_numeric.q.out


 metastore filter pushdown should support between
 --

 Key: HIVE-5512
 URL: https://issues.apache.org/jira/browse/HIVE-5512
 Project: Hive
  Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Fix For: 0.13.0

 Attachments: HIVE-5512.01.patch


 Currently, metastore filter pushdown supports compare operators, and and 
 or. Between is just = and =, so it should be easy to add thru changes 
 to Filter.g or even client-side modification in partition pruner.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5513) Set the short version directly via build script


[ 
https://issues.apache.org/jira/browse/HIVE-5513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793444#comment-13793444
 ] 

Hudson commented on HIVE-5513:
--

FAILURE: Integrated in Hive-trunk-hadoop2-ptest #137 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/137/])
HIVE-5513 - Set the short version directly via build script (Prasad Mujumdar 
via Brock Noland) (brock: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531550)
* /hive/trunk/build.properties
* /hive/trunk/common/build.xml


 Set the short version directly via build script
 ---

 Key: HIVE-5513
 URL: https://issues.apache.org/jira/browse/HIVE-5513
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure, Diagnosability
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Priority: Minor
 Fix For: 0.13.0

 Attachments: HIVE-5513.1.patch


 This is a followup to HIVE-5484. The short version should be configurable 
 directly from build script.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5252) Add ql syntax for inline java code creation


[ 
https://issues.apache.org/jira/browse/HIVE-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793446#comment-13793446
 ] 

Hudson commented on HIVE-5252:
--

FAILURE: Integrated in Hive-trunk-hadoop2-ptest #137 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/137/])
HIVE-5252 - Add ql syntax for inline java code creation (Edward Capriolo via 
Brock Noland) (brock: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531549)
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/ivy/libraries.properties
* /hive/trunk/ql/ivy.xml
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/CommandProcessorFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/CompileProcessor.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/HiveCommand.java
* /hive/trunk/ql/src/test/queries/clientnegative/compile_processor.q
* /hive/trunk/ql/src/test/queries/clientpositive/compile_processor.q
* /hive/trunk/ql/src/test/results/clientnegative/compile_processor.q.out
* /hive/trunk/ql/src/test/results/clientpositive/compile_processor.q.out


 Add ql syntax for inline java code creation
 ---

 Key: HIVE-5252
 URL: https://issues.apache.org/jira/browse/HIVE-5252
 Project: Hive
  Issue Type: Sub-task
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Fix For: 0.13.0

 Attachments: HIVE-5252.1.patch.txt, HIVE-5252.2.patch.txt


 Something to the effect of compile 'my code here' using 'groovycompiler'.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (HIVE-5528) hive log file name in local is .log

2013-10-12 Thread Thejas M Nair (JIRA)

Thejas M Nair created HIVE-5528:
---

 Summary: hive log file name in local is .log
 Key: HIVE-5528
 URL: https://issues.apache.org/jira/browse/HIVE-5528
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.11.0, 0.12.0
Reporter: Thejas M Nair


In local mode the log is getting written to /tmp/{user.name}/.log instead of 
/tmp/{user.name}/hive.log



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: [VOTE] Apache Hive 0.12.0 Release Candidate 1

2013-10-12 Thread Vikram Dixit

+1 non-binding.

Downloaded tarball and verified signatures.
Ran basic queries as well as the join tests on the src code.

On Sat, Oct 12, 2013 at 10:59 AM, Brock Noland br...@cloudera.com wrote:

+1 (non-binding)

Downloaded tar, verified sigs and hashes.
Tested queries such as create, alter, drop, select, join (via mapjoin
and reduce side), order by.
Verified tag and src directory of tar.

I noticed the RELEASE NOTES are different from the tarball and the
tag. I don't think we should block the release for this minor issue.

--- hive-0.12.0/src/RELEASE_NOTES.txt 2013-10-09 20:30:39.0 -0500
+++ release-0.12.0-rc1/RELEASE_NOTES.txt 2013-10-12 12:52:14.720002162
-0500
@@ -320,7 +320,13 @@
* [HIVE-5429] - HiveVarcharWritable length not reset when value is
changed
* [HIVE-5431] - PassthroughOutputFormat SH changes causes
IllegalArgumentException
* [HIVE-5433] - Fix varchar unit tests to work with hadoop-2.1.1
+* [HIVE-5476] - Authorization-provider tests fail in sequential run
* [HIVE-5477] - maven-publish fails because it can't find
hive-metastore-0.12.0.pom
+* [HIVE-5488] - some files are missing apache license headers
+* [HIVE-5489] - NOTICE copyright dates are out of date, README needs
update
+* [HIVE-5493] - duplicate jars with different versions for guava,
commons-logging
+* [HIVE-5497] - Hive trunk broken against hadoop 0.20.2
+

On Sat, Oct 12, 2013 at 11:59 AM, Ashutosh Chauhan hashut...@apache.org
wrote:
Downloaded the tar ball and tested on 1-node pseudo distributed 1.1.2
hadoop cluster.
Tested basic ddl like create / alter / drop
Tested various queries like select, join, group by, over , limit, order
etc.
Everything worked as expected.

On Sat, Oct 12, 2013 at 9:33 AM, Brock Noland br...@cloudera.com
wrote:

I have noticed the .log thing for some time. In fact I have PTest2
code to handle this:

https://github.com/apache/hive/blob/trunk/testutils/ptest2/src/main/resources/batch-exec.vm#L77

AFAICT it only occurs in local mode. I have actually fixed this in the
next patch for the maven branch.

Brock

On Sat, Oct 12, 2013 at 10:32 AM, Edward Capriolo
edlinuxg...@gmail.com
wrote:
I ran a list of queries in local mode, join, select, limit , group,
rank,
they all worked as expected.

I am +1
It would be nice if we can fix the .log thing and one other (if we do
another rc)

There is one long standing bug that I feel I have fixed before.

configuration
property
namehive.hwi.war.file/name
valuelib/hive-hwi-0.12.0.war/value
descriptionThis sets the path to the HWI war file, relative to
${HIVE_HOME}. /description
/property
/configuration

This should be

/lib/hive-hwi-0.12.0.war

Other then that hive_web interface does work out of the box.

On Fri, Oct 11, 2013 at 11:32 PM, Thejas Nair the...@hortonworks.com
wrote:

Hoping to see more activity on this thread!

I would like to clarify that you don't have to be a committer or PMC
member to give feedback on the release candidate. So please try it
out
and give feedback on the mailing list.

Somethings you can try out and use to give feedback -
- run some queries against the release candidate (local mode or
against a cluster)
- run some or all unit tests
- verify the md5 checksums

% md5sum -c hive-X.Y.Z.tar.gz.md5
hive-X.Y.Z.tar.gz: OK

% md5sum -c hive-X.Y.Z-bin.tar.gz.md5
hive-X.Y.Z-bin.tar.gz: OK

On Wed, Oct 9, 2013 at 7:50 PM, Thejas Nair the...@hortonworks.com
wrote:
(Fixing subject of the email!)

On Wed, Oct 9, 2013 at 7:31 PM, Thejas Nair
the...@hortonworks.com
wrote:
Apache Hive 0.12.0 Release Candidate 1 is available here:

http://people.apache.org/~thejas/hive-0.12.0-rc1/

Maven artifacts are available here:

https://repository.apache.org/content/repositories/orgapachehive-157/

Source tag for RC1 is at:
http://svn.apache.org/viewvc/hive/tags/release-0.12.0-rc1/

Voting will conclude in 72 hours.

Hive PMC Members: Please test and vote.

This RC includes the following jira fixes to address some of the
concerns raised about RC0 -
HIVE-5497 : Hive trunk broken against hadoop

Re: [VOTE] Apache Hive 0.12.0 Release Candidate 1

2013-10-12 Thread Thejas Nair

Thanks Edward, Brock and Ashutosh for checking out the RC and the inputs!

Brock, you are right, I just checked and found that the RC1 package is
one commit behind the rc1 tag. The package is missing the final commit
that updated the release notes. r1530840 - Updating RELEASE_NOTES.txt
for commits since 0.12.0 RC0
Sorry about that! Please let me know if anybody thinks we should
create a new RC with just this fix.

Edward, I have created HIVE-5528 to track the .log file name issue. I
checked 0.11 release as well, it has the same issue. As it is not a
regression, and not a serious issue (such as correctness or serious
performance regression), I think we can include the fix for this in
the next release.
Regarding hive-hwi-0.12.0.war location, it looks like the location has
remained lib/hive-hwi-0.12.0.war for a while in
hive-default.xml.template (2010). Maybe your fix is yet to be
committed ? Also, I think we should make then entry in
default.xml.template in sync with what is actually the default in
HiveConf.java (System.getenv(HWI_WAR_FILE)) when we fix the issue.

Regarding the 3 day deadline for voting, that is what is in the hive
bylaws. I also see that has been followed in last few releases I
checked. In my opinion, that is very reasonable, since each release
typically has a few RCs. If we have a 7 day deadline, as most people
have tendency to wait for last day(s) before voting, it can take
several weeks for the release process to finish.

Thanks,
Thejas

On Sat, Oct 12, 2013 at 10:59 AM, Brock Noland br...@cloudera.com wrote:
+1 (non-binding)

Downloaded tar, verified sigs and hashes.
Tested queries such as create, alter, drop, select, join (via mapjoin
and reduce side), order by.
Verified tag and src directory of tar.

I noticed the RELEASE NOTES are different from the tarball and the
tag. I don't think we should block the release for this minor issue.

--- hive-0.12.0/src/RELEASE_NOTES.txt 2013-10-09 20:30:39.0 -0500
+++ release-0.12.0-rc1/RELEASE_NOTES.txt 2013-10-12 12:52:14.720002162 -0500
@@ -320,7 +320,13 @@
* [HIVE-5429] - HiveVarcharWritable length not reset when value is
changed
* [HIVE-5431] - PassthroughOutputFormat SH changes causes
IllegalArgumentException
* [HIVE-5433] - Fix varchar unit tests to work with hadoop-2.1.1
+* [HIVE-5476] - Authorization-provider tests fail in sequential run
* [HIVE-5477] - maven-publish fails because it can't find
hive-metastore-0.12.0.pom
+* [HIVE-5488] - some files are missing apache license headers
+* [HIVE-5489] - NOTICE copyright dates are out of date, README needs
update
+* [HIVE-5493] - duplicate jars with different versions for guava,
commons-logging
+* [HIVE-5497] - Hive trunk broken against hadoop 0.20.2
+

On Sat, Oct 12, 2013 at 9:33 AM, Brock Noland br...@cloudera.com wrote:

I have noticed the .log thing for some time. In fact I have PTest2
code to handle this:

https://github.com/apache/hive/blob/trunk/testutils/ptest2/src/main/resources/batch-exec.vm#L77

AFAICT it only occurs in local mode. I have actually fixed this in the
next patch for the maven branch.

Brock

On Sat, Oct 12, 2013 at 10:32 AM, Edward Capriolo edlinuxg...@gmail.com
wrote:
I ran a list of queries in local mode, join, select, limit , group, rank,
they all worked as expected.

I am +1
It would be nice if we can fix the .log thing and one other (if we do
another rc)

There is one long standing bug that I feel I have fixed before.

configuration
property
namehive.hwi.war.file/name
valuelib/hive-hwi-0.12.0.war/value
descriptionThis sets the path to the HWI war file, relative to
${HIVE_HOME}. /description
/property
/configuration

This should be

/lib/hive-hwi-0.12.0.war

Other then that hive_web interface does work out of the box.

On Fri, Oct 11, 2013 at 11:32 PM, Thejas Nair the...@hortonworks.com
wrote:

Hoping to see more activity on this thread!

I would like to clarify that you don't have to be a committer or PMC
member to give feedback on the release candidate. So please try it out
and give feedback on the mailing list.

Somethings you can try out and use to give feedback -
- run some queries against the release candidate

[jira] [Commented] (HIVE-5520) Use factory methods to instantiate HiveDecimal instead of constructors


[ 
https://issues.apache.org/jira/browse/HIVE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793459#comment-13793459
 ] 

Ashutosh Chauhan commented on HIVE-5520:


Sorry.. I misread the patch thinking its changing behavior. But, refactoring 
hasn't altered the existing behavior of returning null in error conditions.  
+1 

 Use factory methods to instantiate HiveDecimal instead of constructors
 --

 Key: HIVE-5520
 URL: https://issues.apache.org/jira/browse/HIVE-5520
 Project: Hive
  Issue Type: Improvement
  Components: Types
Affects Versions: 0.11.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Fix For: 0.13.0

 Attachments: HIVE-5520.1.patch, HIVE-5520.patch


 Currently HiveDecimal class provided a bunch of constructors that  
 unfortunately also throws a runtime exception. For example,
 {code}
  public HiveDecimal(BigInteger unscaled, int scale) {
 bd = this.normalize(new BigDecimal(unscaled, scale), MAX_PRECISION, 
 false);
 if (bd == null) {
  throw new NumberFormatException(Assignment would result in truncation);
}
 {code}
 As a result, it's hard for the caller to detect error occurrences and the 
 error handling is also complicated. In many cases, the error handling is 
 omitted or missed. For instance,
 {code}
  HiveDecimalWritable result = new 
 HiveDecimalWritable(HiveDecimal.ZERO);
 try {
   result.set(aggregation.sum.divide(new 
 HiveDecimal(aggregation.count)));
 } catch (NumberFormatException e) {
   result = null;
 }
 {code} 
 Throwing runtime exception while expecting caller to catch seems 
 anti-pattern. In the case of constructor, factory class or methods seem more 
 appropriate. With such a change, the apis are cleaner, and the error handling 
 is simplified.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (HIVE-5529) precision and scale for DECIMAL type p=2147483647 s=2147483647

2013-10-12 Thread N Campbell (JIRA)

N Campbell created HIVE-5529:


 Summary: precision and scale for DECIMAL type p=2147483647 
s=2147483647
 Key: HIVE-5529
 URL: https://issues.apache.org/jira/browse/HIVE-5529
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.11.0
Reporter: N Campbell
Priority: Critical


If you declare a column of type DECIMAL you cannot define a precision and scale 
(per ISO-SQL). Should you attempt to describe a column in a resulset etc which 
is a decimal type it reports p=2147483647 s=2147483647.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5494) Vectorization throws exception with nested UDF.


[ 
https://issues.apache.org/jira/browse/HIVE-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793479#comment-13793479
 ] 

Hudson commented on HIVE-5494:
--

ABORTED: Integrated in Hive-trunk-hadoop2 #497 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop2/497/])
HIVE-5494 : Vectorization throws exception with nested UDF. (Jitendra Nath 
Pandey via Ashutosh Chauhan) (hashutosh: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531391)
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/optimizer/physical
* 
/hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/optimizer/physical/TestVectorizer.java
* /hive/trunk/ql/src/test/queries/clientpositive/vectorization_nested_udf.q
* /hive/trunk/ql/src/test/results/clientpositive/vectorization_nested_udf.q.out


 Vectorization throws exception with nested UDF.
 ---

 Key: HIVE-5494
 URL: https://issues.apache.org/jira/browse/HIVE-5494
 Project: Hive
  Issue Type: Sub-task
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
 Fix For: 0.13.0

 Attachments: HIVE-5494.1.patch, HIVE-5494.2.patch


 {code}
 Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Udf: 
 GenericUDFAbs, is not supported
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:465)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:274)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getAggregatorExpression(VectorizationContext.java:1512)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.init(VectorGroupByOperator.java:133)
 ... 41 more
 FAILED: RuntimeException java.lang.reflect.InvocationTargetException
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5512) metastore filter pushdown should support between


[ 
https://issues.apache.org/jira/browse/HIVE-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793484#comment-13793484
 ] 

Hudson commented on HIVE-5512:
--

FAILURE: Integrated in Hive-trunk-hadoop1-ptest #202 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/202/])
HIVE-5512 : metastore filter pushdown should support between (Sergey Shelukhin 
via Ashutosh Chauhan) (hashutosh: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531555)
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/parser/Filter.g
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java
* 
/hive/trunk/ql/src/test/org/apache/hadoop/hive/metastore/TestMetastoreExpr.java
* /hive/trunk/ql/src/test/queries/clientpositive/filter_numeric.q
* /hive/trunk/ql/src/test/results/clientpositive/filter_numeric.q.out


 metastore filter pushdown should support between
 --

 Key: HIVE-5512
 URL: https://issues.apache.org/jira/browse/HIVE-5512
 Project: Hive
  Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Fix For: 0.13.0

 Attachments: HIVE-5512.01.patch


 Currently, metastore filter pushdown supports compare operators, and and 
 or. Between is just = and =, so it should be easy to add thru changes 
 to Filter.g or even client-side modification in partition pruner.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5252) Add ql syntax for inline java code creation


[ 
https://issues.apache.org/jira/browse/HIVE-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793485#comment-13793485
 ] 

Hudson commented on HIVE-5252:
--

FAILURE: Integrated in Hive-trunk-hadoop1-ptest #202 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/202/])
HIVE-5252 - Add ql syntax for inline java code creation (Edward Capriolo via 
Brock Noland) (brock: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531549)
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/ivy/libraries.properties
* /hive/trunk/ql/ivy.xml
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/CommandProcessorFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/CompileProcessor.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/HiveCommand.java
* /hive/trunk/ql/src/test/queries/clientnegative/compile_processor.q
* /hive/trunk/ql/src/test/queries/clientpositive/compile_processor.q
* /hive/trunk/ql/src/test/results/clientnegative/compile_processor.q.out
* /hive/trunk/ql/src/test/results/clientpositive/compile_processor.q.out


 Add ql syntax for inline java code creation
 ---

 Key: HIVE-5252
 URL: https://issues.apache.org/jira/browse/HIVE-5252
 Project: Hive
  Issue Type: Sub-task
Reporter: Edward Capriolo
Assignee: Edward Capriolo
 Fix For: 0.13.0

 Attachments: HIVE-5252.1.patch.txt, HIVE-5252.2.patch.txt


 Something to the effect of compile 'my code here' using 'groovycompiler'.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5513) Set the short version directly via build script


[ 
https://issues.apache.org/jira/browse/HIVE-5513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793483#comment-13793483
 ] 

Hudson commented on HIVE-5513:
--

FAILURE: Integrated in Hive-trunk-hadoop1-ptest #202 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/202/])
HIVE-5513 - Set the short version directly via build script (Prasad Mujumdar 
via Brock Noland) (brock: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1531550)
* /hive/trunk/build.properties
* /hive/trunk/common/build.xml


 Set the short version directly via build script
 ---

 Key: HIVE-5513
 URL: https://issues.apache.org/jira/browse/HIVE-5513
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure, Diagnosability
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Priority: Minor
 Fix For: 0.13.0

 Attachments: HIVE-5513.1.patch


 This is a followup to HIVE-5484. The short version should be configurable 
 directly from build script.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-3173) implement getTypeInfo database metadata method

2013-10-12 Thread N Campbell (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793491#comment-13793491
 ] 

N Campbell commented on HIVE-3173:
--

Hive 0.11 fails to define prefix and suffix information etc.

TYPE_NAME   DATA_TYPE   PRECISION   LITERAL_PREFIX  LITERAL_SUFFIX  
CREATE_PARAMS   NULLABLECASE_SENSITIVE  SEARCHABLE  
UNSIGNED_ATTRIBUTE  FIXED_PREC_SCALEAUTO_INCREMENT  LOCAL_TYPE_NAME 
MINIMUM_SCALE   MAXIMUM_SCALE   SQL_DATA_TYPE   SQL_DATETIME_SUB
NUM_PREC_RADIX
ARRAY   12 [VARCHAR]null  null  null  null  truefalse   no 
support  truefalse   false   null  0   0   null  null  
null
BIGINT  -5 [BIGINT] 19  null  null  null  truefalse   
supports all WHERE  false   false   false   null  0   0   null  
null  10
BINARY  -2 [BINARY] null  null  null  null  truefalse   
supports all WHERE  truefalse   false   null  0   0   null  
null  null
BOOLEAN 16 [BOOLEAN]null  null  null  null  truefalse   
supports all WHERE  truefalse   false   null  0   0   null  
null  null
DECIMAL 3 [DECIMAL] null  null  null  null  truefalse   
supports all WHERE  false   false   false   null  0   0   null  
null  null
DOUBLE  8 [DOUBLE]  15  null  null  null  truefalse   
supports all WHERE  false   false   false   null  0   0   null  
null  2
FLOAT   6 [FLOAT]   7   null  null  null  truefalse   
supports all WHERE  false   false   false   null  0   0   null  
null  2
INT 4 [INTEGER] 10  null  null  null  truefalse   
supports all WHERE  false   false   false   null  0   0   null  
null  10
MAP 12 [VARCHAR]null  null  null  null  truefalse   no 
support  truefalse   false   null  0   0   null  null  
null
SMALLINT5 [SMALLINT]5   null  null  null  truefalse   
supports all WHERE  false   false   false   null  0   0   null  
null  10
STRING  12 [VARCHAR]null  null  null  null  truetrue
supports all WHERE  truefalse   false   null  0   0   null  
null  null
STRUCT  12 [VARCHAR]null  null  null  null  truefalse   no 
support  truefalse   false   null  0   0   null  null  
null
TIMESTAMP   93 [TIMESTAMP]  null  null  null  null  truefalse   
supports all WHERE  truefalse   false   null  0   0   null  
null  null
TINYINT -6 [TINYINT]3   null  null  null  truefalse   
supports all WHERE  false   false   false   null  0   0   null  
null  10
UNIONTYPE   12 [VARCHAR]null  null  null  null  truefalse   
no support  truefalse   false   null  0   0   null  null  
null
null  12 [VARCHAR]null  null  null  null  truefalse   no 
support  truefalse   false   null  0   0   null  null  
null


 implement getTypeInfo database metadata method 
 ---

 Key: HIVE-3173
 URL: https://issues.apache.org/jira/browse/HIVE-3173
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.8.1
Reporter: N Campbell
 Attachments: Hive-3173.patch.txt


 The JDBC driver does not implement the database metadata method getTypeInfo. 
 Hence, an application cannot dynamically determine the available type 
 information and associated properties. 



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-4850) Implement vectorized JOIN operators


 [ 
https://issues.apache.org/jira/browse/HIVE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remus Rusanu updated HIVE-4850:
---

Attachment: HIVE-4850.04.patch

 Implement vectorized JOIN operators
 ---

 Key: HIVE-4850
 URL: https://issues.apache.org/jira/browse/HIVE-4850
 Project: Hive
  Issue Type: Sub-task
Reporter: Remus Rusanu
Assignee: Remus Rusanu
 Attachments: HIVE-4850.03.patch, HIVE-4850.04.patch, 
 HIVE-4850.1.patch, HIVE-4850.2.patch


 Easysauce



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-4850) Implement vectorized JOIN operators


 [ 
https://issues.apache.org/jira/browse/HIVE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remus Rusanu updated HIVE-4850:
---

Status: Patch Available  (was: Open)

Fixed the JoinUtils computeValue regression

 Implement vectorized JOIN operators
 ---

 Key: HIVE-4850
 URL: https://issues.apache.org/jira/browse/HIVE-4850
 Project: Hive
  Issue Type: Sub-task
Reporter: Remus Rusanu
Assignee: Remus Rusanu
 Attachments: HIVE-4850.03.patch, HIVE-4850.04.patch, 
 HIVE-4850.1.patch, HIVE-4850.2.patch


 Easysauce



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-4850) Implement vectorized JOIN operators


 [ 
https://issues.apache.org/jira/browse/HIVE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remus Rusanu updated HIVE-4850:
---

Attachment: HIVE-4850.04.patch

 Implement vectorized JOIN operators
 ---

 Key: HIVE-4850
 URL: https://issues.apache.org/jira/browse/HIVE-4850
 Project: Hive
  Issue Type: Sub-task
Reporter: Remus Rusanu
Assignee: Remus Rusanu
 Attachments: HIVE-4850.03.patch, HIVE-4850.04.patch, 
 HIVE-4850.1.patch, HIVE-4850.2.patch


 Easysauce



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-4850) Implement vectorized JOIN operators


 [ 
https://issues.apache.org/jira/browse/HIVE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Remus Rusanu updated HIVE-4850:
---

Attachment: (was: HIVE-4850.04.patch)

 Implement vectorized JOIN operators
 ---

 Key: HIVE-4850
 URL: https://issues.apache.org/jira/browse/HIVE-4850
 Project: Hive
  Issue Type: Sub-task
Reporter: Remus Rusanu
Assignee: Remus Rusanu
 Attachments: HIVE-4850.03.patch, HIVE-4850.04.patch, 
 HIVE-4850.1.patch, HIVE-4850.2.patch


 Easysauce



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: Review Request 13059: HIVE-4850 Implement vector mode map join

2013-10-12 Thread Remus Rusanu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13059/
---

(Updated Oct. 12, 2013, 9:51 p.m.)


Review request for hive, Eric Hanson and Jitendra Pandey.


Bugs: HIVE-4850
https://issues.apache.org/jira/browse/HIVE-4850


Repository: hive-git


Description
---

This is a working implementation based on current trunk. It is simpler than the 
.1 patch in as it delegates the JOIN entirely to the row-mode MapJoinOperator. 
The vectorized operator is literally calling the row-mode implementaiton for 
each row in the input batch and collects the row-mode forward into the output 
batch. This is not as bad as it seems because the JOIN operators has to resort 
to row-mode operations anyway, due to the small tables (hashtables) being 
row-mode (objects and object-inspectors). By delegating the entire join logic 
to the row mode we piggyback on the correctness of exiting implementation. I do 
plan to come up with a full-vectorized mode implementation but that would 
require changes to the hash table creation-serialization. Note that the 
filtering and key evaluation of the big table does use vectorized operators. 
the row mode applies only to the key HT lookup and to the JOIN logic


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/CommonJoinOperator.java d320b47 
  ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java 86db044 
  ql/src/java/org/apache/hadoop/hive/ql/exec/JoinUtil.java fa9ee35 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java 153b8ea 
  ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 54f2644 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java cde1a59 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 8b4c615 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorColumnAssign.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorColumnAssignFactory.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorHashKeyWrapperBatch.java
 9955d09 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorReduceSinkOperator.java 
6df3551 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java 
0fb763a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 
8f10644 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java 
ff13f89 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorExpressionWriterFactory.java
 9e189c9 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 
02c32cb 
  ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java a72ec8b 
  ql/src/test/queries/clientpositive/vectorized_mapjoin.q PRE-CREATION 
  ql/src/test/results/clientpositive/vectorized_mapjoin.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/13059/diff/


Testing
---

Manually run some join queries on alltypes_orc table.


Thanks,

Remus Rusanu

[jira] [Commented] (HIVE-5277) HBase handler skips rows with null valued first cells when only row key is selected


[ 
https://issues.apache.org/jira/browse/HIVE-5277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793523#comment-13793523
 ] 

Hive QA commented on HIVE-5277:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12608162/HIVE-5277.2.patch.txt

{color:green}SUCCESS:{color} +1 4397 tests passed

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1114/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1114/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

 HBase handler skips rows with null valued first cells when only row key is 
 selected
 ---

 Key: HIVE-5277
 URL: https://issues.apache.org/jira/browse/HIVE-5277
 Project: Hive
  Issue Type: Bug
  Components: HBase Handler
Affects Versions: 0.11.0, 0.11.1, 0.12.0, 0.13.0
Reporter: Teddy Choi
Assignee: Teddy Choi
 Attachments: HIVE-5277.1.patch.txt, HIVE-5277.2.patch.txt


 HBaseStorageHandler skips rows with null valued first cells when only row key 
 is selected.
 {noformat}
 SELECT key, col1, col2 FROM hbase_table;
 key1  cell1   cell2 
 key2  NULLcell3
 SELECT COUNT(key) FROM hbase_table;
 1
 {noformat}
 HiveHBaseTableInputFormat.getRecordReader makes first cell selected to avoid 
 skipping rows. But when the first cell is null, HBase skips that row.
 http://hbase.apache.org/book/perf.reading.html 12.9.6. Optimal Loading of Row 
 Keys describes how to deal with this problem.
 I tried to find an existing issue, but I couldn't. If you find a same issue, 
 please make this issue duplicated.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5529) precision and scale for DECIMAL type p=2147483647 s=2147483647

2013-10-12 Thread Xuefu Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793525#comment-13793525
 ] 

Xuefu Zhang commented on HIVE-5529:
---

This would be address in HIVE-5355 per HIVE-3976.

 precision and scale for DECIMAL type p=2147483647 s=2147483647
 --

 Key: HIVE-5529
 URL: https://issues.apache.org/jira/browse/HIVE-5529
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.11.0
Reporter: N Campbell
Priority: Critical

 If you declare a column of type DECIMAL you cannot define a precision and 
 scale (per ISO-SQL). Should you attempt to describe a column in a resulset 
 etc which is a decimal type it reports p=2147483647 s=2147483647.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-4850) Implement vectorized JOIN operators


[ 
https://issues.apache.org/jira/browse/HIVE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793541#comment-13793541
 ] 

Hive QA commented on HIVE-4850:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12608178/HIVE-4850.04.patch

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 4397 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_mapjoin
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1115/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1115/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

 Implement vectorized JOIN operators
 ---

 Key: HIVE-4850
 URL: https://issues.apache.org/jira/browse/HIVE-4850
 Project: Hive
  Issue Type: Sub-task
Reporter: Remus Rusanu
Assignee: Remus Rusanu
 Attachments: HIVE-4850.03.patch, HIVE-4850.04.patch, 
 HIVE-4850.1.patch, HIVE-4850.2.patch


 Easysauce



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (HIVE-5530) null pointer exception when case returns null

2013-10-12 Thread N Campbell (JIRA)

N Campbell created HIVE-5530:


 Summary: null pointer exception when case returns null
 Key: HIVE-5530
 URL: https://issues.apache.org/jira/browse/HIVE-5530
 Project: Hive
  Issue Type: Bug
  Components: SQL
Affects Versions: 0.11.0
Reporter: N Campbell
Priority: Minor


The following expression will cause an NPE
 
select case when 1 = 1 then null end  from t



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5525) Vectorized query failing for partitioned tables.


 [ 
https://issues.apache.org/jira/browse/HIVE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jitendra Nath Pandey updated HIVE-5525:
---

Status: Patch Available  (was: Open)

 Vectorized query failing for partitioned tables.
 

 Key: HIVE-5525
 URL: https://issues.apache.org/jira/browse/HIVE-5525
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
 Attachments: HIVE-5525.1.patch


 {code}
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getInputColumnIndex(VectorizationContext.java:154)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorBinaryComparisonFilterExpression(VectorizationContext.java:1163)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:436)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:274)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.init(VectorFilterOperator.java:50)
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (HIVE-5525) Vectorized query failing for partitioned tables.


 [ 
https://issues.apache.org/jira/browse/HIVE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jitendra Nath Pandey updated HIVE-5525:
---

Status: Open  (was: Patch Available)

 Vectorized query failing for partitioned tables.
 

 Key: HIVE-5525
 URL: https://issues.apache.org/jira/browse/HIVE-5525
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
 Attachments: HIVE-5525.1.patch


 {code}
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getInputColumnIndex(VectorizationContext.java:154)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorBinaryComparisonFilterExpression(VectorizationContext.java:1163)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:436)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:274)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.init(VectorFilterOperator.java:50)
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5525) Vectorized query failing for partitioned tables.


[ 
https://issues.apache.org/jira/browse/HIVE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13793576#comment-13793576
 ] 

Jitendra Nath Pandey commented on HIVE-5525:


The failed test is not related to the patch, and I ran it multiple times on my 
machine and it passes.

 Vectorized query failing for partitioned tables.
 

 Key: HIVE-5525
 URL: https://issues.apache.org/jira/browse/HIVE-5525
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
 Attachments: HIVE-5525.1.patch


 {code}
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getInputColumnIndex(VectorizationContext.java:154)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorBinaryComparisonFilterExpression(VectorizationContext.java:1163)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:436)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:274)
 at 
 org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.init(VectorFilterOperator.java:50)
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (HIVE-5525) Vectorized query failing for partitioned tables.