Review Request 36086: Removed LazyBinaryUtil dependency for DateWritable by porting byte -- int logic.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36086/ --- Review request for hadoop-common, hive and Owen O'Malley. Repository: hive-git Description --- HIVE-11137: Removing LazyBinaryUtils dependency from DateWritable Diffs - .gitignore 4d341a0c3c1483a5cad45fa9cc0e2c1dc9bcff2e .reviewboardrc abc33f91a44b76573cbba334c33417307c63956f serde/src/java/org/apache/hadoop/hive/serde2/io/DateWritable.java 6cedf4c88dc07926aa5f15d657daeaea2156e58c serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryDate.java d0c2504ba529803be2880dcbb440ef8cdce1590a Diff: https://reviews.apache.org/r/36086/diff/ Testing --- Thanks, Nishant Kelkar
Re: Review Request 35968: 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35968/
---
(Updated June 29, 2015, 9:24 p.m.)
Review request for hive and Alexander Pivovarov.
Changes
---
Removed dependency on commons-math3 FastMath class.
Repository: hive-git
Description
---
1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and
integration tests. 3. Registered the UDF in the FunctionRegistry class.
Diffs (updated)
-
.reviewboardrc abc33f91a44b76573cbba334c33417307c63956f
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
fabc21e2092561cbf98c35a406e4ee40e71fe1de
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFCosineSimilarity.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_error_1.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_1.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_2.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_3.q
PRE-CREATION
ql/src/test/queries/clientpositive/udf_cosine_similarity.q PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_error_1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_error_2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_3.q.out
PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out
5de4ffcd1ace477af026b83fb7bfb8068fc192b3
ql/src/test/results/clientpositive/udf_cosine_similarity.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/35968/diff/
Testing
---
Function signature of the UDF is: cosine_similarity(Text, Text, Text)
Each Text can be one of {S=something,E=empty,N=null}
Unit tests written for the following cases:
1. cosine_similarity(S, S, S)
2. cosine_similarity(S, E, S)
3. cosine_similarity(N, E, S)
4. cosine_similarity(S, S, E)
5. cosine_similarity(N, N, N)
Thanks,
Nishant Kelkar
Re: Review Request 35968: 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35968/#review89800
---
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java (line 4)
https://reviews.apache.org/r/35968/#comment142597
Oh, sorry didn't read well. You mean math3. Yes, you're right about this.
Moving to use
{code}
Math.pow(double, double)
{code}
instead.
- Nishant Kelkar
On June 28, 2015, 11:39 a.m., Nishant Kelkar wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35968/
---
(Updated June 28, 2015, 11:39 a.m.)
Review request for hive and Alexander Pivovarov.
Repository: hive-git
Description
---
1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and
integration tests. 3. Registered the UDF in the FunctionRegistry class.
Diffs
-
.reviewboardrc abc33f91a44b76573cbba334c33417307c63956f
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
fabc21e2092561cbf98c35a406e4ee40e71fe1de
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFCosineSimilarity.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_error_1.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_1.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_2.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_3.q
PRE-CREATION
ql/src/test/queries/clientpositive/udf_cosine_similarity.q PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_error_1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_error_2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_3.q.out
PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out
5de4ffcd1ace477af026b83fb7bfb8068fc192b3
ql/src/test/results/clientpositive/udf_cosine_similarity.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/35968/diff/
Testing
---
Function signature of the UDF is: cosine_similarity(Text, Text, Text)
Each Text can be one of {S=something,E=empty,N=null}
Unit tests written for the following cases:
1. cosine_similarity(S, S, S)
2. cosine_similarity(S, E, S)
3. cosine_similarity(N, E, S)
4. cosine_similarity(S, S, E)
5. cosine_similarity(N, N, N)
Thanks,
Nishant Kelkar
Re: Review Request 35968: 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35968/#review89671 --- ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java (line 32) https://reviews.apache.org/r/35968/#comment142333 You're right, we could potentially extend this method. ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java (line 72) https://reviews.apache.org/r/35968/#comment142335 These still throw HiveException, but I've included all of the code in evaluate() in try/catch, and catch returns null. ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java (line 88) https://reviews.apache.org/r/35968/#comment142336 These still throw HiveException, but I've included all of the code in evaluate() in try/catch, and catch returns null. ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java (line 109) https://reviews.apache.org/r/35968/#comment142337 These still throw HiveException, but I've included all of the code in evaluate() in try/catch, and catch returns null. ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java (line 122) https://reviews.apache.org/r/35968/#comment142338 These still throw HiveException, but I've included all of the code in evaluate() in try/catch, and catch returns null. - Nishant Kelkar On June 28, 2015, 11:22 a.m., Nishant Kelkar wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35968/ --- (Updated June 28, 2015, 11:22 a.m.) Review request for hive and Alexander Pivovarov. Repository: hive-git Description --- 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class. Diffs - .reviewboardrc abc33f91a44b76573cbba334c33417307c63956f ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java fabc21e2092561cbf98c35a406e4ee40e71fe1de ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFCosineSimilarity.java PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_error_1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_3.q PRE-CREATION ql/src/test/queries/clientpositive/udf_cosine_similarity.q PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_error_1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_error_2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_3.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out 5de4ffcd1ace477af026b83fb7bfb8068fc192b3 ql/src/test/results/clientpositive/udf_cosine_similarity.q.out PRE-CREATION Diff: https://reviews.apache.org/r/35968/diff/ Testing --- Thanks, Nishant Kelkar
Re: Review Request 35968: 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35968/ --- (Updated June 28, 2015, 11:29 a.m.) Review request for hive and Alexander Pivovarov. Changes --- Removed unused code and import statements from TestUDFCosineSimilarity. Repository: hive-git Description --- 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class. Diffs (updated) - .reviewboardrc abc33f91a44b76573cbba334c33417307c63956f ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java fabc21e2092561cbf98c35a406e4ee40e71fe1de ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFCosineSimilarity.java PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_error_1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_3.q PRE-CREATION ql/src/test/queries/clientpositive/udf_cosine_similarity.q PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_error_1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_error_2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_3.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out 5de4ffcd1ace477af026b83fb7bfb8068fc192b3 ql/src/test/results/clientpositive/udf_cosine_similarity.q.out PRE-CREATION Diff: https://reviews.apache.org/r/35968/diff/ Testing --- Thanks, Nishant Kelkar
Re: Review Request 35968: 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35968/
---
(Updated June 28, 2015, 11:39 a.m.)
Review request for hive and Alexander Pivovarov.
Changes
---
Added description for tests performed in Tests Done section.
Repository: hive-git
Description
---
1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and
integration tests. 3. Registered the UDF in the FunctionRegistry class.
Diffs
-
.reviewboardrc abc33f91a44b76573cbba334c33417307c63956f
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
fabc21e2092561cbf98c35a406e4ee40e71fe1de
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFCosineSimilarity.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_error_1.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_1.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_2.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_3.q
PRE-CREATION
ql/src/test/queries/clientpositive/udf_cosine_similarity.q PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_error_1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_error_2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_3.q.out
PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out
5de4ffcd1ace477af026b83fb7bfb8068fc192b3
ql/src/test/results/clientpositive/udf_cosine_similarity.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/35968/diff/
Testing (updated)
---
Function signature of the UDF is: cosine_similarity(Text, Text, Text)
Each Text can be one of {S=something,E=empty,N=null}
Unit tests written for the following cases:
1. cosine_similarity(S, S, S)
2. cosine_similarity(S, E, S)
3. cosine_similarity(N, E, S)
4. cosine_similarity(S, S, E)
5. cosine_similarity(N, N, N)
Thanks,
Nishant Kelkar
[jira] [Created] (HIVE-11136) Unused Logger in org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCount
Nishant Kelkar created HIVE-11136:
-
Summary: Unused Logger in
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCount
Key: HIVE-11136
URL: https://issues.apache.org/jira/browse/HIVE-11136
Project: Hive
Issue Type: Improvement
Reporter: Nishant Kelkar
Priority: Trivial
There is an unused logger in the GenericUDAFCount. We could either try/catch
all the code that throws exceptions and log them, or get rid of the logger.
Relevant section of code:
{code}
public class GenericUDAFCount implements GenericUDAFResolver2 {
private static final Log LOG =
LogFactory.getLog(GenericUDAFCount.class.getName());
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 35968: 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35968/ --- (Updated June 28, 2015, 11:22 a.m.) Review request for hive and Alexander Pivovarov. Repository: hive-git Description --- 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class. Diffs (updated) - .reviewboardrc abc33f91a44b76573cbba334c33417307c63956f ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java fabc21e2092561cbf98c35a406e4ee40e71fe1de ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFCosineSimilarity.java PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_error_1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_2.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_3.q PRE-CREATION ql/src/test/queries/clientpositive/udf_cosine_similarity.q PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_error_1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_error_2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_2.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_3.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out 5de4ffcd1ace477af026b83fb7bfb8068fc192b3 ql/src/test/results/clientpositive/udf_cosine_similarity.q.out PRE-CREATION Diff: https://reviews.apache.org/r/35968/diff/ Testing --- Thanks, Nishant Kelkar
Re: Review Request 35968: 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35968/#review89646
---
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java (line 4)
https://reviews.apache.org/r/35968/#comment142310
Hey Alexander,
I found the following in the ~hive/ql/pom.xml:
dependency
groupIdorg.apache.commons/groupId
artifactIdcommons-lang3/artifactId
version${commons-lang3.version}/version
/dependency
Looks like we have this dependency?
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java (line 93)
https://reviews.apache.org/r/35968/#comment142311
Oh, I use Java 1.7, so I don't need to mention HashSetString. But I'll
probably need to, to be backwards compatible with people compiling with Java
1.6.
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java (line 109)
https://reviews.apache.org/r/35968/#comment142320
Do we usually log such edge cases in Hive? If yes, I could add a slf4j
logger to this UDF.
- Nishant Kelkar
On June 28, 2015, 11:22 a.m., Nishant Kelkar wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35968/
---
(Updated June 28, 2015, 11:22 a.m.)
Review request for hive and Alexander Pivovarov.
Repository: hive-git
Description
---
1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and
integration tests. 3. Registered the UDF in the FunctionRegistry class.
Diffs
-
.reviewboardrc abc33f91a44b76573cbba334c33417307c63956f
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
fabc21e2092561cbf98c35a406e4ee40e71fe1de
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFCosineSimilarity.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_error_1.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_1.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_2.q
PRE-CREATION
ql/src/test/queries/clientnegative/udf_cosine_similarity_wrongargs_3.q
PRE-CREATION
ql/src/test/queries/clientpositive/udf_cosine_similarity.q PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_error_1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_error_2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_cosine_similarity_wrongargs_3.q.out
PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out
5de4ffcd1ace477af026b83fb7bfb8068fc192b3
ql/src/test/results/clientpositive/udf_cosine_similarity.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/35968/diff/
Testing
---
Thanks,
Nishant Kelkar
Review Request 35968: 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35968/ --- Review request for hive and Alexander Pivovarov. Repository: hive-git Description --- 1. Added preliminary UDF code for cosine similarity. 2. Added unit tests and integration tests. 3. Registered the UDF in the FunctionRegistry class. Diffs - .reviewboardrc abc33f91a44b76573cbba334c33417307c63956f ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java fabc21e2092561cbf98c35a406e4ee40e71fe1de ql/src/java/org/apache/hadoop/hive/ql/udf/UDFCosineSimilarity.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFCosineSimilarity.java PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_error_1.q PRE-CREATION ql/src/test/queries/clientnegative/udf_cosine_similarity_error_2.q PRE-CREATION ql/src/test/queries/clientpositive/udf_cosine_similarity.q PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_error_1.q.out PRE-CREATION ql/src/test/results/clientnegative/udf_cosine_similarity_error_2.q.out PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out 5de4ffcd1ace477af026b83fb7bfb8068fc192b3 ql/src/test/results/clientpositive/udf_cosine_similarity.q.out PRE-CREATION Diff: https://reviews.apache.org/r/35968/diff/ Testing --- Thanks, Nishant Kelkar
[jira] [Created] (HIVE-11114) Documentation of Pentaho Missing from Maven Central
Nishant Kelkar created HIVE-4: - Summary: Documentation of Pentaho Missing from Maven Central Key: HIVE-4 URL: https://issues.apache.org/jira/browse/HIVE-4 Project: Hive Issue Type: Task Reporter: Nishant Kelkar Assignee: Nishant Kelkar Priority: Minor I recently cloned the Hive Git repository. When I went into the hive/ql sub-project and issued the command 'mvn clean compile -Phadoop-1', I got the following build error: [ERROR] Failed to execute goal on project hive-exec: Could not resolve dependencies for project org.apache.hive:hive-exec:jar:2.0.0-SNAPSHOT: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in US (http://repo.maven.apache.org/maven2) - [Help 1] This is because the pentaho-aggdesigner-algorithm dependency is not supported by Maven central; however, it is supported by Conjars. As a quick fix, I downloaded the jar from Conjars repo, and manually installed this dependency to my local Maven by following the instructions here: http://www.mkyong.com/maven/how-to-include-library-manully-into-maven-local-repository/ However, I feel this dependency should be supported on Maven central (I'm not sure where to create this ticket/whom with, but Hive is my use case, so any pointers greatly appreciated). This ticket tracks the task of documenting this fact on the Hive wiki as an additional Note. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Cloning Hive Trunk: HadoopArchives Dependency Not Found
Hmm, so I just ran the same command with 'mvn clean compile -Phadoop-1' and
'mvn clean compile -Phadoop-2' and they seem to work.
Thanks @all! I guess this wasn't an issue. :)
Thanks and Regards,
Nishant Kelkar
On Wed, Jun 24, 2015 at 1:59 AM, Nishant Kelkar nishant@gmail.com
wrote:
Hi all,
So I am a beginner to Hive, pardon me if I'm doing something wrong. I
recently cloned the Hive dev trunk (master branch) via Git. I then went
into my IDE (IntelliJ) and started looking around. At first, it resolved a
lot of dependencies. However, after my IDE finished pulling from different
sources, I saw that it was still missing the *HadoopArchives* dependency.
Later on, I was trying to build the hive/ql sub-project with 'mvn clean
compile', but I'm getting the following error:
*---BEGIN---*
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile
(default-compile) on project hive-exec: Compilation failure: Compilation
failure:
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:[213,31]
cannot find symbol
[ERROR] symbol: class HadoopArchives
[ERROR] location: package org.apache.hadoop.tools
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java:[42,41]
cannot find symbol
[ERROR] symbol: class TimelineClient
[ERROR] location: package org.apache.hadoop.yarn.client.api
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java:[57,18]
cannot find symbol
[ERROR] symbol: class TimelineClient
[ERROR] location: class org.apache.hadoop.hive.ql.hooks.ATSHook
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:[1351,9]
cannot find symbol
[ERROR] symbol: class HadoopArchives
[ERROR] location: class org.apache.hadoop.hive.ql.exec.DDLTask
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:[1351,34]
cannot find symbol
[ERROR] symbol: class HadoopArchives
[ERROR] location: class org.apache.hadoop.hive.ql.exec.DDLTask
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java:[72,26]
cannot find symbol
[ERROR] symbol: variable TimelineClient
[ERROR] location: class org.apache.hadoop.hive.ql.hooks.ATSHook
[ERROR] - [Help 1]
*---END---*
I looked at the pom.xml, and the following is highlighted in red:
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-archives/artifactId
version${hadoop-23.version}/version
optionaltrue/optional
/dependency
This basically means that IntelliJ/Maven was not able to find the
dependency under the Maven repo. I saw the ticket HIVE-9048
https://issues.apache.org/jira/browse/HIVE-9048 and HadoopArchives
should be a sub-dependency of the hadoop-tools dependency right?
Here's a copy of the relevant mirrors that I'm using in my settings.xml:
mirror
idConjars/id
mirrorOf*/mirrorOf
nameUS Conjars/name
urlhttp://conjars.org/repo/url
/mirror
mirror
idUS/id
mirrorOfcentral/mirrorOf
nameUS Central/name
urlhttp://repo.maven.apache.org/maven2/url
/mirror
Does anyone have an idea why this could be happening? I was kind of
expecting just cloning from Git, and running 'mvn clean compile' and seeing
everything pass and build with success.
Best Regards,
Nishant Kelkar
Re: Cloning Hive Trunk: HadoopArchives Dependency Not Found
Hi all,
So I am a beginner to Hive, pardon me if I'm doing something wrong. I
recently cloned the Hive dev trunk (master branch) via Git. I then went
into my IDE (IntelliJ) and started looking around. At first, it resolved a
lot of dependencies. However, after my IDE finished pulling from different
sources, I saw that it was still missing the *HadoopArchives* dependency.
Later on, I was trying to build the hive/ql sub-project with 'mvn clean
compile', but I'm getting the following error:
*---BEGIN---*
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile
(default-compile) on project hive-exec: Compilation failure: Compilation
failure:
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:[213,31]
cannot find symbol
[ERROR] symbol: class HadoopArchives
[ERROR] location: package org.apache.hadoop.tools
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java:[42,41]
cannot find symbol
[ERROR] symbol: class TimelineClient
[ERROR] location: package org.apache.hadoop.yarn.client.api
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java:[57,18]
cannot find symbol
[ERROR] symbol: class TimelineClient
[ERROR] location: class org.apache.hadoop.hive.ql.hooks.ATSHook
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:[1351,9]
cannot find symbol
[ERROR] symbol: class HadoopArchives
[ERROR] location: class org.apache.hadoop.hive.ql.exec.DDLTask
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:[1351,34]
cannot find symbol
[ERROR] symbol: class HadoopArchives
[ERROR] location: class org.apache.hadoop.hive.ql.exec.DDLTask
[ERROR]
/Users/nishantkelkar/hive/ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java:[72,26]
cannot find symbol
[ERROR] symbol: variable TimelineClient
[ERROR] location: class org.apache.hadoop.hive.ql.hooks.ATSHook
[ERROR] - [Help 1]
*---END---*
I looked at the pom.xml, and the following is highlighted in red:
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-archives/artifactId
version${hadoop-23.version}/version
optionaltrue/optional
/dependency
This basically means that IntelliJ/Maven was not able to find the
dependency under the Maven repo. I saw the ticket HIVE-9048
https://issues.apache.org/jira/browse/HIVE-9048 and HadoopArchives should
be a sub-dependency of the hadoop-tools dependency right?
Here's a copy of the relevant mirrors that I'm using in my settings.xml:
mirror
idConjars/id
mirrorOf*/mirrorOf
nameUS Conjars/name
urlhttp://conjars.org/repo/url
/mirror
mirror
idUS/id
mirrorOfcentral/mirrorOf
nameUS Central/name
urlhttp://repo.maven.apache.org/maven2/url
/mirror
Does anyone have an idea why this could be happening? I was kind of
expecting just cloning from Git, and running 'mvn clean compile' and seeing
everything pass and build with success.
Best Regards,
Nishant Kelkar
[jira] [Updated] (HIVE-8642) Hive stack() UDTF Doesn't Support NULL Insert Values
[
https://issues.apache.org/jira/browse/HIVE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Kelkar updated HIVE-8642:
-
Summary: Hive stack() UDTF Doesn't Support NULL Insert Values (was: Hive
stack() Method Doesn't Support NULL Insert Values)
Hive stack() UDTF Doesn't Support NULL Insert Values
Key: HIVE-8642
URL: https://issues.apache.org/jira/browse/HIVE-8642
Project: Hive
Issue Type: Improvement
Components: SQL, UDF
Affects Versions: 0.13.0
Reporter: Nishant Kelkar
Priority: Minor
The following tables exist:
1. test_table -- schema {field_a:STRING, field_b:STRING,
field_c:STRING}
2. dual -- schema {field_1:INT}
test_table contents:
{code}
endbpedia http://dbpedia.org/schools/CMU
{code}
dual contents:
{code}
1
{code}
I tried the following query, and it fails:
{code}
INSERT INTO TABLE test_table SELECT
stack(1,
en,
dbpedia,
NULL
) AS (
field_a,
field_b,
field_c
) FROM dual;
{code}
The error returned: FAILED: NullPointerException null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8642) Hive stack() UDTF Doesn't Support NULL Insert Values
[
https://issues.apache.org/jira/browse/HIVE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188151#comment-14188151
]
Nishant Kelkar commented on HIVE-8642:
--
Stack trace for the exception:
{code}
2014-10-29 01:23:36,757 INFO parse.SemanticAnalyzer
(SemanticAnalyzer.java:analyzeInternal(8185)) - Starting Semantic Analysis
2014-10-29 01:23:36,757 INFO parse.SemanticAnalyzer
(SemanticAnalyzer.java:analyzeInternal(8215)) - Completed phase 1 of Semantic
Analysis
2014-10-29 01:23:36,757 INFO parse.SemanticAnalyzer
(SemanticAnalyzer.java:getMetaData(951)) - Get metadata for source tables
2014-10-29 01:23:36,771 INFO parse.SemanticAnalyzer
(SemanticAnalyzer.java:getMetaData(1036)) - Get metadata for subqueries
2014-10-29 01:23:36,771 INFO parse.SemanticAnalyzer
(SemanticAnalyzer.java:getMetaData(1056)) - Get metadata for destination tables
2014-10-29 01:23:36,784 INFO parse.SemanticAnalyzer
(SemanticAnalyzer.java:analyzeInternal(8218)) - Completed getting MetaData in
Semantic Analysis
2014-10-29 01:23:36,787 ERROR ql.Driver (SessionState.java:printError(427)) -
FAILED: NullPointerException null
java.lang.NullPointerException
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.supportsConstantObjectInspector(ObjectInspectorUtils.java:943)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:115)
at
org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:215)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:781)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:908)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:87)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:124)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:101)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:166)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:8459)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:2485)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:2284)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:6843)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:6814)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:7547)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8224)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:459)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:349)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:938)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
{code}
Hive stack() UDTF Doesn't Support NULL Insert Values
Key: HIVE-8642
URL: https://issues.apache.org/jira/browse/HIVE-8642
Project: Hive
Issue Type: Improvement
Components: SQL, UDF
Affects Versions: 0.13.0
Reporter: Nishant Kelkar
Priority: Minor
The following tables exist:
1. test_table -- schema {field_a:STRING, field_b:STRING,
field_c:STRING}
2. dual -- schema {field_1:INT}
test_table contents:
{code}
endbpedia http://dbpedia.org/schools/CMU
{code}
dual contents:
{code}
1
{code}
I tried the following query, and it fails:
{code}
INSERT INTO TABLE
[jira] [Commented] (HIVE-8642) Hive stack() UDTF Doesn't Support NULL Insert Values
[
https://issues.apache.org/jira/browse/HIVE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188209#comment-14188209
]
Nishant Kelkar commented on HIVE-8642:
--
Thanks [~navis]! This solves my problem.
However, I feel that queries of the form INSERT INTO TABLE table_a SELECT
UDTF(args) AS (arg_names) FROM table_b should have support for null field
insertions. This is because the output of the UDTF is tightly coupled with the
schema of table_a. Right now, debugging through the code showed that null
arguments to a UDTF generate VoidObjectInspector as input to the initialize()
method (which they shouldn't, since the schema of table_a should be sufficient
information to get correct ObjectInspectors).
Please correct me if I'm wrong (I've only started digging into Hive source
code). Thoughts?
Hive stack() UDTF Doesn't Support NULL Insert Values
Key: HIVE-8642
URL: https://issues.apache.org/jira/browse/HIVE-8642
Project: Hive
Issue Type: Improvement
Components: SQL, UDF
Affects Versions: 0.13.0
Reporter: Nishant Kelkar
Priority: Minor
The following tables exist:
1. test_table -- schema {field_a:STRING, field_b:STRING,
field_c:STRING}
2. dual -- schema {field_1:INT}
test_table contents:
{code}
endbpedia http://dbpedia.org/schools/CMU
{code}
dual contents:
{code}
1
{code}
I tried the following query, and it fails:
{code}
INSERT INTO TABLE test_table SELECT
stack(1,
en,
dbpedia,
NULL
) AS (
field_a,
field_b,
field_c
) FROM dual;
{code}
The error returned: FAILED: NullPointerException null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8642) Hive stack() Method Doesn't Support NULL Insert Values
Nishant Kelkar created HIVE-8642:
Summary: Hive stack() Method Doesn't Support NULL Insert Values
Key: HIVE-8642
URL: https://issues.apache.org/jira/browse/HIVE-8642
Project: Hive
Issue Type: Improvement
Components: SQL, UDF
Affects Versions: 0.13.0
Reporter: Nishant Kelkar
Priority: Minor
The following tables exist:
1. test_table -- schema {field_a:STRING, field_b:STRING, field_c:STRING}
2. dual -- schema {field_1:INT}
test_table contents:
{code}
en dbpedia http://dbpedia.org/schools/CMU
{code}
dual contents:
{code}
1
{code}
I tried the following query, and it fails:
{code}
INSERT INTO TABLE test_table SELECT
stack(1,
en,
dbpedia,
NULL
) AS (
field_a,
field_b,
field_c
) FROM dual;
{code}
The error returned: FAILED: NullPointerException null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8642) Hive stack() Method Doesn't Support NULL Insert Values
[
https://issues.apache.org/jira/browse/HIVE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187864#comment-14187864
]
Nishant Kelkar commented on HIVE-8642:
--
Pardon me if I've missed out on any details in this ticket. This is my first
ticket on this Jira board (well, first technical ticket).
Tagging you in case I've missed something [~leftylev]!
Hive stack() Method Doesn't Support NULL Insert Values
--
Key: HIVE-8642
URL: https://issues.apache.org/jira/browse/HIVE-8642
Project: Hive
Issue Type: Improvement
Components: SQL, UDF
Affects Versions: 0.13.0
Reporter: Nishant Kelkar
Priority: Minor
The following tables exist:
1. test_table -- schema {field_a:STRING, field_b:STRING,
field_c:STRING}
2. dual -- schema {field_1:INT}
test_table contents:
{code}
endbpedia http://dbpedia.org/schools/CMU
{code}
dual contents:
{code}
1
{code}
I tried the following query, and it fails:
{code}
INSERT INTO TABLE test_table SELECT
stack(1,
en,
dbpedia,
NULL
) AS (
field_a,
field_b,
field_c
) FROM dual;
{code}
The error returned: FAILED: NullPointerException null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-7181) Beginner User On Apache Jira
Nishant Kelkar created HIVE-7181: Summary: Beginner User On Apache Jira Key: HIVE-7181 URL: https://issues.apache.org/jira/browse/HIVE-7181 Project: Hive Issue Type: Wish Reporter: Nishant Kelkar Priority: Minor Hi All! I've just started to use Apache's Jira board (I registered today). I've used Jira for my work before, so I know how to navigate within Jira. But my main question, was understanding how issues are handled in the open source community (to which I want to contribute, but I'm a noob here too). So basically, a person comes up with a ticket when he/she thinks that the issue they are facing, is a bug/improvement. Questions: 1. Whom am I supposed to assign the ticket to? (myself?) 2. Who would be the QA assignee? 3. If addressing the issue requires looking at the code, how am I supposed to change the code and bring into effect those changes? (At work, we maintain a Git repo on our private server. So everyone always has access to the latest code). 4. Where can I find a list of all the people who are active on this project (Hive)? It would be nice if I could tag people by their names in my ticket comments. 5. Where can I find well formatted documentation about how to take issues from discovery to fixture on Apache Jira? I apologize in advance, if my questions are too simple. Thanks, and any/all help is appreciated! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7181) Beginner User On Apache Jira
[ https://issues.apache.org/jira/browse/HIVE-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019393#comment-14019393 ] Nishant Kelkar commented on HIVE-7181: -- [~leftylev], thanks so much! These links are really helpful! Specially, the People page and the How to Contribute pages. Also, I've sent a request email at user-subscr...@hive.apache.org and dev-subscr...@hive.apache.org, so I guess I should be hearing soon. Thanks again! Beginner User On Apache Jira Key: HIVE-7181 URL: https://issues.apache.org/jira/browse/HIVE-7181 Project: Hive Issue Type: Wish Reporter: Nishant Kelkar Priority: Minor Labels: documentation, newbie Hi All! I've just started to use Apache's Jira board (I registered today). I've used Jira for my work before, so I know how to navigate within Jira. But my main question, was understanding how issues are handled in the open source community (to which I want to contribute, but I'm a noob here too). So basically, a person comes up with a ticket when he/she thinks that the issue they are facing, is a bug/improvement. Questions: 1. Whom am I supposed to assign the ticket to? (myself?) 2. Who would be the QA assignee? 3. If addressing the issue requires looking at the code, how am I supposed to change the code and bring into effect those changes? (At work, we maintain a Git repo on our private server. So everyone always has access to the latest code). 4. Where can I find a list of all the people who are active on this project (Hive)? It would be nice if I could tag people by their names in my ticket comments. 5. Where can I find well formatted documentation about how to take issues from discovery to fixture on Apache Jira? I apologize in advance, if my questions are too simple. Thanks, and any/all help is appreciated! -- This message was sent by Atlassian JIRA (v6.2#6252)
