Re: Review Request 53845: 'like any' and 'like all' operators in hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---
(Updated April 22, 2017, 11:03 a.m.)
Review request for hive, Carl Steinbach and Vineet Garg.
Repository: hive-git
Description
---
https://issues.apache.org/jira/browse/HIVE-15229
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
Diffs (updated)
-
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 8dc5f2e
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 0721b92
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d98a663
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 8598fae
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out ac5ca41
ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/53845/diff/9/
Changes: https://reviews.apache.org/r/53845/diff/8-9/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 53845: 'like any' and 'like all' operators in hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---
(Updated March 2, 2017, 3:26 p.m.)
Review request for hive, Carl Steinbach and Vineet Garg.
Repository: hive-git
Description
---
https://issues.apache.org/jira/browse/HIVE-15229
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
Diffs (updated)
-
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java aaf2399
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g ad61f83
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 61778f6
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 81efadc
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java f979c14
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out 3c9bb4a
ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/53845/diff/8/
Changes: https://reviews.apache.org/r/53845/diff/7-8/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 53845: 'like any' and 'like all' operators in hive
> On March 2, 2017, 4:57 a.m., Carl Steinbach wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
> > Line 2 (original), 2 (patched)
> > <https://reviews.apache.org/r/53845/diff/7/?file=1634262#file1634262line2>
> >
> > Please make whitespace fixes in a separate patch. Including them here
> > pollutes the diff and draws the attention of reviewers away from the
> > important parts. I saw the same problem in several other files as well.
Added new patch after fixing whitespaces.
> On March 2, 2017, 4:57 a.m., Carl Steinbach wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
> > Lines 51 (patched)
> > <https://reviews.apache.org/r/53845/diff/7/?file=1634266#file1634266line51>
> >
> > Please change to "Returns NULL if the expression on the left hand side
> > is NULL or if one of the patterns in the list is NULL."
Added this message.
- Simanchal
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/#review167648
---
On Feb. 15, 2017, 6:47 a.m., Simanchal Das wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53845/
> ---
>
> (Updated Feb. 15, 2017, 6:47 a.m.)
>
>
> Review request for hive, Carl Steinbach and Vineet Garg.
>
>
> Repository: hive-git
>
>
> Description
> ---
>
> https://issues.apache.org/jira/browse/HIVE-15229
>
>
> In Teradata 'like any' and 'like all' operators are mostly used when we are
> matching a text field with numbers of patterns.
> 'like any' and 'like all' operator are equivalents of multiple like operator
> like example below.
> --like any
> select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
> '%retail%', '%bank%', '%insurance%');
>
> --Can be written using multiple like condition
> select col1 from table1 where col2 like '%accountant%' or col2 like
> '%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
> '%insurance%' ;
>
> --like all
> select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
> '%retail%', '%bank%', '%insurance%');
>
> --Can be written using multiple like operator
> select col1 from table1 where col2 like '%accountant%' and col2 like
> '%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
> '%insurance%' ;
>
> Problem statement:
>
> Now a days so many data warehouse projects are being migrated from Teradata
> to Hive.
> Always Data engineer and Business analyst are searching for these two
> operator.
> If we introduce these two operator in hive then so many scripts will be
> migrated smoothly instead of converting these operators to multiple like
> operators.
>
>
> Diffs
> -
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0f05160
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g f80642b
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g eb81393
> ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 81efadc
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
> f979c14
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
> PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
> PRE-CREATION
>
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
> PRE-CREATION
>
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
> PRE-CREATION
> ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
> ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
> ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
> ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
> ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
> ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/show_functions.q.out 3c9bb4a
> ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
> ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/53845/diff/7/
>
>
> Testing
> ---
>
> Junit test cases and query.q files are attached
>
>
> Thanks,
>
> Simanchal Das
>
>
Re: Review Request 53845: 'like any' and 'like all' operators in hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---
(Updated Feb. 15, 2017, 6:47 a.m.)
Review request for hive, Carl Steinbach and Vineet Garg.
Repository: hive-git
Description
---
https://issues.apache.org/jira/browse/HIVE-15229
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
Diffs (updated)
-
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0f05160
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g f80642b
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g eb81393
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 81efadc
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java f979c14
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out 3c9bb4a
ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/53845/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 53845: 'like any' and 'like all' operators in hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---
(Updated Feb. 14, 2017, 12:39 p.m.)
Review request for hive, Carl Steinbach and Vineet Garg.
Repository: hive-git
Description
---
https://issues.apache.org/jira/browse/HIVE-15229
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
Diffs (updated)
-
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0f05160
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g f80642b
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g eb81393
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 81efadc
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java f979c14
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out 3c9bb4a
ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/53845/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 53845: 'like any' and 'like all' operators in hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---
(Updated Feb. 14, 2017, 7:28 a.m.)
Review request for hive, Carl Steinbach and Vineet Garg.
Repository: hive-git
Description
---
https://issues.apache.org/jira/browse/HIVE-15229
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
Diffs (updated)
-
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0f05160
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g f80642b
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g eb81393
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 81efadc
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java f979c14
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/53845/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 53845: 'like any' and 'like all' operators in hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---
(Updated Feb. 9, 2017, 9:58 a.m.)
Review request for hive, Carl Steinbach and Vineet Garg.
Repository: hive-git
Description
---
https://issues.apache.org/jira/browse/HIVE-15229
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
Diffs (updated)
-
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0dbbc1d
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 4357328
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 55915a6
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g a82083b
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 5e708d3
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/53845/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 53845: 'like any' and 'like all' operators in hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---
(Updated Dec. 8, 2016, 12:31 p.m.)
Review request for hive, Carl Steinbach and Vineet Garg.
Repository: hive-git
Description
---
https://issues.apache.org/jira/browse/HIVE-15229
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
Diffs (updated)
-
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0dbbc1d
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 4357328
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 55915a6
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g a82083b
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 5e708d3
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_like.q.out 8ffcf9b
ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/53845/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 53845: 'like any' and 'like all' operators in hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---
(Updated Nov. 21, 2016, 12:29 p.m.)
Review request for hive and Carl Steinbach.
Repository: hive-git
Description
---
https://issues.apache.org/jira/browse/HIVE-15229
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
Diffs (updated)
-
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0dbbc1d
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 4357328
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 55915a6
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g a82083b
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 5e708d3
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_like.q.out 8ffcf9b
ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/53845/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 53845: 'like any' and 'like all' operators in hive
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---
(Updated Nov. 17, 2016, 12:49 p.m.)
Review request for hive and Carl Steinbach.
Repository: hive-git
Description
---
https://issues.apache.org/jira/browse/HIVE-15229
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
Diffs
-
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0dbbc1d
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 4357328
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 55915a6
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g a82083b
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 5e708d3
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION
ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION
ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION
ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/53845/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
[jira] [Created] (HIVE-15229) 'like any' and 'like all' operators in hive
Simanchal Das created HIVE-15229:
Summary: 'like any' and 'like all' operators in hive
Key: HIVE-15229
URL: https://issues.apache.org/jira/browse/HIVE-15229
Project: Hive
Issue Type: New Feature
Components: Operators
Reporter: Simanchal Das
Assignee: Simanchal Das
Priority: Minor
In Teradata 'like any' and 'like all' operators are mostly used when we are
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator
like example below.
{noformat}
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like condition
select col1 from table1 where col2 like '%accountant%' or col2 like
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like
'%insurance%' ;
--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%',
'%retail%', '%bank%', '%insurance%');
--Can be written using multiple like operator
select col1 from table1 where col2 like '%accountant%' and col2 like
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like
'%insurance%' ;
{noformat}
Problem statement:
Now a days so many data warehouse projects are being migrated from Teradata to
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be
migrated smoothly instead of converting these operators to multiple like
operators.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 49619: sorting of tuple array using multiple fields
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49619/
---
(Updated July 11, 2016, 1:32 p.m.)
Review request for hive, Ashutosh Chauhan and Carl Steinbach.
Repository: hive-git
Description (updated)
---
https://issues.apache.org/jira/browse/HIVE-14159
Problem Statement:
When we are working with complex structure of data like avro.
Most of the times we are encountering array contains multiple tuples and each
tuple have struct schema.
Suppose here struct schema is like below:
{
"name": "employee",
"type": [{
"type": "record",
"name": "Employee",
"namespace": "com.company.Employee",
"fields": [{
"name": "empId",
"type": "int"
}, {
"name": "empName",
"type": "string"
}, {
"name": "age",
"type": "int"
}, {
"name": "salary",
"type": "double"
}]
}]
}
Then while running our hive query complex array looks like array of employee
objects.
Example:
//(array<struct<empId,empName,age,salary>>)
Array[Employee(100,Foo,20,20990),Employee(500,Boo,30,50990),Employee(700,Harry,25,40990),Employee(100,Tom,35,70990)]
When we are implementing business use cases day to day life we are encountering
problems like sorting a tuple array by specific field[s] like
empId,name,salary,etc by ASC or DESC order.
Proposal:
I have developed a udf 'sort_array_by' which will sort a tuple array by one or
more fields in ASC or DESC order provided by user ,default is ascending order .
Example:
1.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Salary","ASC");
output:
array[struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(500,Boo,30,50990),struct(100,Tom,35,70990)]
2.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,80990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","ASC");
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
3.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","Age,"ASC");
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
Diffs
-
itests/src/test/resources/testconfiguration.properties 1ab914d
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 2f4a94c
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArrayByField.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFSortArrayByField.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_by_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_by_wrong2.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_by_wrong3.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_sort_array_by.q PRE-CREATION
ql/src/test/results/beelinepositive/show_functions.q.out 4f3ec40
ql/src/test/results/clientnegative/udf_sort_array_by_wrong1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_sort_array_by_wrong2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_sort_array_by_wrong3.q.out
PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out a811747
ql/src/test/results/clientpositive/udf_sort_array_by.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/49619/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 49619: sorting of tuple array using multiple fields
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49619/
---
(Updated July 11, 2016, 8:37 a.m.)
Review request for hive, Ashutosh Chauhan and Carl Steinbach.
Repository: hive-git
Description
---
Problem Statement:
When we are working with complex structure of data like avro.
Most of the times we are encountering array contains multiple tuples and each
tuple have struct schema.
Suppose here struct schema is like below:
{
"name": "employee",
"type": [{
"type": "record",
"name": "Employee",
"namespace": "com.company.Employee",
"fields": [{
"name": "empId",
"type": "int"
}, {
"name": "empName",
"type": "string"
}, {
"name": "age",
"type": "int"
}, {
"name": "salary",
"type": "double"
}]
}]
}
Then while running our hive query complex array looks like array of employee
objects.
Example:
//(array<struct<empId,empName,age,salary>>)
Array[Employee(100,Foo,20,20990),Employee(500,Boo,30,50990),Employee(700,Harry,25,40990),Employee(100,Tom,35,70990)]
When we are implementing business use cases day to day life we are encountering
problems like sorting a tuple array by specific field[s] like
empId,name,salary,etc by ASC or DESC order.
Proposal:
I have developed a udf 'sort_array_by' which will sort a tuple array by one or
more fields in ASC or DESC order provided by user ,default is ascending order .
Example:
1.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Salary","ASC");
output:
array[struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(500,Boo,30,50990),struct(100,Tom,35,70990)]
2.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,80990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","ASC");
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
3.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","Age,"ASC");
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
Diffs (updated)
-
itests/src/test/resources/testconfiguration.properties 1ab914d
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 2f4a94c
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArrayByField.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFSortArrayByField.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_by_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_by_wrong2.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_by_wrong3.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_sort_array_by.q PRE-CREATION
ql/src/test/results/beelinepositive/show_functions.q.out 4f3ec40
ql/src/test/results/clientnegative/udf_sort_array_by_wrong1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_sort_array_by_wrong2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_sort_array_by_wrong3.q.out
PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out a811747
ql/src/test/results/clientpositive/udf_sort_array_by.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/49619/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 49619: sorting of tuple array using multiple fields
> On July 9, 2016, 8:53 p.m., Carl Steinbach wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArrayByField.java,
> > line 56
> > <https://reviews.apache.org/r/49619/diff/3/?file=1439691#file1439691line56>
> >
> > A couple notes:
> >
> > 1. I think the example should actually return a value that is different
> > than the input. It would also be good to include more than two elements in
> > the input. If screen space is an issue I recommend only including a single
> > element in each of the structs in the example, which I think has the added
> > benefit of making the example clearer by not distracting the reader with
> > irrelevant details.
> >
> > 2. It looks like the default sorting order (ASC) is actually the
> > reverse of what I would expect it to be, i.e. I expect 'b' to come before
> > 'g'.
> >
> > 3. Related to point (2), I think it's important to ensure that the
> > sorting order of this UDF is consisent with ORDER BY, e.g. for a table t
> > containing a single row with a single array of struct field a_struct_array,
> > the queries "SELECT a_struct FROM t LATERAL VIEW explode(a_struct_array)
> > structTable AS a_struct ORDER BY a_struct.col1 DESC" should return the same
> > results as "SELECT a_struct FROM t LATERAL VIEW
> > explode(sort_array_by(a_struct_array, 'col1', 'DESC')) structTable AS
> > a_struct". Note that I probably didn't get the syntax for LATERAL VIEW and
> > explode() correct.
1. Sorry there was a typo.
2. corrected the sorting order in example.
3. As per your instruction I have added test example of LATERAL VIEW
explode(array) and explode(udf). Which gives same results.
> On July 9, 2016, 8:53 p.m., Carl Steinbach wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArrayByField.java,
> > line 93
> > <https://reviews.apache.org/r/49619/diff/3/?file=1439691#file1439691line93>
> >
> > Unnecessary string concatenation operators.
removed
> On July 9, 2016, 8:53 p.m., Carl Steinbach wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArrayByField.java,
> > line 104
> > <https://reviews.apache.org/r/49619/diff/3/?file=1439691#file1439691line104>
> >
> > Unnecessary "+" operator.
removed
> On July 9, 2016, 8:53 p.m., Carl Steinbach wrote:
> > ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFSortArrayByField.java,
> > line 1
> > <https://reviews.apache.org/r/49619/diff/3/?file=1439692#file1439692line1>
> >
> > Does this unit test provide any additional coverage or advantages over
> > the q file tests? Is it necessary to have both?
> >
> > Note that I am a strong advocate of end-to-end qfile tests over unit
> > tests, which is an opinion that not everyone holds.
These are kind of same as q file. I feels test cases on Test classes are good
for doing unit testing while development and takes less time to identify
problem comparare to q files.
Any ways I have removed some test cases from Test class.
- Simanchal
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49619/#review141413
---
On July 8, 2016, 12:35 p.m., Simanchal Das wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49619/
> ---
>
> (Updated July 8, 2016, 12:35 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Carl Steinbach.
>
>
> Repository: hive-git
>
>
> Description
> ---
>
> Problem Statement:
>
> When we are working with complex structure of data like avro.
> Most of the times we are encountering array contains multiple tuples and each
> tuple have struct schema.
> Suppose here struct schema is like below:
> {
> "name": "employee",
> "type": [{
> "type": "record",
> "name": "Employee",
> "namespace": "com.company.Employee",
> "fields": [{
> "name": "empId",
> "type": "int"
> }, {
> "name": "empName",
> "type": "string"
>
Re: Review Request 49619: sorting of tuple array using multiple fields
> On July 7, 2016, 6:45 a.m., Carl Steinbach wrote:
> > ql/src/test/results/beelinepositive/show_functions.q.out, line 183
> > <https://reviews.apache.org/r/49619/diff/2/?file=1438314#file1438314line183>
> >
> > The number of rows is off by 8. This looks like a bug, thought not one
> > caused by this patch.
re run the test
> On July 7, 2016, 6:45 a.m., Carl Steinbach wrote:
> > ql/src/test/results/beelinepositive/show_functions.q.out, line 184
> > <https://reviews.apache.org/r/49619/diff/2/?file=1438314#file1438314line184>
> >
> > It looks like you're stripping whitespace out of the patch. I suspect
> > this is the cause of the failure in show_functions.q
re run the test
- Simanchal
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49619/#review141130
-------
On July 8, 2016, 12:35 p.m., Simanchal Das wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49619/
> ---
>
> (Updated July 8, 2016, 12:35 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Carl Steinbach.
>
>
> Repository: hive-git
>
>
> Description
> ---
>
> Problem Statement:
>
> When we are working with complex structure of data like avro.
> Most of the times we are encountering array contains multiple tuples and each
> tuple have struct schema.
> Suppose here struct schema is like below:
> {
> "name": "employee",
> "type": [{
> "type": "record",
> "name": "Employee",
> "namespace": "com.company.Employee",
> "fields": [{
> "name": "empId",
> "type": "int"
> }, {
> "name": "empName",
> "type": "string"
> }, {
> "name": "age",
> "type": "int"
> }, {
> "name": "salary",
> "type": "double"
> }]
> }]
> }
>
> Then while running our hive query complex array looks like array of employee
> objects.
> Example:
> //(array<struct<empId,empName,age,salary>>)
>
> Array[Employee(100,Foo,20,20990),Employee(500,Boo,30,50990),Employee(700,Harry,25,40990),Employee(100,Tom,35,70990)]
>
> When we are implementing business use cases day to day life we are
> encountering problems like sorting a tuple array by specific field[s] like
> empId,name,salary,etc by ASC or DESC order.
> Proposal:
> I have developed a udf 'sort_array_by' which will sort a tuple array by one
> or more fields in ASC or DESC order provided by user ,default is ascending
> order .
> Example:
> 1.Select
> sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Salary","ASC");
> output:
> array[struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(500,Boo,30,50990),struct(100,Tom,35,70990)]
>
> 2.Select
> sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,80990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","ASC");
> output:
> array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
>
> 3.Select
> sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","Age,"ASC");
> output:
> array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
>
>
> Diffs
> -
>
> itests/src/test/resources/testconfiguration.properties 1ab914d
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 2f4a94c
>
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArrayByField.java
> PRE-CREATION
>
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFSortArrayByField.java
> PRE-CREATION
>
Re: Review Request 49619: sorting of tuple array using multiple fields
> On July 7, 2016, 6:45 a.m., Carl Steinbach wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java, line 427
> > <https://reviews.apache.org/r/49619/diff/2/?file=1438308#file1438308line427>
> >
> > To me "sort_array_field" makes it sound like this function sorts the
> > elements in an array field, as opposed to sorting an array on a particular
> > field, which is what is actually does. I think the purpose of this function
> > would be clearer if the name were changed 'sort_array_on_field' or
> > 'sort_array_by' (I prefer the latter).
fixed
> On July 7, 2016, 6:45 a.m., Carl Steinbach wrote:
> > ql/src/test/queries/clientpositive/udf_sort_array_field.q, line 1
> > <https://reviews.apache.org/r/49619/diff/2/?file=1438313#file1438313line1>
> >
> > Is this really necessary?
removed
> On July 7, 2016, 6:45 a.m., Carl Steinbach wrote:
> > ql/src/test/queries/clientpositive/udf_sort_array_field.q, line 9
> > <https://reviews.apache.org/r/49619/diff/2/?file=1438313#file1438313line9>
> >
> > No need for this. Please remove.
removed
> On July 7, 2016, 6:45 a.m., Carl Steinbach wrote:
> > ql/src/test/queries/clientpositive/udf_sort_array_field.q, line 16
> > <https://reviews.apache.org/r/49619/diff/2/?file=1438313#file1438313line16>
> >
> > The rows should have different struct values.
chnaged the values
> On July 7, 2016, 6:45 a.m., Carl Steinbach wrote:
> > ql/src/test/queries/clientpositive/udf_sort_array_field.q, line 25
> > <https://reviews.apache.org/r/49619/diff/2/?file=1438313#file1438313line25>
> >
> > Consider using named_struct() instead of struct(). This will allow you
> > to provide names for the struct fields.
Used named_struct()
- Simanchal
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49619/#review141130
---
On July 8, 2016, 12:35 p.m., Simanchal Das wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49619/
> ---
>
> (Updated July 8, 2016, 12:35 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Carl Steinbach.
>
>
> Repository: hive-git
>
>
> Description
> ---
>
> Problem Statement:
>
> When we are working with complex structure of data like avro.
> Most of the times we are encountering array contains multiple tuples and each
> tuple have struct schema.
> Suppose here struct schema is like below:
> {
> "name": "employee",
> "type": [{
> "type": "record",
> "name": "Employee",
> "namespace": "com.company.Employee",
> "fields": [{
> "name": "empId",
> "type": "int"
> }, {
> "name": "empName",
> "type": "string"
> }, {
> "name": "age",
> "type": "int"
> }, {
> "name": "salary",
> "type": "double"
> }]
> }]
> }
>
> Then while running our hive query complex array looks like array of employee
> objects.
> Example:
> //(array<struct<empId,empName,age,salary>>)
>
> Array[Employee(100,Foo,20,20990),Employee(500,Boo,30,50990),Employee(700,Harry,25,40990),Employee(100,Tom,35,70990)]
>
> When we are implementing business use cases day to day life we are
> encountering problems like sorting a tuple array by specific field[s] like
> empId,name,salary,etc by ASC or DESC order.
> Proposal:
> I have developed a udf 'sort_array_by' which will sort a tuple array by one
> or more fields in ASC or DESC order provided by user ,default is ascending
> order .
> Example:
> 1.Select
> sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Salary","ASC");
> output:
> array[struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(500,Boo,30,50990),struct(100,Tom,35,70990)]
>
> 2.Select
> sort_array_f
Re: Review Request 49619: sorting of tuple array using multiple fields
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49619/
---
(Updated July 8, 2016, 12:35 p.m.)
Review request for hive, Ashutosh Chauhan and Carl Steinbach.
Changes
---
renamed the udf to sort_array_by and fixed all review the comments
Repository: hive-git
Description (updated)
---
Problem Statement:
When we are working with complex structure of data like avro.
Most of the times we are encountering array contains multiple tuples and each
tuple have struct schema.
Suppose here struct schema is like below:
{
"name": "employee",
"type": [{
"type": "record",
"name": "Employee",
"namespace": "com.company.Employee",
"fields": [{
"name": "empId",
"type": "int"
}, {
"name": "empName",
"type": "string"
}, {
"name": "age",
"type": "int"
}, {
"name": "salary",
"type": "double"
}]
}]
}
Then while running our hive query complex array looks like array of employee
objects.
Example:
//(array<struct<empId,empName,age,salary>>)
Array[Employee(100,Foo,20,20990),Employee(500,Boo,30,50990),Employee(700,Harry,25,40990),Employee(100,Tom,35,70990)]
When we are implementing business use cases day to day life we are encountering
problems like sorting a tuple array by specific field[s] like
empId,name,salary,etc by ASC or DESC order.
Proposal:
I have developed a udf 'sort_array_by' which will sort a tuple array by one or
more fields in ASC or DESC order provided by user ,default is ascending order .
Example:
1.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Salary","ASC");
output:
array[struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(500,Boo,30,50990),struct(100,Tom,35,70990)]
2.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,80990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","ASC");
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
3.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","Age,"ASC");
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
Diffs (updated)
-
itests/src/test/resources/testconfiguration.properties 1ab914d
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 2f4a94c
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArrayByField.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFSortArrayByField.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_by_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_by_wrong2.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_by_wrong3.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_sort_array_by.q PRE-CREATION
ql/src/test/results/beelinepositive/show_functions.q.out 4f3ec40
ql/src/test/results/clientnegative/udf_sort_array_by_wrong1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_sort_array_by_wrong2.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_sort_array_by_wrong3.q.out
PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out a811747
ql/src/test/results/clientpositive/udf_sort_array_by.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/49619/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Re: Review Request 49619: sorting of tuple array using multiple fields
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49619/
---
(Updated July 7, 2016, 5:03 a.m.)
Review request for hive and Carl Steinbach.
Changes
---
added udf name in show function q.out file
Repository: hive-git
Description
---
Problem Statement:
When we are working with complext structure of data like avro.
Most of the times we are encountering array contains multiple tuples and each
tuple have struct schema.
Suppose here struct schema is like below:
{
"name": "employee",
"type": [{
"type": "record",
"name": "Employee",
"namespace": "com.company.Employee",
"fields": [{
"name": "empId",
"type": "int"
}, {
"name": "empName",
"type": "string"
}, {
"name": "age",
"type": "int"
}, {
"name": "salary",
"type": "double"
}]
}]
}
Then while running our hive query complex array looks like array of employee
objects.
Example:
//(array<struct<empId,empName,age,salary>>)
Array[Employee(100,Foo,20,20990),Employee(500,Boo,30,50990),Employee(700,Harry,25,40990),Employee(100,Tom,35,70990)]
When we are implementing business use cases day to day life we are encountering
problems like sorting a tuple array by specific field[s] like empIdm,salary,etc.
Proposal:
I have developed a udf 'sort_array_field' which will sort a tuple array by one
or more fields in naural order.
Example:
1.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Salary");
output:
array[struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(500,Boo,30,50990),struct(100,Tom,35,70990)]
2.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,80990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary");
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
3.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","Age);
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
Diffs (updated)
-
itests/src/test/resources/testconfiguration.properties 1ab914d
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 2f4a94c
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArrayField.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFSortArrayField.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_field_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_field_wrong2.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_sort_array_field.q PRE-CREATION
ql/src/test/results/beelinepositive/show_functions.q.out 4f3ec40
ql/src/test/results/clientnegative/udf_sort_array_field_wrong1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_sort_array_field_wrong2.q.out
PRE-CREATION
ql/src/test/results/clientpositive/udf_sort_array_field.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/49619/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
Review Request 49619: sorting of tuple array using multiple fields
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49619/
---
Review request for hive and Carl Steinbach.
Repository: hive-git
Description
---
Problem Statement:
When we are working with complext structure of data like avro.
Most of the times we are encountering array contains multiple tuples and each
tuple have struct schema.
Suppose here struct schema is like below:
{
"name": "employee",
"type": [{
"type": "record",
"name": "Employee",
"namespace": "com.company.Employee",
"fields": [{
"name": "empId",
"type": "int"
}, {
"name": "empName",
"type": "string"
}, {
"name": "age",
"type": "int"
}, {
"name": "salary",
"type": "double"
}]
}]
}
Then while running our hive query complex array looks like array of employee
objects.
Example:
//(array<struct<empId,empName,age,salary>>)
Array[Employee(100,Foo,20,20990),Employee(500,Boo,30,50990),Employee(700,Harry,25,40990),Employee(100,Tom,35,70990)]
When we are implementing business use cases day to day life we are encountering
problems like sorting a tuple array by specific field[s] like empIdm,salary,etc.
Proposal:
I have developed a udf 'sort_array_field' which will sort a tuple array by one
or more fields in naural order.
Example:
1.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Salary");
output:
array[struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(500,Boo,30,50990),struct(100,Tom,35,70990)]
2.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,80990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary");
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
3.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","Age);
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
Diffs
-
itests/src/test/resources/testconfiguration.properties 1ab914d
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 2f4a94c
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArrayField.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFSortArrayField.java
PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_field_wrong1.q PRE-CREATION
ql/src/test/queries/clientnegative/udf_sort_array_field_wrong2.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_sort_array_field.q PRE-CREATION
ql/src/test/results/clientnegative/udf_sort_array_field_wrong1.q.out
PRE-CREATION
ql/src/test/results/clientnegative/udf_sort_array_field_wrong2.q.out
PRE-CREATION
ql/src/test/results/clientpositive/udf_sort_array_field.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/49619/diff/
Testing
---
Junit test cases and query.q files are attached
Thanks,
Simanchal Das
[jira] [Created] (HIVE-14159) sorting of tuple array using multiple field[s]
Simanchal Das created HIVE-14159:
Summary: sorting of tuple array using multiple field[s]
Key: HIVE-14159
URL: https://issues.apache.org/jira/browse/HIVE-14159
Project: Hive
Issue Type: Improvement
Components: UDF
Reporter: Simanchal Das
Assignee: Simanchal Das
Problem Statement:
When we are working with complex structure of data like avro.
Most of the times we are encountering array contains multiple tuples and each
tuple have struct schema.
Suppose here struct schema is like below:
{noformat}
{
"name": "employee",
"type": [{
"type": "record",
"name": "Employee",
"namespace": "com.company.Employee",
"fields": [{
"name": "empId",
"type": "int"
}, {
"name": "empName",
"type": "string"
}, {
"name": "age",
"type": "int"
}, {
"name": "salary",
"type": "double"
}]
}]
}
{noformat}
Then while running our hive query complex array looks like array of employee
objects.
{noformat}
Example:
//(array<struct<empId,empName,age,salary>>)
Array[Employee(100,Foo,20,20990),Employee(500,Boo,30,50990),Employee(700,Harry,25,40990),Employee(100,Tom,35,70990)]
{noformat}
When we are implementing business use cases day to day life we are encountering
problems like sorting a tuple array by specific field[s] like
empId,name,salary,etc.
Proposal:
I have developed a udf 'sort_array_field' which will sort a tuple array by one
or more fields in naural order.
{noformat}
Example:
1.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Salary");
output:
array[struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(500,Boo,30,50990),struct(100,Tom,35,70990)]
2.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,80990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary");
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
3.Select
sort_array_field(array[struct(100,Foo,20,20990),struct(500,Boo,30,50990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)],"Name","Salary","Age);
output:
array[struct(500,Boo,30,50990),struct(500,Boo,30,80990),struct(100,Foo,20,20990),struct(700,Harry,25,40990),struct(100,Tom,35,70990)]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
