Re: Review Request 71835: HIVE-22508

[Krisztian Kasa]

> On Dec. 7, 2019, 3:14 a.m., Jesús Camacho Rodríguez wrote:
> > ql/src/test/results/clientpositive/hypothetical_set_aggregates.q.out
> > Line 133 (original), 133 (patched)
> > 
> >
> > Were results different here because NULL ordering? Can you elaborate?

When `NullOrdering.NULLS_LAST` was mapped to `NullValueOption.MINVALUE` rank(4) 
was 6 because there are 4 rows with nulls and one row with value 3 in the test 
table. This is the same as NULLS_FIRST setting should result. When I fixed this 
rank(4) become 2.

I also added a test case for testing null ordering:

POSTHOOK: query: select
rank(4) WITHIN GROUP (ORDER BY col1 nulls first),
rank(4) WITHIN GROUP (ORDER BY col1 nulls last),
rank(4) WITHIN GROUP (ORDER BY col1 desc),
rank(4) WITHIN GROUP (ORDER BY col1 desc nulls first),
rank(4) WITHIN GROUP (ORDER BY col1 desc nulls last)
from t_test
POSTHOOK: type: QUERY
POSTHOOK: Input: default@t_test
 A masked pattern was here 
6   2   17  13  17


- Krisztian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71835/#review218966
---


On Dec. 9, 2019, 12:12 p.m., Krisztian Kasa wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71835/
> ---
> 
> (Updated Dec. 9, 2019, 12:12 p.m.)
> 
> 
> Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich.
> 
> 
> Bugs: HIVE-22508
> https://issues.apache.org/jira/browse/HIVE-22508
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> KeyWrapperComparator throws exception
> =
> 1. Check the `KeyWrapper` instances whether they are a clones and use the 
> corresponding object inspectors.
> 2. `NullValueOption.MINVALUE` and `MAXVALUE` was switched in `NullOrdering` 
> enum and hypothetical set functions produced bad return values in case of 
> NULL inputs.
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java fee8081652 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperComparator.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java 
> f1bf9023e1 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java 4e35922e5f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBase.java
>  1bb224917e 
>   ql/src/java/org/apache/hadoop/hive/ql/util/NullOrdering.java 46ff329981 
>   ql/src/test/queries/clientpositive/hypothetical_set_aggregates.q 6b5f3765e9 
>   ql/src/test/results/clientpositive/hypothetical_set_aggregates.q.out 
> 3ea6f1f4e5 
>   
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
>  1f3bbaf87d 
>   
> serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorUtilsCompare.java
>  PRE-CREATION 
>   
> serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorUtilsCompareNull.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71835/diff/2/
> 
> 
> Testing
> ---
> 
> Run q tests using TestMiniLlapLocalCliDriver
> topnkey_order_null.q
> 
> TestCliDriver
> hypothetical_set_aggregates.q
> 
> 
> Thanks,
> 
> Krisztian Kasa
> 
>

Re: Review Request 71835: HIVE-22508

[Krisztian Kasa]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71835/
---

(Updated Dec. 9, 2019, 12:12 p.m.)


Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich.


Bugs: HIVE-22508
https://issues.apache.org/jira/browse/HIVE-22508


Repository: hive-git


Description
---

KeyWrapperComparator throws exception
=
1. Check the `KeyWrapper` instances whether they are a clones and use the 
corresponding object inspectors.
2. `NullValueOption.MINVALUE` and `MAXVALUE` was switched in `NullOrdering` 
enum and hypothetical set functions produced bad return values in case of NULL 
inputs.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java fee8081652 
  ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperComparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java f1bf9023e1 
  ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java 4e35922e5f 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBase.java
 1bb224917e 
  ql/src/java/org/apache/hadoop/hive/ql/util/NullOrdering.java 46ff329981 
  ql/src/test/queries/clientpositive/hypothetical_set_aggregates.q 6b5f3765e9 
  ql/src/test/results/clientpositive/hypothetical_set_aggregates.q.out 
3ea6f1f4e5 
  
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
 1f3bbaf87d 
  
serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorUtilsCompare.java
 PRE-CREATION 
  
serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorUtilsCompareNull.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/71835/diff/2/

Changes: https://reviews.apache.org/r/71835/diff/1-2/


Testing
---

Run q tests using TestMiniLlapLocalCliDriver
topnkey_order_null.q

TestCliDriver
hypothetical_set_aggregates.q


Thanks,

Krisztian Kasa

Re: Review Request 71835: HIVE-22508

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71835/#review218966
---




ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java
Lines 70 (patched)


Can you add comment with meaning for variable?



ql/src/test/results/clientpositive/hypothetical_set_aggregates.q.out
Line 133 (original), 133 (patched)


Were results different here because NULL ordering? Can you elaborate?


- Jesús Camacho Rodríguez


On Nov. 27, 2019, 2:24 p.m., Krisztian Kasa wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71835/
> ---
> 
> (Updated Nov. 27, 2019, 2:24 p.m.)
> 
> 
> Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich.
> 
> 
> Bugs: HIVE-22508
> https://issues.apache.org/jira/browse/HIVE-22508
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> KeyWrapperComparator throws exception
> =
> 1. Check the `KeyWrapper` instances whether they are a clones and use the 
> corresponding object inspectors.
> 2. `NullValueOption.MINVALUE` and `MAXVALUE` was switched in `NullOrdering` 
> enum and hypothetical set functions produced bad return values in case of 
> NULL inputs.
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java fee8081652 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperComparator.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java 
> f1bf9023e1 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java 4e35922e5f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBase.java
>  1bb224917e 
>   ql/src/java/org/apache/hadoop/hive/ql/util/NullOrdering.java 46ff329981 
>   ql/src/test/queries/clientpositive/hypothetical_set_aggregates.q 6b5f3765e9 
>   ql/src/test/results/clientpositive/hypothetical_set_aggregates.q.out 
> 3ea6f1f4e5 
>   
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
>  1f3bbaf87d 
>   
> serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorUtilsCompare.java
>  PRE-CREATION 
>   
> serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorUtilsCompareNull.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71835/diff/1/
> 
> 
> Testing
> ---
> 
> Run q tests using TestMiniLlapLocalCliDriver
> topnkey_order_null.q
> 
> TestCliDriver
> hypothetical_set_aggregates.q
> 
> 
> Thanks,
> 
> Krisztian Kasa
> 
>

Review Request 71835: HIVE-22508

[Krisztian Kasa]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71835/
---

Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich.


Bugs: HIVE-22508
https://issues.apache.org/jira/browse/HIVE-22508


Repository: hive-git


Description
---

KeyWrapperComparator throws exception
=
1. Check the `KeyWrapper` instances whether they are a clones and use the 
corresponding object inspectors.
2. `NullValueOption.MINVALUE` and `MAXVALUE` was switched in `NullOrdering` 
enum and hypothetical set functions produced bad return values in case of NULL 
inputs.


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java fee8081652 
  ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperComparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java f1bf9023e1 
  ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java 4e35922e5f 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/wrapper/VectorHashKeyWrapperBase.java
 1bb224917e 
  ql/src/java/org/apache/hadoop/hive/ql/util/NullOrdering.java 46ff329981 
  ql/src/test/queries/clientpositive/hypothetical_set_aggregates.q 6b5f3765e9 
  ql/src/test/results/clientpositive/hypothetical_set_aggregates.q.out 
3ea6f1f4e5 
  
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
 1f3bbaf87d 
  
serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorUtilsCompare.java
 PRE-CREATION 
  
serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorUtilsCompareNull.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/71835/diff/1/


Testing
---

Run q tests using TestMiniLlapLocalCliDriver
topnkey_order_null.q

TestCliDriver
hypothetical_set_aggregates.q


Thanks,

Krisztian Kasa