[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=539817=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-539817
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 22/Jan/21 06:20
Start Date: 22/Jan/21 06:20
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #1896:
URL: https://github.com/apache/hive/pull/1896#discussion_r562053125
##
File path: ql/src/test/queries/clientnegative/compute_stats_long.q
##
@@ -3,5 +3,5 @@ create table tab_int(a int);
-- insert some data
LOAD DATA LOCAL INPATH "../../data/files/int.txt" INTO TABLE tab_int;
--- compute stats should raise an error since the number of bit vectors > 1024
-select compute_stats(a, 1) from tab_int;
+-- this should raise an error since the number of bit vectors has a hard limit
at 1024
+select compute_stats(a, 'fm', 1) from tab_int;
Review comment:
does this limit apply only to fm algo?
##
File path: ql/src/test/queries/clientnegative/compute_stats_long.q
##
@@ -3,5 +3,5 @@ create table tab_int(a int);
-- insert some data
LOAD DATA LOCAL INPATH "../../data/files/int.txt" INTO TABLE tab_int;
--- compute stats should raise an error since the number of bit vectors > 1024
-select compute_stats(a, 1) from tab_int;
+-- this should raise an error since the number of bit vectors has a hard limit
at 1024
+select compute_stats(a, 'fm', 1) from tab_int;
Review comment:
thanks
##
File path: ql/src/test/queries/clientnegative/compute_stats_long.q
##
@@ -3,5 +3,5 @@ create table tab_int(a int);
-- insert some data
LOAD DATA LOCAL INPATH "../../data/files/int.txt" INTO TABLE tab_int;
--- compute stats should raise an error since the number of bit vectors > 1024
-select compute_stats(a, 1) from tab_int;
+-- this should raise an error since the number of bit vectors has a hard limit
at 1024
+select compute_stats(a, 'fm', 1) from tab_int;
Review comment:
+1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 539817)
Time Spent: 1h 40m (was: 1.5h)
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=539779=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-539779
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 22/Jan/21 06:14
Start Date: 22/Jan/21 06:14
Worklog Time Spent: 10m
Work Description: mustafaiman closed pull request #1896:
URL: https://github.com/apache/hive/pull/1896
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 539779)
Time Spent: 1.5h (was: 1h 20m)
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=539708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-539708
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 22/Jan/21 06:04
Start Date: 22/Jan/21 06:04
Worklog Time Spent: 10m
Work Description: mustafaiman opened a new pull request #1896:
URL: https://github.com/apache/hive/pull/1896
Change-Id: I61d941dcbf86fb2dd45772fc658b3dc887325bd0
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 539708)
Time Spent: 1h 20m (was: 1h 10m)
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=539668=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-539668
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 22/Jan/21 05:58
Start Date: 22/Jan/21 05:58
Worklog Time Spent: 10m
Work Description: mustafaiman commented on a change in pull request #1896:
URL: https://github.com/apache/hive/pull/1896#discussion_r562065002
##
File path: ql/src/test/queries/clientnegative/compute_stats_long.q
##
@@ -3,5 +3,5 @@ create table tab_int(a int);
-- insert some data
LOAD DATA LOCAL INPATH "../../data/files/int.txt" INTO TABLE tab_int;
--- compute stats should raise an error since the number of bit vectors > 1024
-select compute_stats(a, 1) from tab_int;
+-- this should raise an error since the number of bit vectors has a hard limit
at 1024
+select compute_stats(a, 'fm', 1) from tab_int;
Review comment:
yes see
https://github.com/apache/hive/blob/5d735c20ffb6a8624382155a3e90e650db2fb80e/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/common/ndv/NumDistinctValueEstimatorFactory.java#L64
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 539668)
Time Spent: 1h 10m (was: 1h)
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=539459=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-539459
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 22/Jan/21 04:31
Start Date: 22/Jan/21 04:31
Worklog Time Spent: 10m
Work Description: mustafaiman closed pull request #1896:
URL: https://github.com/apache/hive/pull/1896
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 539459)
Time Spent: 1h (was: 50m)
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=539216=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-539216
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 21/Jan/21 17:30
Start Date: 21/Jan/21 17:30
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #1896:
URL: https://github.com/apache/hive/pull/1896#discussion_r562066508
##
File path: ql/src/test/queries/clientnegative/compute_stats_long.q
##
@@ -3,5 +3,5 @@ create table tab_int(a int);
-- insert some data
LOAD DATA LOCAL INPATH "../../data/files/int.txt" INTO TABLE tab_int;
--- compute stats should raise an error since the number of bit vectors > 1024
-select compute_stats(a, 1) from tab_int;
+-- this should raise an error since the number of bit vectors has a hard limit
at 1024
+select compute_stats(a, 'fm', 1) from tab_int;
Review comment:
+1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 539216)
Time Spent: 50m (was: 40m)
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=539214=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-539214
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 21/Jan/21 17:29
Start Date: 21/Jan/21 17:29
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #1896:
URL: https://github.com/apache/hive/pull/1896#discussion_r562065992
##
File path: ql/src/test/queries/clientnegative/compute_stats_long.q
##
@@ -3,5 +3,5 @@ create table tab_int(a int);
-- insert some data
LOAD DATA LOCAL INPATH "../../data/files/int.txt" INTO TABLE tab_int;
--- compute stats should raise an error since the number of bit vectors > 1024
-select compute_stats(a, 1) from tab_int;
+-- this should raise an error since the number of bit vectors has a hard limit
at 1024
+select compute_stats(a, 'fm', 1) from tab_int;
Review comment:
thanks
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 539214)
Time Spent: 40m (was: 0.5h)
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=539212=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-539212
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 21/Jan/21 17:28
Start Date: 21/Jan/21 17:28
Worklog Time Spent: 10m
Work Description: mustafaiman commented on a change in pull request #1896:
URL: https://github.com/apache/hive/pull/1896#discussion_r562065002
##
File path: ql/src/test/queries/clientnegative/compute_stats_long.q
##
@@ -3,5 +3,5 @@ create table tab_int(a int);
-- insert some data
LOAD DATA LOCAL INPATH "../../data/files/int.txt" INTO TABLE tab_int;
--- compute stats should raise an error since the number of bit vectors > 1024
-select compute_stats(a, 1) from tab_int;
+-- this should raise an error since the number of bit vectors has a hard limit
at 1024
+select compute_stats(a, 'fm', 1) from tab_int;
Review comment:
yes see
https://github.com/apache/hive/blob/5d735c20ffb6a8624382155a3e90e650db2fb80e/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/common/ndv/NumDistinctValueEstimatorFactory.java#L64
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 539212)
Time Spent: 0.5h (was: 20m)
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=539197=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-539197
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 21/Jan/21 17:13
Start Date: 21/Jan/21 17:13
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #1896:
URL: https://github.com/apache/hive/pull/1896#discussion_r562053125
##
File path: ql/src/test/queries/clientnegative/compute_stats_long.q
##
@@ -3,5 +3,5 @@ create table tab_int(a int);
-- insert some data
LOAD DATA LOCAL INPATH "../../data/files/int.txt" INTO TABLE tab_int;
--- compute stats should raise an error since the number of bit vectors > 1024
-select compute_stats(a, 1) from tab_int;
+-- this should raise an error since the number of bit vectors has a hard limit
at 1024
+select compute_stats(a, 'fm', 1) from tab_int;
Review comment:
does this limit apply only to fm algo?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 539197)
Time Spent: 20m (was: 10m)
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24672) compute_stats_long.q fails for wrong reasons
[
https://issues.apache.org/jira/browse/HIVE-24672?focusedWorklogId=538758=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-538758
]
ASF GitHub Bot logged work on HIVE-24672:
-
Author: ASF GitHub Bot
Created on: 20/Jan/21 23:13
Start Date: 20/Jan/21 23:13
Worklog Time Spent: 10m
Work Description: mustafaiman opened a new pull request #1896:
URL: https://github.com/apache/hive/pull/1896
Change-Id: I61d941dcbf86fb2dd45772fc658b3dc887325bd0
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 538758)
Remaining Estimate: 0h
Time Spent: 10m
> compute_stats_long.q fails for wrong reasons
>
>
> Key: HIVE-24672
> URL: https://issues.apache.org/jira/browse/HIVE-24672
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa İman
>Assignee: Mustafa İman
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TestNegativeCliDriver[compute_stats_long] intends to test fmsketch has a hard
> limit on number of bit vectors (1024). However, the test fails for the
> following wrong reason.
> {code:java}
> Caused by: java.lang.RuntimeException: Can not recognize 1Caused by:
> java.lang.RuntimeException: Can not recognize 1 at
> org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getEmptyNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:71)
> {code}
> Instead it should fail with
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: The maximum
> allowed value for number of bit vectors is 1024, but was passed 1 bit
> vectorsCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: The
> maximum allowed value for number of bit vectors is 1024, but was passed
> 1 bit vectors at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeBitVectorFMSketch$NumericStatsEvaluator.iterate(GenericUDAFComputeBitVectorFMSketch.java:125)
> ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> {code}
> Since this function is superseeded by compute_bit_vector_fm, it is best if we
> add the same test for compute_bit_vector_fm too.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
