[jira] [Commented] (HIVE-13422) Analyse command not working for column having datatype as decimal(38,0)
[
https://issues.apache.org/jira/browse/HIVE-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392426#comment-15392426
]
Thomas Friedrich commented on HIVE-13422:
-
Failed tests not related to fix.
> Analyse command not working for column having datatype as decimal(38,0)
> ---
>
> Key: HIVE-13422
> URL: https://issues.apache.org/jira/browse/HIVE-13422
> Project: Hive
> Issue Type: Bug
> Components: Hive, Statistics
>Affects Versions: 1.1.0
>Reporter: ashim sinha
>Assignee: Thomas Friedrich
> Attachments: HIVE-13422.patch
>
>
> For the repro
> {code}
> drop table sample_test;
> CREATE TABLE IF NOT EXISTS sample_test( key decimal(38,0),b int ) ROW FORMAT
> DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
> load data local inpath '/home/hive/analyse.txt' into table sample_test;
> ANALYZE TABLE sample_test COMPUTE STATISTICS FOR COLUMNS;
> {code}
> Sample data
> {code}
> 2023456789456749825082498304 0
> 5032080754887849825069508304 0
> 4012080754887849825068718304 0
> 2012080754887849825066778304 0
> 4012080754887849625065678304 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13422) Analyse command not working for column having datatype as decimal(38,0)
[
https://issues.apache.org/jira/browse/HIVE-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391150#comment-15391150
]
Ashutosh Chauhan commented on HIVE-13422:
-
+1
> Analyse command not working for column having datatype as decimal(38,0)
> ---
>
> Key: HIVE-13422
> URL: https://issues.apache.org/jira/browse/HIVE-13422
> Project: Hive
> Issue Type: Bug
> Components: Hive, Statistics
>Affects Versions: 1.1.0
>Reporter: ashim sinha
>Assignee: Thomas Friedrich
> Attachments: HIVE-13422.patch
>
>
> For the repro
> {code}
> drop table sample_test;
> CREATE TABLE IF NOT EXISTS sample_test( key decimal(38,0),b int ) ROW FORMAT
> DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
> load data local inpath '/home/hive/analyse.txt' into table sample_test;
> ANALYZE TABLE sample_test COMPUTE STATISTICS FOR COLUMNS;
> {code}
> Sample data
> {code}
> 2023456789456749825082498304 0
> 5032080754887849825069508304 0
> 4012080754887849825068718304 0
> 2012080754887849825066778304 0
> 4012080754887849625065678304 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13422) Analyse command not working for column having datatype as decimal(38,0)
[
https://issues.apache.org/jira/browse/HIVE-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391129#comment-15391129
]
Hive QA commented on HIVE-13422:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12819735/HIVE-13422.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10350 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testCheckPermissions
org.apache.hadoop.hive.llap.daemon.impl.TestLlapTokenChecker.testGetToken
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testConnections
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/625/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/625/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-625/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12819735 - PreCommit-HIVE-MASTER-Build
> Analyse command not working for column having datatype as decimal(38,0)
> ---
>
> Key: HIVE-13422
> URL: https://issues.apache.org/jira/browse/HIVE-13422
> Project: Hive
> Issue Type: Bug
> Components: Hive, Statistics
>Affects Versions: 1.1.0
>Reporter: ashim sinha
>Assignee: Thomas Friedrich
> Attachments: HIVE-13422.patch
>
>
> For the repro
> {code}
> drop table sample_test;
> CREATE TABLE IF NOT EXISTS sample_test( key decimal(38,0),b int ) ROW FORMAT
> DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
> load data local inpath '/home/hive/analyse.txt' into table sample_test;
> ANALYZE TABLE sample_test COMPUTE STATISTICS FOR COLUMNS;
> {code}
> Sample data
> {code}
> 2023456789456749825082498304 0
> 5032080754887849825069508304 0
> 4012080754887849825068718304 0
> 2012080754887849825066778304 0
> 4012080754887849625065678304 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13422) Analyse command not working for column having datatype as decimal(38,0)
[
https://issues.apache.org/jira/browse/HIVE-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390323#comment-15390323
]
Thomas Friedrich commented on HIVE-13422:
-
The problem is that the GenericUDAFDecimalStatsEvaluator is using the default
HiveDecimalObjectInspector which is initialized with the default precision and
scale of 38,18. While the analyze will work for some decimal columns, it fails
for decimal columns where the difference of precision and scale is larger than
20 (38-18), for example decimal(21,0).
I attached a patch that will initialize the HiveDecimalObjectInspector with the
actual precision and scale of the decimal column. I also updated the
compute_stats_decimal.q test case to test this scenario.
> Analyse command not working for column having datatype as decimal(38,0)
> ---
>
> Key: HIVE-13422
> URL: https://issues.apache.org/jira/browse/HIVE-13422
> Project: Hive
> Issue Type: Bug
> Components: Hive, Statistics
>Affects Versions: 1.1.0
>Reporter: ashim sinha
>Assignee: Thomas Friedrich
> Attachments: HIVE-13422.patch
>
>
> For the repro
> {code}
> drop table sample_test;
> CREATE TABLE IF NOT EXISTS sample_test( key decimal(38,0),b int ) ROW FORMAT
> DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
> load data local inpath '/home/hive/analyse.txt' into table sample_test;
> ANALYZE TABLE sample_test COMPUTE STATISTICS FOR COLUMNS;
> {code}
> Sample data
> {code}
> 2023456789456749825082498304 0
> 5032080754887849825069508304 0
> 4012080754887849825068718304 0
> 2012080754887849825066778304 0
> 4012080754887849625065678304 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13422) Analyse command not working for column having datatype as decimal(38,0)
[
https://issues.apache.org/jira/browse/HIVE-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388097#comment-15388097
]
Thomas Friedrich commented on HIVE-13422:
-
I ran into the same issue and have a patch ready. Will upload shortly.
> Analyse command not working for column having datatype as decimal(38,0)
> ---
>
> Key: HIVE-13422
> URL: https://issues.apache.org/jira/browse/HIVE-13422
> Project: Hive
> Issue Type: Bug
> Components: Hive, Statistics
>Affects Versions: 1.1.0
>Reporter: ashim sinha
>Assignee: Thomas Friedrich
>
> For the repro
> {code}
> drop table sample_test;
> CREATE TABLE IF NOT EXISTS sample_test( key decimal(38,0),b int ) ROW FORMAT
> DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
> load data local inpath '/home/hive/analyse.txt' into table sample_test;
> ANALYZE TABLE sample_test COMPUTE STATISTICS FOR COLUMNS;
> {code}
> Sample data
> {code}
> 2023456789456749825082498304 0
> 5032080754887849825069508304 0
> 4012080754887849825068718304 0
> 2012080754887849825066778304 0
> 4012080754887849625065678304 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13422) Analyse command not working for column having datatype as decimal(38,0)
[ https://issues.apache.org/jira/browse/HIVE-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231994#comment-15231994 ] ashim sinha commented on HIVE-13422: Any update on this i'm blocked on this > Analyse command not working for column having datatype as decimal(38,0) > --- > > Key: HIVE-13422 > URL: https://issues.apache.org/jira/browse/HIVE-13422 > Project: Hive > Issue Type: Bug > Components: Hive, Statistics >Affects Versions: 1.1.0 >Reporter: ashim sinha > > Any update on this? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13422) Analyse command not working for column having datatype as decimal(38,0)
[ https://issues.apache.org/jira/browse/HIVE-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228106#comment-15228106 ] ashim sinha commented on HIVE-13422: For the repro drop table sample_test; CREATE TABLE IF NOT EXISTS sample_test( key decimal(38,0),b int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE; load data local inpath '/home/hive/analyse.txt' into table sample_test; ANALYZE TABLE sample_test COMPUTE STATISTICS FOR COLUMNS; Sample data 20234567894567498250824983040 50320807548878498250695083040 40120807548878498250687183040 20120807548878498250667783040 40120807548878496250656783040 > Analyse command not working for column having datatype as decimal(38,0) > --- > > Key: HIVE-13422 > URL: https://issues.apache.org/jira/browse/HIVE-13422 > Project: Hive > Issue Type: Bug > Components: Hive, Statistics >Affects Versions: 1.1.0 >Reporter: ashim sinha > > Any update on this? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
