[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[
https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403057#comment-15403057
]
Ashutosh Chauhan commented on HIVE-14329:
-
+1
> fix flapping qtests - because of output string ordering
> ---
>
> Key: HIVE-14329
> URL: https://issues.apache.org/jira/browse/HIVE-14329
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-14329.1.patch, HIVE-14329.2.patch,
> HIVE-14329.3.patch, HIVE-14329.4.patch
>
>
> it's a bit misleading to see some tests come and go in testresults; for
> example:
> https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/
> These tests fail occasionally because of the ordering is different in the map.
> The usual cause of these failures is a simple hashmap in
> {{MetaDataFormatUtils}}:
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[
https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15401515#comment-15401515
]
Hive QA commented on HIVE-14329:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12821172/HIVE-14329.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10421 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_nullable_union
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_avro_non_nullable_union
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_list_bucket_dml_2
org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.stringifyValidTxns
org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.testTxnRange
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/716/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/716/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-716/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12821172 - PreCommit-HIVE-MASTER-Build
> fix flapping qtests - because of output string ordering
> ---
>
> Key: HIVE-14329
> URL: https://issues.apache.org/jira/browse/HIVE-14329
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-14329.1.patch, HIVE-14329.2.patch,
> HIVE-14329.3.patch
>
>
> it's a bit annoying to see some tests come and go in testresults; for example:
> https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/
> These tests fail occasionally because of the ordering is different in the map.
> The usual cause of these failures is a simple hashmap in
> {{MetaDataFormatUtils}}:
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[
https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15399565#comment-15399565
]
Hive QA commented on HIVE-14329:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12820719/HIVE-14329.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 60 failed/errored test(s), 10366 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-tez_self_join.q-filter_join_breaktask.q-vector_decimal_precision.q-and-12-more
- did not produce a TEST-*.xml file
TestMsgBusConnection - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_nullable_union
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnStatsUpdateForStatsOptimizer_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_alter_list_bucketing_table1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_skewed_table1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_full
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_partial
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_partial_ndv
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_grouping_operators
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lb_fs_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_list_bucket_dml_10
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_alter_merge_stats_orc
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_column_names_with_leading_and_trailing_spaces
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_ctas
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_deleteAnalyze
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_analyze
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_fetchwork_part
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_fetchwork_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_mapwork_part
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_mapwork_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_vec_mapwork_part
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_vec_mapwork_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_stats
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_nonvec_mapwork_part
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_nonvec_mapwork_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_vec_mapwork_part
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_vec_mapwork_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_vecrow_mapwork_part
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[
https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398313#comment-15398313
]
Zoltan Haindrich commented on HIVE-14329:
-
https://reviews.apache.org/r/50586/
> fix flapping qtests - because of output string ordering
> ---
>
> Key: HIVE-14329
> URL: https://issues.apache.org/jira/browse/HIVE-14329
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-14329.1.patch, HIVE-14329.2.patch
>
>
> it's a bit annoying to see some tests come and go in testresults; for example:
> https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/
> These tests fail occasionally because of the ordering is different in the map.
> The usual cause of these failures is a simple hashmap in
> {{MetaDataFormatUtils}}:
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[
https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396265#comment-15396265
]
Zoltan Haindrich commented on HIVE-14329:
-
removed: inconsitency: {{StatsSetupConst#setColumnStatsState}} would put
lowercase colnames if record is new; original colnames during update/addition
choosen path: case sensitive
> fix flapping qtests - because of output string ordering
> ---
>
> Key: HIVE-14329
> URL: https://issues.apache.org/jira/browse/HIVE-14329
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
>
> it's a bit annoying to see some tests come and go in testresults; for example:
> https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/
> These tests fail occasionally because of the ordering is different in the map.
> The usual cause of these failures is a simple hashmap in
> {{MetaDataFormatUtils}}:
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[
https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396251#comment-15396251
]
Zoltan Haindrich commented on HIVE-14329:
-
removed: upgrade-while-read from: {{StatsSetupConst#areBasicStatsUptoDate}}
> fix flapping qtests - because of output string ordering
> ---
>
> Key: HIVE-14329
> URL: https://issues.apache.org/jira/browse/HIVE-14329
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
>
> it's a bit annoying to see some tests come and go in testresults; for example:
> https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/
> These tests fail occasionally because of the ordering is different in the map.
> The usual cause of these failures is a simple hashmap in
> {{MetaDataFormatUtils}}:
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[
https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396221#comment-15396221
]
Zoltan Haindrich commented on HIVE-14329:
-
contains: fix for {{StatsSetupConst#setBasicStatsState}}: failed to upgrade
basic stats status
> fix flapping qtests - because of output string ordering
> ---
>
> Key: HIVE-14329
> URL: https://issues.apache.org/jira/browse/HIVE-14329
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
>
> it's a bit annoying to see some tests come and go in testresults; for example:
> https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/
> These tests fail occasionally because of the ordering is different in the map.
> The usual cause of these failures is a simple hashmap in
> {{MetaDataFormatUtils}}:
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
