[jira] [Commented] (HIVE-13838) Set basic stats as inaccurate for all ACID tables

2016-06-10 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324748#comment-15324748
 ] 

Pengcheng Xiong commented on HIVE-13838:


Thanks [~ekoifman], i will take a look again. Thanks for finding this!

> Set basic stats as inaccurate for all ACID tables
> -
>
> Key: HIVE-13838
> URL: https://issues.apache.org/jira/browse/HIVE-13838
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Fix For: 2.1.0
>
> Attachments: HIVE-13838.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13838) Set basic stats as inaccurate for all ACID tables

2016-06-10 Thread Eugene Koifman (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324589#comment-15324589
 ] 

Eugene Koifman commented on HIVE-13838:
---

[~pxiong] as far as I can tell this is still not fixed.  Please see
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/70/testReport/
or
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/73/testReport/

the same set of tests keeps failing

> Set basic stats as inaccurate for all ACID tables
> -
>
> Key: HIVE-13838
> URL: https://issues.apache.org/jira/browse/HIVE-13838
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Fix For: 2.1.0
>
> Attachments: HIVE-13838.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13838) Set basic stats as inaccurate for all ACID tables

2016-06-07 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318956#comment-15318956
 ] 

Pengcheng Xiong commented on HIVE-13838:


Thanks [~ashutoshc]. I actually would like to confirm jdk8 with you 
yesterday.. It sounds real now. Thanks.

> Set basic stats as inaccurate for all ACID tables
> -
>
> Key: HIVE-13838
> URL: https://issues.apache.org/jira/browse/HIVE-13838
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Fix For: 2.1.0
>
> Attachments: HIVE-13838.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13838) Set basic stats as inaccurate for all ACID tables

2016-06-07 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318939#comment-15318939
 ] 

Ashutosh Chauhan commented on HIVE-13838:
-

[~pxiong] Two things may result in that diff:
* You are using jdk7. golden files are now generated using jdk8.
* You are on Mac. Tests are run on linux.

> Set basic stats as inaccurate for all ACID tables
> -
>
> Key: HIVE-13838
> URL: https://issues.apache.org/jira/browse/HIVE-13838
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Fix For: 2.1.0
>
> Attachments: HIVE-13838.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13838) Set basic stats as inaccurate for all ACID tables

2016-06-07 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318927#comment-15318927
 ] 

Pengcheng Xiong commented on HIVE-13838:


[~ekoifman], thanks for your attention. I have pushed a patch yesterday trying 
to fix 4 of them.
{code}
commit 0bdf371706d69f6fe8847026a013ed2dd1c86211
Author: Pengcheng Xiong 
Date:   Mon Jun 6 14:51:42 2016 -0700

HIVE-13838: Set basic stats as inaccurate for all ACID tables (Pengcheng 
Xiong, reviewed by Ashutosh Chauhan) (addendum)

{code}
However, it seems that there are still 2 test case failures due to different 
data size and different sequence although none of them can be reproduced 
locally. I will open a following JIRA to track it. Thanks.

> Set basic stats as inaccurate for all ACID tables
> -
>
> Key: HIVE-13838
> URL: https://issues.apache.org/jira/browse/HIVE-13838
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Fix For: 2.1.0
>
> Attachments: HIVE-13838.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13838) Set basic stats as inaccurate for all ACID tables

2016-06-03 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314383#comment-15314383
 ] 

Ashutosh Chauhan commented on HIVE-13838:
-

+1

> Set basic stats as inaccurate for all ACID tables
> -
>
> Key: HIVE-13838
> URL: https://issues.apache.org/jira/browse/HIVE-13838
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-13838.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13838) Set basic stats as inaccurate for all ACID tables

2016-05-31 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307993#comment-15307993
 ] 

Hive QA commented on HIVE-13838:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806226/HIVE-13838.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 10173 tests 
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_join30.q-script_pipe.q-vector_decimal_10_0.q-and-12-more
 - did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_table_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_autoColumnStats_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_values_orig_table_use_metadata
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_llap_acid
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
{noformat}

Test results: 
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/469/testReport
Console output: 
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/469/console
Test logs: 
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-469/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12806226 - PreCommit-HIVE-MASTER-Build

> Set basic stats as inaccurate for all ACID tables
> -
>
> Key: HIVE-13838
> URL: https://issues.apache.org/jira/browse/HIVE-13838
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-13838.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13838) Set basic stats as inaccurate for all ACID tables

2016-05-28 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15305347#comment-15305347
 ] 

Hive QA commented on HIVE-13838:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806226/HIVE-13838.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 250 failed/errored test(s), 6207 tests 
executed
*Failed tests:*
{noformat}
TestCliDriver-alter_file_format.q-bucket_map_join_tez1.q-fetch_aggregation.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-alter_rename_table.q-udf_aes_decrypt.q-udf_add.q-and-12-more - 
did not produce a TEST-*.xml file
TestCliDriver-alter_table_not_sorted.q-cbo_udf_max.q-exim_11_managed_external.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-ambiguitycheck.q-udf_bitwise_and.q-insert_overwrite_directory.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-authorization_1_sql_std.q-disallow_incompatible_type_change_off.q-schema_evol_text_nonvec_mapwork_table.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-authorization_cli_nonsql.q-drop_index.q-vector_date_1.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-authorization_cli_stdconfigauth.q-str_to_map.q-touch.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-authorization_create_table_owner_privs.q-cte_7.q-groupby_sort_7.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-authorization_create_temp_table.q-update_all_partitioned.q-lock4.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-authorization_parts.q-authorization_view_1.q-parquet_map_of_maps.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-authorization_role_grant2.q-union_remove_7.q-alter_view_as_select.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-auto_join18.q-dynamic_partition_pruning.q-date_comparison.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-auto_join30.q-udf_bitmap_empty.q-input17.q-and-12-more - did not 
produce a TEST-*.xml file
TestCliDriver-auto_join_reordering_values.q-subquery_in.q-partition_timestamp2.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-auto_sortmerge_join_7.q-exim_04_evolved_parts.q-query_with_semi.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-autogen_colalias.q-skewjoinopt3.q-rcfile_merge1.q-and-12-more - 
did not produce a TEST-*.xml file
TestCliDriver-avro_add_column.q-bucketmapjoin_negative.q-bucket_map_join_spark2.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-ba_table1.q-udf_notequal.q-input_part1.q-and-12-more - did not 
produce a TEST-*.xml file
TestCliDriver-bucketcontext_4.q-orc_ends_with_nulls.q-correlationoptimizer9.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-bucketsortoptimize_insert_7.q-list_bucket_query_multiskew_1.q-skewjoin_noskew.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-cast_to_int.q-literal_double.q-skewjoinopt20.q-and-12-more - did 
not produce a TEST-*.xml file
TestCliDriver-cbo_rp_gby_empty.q-load_nonpart_authsuccess.q-truncate_table.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_join1.q-union_top_level.q-join18.q-and-12-more - did not 
produce a TEST-*.xml file
TestCliDriver-cbo_rp_limit.q-load_exist_part_authsuccess.q-schema_evol_text_nonvec_mapwork_part_all_primitive.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_lineage2.q-authorization_update.q-udf_pmod.q-and-12-more - 
did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_semijoin.q-custom_input_output_format.q-udf_reflect2.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_udf_udaf.q-date_udf.q-limit0.q-and-12-more - did not 
produce a TEST-*.xml file
TestCliDriver-cbo_rp_views.q-orc_split_elimination.q-authorization_view_disable_cbo_4.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-columnStatsUpdateForStatsOptimizer_2.q-cp_mj_rc.q-show_columns.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-columnstats_partlvl_dp.q-input31.q-udaf_percentile.q-and-12-more 
- did not produce a TEST-*.xml file
TestCliDriver-compute_stats_string.q-noalias_subq1.q-vector_auto_smb_mapjoin_14.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-create_func1.q-bucketmapjoin3.q-enforce_order.q-and-12-more - did 
not produce a TEST-*.xml file
TestCliDriver-create_genericudf.q-dynamic_partition_insert.q-skewjoinopt16.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-create_or_replace_view.q-dbtxnmgr_query5.q-compute_stats_double.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-cte_mat_1.q-groupby_sort_6.q-update_after_multiple_inserts_special_characters.q-and-12-more
 - did not produce a TEST-*.xml file
TestCliDriver-decimal_join.q-mapjoin_addjar.q