HIVE-14502: Convert MiniTez tests to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Project: http://git-wip-us.apache.org/repos/asf/hive/repo Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/5f01dc42 Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/5f01dc42 Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/5f01dc42 Branch: refs/heads/master Commit: 5f01dc42a518044975cb54da2966cdd1ba1ac126 Parents: a4d8c66 Author: Prasanth Jayachandran <[email protected]> Authored: Tue Aug 23 11:45:10 2016 -0700 Committer: Prasanth Jayachandran <[email protected]> Committed: Tue Aug 23 11:45:10 2016 -0700 ---------------------------------------------------------------------- data/conf/llap/hive-site.xml | 6 + .../test/resources/testconfiguration.properties | 311 +- .../queries/clientpositive/constprog_semijoin.q | 1 + ql/src/test/queries/clientpositive/cte_3.q | 1 + ql/src/test/queries/clientpositive/cte_5.q | 1 + ql/src/test/queries/clientpositive/cte_mat_1.q | 1 + ql/src/test/queries/clientpositive/cte_mat_2.q | 1 + ql/src/test/queries/clientpositive/cte_mat_3.q | 1 + ql/src/test/queries/clientpositive/cte_mat_4.q | 1 + ql/src/test/queries/clientpositive/cte_mat_5.q | 1 + .../test/queries/clientpositive/deleteAnalyze.q | 1 + ql/src/test/queries/clientpositive/empty_join.q | 1 + .../clientpositive/nonmr_fetch_threshold.q | 1 + ql/src/test/queries/clientpositive/smb_cache.q | 1 + .../vector_aggregate_without_gby.q | 3 +- .../clientpositive/vector_auto_smb_mapjoin_14.q | 1 + .../test/queries/clientpositive/vector_bround.q | 3 +- .../clientpositive/vector_groupby_mapjoin.q | 2 +- .../clientpositive/vector_join_part_col_char.q | 2 + .../queries/clientpositive/vector_struct_in.q | 3 +- .../clientpositive/vectorized_parquet_types.q | 3 +- .../clientpositive/vectorized_timestamp.q | 3 +- .../test/queries/clientpositive/windowing_gby.q | 1 + .../llap/acid_bucket_pruning.q.out | 153 + .../clientpositive/llap/acid_globallimit.q.out | 52 + .../llap/acid_vectorization_missing_cols.q.out | 56 + .../clientpositive/llap/alter_merge_2_orc.q.out | 2 - .../llap/alter_merge_stats_orc.q.out | 10 +- .../clientpositive/llap/auto_join0.q.out | 46 +- .../clientpositive/llap/auto_join1.q.out | 62 +- .../clientpositive/llap/auto_join21.q.out | 34 +- .../clientpositive/llap/auto_join29.q.out | 323 +- .../clientpositive/llap/auto_join30.q.out | 439 +- .../clientpositive/llap/auto_join_filters.q.out | 48 +- .../clientpositive/llap/auto_join_nulls.q.out | 6 +- .../llap/auto_sortmerge_join_1.q.out | 360 +- .../llap/auto_sortmerge_join_10.q.out | 136 +- .../llap/auto_sortmerge_join_11.q.out | 397 +- .../llap/auto_sortmerge_join_12.q.out | 222 +- .../llap/auto_sortmerge_join_13.q.out | 66 +- .../llap/auto_sortmerge_join_14.q.out | 120 +- .../llap/auto_sortmerge_join_15.q.out | 120 +- .../llap/auto_sortmerge_join_2.q.out | 240 +- .../llap/auto_sortmerge_join_3.q.out | 360 +- .../llap/auto_sortmerge_join_4.q.out | 360 +- .../llap/auto_sortmerge_join_5.q.out | 327 +- .../llap/auto_sortmerge_join_6.q.out | 688 +- .../llap/auto_sortmerge_join_7.q.out | 363 +- .../llap/auto_sortmerge_join_8.q.out | 363 +- .../llap/auto_sortmerge_join_9.q.out | 1474 +-- .../results/clientpositive/llap/bucket2.q.out | 46 +- .../results/clientpositive/llap/bucket3.q.out | 40 +- .../results/clientpositive/llap/bucket4.q.out | 46 +- .../llap/bucket_map_join_tez1.q.out | 432 +- .../llap/bucket_map_join_tez2.q.out | 142 +- .../clientpositive/llap/bucketpruning1.q.out | 1697 +++ ...names_with_leading_and_trailing_spaces.q.out | 85 + .../llap/constprog_semijoin.q.out | 494 + .../llap/correlationoptimizer1.q.out | 1450 +-- .../results/clientpositive/llap/count.q.out | 642 +- .../clientpositive/llap/cross_join.q.out | 272 +- .../llap/cross_product_check_1.q.out | 399 +- .../llap/cross_product_check_2.q.out | 444 +- .../test/results/clientpositive/llap/ctas.q.out | 191 +- .../results/clientpositive/llap/cte_3.q.out | 285 +- .../results/clientpositive/llap/cte_5.q.out | 94 +- .../results/clientpositive/llap/cte_mat_1.q.out | 94 +- .../results/clientpositive/llap/cte_mat_2.q.out | 94 +- .../results/clientpositive/llap/cte_mat_3.q.out | 155 +- .../results/clientpositive/llap/cte_mat_4.q.out | 310 +- .../results/clientpositive/llap/cte_mat_5.q.out | 156 +- .../clientpositive/llap/deleteAnalyze.q.out | 140 + .../llap/disable_merge_for_bucketing.q.out | 54 +- .../llap/dynamic_partition_pruning.q.out | 486 +- .../llap/dynamic_partition_pruning_2.q.out | 114 +- .../llap/dynpart_sort_opt_vectorization.q.out | 260 +- .../llap/dynpart_sort_optimization.q.out | 1022 +- .../llap/dynpart_sort_optimization2.q.out | 104 +- .../clientpositive/llap/empty_join.q.out | 111 + .../clientpositive/llap/explainuser_1.q.out | 9948 ++++++------------ .../clientpositive/llap/explainuser_2.q.out | 7627 ++++++-------- .../clientpositive/llap/explainuser_4.q.out | 482 + .../llap/filter_join_breaktask.q.out | 221 +- .../results/clientpositive/llap/groupby1.q.out | 20 +- .../results/clientpositive/llap/groupby2.q.out | 20 +- .../results/clientpositive/llap/groupby3.q.out | 20 +- .../results/clientpositive/llap/having.q.out | 138 +- .../llap/hybridgrace_hashjoin_1.q.out | 200 +- .../llap/hybridgrace_hashjoin_2.q.out | 316 +- .../llap/identity_project_remove_skip.q.out | 60 +- .../results/clientpositive/llap/insert1.q.out | 7 +- .../clientpositive/llap/insert_into1.q.out | 317 +- .../clientpositive/llap/insert_into2.q.out | 136 +- .../results/clientpositive/llap/join0.q.out | 24 +- .../results/clientpositive/llap/join1.q.out | 43 +- .../clientpositive/llap/join_nullsafe.q.out | 82 +- .../clientpositive/llap/limit_pushdown.q.out | 319 +- .../clientpositive/llap/load_dyn_part1.q.out | 15 +- .../clientpositive/llap/load_dyn_part2.q.out | 12 +- .../clientpositive/llap/load_dyn_part3.q.out | 7 +- .../results/clientpositive/llap/mapjoin2.q.out | 95 + .../clientpositive/llap/mapreduce1.q.out | 15 +- .../clientpositive/llap/mapreduce2.q.out | 15 +- .../results/clientpositive/llap/merge1.q.out | 32 +- .../results/clientpositive/llap/merge2.q.out | 32 +- .../results/clientpositive/llap/mergejoin.q.out | 414 +- .../clientpositive/llap/mergejoin_3way.q.out | 30 + .../llap/metadata_only_queries.q.out | 280 +- .../clientpositive/llap/metadataonly1.q.out | 627 +- .../test/results/clientpositive/llap/mrr.q.out | 181 +- .../llap/nonmr_fetch_threshold.q.out | 205 + .../clientpositive/llap/optimize_nullscan.q.out | 689 +- .../clientpositive/llap/orc_analyze.q.out | 306 +- .../clientpositive/llap/orc_merge1.q.out | 21 +- .../clientpositive/llap/orc_merge10.q.out | 743 ++ .../clientpositive/llap/orc_merge11.q.out | 442 + .../clientpositive/llap/orc_merge2.q.out | 7 +- .../clientpositive/llap/orc_merge3.q.out | 7 +- .../clientpositive/llap/orc_merge4.q.out | 7 +- .../clientpositive/llap/orc_merge5.q.out | 6 +- .../clientpositive/llap/orc_merge6.q.out | 10 +- .../clientpositive/llap/orc_merge7.q.out | 2 + .../clientpositive/llap/orc_merge_diff_fs.q.out | 503 + .../llap/orc_merge_incompat1.q.out | 1 + .../llap/orc_merge_incompat2.q.out | 1 + .../llap/orc_merge_incompat3.q.out | 70 + .../clientpositive/llap/order_null.q.out | 222 + .../results/clientpositive/llap/parallel.q.out | 26 +- .../test/results/clientpositive/llap/ptf.q.out | 862 +- .../clientpositive/llap/ptf_matchpath.q.out | 58 +- .../clientpositive/llap/ptf_streaming.q.out | 493 +- .../results/clientpositive/llap/sample1.q.out | 60 +- .../schema_evol_orc_nonvec_mapwork_part.q.out | 144 +- ...ol_orc_nonvec_mapwork_part_all_complex.q.out | 18 +- ..._orc_nonvec_mapwork_part_all_primitive.q.out | 96 +- .../llap/schema_evol_orc_vec_mapwork_part.q.out | 144 +- ..._evol_orc_vec_mapwork_part_all_complex.q.out | 18 +- ...vol_orc_vec_mapwork_part_all_primitive.q.out | 96 +- .../schema_evol_text_nonvec_mapwork_part.q.out | 144 +- ...l_text_nonvec_mapwork_part_all_complex.q.out | 18 +- ...text_nonvec_mapwork_part_all_primitive.q.out | 96 +- .../schema_evol_text_vec_mapwork_part.q.out | 144 +- ...evol_text_vec_mapwork_part_all_complex.q.out | 18 +- ...ol_text_vec_mapwork_part_all_primitive.q.out | 96 +- .../schema_evol_text_vecrow_mapwork_part.q.out | 144 +- ...l_text_vecrow_mapwork_part_all_complex.q.out | 18 +- ...text_vecrow_mapwork_part_all_primitive.q.out | 96 +- .../clientpositive/llap/script_pipe.q.out | 35 +- .../clientpositive/llap/scriptfile1.q.out | 6 + .../llap/selectDistinctStar.q.out | 138 +- .../results/clientpositive/llap/skewjoin.q.out | 257 +- .../results/clientpositive/llap/smb_cache.q.out | 347 + .../clientpositive/llap/stats_noscan_1.q.out | 28 +- .../clientpositive/llap/stats_only_null.q.out | 28 +- .../clientpositive/llap/subquery_exists.q.out | 140 +- .../clientpositive/llap/subquery_in.q.out | 300 +- .../clientpositive/llap/temp_table.q.out | 147 +- .../llap/tez_bmj_schema_evolution.q.out | 14 +- .../results/clientpositive/llap/tez_dml.q.out | 30 +- .../llap/tez_dynpart_hashjoin_1.q.out | 80 +- .../llap/tez_dynpart_hashjoin_2.q.out | 96 +- .../llap/tez_dynpart_hashjoin_3.q.out | 206 + .../clientpositive/llap/tez_join_hash.q.out | 60 +- .../clientpositive/llap/tez_join_tests.q.out | 34 +- .../clientpositive/llap/tez_joins_explain.q.out | 34 +- .../results/clientpositive/llap/tez_smb_1.q.out | 80 +- .../clientpositive/llap/tez_smb_empty.q.out | 918 ++ .../clientpositive/llap/tez_smb_main.q.out | 52 +- .../results/clientpositive/llap/tez_union.q.out | 270 +- .../clientpositive/llap/tez_union2.q.out | 40 +- .../llap/tez_union_group_by.q.out | 60 +- .../llap/tez_union_multiinsert.q.out | 184 +- .../llap/tez_vector_dynpart_hashjoin_1.q.out | 80 +- .../llap/tez_vector_dynpart_hashjoin_2.q.out | 96 +- .../clientpositive/llap/transform1.q.out | 8 +- .../clientpositive/llap/transform_ppr1.q.out | 106 +- .../clientpositive/llap/transform_ppr2.q.out | 102 +- .../results/clientpositive/llap/union2.q.out | 39 +- .../results/clientpositive/llap/union3.q.out | 67 +- .../results/clientpositive/llap/union4.q.out | 13 +- .../results/clientpositive/llap/union5.q.out | 51 +- .../results/clientpositive/llap/union6.q.out | 17 +- .../results/clientpositive/llap/union7.q.out | 39 +- .../results/clientpositive/llap/union8.q.out | 38 +- .../results/clientpositive/llap/union9.q.out | 54 +- .../clientpositive/llap/unionDistinct_1.q.out | 3311 +++--- .../clientpositive/llap/unionDistinct_2.q.out | 6 + .../clientpositive/llap/union_fast_stats.q.out | 154 +- .../clientpositive/llap/union_stats.q.out | 484 + .../clientpositive/llap/union_type_chk.q.out | 30 + .../clientpositive/llap/update_all_types.q.out | 34 +- .../llap/vector_aggregate_9.q.out | 18 +- .../llap/vector_aggregate_without_gby.q.out | 72 + .../llap/vector_auto_smb_mapjoin_14.q.out | 1521 +-- .../llap/vector_between_columns.q.out | 274 + .../clientpositive/llap/vector_between_in.q.out | 422 +- .../llap/vector_binary_join_groupby.q.out | 177 +- .../clientpositive/llap/vector_bround.q.out | 66 + .../clientpositive/llap/vector_bucket.q.out | 2 + .../llap/vector_cast_constant.q.out | 1 + .../clientpositive/llap/vector_char_2.q.out | 22 +- .../clientpositive/llap/vector_char_4.q.out | 1 + .../llap/vector_char_mapjoin1.q.out | 194 +- .../llap/vector_char_simple.q.out | 39 +- .../clientpositive/llap/vector_coalesce.q.out | 250 +- .../clientpositive/llap/vector_coalesce_2.q.out | 26 +- .../llap/vector_complex_join.q.out | 10 +- .../clientpositive/llap/vector_count.q.out | 322 + .../llap/vector_count_distinct.q.out | 18 +- .../clientpositive/llap/vector_data_types.q.out | 22 +- .../clientpositive/llap/vector_date_1.q.out | 55 +- .../clientpositive/llap/vector_decimal_1.q.out | 72 +- .../llap/vector_decimal_10_0.q.out | 9 +- .../clientpositive/llap/vector_decimal_2.q.out | 253 +- .../clientpositive/llap/vector_decimal_3.q.out | 516 +- .../clientpositive/llap/vector_decimal_4.q.out | 288 +- .../clientpositive/llap/vector_decimal_5.q.out | 180 +- .../clientpositive/llap/vector_decimal_6.q.out | 216 +- .../llap/vector_decimal_aggregate.q.out | 52 +- .../llap/vector_decimal_cast.q.out | 22 +- .../llap/vector_decimal_expressions.q.out | 48 +- .../llap/vector_decimal_mapjoin.q.out | 267 +- .../llap/vector_decimal_math_funcs.q.out | 6 +- .../llap/vector_decimal_precision.q.out | 276 +- .../llap/vector_decimal_round.q.out | 54 +- .../llap/vector_decimal_round_2.q.out | 46 +- .../llap/vector_decimal_trailing.q.out | 42 +- .../llap/vector_decimal_udf.q.out | 1184 ++- .../llap/vector_decimal_udf2.q.out | 12 +- .../clientpositive/llap/vector_distinct_2.q.out | 8 +- .../clientpositive/llap/vector_groupby4.q.out | 138 + .../clientpositive/llap/vector_groupby6.q.out | 138 + .../clientpositive/llap/vector_groupby_3.q.out | 8 +- .../llap/vector_groupby_mapjoin.q.out | 87 + .../llap/vector_groupby_reduce.q.out | 97 +- .../llap/vector_grouping_sets.q.out | 45 +- .../clientpositive/llap/vector_if_expr.q.out | 22 +- .../clientpositive/llap/vector_inner_join.q.out | 508 +- .../clientpositive/llap/vector_interval_1.q.out | 68 +- .../clientpositive/llap/vector_interval_2.q.out | 92 +- .../llap/vector_interval_arithmetic.q.out | 1093 ++ .../llap/vector_interval_mapjoin.q.out | 98 +- .../clientpositive/llap/vector_join30.q.out | 133 +- .../llap/vector_join_filters.q.out | 8 +- .../clientpositive/llap/vector_join_nulls.q.out | 8 +- .../llap/vector_join_part_col_char.q.out | 114 +- .../llap/vector_left_outer_join.q.out | 42 +- .../llap/vector_left_outer_join2.q.out | 393 +- .../llap/vector_leftsemi_mapjoin.q.out | 3189 +++--- .../llap/vector_mapjoin_reduce.q.out | 171 +- .../llap/vector_mr_diff_schema_alias.q.out | 123 +- .../llap/vector_multi_insert.q.out | 2 + .../llap/vector_null_projection.q.out | 54 +- .../llap/vector_nullsafe_join.q.out | 154 +- .../clientpositive/llap/vector_nvl.q.out | 194 + .../clientpositive/llap/vector_orderby_5.q.out | 8 +- .../llap/vector_outer_join0.q.out | 92 +- .../llap/vector_outer_join1.q.out | 152 +- .../llap/vector_outer_join2.q.out | 104 +- .../llap/vector_outer_join3.q.out | 208 +- .../llap/vector_outer_join4.q.out | 152 +- .../llap/vector_outer_join5.q.out | 346 +- .../llap/vector_outer_join6.q.out | 373 + .../llap/vector_partition_diff_num_cols.q.out | 68 +- .../llap/vector_partitioned_date_time.q.out | 120 +- .../clientpositive/llap/vector_reduce1.q.out | 2169 ++++ .../clientpositive/llap/vector_reduce2.q.out | 2169 ++++ .../clientpositive/llap/vector_reduce3.q.out | 2169 ++++ .../llap/vector_reduce_groupby_decimal.q.out | 124 +- .../llap/vector_string_concat.q.out | 10 +- .../clientpositive/llap/vector_struct_in.q.out | 645 ++ .../clientpositive/llap/vector_varchar_4.q.out | 1 + .../llap/vector_varchar_mapjoin1.q.out | 194 +- .../llap/vector_varchar_simple.q.out | 39 +- .../llap/vector_when_case_null.q.out | 99 + .../clientpositive/llap/vectorization_0.q.out | 242 +- .../clientpositive/llap/vectorization_1.q.out | 2 +- .../clientpositive/llap/vectorization_12.q.out | 1064 +- .../clientpositive/llap/vectorization_13.q.out | 118 +- .../clientpositive/llap/vectorization_14.q.out | 634 +- .../clientpositive/llap/vectorization_15.q.out | 122 +- .../clientpositive/llap/vectorization_16.q.out | 748 +- .../clientpositive/llap/vectorization_17.q.out | 38 +- .../clientpositive/llap/vectorization_3.q.out | 1 - .../clientpositive/llap/vectorization_7.q.out | 50 +- .../clientpositive/llap/vectorization_8.q.out | 62 +- .../clientpositive/llap/vectorization_9.q.out | 748 +- .../llap/vectorization_decimal_date.q.out | 4 + .../llap/vectorization_part_project.q.out | 21 +- .../llap/vectorization_part_varchar.q.out | 72 + .../llap/vectorization_pushdown.q.out | 22 +- .../llap/vectorization_short_regress.q.out | 204 +- .../llap/vectorized_bucketmapjoin1.q.out | 63 +- .../clientpositive/llap/vectorized_case.q.out | 111 +- .../clientpositive/llap/vectorized_casts.q.out | 51 +- .../llap/vectorized_context.q.out | 102 +- .../llap/vectorized_date_funcs.q.out | 34 +- .../llap/vectorized_distinct_gby.q.out | 38 +- .../vectorized_dynamic_partition_pruning.q.out | 464 +- .../llap/vectorized_mapjoin.q.out | 38 +- .../llap/vectorized_math_funcs.q.out | 2 +- .../llap/vectorized_nested_mapjoin.q.out | 52 +- .../llap/vectorized_parquet.q.out | 8 +- .../llap/vectorized_parquet_types.q.out | 133 +- .../clientpositive/llap/vectorized_ptf.q.out | 2745 +---- .../llap/vectorized_shufflejoin.q.out | 43 +- .../llap/vectorized_string_funcs.q.out | 2 +- .../llap/vectorized_timestamp.q.out | 157 + .../llap/vectorized_timestamp_funcs.q.out | 66 +- .../clientpositive/llap/windowing_gby.q.out | 66 + 310 files changed, 49490 insertions(+), 35236 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/data/conf/llap/hive-site.xml ---------------------------------------------------------------------- diff --git a/data/conf/llap/hive-site.xml b/data/conf/llap/hive-site.xml index 8b6dc95..d93f8b7 100644 --- a/data/conf/llap/hive-site.xml +++ b/data/conf/llap/hive-site.xml @@ -45,6 +45,12 @@ </property> <property> + <name>hive.stats.fetch.column.stats</name> + <value>true</value> + <description>Use column stats to annotate stats for physical optimization phase</description> +</property> + +<property> <name>hive.merge.tezfiles</name> <value>false</value> <description>Merge small files at the end of a Tez DAG</description> http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/itests/src/test/resources/testconfiguration.properties ---------------------------------------------------------------------- diff --git a/itests/src/test/resources/testconfiguration.properties b/itests/src/test/resources/testconfiguration.properties index 662ae3e..a1bb37c 100644 --- a/itests/src/test/resources/testconfiguration.properties +++ b/itests/src/test/resources/testconfiguration.properties @@ -68,17 +68,59 @@ disabled.query.files=ql_rewrite_gbtoidx.q,\ rcfile_merge1.q,\ smb_mapjoin_8.q -minitez.query.files.shared=acid_globallimit.q,\ - deleteAnalyze.q,\ - empty_join.q,\ +# NOTE: Add tests to minitez only if it is very +# specific to tez and cannot be added to minillap. +minitez.query.files.shared=delete_orig_table.q,\ + orc_merge12.q,\ + orc_vectorization_ppd.q,\ + unionDistinct_2.q,\ + update_orig_table.q,\ + vector_join_part_col_char.q,\ + vector_non_string_partition.q,\ + vectorization_div0.q,\ + vectorization_limit.q + +# NOTE: Add tests to minitez only if it is very +# specific to tez and cannot be added to minillap. +minitez.query.files=explainuser_3.q,\ + hybridgrace_hashjoin_1.q,\ + hybridgrace_hashjoin_2.q,\ + partition_column_names_with_leading_and_trailing_spaces.q,\ + stats_filemetadata.q,\ + tez_union_with_udf.q + +minillap.shared.query.files=acid_globallimit.q,\ alter_merge_2_orc.q,\ alter_merge_orc.q,\ alter_merge_stats_orc.q,\ auto_join0.q,\ auto_join1.q,\ + auto_join21.q,\ + auto_join29.q,\ + auto_join30.q,\ + auto_join_filters.q,\ + auto_join_nulls.q,\ + auto_sortmerge_join_1.q,\ + auto_sortmerge_join_10.q,\ + auto_sortmerge_join_11.q,\ + auto_sortmerge_join_12.q,\ + auto_sortmerge_join_13.q,\ + auto_sortmerge_join_14.q,\ + auto_sortmerge_join_15.q,\ + auto_sortmerge_join_16.q,\ + auto_sortmerge_join_2.q,\ + auto_sortmerge_join_3.q,\ + auto_sortmerge_join_4.q,\ + auto_sortmerge_join_5.q,\ + auto_sortmerge_join_6.q,\ + auto_sortmerge_join_7.q,\ + auto_sortmerge_join_8.q,\ + auto_sortmerge_join_9.q,\ bucket2.q,\ bucket3.q,\ bucket4.q,\ + bucket_map_join_tez1.q,\ + bucket_map_join_tez2.q,\ cbo_gby.q,\ cbo_gby_empty.q,\ cbo_join.q,\ @@ -94,6 +136,7 @@ minitez.query.files.shared=acid_globallimit.q,\ cbo_views.q,\ cbo_windowing.q,\ column_names_with_leading_and_trailing_spaces.q,\ + constprog_dpp.q,\ constprog_semijoin.q,\ correlationoptimizer1.q,\ count.q,\ @@ -113,18 +156,21 @@ minitez.query.files.shared=acid_globallimit.q,\ cte_mat_4.q,\ cte_mat_5.q,\ custom_input_output_format.q,\ + deleteAnalyze.q,\ delete_all_non_partitioned.q,\ delete_all_partitioned.q,\ - delete_orig_table.q,\ delete_tmp_table.q,\ delete_where_no_match.q,\ delete_where_non_partitioned.q,\ delete_where_partitioned.q,\ delete_whole_partition.q,\ disable_merge_for_bucketing.q,\ + dynamic_partition_pruning.q,\ + dynamic_partition_pruning_2.q,\ dynpart_sort_opt_vectorization.q,\ dynpart_sort_optimization.q,\ dynpart_sort_optimization2.q,\ + empty_join.q,\ enforce_order.q,\ filter_join_breaktask.q,\ filter_join_breaktask2.q,\ @@ -132,40 +178,46 @@ minitez.query.files.shared=acid_globallimit.q,\ groupby2.q,\ groupby3.q,\ having.q,\ - identity_project_remove_skip.q\ + identity_project_remove_skip.q,\ insert1.q,\ insert_into1.q,\ insert_into2.q,\ insert_orig_table.q,\ + insert_update_delete.q,\ insert_values_dynamic_partitioned.q,\ insert_values_non_partitioned.q,\ - insert_values_orig_table.q\ + insert_values_orig_table.,\ insert_values_partitioned.q,\ insert_values_tmp_table.q,\ - insert_update_delete.q,\ join0.q,\ join1.q,\ join_nullsafe.q,\ leftsemijoin.q,\ limit_pushdown.q,\ + llap_nullscan.q,\ + llapdecider.q,\ load_dyn_part1.q,\ load_dyn_part2.q,\ load_dyn_part3.q,\ + lvj_mapjoin.q,\ mapjoin2.q,\ + mapjoin_decimal.q,\ mapjoin_mapjoin.q,\ mapreduce1.q,\ mapreduce2.q,\ merge1.q,\ merge2.q,\ mergejoin.q,\ - metadataonly1.q,\ metadata_only_queries.q,\ metadata_only_queries_with_filters.q,\ + metadataonly1.q,\ + mrr.q,\ nonmr_fetch_threshold.q,\ - order_null.q,\ optimize_nullscan.q,\ orc_analyze.q,\ orc_merge1.q,\ + orc_merge10.q,\ + orc_merge11.q,\ orc_merge2.q,\ orc_merge3.q,\ orc_merge4.q,\ @@ -174,28 +226,53 @@ minitez.query.files.shared=acid_globallimit.q,\ orc_merge7.q,\ orc_merge8.q,\ orc_merge9.q,\ - orc_merge10.q,\ - orc_merge11.q,\ - orc_merge12.q,\ orc_merge_incompat1.q,\ orc_merge_incompat2.q,\ orc_merge_incompat3.q,\ + orc_ppd_basic.q,\ orc_ppd_schema_evol_1a.q,\ orc_ppd_schema_evol_1b.q,\ orc_ppd_schema_evol_2a.q,\ orc_ppd_schema_evol_2b.q,\ orc_ppd_schema_evol_3a.q,\ - orc_vectorization_ppd.q,\ + order_null.q,\ parallel.q,\ ptf.q,\ ptf_matchpath.q,\ ptf_streaming.q,\ sample1.q,\ - selectDistinctStar.q,\ + schema_evol_orc_acid_mapwork_part.q,\ + schema_evol_orc_acid_mapwork_table.q,\ + schema_evol_orc_acidvec_mapwork_part.q,\ + schema_evol_orc_acidvec_mapwork_table.q,\ + schema_evol_orc_nonvec_fetchwork_part.q,\ + schema_evol_orc_nonvec_fetchwork_table.q,\ + schema_evol_orc_nonvec_mapwork_part.q,\ + schema_evol_orc_nonvec_mapwork_part_all_complex.q,\ + schema_evol_orc_nonvec_mapwork_part_all_primitive.q,\ + schema_evol_orc_nonvec_mapwork_table.q,\ + schema_evol_orc_vec_mapwork_part.q,\ + schema_evol_orc_vec_mapwork_part_all_complex.q,\ + schema_evol_orc_vec_mapwork_part_all_primitive.q,\ + schema_evol_orc_vec_mapwork_table.q,\ + schema_evol_stats.q,\ + schema_evol_text_nonvec_mapwork_part.q,\ + schema_evol_text_nonvec_mapwork_part_all_complex.q,\ + schema_evol_text_nonvec_mapwork_part_all_primitive.q,\ + schema_evol_text_nonvec_mapwork_table.q,\ + schema_evol_text_vec_mapwork_part.q,\ + schema_evol_text_vec_mapwork_part_all_complex.q,\ + schema_evol_text_vec_mapwork_part_all_primitive.q,\ + schema_evol_text_vec_mapwork_table.q,\ + schema_evol_text_vecrow_mapwork_part.q,\ + schema_evol_text_vecrow_mapwork_part_all_complex.q,\ + schema_evol_text_vecrow_mapwork_part_all_primitive.q,\ + schema_evol_text_vecrow_mapwork_table.q,\ script_env_var1.q,\ script_env_var2.q,\ script_pipe.q,\ scriptfile1.q,\ + selectDistinctStar.q,\ select_dummy_source.q,\ skewjoin.q,\ stats_counter.q,\ @@ -204,8 +281,32 @@ minitez.query.files.shared=acid_globallimit.q,\ stats_only_null.q,\ subquery_exists.q,\ subquery_in.q,\ - tez_join_hash.q,\ temp_table.q,\ + tez_bmj_schema_evolution.q,\ + tez_dml.q,\ + tez_dynpart_hashjoin_1.q,\ + tez_dynpart_hashjoin_2.q,\ + tez_fsstat.q,\ + tez_insert_overwrite_local_directory_1.q,\ + tez_join.q,\ + tez_join_hash.q,\ + tez_join_result_complex.q,\ + tez_join_tests.q,\ + tez_joins_explain.q,\ + tez_multi_union.q,\ + tez_schema_evolution.q,\ + tez_self_join.q,\ + tez_smb_1.q,\ + tez_smb_main.q,\ + tez_union.q,\ + tez_union2.q,\ + tez_union_decimal.q,\ + tez_union_dynamic_partition.q,\ + tez_union_group_by.q,\ + tez_union_multiinsert.q,\ + tez_union_view.q,\ + tez_vector_dynpart_hashjoin_1.q,\ + tez_vector_dynpart_hashjoin_2.q,\ transform1.q,\ transform2.q,\ transform_ppr1.q,\ @@ -219,32 +320,31 @@ minitez.query.files.shared=acid_globallimit.q,\ union8.q,\ union9.q,\ unionDistinct_1.q,\ - unionDistinct_2.q,\ union_fast_stats.q,\ union_stats.q,\ + union_type_chk.q,\ update_after_multiple_inserts.q,\ update_all_non_partitioned.q,\ update_all_partitioned.q,\ update_all_types.q,\ - update_orig_table.q,\ update_tmp_table.q,\ + update_two_cols.q,\ update_where_no_match.q,\ update_where_non_partitioned.q,\ update_where_partitioned.q,\ - update_two_cols.q,\ vector_acid3.q,\ vector_aggregate_9.q,\ vector_aggregate_without_gby.q,\ vector_auto_smb_mapjoin_14.q,\ - vector_between_in.q,\ vector_between_columns.q,\ + vector_between_in.q,\ vector_binary_join_groupby.q,\ vector_bround.q,\ vector_bucket.q,\ - vector_char_cast.q,\ vector_cast_constant.q,\ vector_char_2.q,\ vector_char_4.q,\ + vector_char_cast.q,\ vector_char_mapjoin1.q,\ vector_char_simple.q,\ vector_coalesce.q,\ @@ -297,9 +397,8 @@ minitez.query.files.shared=acid_globallimit.q,\ vector_mapjoin_reduce.q,\ vector_mr_diff_schema_alias.q,\ vector_multi_insert.q,\ - vector_non_string_partition.q,\ - vector_nullsafe_join.q,\ vector_null_projection.q,\ + vector_nullsafe_join.q,\ vector_nvl.q,\ vector_orderby_5.q,\ vector_outer_join0.q,\ @@ -311,13 +410,12 @@ minitez.query.files.shared=acid_globallimit.q,\ vector_outer_join6.q,\ vector_partition_diff_num_cols.q,\ vector_partitioned_date_time.q,\ - vector_reduce_groupby_decimal.q,\ vector_reduce1.q,\ vector_reduce2.q,\ vector_reduce3.q,\ + vector_reduce_groupby_decimal.q,\ vector_string_concat.q,\ vector_struct_in.q,\ - vectorized_timestamp.q,\ vector_varchar_4.q,\ vector_varchar_mapjoin1.q,\ vector_varchar_simple.q,\ @@ -341,8 +439,6 @@ minitez.query.files.shared=acid_globallimit.q,\ vectorization_8.q,\ vectorization_9.q,\ vectorization_decimal_date.q,\ - vectorization_div0.q,\ - vectorization_limit.q,\ vectorization_nested_udf.q,\ vectorization_not.q,\ vectorization_part.q,\ @@ -356,6 +452,7 @@ minitez.query.files.shared=acid_globallimit.q,\ vectorized_context.q,\ vectorized_date_funcs.q,\ vectorized_distinct_gby.q,\ + vectorized_dynamic_partition_pruning.q,\ vectorized_mapjoin.q,\ vectorized_math_funcs.q,\ vectorized_nested_mapjoin.q,\ @@ -365,186 +462,66 @@ minitez.query.files.shared=acid_globallimit.q,\ vectorized_rcfile_columnar.q,\ vectorized_shufflejoin.q,\ vectorized_string_funcs.q,\ + vectorized_timestamp.q,\ vectorized_timestamp_funcs.q,\ - vectorized_timestamp_ints_casts.q,\ - auto_sortmerge_join_1.q,\ - auto_sortmerge_join_10.q,\ - auto_sortmerge_join_11.q,\ - auto_sortmerge_join_12.q,\ - auto_sortmerge_join_13.q,\ - auto_sortmerge_join_14.q,\ - auto_sortmerge_join_15.q,\ - auto_sortmerge_join_16.q,\ - auto_sortmerge_join_2.q,\ - auto_sortmerge_join_3.q,\ - auto_sortmerge_join_4.q,\ - auto_sortmerge_join_5.q,\ - auto_sortmerge_join_6.q,\ - auto_sortmerge_join_7.q,\ - auto_sortmerge_join_8.q,\ - auto_sortmerge_join_9.q,\ - auto_join30.q,\ - auto_join21.q,\ - auto_join29.q,\ - auto_join_filters.q,\ - auto_join_nulls.q,\ - union_type_chk.q - + vectorized_timestamp_ints_casts.q -minitez.query.files=acid_bucket_pruning.q,\ +minillap.query.files=acid_bucket_pruning.q,\ acid_vectorization_missing_cols.q,\ bucket_map_join_tez1.q,\ - smb_cache.q,\ bucket_map_join_tez2.q,\ + bucketpruning1.q,\ constprog_dpp.q,\ dynamic_partition_pruning.q,\ dynamic_partition_pruning_2.q,\ - bucketpruning1.q,\ explainuser_1.q,\ explainuser_2.q,\ - explainuser_3.q,\ explainuser_4.q,\ hybridgrace_hashjoin_1.q,\ hybridgrace_hashjoin_2.q,\ llap_nullscan.q,\ + llap_udf.q,\ llapdecider.q,\ lvj_mapjoin.q,\ mapjoin_decimal.q,\ mergejoin_3way.q,\ mrr.q,\ - orc_ppd_basic.q,\ + orc_llap.q,\ + orc_llap_counters.q,\ + orc_llap_counters1.q,\ + orc_llap_nonvector.q,\ orc_merge_diff_fs.q,\ - partition_column_names_with_leading_and_trailing_spaces.q,\ - stats_filemetadata.q,\ + orc_ppd_basic.q,\ + smb_cache.q,\ tez_aggr_part_stats.q,\ tez_bmj_schema_evolution.q,\ tez_dml.q,\ - tez_fsstat.q,\ - tez_insert_overwrite_local_directory_1.q,\ tez_dynpart_hashjoin_1.q,\ tez_dynpart_hashjoin_2.q,\ tez_dynpart_hashjoin_3.q,\ - tez_vector_dynpart_hashjoin_1.q,\ - tez_vector_dynpart_hashjoin_2.q,\ + tez_fsstat.q,\ + tez_insert_overwrite_local_directory_1.q,\ + tez_join.q,\ tez_join_result_complex.q,\ tez_join_tests.q,\ tez_joins_explain.q,\ + tez_multi_union.q,\ tez_schema_evolution.q,\ tez_self_join.q,\ + tez_smb_1.q,\ + tez_smb_empty.q,\ + tez_smb_main.q,\ tez_union.q,\ tez_union2.q,\ - tez_union_dynamic_partition.q,\ - tez_union_view.q,\ - tez_union_with_udf.q,\ tez_union_decimal.q,\ + tez_union_dynamic_partition.q,\ tez_union_group_by.q,\ - tez_smb_main.q,\ - tez_smb_1.q,\ - tez_smb_empty.q,\ - vector_join_part_col_char.q,\ - vectorized_dynamic_partition_pruning.q,\ - tez_multi_union.q,\ - tez_join.q,\ tez_union_multiinsert.q,\ - windowing_gby.q - - - - -minillap.shared.query.files=bucket_map_join_tez1.q,\ - bucket_map_join_tez2.q,\ - constprog_dpp.q,\ - cte_1.q,\ - cte_2.q,\ - cte_3.q,\ - cte_4.q,\ - cte_5.q,\ - cte_mat_1.q,\ - cte_mat_2.q,\ - cte_mat_3.q,\ - cte_mat_4.q,\ - cte_mat_5.q,\ - dynamic_partition_pruning.q,\ - dynamic_partition_pruning_2.q,\ - hybridgrace_hashjoin_1.q,\ - hybridgrace_hashjoin_2.q,\ - mapjoin_decimal.q,\ - lvj_mapjoin.q,\ - llapdecider.q,\ - llap_nullscan.q,\ - mrr.q,\ - orc_merge12.q,\ - orc_ppd_basic.q,\ - orc_ppd_schema_evol_1a.q,\ - orc_ppd_schema_evol_1b.q,\ - orc_ppd_schema_evol_2a.q,\ - orc_ppd_schema_evol_2b.q,\ - orc_ppd_schema_evol_3a.q,\ - schema_evol_stats.q,\ - schema_evol_orc_acid_mapwork_part.q,\ - schema_evol_orc_acid_mapwork_table.q,\ - schema_evol_orc_acidvec_mapwork_part.q,\ - schema_evol_orc_acidvec_mapwork_table.q,\ - schema_evol_orc_nonvec_fetchwork_part.q,\ - schema_evol_orc_nonvec_fetchwork_table.q,\ - schema_evol_orc_nonvec_mapwork_part.q,\ - schema_evol_orc_nonvec_mapwork_part_all_complex.q,\ - schema_evol_orc_nonvec_mapwork_part_all_primitive.q,\ - schema_evol_orc_nonvec_mapwork_table.q,\ - schema_evol_orc_vec_mapwork_part.q,\ - schema_evol_orc_vec_mapwork_part_all_complex.q,\ - schema_evol_orc_vec_mapwork_part_all_primitive.q,\ - schema_evol_orc_vec_mapwork_table.q,\ - schema_evol_text_nonvec_mapwork_part.q,\ - schema_evol_text_nonvec_mapwork_part_all_complex.q,\ - schema_evol_text_nonvec_mapwork_part_all_primitive.q,\ - schema_evol_text_nonvec_mapwork_table.q,\ - schema_evol_text_vec_mapwork_part.q,\ - schema_evol_text_vec_mapwork_part_all_complex.q,\ - schema_evol_text_vec_mapwork_part_all_primitive.q,\ - schema_evol_text_vec_mapwork_table.q,\ - schema_evol_text_vecrow_mapwork_part.q,\ - schema_evol_text_vecrow_mapwork_part_all_complex.q,\ - schema_evol_text_vecrow_mapwork_part_all_primitive.q,\ - schema_evol_text_vecrow_mapwork_table.q,\ - tez_bmj_schema_evolution.q,\ - tez_dml.q,\ - tez_fsstat.q,\ - tez_insert_overwrite_local_directory_1.q,\ - tez_dynpart_hashjoin_1.q,\ - tez_dynpart_hashjoin_2.q,\ + tez_union_view.q,\ tez_vector_dynpart_hashjoin_1.q,\ tez_vector_dynpart_hashjoin_2.q,\ - tez_join_hash.q,\ - tez_join_result_complex.q,\ - tez_join_tests.q,\ - tez_joins_explain.q,\ - tez_schema_evolution.q,\ - tez_self_join.q,\ - tez_union.q,\ - tez_union2.q,\ - tez_union_dynamic_partition.q,\ - tez_union_view.q,\ - tez_union_decimal.q,\ - tez_union_group_by.q,\ - tez_smb_main.q,\ - tez_smb_1.q,\ - vector_complex_all.q,\ - vector_complex_join.q,\ - vector_include_no_sel.q,\ - vector_join_part_col_char.q,\ vectorized_dynamic_partition_pruning.q,\ - tez_multi_union.q,\ - tez_join.q,\ - tez_union_multiinsert.q - - -minillap.query.files=llap_udf.q,\ - orc_llap.q,\ - orc_llap_counters.q,\ - orc_llap_counters1.q,\ - orc_llap_nonvector.q - + windowing_gby.q encrypted.query.files=encryption_join_unencrypted_tbl.q,\ encryption_insert_partition_static.q,\ http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/constprog_semijoin.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/constprog_semijoin.q b/ql/src/test/queries/clientpositive/constprog_semijoin.q index 34f90a0..f282c27 100644 --- a/ql/src/test/queries/clientpositive/constprog_semijoin.q +++ b/ql/src/test/queries/clientpositive/constprog_semijoin.q @@ -1,4 +1,5 @@ set hive.optimize.constant.propagation=true; +set hive.explain.user=true; create table table1 (id int, val string, val1 string, dimid int); insert into table1 (id, val, val1, dimid) values (1, 't1val01', 'val101', 100), (2, 't1val02', 'val102', 200), (3, 't1val03', 'val103', 103), (3, 't1val01', 'val104', 100), (2, 't1val05', 'val105', 200), (3, 't1val01', 'val106', 103), (1, 't1val07', 'val107', 200), (2, 't1val01', 'val108', 200), (3, 't1val09', 'val109', 103), (4,'t1val01', 'val110', 200); http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_3.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/cte_3.q b/ql/src/test/queries/clientpositive/cte_3.q index aa8adbc..037c0cb 100644 --- a/ql/src/test/queries/clientpositive/cte_3.q +++ b/ql/src/test/queries/clientpositive/cte_3.q @@ -1,5 +1,6 @@ set hive.mapred.mode=nonstrict; set hive.optimize.cte.materialize.threshold=1; +set hive.explain.user=true; explain with q1 as ( select key from src where key = '5') http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_5.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/cte_5.q b/ql/src/test/queries/clientpositive/cte_5.q index 8968688..8f6ac15 100644 --- a/ql/src/test/queries/clientpositive/cte_5.q +++ b/ql/src/test/queries/clientpositive/cte_5.q @@ -1,5 +1,6 @@ set hive.mapred.mode=nonstrict; set hive.optimize.cte.materialize.threshold=-1; +set hive.explain.user=true; create database mydb; use mydb; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_1.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/cte_mat_1.q b/ql/src/test/queries/clientpositive/cte_mat_1.q index 2afb960..1131a4b 100644 --- a/ql/src/test/queries/clientpositive/cte_mat_1.q +++ b/ql/src/test/queries/clientpositive/cte_mat_1.q @@ -1,5 +1,6 @@ set hive.mapred.mode=nonstrict; set hive.optimize.cte.materialize.threshold=-1; +set hive.explain.user=true; explain with q1 as (select * from src where key= '5') http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_2.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/cte_mat_2.q b/ql/src/test/queries/clientpositive/cte_mat_2.q index adcd087..283aa40 100644 --- a/ql/src/test/queries/clientpositive/cte_mat_2.q +++ b/ql/src/test/queries/clientpositive/cte_mat_2.q @@ -1,5 +1,6 @@ set hive.mapred.mode=nonstrict; set hive.optimize.cte.materialize.threshold=3; +set hive.explain.user=true; explain with q1 as (select * from src where key= '5') http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_3.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/cte_mat_3.q b/ql/src/test/queries/clientpositive/cte_mat_3.q index 650cc24..a6c3a86 100644 --- a/ql/src/test/queries/clientpositive/cte_mat_3.q +++ b/ql/src/test/queries/clientpositive/cte_mat_3.q @@ -1,5 +1,6 @@ set hive.mapred.mode=nonstrict; set hive.optimize.cte.materialize.threshold=2; +set hive.explain.user=true; explain with q1 as (select * from src where key= '5') http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_4.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/cte_mat_4.q b/ql/src/test/queries/clientpositive/cte_mat_4.q index 2d75963..c52d9a2 100644 --- a/ql/src/test/queries/clientpositive/cte_mat_4.q +++ b/ql/src/test/queries/clientpositive/cte_mat_4.q @@ -1,5 +1,6 @@ set hive.mapred.mode=nonstrict; set hive.optimize.cte.materialize.threshold=2; +set hive.explain.user=true; create temporary table q1 (a int, b string); insert into q1 values (1, 'A'); http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_5.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/cte_mat_5.q b/ql/src/test/queries/clientpositive/cte_mat_5.q index fd0aeda..83d1eab 100644 --- a/ql/src/test/queries/clientpositive/cte_mat_5.q +++ b/ql/src/test/queries/clientpositive/cte_mat_5.q @@ -1,5 +1,6 @@ set hive.mapred.mode=nonstrict; set hive.optimize.cte.materialize.threshold=1; +set hive.explain.user=true; create database mydb; use mydb; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/deleteAnalyze.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/deleteAnalyze.q b/ql/src/test/queries/clientpositive/deleteAnalyze.q index 7e5371c..26123a6 100644 --- a/ql/src/test/queries/clientpositive/deleteAnalyze.q +++ b/ql/src/test/queries/clientpositive/deleteAnalyze.q @@ -1,4 +1,5 @@ set hive.stats.autogather=true; +set hive.explain.user=true; dfs ${system:test.dfs.mkdir} ${system:test.tmp.dir}/testdeci2; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/empty_join.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/empty_join.q b/ql/src/test/queries/clientpositive/empty_join.q index 0c372a0..088b66a 100644 --- a/ql/src/test/queries/clientpositive/empty_join.q +++ b/ql/src/test/queries/clientpositive/empty_join.q @@ -1,5 +1,6 @@ set hive.auto.convert.join=true; set hive.mapjoin.hybridgrace.hashtable=false; +set hive.explain.user=true; DROP TABLE IF EXISTS test_1; CREATE TABLE test_1 AS SELECT 1 AS id; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q b/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q index bbf33f9..ddb9d82 100644 --- a/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q +++ b/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q @@ -1,4 +1,5 @@ set hive.fetch.task.conversion=more; +set hive.explain.user=true; set hive.mapred.mode=nonstrict; explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10; explain select cast(key as int) * 10, upper(value) from src limit 10; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/smb_cache.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/smb_cache.q b/ql/src/test/queries/clientpositive/smb_cache.q index 5800f3f..e415e51 100644 --- a/ql/src/test/queries/clientpositive/smb_cache.q +++ b/ql/src/test/queries/clientpositive/smb_cache.q @@ -1,3 +1,4 @@ +set hive.explain.user=true; create table bug_201_input_a ( userid int ) clustered by (userid) sorted by (userid) into 64 BUCKETS ; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q b/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q index 01bd33d..8a63635 100644 --- a/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q +++ b/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q @@ -1,4 +1,5 @@ set hive.mapred.mode=nonstrict; +set hive.explain.user=true; create table testvec(id int, dt int, greg_dt string) stored as orc; insert into table testvec values @@ -12,4 +13,4 @@ values set hive.vectorized.execution.enabled=true; set hive.map.aggr=true; explain select max(dt), max(greg_dt) from testvec where id=5; -select max(dt), max(greg_dt) from testvec where id=5; \ No newline at end of file +select max(dt), max(greg_dt) from testvec where id=5; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q b/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q index d7e1e10..2077f8e 100644 --- a/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q +++ b/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q @@ -1,4 +1,5 @@ set hive.mapred.mode=nonstrict; +set hive.explain.user=true; SET hive.vectorized.execution.enabled=true; ; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_bround.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/vector_bround.q b/ql/src/test/queries/clientpositive/vector_bround.q index 9130540..deea00b 100644 --- a/ql/src/test/queries/clientpositive/vector_bround.q +++ b/ql/src/test/queries/clientpositive/vector_bround.q @@ -1,4 +1,5 @@ set hive.mapred.mode=nonstrict; +set hive.explain.user=true; create table test_vector_bround(v0 double, v1 double) stored as orc; insert into table test_vector_bround values @@ -12,4 +13,4 @@ values (3.51, 1.351); set hive.vectorized.execution.enabled=true; explain select bround(v0), bround(v1, 1) from test_vector_bround; -select bround(v0), bround(v1, 1) from test_vector_bround; \ No newline at end of file +select bround(v0), bround(v1, 1) from test_vector_bround; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q b/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q index a3cec04..53df2aa 100644 --- a/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q +++ b/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q @@ -19,4 +19,4 @@ select * from src where not key in (select key from src) -order by key; \ No newline at end of file +order by key; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_join_part_col_char.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/vector_join_part_col_char.q b/ql/src/test/queries/clientpositive/vector_join_part_col_char.q index 45a9165..5cfce37 100644 --- a/ql/src/test/queries/clientpositive/vector_join_part_col_char.q +++ b/ql/src/test/queries/clientpositive/vector_join_part_col_char.q @@ -2,6 +2,8 @@ set hive.mapred.mode=nonstrict; set hive.support.concurrency=true; set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager; set hive.vectorized.execution.enabled=true; +set hive.explain.user=true; +set hive.metastore.fastpath=false; drop table if exists char_part_tbl1 ; drop table if exists char_part_tbl2; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_struct_in.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/vector_struct_in.q b/ql/src/test/queries/clientpositive/vector_struct_in.q index 42ee938..50487db 100644 --- a/ql/src/test/queries/clientpositive/vector_struct_in.q +++ b/ql/src/test/queries/clientpositive/vector_struct_in.q @@ -1,4 +1,5 @@ set hive.cbo.enable=false; +set hive.explain.user=true; set hive.tez.dynamic.partition.pruning=false; set hive.vectorized.execution.enabled=true; SET hive.auto.convert.join=true; @@ -244,4 +245,4 @@ struct(1L, "b", 0.5D), struct(5L, "b", 1.5D), struct(1L, "a", 0.5D), struct(3L, "b", 1.5D) -) as b from test_4; \ No newline at end of file +) as b from test_4; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vectorized_parquet_types.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/vectorized_parquet_types.q b/ql/src/test/queries/clientpositive/vectorized_parquet_types.q index bf41958..297c5af 100644 --- a/ql/src/test/queries/clientpositive/vectorized_parquet_types.q +++ b/ql/src/test/queries/clientpositive/vectorized_parquet_types.q @@ -1,5 +1,6 @@ set hive.mapred.mode=nonstrict; SET hive.vectorized.execution.enabled=true; +set hive.explain.user=true; DROP TABLE parquet_types_staging; DROP TABLE parquet_types; @@ -80,4 +81,4 @@ SELECT ctinyint, MAX(cdecimal) FROM parquet_types GROUP BY ctinyint -ORDER BY ctinyint; \ No newline at end of file +ORDER BY ctinyint; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vectorized_timestamp.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/vectorized_timestamp.q b/ql/src/test/queries/clientpositive/vectorized_timestamp.q index baf0cfa..2784b7a 100644 --- a/ql/src/test/queries/clientpositive/vectorized_timestamp.q +++ b/ql/src/test/queries/clientpositive/vectorized_timestamp.q @@ -1,4 +1,5 @@ set hive.fetch.task.conversion=none; +set hive.explain.user=true; DROP TABLE IF EXISTS test; CREATE TABLE test(ts TIMESTAMP) STORED AS ORC; @@ -24,4 +25,4 @@ SELECT ts FROM test; EXPLAIN SELECT MIN(ts), MAX(ts), MAX(ts) - MIN(ts) FROM test; -SELECT MIN(ts), MAX(ts), MAX(ts) - MIN(ts) FROM test; \ No newline at end of file +SELECT MIN(ts), MAX(ts), MAX(ts) - MIN(ts) FROM test; http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/windowing_gby.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/windowing_gby.q b/ql/src/test/queries/clientpositive/windowing_gby.q index d844f11..e1a536b 100644 --- a/ql/src/test/queries/clientpositive/windowing_gby.q +++ b/ql/src/test/queries/clientpositive/windowing_gby.q @@ -1,4 +1,5 @@ set hive.mapred.mode=nonstrict; +set hive.explain.user=true; explain select rank() over (order by return_ratio) as return_rank from (select sum(wr.cint)/sum(ws.c_int) as return_ratio http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out b/ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out new file mode 100644 index 0000000..4fc4114 --- /dev/null +++ b/ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out @@ -0,0 +1,153 @@ +PREHOOK: query: -- Bucket pruning only works for ACID when split-update (U=D+I) has been enabled for the table. +-- For e.g., this can be done by setting 'transactional_properties' = 'default'. +-- This also means that bucket pruning will not work for ACID tables with legacy behaviour. + +CREATE TABLE acidTblDefault(a INT) CLUSTERED BY(a) INTO 16 BUCKETS STORED AS ORC TBLPROPERTIES ('transactional'='true', 'transactional_properties'='default') +PREHOOK: type: CREATETABLE +PREHOOK: Output: database:default +PREHOOK: Output: default@acidTblDefault +POSTHOOK: query: -- Bucket pruning only works for ACID when split-update (U=D+I) has been enabled for the table. +-- For e.g., this can be done by setting 'transactional_properties' = 'default'. +-- This also means that bucket pruning will not work for ACID tables with legacy behaviour. + +CREATE TABLE acidTblDefault(a INT) CLUSTERED BY(a) INTO 16 BUCKETS STORED AS ORC TBLPROPERTIES ('transactional'='true', 'transactional_properties'='default') +POSTHOOK: type: CREATETABLE +POSTHOOK: Output: database:default +POSTHOOK: Output: default@acidTblDefault +PREHOOK: query: INSERT INTO TABLE acidTblDefault SELECT cint FROM alltypesorc WHERE cint IS NOT NULL ORDER BY cint +PREHOOK: type: QUERY +PREHOOK: Input: default@alltypesorc +PREHOOK: Output: default@acidtbldefault +POSTHOOK: query: INSERT INTO TABLE acidTblDefault SELECT cint FROM alltypesorc WHERE cint IS NOT NULL ORDER BY cint +POSTHOOK: type: QUERY +POSTHOOK: Input: default@alltypesorc +POSTHOOK: Output: default@acidtbldefault +POSTHOOK: Lineage: acidtbldefault.a SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cint, type:int, comment:null), ] +PREHOOK: query: INSERT INTO TABLE acidTblDefault VALUES (1) +PREHOOK: type: QUERY +PREHOOK: Input: default@values__tmp__table__1 +PREHOOK: Output: default@acidtbldefault +POSTHOOK: query: INSERT INTO TABLE acidTblDefault VALUES (1) +POSTHOOK: type: QUERY +POSTHOOK: Input: default@values__tmp__table__1 +POSTHOOK: Output: default@acidtbldefault +POSTHOOK: Lineage: acidtbldefault.a EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ] +PREHOOK: query: -- Exactly one of the buckets should be selected out of the 16 buckets +-- by the following selection query. +EXPLAIN EXTENDED +SELECT * FROM acidTblDefault WHERE a = 1 +PREHOOK: type: QUERY +POSTHOOK: query: -- Exactly one of the buckets should be selected out of the 16 buckets +-- by the following selection query. +EXPLAIN EXTENDED +SELECT * FROM acidTblDefault WHERE a = 1 +POSTHOOK: type: QUERY +STAGE DEPENDENCIES: + Stage-1 is a root stage + Stage-0 depends on stages: Stage-1 + +STAGE PLANS: + Stage: Stage-1 + Tez +#### A masked pattern was here #### + Vertices: + Map 1 + Map Operator Tree: + TableScan + alias: acidtbldefault + filterExpr: (a = 1) (type: boolean) + buckets included: [1,] of 16 + Statistics: Num rows: 8983 Data size: 35932 Basic stats: COMPLETE Column stats: NONE + GatherStats: false + Filter Operator + isSamplingPred: false + predicate: (a = 1) (type: boolean) + Statistics: Num rows: 4491 Data size: 17964 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: 1 (type: int) + outputColumnNames: _col0 + Statistics: Num rows: 4491 Data size: 17964 Basic stats: COMPLETE Column stats: NONE + File Output Operator + compressed: false + GlobalTableId: 0 +#### A masked pattern was here #### + NumFilesPerFileSink: 1 + Statistics: Num rows: 4491 Data size: 17964 Basic stats: COMPLETE Column stats: NONE +#### A masked pattern was here #### + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + properties: + columns _col0 + columns.types int + escape.delim \ + hive.serialization.extend.additional.nesting.levels true + serialization.escape.crlf true + serialization.format 1 + serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe + serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe + TotalFiles: 1 + GatherStats: false + MultiFileSpray: false + Execution mode: llap + LLAP IO: may be used (ACID table) + Path -> Alias: +#### A masked pattern was here #### + Path -> Partition: +#### A masked pattern was here #### + Partition + base file name: acidtbldefault + input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat + output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat + properties: + bucket_count 16 + bucket_field_name a + columns a + columns.comments + columns.types int +#### A masked pattern was here #### + name default.acidtbldefault + numFiles 17 + numRows 0 + rawDataSize 0 + serialization.ddl struct acidtbldefault { i32 a} + serialization.format 1 + serialization.lib org.apache.hadoop.hive.ql.io.orc.OrcSerde + totalSize 35932 + transactional true + transactional_properties default +#### A masked pattern was here #### + serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde + + input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat + output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat + properties: + bucket_count 16 + bucket_field_name a + columns a + columns.comments + columns.types int +#### A masked pattern was here #### + name default.acidtbldefault + numFiles 17 + numRows 0 + rawDataSize 0 + serialization.ddl struct acidtbldefault { i32 a} + serialization.format 1 + serialization.lib org.apache.hadoop.hive.ql.io.orc.OrcSerde + totalSize 35932 + transactional true + transactional_properties default +#### A masked pattern was here #### + serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde + name: default.acidtbldefault + name: default.acidtbldefault + Truncated Path -> Alias: + /acidtbldefault [acidtbldefault] + + Stage: Stage-0 + Fetch Operator + limit: -1 + Processor Tree: + ListSink + http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/acid_globallimit.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/acid_globallimit.q.out b/ql/src/test/results/clientpositive/llap/acid_globallimit.q.out new file mode 100644 index 0000000..6f0e46e --- /dev/null +++ b/ql/src/test/results/clientpositive/llap/acid_globallimit.q.out @@ -0,0 +1,52 @@ +PREHOOK: query: -- Global Limit optimization does not work with ACID table. Make sure to skip it for ACID table. +CREATE TABLE acidtest1(c1 INT, c2 STRING) +CLUSTERED BY (c1) INTO 3 BUCKETS +STORED AS ORC +TBLPROPERTIES ("transactional"="true") +PREHOOK: type: CREATETABLE +PREHOOK: Output: database:default +PREHOOK: Output: default@acidtest1 +POSTHOOK: query: -- Global Limit optimization does not work with ACID table. Make sure to skip it for ACID table. +CREATE TABLE acidtest1(c1 INT, c2 STRING) +CLUSTERED BY (c1) INTO 3 BUCKETS +STORED AS ORC +TBLPROPERTIES ("transactional"="true") +POSTHOOK: type: CREATETABLE +POSTHOOK: Output: database:default +POSTHOOK: Output: default@acidtest1 +PREHOOK: query: insert into table acidtest1 select cint, cstring1 from alltypesorc where cint is not null order by cint +PREHOOK: type: QUERY +PREHOOK: Input: default@alltypesorc +PREHOOK: Output: default@acidtest1 +POSTHOOK: query: insert into table acidtest1 select cint, cstring1 from alltypesorc where cint is not null order by cint +POSTHOOK: type: QUERY +POSTHOOK: Input: default@alltypesorc +POSTHOOK: Output: default@acidtest1 +POSTHOOK: Lineage: acidtest1.c1 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cint, type:int, comment:null), ] +POSTHOOK: Lineage: acidtest1.c2 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cstring1, type:string, comment:null), ] +PREHOOK: query: select cast (c1 as string) from acidtest1 limit 10 +PREHOOK: type: QUERY +PREHOOK: Input: default@acidtest1 +#### A masked pattern was here #### +POSTHOOK: query: select cast (c1 as string) from acidtest1 limit 10 +POSTHOOK: type: QUERY +POSTHOOK: Input: default@acidtest1 +#### A masked pattern was here #### +-340178543 +-339581189 +-339244391 +356535438 +-338184935 +-338131778 +356416560 +844686816 +355274340 +-821957276 +PREHOOK: query: drop table acidtest1 +PREHOOK: type: DROPTABLE +PREHOOK: Input: default@acidtest1 +PREHOOK: Output: default@acidtest1 +POSTHOOK: query: drop table acidtest1 +POSTHOOK: type: DROPTABLE +POSTHOOK: Input: default@acidtest1 +POSTHOOK: Output: default@acidtest1 http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/acid_vectorization_missing_cols.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/acid_vectorization_missing_cols.q.out b/ql/src/test/results/clientpositive/llap/acid_vectorization_missing_cols.q.out new file mode 100644 index 0000000..6acc6e3 --- /dev/null +++ b/ql/src/test/results/clientpositive/llap/acid_vectorization_missing_cols.q.out @@ -0,0 +1,56 @@ +PREHOOK: query: drop table if exists newtable +PREHOOK: type: DROPTABLE +POSTHOOK: query: drop table if exists newtable +POSTHOOK: type: DROPTABLE +PREHOOK: query: create table newtable( + a string, + b int, + c double) +row format delimited +fields terminated by '\t' +stored as textfile +PREHOOK: type: CREATETABLE +PREHOOK: Output: database:default +PREHOOK: Output: default@newtable +POSTHOOK: query: create table newtable( + a string, + b int, + c double) +row format delimited +fields terminated by '\t' +stored as textfile +POSTHOOK: type: CREATETABLE +POSTHOOK: Output: database:default +POSTHOOK: Output: default@newtable +PREHOOK: query: drop table if exists newtable_acid +PREHOOK: type: DROPTABLE +POSTHOOK: query: drop table if exists newtable_acid +POSTHOOK: type: DROPTABLE +PREHOOK: query: create table newtable_acid (b int, a varchar(50),c decimal(3,2), d int) +clustered by (b) into 2 buckets +stored as orc +tblproperties ('transactional'='true') +PREHOOK: type: CREATETABLE +PREHOOK: Output: database:default +PREHOOK: Output: default@newtable_acid +POSTHOOK: query: create table newtable_acid (b int, a varchar(50),c decimal(3,2), d int) +clustered by (b) into 2 buckets +stored as orc +tblproperties ('transactional'='true') +POSTHOOK: type: CREATETABLE +POSTHOOK: Output: database:default +POSTHOOK: Output: default@newtable_acid +PREHOOK: query: insert into newtable_acid(a,b,c) +select * from newtable +PREHOOK: type: QUERY +PREHOOK: Input: default@newtable +PREHOOK: Output: default@newtable_acid +POSTHOOK: query: insert into newtable_acid(a,b,c) +select * from newtable +POSTHOOK: type: QUERY +POSTHOOK: Input: default@newtable +POSTHOOK: Output: default@newtable_acid +POSTHOOK: Lineage: newtable_acid.a EXPRESSION [(newtable)newtable.FieldSchema(name:a, type:string, comment:null), ] +POSTHOOK: Lineage: newtable_acid.b SIMPLE [(newtable)newtable.FieldSchema(name:b, type:int, comment:null), ] +POSTHOOK: Lineage: newtable_acid.c EXPRESSION [(newtable)newtable.FieldSchema(name:c, type:double, comment:null), ] +POSTHOOK: Lineage: newtable_acid.d SIMPLE [] http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out b/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out index 7e30942..caa41b2 100644 --- a/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out +++ b/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out @@ -64,12 +64,10 @@ POSTHOOK: Lineage: src_orc_merge_test_part PARTITION(ds=2012-01-03,ts=2012-01-03 PREHOOK: query: select count(1) from src_orc_merge_test_part where ds='2012-01-03' and ts='2012-01-03+14:46:31' PREHOOK: type: QUERY PREHOOK: Input: default@src_orc_merge_test_part -PREHOOK: Input: default@src_orc_merge_test_part@ds=2012-01-03/ts=2012-01-03+14%3A46%3A31 #### A masked pattern was here #### POSTHOOK: query: select count(1) from src_orc_merge_test_part where ds='2012-01-03' and ts='2012-01-03+14:46:31' POSTHOOK: type: QUERY POSTHOOK: Input: default@src_orc_merge_test_part -POSTHOOK: Input: default@src_orc_merge_test_part@ds=2012-01-03/ts=2012-01-03+14%3A46%3A31 #### A masked pattern was here #### 610 PREHOOK: query: select sum(hash(key)), sum(hash(value)) from src_orc_merge_test_part where ds='2012-01-03' and ts='2012-01-03+14:46:31' http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out b/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out index cefe069..0d5ba01 100644 --- a/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out +++ b/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out @@ -89,7 +89,7 @@ Retention: 0 #### A masked pattern was here #### Table Type: MANAGED_TABLE Table Parameters: - COLUMN_STATS_ACCURATE true + COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} numFiles 3 numRows 1500 rawDataSize 141000 @@ -140,7 +140,7 @@ Retention: 0 #### A masked pattern was here #### Table Type: MANAGED_TABLE Table Parameters: - COLUMN_STATS_ACCURATE true + COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} numFiles 1 numRows 1500 rawDataSize 141000 @@ -241,7 +241,7 @@ Database: default Table: src_orc_merge_test_part_stat #### A masked pattern was here #### Partition Parameters: - COLUMN_STATS_ACCURATE true + COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} numFiles 3 numRows 1500 rawDataSize 141000 @@ -290,7 +290,7 @@ Database: default Table: src_orc_merge_test_part_stat #### A masked pattern was here #### Partition Parameters: - COLUMN_STATS_ACCURATE true + COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} numFiles 3 numRows 1500 rawDataSize 141000 @@ -347,7 +347,7 @@ Database: default Table: src_orc_merge_test_part_stat #### A masked pattern was here #### Partition Parameters: - COLUMN_STATS_ACCURATE true + COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"} numFiles 1 numRows 1500 rawDataSize 141000 http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join0.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/auto_join0.q.out b/ql/src/test/results/clientpositive/llap/auto_join0.q.out index f5be383..3dda24a 100644 --- a/ql/src/test/results/clientpositive/llap/auto_join0.q.out +++ b/ql/src/test/results/clientpositive/llap/auto_join0.q.out @@ -32,6 +32,7 @@ STAGE DEPENDENCIES: STAGE PLANS: Stage: Stage-1 Tez +#### A masked pattern was here #### Edges: Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (BROADCAST_EDGE) Reducer 3 <- Reducer 2 (SIMPLE_EDGE) @@ -43,43 +44,45 @@ STAGE PLANS: Map Operator Tree: TableScan alias: src - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Filter Operator predicate: (key < 10) (type: boolean) - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) outputColumnNames: _col0, _col1 - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: _col0 (type: string), _col1 (type: string) sort order: ++ - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE Execution mode: llap + LLAP IO: no inputs Map 5 Map Operator Tree: TableScan alias: src - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Filter Operator predicate: (key < 10) (type: boolean) - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) outputColumnNames: _col0, _col1 - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: _col0 (type: string), _col1 (type: string) sort order: ++ - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE Execution mode: llap + LLAP IO: no inputs Reducer 2 Execution mode: llap Reduce Operator Tree: Select Operator expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string) outputColumnNames: _col0, _col1 - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE Map Join Operator condition map: Inner Join 0 to 1 @@ -89,42 +92,41 @@ STAGE PLANS: outputColumnNames: _col0, _col1, _col2, _col3 input vertices: 1 Reducer 6 - Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE - HybridGraceHashJoin: true + Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string) sort order: ++++ - Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE Reducer 3 Execution mode: llap Reduce Operator Tree: Select Operator expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string) outputColumnNames: _col0, _col1, _col2, _col3 - Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator aggregations: sum(hash(_col0,_col1,_col2,_col3)) mode: hash outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator sort order: - Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE value expressions: _col0 (type: bigint) Reducer 4 - Execution mode: uber + Execution mode: llap Reduce Operator Tree: Group By Operator aggregations: sum(VALUE._col0) mode: mergepartial outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE File Output Operator compressed: false - Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Reducer 6 Execution mode: llap @@ -132,10 +134,10 @@ STAGE PLANS: Select Operator expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string) outputColumnNames: _col0, _col1 - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator sort order: - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE value expressions: _col0 (type: string), _col1 (type: string) Stage: Stage-0 http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join1.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/auto_join1.q.out b/ql/src/test/results/clientpositive/llap/auto_join1.q.out index 2a771fc..3624062 100644 --- a/ql/src/test/results/clientpositive/llap/auto_join1.q.out +++ b/ql/src/test/results/clientpositive/llap/auto_join1.q.out @@ -27,64 +27,66 @@ STAGE DEPENDENCIES: STAGE PLANS: Stage: Stage-1 Tez +#### A masked pattern was here #### Edges: - Map 1 <- Map 2 (BROADCAST_EDGE) + Map 2 <- Map 1 (BROADCAST_EDGE) #### A masked pattern was here #### Vertices: Map 1 Map Operator Tree: TableScan alias: src1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE + Filter Operator + predicate: key is not null (type: boolean) + Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE + Select Operator + expressions: key (type: string) + outputColumnNames: _col0 + Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE + Execution mode: llap + LLAP IO: no inputs + Map 2 + Map Operator Tree: + TableScan + alias: src2 + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Filter Operator predicate: key is not null (type: boolean) - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Map Join Operator condition map: Inner Join 0 to 1 keys: 0 _col0 (type: string) 1 _col0 (type: string) - outputColumnNames: _col1, _col2 + outputColumnNames: _col0, _col2 input vertices: - 1 Map 2 - Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE - HybridGraceHashJoin: true + 0 Map 1 + Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE Select Operator - expressions: UDFToInteger(_col2) (type: int), _col1 (type: string) + expressions: UDFToInteger(_col0) (type: int), _col2 (type: string) outputColumnNames: _col0, _col1 - Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 1219 Data size: 115805 Basic stats: COMPLETE Column stats: COMPLETE File Output Operator compressed: false - Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 1219 Data size: 115805 Basic stats: COMPLETE Column stats: COMPLETE table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest_j1 Execution mode: llap - Map 2 - Map Operator Tree: - TableScan - alias: src1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Filter Operator - predicate: key is not null (type: boolean) - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: key (type: string) - outputColumnNames: _col0 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - Execution mode: llap + LLAP IO: no inputs Stage: Stage-2 Dependency Collection @@ -113,7 +115,7 @@ POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: default@dest_j1 POSTHOOK: Lineage: dest_j1.key EXPRESSION [(src)src1.FieldSchema(name:key, type:string, comment:default), ] -POSTHOOK: Lineage: dest_j1.value SIMPLE [(src)src1.FieldSchema(name:value, type:string, comment:default), ] +POSTHOOK: Lineage: dest_j1.value SIMPLE [(src)src2.FieldSchema(name:value, type:string, comment:default), ] PREHOOK: query: SELECT sum(hash(dest_j1.key,dest_j1.value)) FROM dest_j1 PREHOOK: type: QUERY PREHOOK: Input: default@dest_j1 http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join21.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/auto_join21.q.out b/ql/src/test/results/clientpositive/llap/auto_join21.q.out index 5ec385c..91ea004 100644 --- a/ql/src/test/results/clientpositive/llap/auto_join21.q.out +++ b/ql/src/test/results/clientpositive/llap/auto_join21.q.out @@ -15,6 +15,7 @@ STAGE DEPENDENCIES: STAGE PLANS: Stage: Stage-1 Tez +#### A masked pattern was here #### Edges: Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE) Reducer 3 <- Reducer 2 (SIMPLE_EDGE) @@ -24,41 +25,44 @@ STAGE PLANS: Map Operator Tree: TableScan alias: src1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: key (type: string) sort order: + Map-reduce partition columns: key (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE value expressions: value (type: string) Execution mode: llap + LLAP IO: no inputs Map 4 Map Operator Tree: TableScan alias: src2 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Filter Operator predicate: (key > 10) (type: boolean) - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: key (type: string) sort order: + Map-reduce partition columns: key (type: string) - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE value expressions: value (type: string) Execution mode: llap + LLAP IO: no inputs Map 5 Map Operator Tree: TableScan alias: src3 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: key (type: string) sort order: + Map-reduce partition columns: key (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE value expressions: value (type: string) Execution mode: llap + LLAP IO: no inputs Reducer 2 Execution mode: llap Reduce Operator Tree: @@ -75,28 +79,28 @@ STAGE PLANS: 1 key (type: string) 2 key (type: string) outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11 - Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string) outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5 - Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string) sort order: ++++++ - Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE Reducer 3 - Execution mode: uber + Execution mode: llap Reduce Operator Tree: Select Operator expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string) outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5 - Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE File Output Operator compressed: false - Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Stage: Stage-0
