[
https://issues.apache.org/jira/browse/HIVE-14877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543321#comment-15543321
]
Prasanth Jayachandran commented on HIVE-14877:
----------------------------------------------
Test runtime after moving to MiniLlap
||QFiles||TestCliDriver elapsed time||TestMiniLlapCliDriver elapsed
time||Comments||
|special_character_in_tabnames_1.q|419.229|62.453| |
|unionDistinct_1.q|278.583|54.171| |
|vector_leftsemi_mapjoin.q|232.313|57.86| |
|join_filters.q|172.436|31.507| |
|escape2.q|167.503|126.536| |
|archive_excludeHadoop20.q|163.522|349.996| (not moved - unsupported) |
|escape1.q|130.217|7.993| |
|lineage3.q|110.935|33.655| |
|insert_into_with_schema.q|107.345|10.33| |
|auto_join_filters.q|104.331|18.879| |
|windowing.q|99.622|17.476| |
|index_compact_binary_search.q|97.637|0.281| (not moved - unsupported) |
|cbo_rp_windowing_2.q|95.108|18.953| |
|vectorized_ptf.q|93.397|17.83| |
|dynpart_sort_optimization_acid.q|91.831|23.627| |
|partition_multilevels.q|90.392|15.522| |
|ptf.q|89.115|13.973| |
|sample_islocalmode_hook.q|88.293|1.527| (not moved - failing) |
|udaf_collect_set_2.q|84.725|8.893| |
|skewjoin.q|84.588|6.687| (not moved - unsupported) |
|lineage2.q|84.187|17.773| |
|correlationoptimizer1.q|80.367|9.688| |
|dynpart_sort_optimization.q|77.07|16.933| |
|orc_ppd_decimal.q|75.523|9.644| |
|orc_ppd_schema_evol_3a.q|75.352|11.042| |
|groupby_sort_skew_1_23.q|75.342|12.619| (not moved - unsupported) |
|cbo_rp_lineage2.q|75.283|13.133| |
|parquet_ppd_decimal.q|74.063|5.132| (not moved - results different) |
|sample_islocalmode_hook_use_metadata.q|73.988|1.572| (not moved - failing) |
|orc_analyze.q|73.803|7.258| |
|join_nulls.q|72.417|11.511| |
|semijoin.q|70.403|10.058| |
|correlationoptimizer6.q|69.151|13.689| |
|table_access_keys_stats.q|68.699|6.431| |
|autoColumnStats_2.q|68.632|15.836| |
|cbo_join.q|68.325|13.588| |
|cbo_rp_join.q|68.317|13.322| |
|sample10.q|64.513|2.43| |
|mergejoin.q|63.647|12.95| (not moved - explain looks different) |
|multi_insert_move_tasks_share_dependencies.q|62.079|11.974| (not moved -
explain looks different) |
|union_view.q|61.772|0.173| |
|autoColumnStats_1.q|61.246|14.669| |
|groupby_sort_1_23.q|61.129|13.585| (not moved - #files different) |
|pcr.q|59.546|23.866| (not moved - diff in results) |
|vectorization_short_regress.q|58.775|30.378| |
|auto_sortmerge_join_9.q|58.3|18.07| |
|correlationoptimizer2.q|56.591|9.867| |
|alter_merge_stats_orc.q|55.202|12.148| |
|vector_join30.q|54.85|6.666| |
|selectDistinctStar.q|53.981|8.516| |
|vector_decimal_udf.q|53.879|10.767| |
|auto_join30.q|53.762|5.655| |
|subquery_notin.q|52.879|6.896| |
|cbo_rp_subq_not_in.q|52.609|5.157| |
|cbo_rp_gby.q|51.866|6.982| |
|cbo_subq_not_in.q|51.672|3.711| |
|cbo_gby.q|50.361|7.05| |
|infer_bucket_sort.q|49.158|18.89| (not moved - unsupported) |
|ptf_streaming.q|48.484|8.996| |
|join_1to1.q|48.268|8.846| (not moved - results different) |
|load_dyn_part5.q|47.796|18.926| |
|limit_join_transpose.q|47.517|9.057| |
|ppd_windowing2.q|47.318|6.27| (not moved - results different) |
|dynpart_sort_opt_vectorization.q|47.208|16.859| |
|vector_number_compare_projection.q|47.024|9.922| |
|correlationoptimizer4.q|45.472|9.19| |
|orc_ppd_date.q|45.19|7.914| |
|global_limit.q|44.438|12.555| |
|union_top_level.q|44.229|5.054| |
|llap_partitioned.q|44.139|26.325| |
|orc_ppd_timestamp.q|43.617|7.195| |
|parquet_ppd_date.q|43.539|5.213| (not moved - results different) |
|multiMapJoin2.q|43.036|8.711| |
|parquet_ppd_timestamp.q|42.665|3.527| (not moved - results different) |
|vector_partitioned_date_time.q|42.511|12.54| |
|auto_sortmerge_join_8.q|42.377|4.668| |
|create_view.q|42.23|11.445| (not moved - failing) |
|windowing_windowspec2.q|42.202|9.947| |
|multiMapJoin1.q|41.176|10.094| |
|vector_decimal_2.q|41.026|7.814| |
|bucket_groupby.q|40.565|7.341| |
|rcfile_merge2.q|39.782|11.38| |
|index_compact_2.q|39.765|1.508| (not moved - unsupported) |
|join_nullsafe.q|39.698|7.956| |
|vector_join_filters.q|39.343|9.178| |
|cbo_rp_auto_join1.q|39.308|1.563| (not moved - failing) |
|vector_auto_smb_mapjoin_14.q|39.17|6.585| |
|vector_udf1.q|38.988|7.532| |
|rcfile_createas1.q|38.932|9.957| |
|cbo_rp_semijoin.q|38.675|5.857| |
|auto_join_nulls.q|38.519|7.52| |
|cbo_rp_unionDistinct_2.q|37.815|4.328| |
|union_remove_26.q|37.672|5.359| |
|rcfile_merge3.q|37.373|11.3| |
|rcfile_merge4.q|37.194|4.317| |
|bucketsortoptimize_insert_2.q|37.187|7.928| |
|cbo_limit.q|37.038|5.17| |
|auto_sortmerge_join_6.q|36.663|8.501| |
|join43.q|36.656|6.192| |
> Move slow CliDriver tests to MiniLlap
> -------------------------------------
>
> Key: HIVE-14877
> URL: https://issues.apache.org/jira/browse/HIVE-14877
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 2.2.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Attachments: HIVE-14877.1.patch
>
>
> When analyzing the test runtimes, there are many CliDriver tests that shows
> up as stragglers and are slow. Most of these tests are not really testing the
> execution engine. For example special_character_in_tabnames_1.q is the
> slowest test case that takes 419s in CliDriver but only 62s in MiniLlap.
> Similarly there are many test cases that can benefit from fast runtimes. We
> should consider moving the tests that are not testing the execution engine to
> MiniLlap (assuming it provides significant performance benefit).
> Here is the list of top 100 slow tests based on build #1055
> ||QFiles||TestCliDriver elapsed time||
> |special_character_in_tabnames_1.q|419.229|
> |unionDistinct_1.q|278.583|
> |vector_leftsemi_mapjoin.q|232.313|
> |join_filters.q|172.436|
> |escape2.q|167.503|
> |archive_excludeHadoop20.q|163.522|
> |escape1.q|130.217|
> |lineage3.q|110.935|
> |insert_into_with_schema.q|107.345|
> |auto_join_filters.q|104.331|
> |windowing.q|99.622|
> |index_compact_binary_search.q|97.637|
> |cbo_rp_windowing_2.q|95.108|
> |vectorized_ptf.q|93.397|
> |dynpart_sort_optimization_acid.q|91.831|
> |partition_multilevels.q|90.392|
> |ptf.q|89.115|
> |sample_islocalmode_hook.q|88.293|
> |udaf_collect_set_2.q|84.725|
> |skewjoin.q|84.588|
> |lineage2.q|84.187|
> |correlationoptimizer1.q|80.367|
> |dynpart_sort_optimization.q|77.07|
> |orc_ppd_decimal.q|75.523|
> |orc_ppd_schema_evol_3a.q|75.352|
> |groupby_sort_skew_1_23.q|75.342|
> |cbo_rp_lineage2.q|75.283|
> |parquet_ppd_decimal.q|74.063|
> |sample_islocalmode_hook_use_metadata.q|73.988|
> |orc_analyze.q|73.803|
> |join_nulls.q|72.417|
> |semijoin.q|70.403|
> |correlationoptimizer6.q|69.151|
> |table_access_keys_stats.q|68.699|
> |autoColumnStats_2.q|68.632|
> |cbo_join.q|68.325|
> |cbo_rp_join.q|68.317|
> |sample10.q|64.513|
> |mergejoin.q|63.647|
> |multi_insert_move_tasks_share_dependencies.q|62.079|
> |union_view.q|61.772|
> |autoColumnStats_1.q|61.246|
> |groupby_sort_1_23.q|61.129|
> |pcr.q|59.546|
> |vectorization_short_regress.q|58.775|
> |auto_sortmerge_join_9.q|58.3|
> |correlationoptimizer2.q|56.591|
> |alter_merge_stats_orc.q|55.202|
> |vector_join30.q|54.85|
> |selectDistinctStar.q|53.981|
> |vector_decimal_udf.q|53.879|
> |auto_join30.q|53.762|
> |subquery_notin.q|52.879|
> |cbo_rp_subq_not_in.q|52.609|
> |cbo_rp_gby.q|51.866|
> |cbo_subq_not_in.q|51.672|
> |cbo_gby.q|50.361|
> |infer_bucket_sort.q|49.158|
> |ptf_streaming.q|48.484|
> |join_1to1.q|48.268|
> |load_dyn_part5.q|47.796|
> |limit_join_transpose.q|47.517|
> |ppd_windowing2.q|47.318|
> |dynpart_sort_opt_vectorization.q|47.208|
> |vector_number_compare_projection.q|47.024|
> |correlationoptimizer4.q|45.472|
> |orc_ppd_date.q|45.19|
> |global_limit.q|44.438|
> |union_top_level.q|44.229|
> |llap_partitioned.q|44.139|
> |orc_ppd_timestamp.q|43.617|
> |parquet_ppd_date.q|43.539|
> |multiMapJoin2.q|43.036|
> |parquet_ppd_timestamp.q|42.665|
> |vector_partitioned_date_time.q|42.511|
> |auto_sortmerge_join_8.q|42.377|
> |create_view.q|42.23|
> |windowing_windowspec2.q|42.202|
> |multiMapJoin1.q|41.176|
> |vector_decimal_2.q|41.026|
> |bucket_groupby.q|40.565|
> |rcfile_merge2.q|39.782|
> |index_compact_2.q|39.765|
> |join_nullsafe.q|39.698|
> |vector_join_filters.q|39.343|
> |cbo_rp_auto_join1.q|39.308|
> |vector_auto_smb_mapjoin_14.q|39.17|
> |vector_udf1.q|38.988|
> |rcfile_createas1.q|38.932|
> |cbo_rp_semijoin.q|38.675|
> |auto_join_nulls.q|38.519|
> |cbo_rp_unionDistinct_2.q|37.815|
> |union_remove_26.q|37.672|
> |rcfile_merge3.q|37.373|
> |rcfile_merge4.q|37.194|
> |bucketsortoptimize_insert_2.q|37.187|
> |cbo_limit.q|37.038|
> |auto_sortmerge_join_6.q|36.663|
> |join43.q|36.656|
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
