[ 
https://issues.apache.org/jira/browse/HIVE-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-14767:
-----------------------------------------
    Attachment:     (was: HIVE-14767.4.patch)

> Migrate slow MiniMr tests to faster options
> -------------------------------------------
>
>                 Key: HIVE-14767
>                 URL: https://issues.apache.org/jira/browse/HIVE-14767
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Tests
>    Affects Versions: 2.2.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-14767.1.patch, HIVE-14767.2.patch, 
> HIVE-14767.3.patch, HIVE-14767.4.patch
>
>
> After analyzing the latest test results 
> https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1200/testReport/org.apache.hadoop.hive.cli/
> there are many Minimr tests that are slow individually. There are also some 
> tests that does not have to run in minimr, some tests are testing hdfs 
> related features that can be migrated to MiniLlap (some are already in 
> MiniSpark). Since we have removed hadoop20 now some MiniMr tests can be moved 
> to TestCliDriver. We should keep the absolute minimum number of tests in 
> minimr and move other slow test to minillap (spark already has most of these 
> tests).
> These are the individual test runtimes for minimr tests
> {code}
> QFiles        TestMinimrCliDriver elapsed time (Build #1055)
> infer_bucket_sort_reducers_power_two.q        356.211
> bucket5.q     257.589
> infer_bucket_sort_bucketed_table.q    251.557
> bucketizedhiveinputformat.q   249.755
> infer_bucket_sort_map_operators.q     249.484
> index_bitmap_auto.q   212.9
> infer_bucket_sort_dyn_part.q  204.401
> skewjoin_onesideskew.q        198.428
> index_bitmap3.q       194.221
> truncate_column_buckets.q     193.966
> auto_sortmerge_join_16.q      171.094
> schemeAuthority.q     170.839
> leftsemijoin_mr.q     150.493
> join_acid_non_acid.q  143.232
> empty_dir_in_table.q  136.463
> bucketmapjoin6.q      129.786
> reduce_deduplicate.q  116.455
> exchgpartition2lel.q  115.678
> bucket_many.q 104.316
> infer_bucket_sort_merge.q     100.976
> quotedid_smb.q        95.801
> external_table_with_space_in_location_path.q  80.265
> schemeAuthority2.q    72.64
> root_dir_external_table.q     72.171
> parallel_orderby.q    71.662
> disable_merge_for_bucketing.q 69.25
> bucket_num_reducers.q 67.618
> remote_script.q       67.029
> groupby2.q    66.72
> bucket4.q     66.02
> udf_using.q   64.985
> file_with_header_footer.q     42.986
> infer_bucket_sort_num_buckets.q       38.795
> scriptfile1.q 35.317
> bucket6.q     34.963
> non_native_window_udf.q       34.129
> join1.q       33.831
> bucketmapjoin7.q      33.71
> bucket_num_reducers2.q        30.645
> list_bucket_dml_10.q  25.751
> insert_dir_distcp.q   25.723
> uber_reduce.q 16.481
> load_hdfs_file_with_space_in_the_name.q       3.899
> table_nonprintable.q  2.097
> load_fs2.q    1.947
> input16_cc.q  0.916
> temp_table_external.q 0.751
> import_exported_table.q       0.508
> scriptfile1_win.q     0.119
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to