I just discovered on Friday that pytest allows for a --durations=N
parameter, which will output the time it takes to execute the slowest N
tests to the console, so I ran both a core and exhaustive private build
over the weekend, and clocked the slowest 50 for each grouping of tests.
These are the results, tidied up to be more legible. (Since it's long,
I'll send another email with the slowest exhaustive tests.)
This was a question that had been asked recently. I wonder we should add
this to the default pytest args in run-all-tests.sh, to track where most
of our test time is being spent? Since it adds noise to the log, we
could pick a number smaller than 50.
------------------------------------------------------------------------------------------------------------------------
TIME TABLE FORMAT Slowest Core Parallel Tests
------------------------------------------------------------------------------------------------------------------------
214.71s text/none
data/test_compute_stats.py::TestComputeStats::test_compute_
194.68s text/none-unique_database0
data/test_ddl.py::TestDdlStatements::test_alter_tab
193.79s text/none
query_test/test_exprs.py::TestExprs::test_expr
185.03s parquet/none
query_test/test_queries.py::TestQueries::test_analytic_fn
164.04s text/none
data/test_show_create_table.py::TestShowCreateTable::test_show_create_tab
157.15s parquet/none
query_test/test_nested_types.py::TestNestedTypes::test_tpch
153.68s parquet/none
query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
143.73s text/none
query_test/test_kudu.py::TestKuduOperations::test_kudu_partition_dd
137.51s parquet/none
query_test/test_join_queries.py::TestTPCHJoinQueries::test_outer_join
133.13s text/none
query_test/test_exprs.py::TestExprs::test_expr
125.87s text/none
query_test/test_insert_permutation.py::TestInsertQueriesWithPermutation::test_insert_permutation
119.45s text/gzip/block
query_test/test_hdfs_caching.py::TestHdfsCaching::test_cache_cancellation
118.59s parquet/none
query_test/test_runtime_filters.py::TestRuntimeRowFilters::test_row_filter
117.99s text/none
query_test/test_hdfs_caching.py::TestHdfsCaching::test_cache_cancellation
104.71s text/none
data/test_compute_stats.py::TestComputeStats::test_compute_stats_incremen
95.86s avro/snap/block
query_test/test_runtime_filters.py::TestRuntimeFilters::test_basic_filter
89.58s rc/snap/block
query_test/test_runtime_filters.py::TestRuntimeFilters::test_basic_filter
88.74s text/none
query_test/test_kudu.py::TestKuduOperations::test_kudu_d
88.68s parquet/none
query_test/test_sort.py::TestQueryFullSort::test_multiple_mem_limits_full_outpu
83.88s seq/snap/block
query_test/test_scanners.py::TestScannersAllTableFormatsWithLimit::test_limi
83.45s text/none
query_test/test_kudu.py::TestKuduOperations::test_kudu_inser
83.00s text/none
query_test/test_runtime_filters.py::TestRuntimeFilters::test_basic_filter
79.15s text/none
query_test/test_scanners.py::TestScannersAllTableFormatsWithLimit::test_limi
79.15s parquet/none
query_test/test_runtime_filters.py::TestRuntimeFilters::test_basic_filter
78.20s seq/snap/block
query_test/test_runtime_filters.py::TestRuntimeFilters::test_basic_filter
76.24s text/none
query_test/test_udfs.py::TestUdfExecution::test_ir_function
74.27s parquet/none
query_test/test_sort.py::TestQueryFullSort::test_multiple_mem_limi
73.40s parquet/none
query_test/test_insert_parquet.py::TestInsertParquetQueries::test_insert_parqu
72.62s rc/snap/block
query_test/test_scanners.py::TestScannersAllTableFormatsWithLimit::test_limi
71.21s text/none
data/test_partition_metadata.py::TestPartitionMetadata::test_partition_metadata_compatibility
70.15s parquet/none
query_test/test_insert_parquet.py::TestInsertParquetQueries::test_insert_parqu
68.99s parquet/none
query_test/test_nested_types.py::TestNestedTypes::test_subplan
68.08s text/none
query_test/test_udfs.py::TestUdfExecution::test_native_function
67.68s parquet/none
query_test/test_sort.py::TestQueryFullSort::test_sort_union
65.39s text/none
query_test/test_udfs.py::TestUdfExecution::test_ir_function
64.80s text/none
query_test/test_udfs.py::TestUdfExecution::test_ir_function
63.34s parquet/none
query_test/test_scanners.py::TestScannersAllTableFormatsWithLimit::test_limi
63.16s text/none
query_test/test_errorlog.py::TestErrorLogs::test_errorlog
63.10s text/none
query_test/test_udfs.py::TestUdfExecution::test_native_function
63.08s text/none-unique_database0
data/test_ddl.py::TestDdlStatements::test_views_dd
62.97s parquet/none
query_test/test_insert_parquet.py::TestInsertParquetQueries::test_insert_parqu
61.12s text/none
query_test/test_kudu.py::TestKuduOperations::test_kudu_upser
61.10s parquet/none
query_test/test_insert_parquet.py::TestInsertParquetQueries::test_insert_parqu
61.05s text/none
query_test/test_kudu.py::TestKuduOperations::test_kudu_scan_nod
60.96s avro/snap/block
query_test/test_scanners.py::TestScannersAllTableFormatsWithLimit::test_limi
58.63s text/none
query_test/test_aggregation.py::TestAggregationQueries::test_aggregation
58.59s text/none
query_test/test_udfs.py::TestUdfExecution::test_native_function
58.42s text/none
query_test/test_udfs.py::TestUdfExecution::test_native_function
58.27s parquet/none
query_test/test_scanners_fuzz.py::TestScannersFuzzing::test_fuzz_allty
57.90s avro/snap/block
query_test/test_scanners_fuzz.py::TestScannersFuzzing::test_fuzz_allty
------------------------------------------------------------------------------------------------------------------------
TIME TABLE FORMAT Slowest Core Serial Tests
------------------------------------------------------------------------------------------------------------------------
301.99s parquet/none
query_test/test_queries.py::TestQueriesParquetTables::test_very_large_string
142.72s text/none
query_test/test_insert.py::TestInsertQueries::test_inser
109.29s text/none
data/test_metadata_query_statements.py::TestMetadataQueryStatements::test_describe_db
83.58s text/none
query_test/test_hdfs_caching.py::TestHdfsCachingDdl::test_cache_reload_validation
65.20s text/none
query_test/test_chars.py::TestStringQueries::test_varchar
63.93s
data/test_hdfs_encryption.py::TestHdfsEncryption::test_drop_partition_encry
52.34s text/none
data/test_hms_integration.py::TestHmsIntegrationSanity::test_sanity
52.33s text/none
query_test/test_hdfs_caching.py::TestHdfsCachingDdl::test_caching_dd
50.18s
query_test/test_insert_behaviour.py::TestInsertBehaviour::test_insert_select_with_empty_resu
49.66s text/none
query_test/test_hdfs_caching.py::TestHdfsCachingDdl::test_caching_ddl_drop_datab
45.89s text/none
query_test/test_kudu.py::TestKuduOperations::test_kudu_alter_tab
44.39s parquet/none
catalog_service/test_hms_failure.py::TestHiveMetaStoreFailure::test_hms_service_di
41.62s text/none
query_test/test_insert.py::TestInsertPartKey::test_insert_part_key
36.80s parquet/none
query_test/test_tpcds_queries.py::TestTpcdsQuery::test_tpcds_coun
33.72s text/none
data/test_hdfs_encryption.py::TestHdfsEncryption::test_load_d
31.41s text/none
query_test/test_insert.py::TestInsertNullQueries::test_insert_nu
30.68s text/none
data/test_ddl.py::TestLibCache::test_create_drop_data_src
29.40s
data/test_hdfs_encryption.py::TestHdfsEncryption::test_drop_table_encry
29.22s text/none
data/test_hdfs_encryption.py::TestHdfsEncryption::test_load_d
27.41s text/none
query_test/test_chars.py::TestStringQueries::test_varchar
24.98s
hs2/test_hs2.py::TestHS2::test_concurrent_session_mixed_idle_timeou
18.77s hbase/none
query_test/test_hbase_queries.py::TestHBaseQueries::test_hbase_inser
18.04s text/none
data/test_compute_stats.py::TestComputeStats::test_compute_stats_keyword
17.65s text/none
query_test/test_insert.py::TestInsertQueries::test_insert_overwri
14.16s parquet/none
query_test/test_cancellation.py::TestCancellationSerial::test_cancel_inser
13.58s text/none
failure/test_failpoints.py::TestFailpoints::test_failpoin
13.34s
hell/test_shell_commandline.py::TestImpalaShell::test_config_fi
11.77s kudu/none
failure/test_failpoints.py::TestFailpoints::test_failpoin
11.13s
hell/test_shell_interactive.py::TestImpalaShellInteractive::test_ddl_queries_are_closed
10.72s
hell/test_shell_commandline.py::TestImpalaShell::test_queries_closed
10.58s
hell/test_shell_interactive.py::TestImpalaShellInteractive::test_cancellation
10.02s kudu/none
failure/test_failpoints.py::TestFailpoints::test_failpoin
9.99s seq/snap/block
query_test/test_cancellation.py::TestCancellationSerial::test_cancel_inser
9.97s text/gzip/block
query_test/test_cancellation.py::TestCancellationSerial::test_cancel_inser
9.63s
hs2/test_fetch_first.py::TestFetchFirst::test_query_stmts_v1
9.01s text/none
query_test/test_cancellation.py::TestCancellationSerial::test_cancel_inser
9.01s avro/snap/block
query_test/test_cancellation.py::TestCancellationSerial::test_cancel_inser
8.95s avro/none
query_test/test_cancellation.py::TestCancellationSerial::test_cancel_inser
------------------------------------------------------------------------------------------------------------------------
TIME TABLE FORMAT Slowest Core Custom Cluster Tests
------------------------------------------------------------------------------------------------------------------------
685.22s
custom_cluster/test_permanent_udfs.py::TestUdfPersistence::test_java_udfs_hive_integration
667.85s parquet/none
custom_cluster/test_spilling.py::TestSpilling::test_spilling
117.89s
custom_cluster/test_permanent_udfs.py::TestUdfPersistence::test_java_udfs_from_i
100.75s text/none
uthorization/test_grant_revoke.py::TestGrantRevoke::test_grant_revok
53.79s
custom_cluster/test_redaction.py::TestRedaction::test_very_verbose_logging
49.12s
custom_cluster/test_permanent_udfs.py::TestUdfPersistence::test_permanent_udf
41.84s text/none
custom_cluster/test_scratch_disk.py::TestScratchDir::test_write_error_failover
39.62s text/none
custom_cluster/test_scratch_disk.py::TestScratchDir::test_non_writable_dir
39.55s text/none
custom_cluster/test_scratch_disk.py::TestScratchDir::test_multiple_dir
39.53s text/none
custom_cluster/test_scratch_disk.py::TestScratchDir::test_no_dir
38.73s text/none
custom_cluster/test_scratch_disk.py::TestScratchDir::test_non_existing_dir
37.12s text/none
custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_flag
33.42s text/none
custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_config
33.19s
custom_cluster/test_redaction.py::TestRedaction::test_redacted
32.50s text/none
custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi
26.87s
custom_cluster/test_redaction.py::TestRedaction::test_unredacted
22.28s text/none
custom_cluster/test_exchange_delays.py::TestExchangeDelays::test_exchange_large_delay
22.02s text/none
custom_cluster/test_alloc_fail.py::TestAllocFail::test_alloc_fail_ini
20.30s text/none
custom_cluster/test_exchange_delays.py::TestExchangeDelays::test_exchange_small_delay
18.07s
custom_cluster/test_redaction.py::TestRedaction::test_bad_ru
--
David Knupp
Software Engineer
Cloudera
415-312-1049
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
