BTW I did see Alex's comment on the related thread about this being CPU time rather than response time, but given how frequently the analytic fn tests show up on that list it seems fair to assume it's contributing a fair amount to the response time. It'll have to be tested, of course.
Filed https://issues.cloudera.org/browse/IMPALA-4960 on myself On Tue, Feb 21, 2017 at 1:20 PM, Matthew Jacobs <[email protected]> wrote: > Thanks David, this is interesting. > > I'll put up a patch to remove some of the tested file formats for > analytic functions since it shouldn't really matter too much, unless > there are differences in timing producing rows to the analytic > functions, but I'm not sure that's the right way to get that kind of > coverage anyway. I'll also re-evaluate if we can trim some of the > admission control test matrix combinations, I suspect there are some. > > On Tue, Feb 21, 2017 at 11:32 AM, David Knupp <[email protected]> wrote: >> ------------------------------------------------------------------------------------------------------------------------ >> TIME TABLE FORMAT Slowest Exhaustive Parallel Tests >> ------------------------------------------------------------------------------------------------------------------------ >> 1098.89s text/none >> data/test_views_compatibility.py::TestViewCompatibility::test_view_compatibility >> 833.21s text/none-unique_database0 >> data/test_ddl.py::TestDdlStatements::test_alter_tab >> 388.71s seq/def/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 387.68s rc/gzip/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 382.00s parquet/none >> query_test/test_join_queries.py::TestSemiJoinQueries::test_semi_joins_exhaustiv >> 381.82s parquet/none >> query_test/test_join_queries.py::TestSemiJoinQueries::test_semi_joins_exhaustiv >> 380.61s text/bzip/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 371.18s seq/none >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 370.11s text/none >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 369.31s seq/bzip/record >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 369.18s text/lzo/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 366.88s seq/gzip/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 366.27s seq/def/record >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 365.91s seq/bzip/record >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 363.56s seq/snap/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 359.66s seq/def/record >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 355.43s text/gzip/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 353.24s text/none >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 351.96s seq/def/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 349.25s text/snap/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 348.43s text/none >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 347.70s seq/def/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 347.54s seq/snap/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 344.59s seq/def/record >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 344.57s seq/gzip/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 344.56s seq/none >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 343.30s seq/gzip/record >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 342.96s rc/gzip/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 341.99s seq/bzip/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 341.57s seq/def/record >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 341.31s seq/bzip/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 340.44s seq/none >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 340.40s text/snap/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 337.42s text/none >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 336.99s seq/snap/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 336.35s seq/gzip/record >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 333.26s seq/gzip/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 332.55s text/lzo/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 332.35s seq/snap/record >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 330.21s text/lzo/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 329.77s seq/bzip/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 329.38s seq/gzip/record >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 325.61s text/lzo/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 325.36s seq/bzip/record >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 325.04s seq/bzip/block >> query_test/test_queries.py::TestHdfsQueries::test_analytic_fn >> 324.84s seq/gzip/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 321.08s text/bzip/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 319.84s text/gzip/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 318.10s rc/none >> query_test/test_queries.py::TestQueries::test_analytic_fn >> 316.15s rc/def/block >> query_test/test_queries.py::TestQueries::test_analytic_fn >> >> >> ------------------------------------------------------------------------------------------------------------------------ >> TIME TABLE FORMAT Slowest Exhaustive Serial Tests >> ------------------------------------------------------------------------------------------------------------------------ >> 389.59s parquet/none >> query_test/test_queries.py::TestQueriesParquetTables::test_very_large_string >> 311.88s parquet/none >> query_test/test_insert.py::TestInsertQueries::test_inser >> 304.41s parquet/none >> query_test/test_queries.py::TestQueriesParquetTables::test_very_large_string >> 154.64s text/none >> query_test/test_insert.py::TestInsertQueries::test_inser >> 148.05s parquet/none >> query_test/test_insert.py::TestInsertQueries::test_inser >> 144.27s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_impala_db_hive_table_add_partition >> 142.35s parquet/none >> query_test/test_insert.py::TestInsertQueries::test_inser >> 125.87s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_hive_db_hive_table_add_partition >> 119.99s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_drop_column_maintains_ >> 99.47s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_impala_db_impala_table_add_partition >> 99.25s text/none >> data/test_compute_stats.py::TestComputeStats::test_compute_stats_many_partition >> 98.33s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_change_parquet_column_ty >> 97.10s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_compute_stats_get_to_i >> 92.11s text/none >> data/test_metadata_query_statements.py::TestMetadataQueryStatements::test_describe_db >> 91.37s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_hive_db_impala_table_add_partition >> 85.77s text/none >> query_test/test_hdfs_caching.py::TestHdfsCachingDdl::test_cache_reload_validation >> 84.09s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_add_column >> 82.05s text/none >> data/test_metadata_query_statements.py::TestMetadataQueryStatements::test_describe_db >> 81.98s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_add_preexisting_partitions_with_d >> 78.03s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_change_column_ty >> 76.09s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_impala_partitions_accessible_in_hiv >> 74.82s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_table_format_chang >> 74.07s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_select_without_compute_ >> 71.81s >> data/test_hdfs_encryption.py::TestHdfsEncryption::test_drop_partition_encry >> 68.44s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_change_table_n >> 67.09s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_drop_partition >> 63.78s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_compute_stats_get_to_hiv >> 60.81s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_add_overlapping_partition >> 60.30s parquet/none >> query_test/test_insert.py::TestInsertQueries::test_insert_overwri >> 58.76s text/none >> query_test/test_hdfs_caching.py::TestHdfsCachingDdl::test_caching_dd >> 54.98s text/none >> query_test/test_chars.py::TestStringQueries::test_varchar >> 52.50s text/none >> data/test_hms_integration.py::TestHmsIntegrationSanity::test_sanity >> 50.16s >> query_test/test_insert_behaviour.py::TestInsertBehaviour::test_insert_select_with_empty_resu >> 50.09s text/none >> query_test/test_kudu.py::TestKuduOperations::test_kudu_alter_tab >> 49.81s text/none >> query_test/test_hdfs_caching.py::TestHdfsCachingDdl::test_caching_ddl_drop_datab >> 49.75s text/none >> query_test/test_kudu.py::TestKuduOperations::test_kudu_alter_tab >> 47.96s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_drop_column_with_d >> 47.68s parquet/none >> catalog_service/test_hms_failure.py::TestHiveMetaStoreFailure::test_hms_service_di >> 43.42s text/none >> data/test_hms_integration.py::TestHmsIntegration::test_add_impala_partition >> 43.31s text/snap/block >> query_test/test_compressed_formats.py::TestLargeCompressedFile::test_query_large_fi >> 41.05s text/none >> query_test/test_insert.py::TestInsertPartKey::test_insert_part_key >> 40.83s text/snap/block >> query_test/test_compressed_formats.py::TestLargeCompressedFile::test_query_large_fi >> 38.52s >> query_test/test_compressed_formats.py::TestCompressedFormats::test_compressed_for >> 37.67s >> query_test/test_compressed_formats.py::TestCompressedFormats::test_compressed_for >> 37.12s >> query_test/test_compressed_formats.py::TestCompressedFormats::test_compressed_for >> 36.44s text/none >> query_test/test_insert.py::TestInsertNullQueries::test_insert_nu >> 36.19s >> query_test/test_compressed_formats.py::TestCompressedFormats::test_compressed_for >> 35.94s >> query_test/test_compressed_formats.py::TestCompressedFormats::test_compressed_for >> >> >> ------------------------------------------------------------------------------------------------------------------------ >> TIME TABLE FORMAT Slowest Exhaustive Custom Cluster >> Tests >> ------------------------------------------------------------------------------------------------------------------------ >> 694.66s >> custom_cluster/test_permanent_udfs.py::TestUdfPersistence::test_java_udfs_hive_integration >> 678.02s parquet/none >> custom_cluster/test_spilling.py::TestSpilling::test_spilling >> 248.32s >> custom_cluster/test_breakpad.py::TestBreakpad::test_dcheck_writes_minidu >> 117.61s >> custom_cluster/test_permanent_udfs.py::TestUdfPersistence::test_java_udfs_from_i >> 84.63s text/none >> authorization/test_grant_revoke.py::TestGrantRevoke::test_grant_revok >> 60.03s >> custom_cluster/test_redaction.py::TestRedaction::test_very_verbose_logging >> 58.19s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_config >> 56.01s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_config >> 55.04s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_flag >> 54.06s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi >> 53.99s text/none >> custom_cluster/test_rpc_timeout.py::TestRPCTimeout::test_random_rpc_timeou >> 52.05s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_flag >> 48.34s >> custom_cluster/test_permanent_udfs.py::TestUdfPersistence::test_permanent_udf >> 47.39s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_config >> 45.23s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi >> 45.12s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi >> 43.06s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_config >> 41.79s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_flag >> 41.61s text/none >> custom_cluster/test_scratch_disk.py::TestScratchDir::test_write_error_failover >> 41.42s text/none >> custom_cluster/test_scratch_disk.py::TestScratchDir::test_multiple_dir >> 41.33s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi >> 41.02s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi >> 39.96s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_flag >> 39.85s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_flag >> 39.02s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_config >> 38.97s text/none >> custom_cluster/test_scratch_disk.py::TestScratchDir::test_non_existing_dir >> 38.72s text/none >> custom_cluster/test_scratch_disk.py::TestScratchDir::test_non_writable_dir >> 38.48s text/none >> custom_cluster/test_scratch_disk.py::TestScratchDir::test_no_dir >> 35.45s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi >> 34.79s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_config >> 34.70s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_config >> 34.19s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi >> 34.13s >> custom_cluster/test_permanent_udfs.py::TestUdfPersistence::test_corrupt_java_udf >> 33.95s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_flag >> 33.07s >> custom_cluster/test_redaction.py::TestRedaction::test_redacted >> 32.82s >> custom_cluster/test_breakpad.py::TestBreakpad::test_limit_minidump_siz >> 31.83s >> custom_cluster/test_breakpad.py::TestBreakpad::test_minidump_cleanu >> 31.30s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_flag >> 30.46s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_config >> 30.40s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi >> 30.15s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_mem_limi >> 29.58s text/none >> custom_cluster/test_rpc_timeout.py::TestRPCTimeout::test_transmitdata_timeou >> 28.66s text/none >> custom_cluster/test_admission_controller.py::TestAdmissionControllerStress::test_admission_controller_with_flag<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach> >>
