[
https://issues.apache.org/jira/browse/IMPALA-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-10620:
------------------------------------
Component/s: Backend
> TestExecutorGroups.test_executor_group_num_queries_executing_metric fails in
> exhaustive release build
> -----------------------------------------------------------------------------------------------------
>
> Key: IMPALA-10620
> URL: https://issues.apache.org/jira/browse/IMPALA-10620
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Quanlong Huang
> Priority: Critical
>
> custom_cluster.test_executor_groups.TestExecutorGroups.test_executor_group_num_queries_executing_metric
> fails in an exhaustive release build.
> *Stacktrace*
> {code:java}
> custom_cluster/test_executor_groups.py:436: in
> test_executor_group_num_queries_executing_metric
> assert self._get_num_queries_executing_for_exec_group(group_names[0]) ==
> 1, \
> E AssertionError: The lingering query should be accounted for when the
> group comes back up.
> E assert 0 == 1
> E + where 0 = <bound method
> TestExecutorGroups._get_num_queries_executing_for_exec_group of
> <test_executor_groups.TestExecutorGroups object at 0x7f870cecba50>>('group1')
> E + where <bound method
> TestExecutorGroups._get_num_queries_executing_for_exec_group of
> <test_executor_groups.TestExecutorGroups object at 0x7f870cecba50>> =
> <test_executor_groups.TestExecutorGroups object at
> 0x7f870cecba50>._get_num_queries_executing_for_exec_group{code}
> *Standard Error*
> {code}
> -- 2021-03-29 03:45:18,032 INFO MainThread: Starting cluster with
> command:
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/bin/start-impala-cluster.py
> '--state_store_args=--statestore_update_frequency_ms=50
> --statestore_priority_update_frequency_ms=50
> --statestore_heartbeat_frequency_ms=50' --cluster_size=1 --num_coordinators=1
> --log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests
> --log_level=1 --use_exclusive_coordinators
> --impalad_args=--enable_orc_scanner=true '--impalad_args=
> -executor_groups=coordinator ' --impalad_args=--default_query_options=
> 03:45:18 MainThread: Starting impala cluster without executors
> 03:45:18 MainThread: Found 0 impalad/0 statestored/0 catalogd process(es)
> 03:45:18 MainThread: Starting State Store logging to
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 03:45:18 MainThread: Starting Catalog Service logging to
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 03:45:18 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 03:45:21 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es)
> 03:45:21 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es)
> 03:45:21 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> 03:45:21 MainThread: Waiting for num_known_live_backends=1. Current value: 0
> 03:45:22 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es)
> 03:45:22 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> 03:45:22 MainThread: Waiting for num_known_live_backends=1. Current value: 0
> 03:45:23 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es)
> 03:45:23 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> 03:45:23 MainThread: num_known_live_backends has reached value: 1
> 03:45:24 MainThread: Impala Cluster Running with 1 nodes (1 coordinators, 0
> executors).
> -- 2021-03-29 03:45:25,419 DEBUG MainThread: Found 1 impalad/1
> statestored/1 catalogd process(es)
> -- 2021-03-29 03:45:25,419 INFO MainThread: Getting metric:
> statestore.live-backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25010
> -- 2021-03-29 03:45:25,422 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:25,425 INFO MainThread: Metric
> 'statestore.live-backends' has reached desired value: 2
> -- 2021-03-29 03:45:25,425 DEBUG MainThread: Getting
> num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:25,428 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:25,430 INFO MainThread: num_known_live_backends has
> reached value: 1
> SET
> client_identifier=custom_cluster/test_executor_groups.py::TestExecutorGroups::()::test_executor_group_num_queries_executing_metric;
> -- connecting to: localhost:21000
> -- connecting to localhost:21050 with impyla
> -- 2021-03-29 03:45:25,640 INFO MainThread: Closing active operation
> -- connecting to localhost:28000 with impyla
> -- 2021-03-29 03:45:25,769 INFO MainThread: Closing active operation
> -- 2021-03-29 03:45:25,778 INFO MainThread: Adding 1 executors to group
> default-pool-group1 with minimum size 1
> -- 2021-03-29 03:45:25,778 INFO MainThread: Starting cluster with
> command:
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/bin/start-impala-cluster.py
> '--state_store_args=--statestore_update_frequency_ms=50
> --statestore_priority_update_frequency_ms=50
> --statestore_heartbeat_frequency_ms=50' --cluster_size=1 --num_coordinators=0
> --log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests
> --log_level=1 --add_executors --impalad_args=--enable_orc_scanner=true
> --impalad_args=-admission_control_slots=1
> --impalad_args=-executor_groups=default-pool-group1:1
> --impalad_args=--default_query_options=
> 03:45:26 MainThread: Found 1 impalad/1 statestored/1 catalogd process(es)
> 03:45:26 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 03:45:29 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es)
> 03:45:29 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es)
> 03:45:29 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> 03:45:29 MainThread: num_known_live_backends has reached value: 2
> 03:45:29 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es)
> 03:45:29 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001
> 03:45:29 MainThread: num_known_live_backends has reached value: 2
> 03:45:29 MainThread: Impala Cluster Running with 1 nodes (0 coordinators, 1
> executors).
> -- 2021-03-29 03:45:30,939 DEBUG MainThread: Found 2 impalad/1
> statestored/1 catalogd process(es)
> -- 2021-03-29 03:45:30,939 INFO MainThread: Getting metric:
> statestore.live-backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25010
> -- 2021-03-29 03:45:30,943 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:30,945 INFO MainThread: Metric
> 'statestore.live-backends' has reached desired value: 3
> -- 2021-03-29 03:45:30,946 DEBUG MainThread: Getting
> num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:30,948 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:30,950 INFO MainThread: num_known_live_backends has
> reached value: 2
> -- 2021-03-29 03:45:30,950 DEBUG MainThread: Getting
> num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001
> -- 2021-03-29 03:45:30,953 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:30,955 INFO MainThread: num_known_live_backends has
> reached value: 2
> -- 2021-03-29 03:45:30,955 INFO MainThread: Adding 2 executors to group
> default-pool-group2 with minimum size 2
> -- 2021-03-29 03:45:30,956 INFO MainThread: Starting cluster with
> command:
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/bin/start-impala-cluster.py
> '--state_store_args=--statestore_update_frequency_ms=50
> --statestore_priority_update_frequency_ms=50
> --statestore_heartbeat_frequency_ms=50' --cluster_size=2 --num_coordinators=0
> --log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests
> --log_level=1 --add_executors --impalad_args=--enable_orc_scanner=true
> --impalad_args=-admission_control_slots=1
> --impalad_args=-executor_groups=default-pool-group2:2
> --impalad_args=--default_query_options=
> 03:45:31 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es)
> 03:45:31 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 03:45:31 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node3.INFO
> 03:45:34 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es)
> 03:45:34 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es)
> 03:45:34 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> 03:45:34 MainThread: num_known_live_backends has reached value: 4
> 03:45:35 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es)
> 03:45:35 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001
> 03:45:35 MainThread: num_known_live_backends has reached value: 4
> 03:45:35 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es)
> 03:45:35 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25002
> 03:45:35 MainThread: num_known_live_backends has reached value: 4
> 03:45:35 MainThread: Found 4 impalad/1 statestored/1 catalogd process(es)
> 03:45:35 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25003
> 03:45:35 MainThread: num_known_live_backends has reached value: 4
> 03:45:35 MainThread: Impala Cluster Running with 2 nodes (0 coordinators, 2
> executors).
> -- 2021-03-29 03:45:36,249 DEBUG MainThread: Found 4 impalad/1
> statestored/1 catalogd process(es)
> -- 2021-03-29 03:45:36,249 INFO MainThread: Getting metric:
> statestore.live-backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25010
> -- 2021-03-29 03:45:36,253 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:36,256 INFO MainThread: Metric
> 'statestore.live-backends' has reached desired value: 5
> -- 2021-03-29 03:45:36,256 DEBUG MainThread: Getting
> num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:36,260 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:36,262 INFO MainThread: num_known_live_backends has
> reached value: 4
> -- 2021-03-29 03:45:36,262 DEBUG MainThread: Getting
> num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001
> -- 2021-03-29 03:45:36,266 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:36,268 INFO MainThread: num_known_live_backends has
> reached value: 4
> -- 2021-03-29 03:45:36,268 DEBUG MainThread: Getting
> num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25002
> -- 2021-03-29 03:45:36,271 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:36,273 INFO MainThread: num_known_live_backends has
> reached value: 4
> -- 2021-03-29 03:45:36,273 DEBUG MainThread: Getting
> num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25003
> -- 2021-03-29 03:45:36,277 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:36,279 INFO MainThread: num_known_live_backends has
> reached value: 4
> -- 2021-03-29 03:45:36,279 INFO MainThread: Getting metric:
> cluster-membership.executor-groups.total-healthy from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:36,282 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:36,286 INFO MainThread: Metric
> 'cluster-membership.executor-groups.total-healthy' has reached desired value:
> 2
> -- 2021-03-29 03:45:36,289 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:36,295 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- executing async: localhost:21000
> select * from functional_parquet.alltypestiny where month < 3
> and id + random() < sleep(500);;
> -- 2021-03-29 03:45:37,146 INFO MainThread: Started query
> cb44d9e10462384b:6dc51c0e00000000
> -- waiting for completion of the admission control processing of the query:
> <tests.common.impala_connection.OperationHandle object at 0x7f870ce96410>
> -- 2021-03-29 03:45:37,201 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:37,207 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- executing async: localhost:21000
> select * from functional_parquet.alltypestiny where month < 3
> and id + random() < sleep(500);;
> -- 2021-03-29 03:45:37,225 INFO MainThread: Started query
> 034cf0551469f82f:2b27a3d500000000
> -- waiting for completion of the admission control processing of the query:
> <tests.common.impala_connection.OperationHandle object at 0x7f870cb4fcd0>
> -- 2021-03-29 03:45:37,279 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:37,285 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f870ce96410>
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f870cb4fcd0>
> -- 2021-03-29 03:45:37,295 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:37,301 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:38,323 INFO MainThread: Killing <ImpaladProcess PID:
> 14833
> (/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/be/build/latest/service/impalad
> -is_coordinator=false -disconnected_session_timeout 21600
> -kudu_client_rpc_timeout_ms 0 -kudu_master_hosts localhost
> -mem_limit=12884901888 -logbufsecs=5 -v=1 -max_log_files=0
> -log_filename=impalad_node3
> -log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests
> -beeswax_port=21003 -hs2_port=21053 -hs2_http_port=28003 -krpc_port=27003
> -state_store_subscriber_port=23003 -webserver_port=25003
> --enable_orc_scanner=true -admission_control_slots=1
> -executor_groups=default-pool-group2:2 --default_query_options=)> with signal
> 9
> -- 2021-03-29 03:45:38,348 INFO MainThread: Getting metric:
> cluster-membership.executor-groups.total-healthy from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:38,352 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:38,355 INFO MainThread: Waiting for metric value
> 'cluster-membership.executor-groups.total-healthy'=1. Current value: 2
> -- 2021-03-29 03:45:38,356 INFO MainThread: Sleeping 1s before next retry.
> -- 2021-03-29 03:45:39,357 INFO MainThread: Getting metric:
> cluster-membership.executor-groups.total-healthy from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:39,360 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:39,363 INFO MainThread: Metric
> 'cluster-membership.executor-groups.total-healthy' has reached desired value:
> 1
> -- 2021-03-29 03:45:39,367 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:39,372 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:39,379 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:40,386 INFO MainThread: Killing <ImpaladProcess PID:
> 14830
> (/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/be/build/latest/service/impalad
> -is_coordinator=false -disconnected_session_timeout 21600
> -kudu_client_rpc_timeout_ms 0 -kudu_master_hosts localhost
> -mem_limit=12884901888 -logbufsecs=5 -v=1 -max_log_files=0
> -log_filename=impalad_node2
> -log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests
> -beeswax_port=21002 -hs2_port=21052 -hs2_http_port=28002 -krpc_port=27002
> -state_store_subscriber_port=23002 -webserver_port=25002
> --enable_orc_scanner=true -admission_control_slots=1
> -executor_groups=default-pool-group2:2 --default_query_options=)> with signal
> 9
> -- 2021-03-29 03:45:40,411 INFO MainThread: Getting metric:
> cluster-membership.executor-groups.total from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:40,414 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:40,417 INFO MainThread: Waiting for metric value
> 'cluster-membership.executor-groups.total'=1. Current value: 2
> -- 2021-03-29 03:45:40,417 INFO MainThread: Sleeping 1s before next retry.
> -- 2021-03-29 03:45:41,418 INFO MainThread: Getting metric:
> cluster-membership.executor-groups.total from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:41,421 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:41,424 INFO MainThread: Metric
> 'cluster-membership.executor-groups.total' has reached desired value: 1
> -- 2021-03-29 03:45:41,427 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:41,434 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> SET
> client_identifier=custom_cluster/test_executor_groups.py::TestExecutorGroups::()::test_executor_group_num_queries_executing_metric;
> SET debug_action=QUERY_CANCELLATION_THREAD:SLEEP@10000;
> -- executing async: localhost:21000
> select * from functional_parquet.alltypestiny where month < 3
> and id + random() < sleep(500);;
> -- 2021-03-29 03:45:41,448 INFO MainThread: Started query
> 314f065fbf045208:62949c6e00000000
> -- waiting for completion of the admission control processing of the query:
> <tests.common.impala_connection.OperationHandle object at 0x7f870cb4f510>
> -- 2021-03-29 03:45:41,503 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:42,501 INFO MainThread: Killing <ImpaladProcess PID:
> 14376
> (/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/be/build/latest/service/impalad
> -is_coordinator=false -disconnected_session_timeout 21600
> -kudu_client_rpc_timeout_ms 0 -kudu_master_hosts localhost
> -mem_limit=12884901888 -logbufsecs=5 -v=1 -max_log_files=0
> -log_filename=impalad_node1
> -log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests
> -beeswax_port=21001 -hs2_port=21051 -hs2_http_port=28001 -krpc_port=27001
> -state_store_subscriber_port=23001 -webserver_port=25001
> --enable_orc_scanner=true -admission_control_slots=1
> -executor_groups=default-pool-group1:1 --default_query_options=)> with signal
> 9
> -- 2021-03-29 03:45:42,523 INFO MainThread: Getting metric:
> cluster-membership.executor-groups.total from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:42,526 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:42,530 INFO MainThread: Waiting for metric value
> 'cluster-membership.executor-groups.total'=0. Current value: 1
> -- 2021-03-29 03:45:42,530 INFO MainThread: Sleeping 1s before next retry.
> -- 2021-03-29 03:45:43,531 INFO MainThread: Getting metric:
> cluster-membership.executor-groups.total from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:43,533 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:43,537 INFO MainThread: Metric
> 'cluster-membership.executor-groups.total' has reached desired value: 0
> -- 2021-03-29 03:45:43,539 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:43,543 INFO MainThread: Starting Impalad process with
> args: ['-is_coordinator=false', '-disconnected_session_timeout', '21600',
> '-kudu_client_rpc_timeout_ms', '0', '-kudu_master_hosts', 'localhost',
> '-mem_limit=12884901888', '-logbufsecs=5', '-v=1', '-max_log_files=0',
> '-log_filename=impalad_node1',
> '-log_dir=/data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests',
> '-beeswax_port=21001', '-hs2_port=21051', '-hs2_http_port=28001',
> '-krpc_port=27001', '-state_store_subscriber_port=23001',
> '-webserver_port=25001', '--enable_orc_scanner=true',
> '-admission_control_slots=1', '-executor_groups=default-pool-group1:1',
> '--default_query_options=']
> -- 2021-03-29 03:45:43,561 INFO MainThread: Getting metric:
> impala-server.ready from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25001
> -- 2021-03-29 03:45:43,564 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:43,565 INFO MainThread: Debug webpage not yet
> available: ('Connection aborted.', error(111, 'Connection refused'))
> Turning perftools heap leak checking off
> Redirecting stderr to
> /data/jenkins/workspace/impala-cdpd-master-staging-exhaustive-release/repos/Impala/logs/custom_cluster_tests/impalad_node1.ERROR
> -- 2021-03-29 03:45:44,569 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:44,569 INFO MainThread: Debug webpage not yet
> available: ('Connection aborted.', error(111, 'Connection refused'))
> -- 2021-03-29 03:45:45,573 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:45,576 INFO MainThread: Metric 'impala-server.ready'
> has reached desired value: True
> -- 2021-03-29 03:45:45,576 INFO MainThread: Getting metric:
> cluster-membership.executor-groups.total-healthy from
> impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com:25000
> -- 2021-03-29 03:45:45,579 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> -- 2021-03-29 03:45:45,582 INFO MainThread: Metric
> 'cluster-membership.executor-groups.total-healthy' has reached desired value:
> 1
> -- 2021-03-29 03:45:45,585 INFO MainThread: Starting new HTTP connection
> (1): impala-ec2-centos74-m5-4xlarge-ondemand-117c.vpc.cloudera.com
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
