Riza Suminto created IMPALA-13874:
-------------------------------------
Summary: Typo in test_coord_only_pool_exec_groups causing minidump
Key: IMPALA-13874
URL: https://issues.apache.org/jira/browse/IMPALA-13874
Project: IMPALA
Issue Type: Test
Components: Backend, Test
Affects Versions: Impala 5.0.0
Reporter: Riza Suminto
Assignee: Riza Suminto
Graceful shutdown following test_coord_only_pool_exec_groups completion produce
FATAL log like this.
{code:java}
F0316 04:27:40.947587 1226 executor-group.cc:150] Tried to remove non-existing
backend from per-host list: 127.0.0.1:27002{code}
This is because the test has typo in executor group name (should be 'large'
instead of small).
[https://github.com/apache/impala/blob/356b7e5ddf7868968fb76ca55a8046d0291388fd/tests/custom_cluster/test_admission_controller.py#L2071]
This typo is causing local backend registration fail with following message:
{noformat}
is configured for executor group name: "root.group-set-small-group-000"
min_size: 2
but group has minimum size 1
W0316 04:27:40.146536 1226 executor-group.cc:93] Ignoring inconsistent backend
for executor group: 127.0.0.1:27002{noformat}
Here is the last 50 lines of the log.
{noformat}
I0316 04:27:40.146524 1226 cluster-membership-mgr.cc:439] Adding local backend
to group name: "root.group-set-small-group-000"
min_size: 2
W0316 04:27:40.146526 1226 executor-group.cc:235] Backend backend_id {
hi: 11695485365096650168
lo: 10250060141956239240
}
address {
hostname: "impala-ec2-centos79-m6i-4xlarge-xldisk-0aeb.vpc.cloudera.com"
port: 27002
uds_address: "@impala-krpc"
}
ip_address: "127.0.0.1"
is_coordinator: false
is_executor: true
debug_http_address {
hostname: "impala-ec2-centos79-m6i-4xlarge-xldisk-0aeb.vpc.cloudera.com"
port: 25002
}
secure_webserver: false
krpc_address {
hostname: "127.0.0.1"
port: 27002
uds_address: "@impala-krpc"
}
admit_mem_limit: 11811160064
is_quiescing: false
executor_groups {
name: "root.group-set-small-group-000"
min_size: 2
}
admission_slots: 16
process_start_time: "2025-03-16 04:27:39.975817000"
version: "4.5.0-SNAPSHOT DEBUG (build 935c5b37f62a9f249db1fa5e48618c6ad62fe9af)"
is configured for executor group name: "root.group-set-small-group-000"
min_size: 2
but group has minimum size 1
W0316 04:27:40.146536 1226 executor-group.cc:93] Ignoring inconsistent backend
for executor group: 127.0.0.1:27002
I0316 04:27:40.146538 1226 cluster-membership-mgr.cc:567] Sending local
backend to statestore
I0316 04:27:40.246865 1226 cluster-membership-mgr.cc:250] Processing
statestore update
I0316 04:27:40.246877 1226 cluster-membership-mgr.cc:267] Received delta
membership update
I0316 04:27:40.847270 1226 cluster-membership-mgr.cc:250] Processing
statestore update
I0316 04:27:40.847282 1226 cluster-membership-mgr.cc:267] Received delta
membership update
I0316 04:27:40.938957 1229 init.cc:260] Shutdown signal received. Current
Shutdown Status: shutdown grace period left: 0, deadline left: 1m, cancel
deadline left: 48s000ms, queries registered on coordinator: 0, queries
executing: 0, fragment instances: 0
I0316 04:27:40.947544 1226 cluster-membership-mgr.cc:251] Local backend
membership needs update
I0316 04:27:40.947552 1226 cluster-membership-mgr.cc:267] Received delta
membership update
I0316 04:27:40.947569 1226 cluster-membership-mgr.cc:436] Removing local
backend from group name: "root.group-set-small-group-000"
min_size: 2
F0316 04:27:40.947587 1226 executor-group.cc:150] Tried to remove non-existing
backend from per-host list: 127.0.0.1:27002
Minidump in thread [1226]StatestoreSubscriber-1 running query
0000000000000000:0000000000000000, fragment instance
0000000000000000:0000000000000000
Wrote minidump to
/data/jenkins/workspace/impala-cdwh-2025.0.19.0-core-jdk17/repos/Impala/logs/custom_cluster_tests/minidumps/impalad/22f92a28-73c2-495a-d5bb87be-bc13640b.dmp{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
