[jira] [Commented] (IMPALA-13024) Several tests timeout waiting for admission
[ https://issues.apache.org/jira/browse/IMPALA-13024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839300#comment-17839300 ] Riza Suminto commented on IMPALA-13024: --- There might be slots starvation happen due to addition of this mandatory exec option. [https://github.com/apache/impala/commit/6abfdbc56c3d0ec3ac201dd4b8c2c35656d24eaf#diff-4b2eec27068bd7a1f30e139b84c0067d9c33c44ffe2efb46a18f9061e9d658c1R760] I'll try to remove it and retests with ASAN build. custom_cluster should not be affected since the default slot_count_strategy is still LARGEST_FRAGMENT. > Several tests timeout waiting for admission > --- > > Key: IMPALA-13024 > URL: https://issues.apache.org/jira/browse/IMPALA-13024 > Project: IMPALA > Issue Type: Bug >Reporter: Csaba Ringhofer >Priority: Critical > > A bunch of seemingly unrelated tests failed with the following message: > Example: > query_test.test_spilling.TestSpillingDebugActionDimensions.test_spilling_aggs[protocol: > beeswax | exec_option: {'mt_dop': 1, 'debug_action': None, > 'default_spillable_buffer_size': '256k'} | table_format: parquet/none] > {code} > ImpalaBeeswaxException: EQuery aborted:Admission for query exceeded > timeout 6ms in pool default-pool. Queued reason: Not enough admission > control slots available on host ... . Needed 1 slots but 18/16 are already in > use. Additional Details: Not Applicable > {code} > This happened in an ASAN build. Another test also failed which may be related > to the cause: > custom_cluster.test_admission_controller.TestAdmissionController.test_queue_reasons_slots > > {code} > Timeout: query 'e1410add778cd7b0:c40812b9' did not reach one of the > expected states [4], last known state 5 > {code} > test_queue_reasons_slots seems to be know flaky test: IMPALA-10338 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12933) Catalogd should set eventTypeSkipList when fetching specifit events for a table
[ https://issues.apache.org/jira/browse/IMPALA-12933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839297#comment-17839297 ] ASF subversion and git services commented on IMPALA-12933: -- Commit db09d58ef767b2b759792412efcc9481777c464b in impala's branch refs/heads/master from stiga-huang [ https://gitbox.apache.org/repos/asf?p=impala.git;h=db09d58ef ] IMPALA-12933: Avoid fetching unneccessary events of unwanted types There are several places where catalogd will fetch all events of a specific type on a table. E.g. in TableLoader#load(), if the table has an old createEventId, catalogd will fetch all CREATE_TABLE events after that createEventId on the table. Fetching the list of events is expensive since the filtering is done on client side, i.e. catalogd fetches all events and filter them locally based on the event type and table name. This could take hours if there are lots of events (e.g 1M) in HMS. This patch sets the eventTypeSkipList with the complement set of the wanted type. So the get_next_notification RPC can filter out some events on HMS side. To avoid bringing too much computation overhead to HMS's underlying RDBMS in evaluating predicates of EVENT_TYPE != 'xxx', rare event types (e.g. DROP_ISCHEMA) are not added in the list. A new flag, common_hms_event_types, is added to specify the common HMS event types. Once HIVE-28146 is resolved, we can set the wanted types directly in the HMS RPC and this approach can be simplified. UPDATE_TBL_COL_STAT_EVENT, UPDATE_PART_COL_STAT_EVENT are the most common unused events for Impala. They are also added to the default skip list. A new flag, default_skipped_hms_event_types, is added to configure this list. This patch also fixes an issue that events of the non-default catalog are not filtered out. In a local perf test, I generated 100K RELOAD events after creating a table in Hive. Then use the table in Impala to trigger metadata loading on it which will fetch the latest CREATE_TABLE event by polling all events after the last known CREATE_TABLE event. Before this patch, fetching the events takes 1s779ms. Now it takes only 395.377ms. Note that in prod env, the event messages are usually larger, we could have a larger speedup. Tests: - Added an FE test - Ran CORE tests Change-Id: Ieabe714328aa2cc605cb62b85ae8aa4bd537dbe9 Reviewed-on: http://gerrit.cloudera.org:8080/21186 Reviewed-by: Csaba Ringhofer Tested-by: Impala Public Jenkins > Catalogd should set eventTypeSkipList when fetching specifit events for a > table > --- > > Key: IMPALA-12933 > URL: https://issues.apache.org/jira/browse/IMPALA-12933 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Quanlong Huang >Assignee: Quanlong Huang >Priority: Critical > > There are several places that catalogd will fetch all events of a specifit > type on a table. E.g. in TableLoader#load(), if the table has an old > createEventId, catalogd will fetch all CREATE_TABLE events after that > createEventId on the table. > Fetching the list of events is expensive since the filtering is done on > client side, i.e. catalogd fetch all events and filter them locally based on > the event type and table name: > [https://github.com/apache/impala/blob/14e3ed4f97292499b2e6ee8d5a756dc648d9/fe/src/main/java/org/apache/impala/catalog/TableLoader.java#L98-L102] > [https://github.com/apache/impala/blob/b7ddbcad0dd6accb559a3f391a897a8c442d1728/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L336] > This could take hours if there are lots of events (e.g 1M) in HMS. In fact, > NotificationEventRequest can specify an eventTypeSkipList. Catalogd can do > the filtering of event type in HMS side. On higher Hive versions that have > HIVE-27499, catalogd can also specify the table name in the request > (IMPALA-12607). > This Jira focus on specifying the eventTypeSkipList when fetching events of a > particular type on a table. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12657) Improve ProcessingCost of ScanNode and NonGroupingAggregator
[ https://issues.apache.org/jira/browse/IMPALA-12657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839296#comment-17839296 ] ASF subversion and git services commented on IMPALA-12657: -- Commit 25a8d70664ce4c91634e1377e020a4d7fa8b09c0 in impala's branch refs/heads/master from David Rorke [ https://gitbox.apache.org/repos/asf?p=impala.git;h=25a8d7066 ] IMPALA-12657: Improve ProcessingCost of ScanNode and NonGroupingAggregator This patch improves the accuracy of the CPU ProcessingCost estimates for several of the CPU intensive operators by basing the costs on benchmark data. The general approach for a given operator was to run a set of queries that exercised the operator under various conditions (e.g. large vs small row sizes and row counts, varying NDV, different file formats, etc) and capture the CPU time spent per unit of work (the unit of work might be measured as some number of rows, some number of bytes, some number of predicates evaluated, or some combination of these). The data was then analyzed in an attempt to fit a simple model that would allow us to predict CPU consumption of a given operator based on information available at planning time. For example, the CPU ProcessingCost for a Parquet scan is estimated as: TotalCost = (0.0144 * BytesMaterialized) + (0.0281 * Rows * Predicate Count) The coefficients (0.0144 and 0.0281) are derived from benchmarking scans under a variety of conditions. Similar cost functions and coefficients were derived for all of the benchmarked operators. The coefficients for all the operators are normalized such that a single unit of cost equates to roughly 100 nanoseconds of CPU time on a r5d.4xlarge instance. So we would predict an operator with a cost of 10,000,000 would complete in roughly one second on a single core. Limitations: * Costing only addresses CPU time spent and doesn't account for any IO or other wait time. * Benchmarking scenarios didn't provide comprehensive coverage of the full range of data types, distributions, etc. More thorough benchmarking could improve the costing estimates further. * This initial patch only covers a subset of the operators, focusing on those that are most common and most CPU intensive. Specifically the following operators are covered by this patch. All others continue to use the previous ProcessingCost code: AggregationNode DataStreamSink (exchange sender) ExchangeNode HashJoinNode HdfsScanNode HdfsTableSink NestedLoopJoinNode SortNode UnionNode Benchmark-based costing of the remaining operators will be covered by a future patch. Future patches will automate the collection and analysis of the benchmark data and the computation of the cost coefficients to simplify maintenance of the costing as performance changes over time. Change-Id: Icf1edd48d4ae255b7b3b7f5b228800d7bac7d2ca Reviewed-on: http://gerrit.cloudera.org:8080/21279 Reviewed-by: Riza Suminto Tested-by: Impala Public Jenkins > Improve ProcessingCost of ScanNode and NonGroupingAggregator > > > Key: IMPALA-12657 > URL: https://issues.apache.org/jira/browse/IMPALA-12657 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Affects Versions: Impala 4.3.0 >Reporter: Riza Suminto >Assignee: David Rorke >Priority: Major > Fix For: Impala 4.4.0 > > Attachments: profile_1f4d7a679a3e12d5_42231157.txt > > > Several benchmark run measuring Impala scan performance indicates some > costing improvement opportunity around ScanNode and NonGroupingAggregator. > [^profile_1f4d7a679a3e12d5_42231157.txt] shows an example of simple > count query. > Key takeaway: > # There is a strong correlation between total materialized bytes (row-size * > cardinality) with total materialized tuple time per fragment. Row > materialization cost should be adjusted to be based on this row-sized instead > of equal cost per scan range. > # NonGroupingAggregator should have much lower cost that GroupingAggregator. > In example above, the cost of NonGroupingAggregator dominates the scan > fragment even though it only does simple counting instead of hash table > operation. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12933) Catalogd should set eventTypeSkipList when fetching specifit events for a table
[ https://issues.apache.org/jira/browse/IMPALA-12933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quanlong Huang resolved IMPALA-12933. - Fix Version/s: Impala 4.4.0 Resolution: Fixed > Catalogd should set eventTypeSkipList when fetching specifit events for a > table > --- > > Key: IMPALA-12933 > URL: https://issues.apache.org/jira/browse/IMPALA-12933 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Quanlong Huang >Assignee: Quanlong Huang >Priority: Critical > Fix For: Impala 4.4.0 > > > There are several places that catalogd will fetch all events of a specifit > type on a table. E.g. in TableLoader#load(), if the table has an old > createEventId, catalogd will fetch all CREATE_TABLE events after that > createEventId on the table. > Fetching the list of events is expensive since the filtering is done on > client side, i.e. catalogd fetch all events and filter them locally based on > the event type and table name: > [https://github.com/apache/impala/blob/14e3ed4f97292499b2e6ee8d5a756dc648d9/fe/src/main/java/org/apache/impala/catalog/TableLoader.java#L98-L102] > [https://github.com/apache/impala/blob/b7ddbcad0dd6accb559a3f391a897a8c442d1728/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L336] > This could take hours if there are lots of events (e.g 1M) in HMS. In fact, > NotificationEventRequest can specify an eventTypeSkipList. Catalogd can do > the filtering of event type in HMS side. On higher Hive versions that have > HIVE-27499, catalogd can also specify the table name in the request > (IMPALA-12607). > This Jira focus on specifying the eventTypeSkipList when fetching events of a > particular type on a table. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-13024) Several tests timeout waiting for admission
[ https://issues.apache.org/jira/browse/IMPALA-13024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839247#comment-17839247 ] Abhishek Rawat commented on IMPALA-13024: - Slot based admission is not enabled when using default groups - IMPALA-8757. `test_queue_reasons_slots` has executor_groups="default-pool-group1" set so slots based admission is enabled. I couldn't say the same about the other one. CC [~rizaon] > Several tests timeout waiting for admission > --- > > Key: IMPALA-13024 > URL: https://issues.apache.org/jira/browse/IMPALA-13024 > Project: IMPALA > Issue Type: Bug >Reporter: Csaba Ringhofer >Priority: Critical > > A bunch of seemingly unrelated tests failed with the following message: > Example: > query_test.test_spilling.TestSpillingDebugActionDimensions.test_spilling_aggs[protocol: > beeswax | exec_option: {'mt_dop': 1, 'debug_action': None, > 'default_spillable_buffer_size': '256k'} | table_format: parquet/none] > {code} > ImpalaBeeswaxException: EQuery aborted:Admission for query exceeded > timeout 6ms in pool default-pool. Queued reason: Not enough admission > control slots available on host ... . Needed 1 slots but 18/16 are already in > use. Additional Details: Not Applicable > {code} > This happened in an ASAN build. Another test also failed which may be related > to the cause: > custom_cluster.test_admission_controller.TestAdmissionController.test_queue_reasons_slots > > {code} > Timeout: query 'e1410add778cd7b0:c40812b9' did not reach one of the > expected states [4], last known state 5 > {code} > test_queue_reasons_slots seems to be know flaky test: IMPALA-10338 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-13004) heap-use-after-free error in ExprTest AiFunctionsTest
[ https://issues.apache.org/jira/browse/IMPALA-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yida Wu resolved IMPALA-13004. -- Fix Version/s: Impala 4.4.0 Resolution: Fixed > heap-use-after-free error in ExprTest AiFunctionsTest > - > > Key: IMPALA-13004 > URL: https://issues.apache.org/jira/browse/IMPALA-13004 > Project: IMPALA > Issue Type: Bug > Components: be >Affects Versions: Impala 4.4.0 >Reporter: Andrew Sherman >Assignee: Yida Wu >Priority: Critical > Fix For: Impala 4.4.0 > > > In an ASAN test, expr-test fails: > {code} > ==1601==ERROR: AddressSanitizer: heap-use-after-free on address > 0x63100152c826 at pc 0x0298f841 bp 0x7ffc91fff460 sp 0x7ffc91fff458 > READ of size 2 at 0x63100152c826 thread T0 > #0 0x298f840 in rapidjson::GenericValue, > rapidjson::MemoryPoolAllocator >::GetType() const > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:936:62 > #1 0x298d852 in bool rapidjson::GenericValue, > rapidjson::MemoryPoolAllocator > >::Accept, > rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8, > rapidjson::CrtAllocator, 0u> > >(rapidjson::Writer, > rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8, > rapidjson::CrtAllocator, 0u>&) const > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1769:16 > #2 0x298d8d0 in bool rapidjson::GenericValue, > rapidjson::MemoryPoolAllocator > >::Accept, > rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8, > rapidjson::CrtAllocator, 0u> > >(rapidjson::Writer, > rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8, > rapidjson::CrtAllocator, 0u>&) const > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1790:21 > #3 0x298d9e8 in bool rapidjson::GenericValue, > rapidjson::MemoryPoolAllocator > >::Accept, > rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8, > rapidjson::CrtAllocator, 0u> > >(rapidjson::Writer, > rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8, > rapidjson::CrtAllocator, 0u>&) const > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1781:21 > #4 0x28a0707 in impala_udf::StringVal > impala::AiFunctions::AiGenerateTextInternal(impala_udf::FunctionContext*, > impala_udf::StringVal const&, impala_udf::StringVal const&, > impala_udf::StringVal const&, impala_udf::StringVal const&, > impala_udf::StringVal const&, bool) > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/ai-functions.inline.h:140:11 > #5 0x286087e in impala::ExprTest_AiFunctionsTest_Test::TestBody() > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/expr-test.cc:11254:12 > #6 0x8aeaa4c in void > testing::internal::HandleExceptionsInMethodIfSupported void>(testing::Test*, void (testing::Test::*)(), char const*) > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8aeaa4c) > #7 0x8ae3ec4 in testing::Test::Run() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae3ec4) > #8 0x8ae4007 in testing::TestInfo::Run() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4007) > #9 0x8ae40e4 in testing::TestCase::Run() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae40e4) > #10 0x8ae45db in testing::internal::UnitTestImpl::RunAllTests() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae45db) > #11 0x8ae4682 in testing::UnitTest::Run() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4682) > #12 0x249ac19 in main > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/service/unified-betest-main.cc:48:10 > #13 0x7f4b0b911554 in __libc_start_main (/lib64/libc.so.6+0x22554) > #14 0x2396af6 in _start > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2396af6) > 0x63100152c826 is located 38 bytes inside of 65560-byte region > [0x63100152c800,0x63100153c818) > freed by thread T0 here: > #0 0x2466ea7 in __interceptor_free > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2466ea7) > #1
[jira] [Resolved] (IMPALA-12988) Calculate an unbounded version of CpuAsk
[ https://issues.apache.org/jira/browse/IMPALA-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Riza Suminto resolved IMPALA-12988. --- Fix Version/s: Impala 4.4.0 Resolution: Fixed > Calculate an unbounded version of CpuAsk > > > Key: IMPALA-12988 > URL: https://issues.apache.org/jira/browse/IMPALA-12988 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Reporter: Riza Suminto >Assignee: Riza Suminto >Priority: Major > Fix For: Impala 4.4.0 > > > CpuAsk is calculated through recursive call beginning at > Planner.computeBlockingAwareCores(), which called after > Planner.computeEffectiveParallelism(). It does blocking operator analysis > over selected degree of parallelism that decided during > computeEffectiveParallelism() traversal. That selected degree of parallelism, > however, is already bounded by min and max parallelism config, derived from > PROCESSING_COST_MIN_THREADS and MAX_FRAGMENT_INSTANCES_PER_NODE options > accordingly. > It is beneficial to have another version of CpuAsk that is not bounded by min > and max parallelism config. It should purely based on the fragment's > ProcessingCost and query plan relationship constraint (ie., num JOIN BUILDER > fragment should be equal as num JOIN fragment for partitioned join). During > executor group set selection, Frontend should use the unbounded CpuAsk number > to avoid assigning query to small executor group set prematurely. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-13016) Fix ambiguous row_regex that check for no-existence
[ https://issues.apache.org/jira/browse/IMPALA-13016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Riza Suminto resolved IMPALA-13016. --- Fix Version/s: Impala 4.4.0 Resolution: Fixed > Fix ambiguous row_regex that check for no-existence > --- > > Key: IMPALA-13016 > URL: https://issues.apache.org/jira/browse/IMPALA-13016 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Riza Suminto >Assignee: Riza Suminto >Priority: Minor > Fix For: Impala 4.4.0 > > > There are few row_regex pattern used in EE test files that is ambiguous on > whether a parttern not exist in all parts of results/runtime filter or at > least one row does not have that pattern: > {code:java} > $ git grep -n "row_regex: (?\!" > testdata/workloads/functional-query/queries/QueryTest/acid-clear-statsaccurate.test:34:row_regex: > (?!.*COLUMN_STATS_ACCURATE) > testdata/workloads/functional-query/queries/QueryTest/acid-truncate.test:47:row_regex: > (?!.*COLUMN_STATS_ACCURATE) > testdata/workloads/functional-query/queries/QueryTest/clear-statsaccurate.test:28:row_regex: > (?!.*COLUMN_STATS_ACCURATE) > testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-directed-mode.test:14:row_regex: > (?!.*F03:JOIN BUILD.*) {code} > They should be replaced either with !row_regex or VERIFY_IS_NOT_IN comment. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13024) Several tests timeout waiting for admission
Csaba Ringhofer created IMPALA-13024: Summary: Several tests timeout waiting for admission Key: IMPALA-13024 URL: https://issues.apache.org/jira/browse/IMPALA-13024 Project: IMPALA Issue Type: Bug Reporter: Csaba Ringhofer A bunch of seemingly unrelated tests failed with the following message: Example: query_test.test_spilling.TestSpillingDebugActionDimensions.test_spilling_aggs[protocol: beeswax | exec_option: {'mt_dop': 1, 'debug_action': None, 'default_spillable_buffer_size': '256k'} | table_format: parquet/none] {code} ImpalaBeeswaxException: EQuery aborted:Admission for query exceeded timeout 6ms in pool default-pool. Queued reason: Not enough admission control slots available on host ... . Needed 1 slots but 18/16 are already in use. Additional Details: Not Applicable {code} This happened in an ASAN build. Another test also failed which may be related to the cause: custom_cluster.test_admission_controller.TestAdmissionController.test_queue_reasons_slots {code} Timeout: query 'e1410add778cd7b0:c40812b9' did not reach one of the expected states [4], last known state 5 {code} test_queue_reasons_slots seems to be know flaky test: IMPALA-10338 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org