alexey-bass opened a new issue, #66398: URL: https://github.com/apache/doris/issues/66398
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version Reproduced identically on two versions (VeloDB Cloud builds of Doris): - selectdb-4.1.3-39dbf23ff3d (Doris 3.1.4) - cloud-4.1.8-737218b90e2 (Doris 3.1.7) - upgraded specifically to re-test, same behavior Deployment: compute-storage decoupled (cloud mode), single BE, parallel_fragment_exec_instance_num=2. ### What's Wrong? A trivial LEFT JOIN with LIMIT 100 hangs in RUNNING state for the entire runtime_filter_wait_time_ms (900 s in our setup) before returning, when the following session variables are set: enable_force_spill=true enable_join_spill=true enable_agg_spill=true enable_sort_spill=true Query: SELECT t.*, srstart.stop_name AS actual_start_stop FROM trip_time_convert t LEFT JOIN stop_on_route_stop_master srstart ON srstart.id = t.actual_start_stop_id LIMIT 100; Table sizes: probe/preserved side ~2.9M rows, other side ~9.79M rows. The planner rewrites this to RIGHT_OUTER_JOIN, pushes LIMIT 100 down to the build side, and attaches two runtime filters to the 9.79M-row scan: RF000[min_max] and RF001[in_or_bloom] on the join key. From the profile: 1. Because enable_force_spill=true, the join runs as RIGHT OUTER JOIN(PARTITIONED) even though the build side is only 100 rows. 2. The build completes in ~2 ms. Two build instances: InputRows = 100 and 0 (one instance receives zero rows). No spill actually occurs - every Spill* counter is 0. 3. The runtime filters are never produced: PublishRuntimeFilterTime = 0ns, RuntimeFilterComputeTime = 0ns. 4. They are also never disabled: profile shows Disabled = false, Ignored = false. 5. As a result the consumer scan never starts: NumScanners = 0, ScanRows = 0, both filters stuck in RuntimeFilterState = NOT_READY, synced_size = -1, has_remote_target = true, WaitTimeMS = 900000. The query "completes" only when the RF wait times out. Key profile excerpt (identical on both versions): OLAP_SCAN_OPERATOR (id=2, table = stop_on_route_stop_master): RuntimeFilter: (id = 0, type = minmax): Info: [Id = 0, IsPushDown = false, RuntimeFilterState = NOT_READY, HasRemoteTarget = true, HasLocalTarget = false, Ignored = false, Disabled = false, WaitTimeMS = 900000] RuntimeFilter: (id = 1, type = in_or_bloomfilter): Info: [Id = 1, ... RuntimeFilterState = NOT_READY, ... Disabled = false] NumScanners: 0 ScanRows: 0 PARTITIONED_HASH_JOIN_SINK_OPERATOR (id=4): InputRows: 100 (instance 0) / 0 (instance 1) ExecTime: ~2ms, all Spill* counters = 0 PARTITIONED_HASH_JOIN_PROBE_OPERATOR (id=4), BuildPhase: RuntimeFilterComputeTime: 0ns PublishRuntimeFilterTime: 0ns Full profiles from both versions attached. ### What You Expected? Per the documented spill design, the partitioned hash join should either (a) publish the runtime filters normally when no spill occurs, or (b) on spilling, disable the RF and signal the consumer so the scan proceeds without waiting. The observed state is a third, invalid one: RF neither published nor disabled, leaving the consumer blocked for the full wait time. Even though enable_force_spill is a debug/testing flag, it should not deadlock a query that never actually spills. Expected result for this query: sub-second. Verified: with enable_force_spill=false (and the other spill variables off), the identical query returns in 90 ms and the join runs as a plain HASH_JOIN with both RFs reaching READY. ### How to Reproduce? 1. Two tables joined on an id column, big enough that the planner attaches min_max + in_or_bloom runtime filters to the larger side's scan (~10M rows on the RF target side in our case). 2. Set parallel_fragment_exec_instance_num=2 (so that with LIMIT pushdown one build instance can end up with 0 input rows - suspected trigger for the RF merge never completing, since both RFs have remote targets and synced_size stays -1). 3. SET enable_force_spill=true; SET enable_join_spill=true; 4. Run: SELECT t.*, d.name FROM fact t LEFT JOIN dim d ON d.id = t.dim_id LIMIT 100; 5. Query hangs until runtime_filter_wait_time_ms elapses. 6. SET enable_force_spill=false; rerun - returns immediately. Repro rate: 100% with the variables above, on both tested versions. ### Anything Else? - Profile IDs for reference: - 3.1.4 run: c6cd07df59084b3d-8da5afe29dfce262 - 3.1.7 run: 8dca88a596834a4b-aacef345cb7ab612 - Workaround confirmed: disable enable_force_spill (900 s -> 90 ms). ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
