BiteTheDDDDt opened a new pull request, #64953:
URL: https://github.com/apache/doris/pull/64953
### What problem does this PR solve?
Issue Number: None
Related PR: #64899
Problem Summary: A runtime filter dependency can wake a pipeline task while
another thread is closing or finalizing the same task.
HybridTaskScheduler::submit() synchronously calls PipelineTask::is_blockable()
before enqueueing the task, and is_blockable() reads _sink and _operators.
After close/finalize starts releasing terminal task resources, a late submit
can therefore dereference cleared task resources and crash.
This change adds a small submit gate on PipelineTask. Terminal
close/finalize closes that gate under a dedicated lock, and
HybridTaskScheduler::submit() checks the gate under the same lock before
calling is_blockable(). The lock only covers the blockable check and does not
extend to the actual queue submit.
(cherry picked from commit 1dd3207f04a7311ddec655990fca58d68b0b0b9d)
### Release note
None
### Check List (For Author)
- Test: Unit Test / Static check
- build-support/clang-format.sh
- build-support/check-format.sh
- git diff --check HEAD~1..HEAD
- Attempted: ./run-be-ut.sh --run
--filter=PipelineTaskTest.TEST_CLOSED_TASK_REJECTS_HYBRID_SUBMIT_BEFORE_FINALIZE:PipelineTaskTest.TEST_FINALIZED_TASK_REJECTS_HYBRID_SUBMIT.
It was interrupted after confirming the changed pipeline_task.cpp and
task_scheduler.cpp objects compiled, because branch-4.1 BE UT uses one large
doris_be_test binary and the filtered run expanded into a near full UT build.
- Attempted: ninja -C be/ut_build_ASAN -j1
test/CMakeFiles/doris_be_test.dir/exec/pipeline/pipeline_task_test.cpp.o. It
was blocked by the pre-existing contrib/faiss submodule checkout at
0276f0bb7e4f while branch-4.1 expects 032afe95f671, causing fatal error:
faiss/invlists/PreadInvertedLists.h file not found.
- Behavior changed: No
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
