jayzhan211 opened a new pull request, #25533:
URL: https://github.com/apache/datafusion/pull/25533
## Which issue does this PR close?
- N/A — small refactor, no issue filed.
## Rationale for this change
The memory-limited fallback path in `NestedLoopJoinExec` carries the left
schema in three places: `LeftSpillData::schema`,
`FallbackCoordinatorInner::left_schema` and `SpillStateActive::left_schema`.
The last two are `Option`s that are only ever populated from the first, so they
add a lock round-trip, a lazy initialisation and an `.expect("left_schema must
be set")` without carrying any information of their own.
`FallbackCoordinator` is already the hardest part of this file to follow.
Removing state that cannot differ from its source makes the leader path in
`next_chunk` shorter and leaves one fewer invariant for readers to check.
## What changes are included in this PR?
No behaviour change.
- Remove `FallbackCoordinatorInner::left_schema`; the leader uses
`spill_data.schema` directly.
- Remove `SpillStateActive::left_schema`; `EmitGlobalRightUnmatched` reads
`left_spill.schema`, which drops the lazy initialisation and the `expect`.
- Add `FallbackCoordinator::abandon_load()` for the "clear
`loader_in_flight` and wake waiters" step that was written out in two error
paths.
- `load_one_chunk` takes `&self` instead of `self: Arc<Self>` and loses its
unused `_chunk_index` parameter.
## What is the testing strategy for this PR?
Covered by the existing `nested_loop_join` unit tests, including the
fallback cancellation tests, which exercise every touched path. No new tests,
since there is no new behaviour.
## Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
