vikrantmehta123 opened a new pull request, #2886:
URL: https://github.com/apache/datafusion-ballista/pull/2886
# Which issue does this PR close?
Closes #2839 .
# Rationale for this change
Standalone setup creates its own `SchedulerConfig`, leaving no way to supply
a `PartitionSliceableResolver`. Custom plan nodes that require partition
slicing therefore cannot run when their stages are split between tasks. The
issue suggested two options. I have chosen the option of registering resolver
next to the codec.
# What changes are included in this PR?
- Add resolver setter and getter methods to `SessionConfigExt`, following
the existing codec extension pattern.
- Forward the registered resolver to `SchedulerConfig` during standalone
setup.
- Add an integration test verifying that the standalone scheduler consults a
resolver registered through session configuration.
- Update the extending-components guide and upgrade notes.
**Note:** AI helped me identify that there are docs to be changed. AI also
helped me come up with the integration test. I have read and understood both
these changes, and they looked reasonable to me.
# Are there any user-facing changes?
Yes, users can optionally register a resolver in standalone mode. The
`SessionConfigExt` gets two more methods. This is documented in the upgrade
guide.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]