HyukjinKwon opened a new pull request, #37253:
URL: https://github.com/apache/spark/pull/37253
### What changes were proposed in this pull request?
This PR factors the Arrow input code path out as `PythonArrowInput` as
symmetry to `PythonArrowOutput`. The current hierarchy is not affected:
```
└── BasePythonRunner
├── ArrowPythonRunner with PythonArrowOutput with PythonArrowInput
├── CoGroupedArrowPythonRunner with PythonArrowOutput
├── PythonRunner
└── PythonUDFRunner
```
In addition, this PR also factors out `handleMetadataAfterExec` and
`handleMetadataBeforeExec` which contains the logic to send and receive the
metadata such as runtime configurations specific to Arrow in/out.
### Why are the changes needed?
https://github.com/apache/spark/commit/40485f4656261231251f1d68b5b7d8b3b8600372
factored `PythonArrowOutput` out. It's better to factor `PythonArrowInput` out
too to be consistent
### Does this PR introduce _any_ user-facing change?
No, this is refactoring.
### How was this patch tested?
Existing test cases should cover.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
