rangadi opened a new pull request, #41343:
URL: https://github.com/apache/spark/pull/41343
### What changes were proposed in this pull request?
This updated Protobuf Pyspark API to allow passing binary FileDescriptorSet
rather than a file name. This is a Python follow up to feature implemented in
Scala in #41192.
### Why are the changes needed?
- This allows flexibility for Pyspark users to provide binary descriptor
set directly.
- Even if users are using file path, Pyspark avoids passing file name to
Scala and reads the descriptor file in Python. This avoids having to read the
file in Scala.
### Does this PR introduce _any_ user-facing change?
- This adds extra arg to `from_protobuf()` and `to_protobuf()` API.
### How was this patch tested?
- Doc tests
- Manual tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
