sven-weber-db commented on code in PR #55716:
URL: https://github.com/apache/spark/pull/55716#discussion_r3282017694
##########
python/pyspark/worker_message.py:
##########
@@ -243,9 +247,14 @@ def from_stream(cls, stream: IO) -> "WorkerInitInfo":
v = utf8_deserializer.loads(stream)
eval_conf[k] = v
- udf_info: Union[bytes, UDTFInfo, list[UDFInfo]]
+ udf_info: UdfInfoType
if eval_type == PythonEvalType.NON_UDF:
+ # If the code in this branch changes
Review Comment:
At the moment, `WorkerInitInfo.from_stream` is only called in `worker.py`,
which will now pass a `stream: ZeroCopyByteStream`. Therefore, in case of
`eval_type == PythonEvalType.NON_UDF`, `udf_info` will now be a `memoryview`
instead of a `bytes` object as the result of the `read` call.
Am I missing another usage of `WorkerInitInfo` that does not match this
behavior?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
