mathewjacob1002 commented on code in PR #41973:
URL: https://github.com/apache/spark/pull/41973#discussion_r1264335585
##########
python/pyspark/ml/torch/distributor.py:
##########
@@ -514,11 +514,54 @@ def _execute_command(
f"Command {cmd} failed with return code {task.returncode}. "
f"The {last_n_msg} included below: {task_output}"
)
+ @staticmethod
+ def _get_output_from_framework_wrapper(framework_wrapper:
Optional[Callable], input_params: Dict, train_object: Union[Callable, str],
run_pytorch_file_fn: Optional[Callable], *args, **kwargs) -> Optional[Any]:
+ """
+ This function is meant to get the output from framework wrapper
function by passing in the correct arguments,
+ depending on the type of train_object.
+
+ Parameters
+ ----------
+ framework_wrapper: Optional[Callable]
+ Function pointer that will be invoked. Can either be the function
that runs distributed training on
Review Comment:
train_object is from the user - it's either a string representing a filepath
or a function pointer that the user wants to run in a distributed fashion. I
will try to make this more explicit in the docstring.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
