Suraj created SPARK-31339:
-----------------------------
Summary: Changed PipelineModel(...) to self.cls(...) in
pyspark.ml.pipeline.PipelineModelReader.load()
Key: SPARK-31339
URL: https://issues.apache.org/jira/browse/SPARK-31339
Project: Spark
Issue Type: Bug
Components: ML, PySpark
Affects Versions: 2.4.5
Reporter: Suraj
PR: [https://github.com/apache/spark/pull/28110]
### What changes were proposed in this pull request?
pypsark.ml.pipeline.py line 245: Change PipelineModel(...) to self.cls(...)
### Why are the changes needed?
This change fixes the loading of class (which inherits from PipelineModel
class) from file.
E.g. Current issue:
```
CustomPipelineModel(PipelineModel):
def _transform(self, df):
...
CustomPipelineModel.save('path/to/file') # works
CustomPipelineModel.load('path/to/file') # wrong: results in PipelineModel()
instead of CustomPipelineModel()
CustomPipelineModel.transform() # wrong: results in calling
PipelineModel.transform() instead of CustomPipelineModel.transform()
```
### Does this introduce any user-facing change?
No.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
