[
https://issues.apache.org/jira/browse/SPARK-31339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suraj updated SPARK-31339:
--------------------------
Description:
PR: [https://github.com/apache/spark/pull/28110]
What changes were proposed in this pull request?
pypsark.ml.pipeline.py line 245: Change PipelineModel(...) to self.cls(...)
Why are the changes needed?
This change fixes the loading of class (which inherits from PipelineModel
class) from file.
E.g. Current issue:
```
CustomPipelineModel(PipelineModel):
def _transform(self, df):
...
CustomPipelineModel.save('path/to/file') # works
CustomPipelineModel.load('path/to/file') # wrong: results in PipelineModel()
instead of CustomPipelineModel()
CustomPipelineModel.transform() # wrong: results in calling
PipelineModel.transform() instead of CustomPipelineModel.transform()
```
Does this introduce any user-facing change?
No.
was:
PR: [https://github.com/apache/spark/pull/28110]
### What changes were proposed in this pull request?
pypsark.ml.pipeline.py line 245: Change PipelineModel(...) to self.cls(...)
### Why are the changes needed?
This change fixes the loading of class (which inherits from PipelineModel
class) from file.
E.g. Current issue:
```
CustomPipelineModel(PipelineModel):
def _transform(self, df):
...
CustomPipelineModel.save('path/to/file') # works
CustomPipelineModel.load('path/to/file') # wrong: results in PipelineModel()
instead of CustomPipelineModel()
CustomPipelineModel.transform() # wrong: results in calling
PipelineModel.transform() instead of CustomPipelineModel.transform()
```
### Does this introduce any user-facing change?
No.
> Changed PipelineModel(...) to self.cls(...) in
> pyspark.ml.pipeline.PipelineModelReader.load()
> ---------------------------------------------------------------------------------------------
>
> Key: SPARK-31339
> URL: https://issues.apache.org/jira/browse/SPARK-31339
> Project: Spark
> Issue Type: Bug
> Components: ML, PySpark
> Affects Versions: 2.4.5
> Reporter: Suraj
> Priority: Minor
> Labels: pull-request-available
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> PR: [https://github.com/apache/spark/pull/28110]
> What changes were proposed in this pull request?
> pypsark.ml.pipeline.py line 245: Change PipelineModel(...) to self.cls(...)
> Why are the changes needed?
> This change fixes the loading of class (which inherits from PipelineModel
> class) from file.
> E.g. Current issue:
> ```
> CustomPipelineModel(PipelineModel):
> def _transform(self, df):
> ...
> CustomPipelineModel.save('path/to/file') # works
> CustomPipelineModel.load('path/to/file') # wrong: results in PipelineModel()
> instead of CustomPipelineModel()
> CustomPipelineModel.transform() # wrong: results in calling
> PipelineModel.transform() instead of CustomPipelineModel.transform()
> ```
> Does this introduce any user-facing change?
> No.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
