[
https://issues.apache.org/jira/browse/SPARK-61752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61752:
---------------------------------
Fix Version/s: connect-rust-4.3.0
> Keep the params of the native ML classes in PySpark's own param maps, of
> PySpark's own Params API
> -------------------------------------------------------------------------------------------------
>
> Key: SPARK-61752
> URL: https://issues.apache.org/jira/browse/SPARK-61752
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> The public classes of the native ML classes (pyspark.ml.connect.*, subclasses
> of PySpark's classes) keep their params in the native object, and their
> Params API is native code. PySpark's own Python code over a Params object's
> param maps does not apply to them: _paramMap and _defaultParamMap, _set,
> _setDefault, _copyValues, _resetUid, _input_kwargs, the order of
> extractParamMap, the TypeConverters' conversions, the messages of a call that
> does not bind, the Params of an old uid after an assignment of uid, and
> PySpark's own getters, setters, setParams and keyword_only constructors.
> This makes the param maps PySpark's, in the instance's attributes: a public
> class takes PySpark's own functions of the Params API (and its getters,
> setters, constructors and copy) as they are, and makes the native object's
> params those of the maps (by name) before each native member that reads them
> (a fit, a transform, an evaluation, a save, a model member). The maps of an
> object that PySpark's constructor did not make (a fit's model, a load, a
> native copy) are made of the native object's values at their first read.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]