Hyukjin Kwon created SPARK-61752:
------------------------------------
Summary: Keep the params of the native ML classes in PySpark's own
param maps, of PySpark's own Params API
Key: SPARK-61752
URL: https://issues.apache.org/jira/browse/SPARK-61752
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
The public classes of the native ML classes (pyspark.ml.connect.*, subclasses
of PySpark's classes) keep their params in the native object, and their Params
API is native code. PySpark's own Python code over a Params object's param maps
does not apply to them: _paramMap and _defaultParamMap, _set, _setDefault,
_copyValues, _resetUid, _input_kwargs, the order of extractParamMap, the
TypeConverters' conversions, the messages of a call that does not bind, the
Params of an old uid after an assignment of uid, and PySpark's own getters,
setters, setParams and keyword_only constructors.
This makes the param maps PySpark's, in the instance's attributes: a public
class takes PySpark's own functions of the Params API (and its getters,
setters, constructors and copy) as they are, and makes the native object's
params those of the maps (by name) before each native member that reads them (a
fit, a transform, an evaluation, a save, a model member). The maps of an object
that PySpark's constructor did not make (a fit's model, a load, a native copy)
are made of the native object's values at their first read.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]