Joseph K. Bradley created SPARK-21166:
-----------------------------------------
Summary: Automated ML persistence
Key: SPARK-21166
URL: https://issues.apache.org/jira/browse/SPARK-21166
Project: Spark
Issue Type: New Feature
Components: ML
Affects Versions: 2.2.0
Reporter: Joseph K. Bradley
This JIRA is for discussing the possibility of automating ML persistence.
Currently, custom save/load methods are written for every Model. However, we
could design a mixin which provides automated persistence, inspecting model
data and Params and reading/writing (known types) automatically. This was
brought up in discussions with developers behind
https://github.com/azure/mmlspark
Some issues we will need to consider:
* Providing generic mixin usable in most or all cases
* Handling corner cases (strange Param types, etc.)
* Backwards compatibility (loading models saved by old Spark versions)
Because of backwards compatibility in particular, it may make sense to
implement testing for that first, before we try to address automated
persistence: [SPARK-15573]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
