GitHub user mengxr opened a pull request:
https://github.com/apache/spark/pull/9454
[WIP][SPARK-11217][ML] save/load for non-meta estimators and transformers
This PR implements the default save/load for non-meta estimators and
transformers using the JSON serialization of param values. The saved metadata
includes:
* class name
* uid
* timestamp
* paramMap
The save/load interface is similar to DataFrames. We use the current active
context by default, which should be sufficient for most use cases.
~~~scala
instance.save.to("path")
instance.save.options("overwrite" -> "true").to("path")
instance.save.context(sqlContext).to("path")
Instance.load.from("path")
~~~
The param handling is different from the design doc. We didn't save default
and user-set params separately, and when we load it back, all parameters are
user-set. This does cause issues. But it also cause other issues if we modify
the default params.
TODOs:
* [ ] Java test
* [ ] a follow-up PR to implement default save/load for all non-meta
estimators and transformers
cc @jkbradley
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mengxr/spark SPARK-11217
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9454.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9454
----
commit cd1c7eae3246f93b6ee4e044443adfe57fdf1386
Author: Xiangrui Meng <[email protected]>
Date: 2015-11-03T18:56:22Z
initial implementation
commit df81d61f73c6a854913df638770f0b0409f046a3
Author: Xiangrui Meng <[email protected]>
Date: 2015-11-03T23:41:58Z
update doc and test
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
