Joseph K. Bradley reassigned SPARK-21088:

    Assignee: Weichen Xu

> CrossValidator, TrainValidationSplit should collect all models when fitting: 
> Python API
> ---------------------------------------------------------------------------------------
>                 Key: SPARK-21088
>                 URL: https://issues.apache.org/jira/browse/SPARK-21088
>             Project: Spark
>          Issue Type: Sub-task
>          Components: ML, PySpark
>    Affects Versions: 2.2.0
>            Reporter: Joseph K. Bradley
>            Assignee: Weichen Xu
>            Priority: Major
> In pyspark:
> We add a parameter whether to collect the full model list when 
> CrossValidator/TrainValidationSplit training (Default is NOT, avoid the 
> change cause OOM)
> Add a method in CrossValidatorModel/TrainValidationSplitModel, allow user to 
> get the model list
> CrossValidatorModelWriter add a “option”, allow user to control whether to 
> persist the model list to disk.
> Note: when persisting the model list, use indices as the sub-model path

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to