Ruifeng Zheng created SPARK-61456:
-------------------------------------

             Summary: Support configurable intermediate storage levels in 
regression estimators
                 Key: SPARK-61456
                 URL: https://issues.apache.org/jira/browse/SPARK-61456
             Project: Spark
          Issue Type: Improvement
          Components: ML
    Affects Versions: 5.0.0
            Reporter: Ruifeng Zheng


LinearRegression, AFTSurvivalRegression, FMRegressor, and IsotonicRegression 
currently persist internally created training data at a hard-coded 
MEMORY_AND_DISK storage level. Caching the input DataFrame does not configure 
these intermediate datasets.

Expose the existing intermediateStorageLevel shared parameter on these four 
regressors in Scala and PySpark. Apply it to their persistence calls while 
retaining MEMORY_AND_DISK as the default. For LinearRegression, the setting 
applies to the block-based iterative solver.

Include PySpark constructor and setParams keyword support, setters, model 
parameter propagation, and checks in the existing estimator suites.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to