Ruifeng Zheng created SPARK-61954:
-------------------------------------

             Summary: Support configurable intermediate storage levels in 
BisectingKMeans
                 Key: SPARK-61954
                 URL: https://issues.apache.org/jira/browse/SPARK-61954
             Project: Spark
          Issue Type: Improvement
          Components: ML
    Affects Versions: 5.0.0
            Reporter: Ruifeng Zheng


BisectingKMeans currently hard-codes MEMORY_AND_DISK for its intermediate 
vectors, norms, and cluster-assignment RDDs.

Expose the shared intermediateStorageLevel parameter in the Scala and PySpark 
ML APIs, including the Python constructor, setParams, and setter. Pass the 
configured level to the shared training implementation and apply it to all 
three intermediate cache sites. Preserve the default MEMORY_AND_DISK storage 
level and the existing choice between caching vectors for uncached input and 
norms for cached input.

Extend the existing Scala and Python test suites. New API annotations use 4.5.0.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to