Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/20124
This basically works by splitting the array of ParamMaps into two. One
that has params that can be optimized by the estimator, and one that can be
parallelized over. These are then grouped together so that the estimator can
fit a sequence of Models. This allows us to reuse the previous API for fitting
multiple Models and still keep the parallelization logic pretty
straightforward. Model specific optimization support is just how it was before
there was any parallelism introduced too. I can explain in further detail or
make a design document if needed.
cc @MLnick @WeichenXu123 @jkbradley
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
