Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/17000
@MLnick It looks like VF-LBFGS has a different scenario. In VF algos, the
vectors will be too large to store in driver memory, so we slice the vectors
into different machines (stored by `RDD[Vector], and the use partitionID as
slice key).
and , in VF-LBFGS, there're only very few large vectors(usually 4-10
vectors) need to aggregate together. so, what this PR do looks different with
VF-LBFGS.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
