zhengruifeng commented on a change in pull request #27519: [SPARK-30770][ML]
avoid vector conversion in GMM.transform
URL: https://github.com/apache/spark/pull/27519#discussion_r376937067
##########
File path:
mllib-local/src/main/scala/org/apache/spark/ml/stat/distribution/MultivariateGaussian.scala
##########
@@ -43,46 +43,40 @@ class MultivariateGaussian @Since("2.0.0") (
require(cov.numCols == cov.numRows, "Covariance matrix must be square")
require(mean.size == cov.numCols, "Mean vector length must match covariance
matrix size")
- /** Private constructor taking Breeze types */
- private[ml] def this(mean: BDV[Double], cov: BDM[Double]) = {
- this(Vectors.fromBreeze(mean), Matrices.fromBreeze(cov))
- }
-
- @transient private lazy val breezeMu = mean.asBreeze.toDenseVector
-
/**
* Compute distribution dependent constants:
* rootSigmaInv = D^(-1/2)^ * U.t, where sigma = U * D * U.t
* u = log((2*pi)^(-k/2)^ * det(sigma)^(-1/2)^)
*/
- @transient private lazy val (rootSigmaInv: BDM[Double], u: Double) =
calculateCovarianceConstants
+ @transient private lazy val tuple3 = {
Review comment:
it is said in
[LeastSquaresAggregator](https://github.com/apache/spark/blob/12e1bbaddbb2ef304b5880a62df6683fcc94ea54/mllib/src/main/scala/org/apache/spark/ml/optim/aggregator/LeastSquaresAggregator.scala#L188)
that
> // do not use tuple assignment above because it will circumvent the
@transient tag
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
