zhengruifeng commented on a change in pull request #27374:
[SPARK-30659][ML][PYSPARK] LogisticRegression blockify input vectors
URL: https://github.com/apache/spark/pull/27374#discussion_r371865596
##########
File path:
mllib/src/main/scala/org/apache/spark/ml/optim/aggregator/LogisticAggregator.scala
##########
@@ -209,6 +208,70 @@ private[ml] class LogisticAggregator(
s"got type ${bcCoefficients.value.getClass}.)")
}
+ // Helper vectors and matrices for binary:
+ @transient private lazy val binaryLinear = {
Review comment:
`binaryLinear`, `binaryIntercept`, `multinomialLinear`,
`multinomialIntercept` are the linear and bias part of coefficients,
repectively.
`binaryLinearGradSumVec` (numFeatures) and `multinomialLinearGradSumMat`
(numClassXnumFeatures) are used to store result of `gemv`/`gemm` if
`fitIntercept==True`, since `gradientSumArray` contains gradient sums of
intercepts and can not be used directly in `gemv`/`gemm`.
`auxiliaryVec` (blockSize) and `multinomialAuxiliaryMat`
(blockSizeXnumClasses) are used to store the intermediate
multiplication(margins) and multipliers.
they can be used among blocks, and if they are used multi-times in one call
we can assign them to local variables.
However I am OK to make them local variables, since I guess they are not the
bottleneck.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
