zhengruifeng commented on a change in pull request #27374:
[SPARK-30659][ML][PYSPARK] LogisticRegression blockify input vectors
URL: https://github.com/apache/spark/pull/27374#discussion_r371824168
##########
File path:
mllib/src/main/scala/org/apache/spark/ml/optim/aggregator/LogisticAggregator.scala
##########
@@ -339,6 +447,116 @@ private[ml] class LogisticAggregator(
lossSum += weight * loss
}
+ /** Update gradient and loss using multinomial (softmax) loss function. */
+ private def multinomialUpdateInPlace(block: InstanceBlock): Unit = {
+ val size = block.size
+ val localGradientSumArray = gradientSumArray
+
+ // mat here represents margins, shape: S X C
+ val mat = if (size == blockSize) {
+ multinomialAuxiliaryMat
+ } else {
+ // the last block within one partition may be of size less than blockSize
+ new DenseMatrix(size, numClasses, Array.ofDim[Double](size * numClasses))
+ }
+
+ if (fitIntercept) {
+ var i = 0
+ while (i < size) {
+ var j = 0
+ while (j < numClasses) {
+ mat.update(i, j, multinomialIntercept(j))
+ j += 1
+ }
+ i += 1
+ }
+ BLAS.gemm(1.0, block.matrix, multinomialLinear.transpose, 1.0, mat)
+ } else {
+ BLAS.gemm(1.0, block.matrix, multinomialLinear.transpose, 0.0, mat)
+ }
+
+ // in-place convert margins to multipliers
+ // then, mat represents multipliers
+ var i = 0
+ val tmp = Array.ofDim[Double](numClasses)
+ while (i < size) {
+ val weight = block.getWeight(i)
+ if (weight > 0) {
+ weightSum += weight
+ val label = block.getLabel(i)
+
+ var maxMargin = Double.NegativeInfinity
+ var j = 0
+ while (j < numClasses) {
+ tmp(j) = mat(i, j)
+ maxMargin = math.max(maxMargin, tmp(j))
+ j += 1
+ }
+
+ // marginOfLabel is margins(label) in the formula
+ val marginOfLabel = tmp(label.toInt)
+
+ var sum = 0.0
+ j = 0
+ while (j < numClasses) {
+ if (maxMargin > 0) tmp(j) -= maxMargin
+ val exp = math.exp(tmp(j))
+ sum += exp
+ tmp(j) = exp
+ j += 1
+ }
+
+ j = 0
+ while (j < numClasses) {
+ val multiplier = weight * (tmp(j) / sum - (if (label == j) 1.0 else
0.0))
+ mat.update(i, j, multiplier)
+ j += 1
+ }
+
+ if (maxMargin > 0) {
+ lossSum += weight * (math.log(sum) - marginOfLabel + maxMargin)
+ } else {
+ lossSum += weight * (math.log(sum) - marginOfLabel)
+ }
+ } else {
+ var j = 0
+ while (j < numClasses) {
+ mat.update(i, j, 0.0)
+ j += 1
+ }
+ }
+ i += 1
+ }
+
+ // block.matrix: S X F, unknown type
+ // mat (multipliers): S X C, dense
+ // multinomialLinearGradSumMat: F X C, dense
+ // gradSumMat(gradientSumArray): C X FPI (numFeaturesPlusIntercept), dense
+ block.matrix match {
+ case dm: DenseMatrix if !fitIntercept =>
+ // If fitIntercept==false, gradientSumArray += mat.T X matrix
+ // GEMM requires block.matrix is dense
+ val gradSumMat = new DenseMatrix(numClasses, numFeatures,
localGradientSumArray)
+ BLAS.gemm(1.0, mat.transpose, dm, 1.0, gradSumMat)
Review comment:
Since `gradientSumArray` is for Matrix of shape CXFPI, and `BLAS.gemm`
requires the output matrix is not transposed. So only if F(numFeature) ==
FPI(numFeaturesPlusIntercept) and input block is dense, can I use `BLAS.gemm`
to directly update `gradientSumArray`.
Otherwise, I need to output the result to a temp matrix
`multinomialLinearGradSumMat`, and then add elements to `gradientSumArray`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
