Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/2451#discussion_r17812518
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/optimization/Updater.scala ---
@@ -145,12 +150,151 @@ class SquaredL2Updater extends Updater {
// w' = w - thisIterStepSize * (gradient + regParam * w)
// w' = (1 - thisIterStepSize * regParam) * w - thisIterStepSize *
gradient
val thisIterStepSize = stepSize / math.sqrt(iter)
- val brzWeights: BV[Double] = weightsOld.toBreeze.toDenseVector
- brzWeights :*= (1.0 - thisIterStepSize * regParam)
- brzAxpy(-thisIterStepSize, gradient.toBreeze, brzWeights)
- val norm = brzNorm(brzWeights, 2.0)
+ scal(1.0 - thisIterStepSize * regParam, weightsOld)
+ axpy(-thisIterStepSize, gradient, weightsOld)
+ val norm = brzNorm(weightsOld.toBreeze, 2.0)
- (Vectors.fromBreeze(brzWeights), 0.5 * regParam * norm * norm)
+ (weightsOld, 0.5 * regParam * norm * norm)
}
}
+/**
+ * :: DeveloperApi ::
+ * Class used to perform steps (weight update) using Gradient Descent
methods.
+ *
+ * For general minimization problems, or for regularized problems of the
form
+ * min L(w) + regParam * R(w),
+ * the compute function performs the actual update step, when given some
+ * (e.g. stochastic) gradient direction for the loss L(w),
+ * and a desired step-size (learning rate).
+ *
+ * The updater is responsible to also perform the update coming from the
+ * regularization term R(w) (if any regularization is used).
+ */
+@DeveloperApi
+abstract class MultiModelUpdater extends Serializable {
+ /**
+ * Compute an updated value for weights given the gradient, stepSize,
iteration number and
+ * regularization parameter. Also returns the regularization value
regParam * R(w)
+ * computed using the *updated* weights.
+ *
+ * @param weightsOld - Column matrix of size dx1 where d is the number
of features.
--- End diff --
update doc (matrix size)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
