zhengruifeng commented on a change in pull request #27374:
[SPARK-30659][ML][PYSPARK] LogisticRegression blockify input vectors
URL: https://github.com/apache/spark/pull/27374#discussion_r371869290
##########
File path:
mllib/src/test/scala/org/apache/spark/ml/optim/aggregator/LogisticAggregatorSuite.scala
##########
@@ -32,21 +32,21 @@ class LogisticAggregatorSuite extends SparkFunSuite with
MLlibTestSparkContext {
override def beforeAll(): Unit = {
super.beforeAll()
- instances = Array(
+ instances = standardize(Array(
Review comment:
Two reasons:
1, we do need to standardize the input at each iteration;
2, if we keep this, then it is hard to use matrix operations in aggregator,
since we need: 1, element division, 2, broadcasting (block.matrix / stdvec),
3, special logic that if std==0 then output=0; the first 2 point may need to
use `breeze` instead of BLAS, the last point may even make `breeze` not
suitable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
