spark git commit: [SPARK-18060][ML] Avoid unnecessary computation for MLOR
Repository: spark
Updated Branches:
refs/heads/branch-2.1 c2ebda443 -> 56859c029
[SPARK-18060][ML] Avoid unnecessary computation for MLOR
## What changes were proposed in this pull request?
Before this patch, the gradient updates for multinomial logistic regression
were computed by an outer loop over the number of classes and an inner loop
over the number of features. Inside the inner loop, we standardized the feature
value (`value / featuresStd(index)`), which means we performed the computation
`numFeatures * numClasses` times. We only need to perform that computation
`numFeatures` times, however. If we re-order the inner and outer loop, we can
avoid this, but then we lose sequential memory access. In this patch, we
instead lay out the coefficients in column major order while we train, so that
we can avoid the extra computation and retain sequential memory access. We
convert back to row-major order when we create the model.
## How was this patch tested?
This is an implementation detail only, so the original behavior should be
maintained. All tests pass. I ran some performance tests to verify speedups.
The results are below, and show significant speedups.
## Performance Tests
**Setup**
3 node bare-metal cluster
120 cores total
384 gb RAM total
**Results**
NOTE: The `currentMasterTime` and `thisPatchTime` are times in seconds for a
single iteration of L-BFGS or OWL-QN.
|| numPoints | numFeatures | numClasses | regParam |
elasticNetParam | currentMasterTime (sec) | thisPatchTime (sec) |
pctSpeedup |
||-|---|--||---|---|---|--|
| 0 | 1e+07 | 100 | 500 | 0.5 |
0 |90 |18 | 80 |
| 1 | 1e+08 | 100 | 50 | 0.5 |
0 |90 |19 | 78 |
| 2 | 1e+08 | 100 | 50 | 0.05 |
1 |72 |19 | 73 |
| 3 | 1e+06 | 100 | 5000 | 0.5 |
0 |93 |53 | 43 |
| 4 | 1e+07 | 100 | 5000 | 0.5 |
0 | 900 | 390 | 56 |
| 5 | 1e+08 | 100 | 500 | 0.5 |
0 | 840 | 174 | 79 |
| 6 | 1e+08 | 100 | 200 | 0.5 |
0 | 360 |72 | 80 |
| 7 | 1e+08 | 1000 |5 | 0.5 |
0 | 9 | 3 | 66 |
Author: sethah
Closes #15593 from sethah/MLOR_PERF_COL_MAJOR_COEF.
(cherry picked from commit 46b2550bcd3690a260b995fd4d024a73b92a0299)
Signed-off-by: DB Tsai
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/56859c02
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/56859c02
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/56859c02
Branch: refs/heads/branch-2.1
Commit: 56859c029476bc41b2d2e05043c119146b287bce
Parents: c2ebda4
Author: sethah
Authored: Sat Nov 12 01:38:26 2016 +
Committer: DB Tsai
Committed: Sat Nov 12 01:41:29 2016 +
--
.../ml/classification/LogisticRegression.scala | 125 +++
1 file changed, 74 insertions(+), 51 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/56859c02/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
--
diff --git
a/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
b/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
index c465105..18b9b30 100644
---
a/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
+++
b/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
@@ -438,18 +438,14 @@ class LogisticRegression @Since("1.2.0") (
val standardizationParam = $(standardization)
def regParamL1Fun = (index: Int) => {
// Remove the L1 penalization on the intercept
-val isIntercept = $(fitIntercept) && ((index + 1) %
numFeaturesPlusIntercept == 0)
+val isIntercept = $(fitIntercept) && index >= numFeatures *
numCoefficientSets
if (isIntercept) {
0.0
} else {
if (standardizationParam) {
spark git commit: [SPARK-18060][ML] Avoid unnecessary computation for MLOR
Repository: spark
Updated Branches:
refs/heads/master ba23f768f -> 46b2550bc
[SPARK-18060][ML] Avoid unnecessary computation for MLOR
## What changes were proposed in this pull request?
Before this patch, the gradient updates for multinomial logistic regression
were computed by an outer loop over the number of classes and an inner loop
over the number of features. Inside the inner loop, we standardized the feature
value (`value / featuresStd(index)`), which means we performed the computation
`numFeatures * numClasses` times. We only need to perform that computation
`numFeatures` times, however. If we re-order the inner and outer loop, we can
avoid this, but then we lose sequential memory access. In this patch, we
instead lay out the coefficients in column major order while we train, so that
we can avoid the extra computation and retain sequential memory access. We
convert back to row-major order when we create the model.
## How was this patch tested?
This is an implementation detail only, so the original behavior should be
maintained. All tests pass. I ran some performance tests to verify speedups.
The results are below, and show significant speedups.
## Performance Tests
**Setup**
3 node bare-metal cluster
120 cores total
384 gb RAM total
**Results**
NOTE: The `currentMasterTime` and `thisPatchTime` are times in seconds for a
single iteration of L-BFGS or OWL-QN.
|| numPoints | numFeatures | numClasses | regParam |
elasticNetParam | currentMasterTime (sec) | thisPatchTime (sec) |
pctSpeedup |
||-|---|--||---|---|---|--|
| 0 | 1e+07 | 100 | 500 | 0.5 |
0 |90 |18 | 80 |
| 1 | 1e+08 | 100 | 50 | 0.5 |
0 |90 |19 | 78 |
| 2 | 1e+08 | 100 | 50 | 0.05 |
1 |72 |19 | 73 |
| 3 | 1e+06 | 100 | 5000 | 0.5 |
0 |93 |53 | 43 |
| 4 | 1e+07 | 100 | 5000 | 0.5 |
0 | 900 | 390 | 56 |
| 5 | 1e+08 | 100 | 500 | 0.5 |
0 | 840 | 174 | 79 |
| 6 | 1e+08 | 100 | 200 | 0.5 |
0 | 360 |72 | 80 |
| 7 | 1e+08 | 1000 |5 | 0.5 |
0 | 9 | 3 | 66 |
Author: sethah
Closes #15593 from sethah/MLOR_PERF_COL_MAJOR_COEF.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/46b2550b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/46b2550b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/46b2550b
Branch: refs/heads/master
Commit: 46b2550bcd3690a260b995fd4d024a73b92a0299
Parents: ba23f76
Author: sethah
Authored: Sat Nov 12 01:38:26 2016 +
Committer: DB Tsai
Committed: Sat Nov 12 01:38:26 2016 +
--
.../ml/classification/LogisticRegression.scala | 125 +++
1 file changed, 74 insertions(+), 51 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/46b2550b/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
--
diff --git
a/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
b/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
index c465105..18b9b30 100644
---
a/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
+++
b/mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
@@ -438,18 +438,14 @@ class LogisticRegression @Since("1.2.0") (
val standardizationParam = $(standardization)
def regParamL1Fun = (index: Int) => {
// Remove the L1 penalization on the intercept
-val isIntercept = $(fitIntercept) && ((index + 1) %
numFeaturesPlusIntercept == 0)
+val isIntercept = $(fitIntercept) && index >= numFeatures *
numCoefficientSets
if (isIntercept) {
0.0
} else {
if (standardizationParam) {
regParamL1
} else {
-val featureIndex = if ($(fitIntercept)) {
