Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22659
In the test, "multinomial logistic regression with intercept with
elasticnet regularization" in the "LogisticRegressionSuite", taking around 1
minute to train 2 logistic regression model.
However after analyzing the training cost over iteration, we can reduce the
computation time by 50%.
Training cost vs iteration for model 1
So, model1 is converging after iteration 200.
Training cost vs iteration for model 2:
image
After around 50 iteration, model2 is converging.
So, if we give maximum iteration for model1 and model2 as 220 and 90
respectively, we can reduce the computation time by half.
Computation time in local setup :
Before change:
~54 sec
After change:
~35 sec
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
