[GitHub] spark issue #22659: [SPARK-25623][TEST] Reduce test time of LogisticRegressi...

[shahidki31]

Github user shahidki31 commented on the issue:

    https://github.com/apache/spark/pull/22659
  
    In the test, "multinomial logistic regression with intercept with 
elasticnet regularization" in the "LogisticRegressionSuite", taking around 1 
minute to train 2 logistic regression model.
    However after analyzing the training cost over iteration, we can reduce the 
computation time by 50%.
    Training cost vs iteration for model 1
    
    
![image](https://user-images.githubusercontent.com/23054875/46590546-c496e880-cad1-11e8-8539-5bc9853c33ca.png)
    
    
    So, model1 is converging after iteration 200.
    
    Training cost vs iteration for model 2:
    image
    
![image](https://user-images.githubusercontent.com/23054875/46590551-ca8cc980-cad1-11e8-8e83-24ad220e1618.png)
    
    After around 50 iteration, model2 is converging.
    So, if we give maximum iteration for model1 and model2 as 220 and 90 
respectively, we can reduce the computation time by half.
    
    Computation time in local setup :
    Before change:
    ~54 sec
    After change:
    ~35 sec


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org