[GitHub] [spark] zhengruifeng commented on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations

2019-12-28 Thread GitBox 
zhengruifeng commented on issue #27011: [SPARK-30354][ML] GBT reuse 
DecisionTreeMetadata among iterations
URL: https://github.com/apache/spark/pull/27011#issuecomment-569471446
 
 
   Thanks @srowen @huaxingao for reviewing!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] zhengruifeng commented on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations

2019-12-25 Thread GitBox 
zhengruifeng commented on issue #27011: [SPARK-30354][ML] GBT reuse 
DecisionTreeMetadata among iterations
URL: https://github.com/apache/spark/pull/27011#issuecomment-568991035
 
 
   testcode:
   ```scala
   import org.apache.spark.ml.regression._
   import org.apache.spark.storage.StorageLevel
   
   var df = spark.read.format("libsvm").load("/data1/Datasets/a9a/a9a")
   
   (0 until 8).foreach{ _ => df = df.union(df) }
   df.persist(StorageLevel.MEMORY_AND_DISK)
   
   df.count
   df.count
   df.count
   
   val gbt = new GBTRegressor().setMaxIter(10)
   val gbtm = gbt.fit(df)
   
   
   val start = System.currentTimeMillis; val gbtm = gbt.fit(df); val end = 
System.currentTimeMillis; end - start
   
   ```
   
   reusing treeMetadata in this PR: 176969
   master: 191657


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org