Re: How to use K-fold validation in spark-1.0?

2014-06-24 Thread holdingonrobin
Anyone knows anything about it? Or should I actually move this topic to a MLlib specif mailing list? Any information is appreciated! Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-K-fold-validation-in-spark-1-0-tp8142p8172.html Sent from

Re: How to use K-fold validation in spark-1.0?

2014-06-24 Thread Eustache DIEMERT
I'm interested in this topic too :) Are the MLLib core devs on this list ? E/ 2014-06-24 14:19 GMT+02:00 holdingonrobin robinholdin...@gmail.com: Anyone knows anything about it? Or should I actually move this topic to a MLlib specif mailing list? Any information is appreciated! Thanks!

Re: How to use K-fold validation in spark-1.0?

2014-06-24 Thread Evan R. Sparks
There is a method in org.apache.spark.mllib.util.MLUtils called kFold which will automatically partition your dataset for you into k train/test splits at which point you can build k different models and aggregate the results. For example (a very rough sketch - assuming I want to do 10-fold cross

Re: How to use K-fold validation in spark-1.0?

2014-06-24 Thread holdingonrobin
Thanks Evan! I think it works! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-K-fold-validation-in-spark-1-0-tp8142p8188.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

How to use K-fold validation in spark-1.0?

2014-06-23 Thread holdingonrobin
Hello, I noticed there are some discussions about implementing K-fold validation to Mllib on Spark and believe it should be in Spark-1.0 now. However there isn't any documentation or example about how to use it in the training. While I am reading the code to find out, does anyone use it