Re: How to find how much data will be train in mllib or how much the spark job is completed ?

This page gives details on the monitoring available http://spark.apache.org/docs/latest/monitoring.html. You can get a UI showing Jobs, Stages and Tasks with an indication how far completed the job is. The UI is usually on port 4040 of the machine where you run the spark driver program. The

Re: How to find how much data will be train in mllib or how much the spark job is completed ?

so you could query the rest api in code. E.g. /applications//stages provides details on the number of active and completed tasks in each stage - Robin East Spark GraphX in Action Michael Malak and Robin East Manning Publications Co. http://www.manning.com/books/spark-graphx-in-action --