Does Spark always wait for stragglers to finish running?

2014-09-15 Thread Pramod Biligiri
Hi, I'm running Spark tasks with speculation enabled. I'm noticing that Spark seems to wait in a given stage for all stragglers to finish, even though the speculated alternative might have finished sooner. Is that correct? Is there a way to indicate to Spark not to wait for stragglers to finish?

Re: Does Spark always wait for stragglers to finish running?

2014-09-15 Thread Du Li
@spark.apache.orgmailto:user@spark.apache.org user@spark.apache.orgmailto:user@spark.apache.org Subject: Does Spark always wait for stragglers to finish running? Hi, I'm running Spark tasks with speculation enabled. I'm noticing that Spark seems to wait in a given stage for all stragglers to finish

Re: Does Spark always wait for stragglers to finish running?

2014-09-15 Thread Pramod Biligiri
Date: Monday, September 15, 2014 at 3:30 PM To: user@spark.apache.org user@spark.apache.org Subject: Does Spark always wait for stragglers to finish running? Hi, I'm running Spark tasks with speculation enabled. I'm noticing that Spark seems to wait in a given stage for all stragglers

Re: Does Spark always wait for stragglers to finish running?

2014-09-15 Thread Matei Zaharia
/latest/configuration.html From: Pramod Biligiri pramodbilig...@gmail.com Date: Monday, September 15, 2014 at 3:30 PM To: user@spark.apache.org user@spark.apache.org Subject: Does Spark always wait for stragglers to finish running? Hi, I'm running Spark tasks with speculation enabled. I'm noticing