Could I be experiencing the same thing?
https://www.dropbox.com/s/egtj1056qeudswj/sparkwut.png?dl=0
On Wed, Nov 16, 2016 at 10:37 AM, Shreya Agarwal
wrote:
> I think that is a bug. I have seen that a lot especially with long running
> jobs where Spark skips a lot of
I think that is a bug. I have seen that a lot especially with long running jobs
where Spark skips a lot of stages because it has pre-computed results. And some
of these are never marked as completed, even though in reality they are. I
figured this out because I was using the interactive shell