GitHub user xuanyuanking opened a pull request:

    [SPARK-24036][SS] Scheduler changes for continuous processing shuffle 

    ## What changes were proposed in this pull request?
    This is the last part of the preview PRs, the mainly change is in 
DAGScheduler, we added a interface to submit whole job at once.
    In order to work and compile properly, this also including several parts in 
#21332 and #21293. I will marked the duplicated part for convenient review. 
Sorry about this...
    ## How was this patch tested?
    Added new UT

You can merge this pull request into a Git repository by running:

    $ git pull SPARK-24304

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21353
commit fbcc88bfb0d2fb6dbaae9664d6f0852b71e64f2b
Author: Yuanjian Li <xyliyuanjian@...>
Date:   2018-05-10T09:18:16Z

    commit for continuous map output tracker

commit 44ae9d917c354d780071a8e112a118674865143d
Author: Yuanjian Li <xyliyuanjian@...>
Date:   2018-05-04T03:58:06Z

    INF-SPARK-1382: Continuous shuffle map task implementation and output 
trackder support

commit af2d60854856e669f40a03b76fffe02dac7b79c2
Author: Yuanjian Li <xyliyuanjian@...>
Date:   2018-05-10T13:23:39Z

    Address comments

commit 56442dc1c7450518d9bc84b4bfeddb017daa967b
Author: Yuanjian Li <xyliyuanjian@...>
Date:   2018-05-11T12:12:29Z

    Fix mima test

commit 2ac9980f30b8b50809aa780035281f6a62ad9573
Author: Yuanjian Li <xyliyuanjian@...>
Date:   2018-05-17T13:01:33Z

    add inteface to submit all stages of one job in DAGScheduler

commit 0f070fc5b26a3e2d5c30fdd7d59c4dd8896255ac
Author: Yuanjian Li <xyliyuanjian@...>
Date:   2018-05-17T13:05:29Z


commit 42e2a63f68ba12264f001ed53ff33c27c1287de8
Author: Yuanjian Li <xyliyuanjian@...>
Date:   2018-05-17T13:36:32Z

    Merge SPARK-24304



To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to