GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/7699
[SPARK-5259][CORE] don't submit stage until its dependencies map outputs
are registered
Before this change, failure & retry could result in a case where a stage
got submitted before the map output from its dependencies get registered. This
was due to an error in the condition for registering map outputs.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark SPARK-5259
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7699.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7699
----
commit 641936f6ac61c21dbe20049e4745e61c2ed41ed9
Author: hushan[è¡ç] <[email protected]>
Date: 2015-01-15T03:52:10Z
Make sure mapStage.pendingtasks is set() while MapStage.isAvailable is true
while stage was retry
commit 741ab4dbbb1be88d290a1d773103645a32615f34
Author: hushan[è¡ç] <[email protected]>
Date: 2015-01-26T07:23:16Z
Add sub-class canEqual to make ShuffleMapTask not equal Result Task while
stageId and partId are same
commit 5ec8a8217832b5f27cf724a2b3f2ee41092c9a4b
Author: hushan[è¡ç] <[email protected]>
Date: 2015-01-29T09:16:48Z
Refine
commit 90447201931bccee87d60a42c4501d90993f10b7
Author: hushan[è¡ç] <[email protected]>
Date: 2015-01-30T03:21:14Z
add parameter type
commit 03db62475d0f0245563ee7d0148b6683c2a254d9
Author: hushan[è¡ç] <[email protected]>
Date: 2015-01-30T07:01:20Z
Refine
commit 8dfdc18e3f1d92930cdd6f818111d88dc4bbc358
Author: hushan[è¡ç] <[email protected]>
Date: 2015-02-27T02:43:56Z
Refine log message use string interpolation
commit c286f7a3bb167817eae423a83901041476a35b4a
Author: hushan[è¡ç] <[email protected]>
Date: 2015-02-27T09:19:15Z
Refine with the latest spark
commit dcf153309894f19271d5db0e069a2a6474e1a664
Author: hushan[è¡ç] <[email protected]>
Date: 2015-06-18T13:59:03Z
Refine solution and fix bug in code
commit 27da8e7e4d76b36d38c8d01d7751bec51132bc0f
Author: hushan[è¡ç] <[email protected]>
Date: 2015-06-18T14:07:30Z
Refine codeStyle
commit bd5fec46125f362e4fc2c96327cc5a169f809dfa
Author: hushan[è¡ç] <[email protected]>
Date: 2015-06-19T07:38:52Z
Refine Tests
commit 4dbe4d32c9364d8c11c417e0e91688ee20160dab
Author: hushan[è¡ç] <[email protected]>
Date: 2015-06-19T07:47:35Z
Refine assert exception
commit 9dfff638ba76883404fb8166622e86089c646b5c
Author: hushan[è¡ç] <[email protected]>
Date: 2015-06-19T08:34:39Z
Refine the test 'ignore late map task completions'
commit e1e0b66abc7430e88b8a58ec0c81a8d500bf6caa
Author: hushan[è¡ç] <[email protected]>
Date: 2015-07-03T07:12:27Z
Refine the testcase
commit 23792509f2061012495035717071c5c196303343
Author: hushan[è¡ç] <[email protected]>
Date: 2015-07-03T12:56:44Z
Refine scala style
commit 314873af23f391254dc89cf4e233d2571f3358d4
Author: hushan[è¡ç] <[email protected]>
Date: 2015-07-21T03:25:24Z
Only tracker partitionId to instead of Task
commit 7ae128e2f2888a5840be785da34cc20902ba8b5c
Author: hushan[è¡ç] <[email protected]>
Date: 2015-07-21T03:31:15Z
Refine test name
commit b2df3fd330325a1bf3dc5782e973506202af92f4
Author: hushan[è¡ç] <[email protected]>
Date: 2015-07-21T12:23:33Z
refine suite code
commit 9f5276f94813636c0980c2c8b25bcfe5e3f1b02a
Author: Imran Rashid <[email protected]>
Date: 2015-07-27T15:54:43Z
Merge branch 'master' into SPARK-5259
commit e4aa266450a867866b4f2000bd85347330e1b5e0
Author: Imran Rashid <[email protected]>
Date: 2015-07-27T15:59:42Z
minor style updates
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
