GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/591
Add tests for FileLogger, EventLoggingListener, and ReplayListenerBus
Modifications to Spark core are limited to exposing functionality to test
files + minor style fixes.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark event-log-tests
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/591.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #591
----
commit e12f4b1ef342b6f0a1b2188904951bb4256263b2
Author: Andrew Or <[email protected]>
Date: 2014-04-11T21:16:49Z
Preliminary tests for EventLoggingListener (need major cleanup)
commit 5d38ffe98d9b15ce88518cdb50420fe6c39a04e2
Author: Andrew Or <[email protected]>
Date: 2014-04-11T21:43:49Z
Clean up EventLoggingListenerSuite + add comments
commit 769336f33e7800c4dc158e3926d74d4480597629
Author: Andrew Or <[email protected]>
Date: 2014-04-28T17:21:09Z
Merge branch 'master' of github.com:apache/spark into event-log-tests
commit 187bb2587b6df07a77a72d4be19e1ed0d12489ff
Author: Andrew Or <[email protected]>
Date: 2014-04-28T17:34:37Z
Formatting and renaming variables
commit ab66a846b23bcc16377afd35ec9b897f921b4626
Author: Andrew Or <[email protected]>
Date: 2014-04-29T03:11:47Z
Tests for FileLogger + delete file after tests
commit b9904537ca19e3e18504a1f448cc77958bbbb909
Author: Andrew Or <[email protected]>
Date: 2014-04-29T05:16:41Z
ReplayListenerBus suite - tests do not all pass yet
commit e0ba2f8e8c37adc4c57684b2a42aa20fc2e19b19
Author: Andrew Or <[email protected]>
Date: 2014-04-29T20:25:04Z
Fix test failures caused by race condition in processing/mutating events
An event can be mutated by the DAGScheduler in between being procssed by one
listener and being processed by another. This causes the ReplayListenerSuite
to be flaky. This commit ensures that the event logged is the same as the
original event received by the EventLoggingListener.
commit 862e752d7500d7f89c0a005123507fecb8911fc4
Author: Andrew Or <[email protected]>
Date: 2014-04-29T20:32:08Z
Merge branch 'master' of github.com:apache/spark into event-log-tests
Conflicts:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala
core/src/main/scala/org/apache/spark/util/FileLogger.scala
commit ad2beff5226b4ff93d36252cfceca883cfb789fa
Author: Andrew Or <[email protected]>
Date: 2014-04-29T21:17:45Z
Clean up EventLoggingListenerSuite + modify a few comments
commit 62010fd3a5f9ca08eedcf6fb41f37d6094b89295
Author: Andrew Or <[email protected]>
Date: 2014-04-29T21:25:11Z
More cleanup (renaming variables, updating comments etc)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
