wypoon commented on issue #25673: [SPARK-28770][CORE][TEST] Fix ReplayListenerSuite tests that sometime… URL: https://github.com/apache/spark/pull/25673#issuecomment-528117897 @HeartSaVioR, on second thoughts, after offline discussion with @squito, I agree that the best way to fix the tests is to reimplement EventMonster. Instead of extending EventLoggingListener, EventMonster should simply buffer any and all events it receives. @squito pointed out that EventMonster was added when ReplayListenerSuite was first added, and that SparkFirehoseListener did not exist then. Extending EventLoggingListener and adding a mode where it simply appends events to a buffer (in its logEvent method) was an easy way of implementing EventMonster. However, as EventLoggingListener evolved, it did not call logEvent on every event, and it also wrote events as side effects that depended on its internal state, leading to the current difficulties. I have reimplemented EventMonster (extending SparkFirehoseListener) to simply buffer each and every event it receives. This matches the needs of ReplayListenerSuite better. With this change, testApplicationReplay no longer needs any logic to filter any events.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
