dongjoon-hyun edited a comment on issue #25514: [SPARK-28784][SS]StreamExecution and StreamingQueryManager should utilize CheckpointFileManager to interact with checkpoint directories URL: https://github.com/apache/spark/pull/25514#issuecomment-523733239 Hi, @shrutig . Did you run the unit tests locally? I'm wondering if this PR pass in your local environment. In the last two Jenkins runs, there are four relevant failures identically. ``` org.apache.spark.sql.streaming.StreamSuite.explain - redaction 0.61 sec 1 org.apache.spark.sql.streaming.StreamSuite.should resolve the checkpoint path 23 ms 1 org.apache.spark.sql.streaming.StreamingQuerySuite.StreamExecution metadata garbage collection 0.78 sec 1 org.apache.spark.sql.streaming.test.DataStreamReaderWriterSuite.source metadataPath ``` I'd like to recommend the following three things. 1. Please don't change the existing UTs. 2. Please add a new UT which fails in the `master` branch without your patch. 3. Run the UTs locally in order to make it sure that your PR is valid.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
