dongjoon-hyun commented on issue #25514: [SPARK-28784][SS]StreamExecution and StreamingQueryManager should utilize CheckpointFileManager to interact with checkpoint directories URL: https://github.com/apache/spark/pull/25514#issuecomment-523733239 Hi, @shrutig . Did you run the unit test locally? Does this PR pass in your environment? In the last two Jenkins runs, there are four relevant failures. ``` org.apache.spark.sql.streaming.StreamSuite.explain - redaction 0.61 sec 1 org.apache.spark.sql.streaming.StreamSuite.should resolve the checkpoint path 23 ms 1 org.apache.spark.sql.streaming.StreamingQuerySuite.StreamExecution metadata garbage collection 0.78 sec 1 org.apache.spark.sql.streaming.test.DataStreamReaderWriterSuite.source metadataPath ``` I'd like to recommend the following three things. 1. Please don't change the existing UTs. 2. Please add a new UT which fails in the `master` branch without your patch. 3. Run the UTs locally in order to make it sure that your PR is valid.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
