HeartSaVioR edited a comment on issue #26671: Revert "[SPARK-26081][SPARK-29999]" URL: https://github.com/apache/spark/pull/26671#issuecomment-558478147 Ah yes you're right that it cannot be reverted cleanly - so there's physically no clean revert. Maybe I overthought here; I thought about how we deal with JIRA issue for SPARK-26081/SPARK-29999. If we reopen them (at least SPARK-26081) and open a chance to try to do the right fix, it'd be ideal if we have a "minimized" commit to revert the SPARK-26081 - so we can track how SPARK-26081 was introduced and reverted later, and re-introduced. If we would want to abandon the original idea of SPARK-26081 and close the issue as won't fix, any approach would be OK for me. Btw, would you mind if I ask for elaboration on the new suggestion on the new UT? > We can also add a new test case to check the behavior that an empty Dataframe will output exactly one empty file. I'm not familiar enough to understand the expectations/requirements on file sink; I feel the UT in SPARK-29999 can reside with reverting commit as the UT tests the regression what we've broken - we're reverting and adding the guard to prevent we don't break again. Is the new UT same case - did SPARK-26081 break the expectation? If not, that sounds to be on different purpose.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
