HeartSaVioR commented on issue #27620: URL: https://github.com/apache/spark/pull/27620#issuecomment-617015907
> Does the modified code behaves the same way you've shown on the pictures attached? Yes, because the code change in review phase excluding tests was only adding lower-bar condition. (Existing tests make sure it doesn't break the original functionality.) From the manual test I assume there're bunch of files (say, 100,000+) obtained via listing the input directory (the main use case of the feature is to catch up existing bunch of files faster) - in this case newly added condition wouldn't take effect before it finally catches up the all files. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
