HyukjinKwon commented on issue #23288: [SPARK-26339][SQL]Throws better exception when reading files that start with underscore URL: https://github.com/apache/spark/pull/23288#issuecomment-450125081 Yea, listing files itself is non-trivial in some cases (in particular when you use, for instance, S3) and extra listing up should be avoided. Looks the change is a bit invasive, and does another file listing. Can we simply do the check when the existing file list happens, rather than doing another file listing?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
