gengliangwang commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-647879761
> Hadoop's PathFilter only gives the user Path without metadata. But this is an interesting point. The user needs a file metadata filter essentially. For example, some user may ask to filter files using the file size. But supporting a filter in multiple languages is hard. +1 with @zsxwing . Supporting multiple languages is important for Spark. Also, it seems a bit complicated if we create an API accepting Hadoop's PathFilter as parameter. Ideally, users should be able to use the APIs without basic data types. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
