cloud-fan commented on PR #41332: URL: https://github.com/apache/spark/pull/41332#issuecomment-1596844463
> Aren't we looking for a generic solution here? Wouldn't this create issues if the string-type "timestamp" column do not always contains standard timestamp string? My take is for a general solution, the generated predicate will be very complicated to handle non-standard timestamp strings, probably with a lot of `Or` predicates, and the file source pushdown may not work well with such predicates. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
