[GitHub] [spark] bart-samwel commented on pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread GitBox
bart-samwel commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-644807406 > @bart-samwel To your point, I wonder if "_fromModifiedDate_" would be more appropriate? How about fromModified_Timestamp_? And then add untilModifiedTimestamp?

[GitHub] [spark] bart-samwel commented on pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

2020-06-16 Thread GitBox
bart-samwel commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-644719642 The option `fileModifiedDate` doesn't say at all that it's a minimum modified date. I can imagine use cases for lower bounds, upper bounds, ranges. That requires at least