[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r449350240 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala ## @@ -83,7 +85,7 @@ abstract class

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r449241653 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala ## @@ -83,7 +85,7 @@ abstract class

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r449243486 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala ## @@ -83,7 +85,7 @@ abstract class

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r449241653 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala ## @@ -83,7 +85,7 @@ abstract class

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r449241653 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala ## @@ -83,7 +85,7 @@ abstract class

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r449241653 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala ## @@ -83,7 +85,7 @@ abstract class

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r443087412 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala ## @@ -356,26 +380,35 @@ object

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r441280950 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala ## @@ -19,18 +19,18 @@ package

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r441141560 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/pathfilters/PathFilterIgnoreOldFiles.scala ## @@ -0,0 +1,51 @@ +/* +

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r440895770 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/pathfilters/PathFilterIgnoreNonData.scala ## @@ -0,0 +1,37 @@ +/* + *

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r440893695 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala ## @@ -356,26 +380,35 @@ object

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r440891685 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala ## @@ -176,9 +180,27 @@ object