Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/16251#discussion_r92487038
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamOptions.scala
---
@@ -53,4 +53,15 @@ class FileStreamOptions(parameters: C
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/16251
[SPARK-18826][SS]Add 'newestFirst' option to FileStreamSource
## What changes were proposed in this pull request?
When starting a stream with a lot of backfill and maxFilesPerTrigger, the
