Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/14731
Sean, I think I've managed to delete the lines where you were asking about
globs
> Am I right that the net change here is not an optimization but an
expansion of the behavior to support globs rather than single dirs?
There's no changes in this source to change the expansion policy; that went
in with [SPARK-14976](https://issues.apache.org/jira/browse/SPARK-14976), "make
StreamingContext.textFileStream support wildcard". This updates the docs to
cover what goes on (the wildcard covering directories, but not the files inside
them), and makes the scan much more efficient on object stores. No changes in
the semantics of what gets found or when things get found
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
