> @udim I see you touched `LocalFileSystem._list()` recently(ish) in > [36a2506](https://github.com/apache/beam/commit/36a250615b4a5d72c80b03f8f87957b5c0aaf791). > Is there a valid reason to not support `foo/*/file.txt` on filesystems with > directories or was it just overlooked?
Hi @joar. I don't remember why exactly, but it was a deliberate choice: https://github.com/apache/beam/blob/b7c2975c16c2a4d3051b0ea21ef5f515a0b8d50b/sdks/python/apache_beam/io/filesystem.py#L554. We used fnmatch.fnmatch() instead of glob.glob() to support both Unix and Windows style paths. To fully add support for paths like `foo/*/file.txt` (and `d:\foo\*\file.txt`): - Convert `*` in patterns to `[!/\]` (see fnmatch docs, I belive this should support both Unix and Windows paths) - Convert `**` in patterns to `*` (optional, this would be a new feature) - Document these changes [ Full content available at: https://github.com/apache/beam/pull/6423 ] This message was relayed via gitbox.apache.org for [email protected]
