[ https://issues.apache.org/jira/browse/BEAM-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Cwik resolved BEAM-4711. ----------------------------- Resolution: Fixed Fix Version/s: 2.8.0 > LocalFileSystem.delete doesn't support globbing > ----------------------------------------------- > > Key: BEAM-4711 > URL: https://issues.apache.org/jira/browse/BEAM-4711 > Project: Beam > Issue Type: Task > Components: sdk-py-core > Affects Versions: 2.5.0 > Reporter: Ryan Williams > Assignee: Ryan Williams > Priority: Minor > Fix For: 2.8.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > I attempted to run {{wordcount_it_test:WordCountIT.test_wordcount_it}} > locally with {{DirectRunner}}: > {code} > python setup.py nosetests \ > --tests > apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it \ > --test-pipeline-options="--output=foo" > {code} > It failed in [the {{delete_files}} cleanup > command|https://github.com/apache/beam/blob/a58f1ffaafb0e2ebcc73a1c5abfb05a15ec6a84b/sdks/python/apache_beam/examples/wordcount_it_test.py#L64]: > {code} > root: WARNING: Retry with exponential backoff: waiting for 11.1454450937 > seconds before retrying delete_files because we caught exception: > BeamIOError: Delete operation failed with exceptions > {'foo/1530557644/results*': IOError(OSError(2, 'No such file or directory'),)} > Traceback for above exception (most recent call last): > File "/Users/ryan/c/beam/sdks/python/apache_beam/utils/retry.py", line 184, > in wrapper > return fun(*args, **kwargs) > File "/Users/ryan/c/beam/sdks/python/apache_beam/testing/test_utils.py", > line 136, in delete_files > FileSystems.delete(file_paths) > File "/Users/ryan/c/beam/sdks/python/apache_beam/io/filesystems.py", line > 282, in delete > return filesystem.delete(paths) > File "/Users/ryan/c/beam/sdks/python/apache_beam/io/localfilesystem.py", > line 304, in delete > raise BeamIOError("Delete operation failed", exceptions) > {code} > The line: > {code} > self.addCleanup(delete_files, [output + '*']) > {code} > works as expected in GCS, and deletes a test's output-directory, but it fails > in on the local-filesystem, which doesn't expand globs before attempting to > delete paths. > It would be good to make these consistent, presumably by adding glob-support > to {{LocalFileSystem}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)