raams95 commented on PR #40549:
URL: https://github.com/apache/beam/pull/40549#issuecomment-6153691119
Thanks for taking a look. Built-in file sources don't hit this anymore
(empty files are skipped and their bundles have no weights), but reading any
`ConcatSource` whose bundles have weight 0 still fails on master, e.g.:
```python
source = ConcatSource([
iobase.SourceBundle(0, RangeSource(0, 4), None, None),
iobase.SourceBundle(0, RangeSource(4, 8), None, None),
])
with TestPipeline() as p:
assert_that(p | beam.io.Read(source), equal_to(list(range(8))))
```
On master this raises `ZeroDivisionError: float division by zero [while
running
'Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction']`.
That's the case from the issue, where weights come from sizes rounded down to
whole MB.
I replaced the unit test with this pipeline as the regression test. It fails
without the change and passes with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
