I heard mention that there is a flatten unzipping optimization implemented
by some runners. I didn't see that in the python optimizations
in translations.py[1]. Just curious what this optimization is?

I think I get the general gist in that you dont necessarily need to combine
the input pcollections to a flatten and instead you can just apply
non-aggregating consuming transforms to all input pcollections, but when is
a good time to do that? Do runners that implement this optimization always
apply this to all flattens?

Cheers,
Joey

[1]
https://github.com/apache/beam/blob/72102b5985b3a13c4a4c3949bf23d129c3999827/sdks/python/apache_beam/runners/portability/fn_api_runner/translations.py

Reply via email to