The shared libraries need to understand all the primitives given to them as they are required to satisfy the contracts that each primitive has. The code within QueryablePipeline/ExecutableStage/GreedyPipelineFuser/... make assumptions around what neighboring primitives exist and how they are compatible with each other.
The runner can choose to intercept the pipeline before it gives it to the shared libraries to do fusion/... and update the representation to be compatible with those libraries or those libraries need to expose hooks for runner known primitives to describe how they should treat these primitives (e.g. should they break fusion, or be fusable with the SDK transforms). [ Full content available at: https://github.com/apache/beam/pull/6328 ] This message was relayed via gitbox.apache.org for [email protected]
