The shared libraries need to understand all the primitives given to them as they are required to satisfy the contracts that each primitive has. The code within QueryablePipeline/ExecutableStage/GreedyPipelineFuser/... make assumptions around what neighboring primitives exist and how they are compatible with each other.
The pipeline representation can use pardo/read/... primitives which the runner intercepts during translation *OR* the runner can choose to intercept the pipeline before it gives it to the shared libraries to do fusion/... and update the representation to be compatible with those libraries *OR* those libraries need to expose hooks for runner known primitives to describe how they should treat these primitives (e.g. should they break fusion, or be fusable with the SDK transforms). [ Full content available at: https://github.com/apache/beam/pull/6328 ] This message was relayed via gitbox.apache.org for [email protected]
