Hi all, I would like to construct a Beam pipeline with Python transforms, where different transforms use different container images, is this possible? Browsing through the design docs for cross-language transforms, it seems like it should in theory be possible, as each step can spawn it's own container if I understand correctly.
Unfortunately, I'm not having any luck piecing together the concrete steps necessary to construct such a pipeline. I was only able to find the `--environment_config` flag, but this applies a single image to the whole pipeline. Ideallly, this would work on the Flink runner, but I'm open to setting up alternative runners if required for this. The motivation behind this request is that we'd like to build a pipeline that chains together various bits of software for processing protein structures. These packages are either poorly packaged or have many dependencies (or both), making it difficult to construct a single environment that contains all of the transitive dependencies without conflicts. It would be much nicer to be able to construct a single image per step, and then chain these together into a unified pipeline with Beam. Many thanks, Kai
