HI all
i was wondering how is it possible to force Beam to run 4 separate
processes for this pipeline
Currently i have this setup
with beam.Pipeline(options=pipeline_options) as p:
source = (p | 'Startup' >> beam.Create([1,2,3,4])
)
lines = run_my_pipeline(source)
Now, as per setup above, run_my_pipeline(source) will run only one process
which wil perform, sequentially, the run_my_pipeline processing [1,2,3,4]
what i want in stead would be to kick off run_my_pipeline(1),
run_my_pipeline(2), run_my_pipeline(3), run_my_pipeline(4),
Is that achievable>? Which tweaks do i need to do to accomplish this?
kind regards
Marco
