Hello Team,
i am using beam for experimenting for my project usecase
i have specific steps for a pipeline. for each step to process i am using do
function in pardo invocation in pipeline
in one of the steps i had implemented multiprocessing implemented
with out multiprocessing the step is working fine
when i introduce multiprocessing in pardo transform step is blocking execution
if use beam supported multiprocessing option vis direct runner number of
workers it is not recognizing my module function
if use multi threading it is again blocking infinitely
can you suggest me how to resolve issue with in the pardo do fn
or any other suggestion when multiprocessing
