Hi Dev's Currently, the Pipeline class in Beam is not Serializable. This is not a problem for the current runners since the pipeline is translated and submitted through a centralized Driver like model. However, if the runner has a decentralized model similar to OpenMPI (MPI), which is also the case with Twister2, which I am developing a runner currently, it would have been better if the pipeline itself was Serializable.
Currently, I am trying to transform the Pipeline into a Twister2 graph and then send over to the workers, however since there are some functions such as "SystemReduceFn" that are not serializable this also is somewhat troublesome. Was the decision to make Pipelines not Serializable made due to some specific reason or because all the current use cases did not present any valid requirement to make them Serializable? Best Regards, Pulasthi -- Pulasthi S. Wickramasinghe PhD Candidate | Research Assistant School of Informatics and Computing | Digital Science Center Indiana University, Bloomington cell: 224-386-9035
