Thanks for the information. I will take a look.
Best Regards,
Pulasthi
On Fri, Nov 15, 2019 at 2:07 PM Luke Cwik wrote:
> They are serialized but not with Java serialization. There is a
> CloudObject serialization[1] layer that only Dataflow uses while all other
> runners who need to serialize
Serializable classes are not required to have default, no-arg constructors.
Reuven
On Fri, Nov 15, 2019 at 11:00 AM Pulasthi Supun Wickramasinghe <
pulasthi...@gmail.com> wrote:
> Hi Luke,
>
> Aren't the coders supposed to be serializable? The doc on the Coder
> interface has the following java
They are serialized but not with Java serialization. There is a CloudObject
serialization[1] layer that only Dataflow uses while all other runners who
need to serialize are using the Coder -> Proto serialization layer[2]. The
CloudObject representation is slated for deletion once we can migrate
Dat
Hi Luke,
Aren't the coders supposed to be serializable? The doc on the Coder
interface has the following java doc comment, which seems to mean that they
should be, and most of the basic coders seem to serializable.
" {@link Coder} instances are serialized during job creation and
deserialized befo
Hi Luke,
That is the approach i am taking currently to handle the functions. I Might
have to do the same for Coders as well since some coders have the same
issue of not having default constructors.
I also initially considered converting the pipeline into a JSON format and
sending that over to the
You should create placeholders inside of your Twister2/OpenMPI
implementation that represent these functions and then instantiate actual
instances of them on the workers if you want to write your own pipeline
representation and format for OpenMPI/Twister2.
Or consider converting the pipeline to it
Hi Dev's
Currently, the Pipeline class in Beam is not Serializable. This is not a
problem for the current runners since the pipeline is translated and
submitted through a centralized Driver like model. However, if the runner
has a decentralized model similar to OpenMPI (MPI), which is also the cas
