Hi Lukasz, For both Gearpump DAG and Storm Topology, they are built and serialized at client side, and sent to server. For things (e.g. Reader) that depends on PipelineOptions to create at server side, they require PipelineOptions to be serialized at client side. (example code <https://github.com/apache/incubator-beam/blob/gearpump-runner/runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/io/GearpumpSource.java#L50> )
Is this what you mean, Taylor ? I'm not sure core Java SDK can do this part for runners since not all runners adopt the serde way for application submit. On Thu, Aug 4, 2016 at 4:45 AM Lukasz Cwik <[email protected]> wrote: > The core Java SDK already contains the code to serialize/deserialize > PipelineOptions using Jackson's ObjectMapper since PipelineOptions has > Jackson databind annotations @JsonSerialize and @JsonDeserialize. > > Is there something more specific that you were looking for? > > On Wed, Aug 3, 2016 at 12:54 PM, P. Taylor Goetz <[email protected]> > wrote: > > > Looking at the runner implementations for Spark, Flink, and Gearpump, > they > > all have the same requirement of being able to serialize/deserialize > > PipelineOptions, and they each seem to use slightly different approaches. > > In working on a Beam runner for Storm I have the same requirement, and > > imagine other runner implementations might as well. > > > > Would it make sense to add this functionality in the core SDK? > > > > -Taylor > > >
