[
https://issues.apache.org/jira/browse/BEAM-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Etienne Chauchot updated BEAM-6663:
-----------------------------------
Description: Many runners, store both PipelineOptions and
SerializablePipelineOptions in their context whereas
SerializablePipelineOptions stores both Options and json serialized Options.
Options can be obtain by SerializablePipelineOptions.get() but json form cannot
be obtained. Similarly, we cannot create a SerializablePipelineOptions from
json. It is needed in some cases (like spark structured streaming) where we
cannot inject anything else than Strings into the framework. (was: Many
runners, store both options and SerializableOptions in their context whereas
SerializableOptions stores both Options and json serialized Options. Options
can be obtain by SerializableOptions.get() and json cannot be obtained. I
propose to use only SerializableOptions inside the runners (as they all have
serialization issues) and simply override toString to get the json version.)
Summary: Add the ability to transform to/from json
SerializablePipelineOptions (was: SerializablePipelineOptions should override
toString())
> Add the ability to transform to/from json SerializablePipelineOptions
> ---------------------------------------------------------------------
>
> Key: BEAM-6663
> URL: https://issues.apache.org/jira/browse/BEAM-6663
> Project: Beam
> Issue Type: Improvement
> Components: runner-core
> Reporter: Etienne Chauchot
> Assignee: Etienne Chauchot
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Many runners, store both PipelineOptions and SerializablePipelineOptions in
> their context whereas SerializablePipelineOptions stores both Options and
> json serialized Options. Options can be obtain by
> SerializablePipelineOptions.get() but json form cannot be obtained.
> Similarly, we cannot create a SerializablePipelineOptions from json. It is
> needed in some cases (like spark structured streaming) where we cannot inject
> anything else than Strings into the framework.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
