[
https://issues.apache.org/jira/browse/BEAM-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balázs Németh updated BEAM-6954:
--------------------------------
Component/s: runner-dataflow
> @Default not called if the options json has null value for a property
> ---------------------------------------------------------------------
>
> Key: BEAM-6954
> URL: https://issues.apache.org/jira/browse/BEAM-6954
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow, sdk-java-core
> Affects Versions: 2.11.0
> Reporter: Balázs Németh
> Priority: Major
>
> When a pipeline options get deserialized from a json with
> [https://github.com/apache/beam/blob/a85ea07b719385ec185e4fc5e4cdcc67b3598599/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L738-L760]
> it creates a map, where properties present in the json - even if with a null
> value - will be added to the map.
> So we can have String->NullNode mappings.
> When you create a ProxyInvocationHandler with this Map (
> [https://github.com/apache/beam/blob/a85ea07b719385ec185e4fc5e4cdcc67b3598599/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L117-L125]
> ) this map will be the backing jsonOptions map.
> Later on when a getter is called on the options it will reach this code:
> [https://github.com/apache/beam/blob/a85ea07b719385ec185e4fc5e4cdcc67b3598599/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L156-L158]
>
> Then the containsKey will return true, even for NullNodes. So we won't
> execute the getDefault() method hence not calculating the default value.
>
> I'm not sure about the expected behaviour, but either:
> - the containsKey check should be expanded with an !isNull check
> OR
> - the dataflow runner have to be modified so it doesn't persist null values
> at options here:
> https://github.com/apache/beam/blob/a85ea07b719385ec185e4fc5e4cdcc67b3598599/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java#L216-L222
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
