Balázs Németh created BEAM-6954:
-----------------------------------
Summary: @Default not called if the options json has null value
for a property
Key: BEAM-6954
URL: https://issues.apache.org/jira/browse/BEAM-6954
Project: Beam
Issue Type: Bug
Components: sdk-java-core
Affects Versions: 2.11.0
Reporter: Balázs Németh
When a pipeline options get deserialized from a json with
[https://github.com/apache/beam/blob/a85ea07b719385ec185e4fc5e4cdcc67b3598599/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L738-L760]
it creates a map, where properties present in the json - even if with a null
value - will be added to the map.
So we can have String->NullNode mappings.
When you create a ProxyInvocationHandler with this Map (
[https://github.com/apache/beam/blob/a85ea07b719385ec185e4fc5e4cdcc67b3598599/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L117-L125]
) this map will be the backing jsonOptions map.
Later on when a getter is called on the options it will reach this code:
[https://github.com/apache/beam/blob/a85ea07b719385ec185e4fc5e4cdcc67b3598599/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L156-L158]
Then the containsKey will return true, even for NullNodes. So we won't execute
the getDefault() method hence not calculating the default value.
I'm not sure about the expected behaviour, but either:
- the containsKey check should be expanded with an !isNull check
OR
- the dataflow runner have to be modified so it doesn't persist null values at
options here:
https://github.com/apache/beam/blob/a85ea07b719385ec185e4fc5e4cdcc67b3598599/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java#L216-L222
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
