[
https://issues.apache.org/jira/browse/BEAM-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aviem Zur updated BEAM-1629:
----------------------------
Description:
Today aggregators and metrics accumulators are instantiated after pipeline is
traversed, and so, if code that runs on driver accesses these singletons but is
not part of a closure a {{NullPointerException}} is thrown.
This is fine for streaming branches of a pipeline as you probably did something
wrong, but for batch branches this should be allowed (In completely batch
pipelines accumulators are instantiated before traversal and everything works
fine, the issue occurs in streaming+batch combined pipelines)
was:
Today aggregators and metrics accumulators are instantiated after pipeline is
traversed, and so, if code that runs on driver accesses these singletons but is
not part of a closure a {{NullPointerException}} is thrown.
This is fine for streaming branches of a pipeline as you probably did something
wrong, but for batch branches this should be allowed (In completely batch
pipelines accumulators are instantiated before traversal and everything works
fine, the issue occurs in streaming/batch pipelines)
> Init metrics/aggregators accumulators before traversing pipeline
> ----------------------------------------------------------------
>
> Key: BEAM-1629
> URL: https://issues.apache.org/jira/browse/BEAM-1629
> Project: Beam
> Issue Type: Bug
> Components: runner-spark
> Reporter: Aviem Zur
> Assignee: Aviem Zur
>
> Today aggregators and metrics accumulators are instantiated after pipeline is
> traversed, and so, if code that runs on driver accesses these singletons but
> is not part of a closure a {{NullPointerException}} is thrown.
> This is fine for streaming branches of a pipeline as you probably did
> something wrong, but for batch branches this should be allowed (In completely
> batch pipelines accumulators are instantiated before traversal and everything
> works fine, the issue occurs in streaming+batch combined pipelines)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
