[jira] [Commented] (BEAM-196) Pipeline options must be available Context in DoFn.startBundle

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320833#comment-15320833
 ] 

ASF GitHub Bot commented on BEAM-196:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/432


> Pipeline options must be available Context in DoFn.startBundle
> --
>
> Key: BEAM-196
> URL: https://issues.apache.org/jira/browse/BEAM-196
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Mark Shields
>Assignee: Maximilian Michels
> Fix For: 0.1.0-incubating
>
>
> Our (not yet merged) Java Pubsub implementation has code like this in a DoFn:
> @Override
> public void startBundle(Context c) throws Exception {
>   Preconditions.checkState(pubsubClient == null);
>   pubsubClient = PubsubClient.newClient(transportType,
>   timestampLabel, idLabel, 
> c.getPipelineOptions().as(PubsubOptions.class));
>   super.startBundle(c);
> }
> This fails with NPE since the pipeline options are not conveyed via the 
> context.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-196) Pipeline options must be available Context in DoFn.startBundle

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320368#comment-15320368
 ] 

ASF GitHub Bot commented on BEAM-196:
-

GitHub user mxm opened a pull request:

https://github.com/apache/incubator-beam/pull/432

[BEAM-196] Additional fix to ensure the PipelineOptions are available in 
DoFns.

Fixes a NullPointException if `PipelineOptions` are acceessed inside a DoFn 
and backs it up with a test.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mxm/incubator-beam BEAM-196

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/432.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #432


commit 05287c4ddcc38378e4e8cd921b6f6394e43eed75
Author: Maximilian Michels 
Date:   2016-06-07T11:57:33Z

[BEAM-196] fix NPE in ParDoWrapper

commit 595146d5759a6f631f057050cdbe71f9849035f9
Author: Maximilian Michels 
Date:   2016-06-08T10:40:50Z

[flink] improve lifecycle of ParDoBoundWrapper




> Pipeline options must be available Context in DoFn.startBundle
> --
>
> Key: BEAM-196
> URL: https://issues.apache.org/jira/browse/BEAM-196
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Mark Shields
>Assignee: Maximilian Michels
> Fix For: 0.1.0-incubating
>
>
> Our (not yet merged) Java Pubsub implementation has code like this in a DoFn:
> @Override
> public void startBundle(Context c) throws Exception {
>   Preconditions.checkState(pubsubClient == null);
>   pubsubClient = PubsubClient.newClient(transportType,
>   timestampLabel, idLabel, 
> c.getPipelineOptions().as(PubsubOptions.class));
>   super.startBundle(c);
> }
> This fails with NPE since the pipeline options are not conveyed via the 
> context.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-196) Pipeline options must be available Context in DoFn.startBundle

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247333#comment-15247333
 ] 

ASF GitHub Bot commented on BEAM-196:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/200


> Pipeline options must be available Context in DoFn.startBundle
> --
>
> Key: BEAM-196
> URL: https://issues.apache.org/jira/browse/BEAM-196
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Mark Shields
>Assignee: Maximilian Michels
>
> Our (not yet merged) Java Pubsub implementation has code like this in a DoFn:
> @Override
> public void startBundle(Context c) throws Exception {
>   Preconditions.checkState(pubsubClient == null);
>   pubsubClient = PubsubClient.newClient(transportType,
>   timestampLabel, idLabel, 
> c.getPipelineOptions().as(PubsubOptions.class));
>   super.startBundle(c);
> }
> This fails with NPE since the pipeline options are not conveyed via the 
> context.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-196) Pipeline options must be available Context in DoFn.startBundle

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245934#comment-15245934
 ] 

ASF GitHub Bot commented on BEAM-196:
-

GitHub user mxm opened a pull request:

https://github.com/apache/incubator-beam/pull/200

[BEAM-196] Pipeline options must be available Context in DoFn.startBundle

This gets rid of the custom Java serialization code by defaulting to 
serialization of the `PipelineOptions` to a byte array. So far, this has been 
proven the most hassle-free method for the Flink Runner. For code reuse and 
avoiding multiple deserialization of the byte array, the 
`SerializedPipelineOptions` class has been introduced.

The changes also make the options accessible in the context of the `DoFn` 
function.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mxm/incubator-beam BEAM-196

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/200.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #200


commit 81577b31c2642522f7dd4ba8eba794df48a0ca56
Author: Maximilian Michels 
Date:   2016-04-18T15:40:38Z

[BEAM-196] abstraction for PipelineOptions serialization

commit 43b5ec743718e63c2d9d9532e3ca55bc87370290
Author: Maximilian Michels 
Date:   2016-04-18T15:40:50Z

[BEAM-196] make use of SerializedPipelineOptions




> Pipeline options must be available Context in DoFn.startBundle
> --
>
> Key: BEAM-196
> URL: https://issues.apache.org/jira/browse/BEAM-196
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Mark Shields
>Assignee: Maximilian Michels
>
> Our (not yet merged) Java Pubsub implementation has code like this in a DoFn:
> @Override
> public void startBundle(Context c) throws Exception {
>   Preconditions.checkState(pubsubClient == null);
>   pubsubClient = PubsubClient.newClient(transportType,
>   timestampLabel, idLabel, 
> c.getPipelineOptions().as(PubsubOptions.class));
>   super.startBundle(c);
> }
> This fails with NPE since the pipeline options are not conveyed via the 
> context.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)