[jira] [Commented] (BEAM-2963) Propagate pipeline protos through Dataflow API from Java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314845#comment-16314845
 ] 

ASF GitHub Bot commented on BEAM-2963:
--

kennknowles opened a new pull request #4356: [BEAM-2963] Pass portable 
transform IDs in Dataflow translation
URL: https://github.com/apache/beam/pull/4356
 
 
   This changes the Dataflow translation to reference the portable pipeline by 
id rather than putting serialized Java bits directly in the Dataflow job.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [x] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
- [x] Each commit in the pull request should have a meaningful subject line 
and body.
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
- [x] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
- [x] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
- [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Propagate pipeline protos through Dataflow API from Java
> 
>
> Key: BEAM-2963
> URL: https://issues.apache.org/jira/browse/BEAM-2963
> Project: Beam
>  Issue Type: Sub-task
>  Components: runner-dataflow
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>  Labels: portability
> Fix For: 2.3.0
>
>
> The Java-specific blobs transmitted to Dataflow need more context, in the 
> form of portability framework protos.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2963) Propagate pipeline protos through Dataflow API from Java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246662#comment-16246662
 ] 

ASF GitHub Bot commented on BEAM-2963:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/4089


> Propagate pipeline protos through Dataflow API from Java
> 
>
> Key: BEAM-2963
> URL: https://issues.apache.org/jira/browse/BEAM-2963
> Project: Beam
>  Issue Type: Sub-task
>  Components: runner-dataflow
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>  Labels: portability
> Fix For: 2.3.0
>
>
> The Java-specific blobs transmitted to Dataflow need more context, in the 
> form of portability framework protos.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2963) Propagate pipeline protos through Dataflow API from Java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209941#comment-16209941
 ] 

ASF GitHub Bot commented on BEAM-2963:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3977


> Propagate pipeline protos through Dataflow API from Java
> 
>
> Key: BEAM-2963
> URL: https://issues.apache.org/jira/browse/BEAM-2963
> Project: Beam
>  Issue Type: Sub-task
>  Components: runner-dataflow
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>  Labels: portability
>
> The Java-specific blobs transmitted to Dataflow need more context, in the 
> form of portability framework protos.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2963) Propagate pipeline protos through Dataflow API from Java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199742#comment-16199742
 ] 

ASF GitHub Bot commented on BEAM-2963:
--

GitHub user kennknowles opened a pull request:

https://github.com/apache/beam/pull/3977

[BEAM-2963] Stage pipeline in DataflowRunner

Follow this checklist to help us incorporate your contribution quickly and 
easily:

 - [x] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
 - [x] Each commit in the pull request should have a meaningful subject 
line and body.
 - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
 - [x] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
 - [x] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
 - [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---

I've confirmed that the pipeline is uploaded and the metadata set to the 
actual path, which has the hash appended to avoid collision.

R: @herohde 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/beam stage-pipeline

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3977.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3977


commit 66a2dc0fddb614d41774dec390cdd1e674e4657c
Author: Kenneth Knowles 
Date:   2017-10-11T03:11:36Z

Make PackageUtil public and @Internal

commit c72e6bbfe58d9f333430e81cae74a98a7d3f54fe
Author: Kenneth Knowles 
Date:   2017-10-11T03:11:47Z

Stage the portable pipeline in Dataflow




> Propagate pipeline protos through Dataflow API from Java
> 
>
> Key: BEAM-2963
> URL: https://issues.apache.org/jira/browse/BEAM-2963
> Project: Beam
>  Issue Type: Sub-task
>  Components: runner-dataflow
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>
> The Java-specific blobs transmitted to Dataflow need more context, in the 
> form of portability framework protos.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2963) Propagate pipeline protos through Dataflow API from Java

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193993#comment-16193993
 ] 

ASF GitHub Bot commented on BEAM-2963:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3920


> Propagate pipeline protos through Dataflow API from Java
> 
>
> Key: BEAM-2963
> URL: https://issues.apache.org/jira/browse/BEAM-2963
> Project: Beam
>  Issue Type: Sub-task
>  Components: runner-dataflow
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>
> The Java-specific blobs transmitted to Dataflow need more context, in the 
> form of portability framework protos.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)