[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037238#comment-16037238
 ] 

ASF GitHub Bot commented on BEAM-1938:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3272


> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032153#comment-16032153
 ] 

ASF GitHub Bot commented on BEAM-1938:
--

GitHub user tgroh opened a pull request:

https://github.com/apache/beam/pull/3272

[BEAM-1938] Remove the requirement to visit PCollectionViews in Dataflow

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/beam reduce_pvalues_in_dataflow

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3272.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3272


commit f106c405daf90544ef1a9b54dbeac5a7142f0700
Author: Thomas Groh 
Date:   2017-05-31T22:42:03Z

Remove the requirement to visit PCollectionViews in Dataflow




> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031574#comment-16031574
 ] 

ASF GitHub Bot commented on BEAM-1938:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3269


> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031479#comment-16031479
 ] 

ASF GitHub Bot commented on BEAM-1938:
--

GitHub user tgroh opened a pull request:

https://github.com/apache/beam/pull/3269

[BEAM-1938] Reduce Prevalence of PValue in the DirectRunner

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---
Use PCollection or PCollectionView explicitly.

Retrieve views from the WriteView transform rather than visiting the
view as an output PValue.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/beam remove_pvalue_from_direct_runner

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3269.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3269


commit d4d6122cc54347036cfcce726142ed8910deea91
Author: Thomas Groh 
Date:   2017-05-31T16:38:02Z

Reduce Prevalence of PValue in the DirectRunner

Use PCollection or PCollectionView explicitly.

Retrieve views from the WriteView transform rather than visiting the
view as an output PValue.




> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020511#comment-16020511
 ] 

ASF GitHub Bot commented on BEAM-1938:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3188


> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017950#comment-16017950
 ] 

ASF GitHub Bot commented on BEAM-1938:
--

GitHub user tgroh opened a pull request:

https://github.com/apache/beam/pull/3188

[BEAM-1938] Include Additional PTransform inputs in Transform Nodes

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---
Add the value of PTransform.getAdditionalInputs in the inputs of a
TransformHierarchy node.

Fork the Node constructor to reduce nullability

This slightly simplifies the constructor implementation(s).

Update the DirectRunner to track main inputs instead of all inputs.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/beam 
additional_inputs_in_transform_nodes

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3188.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3188


commit cd7cd87bd79291278e3c940e3e0013323b1821b9
Author: Thomas Groh 
Date:   2017-04-05T00:43:48Z

Include Additional PTransform inputs in Transform Nodes

Add the value of PTransform.getAdditionalInputs in the inputs of a
TransformHierarchy node.

Fork the Node constructor to reduce nullability

This slightly simplifies the constructor implementation(s).

Update the DirectRunner to track main inputs instead of all inputs.




> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[Thomas Groh (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017749#comment-16017749
 ] 

Thomas Groh commented on BEAM-1938:
---

No. I don't think there's a huge amount of work remaining, but there are a 
couple of changes that need to happen among some assorted Pipeline Visitors to 
handle the fact that there are more inputs than prior.

> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[Daniel Halperin (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016343#comment-16016343
 ] 

Daniel Halperin commented on BEAM-1938:
---

[~tgroh] can this issue be closed?

> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969691#comment-15969691
 ] 

ASF GitHub Bot commented on BEAM-1938:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/2501


> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1938) Side Inputs should be part of the expanded inputs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965072#comment-15965072
 ] 

ASF GitHub Bot commented on BEAM-1938:
--

GitHub user tgroh opened a pull request:

https://github.com/apache/beam/pull/2501

[BEAM-1938] Update Signature of PTransformOverrideFactory

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---
This enables replacements to be reobtained with the entire transform
that is being replaced.

This is required when Side Inputs are part of the input of the
PTransform Application, as PTransforms are not applied to their side
inputs.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/beam override_factory_entire_context

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2501.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2501


commit caf32a9742fabf8e95631e018afa6f981d6f5249
Author: Thomas Groh 
Date:   2017-03-29T22:23:21Z

Update Signature of PTransformOverrideFactory

This enables replacements to be reobtained with the entire transform
that is being replaced.

This is required when Side Inputs are part of the input of the
PTransform Application, as PTransforms are not applied to their side
inputs.




> Side Inputs should be part of the expanded inputs
> -
>
> Key: BEAM-1938
> URL: https://issues.apache.org/jira/browse/BEAM-1938
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> Required for the Java SDK to construct the runner API graphs without 
> inspecting arbitrary transforms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)