[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=102636=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-102636 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 16/May/18 20:17 Start Date: 16/May/18 20:17 Worklog Time Spent: 10m Work Description: lukecwik closed pull request #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/sdks/python/apache_beam/pvalue.py b/sdks/python/apache_beam/pvalue.py index 00597525763..3dedf0abe36 100644 --- a/sdks/python/apache_beam/pvalue.py +++ b/sdks/python/apache_beam/pvalue.py @@ -307,8 +307,7 @@ def _side_input_data(self): return SideInputData( common_urns.side_inputs.ITERABLE.urn, self._window_mapping_fn, -lambda iterable: from_runtime_iterable(iterable, view_options), -self._input_element_coder()) +lambda iterable: from_runtime_iterable(iterable, view_options)) def _input_element_coder(self): return coders.WindowedValueCoder( @@ -346,11 +345,10 @@ def _side_input_data(self): class SideInputData(object): """All of the data about a side input except for the bound PCollection.""" - def __init__(self, access_pattern, window_mapping_fn, view_fn, coder): + def __init__(self, access_pattern, window_mapping_fn, view_fn): self.access_pattern = access_pattern self.window_mapping_fn = window_mapping_fn self.view_fn = view_fn -self.coder = coder def to_runner_api(self, context): return beam_runner_api_pb2.SideInput( @@ -360,7 +358,7 @@ def to_runner_api(self, context): environment_id=context.default_environment_id(), spec=beam_runner_api_pb2.FunctionSpec( urn=python_urns.PICKLED_VIEWFN, -payload=pickler.dumps((self.view_fn, self.coder, +payload=pickler.dumps(self.view_fn))), window_mapping_fn=beam_runner_api_pb2.SdkFunctionSpec( environment_id=context.default_environment_id(), spec=beam_runner_api_pb2.FunctionSpec( @@ -375,7 +373,7 @@ def from_runner_api(proto, unused_context): return SideInputData( proto.access_pattern.urn, pickler.loads(proto.window_mapping_fn.spec.payload), -*pickler.loads(proto.view_fn.spec.payload)) +pickler.loads(proto.view_fn.spec.payload)) class AsSingleton(AsSideInput): @@ -451,8 +449,7 @@ def _side_input_data(self): return SideInputData( common_urns.side_inputs.ITERABLE.urn, self._window_mapping_fn, -lambda iterable: iterable, -self._input_element_coder()) +lambda iterable: iterable) @property def element_type(self): @@ -482,8 +479,7 @@ def _side_input_data(self): return SideInputData( common_urns.side_inputs.ITERABLE.urn, self._window_mapping_fn, -list, -self._input_element_coder()) +list) class AsDict(AsSideInput): @@ -510,8 +506,7 @@ def _side_input_data(self): return SideInputData( common_urns.side_inputs.ITERABLE.urn, self._window_mapping_fn, -dict, -self._input_element_coder()) +dict) class AsMultiMap(AsSideInput): @@ -537,8 +532,7 @@ def _side_input_data(self): return SideInputData( common_urns.side_inputs.MULTIMAP.urn, self._window_mapping_fn, -lambda x: x, -self._input_element_coder()) +lambda x: x) class EmptySideInput(object): diff --git a/sdks/python/apache_beam/runners/common.pxd b/sdks/python/apache_beam/runners/common.pxd index 9b0871fd539..58422b5d201 100644 --- a/sdks/python/apache_beam/runners/common.pxd +++ b/sdks/python/apache_beam/runners/common.pxd @@ -72,6 +72,7 @@ cdef class PerWindowInvoker(DoFnInvoker): cdef dict kwargs_for_process cdef list placeholders cdef bint has_windowed_inputs + cdef bint cache_globally_windowed_args cdef object process_method diff --git a/sdks/python/apache_beam/runners/common.py b/sdks/python/apache_beam/runners/common.py index 1e879a844a7..45e0ed55ba2 100644 --- a/sdks/python/apache_beam/runners/common.py +++ b/sdks/python/apache_beam/runners/common.py @@ -371,16 +371,13 @@ def __init__(self, output_processor, signature, context, # without any additional work. in the process function. # Also cache all the placeholders needed in the process function. -# Fill in sideInputs if they are globally windowed -global_window = GlobalWindow() +# Flag
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[
https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=102609=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-102609
]
ASF GitHub Bot logged work on BEAM-4236:
Author: ASF GitHub Bot
Created on: 16/May/18 18:50
Start Date: 16/May/18 18:50
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5302:
[BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known
coders and also work with Dataflow
URL: https://github.com/apache/beam/pull/5302#discussion_r188734467
##
File path: sdks/python/apache_beam/runners/portability/fn_api_runner.py
##
@@ -308,6 +308,73 @@ def deduplicate_read(self):
safe_coders = {}
+def fix_pcoll_coder(pcoll, pipeline_components):
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 102609)
Time Spent: 7.5h (was: 7h 20m)
> Side inputs do not work on Dataflow
> ---
>
> Key: BEAM-4236
> URL: https://issues.apache.org/jira/browse/BEAM-4236
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-harness
>Affects Versions: Not applicable
>Reporter: Charles Chen
>Assignee: Luke Cwik
>Priority: Blocker
> Fix For: 2.5.0
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Dataflow side inputs do not work in streaming / batch FnApi mode at head.
> The pipeline stalls indefinitely.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[
https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=102606=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-102606
]
ASF GitHub Bot logged work on BEAM-4236:
Author: ASF GitHub Bot
Created on: 16/May/18 18:46
Start Date: 16/May/18 18:46
Worklog Time Spent: 10m
Work Description: robertwb commented on a change in pull request #5302:
[BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known
coders and also work with Dataflow
URL: https://github.com/apache/beam/pull/5302#discussion_r188732416
##
File path: sdks/python/apache_beam/runners/portability/fn_api_runner.py
##
@@ -308,6 +308,73 @@ def deduplicate_read(self):
safe_coders = {}
+def fix_pcoll_coder(pcoll, pipeline_components):
Review comment:
Maybe call this length_prefix_unknown_coders?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 102606)
Time Spent: 7h 20m (was: 7h 10m)
> Side inputs do not work on Dataflow
> ---
>
> Key: BEAM-4236
> URL: https://issues.apache.org/jira/browse/BEAM-4236
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-harness
>Affects Versions: Not applicable
>Reporter: Charles Chen
>Assignee: Luke Cwik
>Priority: Blocker
> Fix For: 2.5.0
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Dataflow side inputs do not work in streaming / batch FnApi mode at head.
> The pipeline stalls indefinitely.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=102569=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-102569 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 16/May/18 17:51 Start Date: 16/May/18 17:51 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-389200149 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 102569) Time Spent: 7h 10m (was: 7h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 7h 10m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=102555=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-102555 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 16/May/18 17:23 Start Date: 16/May/18 17:23 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-389599277 run python precommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 102555) Time Spent: 7h (was: 6h 50m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=102554=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-102554 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 16/May/18 17:22 Start Date: 16/May/18 17:22 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-389599239 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 102554) Time Spent: 6h 50m (was: 6h 40m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 6h 50m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=102149=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-102149 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 15/May/18 15:03 Start Date: 15/May/18 15:03 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388937792 run java precommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 102149) Time Spent: 6h 40m (was: 6.5h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 6h 40m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=102148=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-102148 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 15/May/18 15:03 Start Date: 15/May/18 15:03 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388937819 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 102148) Time Spent: 6.5h (was: 6h 20m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 6.5h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=102147=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-102147 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 15/May/18 15:03 Start Date: 15/May/18 15:03 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-389200149 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 102147) Time Spent: 6h 20m (was: 6h 10m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 6h 20m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=101864=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-101864 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 14/May/18 19:46 Start Date: 14/May/18 19:46 Worklog Time Spent: 10m Work Description: robertwb commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388939376 The refcounts should only be needed for the old DirectRunner. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 101864) Time Spent: 6h 10m (was: 6h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 6h 10m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=101855=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-101855 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 14/May/18 19:40 Start Date: 14/May/18 19:40 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388937792 run java precommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 101855) Time Spent: 5h 20m (was: 5h 10m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 5h 20m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=101856=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-101856 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 14/May/18 19:40 Start Date: 14/May/18 19:40 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388937819 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 101856) Time Spent: 5.5h (was: 5h 20m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=101858=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-101858 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 14/May/18 19:40 Start Date: 14/May/18 19:40 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388134438 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 101858) Time Spent: 5h 50m (was: 5h 40m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 5h 50m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=101859=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-101859 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 14/May/18 19:40 Start Date: 14/May/18 19:40 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388076213 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 101859) Time Spent: 6h (was: 5h 50m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 6h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=101857=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-101857 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 14/May/18 19:40 Start Date: 14/May/18 19:40 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388080435 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 101857) Time Spent: 5h 40m (was: 5.5h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 5h 40m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100921 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 11/May/18 00:17 Start Date: 11/May/18 00:17 Worklog Time Spent: 10m Work Description: charlesccychen commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388223157 @aaltay Yes. @robertwb: I noticed that you very recently touched the ref counting logic in https://github.com/apache/beam/commit/00f3e22fccb. Can you chime in on whether we still need refcounts in PValueCache? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100921) Time Spent: 5h (was: 4h 50m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 5h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100922=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100922 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 11/May/18 00:17 Start Date: 11/May/18 00:17 Worklog Time Spent: 10m Work Description: charlesccychen commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388223157 @aaltay: Yes. @robertwb: I noticed that you very recently touched the ref counting logic in https://github.com/apache/beam/commit/00f3e22fccb. Can you chime in on whether we still need refcounts in PValueCache? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100922) Time Spent: 5h 10m (was: 5h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 5h 10m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100764=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100764 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 18:44 Start Date: 10/May/18 18:44 Worklog Time Spent: 10m Work Description: aaltay commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388147504 @charlesccychen could you run the equivalent of python post commit in your own environment. Let's not wait until we get Jenkin's fixes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100764) Time Spent: 4h 50m (was: 4h 40m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100749=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100749 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 18:33 Start Date: 10/May/18 18:33 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388143881 An issue with virtualenv exists preventing the Dataflow Python postcommit from running, more details on BEAM-4249 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100749) Time Spent: 4h 40m (was: 4.5h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100726=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100726 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 18:01 Start Date: 10/May/18 18:01 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388134438 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100726) Time Spent: 4.5h (was: 4h 20m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 4.5h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100633=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100633 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 15:01 Start Date: 10/May/18 15:01 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388080435 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100633) Time Spent: 4h 20m (was: 4h 10m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100627=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100627 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 14:48 Start Date: 10/May/18 14:48 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387884398 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100627) Time Spent: 4h 10m (was: 4h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100626=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100626 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 14:48 Start Date: 10/May/18 14:48 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387886652 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100626) Time Spent: 4h (was: 3h 50m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 4h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100625=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100625 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 14:48 Start Date: 10/May/18 14:48 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387896265 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100625) Time Spent: 3h 50m (was: 3h 40m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100624=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100624 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 14:47 Start Date: 10/May/18 14:47 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387912415 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100624) Time Spent: 3h 40m (was: 3.5h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100623=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100623 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 14:47 Start Date: 10/May/18 14:47 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-388076213 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100623) Time Spent: 3.5h (was: 3h 20m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100353=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100353 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 00:47 Start Date: 10/May/18 00:47 Worklog Time Spent: 10m Work Description: aaltay commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387917150 I don't know why PValueCache has a refcount. I have the same feeling as you about why it has this feature. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100353) Time Spent: 3h 20m (was: 3h 10m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100345=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100345 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 00:13 Start Date: 10/May/18 00:13 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387912415 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100345) Time Spent: 3h (was: 2h 50m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 3h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100346=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100346 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 00:13 Start Date: 10/May/18 00:13 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387912270 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100346) Time Spent: 3h 10m (was: 3h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100343=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100343 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 10/May/18 00:12 Start Date: 10/May/18 00:12 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387912270 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100343) Time Spent: 2h 50m (was: 2h 40m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100336=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100336 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 23:59 Start Date: 09/May/18 23:59 Worklog Time Spent: 10m Work Description: charlesccychen commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387910144 The issue with the KeyError above happens because of incorrect refcount numbers in PValueCache. The entry is garbage-collected because its refcount reaches zero before it can be used in the side input. Does anyone know why we have refcounts in PValueCache at all? My hunch is that this is not necessary, and is an artifact of when PValueCache was used to hold actual result elements. I am inclined to remove this feature of PValueCache. Does anyone have an opinion here? (CC: @robertwb @aaltay) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100336) Time Spent: 2h 40m (was: 2.5h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100322=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100322 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 23:09 Start Date: 09/May/18 23:09 Worklog Time Spent: 10m Work Description: charlesccychen commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387901868 For the record, the sample in https://github.com/apache/beam/pull/5302#issuecomment-387540559 still fails at this PR for an unrelated reason, likely having to do with PTransform replacement logic. ``` Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/local/google/home/ccy/git/beam/sdks/python/apache_beam/examples/streaming_wordcount.py", line 72, in run() File "/usr/local/google/home/ccy/git/beam/sdks/python/apache_beam/examples/streaming_wordcount.py", line 66, in run result = p.run() File "apache_beam/pipeline.py", line 389, in run self.to_runner_api(), self.runner, self._options).run(False) File "apache_beam/pipeline.py", line 402, in run return self.runner.run_pipeline(self) File "apache_beam/runners/dataflow/dataflow_runner.py", line 347, in run_pipeline super(DataflowRunner, self).run_pipeline(pipeline) File "apache_beam/runners/runner.py", line 170, in run_pipeline pipeline.visit(RunVisitor(self)) File "apache_beam/pipeline.py", line 430, in visit self._root_transform().visit(visitor, self, visited) File "apache_beam/pipeline.py", line 785, in visit part.visit(visitor, pipeline, visited) File "apache_beam/pipeline.py", line 788, in visit visitor.visit_transform(self) File "apache_beam/runners/runner.py", line 165, in visit_transform self.runner.run_transform(transform_node) File "apache_beam/runners/runner.py", line 208, in run_transform return m(transform_node) File "apache_beam/runners/dataflow/dataflow_runner.py", line 581, in run_ParDo input_step = self._cache.get_pvalue(transform_node.inputs[0]) File "apache_beam/runners/runner.py", line 286, in get_pvalue value_with_refcount = self._cache[self.key(pvalue)] KeyError: (u'SingletonSide/Decode Values', None) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100322) Time Spent: 2.5h (was: 2h 20m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100306=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100306 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 22:38 Start Date: 09/May/18 22:38 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387896265 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100306) Time Spent: 2h 20m (was: 2h 10m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100280=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100280 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 21:51 Start Date: 09/May/18 21:51 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387886652 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100280) Time Spent: 2h 10m (was: 2h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100279=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100279 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 21:49 Start Date: 09/May/18 21:49 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387886139 The hardest part was understanding the changes that I could make that would be compatible with Dataflow. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100279) Time Spent: 2h (was: 1h 50m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100268=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100268 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 21:42 Start Date: 09/May/18 21:42 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387818543 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100268) Time Spent: 1h 50m (was: 1h 40m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100266=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100266 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 21:42 Start Date: 09/May/18 21:42 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387884398 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100266) Time Spent: 1.5h (was: 1h 20m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100267=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100267 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 21:42 Start Date: 09/May/18 21:42 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387818859 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100267) Time Spent: 1h 40m (was: 1.5h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100241=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100241 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 20:21 Start Date: 09/May/18 20:21 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387863373 Reverted the removal of tags_and_types and restored the ability for the legacy batch Python Dataflow code paths. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100241) Time Spent: 1h 20m (was: 1h 10m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100205=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100205 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 18:45 Start Date: 09/May/18 18:45 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387834763 @pabloem Your right, this breaks the way that the Dataflow SDK does legacy worker batch side inputs. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100205) Time Spent: 1h 10m (was: 1h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100196=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100196 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 18:36 Start Date: 09/May/18 18:36 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387834763 @pabloem Your right, this breaks the way that the Dataflow SDK does batch side inputs. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100196) Time Spent: 1h (was: 50m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100159=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100159 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 17:51 Start Date: 09/May/18 17:51 Worklog Time Spent: 10m Work Description: pabloem commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387821112 Question: Is this going to change how batch side inputs work in Python Dataflow? Where can I find the c9ode that will fetch the side input from now on? Will we not be using the existing code path in [runners/worker/sideinputs.py](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/worker/sideinputs.py#L53) ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100159) Time Spent: 50m (was: 40m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100157=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100157 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 17:50 Start Date: 09/May/18 17:50 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387820765 CC: @pabloem This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100157) Time Spent: 40m (was: 0.5h) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100155=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100155 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 17:43 Start Date: 09/May/18 17:43 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387818859 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100155) Time Spent: 0.5h (was: 20m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100153=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100153 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 17:42 Start Date: 09/May/18 17:42 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387818543 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100153) Time Spent: 20m (was: 10m) > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4236) Side inputs do not work on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-4236?focusedWorklogId=100149=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-100149 ] ASF GitHub Bot logged work on BEAM-4236: Author: ASF GitHub Bot Created on: 09/May/18 17:39 Start Date: 09/May/18 17:39 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5302: [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow URL: https://github.com/apache/beam/pull/5302#issuecomment-387817586 R: @charlesccychen This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 100149) Time Spent: 10m Remaining Estimate: 0h > Side inputs do not work on Dataflow > --- > > Key: BEAM-4236 > URL: https://issues.apache.org/jira/browse/BEAM-4236 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-harness >Affects Versions: Not applicable >Reporter: Charles Chen >Assignee: Luke Cwik >Priority: Blocker > Fix For: 2.5.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Dataflow side inputs do not work in streaming / batch FnApi mode at head. > The pipeline stalls indefinitely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
