[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=411282=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-411282
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 27/Mar/20 18:55
Start Date: 27/Mar/20 18:55
Worklog Time Spent: 10m 
  Work Description: robertwb commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 411282)
Time Spent: 9h 10m  (was: 9h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 9h 10m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410796=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410796
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 27/Mar/20 02:54
Start Date: 27/Mar/20 02:54
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604788204
 
 
   Run PythonDocker PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410796)
Time Spent: 9h  (was: 8h 50m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 9h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410726=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410726
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 23:53
Start Date: 26/Mar/20 23:53
Worklog Time Spent: 10m 
  Work Description: pabloem commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604744648
 
 
   may need to rebase to get passing Docker PreCommit?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410726)
Time Spent: 8h 50m  (was: 8h 40m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 8h 50m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410693=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410693
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 23:07
Start Date: 26/Mar/20 23:07
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604732125
 
 
   Run PythonDocker PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410693)
Time Spent: 8h 40m  (was: 8.5h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 8h 40m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410549=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410549
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 19:45
Start Date: 26/Mar/20 19:45
Worklog Time Spent: 10m 
  Work Description: robertwb commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604646548
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410549)
Time Spent: 8.5h  (was: 8h 20m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 8.5h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410548=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410548
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 19:45
Start Date: 26/Mar/20 19:45
Worklog Time Spent: 10m 
  Work Description: robertwb commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604646485
 
 
   Task 'pythonDockerBuildPreCommit' not found in root project 'beam'.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410548)
Time Spent: 8h 20m  (was: 8h 10m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 8h 20m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410518=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410518
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 18:59
Start Date: 26/Mar/20 18:59
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604623844
 
 
   Yeah, I was just surprised about the choice, nothing really wrong with being 
PCollection vs PTransform centric.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410518)
Time Spent: 8h 10m  (was: 8h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 8h 10m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410510=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410510
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 18:52
Start Date: 26/Mar/20 18:52
Worklog Time Spent: 10m 
  Work Description: robertwb commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604619996
 
 
   As forusing the input/output PCollection names, rather than the GRPC ids, 
that's a good question. Things are structured this way is mostly to follow the 
existing code, where things are indexed by these PCollections. In particular,
   
   ... -> pcIn -> GBK -> pcOut -> ...
   
   gets replaced by
   
  ... -> pcIn -> GrpcOut
  GrpcIn -> pcOut -> ...
   
   but the original GBK transform itself is kept around (with its references to 
pcIn and pcOut--it's the object that "connects" the two) to construct the 
appropriate grouping buffer. This could be refactored, and may be if GBK 
becomes a full-fledged operator to make it more streamy, but that seemed out of 
scope for this change. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410510)
Time Spent: 8h  (was: 7h 50m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 8h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410484=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410484
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 18:43
Start Date: 26/Mar/20 18:43
Worklog Time Spent: 10m 
  Work Description: robertwb commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#discussion_r398808389
 
 

 ##
 File path: 
sdks/python/apache_beam/runners/portability/fn_api_runner/translations.py
 ##
 @@ -347,6 +348,22 @@ def add_or_get_coder_id(self,
 self.components.coders[new_coder_id].CopyFrom(coder_proto)
 return new_coder_id
 
+  def add_data_channel_coder(self, pcoll_id):
+pcoll = self.components.pcollections[pcoll_id]
+proto = beam_runner_api_pb2.Coder(
+spec=beam_runner_api_pb2.FunctionSpec(
+urn=common_urns.coders.WINDOWED_VALUE.urn),
+component_coder_ids=[
+pcoll.coder_id,
+self.components.windowing_strategies[
+pcoll.windowing_strategy_id].window_coder_id
+])
+channel_coder = self.add_or_get_coder_id(
+proto, pcoll.coder_id + '_windowed')
+if pcoll.coder_id in self.safe_coders:
+  channel_coder = self.length_prefixed_coder(channel_coder)
 
 Review comment:
   Yes, but here `pcoll.coder_id != channel_coder`. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410484)
Time Spent: 7h 50m  (was: 7h 40m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 7h 50m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410479=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410479
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 18:40
Start Date: 26/Mar/20 18:40
Worklog Time Spent: 10m 
  Work Description: robertwb commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604608176
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410479)
Time Spent: 7.5h  (was: 7h 20m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 7.5h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410480=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410480
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 18:40
Start Date: 26/Mar/20 18:40
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#discussion_r398806850
 
 

 ##
 File path: 
sdks/python/apache_beam/runners/portability/fn_api_runner/translations.py
 ##
 @@ -347,6 +348,22 @@ def add_or_get_coder_id(self,
 self.components.coders[new_coder_id].CopyFrom(coder_proto)
 return new_coder_id
 
+  def add_data_channel_coder(self, pcoll_id):
+pcoll = self.components.pcollections[pcoll_id]
+proto = beam_runner_api_pb2.Coder(
+spec=beam_runner_api_pb2.FunctionSpec(
+urn=common_urns.coders.WINDOWED_VALUE.urn),
+component_coder_ids=[
+pcoll.coder_id,
+self.components.windowing_strategies[
+pcoll.windowing_strategy_id].window_coder_id
+])
+channel_coder = self.add_or_get_coder_id(
+proto, pcoll.coder_id + '_windowed')
+if pcoll.coder_id in self.safe_coders:
+  channel_coder = self.length_prefixed_coder(channel_coder)
 
 Review comment:
   I now understand the nuance of your statement.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410480)
Time Spent: 7h 40m  (was: 7.5h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 7h 40m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410468=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410468
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 18:20
Start Date: 26/Mar/20 18:20
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#discussion_r398793631
 
 

 ##
 File path: 
sdks/python/apache_beam/runners/portability/fn_api_runner/translations.py
 ##
 @@ -347,6 +348,22 @@ def add_or_get_coder_id(self,
 self.components.coders[new_coder_id].CopyFrom(coder_proto)
 return new_coder_id
 
+  def add_data_channel_coder(self, pcoll_id):
+pcoll = self.components.pcollections[pcoll_id]
+proto = beam_runner_api_pb2.Coder(
+spec=beam_runner_api_pb2.FunctionSpec(
+urn=common_urns.coders.WINDOWED_VALUE.urn),
+component_coder_ids=[
+pcoll.coder_id,
+self.components.windowing_strategies[
+pcoll.windowing_strategy_id].window_coder_id
+])
+channel_coder = self.add_or_get_coder_id(
+proto, pcoll.coder_id + '_windowed')
+if pcoll.coder_id in self.safe_coders:
+  channel_coder = self.length_prefixed_coder(channel_coder)
 
 Review comment:
   lines 398-399 already do this check as part of the `length_prefixed_coder` 
method.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410468)
Time Spent: 7h 20m  (was: 7h 10m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 7h 20m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410465=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410465
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 18:16
Start Date: 26/Mar/20 18:16
Worklog Time Spent: 10m 
  Work Description: robertwb commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#discussion_r398790807
 
 

 ##
 File path: 
sdks/python/apache_beam/runners/portability/fn_api_runner/translations.py
 ##
 @@ -347,6 +348,22 @@ def add_or_get_coder_id(self,
 self.components.coders[new_coder_id].CopyFrom(coder_proto)
 return new_coder_id
 
+  def add_data_channel_coder(self, pcoll_id):
+pcoll = self.components.pcollections[pcoll_id]
+proto = beam_runner_api_pb2.Coder(
+spec=beam_runner_api_pb2.FunctionSpec(
+urn=common_urns.coders.WINDOWED_VALUE.urn),
+component_coder_ids=[
+pcoll.coder_id,
+self.components.windowing_strategies[
+pcoll.windowing_strategy_id].window_coder_id
+])
+channel_coder = self.add_or_get_coder_id(
+proto, pcoll.coder_id + '_windowed')
+if pcoll.coder_id in self.safe_coders:
+  channel_coder = self.length_prefixed_coder(channel_coder)
 
 Review comment:
   Here I want to populate the safe_coders mapping for channel_coder iff the 
value coder has such a mapping. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410465)
Time Spent: 7h 10m  (was: 7h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410462=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410462
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 18:14
Start Date: 26/Mar/20 18:14
Worklog Time Spent: 10m 
  Work Description: robertwb commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#discussion_r398789812
 
 

 ##
 File path: 
sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py
 ##
 @@ -426,7 +423,8 @@ def _collect_written_timers_and_add_to_deferred_inputs(
   pipeline_components,  # type: beam_runner_api_pb2.Components
   stage,  # type: translations.Stage
   bundle_context_manager,  # type: execution.BundleContextManager
-  deferred_inputs  # type: MutableMapping[str, PartitionableBuffer]
+  deferred_inputs,  # type: MutableMapping[str, PartitionableBuffer]
+  data_channel_coders,
 
 Review comment:
   Thanks for calling me out on this. The types themselves aren't very 
informative as to their meaning, but I added it here and elsewhere. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410462)
Time Spent: 7h  (was: 6h 50m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 7h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410449=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410449
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 17:56
Start Date: 26/Mar/20 17:56
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604583109
 
 
   > fwiw the precommit breakages are unrelated, and already fixed in master.
   
   Thats what I assumed when I looked at the failure causes so I kicked them 
off again.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410449)
Time Spent: 6h 50m  (was: 6h 40m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410448=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410448
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 17:55
Start Date: 26/Mar/20 17:55
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604582850
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410448)
Time Spent: 6h 40m  (was: 6.5h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 6h 40m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410445=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410445
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 17:51
Start Date: 26/Mar/20 17:51
Worklog Time Spent: 10m 
  Work Description: pabloem commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604580447
 
 
   fwiw the precommit breakages are unrelated, and already fixed in master.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410445)
Time Spent: 6.5h  (was: 6h 20m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410366=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410366
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 16:01
Start Date: 26/Mar/20 16:01
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#discussion_r398677414
 
 

 ##
 File path: 
sdks/python/apache_beam/runners/portability/fn_api_runner/translations.py
 ##
 @@ -347,6 +348,22 @@ def add_or_get_coder_id(self,
 self.components.coders[new_coder_id].CopyFrom(coder_proto)
 return new_coder_id
 
+  def add_data_channel_coder(self, pcoll_id):
+pcoll = self.components.pcollections[pcoll_id]
+proto = beam_runner_api_pb2.Coder(
+spec=beam_runner_api_pb2.FunctionSpec(
+urn=common_urns.coders.WINDOWED_VALUE.urn),
+component_coder_ids=[
+pcoll.coder_id,
+self.components.windowing_strategies[
+pcoll.windowing_strategy_id].window_coder_id
+])
+channel_coder = self.add_or_get_coder_id(
+proto, pcoll.coder_id + '_windowed')
+if pcoll.coder_id in self.safe_coders:
+  channel_coder = self.length_prefixed_coder(channel_coder)
 
 Review comment:
   Note this check is already done within `length_prefix_coder` where it will 
return the original coder id if its a safe coder so you can always use it.
   
   ```suggestion
   channel_coder = self.length_prefixed_coder(channel_coder)
   ```
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410366)
Time Spent: 6h 20m  (was: 6h 10m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 6h 20m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410360=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410360
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 15:48
Start Date: 26/Mar/20 15:48
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604507872
 
 
   Run PythonLint PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410360)
Time Spent: 6h 10m  (was: 6h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410356=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410356
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 15:47
Start Date: 26/Mar/20 15:47
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604507442
 
 
   Run Portable_Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410356)
Time Spent: 5h 40m  (was: 5.5h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410359=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410359
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 15:47
Start Date: 26/Mar/20 15:47
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604507596
 
 
   Run Python2_PVR_Flink PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410359)
Time Spent: 6h  (was: 5h 50m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410357=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410357
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 15:47
Start Date: 26/Mar/20 15:47
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #11222: [BEAM-4150] Don't 
window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#issuecomment-604507514
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410357)
Time Spent: 5h 50m  (was: 5h 40m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410351=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410351
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 15:46
Start Date: 26/Mar/20 15:46
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#discussion_r398678099
 
 

 ##
 File path: 
sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py
 ##
 @@ -426,7 +423,8 @@ def _collect_written_timers_and_add_to_deferred_inputs(
   pipeline_components,  # type: beam_runner_api_pb2.Components
   stage,  # type: translations.Stage
   bundle_context_manager,  # type: execution.BundleContextManager
-  deferred_inputs  # type: MutableMapping[str, PartitionableBuffer]
+  deferred_inputs,  # type: MutableMapping[str, PartitionableBuffer]
+  data_channel_coders,
 
 Review comment:
   If possible try and keep the typing information on methods.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410351)
Time Spent: 5.5h  (was: 5h 20m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=410352=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410352
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 26/Mar/20 15:46
Start Date: 26/Mar/20 15:46
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#discussion_r398677414
 
 

 ##
 File path: 
sdks/python/apache_beam/runners/portability/fn_api_runner/translations.py
 ##
 @@ -347,6 +348,22 @@ def add_or_get_coder_id(self,
 self.components.coders[new_coder_id].CopyFrom(coder_proto)
 return new_coder_id
 
+  def add_data_channel_coder(self, pcoll_id):
+pcoll = self.components.pcollections[pcoll_id]
+proto = beam_runner_api_pb2.Coder(
+spec=beam_runner_api_pb2.FunctionSpec(
+urn=common_urns.coders.WINDOWED_VALUE.urn),
+component_coder_ids=[
+pcoll.coder_id,
+self.components.windowing_strategies[
+pcoll.windowing_strategy_id].window_coder_id
+])
+channel_coder = self.add_or_get_coder_id(
+proto, pcoll.coder_id + '_windowed')
+if pcoll.coder_id in self.safe_coders:
+  channel_coder = self.length_prefixed_coder(channel_coder)
 
 Review comment:
   Note this check is already done within `length_prefix_coder` where it will 
return the original coder id if its a safe coder so you 
   
   ```suggestion
   channel_coder = self.length_prefixed_coder(channel_coder)
   ```
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 410352)
Time Spent: 5.5h  (was: 5h 20m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=409950=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409950
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 25/Mar/20 22:59
Start Date: 25/Mar/20 22:59
Worklog Time Spent: 10m 
  Work Description: robertwb commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222#discussion_r398221421
 
 

 ##
 File path: 
sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py
 ##
 @@ -882,66 +885,13 @@ def _add_residuals_and_channel_splits_to_deferred_inputs(
 prev_stops[
 channel_split.transform_id] = channel_split.last_primary_element
 
-  @staticmethod
 
 Review comment:
   This was an extra copy of the method that wasn't used anywhere.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 409950)
Time Spent: 5h 20m  (was: 5h 10m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=409667=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409667
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 25/Mar/20 17:51
Start Date: 25/Mar/20 17:51
Worklog Time Spent: 10m 
  Work Description: robertwb commented on pull request #11222: [BEAM-4150] 
Don't window PCollection coders.
URL: https://github.com/apache/beam/pull/11222
 
 
   Now that no SDKs require it.
   
   
   
   Thank you for your contribution! Follow this checklist to help us 
incorporate your contribution quickly and easily:
   
- [ ] [**Choose 
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and 
mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more 
tips on [how to make review process 
smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[![Build
 

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381903=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381903
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 22:08
Start Date: 04/Feb/20 22:08
Worklog Time Spent: 10m 
  Work Description: lostluck commented on issue #10755: [BEAM-4150] Remove 
fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#issuecomment-582139562
 
 
   Whoops. Dead comment I thought I deleted after answering my own question
   after expanding how much code was revealed.
   
   On Tue, Feb 4, 2020, 1:20 PM Lukasz Cwik  wrote:
   
   > Tthey always have one.
   >
   > On Tue, Feb 4, 2020 at 11:04 AM Robert Burke 
   > wrote:
   >
   > > *@lostluck* commented on this pull request.
   > > --
   > >
   > > In sdks/go/pkg/beam/core/runtime/exec/translate.go
   > > :
   > >
   > > > @@ -64,6 +64,13 @@ func UnmarshalPlan(desc
   > *fnpb.ProcessBundleDescriptor) (*Plan, error) {
   > > }
   > >
   > > u := {UID: b.idgen.New()}
   > > + u.Coder, err = b.coders.Coder(cid) // Expected to be windowed coder
   > > + if err != nil {
   > > + return nil, err
   > > + }
   > > + if !coder.IsW(u.Coder) {
   > > + return nil, errors.Errorf("unwindowed coder %v on DataSource %v: %v",
   > cid, id, u.Coder)
   > > + }
   > >
   > > for key, pid := range transform.GetOutputs() {
   > >
   > > Do GRPC datasources always only have a single output?
   > >
   > > I note that we aren't using the pid, in the new code.
   > >
   > > —
   > > You are receiving this because you authored the thread.
   > > Reply to this email directly, view it on GitHub
   > > <
   > 
https://github.com/apache/beam/pull/10755?email_source=notifications_token=ACM4V3D4ENJGCCU7DCMAVXLRBG333A5CNFSM4KPKV7XKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCUG4ZCI#pullrequestreview-353225865
   > >,
   > > or unsubscribe
   > > <
   > 
https://github.com/notifications/unsubscribe-auth/ACM4V3AXN6VNEWJK3UURZZ3RBG333ANCNFSM4KPKV7XA
   > >
   > > .
   > >
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or unsubscribe
   > 

   > .
   >
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381903)
Time Spent: 5h  (was: 4h 50m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381873=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381873
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 21:20
Start Date: 04/Feb/20 21:20
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #10755: [BEAM-4150] Remove 
fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#issuecomment-582120561
 
 
   Tthey always have one.
   
   On Tue, Feb 4, 2020 at 11:04 AM Robert Burke 
   wrote:
   
   > *@lostluck* commented on this pull request.
   > --
   >
   > In sdks/go/pkg/beam/core/runtime/exec/translate.go
   > :
   >
   > > @@ -64,6 +64,13 @@ func UnmarshalPlan(desc 
*fnpb.ProcessBundleDescriptor) (*Plan, error) {
   >}
   >
   >u := {UID: b.idgen.New()}
   > +  u.Coder, err = b.coders.Coder(cid) // Expected to be windowed 
coder
   > +  if err != nil {
   > +  return nil, err
   > +  }
   > +  if !coder.IsW(u.Coder) {
   > +  return nil, errors.Errorf("unwindowed coder %v on 
DataSource %v: %v", cid, id, u.Coder)
   > +  }
   >
   >for key, pid := range transform.GetOutputs() {
   >
   > Do GRPC datasources always only have a single output?
   >
   > I note that we aren't using the pid, in the new code.
   >
   > —
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or unsubscribe
   > 

   > .
   >
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381873)
Time Spent: 4h 50m  (was: 4h 40m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Assignee: Luke Cwik
>Priority: Major
> Fix For: 2.20.0
>
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381832=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381832
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 19:40
Start Date: 04/Feb/20 19:40
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on pull request #10755: [BEAM-4150] 
Remove fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381832)
Time Spent: 4h 40m  (was: 4.5h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381823=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381823
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 19:04
Start Date: 04/Feb/20 19:04
Worklog Time Spent: 10m 
  Work Description: lostluck commented on pull request #10755: [BEAM-4150] 
Remove fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#discussion_r374863985
 
 

 ##
 File path: sdks/go/pkg/beam/core/runtime/exec/translate.go
 ##
 @@ -64,6 +64,13 @@ func UnmarshalPlan(desc *fnpb.ProcessBundleDescriptor) 
(*Plan, error) {
}
 
u := {UID: b.idgen.New()}
+   u.Coder, err = b.coders.Coder(cid) // Expected to be windowed 
coder
+   if err != nil {
+   return nil, err
+   }
+   if !coder.IsW(u.Coder) {
+   return nil, errors.Errorf("unwindowed coder %v on 
DataSource %v: %v", cid, id, u.Coder)
+   }
 
for key, pid := range transform.GetOutputs() {
 
 Review comment:
   Do GRPC datasources always only have a single output?
   
   I note that we aren't using the pid, in the new code.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381823)
Time Spent: 4h 20m  (was: 4h 10m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381824=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381824
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 19:04
Start Date: 04/Feb/20 19:04
Worklog Time Spent: 10m 
  Work Description: lostluck commented on pull request #10755: [BEAM-4150] 
Remove fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#discussion_r374863985
 
 

 ##
 File path: sdks/go/pkg/beam/core/runtime/exec/translate.go
 ##
 @@ -64,6 +64,13 @@ func UnmarshalPlan(desc *fnpb.ProcessBundleDescriptor) 
(*Plan, error) {
}
 
u := {UID: b.idgen.New()}
+   u.Coder, err = b.coders.Coder(cid) // Expected to be windowed 
coder
+   if err != nil {
+   return nil, err
+   }
+   if !coder.IsW(u.Coder) {
+   return nil, errors.Errorf("unwindowed coder %v on 
DataSource %v: %v", cid, id, u.Coder)
+   }
 
for key, pid := range transform.GetOutputs() {
 
 Review comment:
   Do GRPC datasources always only have a single output?
   
   I note that we aren't using the pid, in the new code.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381824)
Time Spent: 4.5h  (was: 4h 20m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381762=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381762
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 17:49
Start Date: 04/Feb/20 17:49
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #10755: [BEAM-4150] Remove 
fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#issuecomment-582032543
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381762)
Time Spent: 4h 10m  (was: 4h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381761=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381761
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 17:49
Start Date: 04/Feb/20 17:49
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #10755: [BEAM-4150] Remove 
fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#issuecomment-582032495
 
 
   Run Python2_PVR_Flink PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381761)
Time Spent: 4h  (was: 3h 50m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381303=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381303
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 01:58
Start Date: 04/Feb/20 01:58
Worklog Time Spent: 10m 
  Work Description: lostluck commented on issue #10755: [BEAM-4150] Remove 
fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#issuecomment-581705839
 
 
   Run Go Postcommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381303)
Time Spent: 3h 40m  (was: 3.5h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381305=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381305
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 01:58
Start Date: 04/Feb/20 01:58
Worklog Time Spent: 10m 
  Work Description: lostluck commented on pull request #10755: [BEAM-4150] 
Remove fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#discussion_r374436704
 
 

 ##
 File path: sdks/go/pkg/beam/core/runtime/exec/translate.go
 ##
 @@ -64,6 +64,13 @@ func UnmarshalPlan(desc *fnpb.ProcessBundleDescriptor) 
(*Plan, error) {
}
 
u := {UID: b.idgen.New()}
+u.Coder, err = b.coders.Coder(cid) // Expected to be windowed coder
 
 Review comment:
   Please run gofmt on your code. Go uses Tabs for indentation, not spaces.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381305)
Time Spent: 3h 50m  (was: 3h 40m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381286=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381286
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 00:32
Start Date: 04/Feb/20 00:32
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #10755: [BEAM-4150] Remove 
fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#issuecomment-581686417
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381286)
Time Spent: 3.5h  (was: 3h 20m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381271=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381271
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 00:20
Start Date: 04/Feb/20 00:20
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #10755: [BEAM-4150] Remove 
fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#issuecomment-581683444
 
 
   R: @lostluck @ibzib 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381271)
Time Spent: 3h 20m  (was: 3h 10m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381267=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381267
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 04/Feb/20 00:20
Start Date: 04/Feb/20 00:20
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on issue #10755: [BEAM-4150] Remove 
fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755#issuecomment-581683353
 
 
   Tested JRH change internally within Google as well.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 381267)
Time Spent: 3h 10m  (was: 3h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=381159=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381159
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 03/Feb/20 19:27
Start Date: 03/Feb/20 19:27
Worklog Time Spent: 10m 
  Work Description: lukecwik commented on pull request #10755: [BEAM-4150] 
Remove fallback case for coder not specified within RemoteGrpcPort.
URL: https://github.com/apache/beam/pull/10755
 
 
   All runners now specify the coder id.
   
   
   
   Thank you for your contribution! Follow this checklist to help us 
incorporate your contribution quickly and easily:
   
- [ ] [**Choose 
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and 
mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more 
tips on [how to make review process 
smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
 | --- | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)[![Build
 

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=240390=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-240390
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 10/May/19 20:14
Start Date: 10/May/19 20:14
Worklog Time Spent: 10m 
  Work Description: angoenka commented on pull request #8533: [BEAM-4150] 
Downgrade missing coder error logs to info logs.
URL: https://github.com/apache/beam/pull/8533
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 240390)
Time Spent: 2h 50m  (was: 2h 40m)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=24=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-24
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 10/May/19 01:39
Start Date: 10/May/19 01:39
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #8533: [BEAM-4150] Downgrade 
missing coder error logs to info logs.
URL: https://github.com/apache/beam/pull/8533#issuecomment-491123541
 
 
   R: @angoenka could you review this please?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 24)
Time Spent: 2h 40m  (was: 2.5h)

> Standardize use of PCollection coder proto attribute
> 
>
> Key: BEAM-4150
> URL: https://issues.apache.org/jira/browse/BEAM-4150
> Project: Beam
>  Issue Type: Task
>  Components: beam-model
>Reporter: Robert Bradshaw
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> In some places it's expected to be a WindowedCoder, in others the raw 
> ElementCoder. We should use the same convention (decided in discussion to be 
> the raw ElementCoder) everywhere. The WindowCoder can be pulled out of the 
> attached windowing strategy, and the input/output ports should specify the 
> encoding directly rather than read the adjacent PCollection coder fields. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=239483=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-239483
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 08/May/19 22:06
Start Date: 08/May/19 22:06
Worklog Time Spent: 10m 
  Work Description: aaltay commented on pull request #8533: [BEAM-4150] 
Downgrade missing coder error logs to info logs.
URL: https://github.com/apache/beam/pull/8533
 
 
   This log message shows up frequently in the error logs. Users cannot do much 
about it and they are not fatal.
   
   
   
   Thank you for your contribution! Follow this checklist to help us 
incorporate your contribution quickly and easily:
   
- [ ] [**Choose 
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and 
mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Python3_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python3_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)
 | --- | --- | ---
   
   Pre-Commit Tests Status (on master branch)
   

   
   --- |Java | Python | Go | Website
   --- | 

[jira] [Work logged] (BEAM-4150) Standardize use of PCollection coder proto attribute

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4150?focusedWorklogId=173418=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-173418
 ]

ASF GitHub Bot logged work on BEAM-4150:


Author: ASF GitHub Bot
Created on: 10/Dec/18 08:22
Start Date: 10/Dec/18 08:22
Worklog Time Spent: 10m 
  Work Description: robertwb closed pull request #7217: [BEAM-4150] Use 
unwindowed coder in FnApiRunner optimization phases.
URL: https://github.com/apache/beam/pull/7217
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/runners/portability/fn_api_runner.py 
b/sdks/python/apache_beam/runners/portability/fn_api_runner.py
index 76155492fe57..a7cf38e1804e 100644
--- a/sdks/python/apache_beam/runners/portability/fn_api_runner.py
+++ b/sdks/python/apache_beam/runners/portability/fn_api_runner.py
@@ -315,8 +315,11 @@ def __init__(self, name, transforms,
 
   def __repr__(self):
 must_follow = ', '.join(prev.name for prev in self.must_follow)
-downstream_side_inputs = ', '.join(
-str(si) for si in self.downstream_side_inputs)
+if self.downstream_side_inputs is None:
+  downstream_side_inputs = ''
+else:
+  downstream_side_inputs = ', '.join(
+  str(si) for si in self.downstream_side_inputs)
 return "%s\n  %s\n  must follow: %s\n  downstream_side_inputs: %s" % (
 self.name,
 '\n'.join(["%s:%s" % (transform.unique_name, transform.spec.urn)
@@ -467,10 +470,6 @@ def wrap_unknown_coders(coder_id, with_bytes):
 def impulse_to_input(stages):
   bytes_coder_id = add_or_get_coder_id(
   beam.coders.BytesCoder().to_runner_api(None))
-  global_window_coder_id = add_or_get_coder_id(
-  beam.coders.coders.GlobalWindowCoder().to_runner_api(None))
-  globally_windowed_bytes_coder_id = windowed_coder_id(
-  bytes_coder_id, global_window_coder_id)
 
   for stage in stages:
 # First map Reads, if any, to Impulse + triggered read op.
@@ -483,7 +482,7 @@ def impulse_to_input(stages):
 pipeline_components.pcollections[impulse_pc].CopyFrom(
 beam_runner_api_pb2.PCollection(
 unique_name=impulse_pc,
-coder_id=globally_windowed_bytes_coder_id,
+coder_id=bytes_coder_id,
 windowing_strategy_id=read_pc_proto.windowing_strategy_id,
 is_bounded=read_pc_proto.is_bounded))
 stage.transforms.remove(transform)
@@ -539,10 +538,7 @@ def lift_combiners(stages):
   output_pcoll = pipeline_components.pcollections[only_element(
   list(transform.outputs.values()))]
 
-  windowed_input_coder = pipeline_components.coders[
-  input_pcoll.coder_id]
-  element_coder_id, window_coder_id = (
-  windowed_input_coder.component_coder_ids)
+  element_coder_id = input_pcoll.coder_id
   element_coder = pipeline_components.coders[element_coder_id]
   key_coder_id, _ = element_coder.component_coder_ids
   accumulator_coder_id = combine_payload.accumulator_coder_id
@@ -575,8 +571,7 @@ def lift_combiners(stages):
   pipeline_components.pcollections[precombined_pcoll_id].CopyFrom(
   beam_runner_api_pb2.PCollection(
   unique_name=transform.unique_name + '/Precombine.out',
-  coder_id=windowed_coder_id(
-  key_accumulator_coder_id, window_coder_id),
+  coder_id=key_accumulator_coder_id,
   windowing_strategy_id=input_pcoll.windowing_strategy_id,
   is_bounded=input_pcoll.is_bounded))
 
@@ -585,8 +580,7 @@ def lift_combiners(stages):
   pipeline_components.pcollections[grouped_pcoll_id].CopyFrom(
   beam_runner_api_pb2.PCollection(
   unique_name=transform.unique_name + '/Group.out',
-  coder_id=windowed_coder_id(
-  key_accumulator_iter_coder_id, window_coder_id),
+  coder_id=key_accumulator_iter_coder_id,
   windowing_strategy_id=output_pcoll.windowing_strategy_id,
   is_bounded=output_pcoll.is_bounded))
 
@@ -595,8 +589,7 @@ def lift_combiners(stages):
   pipeline_components.pcollections[merged_pcoll_id].CopyFrom(
   beam_runner_api_pb2.PCollection(
   unique_name=transform.unique_name + '/Merge.out',
-  coder_id=windowed_coder_id(
-  key_accumulator_coder_id, window_coder_id),
+