[jira] [Commented] (BEAM-3223) PTransform spec should not reuse FunctionSpec

2017-11-20 Thread Kenneth Knowles (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259408#comment-16259408
 ] 

Kenneth Knowles commented on BEAM-3223:
---

Yea SGTM. I think that will make generated code more readable, less 
error-prone, more typing/analysis/testing friendly.

> PTransform spec should not reuse FunctionSpec
> -
>
> Key: BEAM-3223
> URL: https://issues.apache.org/jira/browse/BEAM-3223
> Project: Beam
>  Issue Type: Sub-task
>  Components: beam-model
>Reporter: Henning Rohde
>Assignee: Henning Rohde
>  Labels: portability
>
> We should add a new type instead, TransformSpec, say, or just inline a URN 
> and payload. It's confusing otherwise.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-3223) PTransform spec should not reuse FunctionSpec

2017-11-17 Thread Luke Cwik (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257763#comment-16257763
 ] 

Luke Cwik commented on BEAM-3223:
-

If we are going to inline all of these, we should drop Udf and define/inline 
ViewFn/WindowFn/... as well.

> PTransform spec should not reuse FunctionSpec
> -
>
> Key: BEAM-3223
> URL: https://issues.apache.org/jira/browse/BEAM-3223
> Project: Beam
>  Issue Type: Sub-task
>  Components: beam-model
>Reporter: Henning Rohde
>Assignee: Henning Rohde
>  Labels: portability
>
> We should add a new type instead, TransformSpec, say, or just inline a URN 
> and payload. It's confusing otherwise.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-3223) PTransform spec should not reuse FunctionSpec

2017-11-17 Thread Kenneth Knowles (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257746#comment-16257746
 ] 

Kenneth Knowles commented on BEAM-3223:
---

Suppose we have {{PTransformSpec}} and {{UdfSpec}} and {{CoderSpec}} that are 
all a pair of URN and payload. This is much more type-friendly and is a 
standard way of avoiding stupid mistakes.

Now they each occur in one place - {{PTransform}}, {{SdkFunctionSpec}}, and 
{{Coder}}, respectively. So we could just inline the fields. That seems much 
better to me.

{code}
message Udf {
  string urn;
  bytes payload;
  string environment_id;
}

messsage Coder {
  string id;
  string urn;
  bytes payload;
  repeated string component_coder_ids;
}

message PTransform {
  .. all the existing stuff ...
  string urn;
  bytes payload;
}
{code}

This makes the differences between the three very obvious.

I am only hesitant because the current protos represent a compromise that made 
all parties satisfied, but none happy. So bikeshedding is a danger.

> PTransform spec should not reuse FunctionSpec
> -
>
> Key: BEAM-3223
> URL: https://issues.apache.org/jira/browse/BEAM-3223
> Project: Beam
>  Issue Type: Sub-task
>  Components: beam-model
>Reporter: Henning Rohde
>Assignee: Henning Rohde
>  Labels: portability
>
> We should add a new type instead, TransformSpec, say, or just inline a URN 
> and payload. It's confusing otherwise.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)