[
https://issues.apache.org/jira/browse/BEAM-5298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746566#comment-16746566
]
Robert Burke commented on BEAM-5298:
------------------------------------
I'm finally in a place where it's worth putting thought into this, but just a
comment based on recent discussion from the dev list:
With the discussion of Schemas and User Coders, it's come to my attention that
the *typed generic* PCollections that Java has are becoming a bit of an
inconvenience. With typed PCollections, essentially *everything* must be a
PCollection<Schema>, or it would fail at compile time. However, a
PCollection<Schema> has the opportunity in Go to be treated as any compatible
concrete type by the user code, mostly because we already do Construction Time
type checking of a pipeline graph. In short, if Beam moves towards broad
inter-language schema support, then even with Generics in Go, the PCollections
will remain untyped, and we'd retain the construction time type checking as we
do now.
> Determine what painpoints exist for the Go SDK WRT the Go Generics Draft
> Design
> -------------------------------------------------------------------------------
>
> Key: BEAM-5298
> URL: https://issues.apache.org/jira/browse/BEAM-5298
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: Major
>
> Specifically, the outcome should be a doc outlining what would need to change
> in the SDK as a consequence of the draft design, with a focus on the user
> experience, and how much unnecessary user toil could be avoided.
> [https://go.googlesource.com/proposal/+/master/design/go2draft-generics-overview.md]
> Initial impressions:
> As the draft design indicates, the current SDK code would not be required to
> change, it would continue working as is, as the change is intended to be
> backwards compatible. Users should be able to use their own generic code with
> the v1 of the SDK.
>
> A v2 of the Go SDK could remove much of the code the SDK spends in doing it's
> own type checking of pipelines at construction time, in favour of the
> compiler.
> PCollections would be properly typed to facilitate this.
> Specific focus should be made to cover the user experience side, and sketch
> implementations of the generic types (like PCollection) and functions (like
> ParDo), as well as on the execution harness side. eg. is it possible to avoid
> reflection or type assertion on the processing path?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
