[
https://issues.apache.org/jira/browse/BEAM-11188?focusedWorklogId=511681&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-511681
]
ASF GitHub Bot logged work on BEAM-11188:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Nov/20 06:18
Start Date: 14/Nov/20 06:18
Worklog Time Spent: 10m
Work Description: youngoli commented on a change in pull request #13325:
URL: https://github.com/apache/beam/pull/13325#discussion_r523383346
##########
File path: sdks/go/pkg/beam/xlang.go
##########
@@ -87,61 +84,16 @@ func TryCrossLanguage(s Scope, ext
*graph.ExternalTransform, ins []*graph.Inboun
// unique namespace can be requested.
ext.Namespace = graph.NewNamespace()
- // Build the ExpansionRequest
-
- // Obtaining the components and transform proto representing this
transform
- // TODO(BEAM-11188): Move proto handling code into xlangx or graphx
package.
- p, err := graphx.Marshal([]*graph.MultiEdge{edge}, &graphx.Options{})
+ expanded, err := xlangx.Expand(edge, ext)
if err != nil {
- return nil, errors.Wrapf(err, "unable to generate proto
representation of %v", ext)
- }
-
- transforms := p.GetComponents().GetTransforms()
-
- // Transforms consist of only External transform and composites.
Composites
- // should be removed from proto before submitting expansion request.
- extTransformID := p.GetRootTransformIds()[0]
- extTransform := transforms[extTransformID]
- for extTransform.UniqueName != "External" {
- delete(transforms, extTransformID)
- p, err := pipelinex.Normalize(p)
- if err != nil {
- return nil, err
- }
- extTransformID = p.GetRootTransformIds()[0]
- extTransform = transforms[extTransformID]
- }
-
- // Scoping the ExternalTransform with respect to it's unique namespace,
thus
- // avoiding future collisions
- xlangx.AddNamespace(extTransform, p.GetComponents(), ext.Namespace)
-
- xlangx.AddFakeImpulses(p) // Inputs need to have sources
- delete(transforms, extTransformID)
-
- // Querying the expansion service
- res, err := xlangx.Expand(context.Background(), p.GetComponents(),
extTransform, ext.Namespace, ext.ExpansionAddr)
- if err != nil {
- return nil, err
- }
-
- // Handling ExpansionResponse
-
- // Previously added fake impulses need to be removed to avoid having
- // multiple sources to the same pcollection in the graph
- xlangx.RemoveFakeImpulses(res.GetComponents(), res.GetTransform())
-
- exp := &graph.ExpandedTransform{
- Components: res.GetComponents(),
- Transform: res.GetTransform(),
- Requirements: res.GetRequirements(),
+ return nil, errors.WithContext(err, "expanding external
transform")
}
- ext.Expanded = exp
+ ext.Expanded = expanded
Review comment:
Sounds fine with me. My instinct is usually to prefer returning values
over modifying parameters, but Expand has such specific usage that I don't
think it matters much here. Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 511681)
Time Spent: 3h 10m (was: 3h)
> Go Cross-Language UX polish and refactoring
> -------------------------------------------
>
> Key: BEAM-11188
> URL: https://issues.apache.org/jira/browse/BEAM-11188
> Project: Beam
> Issue Type: Improvement
> Components: cross-language, sdk-go
> Reporter: Daniel Oliveira
> Assignee: Daniel Oliveira
> Priority: P2
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> This is a bug for progress on various small usability and UX improvements to
> the Go SDK implementation of Cross-Language. I don't feel each one
> individually is important enough for a Jira, but together it's worth
> recording progress.
> Tasks included:
> * Adjust user-facing XLang functions so that Sink and Source versions are
> actually sinks and sources (no outputs and no inputs respectively).
> * Rename SourceInputTag and SinkOutputTag since they are no longer used with
> source/sink versions of the methods.
> * Adjust beam/xlang.go so that it doesn't need to import job_management
> protos. Move the proto creation down into the method the proto is passed to
> (which is xlangx.Expand).
> * Refactor the functions in xlangx/translate.go and how they are used, since
> right now the functions just get called one after another in sequence.
> * Move as many xlang calls out of universal.go as possible. They should be
> handled as part of the normal sequence of the SDK, such as in proto
> marshalling and unmarshalling.
> * Add wrappers around xlang calls in existing examples, to both give a
> cleaner interface and provide an example of how xlang transforms should be
> implemented.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
