[
https://issues.apache.org/jira/browse/BEAM-13399?focusedWorklogId=718351&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-718351
]
ASF GitHub Bot logged work on BEAM-13399:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Feb/22 00:35
Start Date: 01/Feb/22 00:35
Worklog Time Spent: 10m
Work Description: lostluck commented on a change in pull request #16671:
URL: https://github.com/apache/beam/pull/16671#discussion_r796167396
##########
File path: sdks/go/pkg/beam/transforms/sql/sql.go
##########
@@ -79,6 +82,33 @@ func ExpansionAddr(addr string) Option {
}
}
+const serviceGradleTarget =
":sdks:java:extensions:sql:expansion-service:shadowJar"
+
+// StartAutomatedExpansionService pulls a JAR from Maven containing
+// a Beam expansion service that handles the SQL transform URN,
+// creates a runner type to manage the service, starts the service,
+// and arranges the endpoint with the appropriate namespace tag.
+//
+// Returns the StopService() function for the runner (should be
+// deferred so the service is stopped before the program exits,)
+// the tagged expansion address, and an error.
+func StartAutomatedExpansionService() (func() error, string, error) {
Review comment:
I think my question is why is this all done at the wrapper package
level, rather than hidden away deeper? As implemented, every Jar based Xlang
wrapper has to duplicate all this code and the code at the transform call
itself, which is not a great experience, especially since there's nothing
specific about it.
Ideally, wrapper authors should only be writing
defaultExpansion := xlangx.AutoNamespace + xlangx.Separator + <gradle target>
Or similar (or a wrapper function like Require does).
Then they pass that in, and the auto startup code will take care of the
rest. Kinda like how the Python versions just take in the gradle target, and
then the code takes care of the rest. In this case, everything should be
happening within the scope of the auto HandlerFunc.
We do not need to optimize for repeated calls to the same expansion service.
Users with complex enough pipelines hitting the same service should likely be
standing up their own anyway.
If we don't want to use up "auto" just for jar startup we can always have
something else to specify what kind of auto thing it's starting up: eg.
"auto:jar:" to have it select sub name space for handling, or "autojar:" and
just lean on the existing selector.
##########
File path: sdks/go/pkg/beam/core/runtime/xlangx/expand.go
##########
@@ -140,3 +140,45 @@ func QueryExpansionService(ctx context.Context, p
*HandlerParams) (*jobpb.Expans
}
return res, nil
}
+
+// QueryAutomatedExpansionService submits an external transform to be expanded
by the
+// expansion service and then eagerly materializes the artifacts for staging.
The given
+// transform should be the external transform, and the componenets are any
additional
Review comment:
```suggestion
// transform should be the external transform, and the components are any
additional
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 718351)
Time Spent: 17h 40m (was: 17.5h)
> [Go SDK] Add automated expansion service start up feature
> ---------------------------------------------------------
>
> Key: BEAM-13399
> URL: https://issues.apache.org/jira/browse/BEAM-13399
> Project: Beam
> Issue Type: Improvement
> Components: cross-language, sdk-go
> Reporter: Jack McCluskey
> Assignee: Jack McCluskey
> Priority: P2
> Time Spent: 17h 40m
> Remaining Estimate: 0h
>
> Add Go SDK support for automated expansion service start-up if one is not
> provided, following the example of the Python implementation.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
