lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and
implement dependency-aware artifact staging service.
URL: https://github.com/apache/beam/pull/11203#discussion_r400467966
##########
File path: model/job-management/src/main/proto/beam_artifact_api.proto
##########
@@ -31,8 +31,92 @@ option java_outer_classname = "ArtifactApi";
import "beam_runner_api.proto";
-// A service to stage artifacts for use in a Job.
+// A service to retrieve artifacts for use in a Job.
+service ArtifactRetrievalService {
+ // Resolves the given artifact references into one or more replacement
+ // artifact references (e.g. a Maven dependency into a (transitive) set
+ // of jars.
+ rpc ResolveArtifact(ResolveArtifactRequest) returns
(ResolveArtifactResponse);
+
+ // Retrieves the given artifact as a stream of bytes.
+ rpc GetArtifact(GetArtifactRequest) returns (stream GetArtifactResponse);
+}
+
+// A service that allows the client to act as an ArtifactRetrievalService,
+// for a particular job with the server initiating requests and receiving
+// responses.
+//
+// A client calls the service with an ArtifactResponseWrapper that has the
+// staging token set, and thereafter responds to the server's requests.
service ArtifactStagingService {
+ rpc ReverseArtifactRetrievalService(stream ArtifactResponseWrapper)
+ returns (stream ArtifactRequestWrapper);
+}
+
+// A request for artifact resolution.
+message ResolveArtifactRequest {
+ // A set of artifacts to (jointly) resolve.
+ repeated org.apache.beam.model.pipeline.v1.ArtifactInformation artifacts = 1;
Review comment:
It is important to point out that the artifact information / ordering is
important in both the request and the response since in Java this would
represent the classpath. So artifacts should be passed in the expected order of
usage and returned in an expected order of usage.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
