This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 2f12213 Merge pull request #11254: [BEAM-7961] Refactors X-Lang test
pipelines
add b6f2db5 [BEAM-9577] Rename the Artifact{Staging,Retrieval}Service.
add 4c25de6 [BEAM-9577] Define the new Artifact{Staging,Retrieval}Service.
add f2a58b4 [BEAM-9577] Regenerate protos.
add b2ad0f9 [BEAM-9577] Implement the new
Artifact{Staging,Retrieval}Services in Python.
new dbc21d9 Merge pull request #11203 [BEAM-9577] Define and implement
dependency-aware artifact staging service.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../src/main/proto/beam_provision_api.proto | 2 +-
.../src/main/proto/beam_artifact_api.proto | 92 +-
.../core/construction/ArtifactServiceStager.java | 16 +-
.../InMemoryArtifactStagerService.java | 8 +-
.../runners/flink/FlinkJobServerDriverTest.java | 2 +-
...=> AbstractLegacyArtifactRetrievalService.java} | 22 +-
...a => AbstractLegacyArtifactStagingService.java} | 13 +-
...mFileSystemLegacyArtifactRetrievalService.java} | 17 +-
...eamFileSystemLegacyArtifactStagingService.java} | 22 +-
...ClassLoaderLegacyArtifactRetrievalService.java} | 11 +-
...avaFilesystemLegacyArtifactStagingService.java} | 11 +-
...ce.java => LegacyArtifactRetrievalService.java} | 2 +-
.../control/DefaultJobBundleFactory.java | 18 +-
.../environment/DockerEnvironmentFactory.java | 10 +-
.../environment/EmbeddedEnvironmentFactory.java | 4 +-
.../environment/EnvironmentFactory.java | 4 +-
.../environment/ExternalEnvironmentFactory.java | 10 +-
.../environment/ProcessEnvironmentFactory.java | 10 +-
.../StaticRemoteEnvironmentFactory.java | 4 +-
.../fnexecution/jobsubmission/JobServerDriver.java | 19 +-
.../jobsubmission/PortablePipelineJarCreator.java | 29 +-
.../BeamFileSystemArtifactServicesTest.java | 72 +-
.../artifact/ClassLoaderArtifactServiceTest.java | 84 +-
.../control/DefaultJobBundleFactoryTest.java | 6 +-
.../environment/DockerEnvironmentFactoryTest.java | 4 +-
.../environment/ProcessEnvironmentFactoryTest.java | 4 +-
.../beam/runners/samza/SamzaJobServerDriver.java | 4 +-
sdks/go/cmd/beamctl/cmd/artifact.go | 4 +-
sdks/go/pkg/beam/artifact/gcsproxy/retrieval.go | 2 +-
sdks/go/pkg/beam/artifact/gcsproxy/staging.go | 4 +-
sdks/go/pkg/beam/artifact/materialize.go | 12 +-
sdks/go/pkg/beam/artifact/materialize_test.go | 8 +-
sdks/go/pkg/beam/artifact/server_test.go | 8 +-
sdks/go/pkg/beam/artifact/stage.go | 10 +-
sdks/go/pkg/beam/artifact/stage_test.go | 8 +-
.../model/fnexecution_v1/beam_provision_api.pb.go | 2 +-
.../model/jobmanagement_v1/beam_artifact_api.pb.go | 929 +++++++++++++++++----
.../pkg/beam/runners/universal/runnerlib/stage.go | 2 +-
sdks/python/.pylintrc | 1 +
.../runners/portability/abstract_job_service.py | 2 +-
.../runners/portability/artifact_service.py | 263 +++++-
.../runners/portability/artifact_service_test.py | 170 +++-
.../runners/portability/fn_api_runner/fn_runner.py | 4 -
.../portability/fn_api_runner/worker_handlers.py | 8 +-
.../runners/portability/local_job_service.py | 2 +-
.../runners/portability/local_job_service_test.py | 2 +-
.../runners/portability/portable_stager.py | 14 +-
.../runners/portability/portable_stager_test.py | 11 +-
.../portability/spark_uber_jar_job_server_test.py | 2 +-
49 files changed, 1552 insertions(+), 416 deletions(-)
rename
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/{AbstractArtifactRetrievalService.java
=> AbstractLegacyArtifactRetrievalService.java} (90%)
rename
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/{AbstractArtifactStagingService.java
=> AbstractLegacyArtifactStagingService.java} (95%)
rename
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/{BeamFileSystemArtifactRetrievalService.java
=> BeamFileSystemLegacyArtifactRetrievalService.java} (84%)
rename
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/{BeamFileSystemArtifactStagingService.java
=> BeamFileSystemLegacyArtifactStagingService.java} (90%)
rename
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/{ClassLoaderArtifactRetrievalService.java
=> ClassLoaderLegacyArtifactRetrievalService.java} (80%)
rename
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/{JavaFilesystemArtifactStagingService.java
=> JavaFilesystemLegacyArtifactStagingService.java} (87%)
rename
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/{ArtifactRetrievalService.java
=> LegacyArtifactRetrievalService.java} (93%)
