This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a change to branch prismGrace1
in repository https://gitbox.apache.org/repos/asf/beam.git
discard a3c259336f0 Increase prism server receive size to max.
discard 1969ad8f329 add swift to git ignore
discard 874ba9c72b3 Changes to allow Swift SDK to operate successfully with
the Flink Portable Runner as well as the Python Portable Runner. Modified the
PInput/POutput functionality to just be structs, this allows us to use them for
both closures as well as the eventual DoFn interface and cleans up the function
signatures.
discard 3b5062c2981 Rename previous stream-based pardo implementation to
pstream and start to introduce an element-wise pardo. This better matches other
SDK conventions (i.e. processElement) and allows us to implicitly transfer the
timestamp and window from the input to an output. Repurposes the PInput and
POutput protocols to handle this functionality. Also update the map and flatmap
implementations to use pardo rather than pstream to get a sense of how well it
works (well).
discard a332bfe0c2e Added the primitives for a FileIO implementation with
Google Storage support to start. Doesn't do anything fancy, but does implement
a true version of wordcount with the classic Shakespeare input.
discard 1eb6253e1ba Added backwards links in PCollections to support Composite
transforms. Added a very basic composite version of WordCount to the test
suite. Still need to hook up the graph generation, which requires searching for
the roots from the leaves to give it more of that "SwiftUI" style.
discard b01ea078943 Start stubbing out more of the composite transform
implementation. Start stubbing out External Transforms and Schema support.
Improve automatic naming of pardos and make explicit name implementation
cleaner. Add S3 and GCS dependencies to implement basic internal IOs.
discard fbe4fa7d2ea Remove random output file
discard fa7690f86f7 Second attempt and fixing the line values.
discard eec7c1991e3 Turns out String interpolation breaks the #file and #line
macros. Try a new approach with optional "real" names to get more stable
transform names. Start adding "ObjectFn" for more complicated DoFn
implementations.
discard 90db849ed8a Move typealiases for model protos to make it easier to
find and less likely to be ignored by .gitignore
discard 00e461bd1d9 safeAdvance to try to address issues with zero-length data
elements on non-mac platforms.
discard 5b3d21d4961 Trying to stop advanced(by:) crashes on non-macOS platforms
discard 2822f1a3453 Small change to try to stop crashing on Linux
discard 2145eb40a6c Use a single advance in varint implementation
discard 7fbddbc04b7 Add the generated proto files so other people don't have
to do it along with the typealias data
discard 1c1e305a30c Add licenses everywhere
discard 3cd40d267f4 Add the rest of the code. Basic word count integration
test now completes successfully with the Python Portable Runner.
discard 84fca5fd140 Add missing file
discard 0cdfa2a4c49 Moving some stuff around to make the proto construction
code easier to read
discard cbe751e83fa Refactor Unary and Tuple SerializableFn into a single
ClosureFn since one is just a special case of the other. Add the wordcount
integration test to verify the refactoring does indeed work and start moving
the pipeline code into the branch.
discard 0bdfd01cb35 Adding more of the Swift SDK implementation
discard 1afa7faf567 Moving experimental Swift SDK code into Beam repo
structure. Cleaning up and adding tests and Apache License labels where needed.
add 185636c65a4 Increase prism server receive size to max.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a3c259336f0)
\
N -- N -- N refs/heads/prismGrace1 (185636c65a4)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.gitignore | 4 -
sdks/swift/Package.resolved | 194 -
sdks/swift/Package.swift | 70 -
sdks/swift/README.md | 25 -
sdks/swift/Scripts/generate_protobuf.sh | 97 -
sdks/swift/Sources/ApacheBeam/ApacheBeam.swift | 20 -
.../ApacheBeam/Client/ApiServiceDescriptor.swift | 91 -
.../Sources/ApacheBeam/Coders/BeamValue.swift | 72 -
.../swift/Sources/ApacheBeam/Coders/Beamable.swift | 41 -
.../Sources/ApacheBeam/Coders/Coder+Decoding.swift | 82 -
.../Sources/ApacheBeam/Coders/Coder+Encoding.swift | 109 -
sdks/swift/Sources/ApacheBeam/Coders/Coder.swift | 213 -
.../Sources/ApacheBeam/Core/ArtifactInfo.swift | 29 -
.../ApacheBeam/Core/DynamicProperties.swift | 38 -
.../Sources/ApacheBeam/Core/Environment.swift | 94 -
.../Sources/ApacheBeam/Core/Fn/ClosureFn.swift | 71 -
sdks/swift/Sources/ApacheBeam/Core/Fn/DoFn.swift | 59 -
.../Sources/ApacheBeam/Core/Fn/ObjectFn.swift | 36 -
.../Core/Fn/ParameterizedClosureFn.swift | 74 -
.../ApacheBeam/Core/Fn/SerializableFn.swift | 40 -
sdks/swift/Sources/ApacheBeam/Core/KeyValue.swift | 66 -
.../Core/PCollection/AnyPCollection.swift | 92 -
.../Core/PCollection/AnyPCollectionStream.swift | 97 -
.../ApacheBeam/Core/PCollection/PCollection.swift | 83 -
.../Core/PCollection/PCollectionStream.swift | 120 -
.../ApacheBeam/Core/PTransform/AnyPTransform.swift | 43 -
.../ApacheBeam/Core/PTransform/Output.swift | 18 -
.../ApacheBeam/Core/PTransform/PTransform.swift | 52 -
.../Core/PTransform/PTransformBuilder.swift | 51 -
.../ApacheBeam/Core/PTransform/Transform.swift | 7 -
.../Core/PTransform/TuplePTransform.swift | 20 -
.../ApacheBeam/Core/Pipeline/Pipeline.swift | 282 -
.../ApacheBeam/Core/Pipeline/PipelineContext.swift | 38 -
.../ApacheBeam/Core/Pipeline/PipelineRunner.swift | 21 -
.../Core/Pipeline/PipelineTransform.swift | 56 -
sdks/swift/Sources/ApacheBeam/Core/Windowing.swift | 51 -
.../model/fn_execution/v1/beam_fn_api.grpc.swift | 2061 -----
.../fn_execution/v1/beam_provision_api.grpc.swift | 332 -
.../job_management/v1/beam_artifact_api.grpc.swift | 1579 ----
.../v1/beam_expansion_api.grpc.swift | 425 -
.../job_management/v1/beam_job_api.grpc.swift | 1237 ---
.../model/pipeline/v1/beam_runner_api.grpc.swift | 322 -
.../model/fn_execution/v1/beam_fn_api.pb.swift | 5317 ------------
.../fn_execution/v1/beam_provision_api.pb.swift | 383 -
.../job_management/v1/beam_artifact_api.pb.swift | 1372 ---
.../job_management/v1/beam_expansion_api.pb.swift | 413 -
.../model/job_management/v1/beam_job_api.pb.swift | 1708 ----
.../model/pipeline/v1/beam_runner_api.pb.swift | 9135 --------------------
.../beam/model/pipeline/v1/endpoints.pb.swift | 178 -
.../model/pipeline/v1/external_transforms.pb.swift | 482 --
.../apache/beam/model/pipeline/v1/metrics.pb.swift | 1029 ---
.../apache/beam/model/pipeline/v1/schema.pb.swift | 2273 -----
.../model/pipeline/v1/standard_window_fns.pb.swift | 515 --
.../ApacheBeam/Internal/Array+Helpers.swift | 23 -
.../ApacheBeam/Internal/Data+Decoding.swift | 94 -
.../ApacheBeam/Internal/Data+Encoding.swift | 41 -
.../ApacheBeam/Internal/Date+Timestamp.swift | 27 -
.../ApacheBeam/Internal/Dictionary+Helpers.swift | 24 -
.../ApacheBeam/Internal/Model+Typealias.swift | 29 -
.../ApacheBeam/Internal/PipelineProto+Coder.swift | 46 -
.../Internal/PipelineProto+Environment.swift | 28 -
.../Internal/PipelineProto+Initializers.swift | 93 -
.../ApacheBeam/Internal/ProtoConversion.swift | 25 -
.../ApacheBeam/Internal/Server+Endpoint.swift | 35 -
.../Sources/ApacheBeam/Internal/String+Urns.swift | 38 -
.../ApacheBeam/Runners/PortableRunner.swift | 77 -
.../Runtime/Bundle/BundleProcessor.swift | 142 -
.../Sources/ApacheBeam/Runtime/Bundle/Sink.swift | 45 -
.../Sources/ApacheBeam/Runtime/Bundle/Source.swift | 71 -
.../ApacheBeam/Runtime/DataplaneClient.swift | 239 -
.../Runtime/Metrics/MetricReporter.swift | 4 -
.../Sources/ApacheBeam/Runtime/Worker/Worker.swift | 100 -
.../ApacheBeam/Runtime/Worker/WorkerProvider.swift | 83 -
sdks/swift/Sources/ApacheBeam/Schema/Row.swift | 28 -
.../Sources/ApacheBeam/Schema/RowProtocol.swift | 3 -
sdks/swift/Sources/ApacheBeam/Schema/Schema.swift | 22 -
.../ApacheBeam/Testing/PCollection+Testing.swift | 19 -
.../ApacheBeam/Testing/PCollectionTest.swift | 43 -
.../Sources/ApacheBeam/Transforms/Basic.swift | 100 -
.../ApacheBeam/Transforms/BuiltIn+Elements.swift | 56 -
.../Sources/ApacheBeam/Transforms/BuiltIn.swift | 179 -
.../Sources/ApacheBeam/Transforms/Combining.swift | 41 -
.../Sources/ApacheBeam/Transforms/Composite.swift | 20 -
.../Sources/ApacheBeam/Transforms/Grouping.swift | 26 -
.../Sources/ApacheBeam/Transforms/IO/DuckIO.swift | 13 -
.../Sources/ApacheBeam/Transforms/IO/FileIO.swift | 17 -
.../Transforms/IO/GoogleCloud/GoogleStorage.swift | 113 -
.../Documentation.docc/ApacheBeam.md | 37 -
.../Sources/Examples/Wordcount/Wordcount.swift | 2 -
.../Tests/ApacheBeamTests/ApacheBeamTests.swift | 30 -
.../Tests/ApacheBeamTests/Coders/CoderTests.swift | 71 -
.../Pipeline/CompositeIntegrationTests.swift | 77 -
.../ApacheBeamTests/Pipeline/FileIOTests.swift | 84 -
.../ApacheBeamTests/Pipeline/Fixtures/file1.txt | 4 -
.../ApacheBeamTests/Pipeline/Fixtures/file2.txt | 3 -
.../Pipeline/IntegrationTests.swift | 92 -
96 files changed, 33661 deletions(-)
delete mode 100644 sdks/swift/Package.resolved
delete mode 100644 sdks/swift/Package.swift
delete mode 100644 sdks/swift/README.md
delete mode 100755 sdks/swift/Scripts/generate_protobuf.sh
delete mode 100644 sdks/swift/Sources/ApacheBeam/ApacheBeam.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Client/ApiServiceDescriptor.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Coders/BeamValue.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Coders/Beamable.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Coders/Coder+Decoding.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Coders/Coder+Encoding.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Coders/Coder.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/ArtifactInfo.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/DynamicProperties.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/Environment.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/Fn/ClosureFn.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/Fn/DoFn.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/Fn/ObjectFn.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/Fn/ParameterizedClosureFn.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/Fn/SerializableFn.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/KeyValue.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/PCollection/AnyPCollection.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/PCollection/AnyPCollectionStream.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/PCollection/PCollection.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/PCollection/PCollectionStream.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/PTransform/AnyPTransform.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/PTransform/Output.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/PTransform/PTransform.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/PTransform/PTransformBuilder.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/PTransform/Transform.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/PTransform/TuplePTransform.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/Pipeline/Pipeline.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/Pipeline/PipelineContext.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/Pipeline/PipelineRunner.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Core/Pipeline/PipelineTransform.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Core/Windowing.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/GRPC/org/apache/beam/model/fn_execution/v1/beam_fn_api.grpc.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/GRPC/org/apache/beam/model/fn_execution/v1/beam_provision_api.grpc.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/GRPC/org/apache/beam/model/job_management/v1/beam_artifact_api.grpc.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/GRPC/org/apache/beam/model/job_management/v1/beam_expansion_api.grpc.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/GRPC/org/apache/beam/model/job_management/v1/beam_job_api.grpc.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/GRPC/org/apache/beam/model/pipeline/v1/beam_runner_api.grpc.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/fn_execution/v1/beam_fn_api.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/fn_execution/v1/beam_provision_api.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/job_management/v1/beam_artifact_api.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/job_management/v1/beam_expansion_api.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/job_management/v1/beam_job_api.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/pipeline/v1/beam_runner_api.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/pipeline/v1/endpoints.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/pipeline/v1/external_transforms.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/pipeline/v1/metrics.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/pipeline/v1/schema.pb.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Generated/Model/org/apache/beam/model/pipeline/v1/standard_window_fns.pb.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Internal/Array+Helpers.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Internal/Data+Decoding.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Internal/Data+Encoding.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Internal/Date+Timestamp.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Internal/Dictionary+Helpers.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Internal/Model+Typealias.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Internal/PipelineProto+Coder.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Internal/PipelineProto+Environment.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Internal/PipelineProto+Initializers.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Internal/ProtoConversion.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Internal/Server+Endpoint.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Internal/String+Urns.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Runners/PortableRunner.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Runtime/Bundle/BundleProcessor.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Runtime/Bundle/Sink.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Runtime/Bundle/Source.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Runtime/DataplaneClient.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Runtime/Metrics/MetricReporter.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Runtime/Worker/Worker.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Runtime/Worker/WorkerProvider.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Schema/Row.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Schema/RowProtocol.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Schema/Schema.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Testing/PCollection+Testing.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Testing/PCollectionTest.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Transforms/Basic.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Transforms/BuiltIn+Elements.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Transforms/BuiltIn.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Transforms/Combining.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Transforms/Composite.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Transforms/Grouping.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Transforms/IO/DuckIO.swift
delete mode 100644 sdks/swift/Sources/ApacheBeam/Transforms/IO/FileIO.swift
delete mode 100644
sdks/swift/Sources/ApacheBeam/Transforms/IO/GoogleCloud/GoogleStorage.swift
delete mode 100644
sdks/swift/Sources/ApacheBeamDocumentation/Documentation.docc/ApacheBeam.md
delete mode 100644 sdks/swift/Sources/Examples/Wordcount/Wordcount.swift
delete mode 100644 sdks/swift/Tests/ApacheBeamTests/ApacheBeamTests.swift
delete mode 100644 sdks/swift/Tests/ApacheBeamTests/Coders/CoderTests.swift
delete mode 100644
sdks/swift/Tests/ApacheBeamTests/Pipeline/CompositeIntegrationTests.swift
delete mode 100644 sdks/swift/Tests/ApacheBeamTests/Pipeline/FileIOTests.swift
delete mode 100644 sdks/swift/Tests/ApacheBeamTests/Pipeline/Fixtures/file1.txt
delete mode 100644 sdks/swift/Tests/ApacheBeamTests/Pipeline/Fixtures/file2.txt
delete mode 100644
sdks/swift/Tests/ApacheBeamTests/Pipeline/IntegrationTests.swift
