This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a change to branch prismGrace1
in repository https://gitbox.apache.org/repos/asf/beam.git
omit db58501585e Increase prism server receive size to max.
omit 92b22b4b4a8 add swift to git ignore
omit ec0de9a2f9b Changes to allow Swift SDK to operate successfully with
the Flink Portable Runner as well as the Python Portable Runner. Modified the
PInput/POutput functionality to just be structs, this allows us to use them for
both closures as well as the eventual DoFn interface and cleans up the function
signatures.
omit 12b1a281600 Rename previous stream-based pardo implementation to
pstream and start to introduce an element-wise pardo. This better matches other
SDK conventions (i.e. processElement) and allows us to implicitly transfer the
timestamp and window from the input to an output. Repurposes the PInput and
POutput protocols to handle this functionality. Also update the map and flatmap
implementations to use pardo rather than pstream to get a sense of how well it
works (well).
omit c3ce7d6bab8 Added the primitives for a FileIO implementation with
Google Storage support to start. Doesn't do anything fancy, but does implement
a true version of wordcount with the classic Shakespeare input.
omit 337e6d02235 Added backwards links in PCollections to support Composite
transforms. Added a very basic composite version of WordCount to the test
suite. Still need to hook up the graph generation, which requires searching for
the roots from the leaves to give it more of that "SwiftUI" style.
omit be9b2aefef6 Start stubbing out more of the composite transform
implementation. Start stubbing out External Transforms and Schema support.
Improve automatic naming of pardos and make explicit name implementation
cleaner. Add S3 and GCS dependencies to implement basic internal IOs.
omit 82af74a6c57 Remove random output file
omit a5c5adaece9 Second attempt and fixing the line values.
omit e368cb229ad Turns out String interpolation breaks the #file and #line
macros. Try a new approach with optional "real" names to get more stable
transform names. Start adding "ObjectFn" for more complicated DoFn
implementations.
omit 1144716a510 Move typealiases for model protos to make it easier to
find and less likely to be ignored by .gitignore
omit a20a0728e46 safeAdvance to try to address issues with zero-length data
elements on non-mac platforms.
omit 840f2c618bd Trying to stop advanced(by:) crashes on non-macOS platforms
omit 0dd6adb7c60 Small change to try to stop crashing on Linux
omit 0221d9a6fb2 Use a single advance in varint implementation
omit edff60efc7d Add the generated proto files so other people don't have
to do it along with the typealias data
omit fc4aedecd67 Add licenses everywhere
omit d1605ed80fa Add the rest of the code. Basic word count integration
test now completes successfully with the Python Portable Runner.
omit 292632e6341 Add missing file
omit d48fc9f51c9 Moving some stuff around to make the proto construction
code easier to read
omit f77d6ae9ae6 Refactor Unary and Tuple SerializableFn into a single
ClosureFn since one is just a special case of the other. Add the wordcount
integration test to verify the refactoring does indeed work and start moving
the pipeline code into the branch.
omit 029ba71be57 Adding more of the Swift SDK implementation
omit eb732077f59 Moving experimental Swift SDK code into Beam repo
structure. Cleaning up and adding tests and Apache License labels where needed.
omit 54a235ff1c8 [prism] worker shutdown, cleanup, log fail
add 505f9421387 Fix flaky test StreamingDataflowWorkerTest (#28173)
add 45ea758b615 [yaml] yaml_transform.py unit tests python - PR 3/3
(#27356)
add 1d82bdcd42a [prism] worker shutdown, cleanup, log fail
add 1afa7faf567 Moving experimental Swift SDK code into Beam repo
structure. Cleaning up and adding tests and Apache License labels where needed.
add 0bdfd01cb35 Adding more of the Swift SDK implementation
add cbe751e83fa Refactor Unary and Tuple SerializableFn into a single
ClosureFn since one is just a special case of the other. Add the wordcount
integration test to verify the refactoring does indeed work and start moving
the pipeline code into the branch.
add 0cdfa2a4c49 Moving some stuff around to make the proto construction
code easier to read
add 84fca5fd140 Add missing file
add 3cd40d267f4 Add the rest of the code. Basic word count integration
test now completes successfully with the Python Portable Runner.
add 1c1e305a30c Add licenses everywhere
add 7fbddbc04b7 Add the generated proto files so other people don't have
to do it along with the typealias data
add 2145eb40a6c Use a single advance in varint implementation
add 2822f1a3453 Small change to try to stop crashing on Linux
add 5b3d21d4961 Trying to stop advanced(by:) crashes on non-macOS platforms
add 00e461bd1d9 safeAdvance to try to address issues with zero-length data
elements on non-mac platforms.
add 90db849ed8a Move typealiases for model protos to make it easier to
find and less likely to be ignored by .gitignore
add eec7c1991e3 Turns out String interpolation breaks the #file and #line
macros. Try a new approach with optional "real" names to get more stable
transform names. Start adding "ObjectFn" for more complicated DoFn
implementations.
add fa7690f86f7 Second attempt and fixing the line values.
add fbe4fa7d2ea Remove random output file
add b01ea078943 Start stubbing out more of the composite transform
implementation. Start stubbing out External Transforms and Schema support.
Improve automatic naming of pardos and make explicit name implementation
cleaner. Add S3 and GCS dependencies to implement basic internal IOs.
add 1eb6253e1ba Added backwards links in PCollections to support Composite
transforms. Added a very basic composite version of WordCount to the test
suite. Still need to hook up the graph generation, which requires searching for
the roots from the leaves to give it more of that "SwiftUI" style.
add a332bfe0c2e Added the primitives for a FileIO implementation with
Google Storage support to start. Doesn't do anything fancy, but does implement
a true version of wordcount with the classic Shakespeare input.
add 3b5062c2981 Rename previous stream-based pardo implementation to
pstream and start to introduce an element-wise pardo. This better matches other
SDK conventions (i.e. processElement) and allows us to implicitly transfer the
timestamp and window from the input to an output. Repurposes the PInput and
POutput protocols to handle this functionality. Also update the map and flatmap
implementations to use pardo rather than pstream to get a sense of how well it
works (well).
add 874ba9c72b3 Changes to allow Swift SDK to operate successfully with
the Flink Portable Runner as well as the Python Portable Runner. Modified the
PInput/POutput functionality to just be structs, this allows us to use them for
both closures as well as the eventual DoFn interface and cleans up the function
signatures.
add 1969ad8f329 add swift to git ignore
add a3c259336f0 Increase prism server receive size to max.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (db58501585e)
\
N -- N -- N refs/heads/prismGrace1 (a3c259336f0)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../dataflow/worker/util/BoundedQueueExecutor.java | 24 +-
.../worker/StreamingDataflowWorkerTest.java | 49 +-
sdks/python/apache_beam/yaml/yaml_transform.py | 14 +-
.../apache_beam/yaml/yaml_transform_unit_test.py | 854 ++++++++++++++++++++-
4 files changed, 897 insertions(+), 44 deletions(-)
