This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 5e499388670 KeyCommitTooLarge logging improvements (#39316)
add 7356d4be850 Replace port with listeners in LocalKafka configuration
add 4f0044b5de9 Merge pull request #39438 from
sjvanrossum/testing-kafka-service-fix
add c7ab90ebad3 Bump google.golang.org/api from 0.289.0 to 0.290.0 in
/sdks (#39449)
add b0a7099e1f1 Fix lz4 conflicts (#39454)
add 1c9fd1bc1b9 Remove isort dependency, migrate to ruff equivalent
(#39411)
add c8aa28f7e2e docs: Document Fn Execution API protos (#39293)
add d688e1b9505 [Java SDK] Warn when ValueState contains collection types
(#37530)
add a72d6cba37a [KafkaIO] Use consumer position and lag to estimate end
offsets (#39285)
add 0140552d6d5 [#30019] Fix type checking failures around unions with
nested complex types (#39413)
add 9833974946c Remove legacy HttpError usage in Bigtable I/O IT test
(#39441)
add 9c3f51e56e0 [Python] Add Watch transform with growth_of polling SDF
(#39023)
add eb9e7a3dacd Override default fadvise to fix regression from
gcs-connector v3 upgrade (#39445)
add a3703d38100 [Dataflow Streaming][Multikey] Support MultiKey commits in
windmill clients (#38768)
add b15e5f47d40 Add equal_to_approx matcher for approximate numeric
assertions (#39443)
No new revisions were added by this update.
Summary of changes:
CHANGES.md | 1 +
it/build.gradle | 2 +-
.../beam/model/fn_execution/v1/beam_fn_api.proto | 63 +-
.../model/fn_execution/v1/beam_provision_api.proto | 2 +-
.../worker/windmill/client/WindmillStream.java | 5 +
.../worker/windmill/client/commits/Commit.java | 72 +-
.../worker/windmill/client/commits/Commits.java | 2 +-
.../windmill/client/commits/CompleteCommit.java | 15 -
.../commits/StreamingApplianceWorkCommitter.java | 10 +-
.../commits/StreamingEngineWorkCommitter.java | 92 ++-
.../windmill/client/grpc/GrpcCommitWorkStream.java | 93 ++-
.../dataflow/worker/FakeWindmillServer.java | 67 +-
.../StreamingApplianceWorkCommitterTest.java | 11 +-
.../commits/StreamingEngineWorkCommitterTest.java | 281 +++++++-
.../client/grpc/GrpcCommitWorkStreamTest.java | 85 +++
.../worker/windmill/src/main/proto/windmill.proto | 23 +
sdks/go.mod | 2 +-
sdks/go.sum | 4 +-
.../sdk/transforms/reflect/DoFnSignatures.java | 49 ++
.../sdk/transforms/reflect/DoFnSignaturesTest.java | 117 ++++
.../google-cloud-platform-core/build.gradle | 1 +
.../sdk/extensions/gcp/options/GcsOptions.java | 85 ++-
.../sdk/extensions/gcp/GcpCoreApiSurfaceTest.java | 2 +
.../sdk/extensions/gcp/options/GcsOptionsTest.java | 47 ++
sdks/java/io/hadoop-format/build.gradle | 1 +
.../beam/sdk/io/kafka/ReadFromKafkaDoFn.java | 114 +---
sdks/java/io/sparkreceiver/3/build.gradle | 6 +
.../apache/beam/sdk/testing/kafka/LocalKafka.java | 2 +-
.../apache_beam/io/gcp/bigtableio_it_test.py | 10 +-
sdks/python/apache_beam/io/watch.py | 730 +++++++++++++++++++++
sdks/python/apache_beam/io/watch_test.py | 456 +++++++++++++
sdks/python/apache_beam/testing/util.py | 43 ++
sdks/python/apache_beam/testing/util_test.py | 44 ++
sdks/python/apache_beam/typehints/typehints.py | 13 +-
.../python/apache_beam/typehints/typehints_test.py | 18 +
sdks/python/pyproject.toml | 105 +--
sdks/python/scripts/run_lint.sh | 37 +-
sdks/python/setup.py | 1 -
38 files changed, 2410 insertions(+), 301 deletions(-)
create mode 100644 sdks/python/apache_beam/io/watch.py
create mode 100644 sdks/python/apache_beam/io/watch_test.py
