This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/cloud.google.com/go/bigtable-1.42.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 19116300798 Bump cloud.google.com/go/bigtable from 1.41.0 to 1.42.0 in
/sdks
add 1164f941f9c Bump filippo.io/edwards25519 from 1.1.0 to 1.1.1 in /sdks
(#37644)
add 41779ad88d3 [Dataflow Streaming] Update windmill timer clasification
logic to work with windmill state tag encoding v2 (#37604)
add 120ceab62a7 Fix renaming of embeddable_to_dict_fn (#37646)
add e41b9b6ba69 runQuickstartJavaFlinkLocal on Flink 2.0 (#37649)
add 872c671b15b Update cibuildwheel dependency (#37504)
add b9d48fa1750 Fix type checking failure with Python 3.10+ union pipe
syntax (int | None) (#37645)
add dfb3f872192 [yaml] : add jinja inheritance example (#37601)
add 59922a390e0 Fix breaking changes from renaming Chunk to EmbeddableItem
(#37651)
add 87c50c67e37 [Python] Support large pipeline options via file (#37379)
add 1052216416f Migrate remaining checkNotNull to
checkArgumentNotNull/checkStateNotNull in options package #18719 (#37654)
add 9ff96dfd329 Update Golang - DebeziumIO 3.1.3 (#37667)
add caa18faf4ff Adds triggerring configuration to KafkaIO eos. (#37648)
add cb7caaad2ca [Dataflow Streaming] Remove dead code around redundant
experiments (#37684)
add d614058ca95 Bump go.mongodb.org/mongo-driver from 1.17.8 to 1.17.9 in
/sdks (#37657)
add 9bfb7ab8c42 Fix resource leak in KafkaIO GCS truststore file download
(#37681)
add 4fe61733f77 Update Python Dependencies (#37678)
add bc35bef5d8f fix some python deps (#37668)
add 3a41311a1f9 unpin grpc. (#37633)
add 26b18c6e6f1 Revert "[yaml] : switch js2py to pythonmonkey (#37560)"
(#37665)
add 7095c880f34 [Dataflow Streaming] Mark support for windmill state tag
encoding v2 (#37683)
add ace4a54a15a dataframes -> dataframe
add 71c46ef01e4 Merge pull request #37697 from apache/fix-dask
add 451be0e289d Bump github.com/aws/smithy-go from 1.24.0 to 1.24.1 in
/sdks (#37695)
add 1541d027c52 Remove test-only functions in GcsUtil and change related
unit tests to use V1. (#37597)
add 6a1618eca88 fix(python): Register all output pcollections of a
transform rather than only ones that happened to be accessed in DoOutputsTuple
(#37556)
add 015b6417f97 Improve PR Bot reviewer assignment / pinging
add 6cab6eb17a8 Merge pull request #37663: Improve PR Bot reviewer
assignment / pinging
add eac11fed1d4 Simply wheel validation logic. (#37687)
add d503869c450 Bump lodash from 4.17.21 to 4.17.23 in /website/www
add df59030c9ba Merge pull request #37403: Bump lodash from 4.17.21 to
4.17.23 in /website/www
add f47f81cf244 [Python] Update dev SDK container tag for Python SDK
(#37661)
add 2341d3f7b7d Remove relative change threshold condition (#37643)
add 80b49029c64 Fix Playground Frontend Test job (#37642)
add 9ee4fe64f2b [yaml] : add available yaml io connectors to website
(#37605)
add e755b53b5a5 Add an instruction for using Docker buildx when building
containers. (#37706)
add 27a5fca9cca Moves expectMessage to before creating BeamFnLoggingClient
since exception can be thrown inside createAndStart method.
add 6b647c7e3d6 Merge pull request #37692: Moves expectMessage to before
creating BeamFnLoggingClient
add 940a8ca5db9 Bigtable: clean up client side metrics flag
add 193db392a93 Merge pull request #37596: Bigtable: clean up client side
metrics flag
add caf661798f7 Fix PreCommit tests involving docker-in-docker after
Milvus dependency rearrangements (#37711)
add 06dd48e9313 Revert "Update cibuildwheel dependency (#37504)" (#37713)
add 72027e11d1f Fix DebeziumIO resuming from worker restart (#37689)
add c51e244bb95 revert 'Optimize data size estimation for proto coder.'
(#37709)
add 2403014afce Stabilize
test_check_standard_external_transforms_config_in_sync (#37717)
add 9eb3a18ae70 fix conflict (#37594)
add 32d6bce8c23 Enable pickling main by reference option within
CloudpickleConfig (#37554)
add 5c2c3530411 Add Pyrefly configuration for Beam Python (#37700)
add e9f4c99206a fix Python 3.12 PostCommit by installing setuptools
(#37719)
add 67c3183913a Revise title and header in YAML documentation (#37722)
add 9524b56c148 Add observability (#37716)
add 30518c840bf [Stateful] Implement length-aware keying to minimize
padding in BatchElements (Part 2/3) (#37565)
add 5bee123ce8e Add monitoring for model manager (#37703)
add 4b3dcf26142 Fix GroupBy snippet tests for issue #30778 (#37672)
add 342de16b2f2 Fix Python PostCommit Flink runner log spam switching to
simple logger (#37720)
add af81a0c1635 Fix ml_base dependency (#37721)
add 5a6f763da54 [Drain] OnTimer - propagate caused by drain bit up to
DoFnRunner (#37012)
add fcbb0340800 Bump go.opentelemetry.io/otel/sdk from 1.38.0 to 1.40.0 in
/sdks (#37730)
add d8da4c9454a Bump cloud.google.com/go/bigtable from 1.41.0 to 1.42.0 in
/sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (19116300798)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/cloud.google.com/go/bigtable-1.42.0
(d8da4c9454a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.agent/skills/gradle-build/SKILL.md | 26 ++-
.agent/skills/python-development/SKILL.md | 12 +-
.../beam_PostCommit_Python_Xlang_Gcp_Direct.json | 2 +-
.../workflows/beam_PreCommit_Python_Coverage.yml | 35 +--
.github/workflows/beam_PreCommit_Python_ML.yml | 6 +-
.github/workflows/dask_runner_tests.yml | 2 +-
.../run_rc_validation_java_quickstart.yml | 2 +-
CHANGES.md | 4 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 13 +-
.../bigquery_vector_ingestion_and_search.ipynb | 2 +-
examples/terraform/envoy-ratelimiter/README.md | 58 ++++-
examples/terraform/envoy-ratelimiter/deploy.sh | 66 ++++++
.../terraform/envoy-ratelimiter/prerequisites.tf | 1 +
examples/terraform/envoy-ratelimiter/ratelimit.tf | 114 +++++-----
examples/terraform/envoy-ratelimiter/variables.tf | 4 +-
playground/frontend/build.gradle | 3 +-
.../integration_test/embedded_run_test.dart | 2 +-
.../frontend/playground_components/pubspec.yaml | 6 +-
.../lib/src/widget_tester.dart | 13 +-
.../playground_components_dev/pubspec.yaml | 4 +-
playground/frontend/pubspec.yaml | 4 +-
release/build.gradle.kts | 2 +-
.../org/apache/beam/runners/core/DoFnRunner.java | 4 +-
.../runners/core/LateDataDroppingDoFnRunner.java | 7 +-
...TimeBoundedSplittableProcessElementInvoker.java | 6 +
.../apache/beam/runners/core/ProcessFnRunner.java | 4 +-
.../runners/core/PushbackSideInputDoFnRunner.java | 4 +-
.../apache/beam/runners/core/SimpleDoFnRunner.java | 25 ++-
.../core/SimplePushbackSideInputDoFnRunner.java | 7 +-
.../core/SplittableParDoViaKeyedWorkItems.java | 2 +-
.../beam/runners/core/StatefulDoFnRunner.java | 16 +-
.../beam/runners/core/SimpleDoFnRunnerTest.java | 17 +-
.../SimplePushbackSideInputDoFnRunnerTest.java | 20 +-
.../beam/runners/core/StatefulDoFnRunnerTest.java | 3 +-
.../apache/beam/runners/direct/ParDoEvaluator.java | 3 +-
.../wrappers/streaming/DoFnOperator.java | 3 +-
.../functions/FlinkStatefulDoFnFunction.java | 3 +-
.../wrappers/streaming/DoFnOperator.java | 3 +-
.../flink/metrics/DoFnRunnerWithMetricsUpdate.java | 14 +-
.../functions/FlinkStatefulDoFnFunction.java | 3 +-
.../wrappers/streaming/DoFnOperator.java | 12 +-
.../streaming/ExecutableStageDoFnOperator.java | 7 +-
.../streaming/stableinput/BufferedElements.java | 10 +-
.../streaming/stableinput/BufferingDoFnRunner.java | 4 +-
.../wrappers/streaming/DoFnOperatorTest.java | 4 +-
.../streaming/ExecutableStageDoFnOperatorTest.java | 13 +-
.../beam/runners/dataflow/DataflowRunner.java | 19 +-
.../beam/runners/dataflow/DataflowRunnerTest.java | 5 +-
.../dataflow/worker/DataflowProcessFnRunner.java | 4 +-
.../dataflow/worker/GroupAlsoByWindowFnRunner.java | 4 +-
.../runners/dataflow/worker/SimpleParDoFn.java | 3 +-
.../StreamingKeyedWorkItemSideInputDoFnRunner.java | 4 +-
.../worker/StreamingModeExecutionContext.java | 34 ++-
.../worker/StreamingSideInputDoFnRunner.java | 4 +-
.../dataflow/worker/WindmillKeyedWorkItem.java | 16 +-
.../dataflow/worker/WindmillNamespacePrefix.java | 47 ----
.../dataflow/worker/WindmillTimerInternals.java | 28 +--
.../runners/dataflow/worker/WindmillTimerType.java | 10 +-
.../worker/windmill/state/WindmillTagEncoding.java | 24 +-
.../windmill/state/WindmillTagEncodingV1.java | 85 ++++---
.../windmill/state/WindmillTagEncodingV2.java | 42 ++--
...ractWindmillMap.java => WindmillTimerData.java} | 17 +-
.../worker/StreamingGroupAlsoByWindowFnsTest.java | 2 +-
.../dataflow/worker/WindmillKeyedWorkItemTest.java | 2 +-
.../windmill/state/WindmillTagEncodingV1Test.java | 23 +-
.../windmill/state/WindmillTagEncodingV2Test.java | 61 ++---
.../fnexecution/control/TimerReceiverFactory.java | 3 +-
.../translation/PipelineTranslatorUtils.java | 3 +-
.../fnexecution/control/RemoteExecutionTest.java | 4 +-
.../runners/fnexecution/wire/CommonCoderTest.java | 4 +-
.../runners/jet/processors/StatefulParDoP.java | 3 +-
.../samza/metrics/DoFnRunnerWithMetrics.java | 13 +-
.../runners/samza/runtime/AsyncDoFnRunner.java | 7 +-
.../apache/beam/runners/samza/runtime/DoFnOp.java | 3 +-
.../runtime/DoFnRunnerWithKeyedInternals.java | 13 +-
.../beam/runners/samza/runtime/PortableDoFnOp.java | 3 +-
.../runners/samza/runtime/SamzaDoFnRunners.java | 7 +-
.../translation/batch/DoFnRunnerFactory.java | 4 +-
.../translation/batch/DoFnRunnerWithMetrics.java | 14 +-
.../spark/translation/AbstractInOutIterator.java | 3 +-
.../spark/translation/DoFnRunnerWithMetrics.java | 14 +-
.../translation/AbstractInOutIteratorTest.java | 9 +-
.../translation/SparkInputDataProcessorTest.java | 4 +-
scripts/ci/pr-bot/findPrsNeedingAttention.ts | 30 +--
scripts/ci/pr-bot/processNewPrs.ts | 41 +++-
scripts/ci/pr-bot/processPrUpdate.ts | 52 +++--
sdks/go.mod | 16 +-
sdks/go.sum | 32 +--
sdks/go/pkg/beam/io/xlang/debeziumio/debezium.go | 8 +
.../test/integration/io/xlang/debezium/debezium.go | 3 +-
.../integration/io/xlang/debezium/debezium_test.go | 3 +-
.../apache/beam/sdk/metrics/MetricNameFilter.java | 8 +-
.../beam/sdk/options/PipelineOptionsFactory.java | 48 ++--
.../beam/sdk/options/PipelineOptionsValidator.java | 6 +-
.../beam/sdk/options/ProxyInvocationHandler.java | 63 ++++--
.../apache/beam/sdk/options/SdkHarnessOptions.java | 15 +-
.../org/apache/beam/sdk/options/ValueProvider.java | 15 +-
.../apache/beam/sdk/options/ValueProviders.java | 4 +-
.../java/org/apache/beam/sdk/transforms/DoFn.java | 7 +
.../org/apache/beam/sdk/transforms/DoFnTester.java | 6 +
.../construction/SplittableParDoNaiveBounded.java | 6 +
.../apache/beam/sdk/util/construction/Timer.java | 29 ++-
.../beam/sdk/util/construction/TimerTest.java | 28 ++-
.../beam/sdk/extensions/gcp/util/GcsUtil.java | 156 +------------
.../beam/sdk/extensions/gcp/util/GcsUtilTest.java | 207 ++++++++---------
.../apache/beam/fn/harness/FnApiDoFnRunner.java | 29 ++-
.../beam/fn/harness/FnApiDoFnRunnerTest.java | 4 +-
...plittablePairWithRestrictionDoFnRunnerTest.java | 4 +-
.../harness/control/ProcessBundleHandlerTest.java | 13 +-
.../harness/logging/BeamFnLoggingClientTest.java | 2 +-
sdks/java/io/debezium/src/README.md | 7 +-
.../org/apache/beam/io/debezium/DebeziumIO.java | 28 +--
.../io/debezium/DebeziumTransformRegistrar.java | 9 +
.../beam/io/debezium/KafkaSourceConsumerFn.java | 151 ++++++-------
.../io/debezium/KafkaSourceConsumerFnTest.java | 89 +++++++-
.../apache/beam/io/debezium/OffsetTrackerTest.java | 25 +--
.../io/gcp/bigtable/BigtableServiceFactory.java | 13 --
.../beam/sdk/io/kafka/KafkaExactlyOnceSink.java | 28 ++-
.../java/org/apache/beam/sdk/io/kafka/KafkaIO.java | 32 +++
.../io/kafka/KafkaReadSchemaTransformProvider.java | 31 +--
.../sdk/io/kafka/upgrade/KafkaIOTranslation.java | 7 +
.../io/kafka/upgrade/KafkaIOTranslationTest.java | 2 +
sdks/python/apache_beam/coders/coder_impl.py | 6 -
.../transforms/aggregation/groupby_attr.py | 13 +-
.../transforms/aggregation/groupby_attr_expr.py | 10 +-
.../transforms/aggregation/groupby_expr.py | 5 +-
.../aggregation/groupby_expr_aggregate.py | 10 +-
.../aggregation/groupby_global_aggregate.py | 6 +-
.../aggregation/groupby_simple_aggregate.py | 5 +-
.../transforms/aggregation/groupby_test.py | 20 +-
.../transforms/aggregation/groupby_two_exprs.py | 9 +-
.../internal/cloudpickle/cloudpickle.py | 23 +-
sdks/python/apache_beam/metrics/metric.py | 30 ++-
sdks/python/apache_beam/ml/inference/base.py | 17 ++
sdks/python/apache_beam/ml/inference/base_test.py | 39 ++++
.../apache_beam/ml/inference/model_manager.py | 28 +++
.../apache_beam/ml/inference/model_manager_test.py | 65 ++++++
.../ml/rag/enrichment/bigquery_vector_search.py | 2 +-
.../apache_beam/ml/rag/ingestion/bigquery.py | 81 ++++---
.../ml/rag/ingestion/bigquery_it_test.py | 2 +-
.../ml/rag/ingestion/milvus_search_it_test.py | 21 +-
sdks/python/apache_beam/pipeline.py | 5 +-
sdks/python/apache_beam/pipeline_test.py | 7 +-
.../apache_beam/runners/dataflow/internal/names.py | 2 +-
.../apache_beam/runners/worker/sdk_worker_main.py | 27 ++-
.../testing/analyzers/perf_analysis_utils.py | 6 -
.../external_transform_provider_it_test.py | 12 +-
sdks/python/apache_beam/transforms/util.py | 60 ++++-
sdks/python/apache_beam/transforms/util_test.py | 233 +++++++++++++++++++
sdks/python/apache_beam/typehints/typehints.py | 7 +-
.../python/apache_beam/typehints/typehints_test.py | 24 ++
.../yaml/examples/testing/examples_test.py | 18 +-
.../yaml/examples/testing/input_data.py | 5 +
.../examples/transforms/io/bigquery_write.yaml | 59 +++++
.../jinja/{include => inheritance}/README.md | 16 +-
.../base/base_pipeline.yaml} | 40 +++-
.../wordCountInheritance.yaml} | 19 +-
sdks/python/apache_beam/yaml/standard_io.yaml | 1 -
sdks/python/apache_beam/yaml/yaml_mapping.py | 249 +++++++--------------
sdks/python/apache_beam/yaml/yaml_udf_test.py | 21 +-
sdks/python/container/boot.go | 33 +--
.../container/ml/py310/base_image_requirements.txt | 27 +--
.../container/ml/py310/gpu_image_requirements.txt | 29 ++-
.../container/ml/py311/base_image_requirements.txt | 27 +--
.../container/ml/py311/gpu_image_requirements.txt | 29 ++-
.../container/ml/py312/base_image_requirements.txt | 27 +--
.../container/ml/py312/gpu_image_requirements.txt | 29 ++-
.../container/ml/py313/base_image_requirements.txt | 31 +--
sdks/python/container/piputil.go | 39 +++-
.../container/py310/base_image_requirements.txt | 21 +-
.../container/py311/base_image_requirements.txt | 21 +-
.../container/py312/base_image_requirements.txt | 21 +-
.../container/py313/base_image_requirements.txt | 25 +--
sdks/python/gen_xlang_wrappers.py | 6 +-
sdks/python/pyrefly.toml | 62 +++++
sdks/python/setup.py | 40 ++--
sdks/python/test-suites/portable/common.gradle | 37 ++-
sdks/python/tox.ini | 2 +-
start-build-env.sh | 4 +-
.../site/content/en/documentation/io/connectors.md | 140 ++++++++++--
.../www/site/content/en/documentation/sdks/yaml.md | 6 +-
website/www/yarn.lock | 6 +-
182 files changed, 2635 insertions(+), 1607 deletions(-)
create mode 100755 examples/terraform/envoy-ratelimiter/deploy.sh
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillNamespacePrefix.java
copy sdks/java/core/src/main/java/org/apache/beam/sdk/util/Weighted.java =>
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillTimerType.java
(80%)
copy
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/{AbstractWindmillMap.java
=> WindmillTimerData.java} (63%)
create mode 100644
sdks/python/apache_beam/yaml/examples/transforms/io/bigquery_write.yaml
copy sdks/python/apache_beam/yaml/examples/transforms/jinja/{include =>
inheritance}/README.md (77%)
copy
sdks/python/apache_beam/yaml/examples/transforms/jinja/{include/submodules/mapToFieldsSplitConfig.yaml
=> inheritance/base/base_pipeline.yaml} (53%)
copy
sdks/python/apache_beam/yaml/examples/transforms/jinja/{include/submodules/combineTransform.yaml
=> inheritance/wordCountInheritance.yaml} (69%)
create mode 100644 sdks/python/pyrefly.toml
