This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/golang.org/x/oauth2-0.28.0
in repository https://gitbox.apache.org/repos/asf/beam.git
omit 395b2c71a07 Bump golang.org/x/oauth2 from 0.26.0 to 0.28.0 in /sdks
add 5bb7cf54a1d Remove retry count assertion from
testInvalidRecordReceived (#34192)
add 8870dd81759 Add more time for grpc cleanup (#34180)
add 9303e3ead19 Fix failing test: Increase timeout (#34181)
add 57c644a9ae3 Fix test (#34162)
add c1d0fa4f850 Clarify BigQuery InsertRetryPolicy behavior for non-200
responses (#34118)
add a2d9e213b82 Bump dataflow java container version to
beam-master-20250307 (#34211)
add 1db2a5ae995 cloudpickle support is no longer experimental. (#34200)
add 0415bd6a771 Use individual windows rather than window sets in the
combining table. (#34193)
add dfc45f0d1fd Fix PostCommit Java DataflowV2 (#34209)
add 2507861de9f Updates the information regarding Managed I/O
add 8297f112c20 Merge pull request #34152: Updates the information
regarding Managed I/O
add 3096c31e3ee Bump jinja2 from 3.1.4 to 3.1.6 in
/.test-infra/jenkins/metrics_report (#34189)
add d89ccc96ca3 Implement ClickHouse Integration Testing Resource Manager
(#34230)
add 9015b2b180b [AnomalyDetection] Add detectors and some necessary
univariate trackers. (#34232)
add 9429df9b13b Clean up ExternalTransformRegistrar (#34237)
add caf80e844b9 Pin python-snappy==0.6.1 (#34233)
add 53baae84bd8 Kafka add counters v1 uw2 (#33503)
add c23dea40e4f Update go_tests.yml (#34241)
add 060d6f7f6dc call out the snappy version in the warning message (#34225)
add 1ffea2a3d5f Fix Docker dev environment set up (#34142)
add 611eb6a7e4a Remove obsolete release script. (#34239)
add 5d38b8efedc Factor out data file discovery. (#34238)
add 62bdf56e210 Issue warning on long running DoFn.Setup on legacy
Dataflow runner (#34215)
add a08b1234333 Fix typo on resource hints page. (#34216)
add df09b1d624f Don't attempt to cache the bulk multimap lookup call.
add cad3dc0174e Merge pull request #34150 Don't attempt to cache the bulk
multimap lookup call.
add a1b6a2aa8cf [AnomalyDetection] Add main and auxiliary transforms.
(#34234)
add 747141117d6 Refactor code and add some docstrings. (#34235)
add 2f804c2db18 Bump dataflow java container version - 20250312 (#34256)
add 12bc35874f5 Use same format for next and current release versions
(#34262)
add a69094861cb Putting the default retry settings which was overridden in
Dataflow templates (#34161)
add 86f43f29a0a Update Python container (#34255)
add 6dceef51658 Add yaml examples generation to the release scripts.
(#34217)
add 404f7e17d03 [Java] Added Metrics Configuration Support to Iceberg Data
Writers (#34140)
add 3708ad6563c Fix PostCommit Java PVR Spark3 Streaming job (#34253)
add d7ef7b1e9ff Update to latest bom (#34231)
add 01092b043fd Update pinned kafka version to 3.9.0 to support OAuth in
python pipelines on KafkaIO. (#34197)
add ee62d7def54 Fix PostCommit Python ValidatesContainer Dataflow With RC
job (#34246)
add 5892df81b24 Update code-change-guide.md (#34266)
add 89c6402690e Clean up per worker histogram reporting path (#34244)
add bd02aa62fda Remove 2.1 and 2.2 and add 2.8 for Kafka compatibility
test (#34265)
add 347d1f15f03 Fix Stress Tests Java Bigquery (#34078)
add fd30941a67c Fix spotless (#34279)
add f831f91268b Eliminate springframework dependency in KafkaIO (#34278)
add bdb26939350 Spark Runner : Fix invalid translateImpulse (#34272)
add 5950a17fdf8 Update join beam doc (#34284)
add 8f7513aee3c Update Lineage to use BoundedTrie
add 8b420bd5412 Update IOs to work with Lineage based on BoundedTrie
add 5a7311c5fcd Escape last segment path as whole
add df98cbc433e Fix comments
add a00f1840935 Merge pull request #33891 Use BoundedTrie metric to track
lineage in IO.
add 00861d8276a Updates 2.64.0 release notes to highlight recent additions
to the Managed API
add 56409b2e052 Merge pull request #34291: Updates 2.64.0 release notes to
highlight recent additions to the Managed API
add 085140af5b4 Use Avro 1.8-compatible Schema constructors in Storage
Write API translator (#34281)
add 4d8710d43d5 [AnomalyDetection] Support functions and classes as init
arguments in specifiable. (#34273)
add d42bbec28c6 add default port for HostAndPort instances used in
Windmill (#34061)
add c51d6d31009 docs: update CHANGES.md for Spark Runner (#34295)
add d4c2028e084 docs: fix typo in snippets_test.py (#34297)
add d6ce383b054 Update REVIEWERS.yml (#34299)
add d3b1312e882 Fix pre/postprocess type hints (#34298)
add 0aca8f1dfdc refactor main.py so that it can be called by
runner.run_async().
add 8a418f513d9 Merge pull request #34290 Refactor main.py so that it can
be called by runner.run_async().
add c7d67a8958a Better error messages for unavailable providers.
add fbdd8b32107 Merge pull request #34304 Better error messages for
unavailable providers.
add bef5238b76b Add a STRING format to PubSub reading that interpretes the
payload as utf-8 encoded. (#34301)
add e75577c8079 Add copyright year and Apache Beam (#34306)
add fa064b260b1 Honor JAVA_HOME when it is set (#34313)
add 76b0df92051 Consider workflow flaky if the 5 last runs were failed
(#34321)
add 82a9be5709d added test_accessing_valueprovider_info_after_run to test
accessing_valueprovider_info_after_run (#34315)
add 41cac5956e0 Fix Precommit SQL (#34320)
add b8505511d9b Fix PostCommit Python ValidatesContainer with RC workflow
(#34332)
add 1bf14ad0745 Bump golang.org/x/net from 0.35.0 to 0.37.0 in /sdks
(#34328)
add a2879debe09 Bump golang.org/x/oauth2 from 0.26.0 to 0.28.0 in /sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (395b2c71a07)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/golang.org/x/oauth2-0.28.0 (a2879debe09)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/REVIEWERS.yml | 4 -
.../IO_Iceberg_Integration_Tests.json | 2 +-
.../beam_PostCommit_Java_PVR_Spark3_Streaming.json | 2 +-
.github/trigger_files/beam_PostCommit_Python.json | 2 +-
.../beam_PostCommit_Python_Xlang_Gcp_Dataflow.json | 2 +-
.../beam_PostCommit_Python_Xlang_IO_Direct.json | 2 +-
..._Python_ValidatesContainer_Dataflow_With_RC.yml | 4 +-
.github/workflows/beam_PreCommit_Spotless.yml | 12 +-
.github/workflows/build_release_candidate.yml | 1 +
.github/workflows/cut_release_branch.yml | 8 +-
.github/workflows/go_tests.yml | 2 +-
.github/workflows/python_dependency_tests.yml | 2 +
.../jenkins/metrics_report/requirements.txt | 2 +-
.../github/github_runs_prefetcher/code/main.py | 8 +-
CHANGES.md | 4 +
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 12 +-
contributor-docs/code-change-guide.md | 2 +-
dev-support/docker/Dockerfile | 5 -
dev-support/docker/pkglist | 3 +-
it/{elasticsearch => clickhouse}/build.gradle | 35 +-
.../it/clickhouse/ClickHouseResourceManager.java | 353 ++++++++++++++
.../ClickHouseResourceManagerException.java | 14 +-
.../apache/beam/it/clickhouse/ClickHouseUtils.java | 58 +++
.../apache/beam/it/clickhouse}/package-info.java | 4 +-
.../it/clickhouse/ClickHouseResourceManagerIT.java | 85 ++++
.../clickhouse/ClickHouseResourceManagerTest.java | 186 ++++++++
.../beam/it/clickhouse/ClickHouseUtilsTest.java | 96 ++++
.../apache/beam/it/gcp/bigquery/BigQueryIOST.java | 8 +-
.../apache/beam/model/pipeline/v1/metrics.proto | 3 +
.../src/main/scripts/build_release_candidate.sh | 439 -----------------
.../runners/core/metrics/MetricsContainerImpl.java | 7 +
.../core/metrics/MonitoringInfoConstants.java | 13 +
.../core/metrics/MonitoringInfoMetricName.java | 1 +
.../core/metrics/MetricsContainerImplTest.java | 38 ++
.../core/metrics/MonitoringInfoConstantsTest.java | 18 +
.../unbounded/FlinkUnboundedSourceReaderTest.java | 4 +-
runners/google-cloud-dataflow-java/build.gradle | 4 +-
.../dataflow/worker/DataflowMetricsContainer.java | 11 +-
.../dataflow/worker/DoFnInstanceManagers.java | 14 +
.../worker/StreamingStepMetricsContainer.java | 64 +--
.../worker/windmill/WindmillEndpoints.java | 4 +-
.../client/grpc/stubs/WindmillChannelFactory.java | 13 +-
.../worker/StreamingStepMetricsContainerTest.java | 88 +++-
.../dataflow/worker/streaming/StageInfoTest.java | 2 +-
.../FanOutStreamingEngineWorkerHarnessTest.java | 5 +-
.../SparkStreamingPortablePipelineTranslator.java | 8 +-
sdks/go.mod | 13 +-
sdks/go.sum | 24 +-
.../container/license_scripts/dep_urls_java.yaml | 2 +-
.../apache/beam/sdk/metrics/DelegatingGauge.java | 25 +-
.../beam/sdk/metrics/DelegatingHistogram.java | 25 +-
.../beam/sdk/metrics/LabeledMetricNameUtils.java | 16 +-
.../java/org/apache/beam/sdk/metrics/Lineage.java | 197 +++++---
.../org/apache/beam/sdk/metrics/MetricName.java | 21 +-
.../java/org/apache/beam/sdk/metrics/Metrics.java | 8 +
.../apache/beam/sdk/metrics/MetricsContainer.java | 17 -
.../org/apache/beam/sdk/metrics/NoOpGauge.java | 42 --
.../sdk/metrics/LabeledMetricNameUtilsTest.java | 27 +-
.../org/apache/beam/sdk/metrics/LineageTest.java | 64 ++-
.../sdk/extensions/gcp/storage/GcsFileSystem.java | 2 +-
.../extensions/gcp/storage/GcsFileSystemTest.java | 2 +-
.../ExternalSchemaIOTransformRegistrar.java | 41 +-
.../beam/fn/harness/state/MultimapSideInput.java | 4 +-
.../apache/beam/sdk/io/aws2/s3/S3FileSystem.java | 2 +-
.../beam/sdk/io/aws2/s3/S3FileSystemTest.java | 2 +-
.../azure/blobstore/AzureBlobStoreFileSystem.java | 5 +-
.../blobstore/AzureBlobStoreFileSystemTest.java | 2 +-
sdks/java/io/expansion-service/build.gradle | 8 +-
.../AvroGenericRecordToStorageApiProto.java | 9 +-
.../sdk/io/gcp/bigquery/BigQuerySinkMetrics.java | 4 +-
.../sdk/io/gcp/bigquery/InsertRetryPolicy.java | 17 +-
.../beam/sdk/io/gcp/firestore/RpcQosImpl.java | 7 +
.../sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java | 6 +-
.../apache/beam/sdk/io/gcp/pubsub/PubsubIO.java | 5 +-
.../beam/sdk/io/gcp/spanner/SpannerAccessor.java | 23 +-
.../io/gcp/bigquery/BigQuerySinkMetricsTest.java | 18 +-
.../gcp/bigquery/StorageApiSinkSchemaUpdateIT.java | 15 +-
.../gcp/bigquery/StreamingInsertsMetricsTest.java | 6 +-
.../beam/sdk/io/gcp/bigtable/BigtableReadIT.java | 5 +-
.../beam/sdk/io/gcp/bigtable/BigtableWriteIT.java | 4 +-
.../beam/sdk/io/gcp/spanner/SpannerIOReadTest.java | 50 +-
.../sdk/io/gcp/spanner/SpannerIOWriteTest.java | 77 +--
.../SpannerChangeStreamErrorTest.java | 5 -
.../apache/beam/sdk/io/iceberg/RecordWriter.java | 6 +
.../sdk/io/iceberg/RecordWriterManagerTest.java | 184 ++++++-
.../org/apache/beam/sdk/io/jdbc/JdbcIOTest.java | 11 +-
sdks/java/io/kafka/build.gradle | 21 +-
.../io/kafka/{kafka-211 => kafka-282}/build.gradle | 4 +-
.../io/kafka/{kafka-222 => kafka-390}/build.gradle | 4 +-
.../org/apache/beam/sdk/io/kafka/ConsumerSpEL.java | 31 --
.../org/apache/beam/sdk/io/kafka/KafkaMetrics.java | 40 +-
.../apache/beam/sdk/io/kafka/KafkaSinkMetrics.java | 26 +-
.../beam/sdk/io/kafka/KafkaUnboundedReader.java | 6 +-
.../beam/sdk/io/kafka/ReadFromKafkaDoFn.java | 17 +-
.../apache/beam/sdk/io/kafka/KafkaMetricsTest.java | 53 ++-
.../beam/sdk/io/kafka/KafkaSinkMetricsTest.java | 23 +-
.../apache_beam/examples/snippets/snippets.py | 2 +-
.../apache_beam/examples/snippets/snippets_test.py | 14 +-
.../io/external/xlang_jdbcio_it_test.py | 16 +-
sdks/python/apache_beam/io/tfrecordio.py | 2 +-
sdks/python/apache_beam/ml/anomaly/aggregations.py | 23 +-
sdks/python/apache_beam/ml/anomaly/base.py | 15 +-
.../{internal => ml/anomaly/detectors}/__init__.py | 4 +-
.../python/apache_beam/ml/anomaly/detectors/iqr.py | 129 +++++
.../anomaly/detectors/iqr_test.py} | 35 +-
.../ml/anomaly/detectors/robust_zscore.py | 116 +++++
.../anomaly/detectors/robust_zscore_test.py} | 34 +-
.../apache_beam/ml/anomaly/detectors/zscore.py | 126 +++++
.../ml/anomaly/detectors/zscore_test.py | 83 ++++
sdks/python/apache_beam/ml/anomaly/specifiable.py | 97 +++-
.../apache_beam/ml/anomaly/specifiable_test.py | 111 ++++-
sdks/python/apache_beam/ml/anomaly/thresholds.py | 159 -------
.../apache_beam/ml/anomaly/thresholds_test.py | 157 ------
sdks/python/apache_beam/ml/anomaly/transforms.py | 529 +++++++++++++++++++++
.../apache_beam/ml/anomaly/transforms_test.py | 413 ++++++++++++++++
.../apache_beam/ml/anomaly/univariate/base.py | 2 +
.../apache_beam/ml/anomaly/univariate/mad.py | 87 ++++
.../anomaly/univariate/mad_test.py} | 36 +-
.../apache_beam/ml/anomaly/univariate/median.py | 67 +++
.../ml/anomaly/univariate/median_test.py | 72 +++
.../apache_beam/ml/anomaly/univariate/quantile.py | 58 ++-
sdks/python/apache_beam/ml/inference/base.py | 4 +-
.../apache_beam/runners/dataflow/internal/names.py | 2 +-
.../portability/fn_api_runner/fn_runner_test.py | 21 +
.../apache_beam/runners/worker/operations.pxd | 1 +
.../apache_beam/runners/worker/operations.py | 63 +--
sdks/python/apache_beam/utils/subprocess_server.py | 23 +-
sdks/python/apache_beam/yaml/main.py | 70 ++-
sdks/python/apache_beam/yaml/yaml_io.py | 11 +-
sdks/python/apache_beam/yaml/yaml_io_test.py | 22 +
sdks/python/apache_beam/yaml/yaml_provider.py | 39 +-
sdks/python/apache_beam/yaml/yaml_transform.py | 37 +-
.../python/apache_beam/yaml/yaml_transform_test.py | 2 +-
sdks/python/apache_beam/yaml/yaml_utils.py | 5 +
sdks/python/build.gradle | 3 +-
.../container/base_image_requirements_manual.txt | 3 +-
.../container/py310/base_image_requirements.txt | 2 +-
.../container/py311/base_image_requirements.txt | 2 +-
.../container/py312/base_image_requirements.txt | 2 +-
.../container/py39/base_image_requirements.txt | 2 +-
sdks/python/container/run_validatescontainer.sh | 11 +-
sdks/python/scripts/generate_pydoc.sh | 1 +
sdks/python/test-suites/portable/common.gradle | 3 +-
settings.gradle.kts | 12 +-
.../www/site/content/en/contribute/attributes.md | 2 +-
.../site/content/en/documentation/io/connectors.md | 9 +-
.../en/documentation/runtime/resource-hints.md | 2 +-
.../sdks/python-pipeline-dependencies.md | 2 +-
website/www/site/data/en/join_beam.yaml | 2 +-
149 files changed, 4017 insertions(+), 1608 deletions(-)
copy it/{elasticsearch => clickhouse}/build.gradle (58%)
create mode 100644
it/clickhouse/src/main/java/org/apache/beam/it/clickhouse/ClickHouseResourceManager.java
copy learning/katas/java/IO/Built-in IOs/Built-in
IOs/test/org/apache/beam/learning/katas/io/builtinios/TaskTest.java =>
it/clickhouse/src/main/java/org/apache/beam/it/clickhouse/ClickHouseResourceManagerException.java
(73%)
create mode 100644
it/clickhouse/src/main/java/org/apache/beam/it/clickhouse/ClickHouseUtils.java
copy it/{google-cloud-platform/src/main/java/org/apache/beam/it/gcp/datastream
=> clickhouse/src/main/java/org/apache/beam/it/clickhouse}/package-info.java
(88%)
create mode 100644
it/clickhouse/src/test/java/org/apache/beam/it/clickhouse/ClickHouseResourceManagerIT.java
create mode 100644
it/clickhouse/src/test/java/org/apache/beam/it/clickhouse/ClickHouseResourceManagerTest.java
create mode 100644
it/clickhouse/src/test/java/org/apache/beam/it/clickhouse/ClickHouseUtilsTest.java
delete mode 100755 release/src/main/scripts/build_release_candidate.sh
delete mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/metrics/NoOpGauge.java
rename sdks/java/io/kafka/{kafka-211 => kafka-282}/build.gradle (90%)
rename sdks/java/io/kafka/{kafka-222 => kafka-390}/build.gradle (90%)
copy sdks/python/apache_beam/{internal => ml/anomaly/detectors}/__init__.py
(81%)
create mode 100644 sdks/python/apache_beam/ml/anomaly/detectors/iqr.py
copy sdks/python/apache_beam/{io/flink/flink_streaming_impulse_source_test.py
=> ml/anomaly/detectors/iqr_test.py} (62%)
create mode 100644
sdks/python/apache_beam/ml/anomaly/detectors/robust_zscore.py
copy sdks/python/apache_beam/{io/flink/flink_streaming_impulse_source_test.py
=> ml/anomaly/detectors/robust_zscore_test.py} (60%)
create mode 100644 sdks/python/apache_beam/ml/anomaly/detectors/zscore.py
create mode 100644 sdks/python/apache_beam/ml/anomaly/detectors/zscore_test.py
create mode 100644 sdks/python/apache_beam/ml/anomaly/transforms.py
create mode 100644 sdks/python/apache_beam/ml/anomaly/transforms_test.py
create mode 100644 sdks/python/apache_beam/ml/anomaly/univariate/mad.py
copy sdks/python/apache_beam/{io/flink/flink_streaming_impulse_source_test.py
=> ml/anomaly/univariate/mad_test.py} (55%)
create mode 100644 sdks/python/apache_beam/ml/anomaly/univariate/median.py
create mode 100644 sdks/python/apache_beam/ml/anomaly/univariate/median_test.py
