This is an automated email from the ASF dual-hosted git repository.
derrickaw pushed a change to branch 20260624_zizmorInitial
in repository https://gitbox.apache.org/repos/asf/beam.git
omit 29141942b5a redo build release candidate workflow after change
omit ea94c6be8fb fix another workflow
omit 0d262d27c28 add comment
omit fee656a2534 add rule about action hashes
omit d2e1d913f33 Initial zizmor findings
add 2940f8fc6a0 Added Criteo TFT fill_in_missing helper (#39011)
add 0b407cd78ce Fix lost state updates in Set state (#39175)
add 09d63ccbb36 Bump pyo3 from 0.27.2 to 0.29.0 in
/sdks/python/apache_beam/examples/wordcount_rust/word_processing (#38953)
add 4d975e5ec60 Avoid RequestResponseIO throttler access when throttler is
absent (#39112)
add 904f8898ab2 Fix PostCommit Java DataflowV2 workflow (#39232)
add 4bf5a6afb50 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks
(#39227)
add 4cde7ed933a Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#39229)
add 7502ae2d529 Bump golang.org/x/text from 0.38.0 to 0.39.0 in /sdks
(#39230)
add f35fa826f44 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#39231)
add bdc26675c91 Drop envoy-data-plane/betterproto dependency from Python
SDK (#39213)
add c6e13b60fa4 Revert "Fix Python VarIntCoder OverflowError on uint64
values (#39047)" (#39218)
add 20494736bb9 Add MLTransform embedding benchmarks (#38917)
add d1c1a681d83 [aws2] Make SqsIOWriteBatchesTest timeout tests
timing-independent (#38971)
add 4a82e692e78 remove todos etc with pubsublite deprecation (#39237)
add 123dc7feaa4 Bump CI Java version to Java21 (#39205)
add fdd6b7b94c4 Adding release-2.75.0-postrelease to protected branches in
.asf.yaml
add 93d3aae1a4b Update managed-io.md for release 2.75.0-RC1.
add c6193b06ced Merge pull request #39178 from
apache/updates_managed_io_docs_2.75.0_rc1
add 954de0d7846 Update Beam website to release 2.75.0
add 5b7064e44a9 Update release date
add 5fd72c328d4 Update CHANGES.md
add 477c7477d62 Merge pull request #39190 from apache/website-2-75
add 2cecde92a13 Bump google.golang.org/api from 0.287.0 to 0.287.1 in
/sdks (#39239)
add bbb1c7592c7 Bump cloud.google.com/go/pubsub from 1.50.2 to 1.50.3 in
/sdks (#39240)
add 27d699eb721 [Gemini] Add client-side throttling to Java Remote
Inference (#39194)
add 227bcded5aa add upgrade gcp bom workflow (#38711)
add 401844f7131 add mongodbio and datadogio to io connectors for yaml
(#39223)
add 704f7d870d3 update action versions (#39234)
add 89fb1dcfedf [Python] Add UnboundedSource ValidatesRunner test (#38892)
add 1f4f5590a40 Pass --add-opens JVM flags to Spark job server (#39242)
add 3c28e775ee0 Update Bigquery Client (#38950)
add 6170ce68de1 Fix download_files() in python-release.
add 51b9e4d3266 Quote the url
add cb9f9473ba0 Merge pull request #39246 from
shunping/fix-python-release-scripts
add eba9ffa57e3 Update republish_released_docker_containers default
release version
add 955601bdb48 Bump actions/checkout from 6 to 7 (#39261)
add 81dd99c2d03 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#39260)
add 1c9175b4c64 Upgrade GCP Libraries BOM to 26.85.0 (#39243)
add d97ed99f191 Bump golang.org/x/sys from 0.46.0 to 0.47.0 in /sdks
(#39259)
add 1eba2f51d04 Bump cloud.google.com/go/pubsub from 1.50.3 to 1.50.4 in
/sdks (#39256)
add 9e4d607527a Initial zizmor findings
add 42beba04b3f add rule about action hashes
add 062522ee351 add comment
add 61f061cacb2 fix another workflow
add 9bb84c3e30f redo build release candidate workflow after change
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (29141942b5a)
\
N -- N -- N refs/heads/20260624_zizmorInitial (9bb84c3e30f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 1 +
.../actions/setup-environment-action/action.yml | 6 +-
.../arc/images/Dockerfile | 18 +-
...PostCommit_Python_ValidatesRunner_Dataflow.json | 3 +-
...am_PostCommit_Python_ValidatesRunner_Flink.json | 3 +-
...am_PostCommit_Python_ValidatesRunner_Spark.json | 3 +-
.../trigger_files/beam_PostCommit_XVR_Spark3.json | 2 +-
.github/workflows/README.md | 3 +-
.../beam_Inference_Python_Benchmarks_Dataflow.yml | 36 +
.../beam_PerformanceTests_xlang_KafkaIO_Python.yml | 6 +-
.../workflows/beam_PostCommit_Java_DataflowV2.yml | 2 +-
.../beam_PostCommit_Java_Jpms_Dataflow.yml | 2 -
.../workflows/beam_PostCommit_Java_Jpms_Direct.yml | 2 -
.../beam_PostCommit_Python_Xlang_Gcp_Dataflow.yml | 7 +-
.../beam_PostCommit_Python_Xlang_Gcp_Direct.yml | 6 +-
.../beam_PostCommit_Python_Xlang_IO_Dataflow.yml | 7 +-
.../beam_PostCommit_Python_Xlang_IO_Direct.yml | 7 +-
.../beam_PostCommit_TransformService_Direct.yml | 9 +-
.github/workflows/beam_PostCommit_XVR_Direct.yml | 5 +-
.github/workflows/beam_PostCommit_XVR_Flink.yml | 5 +-
.../beam_PostCommit_XVR_GoUsingJava_Dataflow.yml | 5 +-
...eam_PostCommit_XVR_JavaUsingPython_Dataflow.yml | 7 +-
..._PostCommit_XVR_PythonUsingJavaSQL_Dataflow.yml | 7 +-
...eam_PostCommit_XVR_PythonUsingJava_Dataflow.yml | 7 +-
.github/workflows/beam_PostCommit_XVR_Spark3.yml | 5 +-
.../beam_PreCommit_Java_Cassandra_IO_Direct.yml | 2 +
.../beam_PreCommit_Java_Debezium_IO_Direct.yml | 12 +-
...am_PreCommit_Java_Examples_Dataflow_Java11.yml} | 18 +-
.../beam_PreCommit_Java_Flink_Versions.yml | 2 +-
.../beam_PreCommit_Java_HCatalog_IO_Direct.yml | 2 +
.../workflows/beam_PreCommit_Java_IOs_Direct.yml | 13 +
.../beam_PreCommit_Java_Spark_Versions.yml | 4 +-
.../workflows/beam_PreCommit_Kotlin_Examples.yml | 6 +
.../workflows/beam_Publish_Beam_SDK_Snapshots.yml | 5 +-
.github/workflows/beam_Upgrade_GCP_BOM.yml | 94 +
.github/workflows/build_release_candidate.yml | 10 +-
...flow_MLTransform_Image_Embedding_CPU_Batch.txt} | 38 +-
...flow_MLTransform_Image_Embedding_GPU_Batch.txt} | 40 +-
..._Dataflow_MLTransform_Text_Embedding_Batch.txt} | 40 +-
.../republish_released_docker_containers.yml | 4 +-
.github/workflows/typescript_tests.yml | 4 +-
.test-infra/tools/refresh_looker_metrics.py | 5 +-
CHANGES.md | 54 +-
build.gradle.kts | 7 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 12 +-
contributor-docs/release-guide.md | 12 +-
examples/java/build.gradle | 7 +
examples/kotlin/build.gradle | 7 +
infra/iam/roles/roles_config.yaml | 2 +-
.../python_release_automation_utils.sh | 24 +-
.../runners/fnexecution/ServerFactoryTest.java | 19 +-
runners/spark/job-server/spark_job_server.gradle | 17 +-
runners/spark/spark_runner.gradle | 5 +-
scripts/tools/bomupgrader.py | 69 +-
sdks/go.mod | 36 +-
sdks/go.sum | 72 +-
.../container/license_scripts/dep_urls_java.yaml | 10 +-
.../java/org/apache/beam/sdk/util/ApiSurface.java | 7 +-
.../sdk/options/PipelineOptionsFactoryTest.java | 56 +-
.../providers/JavaFilterTransformProviderTest.java | 8 +-
.../beam/sdk/util/common/ReflectHelpersTest.java | 15 +-
sdks/java/extensions/arrow/build.gradle | 7 +-
sdks/java/extensions/avro/build.gradle | 7 +
.../sdk/io/aws2/sqs/SqsIOWriteBatchesTest.java | 97 +-
sdks/java/io/cassandra/build.gradle | 21 +
sdks/java/io/cdap/build.gradle | 5 +
.../org/apache/beam/sdk/io/csv/CsvIOParseTest.java | 6 +-
sdks/java/io/influxdb/build.gradle | 10 +-
.../apache/beam/sdk/io/influxdb/InfluxDbIO.java | 22 +
.../beam/sdk/io/influxdb/InfluxDbIOTest.java | 104 +-
sdks/java/io/kafka/build.gradle | 1 +
.../singlestore/SingleStoreDefaultRowMapper.java | 3 +-
sdks/java/ml/inference/remote/build.gradle | 1 +
.../sdk/ml/inference/remote/RemoteInference.java | 102 +-
.../ml/inference/remote/RemoteInferenceTest.java | 100 +
sdks/python/.yapfignore | 1 +
sdks/python/apache_beam/coders/coder_impl.pxd | 1 +
sdks/python/apache_beam/coders/coder_impl.py | 34 +-
.../apache_beam/coders/coders_test_common.py | 18 -
.../ml_transform/mltransform_image_embedding.py | 263 +
.../mltransform_image_embedding_test.py | 105 +
.../ml_transform/mltransform_text_embedding.py | 186 +
.../mltransform_text_embedding_test.py | 94 +
.../wordcount_rust/word_processing/Cargo.lock | 65 +-
.../wordcount_rust/word_processing/Cargo.toml | 2 +-
.../apache_beam/io/components/rate_limit.proto | 64 +
.../apache_beam/io/components/rate_limit_pb2.py | 63 +
.../apache_beam/io/components/rate_limiter.py | 39 +-
.../apache_beam/io/components/rate_limiter_test.py | 62 +-
.../clients/bigquery/bigquery_v2_client.py | 799 +-
.../clients/bigquery/bigquery_v2_messages.py | 9263 ++++++++++++++++----
sdks/python/apache_beam/io/iobase_test.py | 8 +-
sdks/python/apache_beam/io/requestresponse.py | 5 +-
sdks/python/apache_beam/io/requestresponse_test.py | 10 +
sdks/python/apache_beam/io/unbounded_source.py | 140 +
.../python/apache_beam/io/unbounded_source_test.py | 136 +-
.../mltransform_embedding_tests_requirements.txt} | 5 +-
.../portability/fn_api_runner/fn_runner_test.py | 29 +
.../runners/portability/spark_runner_test.py | 4 +
.../apache_beam/runners/worker/bundle_processor.py | 37 +-
.../benchmarks/cloudml/criteo_tft/criteo.py | 30 +-
.../benchmarks/cloudml/criteo_tft/criteo_test.py | 93 +
.../mltransform_image_embedding_benchmark.py | 128 +
.../mltransform_one_hot_encoding_benchmark.py | 79 +-
.../mltransform_text_embedding_benchmark.py | 117 +
.../testing/load_tests/dataflow_cost_benchmark.py | 143 +-
.../apache_beam/transforms/userstate_test.py | 69 +
.../container/ml/py310/base_image_requirements.txt | 2 -
.../container/ml/py310/gpu_image_requirements.txt | 2 -
.../container/ml/py311/base_image_requirements.txt | 2 -
.../container/ml/py311/gpu_image_requirements.txt | 2 -
.../container/ml/py312/base_image_requirements.txt | 2 -
.../container/ml/py312/gpu_image_requirements.txt | 2 -
.../container/ml/py313/base_image_requirements.txt | 2 -
.../container/py310/base_image_requirements.txt | 2 -
.../container/py311/base_image_requirements.txt | 2 -
.../container/py312/base_image_requirements.txt | 2 -
.../container/py313/base_image_requirements.txt | 2 -
.../container/py314/base_image_requirements.txt | 2 -
sdks/python/pyproject.toml | 1 +
sdks/python/scripts/run_job_server.sh | 8 +-
sdks/python/scripts/run_lint.sh | 1 +
sdks/python/setup.py | 4 -
website/www/site/config.toml | 2 +-
website/www/site/content/en/blog/beam-2.68.0.md | 2 +-
website/www/site/content/en/blog/beam-2.75.0.md | 72 +
.../dsls/sql/extensions/create-external-table.md | 52 -
.../site/content/en/documentation/io/connectors.md | 12 +-
.../site/content/en/documentation/io/managed-io.md | 546 +-
.../www/site/content/en/get-started/downloads.md | 18 +-
website/www/site/content/en/performance/_index.md | 3 +
.../mltransform-image-embedding-cpu/_index.md | 48 +
.../mltransform-image-embedding-gpu/_index.md | 48 +
.../_index.md | 24 +-
website/www/site/data/performance.yaml | 49 +-
135 files changed, 11313 insertions(+), 3081 deletions(-)
rename .github/workflows/{beam_PreCommit_Java_Examples_Dataflow_Java21.yml =>
beam_PreCommit_Java_Examples_Dataflow_Java11.yml} (93%)
create mode 100644 .github/workflows/beam_Upgrade_GCP_BOM.yml
copy
.github/workflows/load-tests-pipeline-options/{beam_Inference_Python_Benchmarks_Dataflow_Table_Row_Inference_Stream.txt
=>
beam_Inference_Python_Benchmarks_Dataflow_MLTransform_Image_Embedding_CPU_Batch.txt}
(59%)
copy
.github/workflows/load-tests-pipeline-options/{beam_Inference_Python_Benchmarks_Dataflow_Table_Row_Inference_Stream.txt
=>
beam_Inference_Python_Benchmarks_Dataflow_MLTransform_Image_Embedding_GPU_Batch.txt}
(54%)
copy
.github/workflows/load-tests-pipeline-options/{beam_Inference_Python_Benchmarks_Dataflow_MLTransform_Generate_Vocab_Batch.txt
=>
beam_Inference_Python_Benchmarks_Dataflow_MLTransform_Text_Embedding_Batch.txt}
(62%)
create mode 100644
sdks/python/apache_beam/examples/ml_transform/mltransform_image_embedding.py
create mode 100644
sdks/python/apache_beam/examples/ml_transform/mltransform_image_embedding_test.py
create mode 100644
sdks/python/apache_beam/examples/ml_transform/mltransform_text_embedding.py
create mode 100644
sdks/python/apache_beam/examples/ml_transform/mltransform_text_embedding_test.py
create mode 100644 sdks/python/apache_beam/io/components/rate_limit.proto
create mode 100644 sdks/python/apache_beam/io/components/rate_limit_pb2.py
copy sdks/python/{.coveragerc =>
apache_beam/ml/transforms/mltransform_embedding_tests_requirements.txt} (91%)
create mode 100644
sdks/python/apache_beam/testing/benchmarks/cloudml/criteo_tft/criteo_test.py
create mode 100644
sdks/python/apache_beam/testing/benchmarks/inference/mltransform_image_embedding_benchmark.py
create mode 100644
sdks/python/apache_beam/testing/benchmarks/inference/mltransform_text_embedding_benchmark.py
create mode 100644 website/www/site/content/en/blog/beam-2.75.0.md
create mode 100644
website/www/site/content/en/performance/mltransform-image-embedding-cpu/_index.md
create mode 100644
website/www/site/content/en/performance/mltransform-image-embedding-gpu/_index.md
copy website/www/site/content/en/performance/{mltransformonehot =>
mltransform-text-embedding}/_index.md (55%)
