This is an automated email from the ASF dual-hosted git repository.
derrickaw pushed a change to branch 20260624_zizmorInitial
in repository https://gitbox.apache.org/repos/asf/beam.git
omit acfb341637d another fix
omit 8f78fd98a56 redo build release candidate workflow after change
omit 797dc002098 fix another workflow
omit fe21a137890 add comment
omit 9b519a94fec add rule about action hashes
omit 0712cf6722d Initial zizmor findings
add 63aecf2dc67 Add JVM arguments to fix hadoop-format tests on JDK 17+
(#39283)
add 1081d592dcb Remove previous workarounds for when `super()` could not
be pickled on Py3 (#39187)
add a3de8e1aa30 Remove redundant argument (#39289)
add cccb6494667 Fix Community Metrics Prober job (#39290)
add 09af958190b Fix BeamRowWrapperTest microsecond precision flake (#39287)
add 23e23981b63 Fix GCP IO tests on Java 21 (#39295)
add ad6762a3bee --add-opens JVM flags (#39302)
add f20fca2d9e4 Add JVM --add-opens flags for Flink PVR on Java 21 (#39303)
add 5a3bbbaba0f rename scanner (#39304)
add ab498be6ef3 Use sbt 1.x
add 1e05fcd9815 Merge pull request #39291 from apache/fix-scio
add 42daed36b39 Bump jackson version. (#39281)
add 20ed72c83e3 [IcebergIO] Support TableIdentifiers with special
characters (#38876)
add 5af82d31aed Bump golang.org/x/crypto from 0.45.0 to 0.52.0 in
/.test-infra/mock-apis (#39314)
add 18f6800625b Add Cloud Spanner Directed Reads support to SpannerIO
(#39241)
add d53b0b176ca Bump cloud.google.com/go/bigquery from 1.78.0 to 1.79.0 in
/sdks (#39320)
add 0c2a58c27a6 Bump actions/setup-node from 6 to 7 (#39317)
add 8e588f6fa96 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks
(#39318)
add 73162ce3e3b Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#39321)
add db6686992c8 Log temp/staging location options parsing errors. (#39305)
add f8e4de0c634 Improve BigQueryIO Storage Write API error messages
(#38948)
add a6f2ff42e9e Remove google-perftools from the default image. (#39323)
add 92b92a1682e Bump soupsieve from 2.5 to 2.8.4 in /.test-infra/mock-apis
(#39315)
add a317020356c Adds a Dataflow integration test suite for Delta Lake
(#38954)
add d4c11ccd536 Bump cloud.google.com/go/storage from 1.63.0 to 1.63.1 in
/sdks (#39319)
add de21b1ce3c5 Fix republish container workflow as #39181 (#39326)
add de47138d5cd Bump actions/checkout from 6 to 7 (#39335)
add 1437711c969 Bump cloud.google.com/go/datastore from 1.24.0 to 1.25.0
in /sdks (#39333)
add 477a47fbeff Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#39334)
add 37cac0603bc Fix DataflowV1 test failure by fixing getSimpleName access
(#39330)
add 4de8bde8f49 add jdkAddOpenModules for Dataflow V2 (#39306)
add 48eab4f5262 Add Spark JVM --add-opens for (Nexmark, TPC-DS,
PortableJar) (#39337)
add d32207058b0 add Spark job-server --add-opens (#39339)
add e65186e1bb0 fix: add retries and query parameter encoding for GitHub
API requests (closes #39188)
add 7a16876ae37 fix: add Apache license header to test_sending.py
add 0d1f25d982d Merge pull request #39196 from
raman118/fix/issue-39188-github-api-retries
add 96ddb0cd511 Initial zizmor findings
add ef1abb0d80d add rule about action hashes
add ba37b2ff3aa add comment
add fafe46f00c5 fix another workflow
add c6243e78541 redo build release candidate workflow after change
add 76c44e72e77 another fix
add 354a6ae6a7e add more initial findings
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (acfb341637d)
\
N -- N -- N refs/heads/20260624_zizmorInitial (354a6ae6a7e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../IO_Iceberg_Integration_Tests.json | 2 +-
.../trigger_files/beam_PostCommit_Go_VR_Spark.json | 4 +-
.../beam_PostCommit_Java_DataflowV1.json | 2 +-
.../beam_PostCommit_Java_DataflowV2.json | 2 +-
...=> beam_PostCommit_Java_Delta_IO_Dataflow.json} | 2 +-
...son => beam_PostCommit_Java_Nexmark_Spark.json} | 0
.../beam_PostCommit_Java_PVR_Flink_Streaming.json | 2 +-
....json => beam_PostCommit_Java_Tpcds_Spark.json} | 0
...json => beam_PostCommit_PortableJar_Spark.json} | 0
... => beam_PostCommit_Python_Examples_Spark.json} | 0
...am_PostCommit_Python_ValidatesRunner_Spark.json | 3 +-
.github/workflows/README.md | 1 +
... => beam_PostCommit_Java_Delta_IO_Dataflow.yml} | 16 +-
.../beam_PostCommit_Java_Jpms_Dataflow.yml | 1 -
.github/workflows/build_release_candidate.yml | 2 +-
.github/workflows/pr-bot-new-prs.yml | 2 +-
.github/workflows/pr-bot-pr-updates.yml | 2 +-
.github/workflows/pr-bot-prs-needing-attention.yml | 2 +-
.github/workflows/reportGenerator.yml | 2 +-
.../republish_released_docker_containers.yml | 5 +-
.github/workflows/typescript_tests.yml | 6 +-
.../metrics/src/test/groovy/ProberTests.groovy | 8 +-
.test-infra/mock-apis/go.mod | 10 +-
.test-infra/mock-apis/go.sum | 20 +-
.test-infra/mock-apis/poetry.lock | 18 +-
CHANGES.md | 3 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +-
infra/enforcement/account_keys.py | 7 +-
infra/enforcement/sending.py | 66 +++++-
infra/enforcement/test_sending.py | 103 ++++++++
playground/backend/containers/scio/Dockerfile | 22 +-
runners/flink/job-server/flink_job_server.gradle | 25 +-
runners/portability/test_pipeline_jar.sh | 11 +-
sdks/go.mod | 24 +-
sdks/go.sum | 48 ++--
sdks/go/test/run_validatesrunner_tests.sh | 4 +
sdks/java/container/common.gradle | 1 +
sdks/java/container/java17/option-arrow.json | 9 +
sdks/java/container/java21/option-arrow.json | 9 +
sdks/java/container/java25/option-arrow.json | 9 +
.../container/license_scripts/dep_urls_java.yaml | 2 +-
.../beam/sdk/transforms/display/DisplayData.java | 7 +-
.../sdk/transforms/reflect/DoFnSignatures.java | 3 +-
.../beam/sdk/util/common/ReflectHelpers.java | 22 +-
sdks/java/io/delta/build.gradle | 38 +++
.../org/apache/beam/sdk/io/delta/DeltaIOIT.java | 264 +++++++++++++++++++++
sdks/java/io/google-cloud-platform/build.gradle | 5 +
.../bigquery/StorageApiWriteUnshardedRecords.java | 12 +-
.../bigquery/StorageApiWritesShardedRecords.java | 20 +-
.../beam/sdk/io/gcp/spanner/SpannerAccessor.java | 5 +
.../beam/sdk/io/gcp/spanner/SpannerConfig.java | 41 ++++
.../apache/beam/sdk/io/gcp/spanner/SpannerIO.java | 59 +++++
.../sdk/io/gcp/testing/FakeDatasetService.java | 18 ++
.../gcp/bigquery/BeamRowToStorageApiProtoTest.java | 11 +-
.../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 63 +++++
.../sdk/io/gcp/spanner/SpannerAccessorTest.java | 51 ++++
.../gcp/spanner/SpannerIOReadChangeStreamTest.java | 16 ++
sdks/java/io/hadoop-format/build.gradle | 21 ++
.../org/apache/beam/sdk/io/iceberg/AddFiles.java | 8 +-
.../beam/sdk/io/iceberg/AppendFilesToTables.java | 5 +-
.../iceberg/AssignDestinationsAndPartitions.java | 3 +-
.../beam/sdk/io/iceberg/FileWriteResult.java | 4 +-
.../beam/sdk/io/iceberg/IcebergCatalogConfig.java | 6 +-
.../IcebergCdcReadSchemaTransformProvider.java | 3 +-
.../IcebergReadSchemaTransformProvider.java | 3 +-
.../beam/sdk/io/iceberg/IcebergScanConfig.java | 6 +-
.../apache/beam/sdk/io/iceberg/IcebergUtils.java | 54 +++++
.../beam/sdk/io/iceberg/IncrementalScanSource.java | 4 +-
.../io/iceberg/OneTableDynamicDestinations.java | 6 +-
.../io/iceberg/PortableIcebergDestinations.java | 3 +-
.../apache/beam/sdk/io/iceberg/SnapshotInfo.java | 3 +-
.../org/apache/beam/sdk/io/iceberg/TableCache.java | 6 +-
.../beam/sdk/io/iceberg/cdc/ChangelogScanner.java | 4 +-
....java => BeamBaseIncrementalChangelogScan.java} | 10 +-
.../beam/sdk/io/iceberg/BeamRowWrapperTest.java | 9 +-
.../beam/sdk/io/iceberg/IcebergIOReadTest.java | 3 +-
.../beam/sdk/io/iceberg/IcebergIOWriteTest.java | 3 +-
.../beam/sdk/io/iceberg/IcebergUtilsTest.java | 64 +++++
sdks/java/testing/load-tests/build.gradle | 14 ++
sdks/java/testing/nexmark/build.gradle | 14 ++
sdks/java/testing/tpcds/build.gradle | 14 ++
.../apache_beam/examples/complete/autocomplete.py | 5 +-
.../examples/complete/game/game_stats.py | 14 +-
.../examples/complete/game/hourly_team_score.py | 19 +-
.../examples/complete/game/leader_board.py | 24 +-
.../examples/complete/game/user_score.py | 9 +-
.../examples/complete/top_wikipedia_sessions.py | 5 +-
.../examples/cookbook/bigtableio_it_test.py | 4 +-
.../apache_beam/examples/wordcount_debugging.py | 4 +-
.../apache_beam/examples/wordcount_with_metrics.py | 4 +-
.../python/apache_beam/options/pipeline_options.py | 14 +-
.../options/pipeline_options_validator.py | 10 +-
.../apache_beam/runners/portability/job_server.py | 4 +-
.../runners/portability/job_server_test.py | 2 +-
.../runners/portability/spark_runner.py | 11 +
.../runners/portability/spark_runner_test.py | 2 +
sdks/python/container/Dockerfile | 2 -
97 files changed, 1259 insertions(+), 236 deletions(-)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Samza.json =>
beam_PostCommit_Java_Delta_IO_Dataflow.json} (82%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Samza.json =>
beam_PostCommit_Java_Nexmark_Spark.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Samza.json =>
beam_PostCommit_Java_Tpcds_Spark.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Samza.json =>
beam_PostCommit_PortableJar_Spark.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Samza.json =>
beam_PostCommit_Python_Examples_Spark.json} (100%)
copy .github/workflows/{IO_Iceberg_Integration_Tests_Dataflow.yml =>
beam_PostCommit_Java_Delta_IO_Dataflow.yml} (85%)
create mode 100644 infra/enforcement/test_sending.py
create mode 100644 sdks/java/container/java17/option-arrow.json
create mode 100644 sdks/java/container/java21/option-arrow.json
create mode 100644 sdks/java/container/java25/option-arrow.json
create mode 100644
sdks/java/io/delta/src/test/java/org/apache/beam/sdk/io/delta/DeltaIOIT.java
rename
sdks/java/io/iceberg/src/main/java/org/apache/iceberg/{BaseIncrementalChangelogScan.java
=> BeamBaseIncrementalChangelogScan.java} (99%)
