This is an automated email from the ASF dual-hosted git repository.
Amar3tto pushed a change to branch fix-beaminference
in repository https://gitbox.apache.org/repos/asf/beam.git
discard b513d66f25d fix
discard b0e4997df34 fix
discard 99c34b6d835 Fix
add e55a1f7399b fix failure in opentelemetry-gcp-auth-extension jar
(#38938)
add 33b34574b3c Bump jsonpickle upper bound. (#38769)
add 2c53494f735 [Dataflow Java] Clarify which portions of
DataflowWorkerLoggingOptions are deprecated (#38960)
add 0bc785253eb Fix WriteToPubSub to pass ordering_key to publish() method
(#37345)
add d032f0e3ef4 Add drain support for Dataflow and Flink (#38786)
add c608b4da8de Bump google.golang.org/api from 0.284.0 to 0.285.0 in
/sdks (#38997)
add 9765e29de68 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks
(#38998)
add 6ccd0c9c861 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#38999)
add b9342767cc5 fix sentence (#38989)
add 597f92530f1 address DOM text reinterpreted as HTML alert (#38949)
add 56d4ffdab31 Align gcpauth extension implementation with ability to
turn it off easily while still being added as dependency.
add 2802de39b2e Merge pull request #38846: [OpenTelemetry] Add turning off
to gcpauth extension
add ee92a553e5d Disable public IP for Dataflow Python and Go Validate
runner tests (#38990)
add 1c182507444 Add support for Apache Flink 2.1.3 (#38961)
add 1d703fe904f Make ModelManager import more robust (#38936)
add 849fcf89c81 [Beam SQL Shell] Add Iceberg deps when using icebergio
(#38994)
add a3f9e0e7d96 Add publish_time_field to ReadFromPubSub YAML transform
(#38985)
add 15d00559d75 Bump cloud.google.com/go/bigtable from 1.49.0 to 1.50.0 in
/sdks (#39012)
add 5c34e1c1c85 Exclude tests for batch-dataset (#39014)
add 9e465a7cfd7 fix parsing error for typescript (#38898)
add c81059fb246 Drop Flink 1.17,1.18 support (#39006)
add 466989d50a4 cancel user code with interruption after timeout (#39018)
add 09c962c4aef Fix Milvus ML PreCommit xdist, container startup (#39007)
add b4e023afece Bump nodemailer from 8.0.5 to 9.0.1 in
/scripts/ci/issue-report (#39019)
add 17bed431e05 remove shardedKey (#39024)
add 63bcd3dd2b5 Validate Project of staging bucket (#39008)
add 8ba04741b47 Sanitize logging in FanOutStreamingEngineWorkerHarness
(#39029)
add 974ad17b60d Increase CallTest sleep durations to keep test from being
flaky due to timeout not being observed. (#39016)
add caba399f96e [Dataflow Java] Fix incorrect warning log when specifying
logger override. (#39034)
add 1df57236fe3 Bump github.com/moby/moby/api from 1.54.2 to 1.55.0 in
/sdks (#39031)
add fc370c9fdc8 [FnApi Java] Add support for separate named data streams
to provide bundle isolation (#38863)
add 630a34c3bfa Bump github.com/moby/moby/client from 0.4.1 to 0.5.0 in
/sdks (#39032)
add a21ca397b42 [Iceberg AddFiles] Add NameMapping when creating table
(#39022)
add 5334dc1d92a Add Flink 2.2.1 runner support (#38980)
add 06aa37bdffd use proper table naming in iceberg sql (#39035)
add e948ef62041 Roll forward io expansion service to Java17 (#38974)
add eaba570c569 Bump github.com/golang-cz/devslog from 0.0.16 to 0.0.17 in
/sdks (#39054)
new c752c4ddf53 Fix
new 87f8fbca975 fix
new 64901ed5d5a fix
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b513d66f25d)
\
N -- N -- N refs/heads/fix-beaminference (64901ed5d5a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../test-properties.json | 2 +-
.../actions/setup-environment-action/action.yml | 2 +-
.../IO_Iceberg_Integration_Tests.json | 2 +-
.github/trigger_files/beam_PostCommit_Go.json | 2 +-
.../beam_PostCommit_Java_DataflowV2.json | 2 +-
...ostCommit_Java_ValidatesRunner_Dataflow_V2.json | 2 +-
...beam_PostCommit_Java_ValidatesRunner_Spark.json | 18 +-
.github/trigger_files/beam_PostCommit_Python.json | 4 +-
...PostCommit_Python_ValidatesRunner_Dataflow.json | 2 +-
.../trigger_files/beam_PreCommit_Python_ML.json | 2 +-
.../beam_PerformanceTests_xlang_KafkaIO_Python.yml | 5 +
.../beam_PostCommit_Java_Examples_Flink.yml | 2 +-
.../beam_PostCommit_Java_Nexmark_Flink.yml | 4 +-
.../beam_PostCommit_Java_PVR_Flink_Batch.yml | 2 +-
.../beam_PostCommit_Java_PVR_Flink_Streaming.yml | 4 +-
.../workflows/beam_PostCommit_Java_Tpcds_Flink.yml | 2 +-
.../beam_PostCommit_Java_ValidatesRunner_Flink.yml | 4 +-
.../beam_PostCommit_Python_Xlang_Gcp_Dataflow.yml | 4 +
.../beam_PostCommit_Python_Xlang_Gcp_Direct.yml | 5 +
.../beam_PostCommit_Python_Xlang_IO_Dataflow.yml | 4 +
.../beam_PostCommit_Python_Xlang_IO_Direct.yml | 7 +-
.../beam_PostCommit_TransformService_Direct.yml | 7 +-
.github/workflows/beam_PostCommit_XVR_Direct.yml | 4 +
.github/workflows/beam_PostCommit_XVR_Flink.yml | 2 +-
.../beam_PostCommit_Yaml_Xlang_Direct.yml | 6 +-
.../workflows/beam_PreCommit_Flink_Container.yml | 2 +-
.../beam_PreCommit_Java_PVR_Flink_Batch.yml | 2 +-
.../beam_PreCommit_Java_PVR_Flink_Docker.yml | 2 +-
.../beam_PreCommit_Xlang_Generated_Transforms.yml | 6 +-
.../workflows/beam_PreCommit_Yaml_Xlang_Direct.yml | 5 +
.../workflows/beam_Publish_Beam_SDK_Snapshots.yml | 5 +-
.../workflows/beam_Publish_Docker_Snapshots.yml | 4 +-
.github/workflows/build_release_candidate.yml | 10 +-
.../run_rc_validation_java_quickstart.yml | 2 +-
.github/workflows/typescript_tests.yml | 10 +
.test-infra/validate-runner/build.gradle | 13 +
CHANGES.md | 4 +
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 7 +
examples/java/common.gradle | 10 +
gradle.properties | 2 +-
.../beam/model/fn_execution/v1/beam_fn_api.proto | 38 ++-
.../beam/model/pipeline/v1/beam_runner_api.proto | 4 +
release/build.gradle.kts | 2 +-
runners/flink/1.17/build.gradle | 25 --
runners/flink/1.18/build.gradle | 25 --
.../flink/streaming/MemoryStateBackendWrapper.java | 80 ------
.../runners/flink/streaming/StreamSources.java | 61 -----
runners/flink/{2.0 => 2.1}/build.gradle | 17 +-
.../job-server-container/build.gradle | 0
.../flink/{1.17 => 2.1}/job-server/build.gradle | 15 +-
runners/flink/{2.0 => 2.2}/build.gradle | 32 ++-
.../job-server-container/build.gradle | 0
.../flink/{1.18 => 2.2}/job-server/build.gradle | 15 +-
.../runners/flink/FlinkExecutionEnvironments.java | 4 +-
.../wrappers/streaming/DoFnOperator.java | 2 +-
.../runners/flink/FlinkPipelineOptionsTest.java | 16 +-
runners/flink/job-server/flink_job_server.gradle | 2 +-
.../runners/flink/FlinkDetachedRunnerResult.java | 60 ++++-
.../beam/runners/flink/FlinkRunnerResult.java | 6 +
.../beam/runners/flink/FlinkRunnerResultTest.java | 92 +++++++
.../flink/streaming/MemoryStateBackendWrapper.java | 28 ++-
.../runners/flink/streaming/StreamSources.java | 7 +
.../beam/runners/dataflow/DataflowPipelineJob.java | 67 +++--
.../options/DataflowWorkerLoggingOptions.java | 56 ++++-
.../runners/dataflow/DataflowPipelineJobTest.java | 46 ++++
.../logging/DataflowWorkerLoggingInitializer.java | 12 +-
.../FanOutStreamingEngineWorkerHarness.java | 7 +-
.../DataflowWorkerLoggingInitializerTest.java | 44 +++-
.../fnexecution/control/SdkHarnessClient.java | 2 +-
.../runners/fnexecution/data/FnDataService.java | 3 +-
.../runners/fnexecution/data/GrpcDataService.java | 13 +-
.../fnexecution/data/GrpcDataServiceTest.java | 2 +-
scripts/beam-sql.sh | 73 +++++-
scripts/ci/issue-report/package-lock.json | 14 +-
scripts/ci/issue-report/package.json | 2 +-
sdks/go.mod | 16 +-
sdks/go.sum | 32 +--
sdks/go/examples/wasm/README.md | 6 +-
sdks/go/test/build.gradle | 5 +-
.../java/org/apache/beam/sdk/PipelineResult.java | 13 +
.../sdk/fn/data/BeamFnDataGrpcMultiplexer.java | 2 +-
.../sdk/fn/data/BeamFnDataOutboundAggregator.java | 131 +++++++---
.../beam/sdk/util/construction/Environments.java | 1 +
.../fn/data/BeamFnDataOutboundAggregatorTest.java | 135 +++++-----
sdks/java/expansion-service/container/Dockerfile | 2 +-
.../opentelemetry-gcp-auth-extension/build.gradle | 1 +
.../opentelemetry/gcp/auth/ConfigurableOption.java | 1 +
...GcpAuthAutoConfigurationCustomizerProvider.java | 196 ++++++++-------
...nfigure.spi.AutoConfigurationCustomizerProvider | 16 --
...uthAutoConfigurationCustomizerProviderTest.java | 276 +++++++++++++--------
.../meta/provider/iceberg/IcebergMetastore.java | 8 +-
.../provider/iceberg/IcebergMetastoreTest.java | 13 +
.../java/org/apache/beam/fn/harness/FnHarness.java | 49 ++--
.../fn/harness/control/ProcessBundleHandler.java | 50 ++--
.../beam/fn/harness/data/BeamFnDataClient.java | 28 +--
.../beam/fn/harness/data/BeamFnDataGrpcClient.java | 91 ++++---
.../beam/fn/harness/BeamFnDataWriteRunnerTest.java | 44 ++--
.../beam/fn/harness/FnApiDoFnRunnerTest.java | 2 +-
.../PTransformRunnerFactoryTestContext.java | 12 +-
.../harness/control/ProcessBundleHandlerTest.java | 47 ++--
.../fn/harness/data/BeamFnDataGrpcClientTest.java | 41 ++-
sdks/java/io/expansion-service/build.gradle | 4 +-
.../beam/sdk/io/gcp/pubsub/ExternalWrite.java | 6 +
.../apache/beam/sdk/io/gcp/pubsub/PubsubIO.java | 5 +-
.../org/apache/beam/sdk/io/iceberg/AddFiles.java | 39 +--
.../beam/sdk/io/iceberg/IcebergCatalogConfig.java | 2 +-
.../org/apache/beam/io/requestresponse/Call.java | 1 +
.../apache/beam/io/requestresponse/CallTest.java | 53 ++--
sdks/python/apache_beam/io/external/gcp/pubsub.py | 11 +-
sdks/python/apache_beam/io/gcp/gcsio.py | 50 +++-
sdks/python/apache_beam/io/gcp/gcsio_test.py | 92 +++++++
sdks/python/apache_beam/io/gcp/pubsub.py | 48 +++-
.../apache_beam/io/gcp/pubsub_integration_test.py | 90 +++++++
sdks/python/apache_beam/io/gcp/pubsub_test.py | 72 ++++++
sdks/python/apache_beam/ml/inference/base.py | 6 +
.../ml/rag/enrichment/milvus_search_it_test.py | 1 +
.../ml/rag/ingestion/milvus_search_it_test.py | 1 +
sdks/python/apache_beam/ml/rag/test_utils.py | 1 +
.../python/apache_beam/options/pipeline_options.py | 2 +-
sdks/python/apache_beam/yaml/yaml_io.py | 21 +-
sdks/python/apache_beam/yaml/yaml_io_test.py | 103 ++++++++
sdks/python/scripts/run_integration_test.sh | 11 +-
sdks/python/setup.py | 3 +-
sdks/python/test-suites/dataflow/common.gradle | 2 +
sdks/typescript/src/apache_beam/runners/flink.ts | 2 +-
website/www/site/assets/js/bootstrap.js | 46 +++-
website/www/site/assets/js/bootstrap/alert.js | 7 +-
website/www/site/assets/js/bootstrap/carousel.js | 9 +-
website/www/site/assets/js/bootstrap/collapse.js | 6 +-
website/www/site/assets/js/bootstrap/dropdown.js | 7 +-
website/www/site/assets/js/bootstrap/modal.js | 8 +-
website/www/site/assets/js/bootstrap/tooltip.js | 9 +-
website/www/site/assets/js/page-nav.js | 2 +-
.../site/content/en/documentation/dsls/sql/ddl.md | 2 +-
.../site/content/en/documentation/runners/flink.md | 10 +
135 files changed, 2068 insertions(+), 938 deletions(-)
delete mode 100644 runners/flink/1.17/build.gradle
delete mode 100644 runners/flink/1.18/build.gradle
delete mode 100644
runners/flink/1.19/src/test/java/org/apache/beam/runners/flink/streaming/MemoryStateBackendWrapper.java
delete mode 100644
runners/flink/1.19/src/test/java/org/apache/beam/runners/flink/streaming/StreamSources.java
copy runners/flink/{2.0 => 2.1}/build.gradle (79%)
rename runners/flink/{1.17 => 2.1}/job-server-container/build.gradle (100%)
rename runners/flink/{1.17 => 2.1}/job-server/build.gradle (71%)
copy runners/flink/{2.0 => 2.2}/build.gradle (54%)
rename runners/flink/{1.18 => 2.2}/job-server-container/build.gradle (100%)
rename runners/flink/{1.18 => 2.2}/job-server/build.gradle (71%)
copy runners/flink/{2.0 =>
2.2}/src/main/java/org/apache/beam/runners/flink/FlinkExecutionEnvironments.java
(99%)
copy runners/flink/{2.0 =>
2.2}/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/DoFnOperator.java
(99%)
copy runners/flink/{2.0 =>
2.2}/src/test/java/org/apache/beam/runners/flink/FlinkPipelineOptionsTest.java
(92%)
delete mode 100644
sdks/java/extensions/opentelemetry-gcp-auth-extension/src/main/resources/META-INF/services/io.opentelemetry.sdk.autoconfigure.spi.AutoConfigurationCustomizerProvider
