This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from df3384f96a2 Also check the existence of _fields in addition
to__annotations__ for NamedTuple. (#36337)
add 1c8bcf9bae1 Fix race condition and nil pointer dereferencing (#36370)
add 45c36901859 Bump google.golang.org/protobuf from 1.36.8 to 1.36.10 in
/sdks (#36371)
add 24d3a7bac85 Fix errorprone move serialization to private method
(#36352)
add a03e96ad981 Update GitHub Actions to run Python 3.13 Tests (#35056)
add bf39e48ebdc Update Python SDK container (#36374)
add bd5cfdacc07 Make ml tests portable (#36377)
add 586cb119224 [Prism] Support AfterSynchronizedProcessingTime and enable
java processing-time trigger tests (#36379)
add c0774c9b636 Trigger python coverage workflow on prism changes (#36378)
add 659cc4dfbd6 Add pipeline option to enforce gbek (#36321)
add 9f8e00e4393 Switch streaming engine worker harness based on job
settings (#35901)
add b9c277244f2 Add Jpms tests for Java 25 (#36380)
add 800e2d7da5f Implement histogram user metric for python SDK (#36335)
add 82952c8c29a Update python metrics example in programming guide (#36334)
add 5af82f5f78c feat(subprocess_server): add fallback to Google Maven
mirror (#36365)
add 1e973630e67 fix(sdk:python): Avoid AttributeError for transforms
without hints (#36251)
add 5485467f230 Handle pending adjustment for processing time bundle
correctly. (#36384)
add 312509fd019 Add back missing licenses (#36386)
No new revisions were added by this update.
Summary of changes:
.github/workflows/README.md | 9 +-
....yml => beam_PostCommit_Java_Jpms_Dataflow.yml} | 16 +-
...eam_PostCommit_Java_Jpms_Dataflow_Versions.yml} | 23 +-
...11.yml => beam_PostCommit_Java_Jpms_Direct.yml} | 17 +-
.../beam_PostCommit_Java_Jpms_Direct_Java21.yml | 101 ----
... beam_PostCommit_Java_Jpms_Direct_Versions.yml} | 23 +-
.github/workflows/beam_PostCommit_Python.yml | 2 +-
.github/workflows/beam_PostCommit_Python_Arm.yml | 2 +-
.../beam_PostCommit_Python_Dependency.yml | 2 +-
.../beam_PostCommit_Python_Examples_Dataflow.yml | 2 +-
.../beam_PostCommit_Python_Examples_Direct.yml | 2 +-
.../beam_PostCommit_Python_Examples_Flink.yml | 2 +-
.../beam_PostCommit_Python_Examples_Spark.yml | 2 +-
.../beam_PostCommit_Python_MongoDBIO_IT.yml | 2 +-
...stCommit_Python_ValidatesContainer_Dataflow.yml | 2 +-
..._Python_ValidatesContainer_Dataflow_With_RC.yml | 2 +-
..._PostCommit_Python_ValidatesRunner_Dataflow.yml | 2 +-
...eam_PostCommit_Python_ValidatesRunner_Flink.yml | 2 +-
...eam_PostCommit_Python_ValidatesRunner_Samza.yml | 2 +-
...eam_PostCommit_Python_ValidatesRunner_Spark.yml | 2 +-
.../beam_PostCommit_Python_Xlang_Gcp_Dataflow.yml | 2 +-
.../beam_PostCommit_Python_Xlang_Gcp_Direct.yml | 2 +-
.../beam_PostCommit_Python_Xlang_IO_Dataflow.yml | 2 +-
.../beam_PostCommit_Python_Xlang_IO_Direct.yml | 2 +-
.../beam_PostCommit_TransformService_Direct.yml | 2 +-
.github/workflows/beam_PostCommit_XVR_Direct.yml | 2 +-
.github/workflows/beam_PostCommit_XVR_Flink.yml | 2 +-
...eam_PostCommit_XVR_JavaUsingPython_Dataflow.yml | 2 +-
..._PostCommit_XVR_PythonUsingJavaSQL_Dataflow.yml | 2 +-
...eam_PostCommit_XVR_PythonUsingJava_Dataflow.yml | 2 +-
.github/workflows/beam_PostCommit_XVR_Samza.yml | 2 +-
.github/workflows/beam_PostCommit_XVR_Spark3.yml | 2 +-
.github/workflows/beam_PreCommit_Prism_Python.yml | 2 +-
.github/workflows/beam_PreCommit_Python.yml | 2 +-
.github/workflows/beam_PreCommit_PythonDocker.yml | 2 +-
.../workflows/beam_PreCommit_Python_Coverage.yml | 4 +-
.../workflows/beam_PreCommit_Python_Dataframes.yml | 2 +-
.../workflows/beam_PreCommit_Python_Examples.yml | 2 +-
.../beam_PreCommit_Python_Integration.yml | 2 +-
.github/workflows/beam_PreCommit_Python_ML.yml | 6 +-
.../workflows/beam_PreCommit_Python_PVR_Flink.yml | 2 +-
.../workflows/beam_PreCommit_Python_Runners.yml | 2 +-
.../workflows/beam_PreCommit_Python_Transforms.yml | 2 +-
.../workflows/beam_Publish_Beam_SDK_Snapshots.yml | 2 +
...beam_Python_ValidatesContainer_Dataflow_ARM.yml | 2 +-
.github/workflows/build_wheels.yml | 6 +-
.github/workflows/python_dependency_tests.yml | 1 +
.github/workflows/python_tests.yml | 3 +-
CHANGES.md | 1 +
build.gradle.kts | 3 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 16 +-
runners/google-cloud-dataflow-java/build.gradle | 4 +-
.../options/DataflowStreamingPipelineOptions.java | 3 +
.../dataflow/worker/StreamingDataflowWorker.java | 512 +++++++++++++++------
.../harness/SingleSourceWorkerHarness.java | 13 +-
.../dataflow/worker/FakeWindmillServer.java | 47 ++
.../worker/StreamingDataflowWorkerTest.java | 148 ++++++
.../worker/windmill/src/main/proto/windmill.proto | 9 +
runners/prism/java/build.gradle | 15 +-
sdks/go.mod | 2 +-
sdks/go.sum | 4 +-
.../prism/internal/engine/elementmanager.go | 54 ++-
.../beam/runners/prism/internal/engine/strategy.go | 38 +-
sdks/go/pkg/beam/runners/prism/internal/execute.go | 4 +-
.../prism/internal/jobservices/management.go | 15 +-
.../runners/prism/internal/unimplemented_test.go | 5 +-
.../extensions/protobuf/ProtoBeamConverter.java | 9 +-
.../beam/sdk/extensions/protobuf/ProtobufUtil.java | 18 -
sdks/python/apache_beam/internal/metrics/cells.py | 21 +-
.../apache_beam/internal/metrics/metric_test.py | 39 ++
sdks/python/apache_beam/metrics/execution.py | 12 +-
sdks/python/apache_beam/metrics/metric.py | 1 +
.../python/apache_beam/metrics/monitoring_infos.py | 39 +-
.../apache_beam/metrics/monitoring_infos_test.py | 35 ++
sdks/python/apache_beam/ml/inference/base_test.py | 4 +-
.../python/apache_beam/options/pipeline_options.py | 15 +
sdks/python/apache_beam/pipeline.py | 33 +-
.../iqr_test.py => programming_guide_test.py} | 44 +-
.../apache_beam/runners/dataflow/internal/names.py | 2 +-
.../apache_beam/runners/direct/direct_metrics.py | 33 ++
.../testing/integration/tests/screen_diff_test.py | 2 -
sdks/python/apache_beam/transforms/core.py | 21 +-
sdks/python/apache_beam/transforms/util.py | 54 ++-
sdks/python/apache_beam/transforms/util_test.py | 77 +++-
sdks/python/apache_beam/utils/histogram.py | 72 ++-
sdks/python/apache_beam/utils/subprocess_server.py | 69 ++-
.../apache_beam/utils/subprocess_server_test.py | 50 ++
sdks/python/build.gradle | 2 +-
.../ml_image_requirements.txt} | 126 +++--
.../container/py310/base_image_requirements.txt | 91 ++--
.../container/py311/base_image_requirements.txt | 93 ++--
.../container/py312/base_image_requirements.txt | 93 ++--
.../container/py313/base_image_requirements.txt | 100 ++--
.../container/py39/base_image_requirements.txt | 81 ++--
sdks/python/pyproject.toml | 2 +-
sdks/python/setup.py | 74 ++-
sdks/python/test-suites/gradle.properties | 28 +-
sdks/python/tox.ini | 12 +
.../content/en/documentation/programming-guide.md | 16 +-
99 files changed, 1702 insertions(+), 842 deletions(-)
rename .github/workflows/{beam_PostCommit_Java_Jpms_Dataflow_Java11.yml =>
beam_PostCommit_Java_Jpms_Dataflow.yml} (88%)
rename .github/workflows/{beam_PostCommit_Java_Jpms_Dataflow_Java17.yml =>
beam_PostCommit_Java_Jpms_Dataflow_Versions.yml} (83%)
rename .github/workflows/{beam_PostCommit_Java_Jpms_Direct_Java11.yml =>
beam_PostCommit_Java_Jpms_Direct.yml} (87%)
delete mode 100644
.github/workflows/beam_PostCommit_Java_Jpms_Direct_Java21.yml
rename .github/workflows/{beam_PostCommit_Java_Jpms_Direct_Java17.yml =>
beam_PostCommit_Java_Jpms_Direct_Versions.yml} (83%)
copy sdks/python/apache_beam/{ml/anomaly/detectors/iqr_test.py =>
programming_guide_test.py} (50%)
copy sdks/python/container/ml/{py312/base_image_requirements.txt =>
py313/ml_image_requirements.txt} (75%)
