This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a change to branch users/damccorm/test-revert
in repository https://gitbox.apache.org/repos/asf/beam.git
from 07d7dc7bd6f Merge branch 'master' of https://github.com/apache/beam
into users/damccorm/test-revert
add b60ce5be96b Add type checking to confirm that the flatten inputs are
actually pco… (#35874)
add c0199f8a5b7 fix(yaml): handle grpc channel serialization in yaml
testing framework (#35882)
add 8275cc69fa4 [Java] Throw an exception for invalid empty pubsub
messages before attempting to publish. (#35865)
add b17488ffddb Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks
(#35878)
add 42dfd51545f [#35869] [Go SDK] Fix Duplicated elements on Blind Writes
to Bag State. (#35870)
add 2b4ca6922ce Update batching test to use FnApiRunner (#35895)
add e591ef3dc1f YAML A Batch ML Pipeline Example (#35810)
add 37e3bf971f6 Fix pymilvus dependency
add 8f888fb171a Fix Python Arm workflows (#35900)
new f11ebca7695 Merge branch 'master' of https://github.com/apache/beam
into users/damccorm/test-revert
new b2d8391fa43 Update test code
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGES.md | 2 +
sdks/go.mod | 16 +--
sdks/go.sum | 32 +++---
sdks/go/pkg/beam/core/runtime/exec/userstate.go | 67 ++++++-----
sdks/go/test/integration/primitives/state.go | 40 +++++++
sdks/go/test/integration/primitives/state_test.go | 5 +
.../sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java | 11 +-
.../apache/beam/sdk/io/gcp/pubsub/PubsubIO.java | 2 +-
.../io/gcp/pubsub/PreparePubsubWriteDoFnTest.java | 78 ++++++-------
sdks/python/apache_beam/ml/inference/base_test.py | 2 +-
sdks/python/apache_beam/transforms/core.py | 9 ++
.../python/apache_beam/typehints/typecheck_test.py | 9 ++
sdks/python/apache_beam/yaml/examples/README.md | 5 +-
.../yaml/examples/testing/examples_test.py | 125 +++++++++++++++++++--
.../yaml/examples/testing/input_data.py | 43 +++++++
.../examples/transforms/ml/log_analysis/README.md | 98 ++++++++++++++++
.../ml/log_analysis/anomaly_scoring.yaml | 93 +++++++++++++++
.../ml/log_analysis/batch_log_analysis.sh | 108 ++++++++++++++++++
.../ml/log_analysis/iceberg_migration.yaml | 45 ++++++++
.../ml/log_analysis/ml_preprocessing.yaml | 124 ++++++++++++++++++++
.../transforms/ml/log_analysis}/requirements.txt | 8 +-
.../examples/transforms/ml/log_analysis/train.py | 89 +++++++++++++++
sdks/python/apache_beam/yaml/yaml_testing.py | 36 ++++--
sdks/python/apache_beam/yaml/yaml_testing_test.py | 99 +++++++++++++++-
.../container/py310/base_image_requirements.txt | 4 +-
.../container/py310/ml_image_requirements.txt | 4 +-
.../container/py311/base_image_requirements.txt | 4 +-
.../container/py311/ml_image_requirements.txt | 4 +-
.../container/py312/base_image_requirements.txt | 4 +-
.../container/py312/ml_image_requirements.txt | 4 +-
.../container/py313/base_image_requirements.txt | 4 +-
.../container/py39/base_image_requirements.txt | 4 +-
.../container/py39/ml_image_requirements.txt | 4 +-
sdks/python/setup.py | 11 +-
34 files changed, 1063 insertions(+), 130 deletions(-)
create mode 100644
sdks/python/apache_beam/yaml/examples/transforms/ml/log_analysis/README.md
create mode 100644
sdks/python/apache_beam/yaml/examples/transforms/ml/log_analysis/anomaly_scoring.yaml
create mode 100755
sdks/python/apache_beam/yaml/examples/transforms/ml/log_analysis/batch_log_analysis.sh
create mode 100644
sdks/python/apache_beam/yaml/examples/transforms/ml/log_analysis/iceberg_migration.yaml
create mode 100644
sdks/python/apache_beam/yaml/examples/transforms/ml/log_analysis/ml_preprocessing.yaml
copy sdks/python/apache_beam/{testing/benchmarks/wordcount =>
yaml/examples/transforms/ml/log_analysis}/requirements.txt (73%)
create mode 100644
sdks/python/apache_beam/yaml/examples/transforms/ml/log_analysis/train.py
