This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.17.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard ec49b84e3b7 Bump github.com/testcontainers/testcontainers-go in /sdks
add 177306792ce Better naming and inference for PythonCallable. (#24735)
add 208a14e05b5 [Spark Dataset runner] Reuse SparkSession when testing
using class rule. (#24812)
add 7c736e27b4f [#24515] Ensure that portable Java pipelines on Dataflow
are not able to opt out of runner v2. (#24805)
add 5b80a8141ff 24802 fix missing test resources (#24803)
add c8bf5d55fd1 Add Python xlang KafkaIO performance test (#24633)
add 598324a8847 Shard python precommit (#24204)
add ea6cf404b9f Implement sibling protocol for Typescript.
add c5c5b35c403 Merge pull request #23975 Implement sibling protocol for
Typescript.
add 80de3ef96d7 Refactor integration tests to use internal containers
package (#24823)
add ba3dcd1cb98 [#24789][Go SDK] Fix Minor race conditions (#24808)
add c38074ac024 Various cleanups to the typescript SDK: * Better errors
for bad serialization imports. * Semver-correct version for fake worker
package. * Package source into temporary directory. * Allow "latest" as a
xlang beam jar version for non-released SDKs. * Less verbose logging.
add 4f96e3594ef Add timeout to cross-language tests.
add 3f28d55dbe1 Merge pull request #24825 Various cleanups to the
typescript SDK.
add 0bba43969f6 [#24789] Spot fix fullvalue wrapping for SDF. (#24826)
add 5ab768bc6c1 Add gradle files for typescript containers and update the
version info.
add 5911ea20fc0 Fix issue with typescript sibling sdks.
add a67fc244f97 npm install before build
add 3bd5f780e83 Format go file.
add 889fd8bcdc9 Merge pull request #24827 Add gradle files for typescript
containers and update the version info.
add ad72baceaef Fix Cassandra read bug when user query has no where clause
(fixes #24829) (#24830)
add 3ee5b86a3f1 Retry create database in InfluxDbIOIT (#24800)
add 587e41eccef [Website] update links from absolute to relative in md
files
add 0f423dd6cc1 Merge pull request #24747: [Website] update links from
absolute to relative in md files
add 0d835cdc72f Support SqlTypes Date in AvroUtils (sync)
add 37358ff0196 Merge pull request #24854: Support SqlTypes Date in
AvroUtils (sync)
add b802c023b27 Revert "Shard python precommit (#24204)" (#24855)
add a4cf26fdc99 [Spark Dataset runner] Fix initialization of metrics
accumulator on driver (fixes #24809) (#24810)
add 1b1acb08baf Improve error message (#24843)
add 1fa97825e62 Playground Example CI fix for 24807 (#24819)
add 803bfa6e2b1 Bump github.com/tetratelabs/wazero from 1.0.0-pre.4 to
1.0.0-pre.7 in /sdks (#24858)
add 3b9113f3fa9 Bump github.com/testcontainers/testcontainers-go in /sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ec49b84e3b7)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.17.0
(3b9113f3fa9)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/typescript_tests.yml | 1 +
.test-infra/jenkins/README.md | 7 +-
.../jenkins/job_PerformanceTests_KafkaIO_IT.groovy | 8 +-
...ob_PerformanceTests_xlang_KafkaIO_Python.groovy | 123 ++++++++++++++++++
...vy => job_PostCommit_Java_InfluxDBIO_IT.groovy} | 15 ++-
.../Java_IO_IT_Tests_Dataflow.json | 70 ++++++++++-
.../Python_IO_IT_Tests_Dataflow.json | 122 ++++++++++++++++++
CHANGES.md | 11 +-
build.gradle.kts | 3 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 3 +-
playground/infrastructure/checker.py | 7 +-
playground/infrastructure/test_checker.py | 19 ++-
release/src/main/scripts/set_version.sh | 2 +
.../beam/runners/dataflow/DataflowRunner.java | 21 +++-
.../beam/runners/dataflow/DataflowRunnerTest.java | 57 +++++++++
.../SparkStructuredStreamingPipelineResult.java | 9 +-
.../SparkStructuredStreamingRunner.java | 37 +++---
.../metrics/MetricsAccumulator.java | 117 ++++++++++++-----
.../MetricsContainerStepMapAccumulator.java | 65 ----------
.../metrics/SparkBeamMetric.java | 12 +-
.../metrics/SparkBeamMetricSource.java | 4 +-
.../metrics/SparkMetricsContainerStepMap.java | 43 -------
.../batch/DoFnPartitionIteratorFactory.java | 26 ++--
.../translation/batch/DoFnRunnerWithMetrics.java | 31 ++---
.../translation/batch/ParDoTranslatorBatch.java | 5 +-
.../translation/batch/CombineGloballyTest.java | 18 +--
.../batch/CombineGroupedValuesTest.java | 18 +--
.../translation/batch/CombinePerKeyTest.java | 18 +--
.../translation/batch/ComplexSourceTest.java | 17 +--
.../translation/batch/FlattenTest.java | 24 ++--
.../translation/batch/GroupByKeyTest.java | 18 +--
.../translation/batch/SimpleSourceTest.java | 18 +--
.../translation/batch/WindowAssignTest.java | 18 +--
sdks/go.mod | 2 +-
sdks/go.sum | 4 +-
sdks/go/examples/wasm/wasm.go | 11 +-
sdks/go/pkg/beam/core/runtime/exec/sdf.go | 43 ++++---
sdks/go/pkg/beam/log/log.go | 19 ++-
.../beam/runners/universal/extworker/extworker.go | 19 ++-
.../integration/internal/containers/containers.go | 45 ++++++-
.../test/integration/io/mongodbio/helper_test.go | 6 +-
.../integration/io/xlang/debezium/debezium_test.go | 44 +++----
.../go/test/integration/io/xlang/jdbc/jdbc_test.go | 75 +++++------
.../extensions/avro/schemas/utils/AvroUtils.java | 10 +-
.../avro/schemas/utils/AvroUtilsTest.java | 20 +++
.../beam/sdk/bigqueryioperftests/BigQueryIOIT.java | 4 +-
.../org/apache/beam/sdk/io/cassandra/ReadFn.java | 5 +-
.../beam/sdk/io/cassandra/CassandraIOTest.java | 41 +++++-
.../beam/sdk/io/gcp/pubsub/PubsubClient.java | 4 +-
.../beam/sdk/io/gcp/pubsub/PubsubGrpcClient.java | 18 +--
.../beam/sdk/io/gcp/pubsub/PubsubJsonClient.java | 4 +-
.../beam/sdk/io/gcp/pubsub/PubsubTestClient.java | 2 +-
.../beam/sdk/io/gcp/pubsub/PubsubSchemaIT.java | 53 +++++++-
.../io/gcp/pubsub/all_data_types_flat_schema.proto | 27 ----
.../pubsub/avro_all_data_types_flat_schema.json | 30 -----
.../sdk/io/hadoop/format/HadoopFormatIOIT.java | 4 +-
.../apache/beam/sdk/io/influxdb/InfluxDbIOIT.java | 24 +++-
.../java/org/apache/beam/sdk/io/jdbc/JdbcIOIT.java | 4 +-
.../org/apache/beam/sdk/io/kafka/KafkaIOIT.java | 35 ++++--
.../io/external/xlang_kafkaio_perf_test.py | 140 +++++++++++++++++++++
.../python/apache_beam/io/filebasedio_perf_test.py | 2 +
.../testing/load_tests/load_test_metrics_utils.py | 32 ++++-
sdks/python/apache_beam/transforms/util.py | 1 +
.../apache_beam/typehints/trivial_inference.py | 11 +-
.../typehints/trivial_inference_test.py | 7 ++
sdks/python/apache_beam/utils/python_callable.py | 13 ++
.../apache_beam/utils/python_callable_test.py | 9 ++
sdks/typescript/build.gradle | 67 ++++++++++
sdks/typescript/build.sh | 5 +
sdks/typescript/build_container.sh | 24 ----
sdks/typescript/{ => container}/Dockerfile | 9 +-
sdks/typescript/{ => container}/boot.go | 16 ++-
.../container/build.gradle} | 60 +++------
sdks/typescript/package-lock.json | 11 +-
sdks/typescript/package.json | 6 +-
.../src/apache_beam/internal/environments.ts | 8 +-
.../typescript/src/apache_beam/runners/dataflow.ts | 4 +-
sdks/typescript/src/apache_beam/runners/flink.ts | 2 +-
.../apache_beam/runners/portable_runner/runner.ts | 23 +++-
sdks/typescript/src/apache_beam/utils/service.ts | 56 +++++++--
sdks/typescript/src/apache_beam/version.ts | 26 ----
sdks/typescript/src/apache_beam/worker/data.ts | 1 -
.../typescript/src/apache_beam/worker/operators.ts | 1 -
.../src/apache_beam/worker/worker_main.ts | 16 ++-
settings.gradle.kts | 2 +
.../www/site/content/en/blog/ApachePlayground.md | 8 +-
.../content/en/blog/adding-data-sources-to-sql.md | 2 +-
website/www/site/content/en/blog/beam-2.21.0.md | 4 +-
website/www/site/content/en/blog/beam-2.25.0.md | 4 +-
website/www/site/content/en/blog/beam-2.32.0.md | 6 +-
website/www/site/content/en/blog/beam-2.38.0.md | 2 +-
website/www/site/content/en/blog/beam-2.42.0.md | 2 +-
website/www/site/content/en/blog/beam-2.8.0.md | 2 +-
.../content/en/blog/beam-katas-kotlin-release.md | 4 +-
.../content/en/blog/beam-sql-with-notebooks.md | 8 +-
.../site/content/en/blog/beam-starter-projects.md | 2 +-
.../content/en/blog/beam-summit-europe-2019.md | 4 +-
.../en/blog/dataframe-api-preview-available.md | 6 +-
website/www/site/content/en/blog/go-2.40.md | 12 +-
website/www/site/content/en/blog/gsoc-19.md | 4 +-
website/www/site/content/en/blog/hop-web-cloud.md | 6 +-
.../content/en/blog/kafka-to-pubsub-example.md | 6 +-
website/www/site/content/en/blog/ml-resources.md | 8 +-
.../content/en/blog/python-improved-annotations.md | 4 +-
.../python-performance-runtime-type-checking.md | 2 +-
website/www/site/content/en/blog/python-typing.md | 4 +-
website/www/site/content/en/blog/season-of-docs.md | 6 +-
.../en/blog/splittable-do-fn-is-available.md | 4 +-
website/www/site/content/en/case-studies/hop.md | 2 +-
.../www/site/content/en/case-studies/paloalto.md | 14 +--
.../www/site/content/en/case-studies/ricardo.md | 4 +-
website/www/site/content/en/case-studies/seznam.md | 8 +-
.../www/site/content/en/case-studies/snowflake.md | 2 +-
website/www/site/content/en/community/in-person.md | 2 +-
website/www/site/content/en/contribute/_index.md | 16 +--
.../en/contribute/get-started-contributing.md | 34 ++---
.../site/content/en/contribute/release-guide.md | 4 +-
.../dsls/dataframes/differences-from-pandas.md | 8 +-
.../en/documentation/dsls/dataframes/overview.md | 8 +-
.../dsls/sql/extensions/create-external-table.md | 2 +-
.../en/documentation/io/built-in/snowflake.md | 16 +--
.../content/en/documentation/ml/data-processing.md | 4 +-
.../en/documentation/ml/large-language-modeling.md | 2 +-
.../en/documentation/ml/multi-model-pipelines.md | 6 +-
.../en/documentation/ml/online-clustering.md | 4 +-
.../content/en/documentation/ml/orchestration.md | 2 +-
.../site/content/en/documentation/ml/overview.md | 6 +-
.../en/documentation/ml/runinference-metrics.md | 2 +-
.../site/content/en/documentation/patterns/bqml.md | 2 +-
...lements-for-efficient-external-service-calls.md | 2 +-
.../content/en/documentation/programming-guide.md | 6 +-
.../content/en/documentation/runners/direct.md | 2 +-
.../site/content/en/documentation/runners/spark.md | 2 +-
.../site/content/en/documentation/runtime/model.md | 2 +-
.../sdks/java-multi-language-pipelines.md | 4 +-
.../documentation/sdks/python-machine-learning.md | 4 +-
.../sdks/python-pipeline-dependencies.md | 2 +-
.../en/documentation/sdks/python-streaming.md | 4 +-
.../java/aggregation/approximatequantiles.md | 2 +-
.../java/aggregation/approximateunique.md | 2 +-
.../transforms/java/aggregation/cogroupbykey.md | 2 +-
.../transforms/java/aggregation/combine.md | 2 +-
.../java/aggregation/combinewithcontext.md | 2 +-
.../transforms/java/aggregation/count.md | 2 +-
.../transforms/java/aggregation/distinct.md | 2 +-
.../transforms/java/aggregation/groupbykey.md | 2 +-
.../java/aggregation/groupintobatches.md | 2 +-
.../transforms/java/aggregation/hllcount.md | 2 +-
.../transforms/java/aggregation/latest.md | 2 +-
.../transforms/java/aggregation/max.md | 2 +-
.../transforms/java/aggregation/mean.md | 2 +-
.../transforms/java/aggregation/min.md | 2 +-
.../transforms/java/aggregation/sample.md | 2 +-
.../transforms/java/aggregation/sum.md | 2 +-
.../transforms/java/aggregation/top.md | 2 +-
.../transforms/java/elementwise/filter.md | 2 +-
.../transforms/java/elementwise/flatmapelements.md | 2 +-
.../transforms/java/elementwise/keys.md | 2 +-
.../transforms/java/elementwise/kvswap.md | 2 +-
.../transforms/java/elementwise/mapelements.md | 2 +-
.../transforms/java/elementwise/pardo.md | 2 +-
.../transforms/java/elementwise/partition.md | 2 +-
.../transforms/java/elementwise/regex.md | 2 +-
.../transforms/java/elementwise/reify.md | 2 +-
.../transforms/java/elementwise/tostring.md | 2 +-
.../transforms/java/elementwise/values.md | 2 +-
.../transforms/java/elementwise/withkeys.md | 2 +-
.../transforms/java/elementwise/withtimestamps.md | 2 +-
.../documentation/transforms/java/other/create.md | 2 +-
.../documentation/transforms/java/other/flatten.md | 2 +-
.../documentation/transforms/java/other/passert.md | 2 +-
.../en/documentation/transforms/java/other/view.md | 2 +-
.../documentation/transforms/java/other/window.md | 2 +-
.../transforms/python/elementwise/pardo.md | 2 +-
.../transforms/python/other/create.md | 2 +-
.../transforms/python/other/flatten.md | 2 +-
.../transforms/python/other/reshuffle.md | 2 +-
.../transforms/python/other/windowinto.md | 2 +-
.../www/site/content/en/get-started/from-spark.md | 2 +-
.../site/content/en/get-started/quickstart/java.md | 4 +-
.../en/get-started/resources/learning-resources.md | 28 ++---
.../site/content/en/get-started/tour-of-beam.md | 2 +-
.../content/en/get-started/wordcount-example.md | 2 +-
.../content/en/roadmap/connectors-multi-sdk.md | 4 +-
184 files changed, 1609 insertions(+), 871 deletions(-)
create mode 100644
.test-infra/jenkins/job_PerformanceTests_xlang_KafkaIO_Python.groovy
rename .test-infra/jenkins/{job_PerformanceTests_InfluxDBIO_IT.groovy =>
job_PostCommit_Java_InfluxDBIO_IT.groovy} (83%)
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/MetricsContainerStepMapAccumulator.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkMetricsContainerStepMap.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/resources/org/apache/beam/sdk/io/gcp/pubsub/all_data_types_flat_schema.proto
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/resources/org/apache/beam/sdk/io/gcp/pubsub/avro_all_data_types_flat_schema.json
create mode 100644
sdks/python/apache_beam/io/external/xlang_kafkaio_perf_test.py
create mode 100644 sdks/typescript/build.gradle
delete mode 100755 sdks/typescript/build_container.sh
rename sdks/typescript/{ => container}/Dockerfile (86%)
rename sdks/typescript/{ => container}/boot.go (92%)
copy sdks/{python/container/common.gradle =>
typescript/container/build.gradle} (52%)
delete mode 100644 sdks/typescript/src/apache_beam/version.ts
