This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/gradle/com.gradle.common-custom-user-data-gradle-plugin-2.0.1
in repository https://gitbox.apache.org/repos/asf/beam.git
discard d6690fae8ff Bump com.gradle.common-custom-user-data-gradle-plugin
add bcb40cf4e4a Change caching of global window inputs to be guarded by
experiment (#31013)
add 2eb1a756258 [Python] Clean doc related to write data in bigquery.py
(#30887)
add 04ff4bdd7fc Support BQ clustering with value provider (#30460)
add 61153bbda6a Update documentation of @SchemaFieldNumber (#30273)
(#30277)
add bb0b63cb940 Bump TPCDS test Flink version (#31041)
add 70e067e1fde fix url for content security (#31043)
add bb310e7e907 Change type for UnboundedReaderMaxReadTimeSec (#31037)
add b69e8c615af Updates Python Dev container used by Dataflow (#31029)
add 76c77cd28ae Fix typo in count_unique_words() (#31023)
add 3e52e3554a0 Add code change guide contributor-doc (#30879)
add d05196dea8a Upgrade the version of GRPC to pick up a fix for #30867
(#31044)
add fd1003034ff Add ScaleToGaussian TFTransform (#31026)
add 98b1f03125c Better and earlier error for missing service loader
metadata. (#31042)
add 40ec29ad289 fix website embedding issue (#31019)
add 31f1c7968d4 fix url for content security
add 93aa62c2900 Merge pull request #31058 from svetakvsundhar/broken_links
add 49da7a34d99 Bump golang.org/x/net from 0.22.0 to 0.23.0 in /sdks
(#31054)
add 516ac97659f fix examples on website (#31066)
add 1a5dc1c7fab Replace Source type for BigTable and Kafka stress tests
(#30999)
add 8dcbf967a1a [#30083][prism] Stabilize additional teststream cases.
(#31046)
add 2472695eb04 filter out outdated workflow runs (#31071)
add f8b81211049 Fix test failure due to locale sensitivity. (#31069)
add 718af0d7652 Don't use isolated builds when building an sdist for a
pipeline package supplied in --setup_file option. (#31070)
add 5a86b3fc922 Bump golang.org/x/net from 0.17.0 to 0.23.0 in
/learning/katas/go (#31050)
add 3dbbad98c80 Bump golang.org/x/net from 0.17.0 to 0.23.0 in
/.test-infra/mock-apis (#31051)
add a0dad088980 Bump golang.org/x/net from 0.22.0 to 0.24.0 in /sdks
(#31065)
add 37609ba70fa Managed Transform protos & translation; Iceberg
SchemaTransforms & translation (#30910)
add de665b77d9f [YAML] - Normalize YAML PubSub format (#31068)
add ba7ea856ffe Revert "Change type for UnboundedReaderMaxReadTimeSec
(#31037)" (#31082)
add 58bea3f19e5 Fix building release candidate (#31084)
add ab27f6f0358 Refactor Residuals to prepare for processing time
scheduling. (#31057)
add 1ebd5039a97 Simplify intermediate data in Iceberg sink; use manifest
files
add 485c5198384 Merge pull request #31086: Simplify intermediate data in
Iceberg sink; use manifest files
add 45e78572e8f python sdk: fix several bugs regarding avto <-> beam
schema conversion (#30770)
add 5cce8d79925 Add usage counters to ListView side inputs.
add 83a90f2bbb4 Merge pull request #31083 Add usage counters to ListView
side inputs.
add 495e9b2eb7e Python package string change (#31093)
add 8454cc95030 Change assertion and timeout for kafka stress test (#31089)
add 2db9b8097b7 Mention known issues in release notes. (#31088)
add e7847998a44 use strimzi kafka on beam_utility cluster (#30967)
add 0f69dc19a5d remove cloudsdk environment variable, not needed here
(#31102)
add 3d3669e2bbc Use java 11 for java_tests workflow (#31101)
add 3c8a8815c00 Add ability to load multiple copies of a model across
processes (#31052)
add 088c854d7a3 Fix :examples:java:twitter:execute task (#31103)
add b33a8438ad3 Revert global snake_case convention for SchemaTransforms
(#31109)
add 15f7f02a79d Add UnboundedReaderMaxReadTimeMs to
DataflowPipelineDebugOptions, deprecate UnboundedReaderMaxReadTimeSec (#31091)
add 5c3786e3eb0 Try fixing build wheels (#31115)
add 3e5a658498e fix test added in previous commit (#31116)
add c20e329a326 Fix reporting metrics not supported warning for BigQueryIO
Direct read (#31096)
add 3329edba79f [Python] Add feast feature store handler for enrichment
transform (#30957)
add 68f6b551541 One more macos change (#31123)
add 28a2682d54c add terraform for utility cluster. Add name override to
gke (#30847)
add 673da546c14 [#30083][prism] Factor out hold tracking to dedicated
structures (#31105)
add 08fc33b26cc Touch trigger files (add some comments as well)
add 7ade06f6f77 Add Redistribute to Beam model protos
add f49ac7d10a0 Add Redistribute transform to Java SDK
add 18ebc454928 Add direct Dataflow translation of Redistribute transform
add 21e3fa1eedf Add Redistribute translation to Spark runner
add 8f1d3da4399 Add Redistribute translation to Samza runner
add bdfc7812fdf Add Redistribute translation to FlinkRunner
add 6fa6a28b397 Merge pull request #30545: Add Redistribute transform to
model, Java SDK, and most active runners
add 61ca405d372 Implementing lull reporting at bundle level processing
(#30693)
add b00958af706 rename cluster used (#31113)
add 0696d00d09b Add ReadChangeStream IO param to adjust backlog estimates
for replication delay (#30995)
add 970109e0553 Also allow links to Drive materials. (#31131)
add f0d06051072 add finalizers to kafka loadbalancer services (#31130)
add 413af128937 Allow users to configure wait options for new Neo4j
databases (#31129)
add 22535521e7b Bump com.gradle.enterprise from 3.15.1 to 3.17.2
add e3b4d41f6b8 Merge pull request #31032: Bump com.gradle.enterprise from
3.15.1 to 3.17.2
add 885958952da Bump com.gradle.common-custom-user-data-gradle-plugin
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (d6690fae8ff)
\
N -- N -- N
refs/heads/dependabot/gradle/com.gradle.common-custom-user-data-gradle-plugin-2.0.1
(885958952da)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
..._PostCommit_Java_ValidatesRunner_Dataflow.json} | 0
...t_Java_ValidatesRunner_Dataflow_Streaming.json} | 0
...eam_PostCommit_Java_ValidatesRunner_Flink.json} | 0
...eam_PostCommit_Java_ValidatesRunner_Samza.json} | 0
...eam_PostCommit_Java_ValidatesRunner_Spark.json} | 0
.github/trigger_files/beam_PostCommit_Python.json | 1 -
.../beam_PostCommit_Python_Xlang_Gcp_Direct.json | 3 +-
.../beam_PostCommit_Python_Xlang_IO_Dataflow.json | 3 +-
.../beam_PerformanceTests_xlang_KafkaIO_Python.yml | 31 +-
.../workflows/beam_PostCommit_Java_Tpcds_Flink.yml | 2 +-
.github/workflows/beam_PostCommit_Python_Arm.yml | 1 -
.../beam_PreCommit_Xlang_Generated_Transforms.yml | 2 +-
.../workflows/beam_StressTests_Java_KafkaIO.yml | 2 +-
.github/workflows/build_release_candidate.yml | 10 +-
.github/workflows/build_wheels.yml | 8 +-
.../workflows/deploy_release_candidate_pypi.yaml | 4 +-
.github/workflows/java_tests.yml | 6 +-
.gitignore | 11 +-
.../01-strimzi-operator/.terraform.lock.hcl | 21 +
.../kafka/strimzi/01-strimzi-operator/README.md | 28 -
.../strimzi/01-strimzi-operator/common.tfvars | 8 +-
.../strimzi/01-strimzi-operator/kafka.tf} | 20 +-
.../strimzi/01-strimzi-operator/kustomization.yaml | 23 -
.../strimzi/01-strimzi-operator/namespace.yaml | 22 -
.../kafka/strimzi/01-strimzi-operator/provider.tf | 9 +-
...-ClusterRole-strimzi-cluster-operator-role.yaml | 170 -
.../020-RoleBinding-strimzi-cluster-operator.yaml | 31 -
...-ClusterRole-strimzi-cluster-operator-role.yaml | 55 -
...lusterRoleBinding-strimzi-cluster-operator.yaml | 31 -
...-ClusterRole-strimzi-cluster-operator-role.yaml | 48 -
.../022-RoleBinding-strimzi-cluster-operator.yaml | 31 -
...-ClusterRole-strimzi-cluster-operator-role.yaml | 82 -
.../023-RoleBinding-strimzi-cluster-operator.yaml | 31 -
.../030-ClusterRole-strimzi-kafka-broker.yaml | 32 -
...i-cluster-operator-kafka-broker-delegation.yaml | 33 -
.../031-ClusterRole-strimzi-entity-operator.yaml | 61 -
...luster-operator-entity-operator-delegation.yaml | 33 -
.../033-ClusterRole-strimzi-kafka-client.yaml | 33 -
...i-cluster-operator-kafka-client-delegation.yaml | 34 -
.../01-strimzi-operator/v0.33.2/040-Crd-kafka.yaml | 6208 --------------------
.../v0.33.2/041-Crd-kafkaconnect.yaml | 1952 ------
.../v0.33.2/042-Crd-strimzipodset.yaml | 135 -
.../v0.33.2/043-Crd-kafkatopic.yaml | 270 -
.../v0.33.2/044-Crd-kafkauser.yaml | 702 ---
.../v0.33.2/045-Crd-kafkamirrormaker.yaml | 1228 ----
.../v0.33.2/046-Crd-kafkabridge.yaml | 1178 ----
.../v0.33.2/047-Crd-kafkaconnector.yaml | 146 -
.../v0.33.2/048-Crd-kafkamirrormaker2.yaml | 1992 -------
.../v0.33.2/049-Crd-kafkarebalance.yaml | 159 -
.../050-ConfigMap-strimzi-cluster-operator.yaml | 52 -
.../060-Deployment-strimzi-cluster-operator.yaml | 155 -
.../01-strimzi-operator/v0.33.2/kustomization.yaml | 46 -
.../strimzi/01-strimzi-operator}/variables.tf | 24 +-
.../kafka/strimzi/02-kafka-persistent/README.md | 17 +-
.../base/v0.33.2/kafka-persistent.yaml | 14 +-
.../gke-internal-load-balanced/kustomization.yaml | 8 +-
.../gke-internal-load-balanced/listeners.yaml | 3 +
.test-infra/kafka/strimzi/README.md | 133 +-
.../github/github_runs_prefetcher/code/main.py | 6 +-
.test-infra/mock-apis/go.mod | 6 +-
.test-infra/mock-apis/go.sum | 12 +-
.../.kafka-workflows.apache-beam-testing.tfbackend | 4 +-
.../google-kubernetes-engine/.terraform.lock.hcl | 42 +
.../google-kubernetes-engine/README.md | 84 +-
.../google-kubernetes-engine/cluster.tf | 15 +-
.../copyme.apache-beam-testing.tfbackend | 4 +-
.../{provider.tf => iam.tf} | 15 +-
... => kafka-workflows.apache-beam-testing.tfvars} | 7 +-
.../google-kubernetes-engine/outputs.tf | 5 +-
.../google-kubernetes-engine/prerequisites.tf | 24 +
.../google-kubernetes-engine/provider.tf | 6 +
.../us-central1.apache-beam-testing.tfvars | 4 +-
.../us-west1.apache-beam-testing.tfvars | 4 +-
.../google-kubernetes-engine/variables.tf | 16 +
CHANGES.md | 37 +
build.gradle.kts | 3 +
contributor-docs/code-change-guide.md | 525 ++
examples/java/twitter/build.gradle | 1 +
.../it/gcp/bigtable/BigtableResourceManager.java | 2 +-
.../apache/beam/it/gcp/bigtable/BigTableIOST.java | 41 +-
.../java/org/apache/beam/it/kafka/KafkaIOST.java | 34 +-
.../apache/beam/it/neo4j/DatabaseWaitOption.java | 4 +-
.../apache/beam/it/neo4j/DatabaseWaitOptions.java | 68 +
.../apache/beam/it/neo4j/Neo4jResourceManager.java | 41 +-
.../beam/it/neo4j/Neo4jResourceManagerIT.java | 2 +-
.../beam/it/neo4j/Neo4jResourceManagerTest.java | 15 +-
learning/katas/go/go.mod | 2 +-
learning/katas/go/go.sum | 20 +-
.../beam/model/pipeline/v1/beam_runner_api.proto | 8 +
.../model/pipeline/v1/external_transforms.proto | 4 +
.../core/metrics/ExecutionStateTracker.java | 41 +-
runners/flink/flink_runner.gradle | 1 +
runners/flink/job-server/flink_job_server.gradle | 2 +
.../FlinkBatchPortablePipelineTranslator.java | 17 +-
.../flink/FlinkBatchTransformTranslators.java | 78 +
.../FlinkStreamingPortablePipelineTranslator.java | 22 +
.../flink/FlinkStreamingTransformTranslators.java | 37 +
.../dataflow/DataflowPipelineTranslator.java | 37 +
.../options/DataflowPipelineDebugOptions.java | 27 +-
.../beam/runners/dataflow/util/PropertyNames.java | 1 +
.../worker/DataflowBatchWorkerHarness.java | 3 +
.../dataflow/worker/DataflowExecutionContext.java | 96 +
.../dataflow/worker/DataflowOperationContext.java | 31 +-
.../runners/dataflow/worker/StackTraceUtil.java | 66 +
.../dataflow/worker/StreamingDataflowWorker.java | 3 +
.../dataflow/worker/WorkerCustomSources.java | 5 +-
.../worker/DataflowExecutionStateTrackerTest.java | 115 +-
.../dataflow/worker/WorkerCustomSourcesTest.java | 6 +-
.../util/common/worker/MapTaskExecutorTest.java | 28 +-
.../translation/RedistributeByKeyTranslator.java | 64 +
.../samza/translation/ReshuffleTranslator.java | 14 +-
.../samza/translation/SamzaPipelineTranslator.java | 1 +
.../spark/translation/TransformTranslator.java | 66 +
.../streaming/StreamingTransformTranslator.java | 70 +
sdks/go.mod | 6 +-
sdks/go.sum | 12 +-
.../prism/internal/engine/elementmanager.go | 213 +-
.../prism/internal/engine/elementmanager_test.go | 20 +-
.../runners/prism/internal/engine/engine_test.go | 5 +
.../beam/runners/prism/internal/engine/holds.go | 105 +
.../runners/prism/internal/engine/holds_test.go | 115 +
.../runners/prism/internal/engine/teststream.go | 43 +-
sdks/go/pkg/beam/runners/prism/internal/stage.go | 48 +-
.../beam/runners/prism/internal/worker/bundle.go | 5 +-
sdks/go/test/integration/primitives/teststream.go | 74 +
.../test/integration/primitives/teststream_test.go | 20 +
sdks/go/test/integration/primitives/timers.go | 2 +-
.../main/java/org/apache/beam/sdk/Pipeline.java | 2 +
.../java/org/apache/beam/sdk/schemas/Schema.java | 54 +-
.../apache/beam/sdk/schemas/SchemaRegistry.java | 23 +
.../sdk/schemas/annotations/DefaultSchema.java | 18 +
.../sdk/schemas/annotations/SchemaFieldNumber.java | 7 +-
.../transforms/SchemaTransformTranslation.java | 79 +
.../transforms/TypedSchemaTransformProvider.java | 9 +-
.../apache/beam/sdk/schemas/utils/YamlUtils.java | 21 +-
.../apache/beam/sdk/transforms/Redistribute.java | 300 +
.../sdk/util/construction/CoderTranslation.java | 17 +
.../util/construction/PTransformTranslation.java | 19 +
.../apache/beam/sdk/values/PCollectionViews.java | 52 +-
.../main/java/org/apache/beam/sdk/values/Row.java | 49 +
.../beam/sdk/schemas/SchemaRegistryTest.java | 49 +
.../org/apache/beam/sdk/schemas/SchemaTest.java | 100 +
.../TypedSchemaTransformProviderTest.java | 24 +-
.../beam/sdk/transforms/RedistributeTest.java | 410 ++
.../org/apache/beam/sdk/util/YamlUtilsTest.java | 33 +
.../java/org/apache/beam/sdk/values/RowTest.java | 90 +
.../sdk/expansion/service/ExpansionService.java | 41 +-
.../java/org/apache/beam/fn/harness/FnHarness.java | 2 +
.../org/apache/beam/fn/harness/CachesTest.java | 5 +-
sdks/java/io/expansion-service/build.gradle | 2 +
.../FileWriteSchemaTransformProviderTest.java | 32 +-
...ueryExportReadSchemaTransformConfiguration.java | 14 -
...FileLoadsWriteSchemaTransformConfiguration.java | 14 -
.../beam/sdk/io/gcp/bigquery/BigQueryHelpers.java | 22 +
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 31 +-
.../sdk/io/gcp/bigquery/BigQueryIOTranslation.java | 18 +-
.../beam/sdk/io/gcp/bigquery/BigQueryServices.java | 8 +
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 38 +-
.../gcp/bigquery/BigQueryStorageStreamSource.java | 7 +-
.../gcp/bigquery/DynamicDestinationsHelpers.java | 8 +
.../beam/sdk/io/gcp/bigtable/BigtableIO.java | 34 +-
.../dofn/ReadChangeStreamPartitionDoFn.java | 33 +-
.../changestreams/dofn/SerializableSupplier.java | 11 +-
.../sdk/io/gcp/bigquery/BigQueryClusteringIT.java | 5 +-
...QueryExportReadSchemaTransformProviderTest.java | 25 +-
...yFileLoadsWriteSchemaTransformProviderTest.java | 12 +-
.../sdk/io/gcp/bigquery/BigQueryHelpersTest.java | 11 +
.../io/gcp/bigquery/BigQueryIOTranslationTest.java | 12 +-
.../io/gcp/bigquery/BigQueryServicesImplTest.java | 12 +-
.../BigQueryTimePartitioningClusteringIT.java | 1 +
.../dofn/ReadChangeStreamPartitionDoFnTest.java | 49 +-
sdks/java/io/iceberg/build.gradle | 2 +-
.../apache/beam/io/iceberg/FileWriteResult.java | 203 -
.../java/org/apache/beam/io/iceberg/IcebergIO.java | 93 -
.../{ => sdk}/io/iceberg/AppendFilesToTables.java | 19 +-
.../{ => sdk}/io/iceberg/AssignDestinations.java | 2 +-
.../{ => sdk}/io/iceberg/DynamicDestinations.java | 2 +-
.../beam/sdk/io/iceberg/FileWriteResult.java | 85 +
.../{ => sdk}/io/iceberg/IcebergCatalogConfig.java | 2 +-
.../{ => sdk}/io/iceberg/IcebergDestination.java | 2 +-
.../org/apache/beam/sdk/io/iceberg/IcebergIO.java | 136 +
.../IcebergReadSchemaTransformProvider.java | 146 +
.../{ => sdk}/io/iceberg/IcebergScanConfig.java | 2 +-
.../IcebergSchemaTransformCatalogConfig.java | 107 +
.../iceberg/IcebergSchemaTransformTranslation.java | 88 +
.../io/iceberg/IcebergTableCreateConfig.java | 2 +-
.../{ => sdk}/io/iceberg/IcebergWriteResult.java | 13 +-
.../IcebergWriteSchemaTransformProvider.java | 190 +
.../io/iceberg/OneTableDynamicDestinations.java | 31 +-
.../beam/{ => sdk}/io/iceberg/PropertyBuilder.java | 2 +-
.../beam/{ => sdk}/io/iceberg/RecordWriter.java | 24 +-
.../beam/{ => sdk}/io/iceberg/ScanSource.java | 2 +-
.../beam/{ => sdk}/io/iceberg/ScanTaskReader.java | 2 +-
.../beam/{ => sdk}/io/iceberg/ScanTaskSource.java | 2 +-
.../io/iceberg/SchemaAndRowConversions.java | 2 +-
.../apache/beam/sdk/io/iceberg/SnapshotInfo.java | 118 +
.../io/iceberg/WriteGroupedRowsToFiles.java | 5 +-
.../{ => sdk}/io/iceberg/WriteToDestinations.java | 5 +-
.../io/iceberg/WriteUngroupedRowsToFiles.java | 7 +-
.../beam/{ => sdk}/io/iceberg/package-info.java | 2 +-
.../beam/io/iceberg/FileWriteResultTest.java | 151 -
.../{ => sdk}/io/iceberg/IcebergIOReadTest.java | 4 +-
.../{ => sdk}/io/iceberg/IcebergIOWriteTest.java | 16 +-
.../IcebergReadSchemaTransformProviderTest.java | 183 +
.../IcebergSchemaTransformTranslationTest.java | 248 +
.../IcebergWriteSchemaTransformProviderTest.java | 175 +
.../beam/{ => sdk}/io/iceberg/ScanSourceTest.java | 2 +-
.../io/iceberg/SchemaAndRowConversionsTest.java | 2 +-
.../{ => sdk}/io/iceberg/TestDataWarehouse.java | 2 +-
.../beam/{ => sdk}/io/iceberg/TestFixtures.java | 2 +-
sdks/java/managed/build.gradle | 1 -
.../java/org/apache/beam/sdk/managed/Managed.java | 39 +-
.../managed/ManagedSchemaTransformProvider.java | 129 +-
.../managed/ManagedSchemaTransformTranslation.java | 59 +
.../sdk/managed/ManagedTransformConstants.java} | 9 +-
.../testing}/TestSchemaTransformProvider.java | 8 +-
.../beam/sdk/managed/testing}/package-info.java | 4 +-
.../ManagedSchemaTransformProviderTest.java | 48 +-
.../ManagedSchemaTransformTranslationTest.java | 185 +
.../org/apache/beam/sdk/managed/ManagedTest.java | 3 +-
.../managed/src/test/resources/test_config.yaml | 4 +-
sdks/python/apache_beam/io/avroio.py | 132 +-
sdks/python/apache_beam/io/avroio_test.py | 80 +-
sdks/python/apache_beam/io/gcp/bigquery.py | 6 +-
sdks/python/apache_beam/ml/inference/base.py | 110 +-
sdks/python/apache_beam/ml/inference/base_test.py | 71 +
.../ml/inference/huggingface_inference.py | 36 +-
.../apache_beam/ml/inference/onnx_inference.py | 12 +-
.../apache_beam/ml/inference/pytorch_inference.py | 24 +-
.../apache_beam/ml/inference/sklearn_inference.py | 24 +-
.../ml/inference/tensorflow_inference.py | 24 +-
.../apache_beam/ml/inference/tensorrt_inference.py | 12 +-
sdks/python/apache_beam/ml/transforms/tft.py | 41 +-
sdks/python/apache_beam/ml/transforms/tft_test.py | 126 +
sdks/python/apache_beam/runners/common.pxd | 4 +-
sdks/python/apache_beam/runners/common.py | 75 +-
.../apache_beam/runners/dataflow/internal/names.py | 2 +-
.../apache_beam/runners/portability/stager.py | 1 +
.../enrichment_handlers/feast_feature_store.py | 200 +
.../feast_feature_store_it_test.py | 121 +
.../feast_feature_store_test.py | 67 +
.../feast_tests_requirements.txt | 7 +-
sdks/python/apache_beam/yaml/yaml_io.py | 39 +-
sdks/python/apache_beam/yaml/yaml_io_test.py | 30 +-
sdks/python/build.gradle | 2 +-
.../container/py310/base_image_requirements.txt | 8 +-
.../container/py311/base_image_requirements.txt | 8 +-
.../container/py38/base_image_requirements.txt | 8 +-
.../container/py39/base_image_requirements.txt | 8 +-
sdks/python/pytest.ini | 1 +
sdks/python/scripts/generate_pydoc.sh | 2 +-
sdks/python/setup.py | 6 +-
sdks/python/test-suites/direct/common.gradle | 28 +
sdks/standard_expansion_services.yaml | 3 +
sdks/standard_external_transforms.yaml | 5 +-
settings.gradle.kts | 2 +-
.../content/en/documentation/runners/dataflow.md | 5 +-
.../www/site/content/en/documentation/sdks/yaml.md | 14 +-
.../site/content/en/get-started/quickstart-java.md | 2 +-
.../get-started/resources/videos-and-podcasts.md | 8 +-
website/www/site/static/.htaccess | 2 +-
261 files changed, 7221 insertions(+), 16410 deletions(-)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Dataflow.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Flink.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Samza.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Spark.json} (100%)
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/.terraform.lock.hcl
delete mode 100644 .test-infra/kafka/strimzi/01-strimzi-operator/README.md
copy
sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/package-info.java
=> .test-infra/kafka/strimzi/01-strimzi-operator/common.tfvars (79%)
copy
.test-infra/{terraform/google-cloud-platform/google-kubernetes-engine/provider.tf
=> kafka/strimzi/01-strimzi-operator/kafka.tf} (69%)
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/kustomization.yaml
delete mode 100644 .test-infra/kafka/strimzi/01-strimzi-operator/namespace.yaml
copy sdks/java/managed/src/test/resources/test_config.yaml =>
.test-infra/kafka/strimzi/01-strimzi-operator/provider.tf (85%)
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/020-ClusterRole-strimzi-cluster-operator-role.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/020-RoleBinding-strimzi-cluster-operator.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/021-ClusterRole-strimzi-cluster-operator-role.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/021-ClusterRoleBinding-strimzi-cluster-operator.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/022-ClusterRole-strimzi-cluster-operator-role.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/022-RoleBinding-strimzi-cluster-operator.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/023-ClusterRole-strimzi-cluster-operator-role.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/023-RoleBinding-strimzi-cluster-operator.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/030-ClusterRole-strimzi-kafka-broker.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/030-ClusterRoleBinding-strimzi-cluster-operator-kafka-broker-delegation.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/031-ClusterRole-strimzi-entity-operator.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/031-RoleBinding-strimzi-cluster-operator-entity-operator-delegation.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/033-ClusterRole-strimzi-kafka-client.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/033-ClusterRoleBinding-strimzi-cluster-operator-kafka-client-delegation.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/040-Crd-kafka.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/041-Crd-kafkaconnect.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/042-Crd-strimzipodset.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/043-Crd-kafkatopic.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/044-Crd-kafkauser.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/045-Crd-kafkamirrormaker.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/046-Crd-kafkabridge.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/047-Crd-kafkaconnector.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/048-Crd-kafkamirrormaker2.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/049-Crd-kafkarebalance.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/050-ConfigMap-strimzi-cluster-operator.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/060-Deployment-strimzi-cluster-operator.yaml
delete mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/kustomization.yaml
copy .test-infra/{terraform/google-cloud-platform/google-kubernetes-engine =>
kafka/strimzi/01-strimzi-operator}/variables.tf (56%)
copy
sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/package-info.java
=>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/.kafka-workflows.apache-beam-testing.tfbackend
(84%)
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/.terraform.lock.hcl
copy
sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/package-info.java
=>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/copyme.apache-beam-testing.tfbackend
(84%)
copy
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/{provider.tf
=> iam.tf} (59%)
copy
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/{us-central1.apache-beam-testing.tfvars
=> kafka-workflows.apache-beam-testing.tfvars} (79%)
copy
sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/package-info.java
=>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/outputs.tf
(79%)
create mode 100644 contributor-docs/code-change-guide.md
copy
sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/package-info.java
=> it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOption.java (91%)
create mode 100644
it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOptions.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StackTraceUtil.java
create mode 100644
runners/samza/src/main/java/org/apache/beam/runners/samza/translation/RedistributeByKeyTranslator.java
create mode 100644 sdks/go/pkg/beam/runners/prism/internal/engine/holds.go
create mode 100644 sdks/go/pkg/beam/runners/prism/internal/engine/holds_test.go
create mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/SchemaTransformTranslation.java
create mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Redistribute.java
create mode 100644
sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/RedistributeTest.java
copy
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/us-west1.apache-beam-testing.tfvars
=>
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/SerializableSupplier.java
(71%)
delete mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/FileWriteResult.java
delete mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/IcebergIO.java
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/AppendFilesToTables.java (86%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/AssignDestinations.java (98%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/DynamicDestinations.java (96%)
create mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/FileWriteResult.java
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/IcebergCatalogConfig.java (99%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/IcebergDestination.java (98%)
create mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergIO.java
create mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergReadSchemaTransformProvider.java
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/IcebergScanConfig.java (99%)
create mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergSchemaTransformCatalogConfig.java
create mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergSchemaTransformTranslation.java
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/IcebergTableCreateConfig.java (97%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/IcebergWriteResult.java (85%)
create mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergWriteSchemaTransformProvider.java
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/OneTableDynamicDestinations.java (66%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/PropertyBuilder.java (97%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/RecordWriter.java (77%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/ScanSource.java (99%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/ScanTaskReader.java (99%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/ScanTaskSource.java (98%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/SchemaAndRowConversions.java (99%)
create mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/SnapshotInfo.java
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/WriteGroupedRowsToFiles.java (96%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/WriteToDestinations.java (97%)
rename sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/WriteUngroupedRowsToFiles.java (98%)
copy sdks/java/io/iceberg/src/main/java/org/apache/beam/{ =>
sdk}/io/iceberg/package-info.java (95%)
delete mode 100644
sdks/java/io/iceberg/src/test/java/org/apache/beam/io/iceberg/FileWriteResultTest.java
rename sdks/java/io/iceberg/src/test/java/org/apache/beam/{ =>
sdk}/io/iceberg/IcebergIOReadTest.java (97%)
rename sdks/java/io/iceberg/src/test/java/org/apache/beam/{ =>
sdk}/io/iceberg/IcebergIOWriteTest.java (95%)
create mode 100644
sdks/java/io/iceberg/src/test/java/org/apache/beam/sdk/io/iceberg/IcebergReadSchemaTransformProviderTest.java
create mode 100644
sdks/java/io/iceberg/src/test/java/org/apache/beam/sdk/io/iceberg/IcebergSchemaTransformTranslationTest.java
create mode 100644
sdks/java/io/iceberg/src/test/java/org/apache/beam/sdk/io/iceberg/IcebergWriteSchemaTransformProviderTest.java
rename sdks/java/io/iceberg/src/test/java/org/apache/beam/{ =>
sdk}/io/iceberg/ScanSourceTest.java (99%)
rename sdks/java/io/iceberg/src/test/java/org/apache/beam/{ =>
sdk}/io/iceberg/SchemaAndRowConversionsTest.java (99%)
rename sdks/java/io/iceberg/src/test/java/org/apache/beam/{ =>
sdk}/io/iceberg/TestDataWarehouse.java (99%)
rename sdks/java/io/iceberg/src/test/java/org/apache/beam/{ =>
sdk}/io/iceberg/TestFixtures.java (99%)
create mode 100644
sdks/java/managed/src/main/java/org/apache/beam/sdk/managed/ManagedSchemaTransformTranslation.java
copy
sdks/java/{io/iceberg/src/main/java/org/apache/beam/io/iceberg/package-info.java
=>
managed/src/main/java/org/apache/beam/sdk/managed/ManagedTransformConstants.java}
(68%)
rename sdks/java/managed/src/{test/java/org/apache/beam/sdk/managed =>
main/java/org/apache/beam/sdk/managed/testing}/TestSchemaTransformProvider.java
(91%)
rename sdks/java/{io/iceberg/src/main/java/org/apache/beam/io/iceberg =>
managed/src/main/java/org/apache/beam/sdk/managed/testing}/package-info.java
(90%)
create mode 100644
sdks/java/managed/src/test/java/org/apache/beam/sdk/managed/ManagedSchemaTransformTranslationTest.java
create mode 100644
sdks/python/apache_beam/transforms/enrichment_handlers/feast_feature_store.py
create mode 100644
sdks/python/apache_beam/transforms/enrichment_handlers/feast_feature_store_it_test.py
create mode 100644
sdks/python/apache_beam/transforms/enrichment_handlers/feast_feature_store_test.py
rename
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/010-ServiceAccount-strimzi-cluster-operator.yaml
=>
sdks/python/apache_beam/transforms/enrichment_handlers/feast_tests_requirements.txt
(88%)
