This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/cloud.google.com/go/storage-1.56.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 54ee1f88347 Bump cloud.google.com/go/storage from 1.55.0 to 1.56.0 in
/sdks
add 9ae3c5ee151 Bump golang.org/x/oauth2 from 0.12.0 to 0.27.0 in
/.test-infra/mock-apis (#35630)
add 3ce1b2bbf5d Bump github.com/go-sql-driver/mysql from 1.9.2 to 1.9.3 in
/sdks (#35365)
add f7114e0ac44 Update README.md (#35689)
add c99b84f00e8 Remove tox dependency
add e30edfc92a3 Remove tox dependency (#35679)
add 80f1052b530 Fix flink runner test (#34913)
add 0dc5175e659 Cloudsql pg colab (#35690)
add ecfbc24b4a5 sdks/python/scripts: support pytest user markers (#35655)
add e80ffd474d9 Upgrade base image for dev-support docker image(Ubuntu
20.04 -> Ubuntu 24.04) (#35694)
add f5fb6cc67ce fix the version string checks in KafkaIO (#35703)
add 999ae4762b5 Reflect that my previous PR was cherrypicked into 2.67
(#35676)
add 420d3f32f45 Update Python Dependencies (#35705)
add 125082c4513 Update beam_PostCommit_Python_Portable_Flink.yml (#35699)
add 800f78bf5e9 Revert "Refactor: separate
SplittableTruncateSizedRestrictions" (#35700)
add 02aa78d831c Bump Dataflow container version (#35712)
add 4043c901e99 [Java] Simplify MoreFutures usage in case it is cause of
issues with ForkJoin pool stuckness (#35673)
add 4ae92f52e02 Reapply "Refactor: separate
SplittableTruncateSizedRestrictions" (#35700)
add 2240a24bd48 Set use_staged_dataflow_worker_jar for dev Beam
add ef0486285dd Merge pull request #35714 from Abacn/reapply
add 2d33d1d59bb Remove ZetaSQL
add 93c0f1bda2a add CHANGES.md
add 06b622c941f Merge pull request #35662 from Abacn/remove-zetasql
add fb1fab07ede YAML Test CLI Flag Fix (#35721)
add 0331f20098a Add Example for Iceberg REST Catalog CDC (#35649)
add 31ce80fa6d2 Use vendored Calcite 1.40.0
add a91d10617eb Fix after upgrading to Calcite 1.40
add 6a0f130c4c8 Fix Iceberg SQL test
add 531b5fc093c trigger tests
add 643d5e56efe Fix tpcds json dependency change
add 377dff58d67 Update CHANGES.md
add 805b68c9553 Merge pull request #35718 from Abacn/use-vendor-calcite
add e2a3ab812f9 workflows: run ML tests on `ubuntu-latest` (#35698)
add c52993ebb62 Add documentation for Wait.On and WaitOn transforms
(#35647)
add 749e274ecd0 add note on spammy logs and mitigation (#35726)
add 4df9762a0f7 .github: remove `ubuntu-latest` environment from ml
add ac654fd7911 Merge pull request #35734 from
mohamedawnallah/revertUbuntuLatest
add 1e1c62a3627 Iobase streaming support in FileBasedSink, exposed in
TextIO (#35253)
add 480fcc87536 sdks/python/scripts: fix posargs for `run_pytest.sh`
(#35733)
add aee8edd9fa0 adapt to new np outputs (#35691) (#35741)
add 5dad7a6659c Fix PostCommit Python job (#35709)
add f117e93b0c8 Update Java beam-master container (#35732)
add 72ecb475fc1 sdks/python/scripts: capture `(...)` pytest marker group
(#35740)
add d493275bb09 [yaml] Fix ml embeddings issue (#35715)
add f2f088283c4 Add Terraform configuration and IAM management scripts for
GCP project (#35701)
add 1aa3592f2a6 [YAML] A Streaming Inference Pipeline - Taxi Fare
Estimation (#35568)
add 9fc413e0f38 Lower log level for harmless SchemaCoder inference failure
in JdbcIO (#35747)
add b0074a964b4 Add workflow dispatch (#35750)
add e5c5efaf338 [Java Harness] Improve caching performance and measure
overhead for cache internals. (#35632)
add b3c1dd30fb4 Add dataflow option go (#35413)
add 04a25706b79 [Java SDK] Add option elementProcessingTimeoutMinutes that
terminates SDK if dofn processing exceeds this (#35120)
add 132b24b32e2 Revert 35715 (#35753)
add f7e5ee593a8 feat(yaml): add a simple schema unification for Flatten
transform (#35728)
add 6fe69fe703e Fix BeamSQL CalcRel DATETIME expression (#35743)
add dbc3f0da578 update Bigtable reviewer (#35761)
add cdb89891d2e docs: Update 2025 discussion documents list up to July
(#35763)
add b81f819d821 Remove OSSRH Sonatype from list of repos (#35764)
add 804dd3559a4 Update config.yaml (#35765)
add 6b91e5c4d7d [Yaml] - fix ml embeddings issue round2 (#35756)
add 00c1f10fb51 [yaml] - add nullable field test for readFromBigQuery and
update create core logic (#35692)
add 5395c9dd8e9 Add next steps to hugging face notebook (#35767)
add 2d803011ac6 Fix ValueError affecting XVR tests (#35766)
add 138640827e0 Bump github.com/aws/smithy-go from 1.22.4 to 1.22.5 in
/sdks (#35682)
add 30dc617d1f2 Fix(test): Make template path unique in
DataflowJobServiceTest (#35776)
add 89ae2763844 Fix: Dedup input list of Topics/TopicPartitions in KafkaIO
(#35758)
add 689b86410d0 fix windows slashes for dask (#30528) (#35781)
add 185dee0c994 Add a pipeline option for setting prism log level. (#35778)
add 5593f405cbe Fix race conditions in apache.beam.sdks.schemas (#35752)
add ed992508c1d Fix a split bundle bug when using timers in stateful dofn
(#35770)
add a247319eb61 Fix options.go flag comments (#35785)
add 720f9617be0 Metastore example (#35722)
add ac2706a291b Fix timestamp issue in a yaml pipeline that calls SQL
transform (#35789)
add 624b2fcd3fd Avoid unreasonably long stage names for @ptransform_fn.
(#35660)
add dab36a365c8 feat(io): Add option to include source filename in file
reads (#35775)
add 8aec418c9f2 Bump github.com/docker/docker in /sdks (#35731)
add 73b1090093b Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks
(#35782)
add 95fd03eb2ab Bump cloud.google.com/go/storage from 1.55.0 to 1.56.0 in
/sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (54ee1f88347)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/cloud.google.com/go/storage-1.56.0
(95fd03eb2ab)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.gemini/config.yaml | 2 +-
.github/REVIEWERS.yml | 3 -
.github/trigger_files/beam_PostCommit_Python.json | 2 +-
.../beam_PostCommit_Python_Portable_Flink.json | 3 +
.../beam_PostCommit_Python_Xlang_IO_Dataflow.json | 2 +-
.github/trigger_files/beam_PostCommit_SQL.json | 2 +-
.../trigger_files/beam_PostCommit_XVR_Direct.json | 3 +-
.../trigger_files/beam_PostCommit_XVR_Flink.json | 3 +-
.github/workflows/beam_PostCommit_Python.yml | 14 +-
...l => beam_PostCommit_Python_Portable_Flink.yml} | 36 +-
.../workflows/beam_PreCommit_Portable_Python.yml | 1 +
.github/workflows/beam_PreCommit_Prism_Python.yml | 1 +
.github/workflows/beam_PreCommit_Python_ML.yml | 15 +-
.github/workflows/beam_UserRoles.yml | 62 +
.test-infra/mock-apis/go.mod | 8 +-
.test-infra/mock-apis/go.sum | 10 +-
CHANGES.md | 20 +-
build.gradle.kts | 1 -
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 9 +-
.../org/apache/beam/gradle/Repositories.groovy | 3 -
contributor-docs/discussion-docs/2025.md | 22 +-
dev-support/docker/Dockerfile | 7 +-
examples/java/build.gradle | 1 +
.../cookbook/IcebergBatchWriteExample.java | 208 +
.../cookbook/IcebergRestCatalogCDCExample.java | 235 ++
.../IcebergRestCatalogStreamingWriteExample.java | 235 +-
...dsql_postgres_product_catalog_embeddings.ipynb} | 680 ++--
.../huggingface_text_embeddings.ipynb | 15 +
infra/iam/.terraform.lock.hcl | 21 +
infra/iam/README.md | 63 +
.../iam/config.auto.tfvars | 6 +-
infra/iam/generate.py | 212 +
.../config_aliases.yaml => infra/iam/main.tf | 23 +-
.../iam/requirements.txt | 9 +-
infra/iam/users.tf | 60 +
infra/iam/users.yml | 547 +++
.../it/gcp/dataflow/DefaultPipelineLauncher.java | 9 +
runners/google-cloud-dataflow-java/build.gradle | 4 +-
sdks/go.mod | 26 +-
sdks/go.sum | 52 +-
sdks/go/examples/xlang/sql/sql.go | 2 +-
sdks/go/pkg/beam/core/metrics/sampler.go | 14 +-
sdks/go/pkg/beam/core/metrics/sampler_test.go | 53 +-
sdks/go/pkg/beam/core/runtime/harness/harness.go | 22 +-
.../pkg/beam/core/runtime/harness/harness_test.go | 27 +
sdks/go/pkg/beam/core/runtime/harness/sampler.go | 19 +-
sdks/go/pkg/beam/options/jobopts/options.go | 19 +-
sdks/go/pkg/beam/options/jobopts/options_test.go | 16 +
.../prism/internal/engine/elementmanager.go | 34 +-
sdks/go/pkg/beam/runners/prism/internal/stage.go | 2 +-
.../beam/runners/prism/internal/worker/bundle.go | 4 +-
sdks/go/pkg/beam/transforms/sql/sql.go | 2 +-
sdks/go/pkg/beam/transforms/sql/sql_test.go | 1 -
.../resources/beam/checkstyle/suppressions.xml | 10 -
.../org/apache/beam/sdk/fn/data/WeightedList.java | 8 +-
.../org/apache/beam/sdk/fn/stream/DataStreams.java | 2 +-
.../java/org/apache/beam/sdk/io/WriteFiles.java | 6 +-
.../apache/beam/sdk/options/SdkHarnessOptions.java | 12 +
.../apache/beam/sdk/schemas/CachingFactory.java | 16 +-
.../beam/sdk/schemas/FromRowUsingCreator.java | 2 +-
.../beam/sdk/schemas/logicaltypes/SqlTypes.java | 8 +-
.../java/org/apache/beam/sdk/util/MoreFutures.java | 7 +
.../beam/sdk/extensions/gcp/util/GcsUtil.java | 2 +-
sdks/java/extensions/sql/build.gradle | 14 +-
sdks/java/extensions/sql/datacatalog/build.gradle | 1 -
.../datacatalog/DataCatalogBigQueryIT.java | 7 +-
.../extensions/sql/expansion-service/build.gradle | 1 -
.../expansion/ExternalSqlTransformRegistrar.java | 2 -
.../SqlTransformSchemaTransformProvider.java | 4 +-
sdks/java/extensions/sql/jdbc/build.gradle | 4 +-
.../sdk/extensions/sql/jdbc/BeamSqlLineIT.java | 10 +-
.../sdk/extensions/sql/jdbc/BeamSqlLineTest.java | 13 +-
.../provider/bigquery/BigQueryIOPushDownIT.java | 6 +-
.../extensions/sql/src/main/codegen/config.fmpp | 42 +-
.../extensions/sql/TableNameExtractionUtils.java | 14 +-
.../sdk/extensions/sql/impl/BeamCalciteSchema.java | 16 +-
.../sql/impl/BeamCalciteSchemaFactory.java | 18 +-
.../sdk/extensions/sql/impl/BeamCalciteTable.java | 28 +-
.../beam/sdk/extensions/sql/impl/BeamSqlEnv.java | 10 +-
.../extensions/sql/impl/BeamTableStatistics.java | 12 +-
.../sql/impl/CalciteConnectionWrapper.java | 21 +-
.../extensions/sql/impl/CalciteFactoryWrapper.java | 24 +-
.../extensions/sql/impl/CalciteQueryPlanner.java | 77 +-
.../sdk/extensions/sql/impl/JavaUdfLoader.java | 2 +-
.../sdk/extensions/sql/impl/JdbcConnection.java | 6 +-
.../beam/sdk/extensions/sql/impl/JdbcDriver.java | 28 +-
.../beam/sdk/extensions/sql/impl/JdbcFactory.java | 32 +-
.../beam/sdk/extensions/sql/impl/QueryPlanner.java | 4 +-
.../extensions/sql/impl/ScalarFunctionImpl.java | 42 +-
.../extensions/sql/impl/TableResolutionUtils.java | 6 +-
.../beam/sdk/extensions/sql/impl/UdafImpl.java | 12 +-
.../beam/sdk/extensions/sql/impl/UdfImpl.java | 10 +-
.../sql/impl/UdfImplReflectiveFunctionBase.java | 14 +-
...SqlUserDefinedSQLNativeTableValuedFunction.java | 14 +-
.../beam/sdk/extensions/sql/impl/cep/CEPCall.java | 10 +-
.../sdk/extensions/sql/impl/cep/CEPFieldRef.java | 2 +-
.../sdk/extensions/sql/impl/cep/CEPLiteral.java | 2 +-
.../sdk/extensions/sql/impl/cep/CEPOperation.java | 8 +-
.../sdk/extensions/sql/impl/cep/CEPOperator.java | 4 +-
.../sdk/extensions/sql/impl/cep/CEPPattern.java | 2 +-
.../beam/sdk/extensions/sql/impl/cep/CEPUtils.java | 16 +-
.../beam/sdk/extensions/sql/impl/cep/OrderKey.java | 2 +-
.../extensions/sql/impl/parser/BeamSqlParser.java | 8 +-
.../sql/impl/parser/SqlCheckConstraint.java | 18 +-
.../sql/impl/parser/SqlColumnDeclaration.java | 18 +-
.../sql/impl/parser/SqlCreateCatalog.java | 30 +-
.../sql/impl/parser/SqlCreateDatabase.java | 28 +-
.../sql/impl/parser/SqlCreateExternalTable.java | 28 +-
.../sql/impl/parser/SqlCreateFunction.java | 32 +-
.../extensions/sql/impl/parser/SqlDdlNodes.java | 20 +-
.../extensions/sql/impl/parser/SqlDropCatalog.java | 28 +-
.../sql/impl/parser/SqlDropDatabase.java | 28 +-
.../extensions/sql/impl/parser/SqlDropObject.java | 20 +-
.../extensions/sql/impl/parser/SqlDropTable.java | 10 +-
.../sql/impl/parser/SqlSetOptionBeam.java | 18 +-
.../extensions/sql/impl/parser/SqlUseCatalog.java | 26 +-
.../extensions/sql/impl/parser/SqlUseDatabase.java | 26 +-
.../extensions/sql/impl/planner/BeamCostModel.java | 8 +-
.../sql/impl/planner/BeamJavaTypeFactory.java | 12 +-
.../sql/impl/planner/BeamRelDataTypeSystem.java | 6 +-
.../sql/impl/planner/BeamRelMetadataQuery.java | 6 +-
.../extensions/sql/impl/planner/BeamRuleSets.java | 13 +-
.../sql/impl/planner/NodeStatsMetadata.java | 12 +-
.../sql/impl/planner/RelMdNodeStats.java | 16 +-
.../sql/impl/rel/AbstractBeamCalcRel.java | 18 +-
.../sql/impl/rel/BeamAggregationRel.java | 16 +-
.../sdk/extensions/sql/impl/rel/BeamCalcRel.java | 71 +-
.../extensions/sql/impl/rel/BeamCoGBKJoinRel.java | 16 +-
.../sql/impl/rel/BeamEnumerableConverter.java | 36 +-
.../sdk/extensions/sql/impl/rel/BeamIOSinkRel.java | 73 +-
.../extensions/sql/impl/rel/BeamIOSourceRel.java | 18 +-
.../extensions/sql/impl/rel/BeamIntersectRel.java | 12 +-
.../sdk/extensions/sql/impl/rel/BeamJoinRel.java | 28 +-
.../sql/impl/rel/BeamLogicalConvention.java | 12 +-
.../sdk/extensions/sql/impl/rel/BeamMatchRel.java | 22 +-
.../sdk/extensions/sql/impl/rel/BeamMinusRel.java | 12 +-
.../sql/impl/rel/BeamPushDownIOSourceRel.java | 10 +-
.../sdk/extensions/sql/impl/rel/BeamRelNode.java | 8 +-
.../sql/impl/rel/BeamSideInputJoinRel.java | 16 +-
.../sql/impl/rel/BeamSideInputLookupJoinRel.java | 14 +-
.../sdk/extensions/sql/impl/rel/BeamSortRel.java | 24 +-
.../extensions/sql/impl/rel/BeamSqlRelUtils.java | 6 +-
.../sql/impl/rel/BeamTableFunctionScanRel.java | 22 +-
.../extensions/sql/impl/rel/BeamUncollectRel.java | 10 +-
.../sdk/extensions/sql/impl/rel/BeamUnionRel.java | 12 +-
.../sdk/extensions/sql/impl/rel/BeamUnnestRel.java | 20 +-
.../sdk/extensions/sql/impl/rel/BeamValuesRel.java | 14 +-
.../sdk/extensions/sql/impl/rel/BeamWindowRel.java | 27 +-
.../extensions/sql/impl/rel/CalcRelSplitter.java | 54 +-
.../sdk/extensions/sql/impl/rel/package-info.java | 2 +-
.../impl/rule/BeamAggregateProjectMergeRule.java | 20 +-
.../sql/impl/rule/BeamAggregationRule.java | 24 +-
.../sql/impl/rule/BeamBasicAggregationRule.java | 26 +-
.../sql/impl/rule/BeamCalcMergeRule.java | 10 +-
.../sdk/extensions/sql/impl/rule/BeamCalcRule.java | 18 +-
.../sql/impl/rule/BeamCalcSplittingRule.java | 14 +-
.../sql/impl/rule/BeamCoGBKJoinRule.java | 12 +-
.../sql/impl/rule/BeamEnumerableConverterRule.java | 8 +-
.../sql/impl/rule/BeamIOPushDownRule.java | 38 +-
.../extensions/sql/impl/rule/BeamIOSinkRule.java | 6 +-
.../sql/impl/rule/BeamIntersectRule.java | 10 +-
.../sql/impl/rule/BeamJoinAssociateRule.java | 12 +-
.../sql/impl/rule/BeamJoinPushThroughJoinRule.java | 16 +-
.../extensions/sql/impl/rule/BeamMatchRule.java | 10 +-
.../extensions/sql/impl/rule/BeamMinusRule.java | 10 +-
.../sql/impl/rule/BeamSideInputJoinRule.java | 12 +-
.../sql/impl/rule/BeamSideInputLookupJoinRule.java | 12 +-
.../sdk/extensions/sql/impl/rule/BeamSortRule.java | 10 +-
.../sql/impl/rule/BeamTableFunctionScanRule.java | 10 +-
.../sql/impl/rule/BeamUncollectRule.java | 8 +-
.../extensions/sql/impl/rule/BeamUnionRule.java | 12 +-
.../extensions/sql/impl/rule/BeamUnnestRule.java | 24 +-
.../extensions/sql/impl/rule/BeamValuesRule.java | 10 +-
.../extensions/sql/impl/rule/BeamWindowRule.java | 10 +-
.../sql/impl/rule/JoinRelOptRuleCall.java | 16 +-
.../sdk/extensions/sql/impl/rule/package-info.java | 2 +-
.../extensions/sql/impl/schema/BeamTableUtils.java | 14 +-
.../sql/impl/transform/BeamJoinTransforms.java | 8 +-
.../transform/agg/AggregationCombineFnAdapter.java | 4 +-
.../sql/impl/transform/agg/CovarianceFn.java | 2 +-
.../sql/impl/transform/agg/VarianceFn.java | 2 +-
.../sql/impl/udf/BuiltinHashFunctions.java | 2 +-
.../sql/impl/udf/BuiltinStringFunctions.java | 2 +-
.../extensions/sql/impl/utils/CalciteUtils.java | 12 +-
.../sql/impl/utils/SerializableRexFieldAccess.java | 4 +-
.../sql/impl/utils/SerializableRexInputRef.java | 2 +-
.../sql/impl/utils/SerializableRexNode.java | 6 +-
.../sdk/extensions/sql/meta/BaseBeamTable.java | 2 +-
.../beam/sdk/extensions/sql/meta/BeamSqlTable.java | 2 +-
.../extensions/sql/meta/BeamSqlTableFilter.java | 4 +-
.../extensions/sql/meta/DefaultTableFilter.java | 2 +-
.../provider/bigquery/BeamBigQuerySqlDialect.java | 22 +-
.../provider/bigquery/BeamSqlUnparseContext.java | 52 +-
.../sql/meta/provider/bigquery/BigQueryFilter.java | 34 +-
.../sql/meta/provider/bigquery/BigQueryTable.java | 12 +-
.../sql/meta/provider/bigtable/BigtableFilter.java | 10 +-
.../sql/meta/provider/bigtable/BigtableTable.java | 2 +-
.../meta/provider/datagen/DataGeneratorRowFn.java | 4 +-
.../sql/meta/provider/iceberg/IcebergFilter.java | 16 +-
.../sql/meta/provider/iceberg/IcebergTable.java | 14 +-
.../sql/meta/provider/mongodb/MongoDbTable.java | 18 +-
.../sql/meta/provider/test/TestTableFilter.java | 14 +-
.../sql/meta/provider/test/TestTableProvider.java | 10 +-
.../sql/meta/provider/test/TestUnboundedTable.java | 2 +-
..._40_0.org.codehaus.commons.compiler.properties} | 2 +-
.../sdk/extensions/sql/BeamComplexTypeTest.java | 40 +
.../sdk/extensions/sql/BeamSqlCliDatabaseTest.java | 2 +-
.../beam/sdk/extensions/sql/BeamSqlCliTest.java | 2 +-
.../sql/BeamSqlDslSqlStdOperatorsTest.java | 43 +-
.../sdk/extensions/sql/BeamSqlDslUdfUdafTest.java | 8 +-
.../sdk/extensions/sql/BeamSqlExplainTest.java | 6 +-
.../extensions/sql/TypedCombineFnDelegateTest.java | 10 +-
.../sdk/extensions/sql/impl/JdbcDriverTest.java | 6 +-
.../sql/impl/LazyAggregateCombineFnTest.java | 12 +-
.../sql/impl/parser/BeamDDLNestedTypesTest.java | 2 +-
.../extensions/sql/impl/parser/BeamDDLTest.java | 12 +-
.../extensions/sql/impl/planner/NodeStatsTest.java | 12 +-
.../sql/impl/rel/BeamAggregationRelTest.java | 2 +-
.../extensions/sql/impl/rel/BeamCalcRelTest.java | 2 +-
.../rel/BeamCoGBKJoinRelBoundedVsBoundedTest.java | 2 +-
.../BeamCoGBKJoinRelUnboundedVsUnboundedTest.java | 2 +-
.../sql/impl/rel/BeamEnumerableConverterTest.java | 37 +-
.../sql/impl/rel/BeamIOSourceRelTest.java | 4 +-
.../sql/impl/rel/BeamIntersectRelTest.java | 2 +-
.../extensions/sql/impl/rel/BeamMinusRelTest.java | 2 +-
.../sql/impl/rel/BeamSideInputJoinRelTest.java | 2 +-
.../extensions/sql/impl/rel/BeamSortRelTest.java | 2 +-
.../sql/impl/rel/BeamUncollectRelTest.java | 2 +-
.../extensions/sql/impl/rel/BeamUnionRelTest.java | 2 +-
.../extensions/sql/impl/rel/BeamValuesRelTest.java | 2 +-
.../sql/impl/rule/IOPushDownRuleTest.java | 14 +-
.../sql/impl/rule/JoinReorderingTest.java | 72 +-
.../sql/impl/schema/BeamSqlRowCoderTest.java | 8 +-
.../sql/impl/utils/CalciteUtilsTest.java | 10 +-
.../BeamSqlDateFunctionsIntegrationTest.java | 4 +-
.../sql/meta/CustomTableResolverTest.java | 2 +-
.../provider/datastore/DataStoreReadWriteIT.java | 2 +-
.../provider/iceberg/BeamSqlCliIcebergTest.java | 2 +-
.../meta/provider/iceberg/IcebergFilterTest.java | 2 +-
.../meta/provider/iceberg/IcebergReadWriteIT.java | 6 +-
.../provider/iceberg/IcebergTableProviderTest.java | 2 +-
.../provider/pubsub/PubsubTableProviderIT.java | 2 +-
...tTableProviderWithFilterAndProjectPushDown.java | 6 +-
.../test/TestTableProviderWithFilterPushDown.java | 8 +-
.../test/TestTableProviderWithProjectPushDown.java | 6 +-
sdks/java/extensions/sql/zetasql/build.gradle | 75 -
.../extensions/sql/zetasql/BeamCalcRelType.java | 159 -
.../sql/zetasql/BeamJavaUdfCalcRule.java | 48 -
.../sql/zetasql/BeamZetaSqlCalcMergeRule.java | 45 -
.../extensions/sql/zetasql/BeamZetaSqlCalcRel.java | 404 --
.../sql/zetasql/BeamZetaSqlCalcRule.java | 40 -
.../sql/zetasql/BeamZetaSqlCalcSplittingRule.java | 44 -
.../extensions/sql/zetasql/BeamZetaSqlCatalog.java | 593 ---
.../extensions/sql/zetasql/BeamZetaSqlRelType.java | 85 -
.../sdk/extensions/sql/zetasql/DateTimeUtils.java | 244 --
.../sdk/extensions/sql/zetasql/QueryTrait.java | 79 -
.../sdk/extensions/sql/zetasql/SqlAnalyzer.java | 180 -
.../zetasql/SupportedZetaSqlBuiltinFunctions.java | 663 ----
.../extensions/sql/zetasql/TableResolution.java | 113 -
.../extensions/sql/zetasql/ZetaSQLPlannerImpl.java | 118 -
.../sql/zetasql/ZetaSQLQueryPlanner.java | 256 --
.../sql/zetasql/ZetaSqlBeamTranslationUtils.java | 296 --
.../zetasql/ZetaSqlCalciteTranslationUtils.java | 366 --
.../extensions/sql/zetasql/ZetaSqlException.java | 37 -
.../sdk/extensions/sql/zetasql/package-info.java | 26 -
.../translation/AggregateScanConverter.java | 283 --
.../translation/ArrayScanColumnRefToUncollect.java | 127 -
.../ArrayScanLiteralToUncollectConverter.java | 66 -
.../translation/ArrayScanToJoinConverter.java | 126 -
.../sql/zetasql/translation/ConversionContext.java | 96 -
.../zetasql/translation/ExpressionConverter.java | 965 -----
.../zetasql/translation/FilterScanConverter.java | 53 -
.../sql/zetasql/translation/JoinScanConverter.java | 108 -
.../LimitOffsetScanToLimitConverter.java | 81 -
.../LimitOffsetScanToOrderByLimitConverter.java | 116 -
.../OrderByScanUnsupportedConverter.java | 39 -
.../zetasql/translation/ProjectScanConverter.java | 50 -
.../translation/QueryStatementConverter.java | 119 -
.../sql/zetasql/translation/RelConverter.java | 100 -
.../translation/SetOperationScanConverter.java | 114 -
.../translation/SingleRowScanConverter.java | 40 -
.../SqlCaseWithValueOperatorRewriter.java | 77 -
.../translation/SqlCoalesceOperatorRewriter.java | 67 -
.../translation/SqlIfNullOperatorRewriter.java | 51 -
.../zetasql/translation/SqlInOperatorRewriter.java | 45 -
.../translation/SqlNullIfOperatorRewriter.java | 58 -
.../translation/SqlOperatorMappingTable.java | 128 -
.../zetasql/translation/SqlOperatorRewriter.java | 35 -
.../sql/zetasql/translation/SqlOperators.java | 352 --
.../translation/SqlWindowTableFunction.java | 119 -
.../sql/zetasql/translation/TVFScanConverter.java | 105 -
.../zetasql/translation/TableScanConverter.java | 120 -
.../translation/UserFunctionDefinitions.java | 81 -
.../zetasql/translation/WithRefScanConverter.java | 56 -
.../sql/zetasql/translation/WithScanConverter.java | 55 -
.../translation/ZetaSqlScalarFunctionImpl.java | 87 -
.../translation/impl/BeamBuiltinMethods.java | 73 -
.../zetasql/translation/impl/BeamCodegenUtils.java | 40 -
.../zetasql/translation/impl/CastFunctionImpl.java | 107 -
.../zetasql/translation/impl/DateFunctions.java | 41 -
.../zetasql/translation/impl/StringFunctions.java | 184 -
.../translation/impl/TimestampFunctions.java | 51 -
.../sql/zetasql/translation/impl/package-info.java | 24 -
.../sql/zetasql/translation/package-info.java | 20 -
.../zetasql/unnest/BeamZetaSqlUncollectRel.java | 120 -
.../zetasql/unnest/BeamZetaSqlUncollectRule.java | 54 -
.../sql/zetasql/unnest/BeamZetaSqlUnnestRel.java | 164 -
.../sql/zetasql/unnest/BeamZetaSqlUnnestRule.java | 116 -
.../sql/zetasql/unnest/ZetaSqlUnnest.java | 156 -
.../sql/zetasql/unnest/package-info.java | 20 -
.../sql/zetasql/BeamJavaUdfCalcRuleTest.java | 84 -
.../sql/zetasql/BeamZetaSqlCalcRelTest.java | 120 -
.../sql/zetasql/BeamZetaSqlCatalogTest.java | 165 -
.../extensions/sql/zetasql/StreamingSqlTest.java | 550 ---
.../sql/zetasql/TableResolutionTest.java | 128 -
.../beam/sdk/extensions/sql/zetasql/TestInput.java | 397 --
.../sql/zetasql/ZetaSQLPushDownTest.java | 228 --
.../zetasql/ZetaSqlBeamTranslationUtilsTest.java | 149 -
.../sql/zetasql/ZetaSqlDialectSpecTest.java | 4122 --------------------
.../extensions/sql/zetasql/ZetaSqlJavaUdfTest.java | 460 ---
.../sql/zetasql/ZetaSqlJavaUdfTypeTest.java | 586 ---
.../sql/zetasql/ZetaSqlMathFunctionsTest.java | 1034 -----
.../sql/zetasql/ZetaSqlNativeUdfTest.java | 264 --
.../sql/zetasql/ZetaSqlNumberTypesTest.java | 76 -
.../extensions/sql/zetasql/ZetaSqlTestBase.java | 97 -
.../sql/zetasql/ZetaSqlTimeFunctionsTest.java | 1779 ---------
.../extensions/sql/zetasql/ZetaSqlTypesUtils.java | 42 -
.../control/ExecutionStateSamplerBenchmark.java | 2 +-
.../java/org/apache/beam/fn/harness/Caches.java | 38 +-
.../java/org/apache/beam/fn/harness/FnHarness.java | 15 +-
.../fn/harness/control/ExecutionStateSampler.java | 81 +-
.../apache/beam/fn/harness/state/BagUserState.java | 4 +-
.../beam/fn/harness/state/FnApiStateAccessor.java | 145 +-
.../fn/harness/state/StateFetchingIterators.java | 108 +-
.../org/apache/beam/fn/harness/CachesTest.java | 14 +-
.../harness/control/ExecutionStateSamplerTest.java | 184 +-
.../harness/control/ProcessBundleHandlerTest.java | 2 +-
.../data/PCollectionConsumerRegistryTest.java | 5 +-
.../data/PTransformFunctionRegistryTest.java | 3 +-
.../harness/logging/BeamFnLoggingClientTest.java | 2 +-
.../harness/state/StateFetchingIteratorsTest.java | 12 +-
sdks/java/io/iceberg/build.gradle | 2 +-
.../apache/beam/sdk/io/iceberg/FilterUtils.java | 44 +-
.../beam/sdk/io/iceberg/FilterUtilsTest.java | 2 +-
.../java/org/apache/beam/sdk/io/jdbc/JdbcIO.java | 2 +-
.../java/org/apache/beam/sdk/io/kafka/KafkaIO.java | 31 +-
...KafkaIOReadImplementationCompatibilityTest.java | 3 +-
.../org/apache/beam/sdk/io/kafka/KafkaIOTest.java | 63 +-
.../queries/sql/SqlBoundedSideInputJoin.java | 4 +-
.../beam/sdk/nexmark/queries/sql/SqlQuery1.java | 4 +-
.../beam/sdk/nexmark/queries/sql/SqlQuery7.java | 15 +-
sdks/java/testing/tpcds/build.gradle | 4 +-
.../org/apache/beam/sdk/tpcds/QueryReader.java | 6 +-
.../apache/beam/sdk/tpcds/SqlTransformRunner.java | 4 +-
.../beam/sdk/tpcds/TableSchemaJSONLoader.java | 25 +-
sdks/python/apache_beam/dataframe/doctests.py | 2 +-
sdks/python/apache_beam/dataframe/io.py | 31 +-
.../inference/gemini_text_classification.py | 7 +-
sdks/python/apache_beam/io/debezium.py | 2 +-
sdks/python/apache_beam/io/filebasedsink.py | 168 +-
sdks/python/apache_beam/io/textio.py | 77 +-
sdks/python/apache_beam/io/textio_test.py | 431 ++
sdks/python/apache_beam/ml/transforms/base.py | 29 +-
.../python/apache_beam/options/pipeline_options.py | 9 +-
sdks/python/apache_beam/pipeline.py | 10 +-
.../runners/dataflow/dataflow_job_service_test.py | 5 +-
.../apache_beam/runners/dataflow/internal/names.py | 2 +-
.../runners/portability/flink_runner_test.py | 32 +
.../runners/portability/prism_runner.py | 4 +
sdks/python/apache_beam/transforms/core.py | 39 +-
sdks/python/apache_beam/transforms/core_test.py | 31 +
.../python/apache_beam/transforms/external_test.py | 2 +-
sdks/python/apache_beam/transforms/ptransform.py | 28 +-
.../apache_beam/transforms/ptransform_test.py | 33 +
sdks/python/apache_beam/transforms/sql.py | 2 +-
sdks/python/apache_beam/transforms/sql_test.py | 32 -
sdks/python/apache_beam/typehints/schemas.py | 12 +-
sdks/python/apache_beam/yaml/examples/README.md | 6 +-
.../yaml/examples/testing/examples_test.py | 221 +-
.../yaml/examples/testing/input_data.py | 67 +
.../ml/{sentiment_analysis => taxi_fare}/README.md | 57 +-
.../custom_nyc_taxifare_model_deployment.ipynb | 810 ++++
.../taxi_fare/streaming_taxifare_prediction.yaml | 320 ++
.../yaml/extended_tests/databases/bigquery.yaml | 79 +-
sdks/python/apache_beam/yaml/main.py | 3 +-
sdks/python/apache_beam/yaml/standard_io.yaml | 1 +
sdks/python/apache_beam/yaml/tests/create.yaml | 34 +
sdks/python/apache_beam/yaml/tests/csv.yaml | 22 +
sdks/python/apache_beam/yaml/tests/sql.yaml | 21 +-
sdks/python/apache_beam/yaml/yaml_ml.py | 33 +-
sdks/python/apache_beam/yaml/yaml_ml_test.py | 62 +
sdks/python/apache_beam/yaml/yaml_provider.py | 101 +-
sdks/python/apache_beam/yaml/yaml_transform.py | 4 +-
.../python/apache_beam/yaml/yaml_transform_test.py | 210 +
.../container/py310/base_image_requirements.txt | 60 +-
.../container/py310/ml_image_requirements.txt | 70 +-
.../container/py311/base_image_requirements.txt | 60 +-
.../container/py311/ml_image_requirements.txt | 70 +-
.../container/py312/base_image_requirements.txt | 60 +-
.../container/py312/ml_image_requirements.txt | 70 +-
.../container/py39/base_image_requirements.txt | 58 +-
.../container/py39/ml_image_requirements.txt | 68 +-
sdks/python/scripts/run_pytest.sh | 113 +-
sdks/python/setup.py | 6 +-
sdks/python/test-suites/portable/common.gradle | 5 +-
settings.gradle.kts | 1 -
vendor/calcite-1_28_0/build.gradle | 124 -
.../en/documentation/transforms/java/other/wait.md | 106 +
.../en/documentation/transforms/java/overview.md | 3 +-
.../transforms/python/other/waiton.md | 61 +
.../en/documentation/transforms/python/overview.md | 2 +
.../partials/section-menu/en/documentation.html | 2 +
412 files changed, 7701 insertions(+), 21793 deletions(-)
create mode 100644
.github/trigger_files/beam_PostCommit_Python_Portable_Flink.json
copy .github/workflows/{beam_PostCommit_PortableJar_Flink.yml =>
beam_PostCommit_Python_Portable_Flink.yml} (75%)
create mode 100644 .github/workflows/beam_UserRoles.yml
create mode 100644
examples/java/src/main/java/org/apache/beam/examples/cookbook/IcebergBatchWriteExample.java
create mode 100644
examples/java/src/main/java/org/apache/beam/examples/cookbook/IcebergRestCatalogCDCExample.java
copy examples/notebooks/beam-ml/{alloydb_product_catalog_embeddings.ipynb =>
cloudsql_postgres_product_catalog_embeddings.ipynb} (83%)
create mode 100644 infra/iam/.terraform.lock.hcl
create mode 100644 infra/iam/README.md
copy sdks/python/apache_beam/internal/code_object_pickler.py =>
infra/iam/config.auto.tfvars (86%)
create mode 100644 infra/iam/generate.py
copy sdks/java/managed/src/main/resources/config_aliases.yaml =>
infra/iam/main.tf (67%)
copy sdks/python/apache_beam/internal/code_object_pickler.py =>
infra/iam/requirements.txt (82%)
create mode 100644 infra/iam/users.tf
create mode 100644 infra/iam/users.yml
rename
sdks/java/extensions/sql/src/main/resources/{org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.commons.compiler.properties
=>
org.apache.beam.vendor.calcite.v1_40_0.org.codehaus.commons.compiler.properties}
(93%)
delete mode 100644 sdks/java/extensions/sql/zetasql/build.gradle
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamCalcRelType.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamJavaUdfCalcRule.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcMergeRule.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcRel.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcRule.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcSplittingRule.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCatalog.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlRelType.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/DateTimeUtils.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/QueryTrait.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/SqlAnalyzer.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/SupportedZetaSqlBuiltinFunctions.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/TableResolution.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSQLPlannerImpl.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSQLQueryPlanner.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlBeamTranslationUtils.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlCalciteTranslationUtils.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlException.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/package-info.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/AggregateScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ArrayScanColumnRefToUncollect.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ArrayScanLiteralToUncollectConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ArrayScanToJoinConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ConversionContext.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ExpressionConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/FilterScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/JoinScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/LimitOffsetScanToLimitConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/LimitOffsetScanToOrderByLimitConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/OrderByScanUnsupportedConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ProjectScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/QueryStatementConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/RelConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SetOperationScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SingleRowScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlCaseWithValueOperatorRewriter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlCoalesceOperatorRewriter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlIfNullOperatorRewriter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlInOperatorRewriter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlNullIfOperatorRewriter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperatorMappingTable.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperatorRewriter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlWindowTableFunction.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/TVFScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/TableScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/UserFunctionDefinitions.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/WithRefScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/WithScanConverter.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ZetaSqlScalarFunctionImpl.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/impl/BeamBuiltinMethods.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/impl/BeamCodegenUtils.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/impl/CastFunctionImpl.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/impl/DateFunctions.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/impl/StringFunctions.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/impl/TimestampFunctions.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/impl/package-info.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/package-info.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/unnest/BeamZetaSqlUncollectRel.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/unnest/BeamZetaSqlUncollectRule.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/unnest/BeamZetaSqlUnnestRel.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/unnest/BeamZetaSqlUnnestRule.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/unnest/ZetaSqlUnnest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/unnest/package-info.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamJavaUdfCalcRuleTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcRelTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCatalogTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/StreamingSqlTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/TableResolutionTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/TestInput.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSQLPushDownTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlBeamTranslationUtilsTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlDialectSpecTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlJavaUdfTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlJavaUdfTypeTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlMathFunctionsTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlNativeUdfTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlNumberTypesTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlTestBase.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlTimeFunctionsTest.java
delete mode 100644
sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlTypesUtils.java
copy sdks/python/apache_beam/yaml/examples/transforms/ml/{sentiment_analysis
=> taxi_fare}/README.md (56%)
create mode 100644
sdks/python/apache_beam/yaml/examples/transforms/ml/taxi_fare/custom_nyc_taxifare_model_deployment.ipynb
create mode 100644
sdks/python/apache_beam/yaml/examples/transforms/ml/taxi_fare/streaming_taxifare_prediction.yaml
delete mode 100644 vendor/calcite-1_28_0/build.gradle
create mode 100644
website/www/site/content/en/documentation/transforms/java/other/wait.md
create mode 100644
website/www/site/content/en/documentation/transforms/python/other/waiton.md
