This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.13.18
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 2bf0ed747a9 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks
add b2bd2a8cbe2 Refactoring: Moved hard coded environment variables' names
into constants. (#25365)
add 942e3fa5b6d Adds multi-language Python and Java quickstart release
validations (#25910)
add fcf51fb7d98 Update gen_protos to work with new protobuf on os-x.
(#25914)
add 545cadc5199 Move inference tests from py37 to py38 (#25936)
add 92abdf636c5 [Playground] Resolve issue with SCIO examples failing on
start due to a timeout (#24946)
add ea50d98048f Increase logging buffer size to 256kb (#25922)
add 43b3f88c68c [Python] TF with TFHub Notebook Example (#25933)
add 99202b237e3 Fix Illegal to use destination for jms producer (#25886)
add 40ca181ebea Update chromedriver-binary requirement in /sdks/python
(#25898)
add 1d5d86cf4b5 Update Dataflow Python container to 20230322
add 457391a0ccd Merge pull request #25948: Update Dataflow Python
container to 20230322
add c90ac7c7a0e [Python] Ignore unknown args with single dash (#25920)
add 4deedf68530 Add a source and sink option to yaml pipelines.
add 0ad52ff4a65 Merge pull request #25935 Add a source and sink option to
yaml pipelines.
add f83b29c1d01 Move license to a code block (#25950)
add fee461df025 Allow PTransforms to be applied directly to dataframes.
add 029451b32ea Add note about efficiency.
add e761360bfdf mypy
add 4484c19ae0a Merge pull request #25919 Allow PTransforms to be applied
directly to dataframes.
add 16cd2a98722 [#23106] Add periodic.Sequence and periodic.Impulse
transforms to Go SDK (#25808)
add ddae966f334 [25949] Add csv and file-schema-transform modules to
PreCommit_Java_IO checks (#25951)
add 038864fd58a Add typescript docs to beam site.
add 6065d7bebef Language consistency.
add 0b981e22fd7 Clarification about batch and streaming features.
add 68d4155b5f2 Merge pull request #25941 Add typescript docs to beam site.
add 5f635bbf558 Use released docker images rather than one from the
testing project.
add 186973b110d Merge pull request #25939 Use released docker images
rather than one from the testing project.
add 5be26977f02 yield instead of return in
tensorflow_imagenet_segmentation.py (#25952)
add 4bef4da18de Use built-in csv and json readers.
add 34485361e20 Validate this still works with a test.
add c637363fa3c Fix windows paths.
add d80f1ff499a Merge pull request #25916 Use built-in csv and json
readers.
add 1d26d399f78 [Playground] GetMetadata() endpoint (#25610)
add 3ebece3c998 Bump gradle-docker plugin for xbuild capabilities.
add 3aa3c744251 [BEAM-20807] Support Beam docker images on ARM64.
add 874b687db01 Merge pull request #25665 [BEAM-20807] Support Beam docker
images on ARM64.
add 0557c338b71 [Python] Update tensorflow doc (#25954)
add 2195f54d437 Fix :examples:java:execute so it can be used on the
command line
add 3a1b6410665 Merge pull request #25868: Fix :examples:java:execute so
it can be used on the command line
add c63a9b19d7f Remove exec calls to terraform from
playground/terraform/build.gradle.kts (#25884)
add 10611f6d2c0 Bump webpack (#25846)
add 94e72e35b69 Add links to tfma notebook for running in colab/github
(#25903)
add 0e02b8b1b6d validated the input schema matches with the bigquery
output schema (#25915)
add cff22144756 Issue25731 beam shortcut key sequence (#25876)
add 02d3084cabb Lock ReadChangeStreamPartition so only one DoFn can work
on one partition (#25938)
add bd51b1337a5 Add write permissions to perf analysis yml file (#25999)
add 07f52a47817 Fix tfhub example + instructions (#25995)
add cc428da14fc Force version of sklearn in examples (#25968)
add f50c8d503af Deprecate Cloud Debugger and its pipeline options (#25998)
add fc7a240eb4e BigQuery Storage Write API wrapper for Python SDK (#25685)
add 91bfbac7a90 Update Java and Python base image versions, resolve CI
build issue (#25992)
add a4f55afa052 Bump tensorflow (#25974)
add 08ae6c7acff [Playground] Implement Java & Go multifile examples
execution (#24874)
add 567bdaa4563 fix: Porting more BigtableOptions (#25955)
add 469101dd378 [Playground] "always_run" flag in examples' tag (#25928)
add c0f0f2d3c15 Overload HbaseIO with KV<RowKey, RowMutations> (#25864)
add 14a77825928 Add explicit language_level=3 arg to Cythonize (#26015)
add 6484a47c3d5 Fix TestPipeline.runWithAdditionalOptionArgs ignore
additionalArgs (#25937)
add 3efd3c38b99 Update pip-licenses in Dockerfile to support Python 3.11
(#26019)
add 5bdc4527a47 [#22737] Re-write Go SDK data plane to support timers.
(#25982)
add 4911a3e150b [Playground] Guard router methods from being executed on
runners (#25931)
add 7bfbdf2d0f1 Remove dead session code & noisy logging (#22241) (#26029)
add 04288fed142 use example id for cloud path (#26027)
add 5569085c0b0 [#26030] Add PTransformID to logged messages. (#26031)
add 652e30a383b Update container tags (#26022)
add c196cd15e99 Update CHANGES.md for HBaseIO.WriteRowMutations (#26032)
add 87783393936 Fix small doc typo (#26033)
add 9c614557c51 [Go SDK]: Add fileio MatchFiles, MatchAll and ReadMatches
transforms (#25809)
add 9f582e564fe Add key param support for helper Combiners (#25895)
add 52531b73cec Add #24599 to CHANGES.md (#26020)
add 915da25323d Update pillow requirement from <8,>=7.1.1 to >=7.1.1,<10
in /sdks/python (#25788)
add d9321d8cdb3 Recommend `yield from` for emitting elements from
iterables. (#25893)
add 1e306fd7bf0 refactored notebooks (#26018)
add 40382c84130 BigQueryHelpers: log statuses using PrettyString (#26037)
add 9c52e0594d6 Make DetectNewPartitions handle Splits and Merges (#25997)
add be725c8a43a Bump com.avast.gradle:gradle-docker-compose-plugin (#26008)
add 57ac3180fc5 Remove unused checkProto2Syntax from ProtobufUtil.java and
test. (#25885)
add 87a0557fd6d Add pytorch 2.0 tests (#26014)
add e0d8b58f397 new file for bq xlang tests (#26046)
add 004c7f612a5 Add documentation for the auto model updates (#25947)
add 6b006531b07 Adding support for GCS-stored files for consumer config
overrides (#25773)
add dc7a7a02c7a Add partition metadata in error logs in the restriction
tracker for change streams and delete unused files (#26040)
add 5e368b4d7ce Add partition reconciler to handle known cases where
partitions can get stuck (#26045)
add 492e2c95f52 (#25314) Have boot loaders log to logging service where
possible. (#26035)
add af5e0a85cc8 [Go SDK] Update slog in prism to fix breaking changes.
(#26053)
add 958f7fcf519 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (2bf0ed747a9)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.13.18
(958f7fcf519)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/go_tests.yml | 4 +-
.github/workflows/playground_backend_precommit.yml | 4 +-
.../workflows/playground_examples_ci_reusable.yml | 6 +-
.github/workflows/playground_frontend_test.yml | 1 -
.github/workflows/run_perf_alert_tool.yml | 2 +
...Commit_Python_CrossLanguage_Gcp_Dataflow.groovy | 55 ++
...stCommit_Python_CrossLanguage_Gcp_Direct.groovy | 55 ++
.test-infra/jenkins/job_PreCommit_Java_IOs.groovy | 2 +
CHANGES.md | 15 +
build.gradle.kts | 4 +-
buildSrc/build.gradle.kts | 4 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 182 ++++-
examples/java/build.gradle | 13 +-
.../beam-ml/run_inference_tensorflow.ipynb | 250 ++++--
.../beam-ml/run_inference_tensorflow_hub.ipynb | 560 --------------
.../run_inference_with_tensorflow_hub.ipynb | 245 ++++++
examples/notebooks/beam-ml/tfma_beam.ipynb | 28 +-
.../get-started/learn_beam_basics_by_doing.ipynb | 46 +-
.../notebooks/get-started/try-apache-beam-go.ipynb | 46 +-
.../get-started/try-apache-beam-java.ipynb | 46 +-
.../notebooks/get-started/try-apache-beam-py.ipynb | 46 +-
learning/tour-of-beam/backend/README.md | 1 +
.../backend/playground_api/api/v1/api.pb.go | 761 +++++++++++-------
.../backend/playground_api/api/v1/api_grpc.pb.go | 38 +
.../backend/playground_api/api/v1/mock.go | 56 ++
playground/README.md | 41 +-
playground/api/v1/api.proto | 15 +
playground/backend/README.md | 61 +-
.../fs_tool/go_fs.go => cmd/server/build_info.go} | 20 +-
playground/backend/cmd/server/controller.go | 78 +-
playground/backend/cmd/server/controller_test.go | 80 +-
.../containers/git-functions.gradle} | 34 +-
playground/backend/containers/go/Dockerfile | 5 +-
playground/backend/containers/go/build.gradle | 34 +-
playground/backend/containers/java/Dockerfile | 15 +-
playground/backend/containers/java/build.gradle | 28 +-
playground/backend/containers/python/Dockerfile | 10 +-
playground/backend/containers/python/build.gradle | 33 +-
playground/backend/containers/python/entrypoint.sh | 2 +-
playground/backend/containers/router/Dockerfile | 5 +-
playground/backend/containers/router/build.gradle | 5 +-
playground/backend/containers/scio/Dockerfile | 23 +-
playground/backend/containers/scio/build.gradle | 28 +-
playground/backend/internal/api/v1/api.pb.go | 761 +++++++++++-------
playground/backend/internal/api/v1/api_grpc.pb.go | 38 +
.../internal/code_processing/code_processing.go | 18 +-
.../code_processing/code_processing_test.go | 2 +-
.../backend/internal/db/datastore/datastore_db.go | 2 +-
.../backend/internal/db/dto/precompiled_object.go | 1 +
playground/backend/internal/db/entity/example.go | 1 +
playground/backend/internal/db/mapper/mapper.go | 2 +-
.../db/mapper/precompiled_object_mapper.go | 23 +-
.../db/mapper/precompiled_object_mapper_test.go | 2 +-
playground/backend/internal/environment/beam.go | 5 +-
.../internal/environment/environment_service.go | 10 +-
.../environment/environment_service_test.go | 38 +-
playground/backend/internal/executors/executor.go | 10 +-
.../backend/internal/executors/executor_builder.go | 30 +-
.../internal/executors/executor_builder_test.go | 2 +-
.../backend/internal/executors/executor_test.go | 8 +-
playground/backend/internal/fs_tool/fs.go | 42 +-
playground/backend/internal/fs_tool/fs_test.go | 10 +-
playground/backend/internal/fs_tool/go_fs.go | 4 +-
playground/backend/internal/fs_tool/go_fs_test.go | 4 +-
playground/backend/internal/fs_tool/java_fs.go | 130 +++-
.../backend/internal/fs_tool/java_fs_test.go | 215 +++++-
.../fs_tool/java_testdata/HasIncorrectMain.java} | 17 +-
.../fs_tool/java_testdata/HasMainTest1.java} | 17 +-
.../fs_tool/java_testdata/HasMainTest2.java} | 16 +-
.../internal/fs_tool/java_testdata/HasNoMain.java} | 17 +-
playground/backend/internal/fs_tool/scio_fs.go | 2 +-
.../internal/setup_tools/builder/setup_builder.go | 51 +-
.../setup_tools/builder/setup_builder_test.go | 161 +++-
.../setup_tools/life_cycle/life_cycle_setuper.go | 51 +-
.../life_cycle/life_cycle_setuper_test.go | 8 +-
playground/backend/internal/utils/common.go | 51 +-
playground/backend/internal/utils/common_test.go | 87 ---
playground/backend/internal/utils/file_utils.go | 4 +
.../backend/internal/utils/preparers_utils.go | 7 +-
.../backend/internal/utils/preparers_utils_test.go | 61 ++
.../internal/utils/test_data/JavaFileName.java} | 12 +-
.../backend/internal/utils/test_data}/wordcount.py | 0
playground/backend/new_scio_project.sh | 4 +-
playground/docker-compose.local.yaml | 4 +
playground/frontend/CONTRIBUTE.md | 19 -
playground/frontend/Dockerfile | 10 +-
playground/frontend/README.md | 22 +-
playground/frontend/build.gradle | 5 +
.../frontend/integration_test/common/common.dart | 2 +-
.../integration_test/common/common_finders.dart | 4 +-
.../miscellaneous_ui/shortcuts_modal_test.dart | 2 +-
.../integration_test/standalone_editing_test.dart | 4 +-
.../standalone_run_shortcuts_test.dart | 6 +-
playground/frontend/lib/config.example.dart | 29 -
.../lib/src/locator.dart => lib/config.g.dart} | 14 +-
playground/frontend/lib/locator.dart | 30 +-
playground/frontend/lib/main.dart | 2 +-
.../shortcuts/components/shortcuts_dialog.dart | 77 ++
.../shortcuts/components/shortcuts_modal.dart | 97 ---
.../shortcuts/constants/global_shortcuts.dart | 8 +-
.../widgets/more_actions.dart | 31 +-
.../assets/translations/en.yaml | 12 +
.../lib/playground_components.dart | 8 +
.../lib/src/api/v1/api.pb.dart | 133 ++++
.../lib/src/api/v1/api.pbgrpc.dart | 28 +
.../lib/src/api/v1/api.pbjson.dart | 23 +-
.../src/{locator.dart => build_metadata.g.dart} | 17 +-
.../lib/src/cache/example_cache.dart | 48 +-
.../lib/src/constants/backend_urls.dart | 42 +
.../lib/src/controllers/build_metadata.dart | 72 ++
.../example_loaders/content_example_loader.dart | 1 +
.../lib/src/controllers/playground_controller.dart | 12 +-
.../controllers/snippet_editing_controller.dart | 1 +
.../playground_components/lib/src/locator.dart | 2 +
.../{shortcut.dart => component_version.dart} | 46 +-
.../lib/src/models/example_base.dart | 2 +
.../content_example_loading_descriptor.dart | 11 +-
.../lib/src/models/shortcut.dart | 18 +-
.../lib/src/repositories/backend_urls.dart | 113 +++
.../src/repositories/code_client/code_client.dart | 4 +
.../repositories/code_client/grpc_code_client.dart | 26 +-
.../lib/src/repositories/code_repository.dart | 6 +-
.../example_client/example_client.dart | 3 +
.../example_client/grpc_example_client.dart | 16 +-
.../get_metadata_response_grpc_extension.dart} | 28 +-
.../lib/src/{locator.dart => util/async.dart} | 12 +-
.../lib/src/{locator.dart => util/iterable.dart} | 15 +-
.../shortcut.dart => widgets/close_button.dart} | 47 +-
.../{shortcuts_manager.dart => dialog.dart} | 63 +-
.../lib/src/widgets/shortcuts_manager.dart | 2 +-
.../lib/src/widgets/versions/versions.dart | 162 ++++
.../frontend/playground_components/pubspec.yaml | 1 +
.../test/src/common/example_cache.dart | 4 +-
.../src/common/example_repository_mock.mocks.dart | 128 +--
.../content_example_loading_descriptor_test.dart | 2 +
.../test/src/models/shortcut_test.dart | 86 +++
.../repositories/code_repository_test.mocks.dart | 125 +--
.../example_repository_test.mocks.dart | 176 +++--
.../tools/generate_build_metadata.sh | 37 +
.../lib/src/examples/python/aggregation_mean.dart | 2 +-
.../lib/src/widget_tester.dart | 2 +-
playground/frontend/pubspec.lock | 4 +-
playground/infrastructure/api/v1/api_pb2.py | 116 +--
playground/infrastructure/api/v1/api_pb2.pyi | 22 +-
playground/infrastructure/api/v1/api_pb2_grpc.py | 34 +
playground/infrastructure/checker.py | 9 +-
playground/infrastructure/ci_cd.py | 5 +-
.../cloudbuild/cloudbuild_pg_to_gke.yaml | 1 -
playground/infrastructure/config.py | 5 +-
.../infrastructure/constants.py | 8 +-
playground/infrastructure/datastore_client.py | 10 +-
playground/infrastructure/grpc_client.py | 5 +-
playground/infrastructure/helper.py | 3 +-
playground/infrastructure/models.py | 1 +
playground/terraform/README.md | 9 +-
playground/terraform/build.gradle.kts | 309 ++++----
release/src/main/scripts/run_rc_validation.sh | 251 +++++-
release/src/main/scripts/script.config | 4 +
.../beam/runners/dataflow/DataflowRunner.java | 6 +-
.../dataflow/options/CloudDebuggerOptions.java | 13 +-
.../logging/DataflowWorkerLoggingHandler.java | 10 +-
sdks/go/container/boot.go | 33 +-
sdks/go/container/boot_test.go | 16 +-
sdks/go/container/tools/logging.go | 117 +++
sdks/go/container/tools/logging_test.go | 115 +++
.../provision => container/tools}/provision.go | 19 +-
sdks/go/container/tools/provision_test.go | 110 +++
.../slowly_updating_side_input.go | 141 ++++
sdks/go/pkg/beam/core/metrics/metrics.go | 32 +-
sdks/go/pkg/beam/core/runtime/exec/data.go | 16 +-
sdks/go/pkg/beam/core/runtime/exec/datasource.go | 182 +++--
.../pkg/beam/core/runtime/exec/datasource_test.go | 115 +--
.../go/pkg/beam/core/runtime/exec/dynsplit_test.go | 14 +-
sdks/go/pkg/beam/core/runtime/harness/datamgr.go | 514 +++++++-----
.../pkg/beam/core/runtime/harness/datamgr_test.go | 403 ++++++++--
sdks/go/pkg/beam/core/runtime/harness/harness.go | 25 +-
sdks/go/pkg/beam/core/runtime/harness/logging.go | 92 ++-
.../pkg/beam/core/runtime/harness/logging_test.go | 65 +-
sdks/go/pkg/beam/core/runtime/harness/session.go | 232 ------
.../go/pkg/beam/core/runtime/harness/session.proto | 71 --
.../core/runtime/harness/session/session.pb.go | 628 ---------------
sdks/go/pkg/beam/io/fileio/example_test.go | 74 ++
sdks/go/pkg/beam/io/fileio/file.go | 142 ++++
sdks/go/pkg/beam/io/fileio/file_test.go | 254 ++++++
sdks/go/pkg/beam/io/fileio/gzip.go | 60 ++
sdks/go/pkg/beam/io/fileio/helper_test.go | 88 +++
sdks/go/pkg/beam/io/fileio/match.go | 189 +++++
sdks/go/pkg/beam/io/fileio/match_test.go | 285 +++++++
sdks/go/pkg/beam/io/fileio/read.go | 140 ++++
sdks/go/pkg/beam/io/fileio/read_test.go | 181 +++++
sdks/go/pkg/beam/provision/provision.go | 59 +-
sdks/go/pkg/beam/runners/prism/internal/coders.go | 3 +-
sdks/go/pkg/beam/runners/prism/internal/stage.go | 3 +-
.../beam/runners/prism/internal/worker/worker.go | 9 +-
.../go/pkg/beam/runners/universal/runnerlib/job.go | 4 +-
sdks/go/pkg/beam/transforms/periodic/periodic.go | 188 +++++
.../pkg/beam/transforms/periodic/periodic_test.go | 54 ++
sdks/java/container/Dockerfile | 4 +-
sdks/java/container/boot.go | 47 +-
sdks/java/container/common.gradle | 2 +
.../java/org/apache/beam/sdk/schemas/Schema.java | 6 +-
.../org/apache/beam/sdk/testing/TestPipeline.java | 23 +-
.../org/apache/beam/sdk/schemas/SchemaTest.java | 1 -
.../apache/beam/sdk/testing/TestPipelineTest.java | 26 +
...xpansionServiceSchemaTransformProviderTest.java | 14 +-
.../beam/sdk/extensions/protobuf/ProtobufUtil.java | 20 -
.../sdk/extensions/protobuf/ProtobufUtilTest.java | 31 -
.../expansion-service/build.gradle | 1 -
.../io/gcp/bigquery/BeamRowToStorageApiProto.java | 2 +-
.../beam/sdk/io/gcp/bigquery/BigQueryHelpers.java | 23 +-
.../beam/sdk/io/gcp/bigquery/BigQueryUtils.java | 11 +-
...ueryStorageWriteApiSchemaTransformProvider.java | 110 ++-
.../io/gcp/bigtable/BigtableConfigTranslator.java | 9 +
.../beam/sdk/io/gcp/bigtable/BigtableIO.java | 8 +-
.../sdk/io/gcp/bigtable/BigtableReadOptions.java | 7 +
.../io/gcp/bigtable/BigtableServiceFactory.java | 7 +-
.../sdk/io/gcp/bigtable/BigtableServiceImpl.java | 24 +-
.../sdk/io/gcp/bigtable/BigtableWriteOptions.java | 5 +
.../changestreams/ByteStringRangeHelper.java | 112 ++-
.../ChangeStreamContinuationTokenHelper.java | 47 ++
.../changestreams/ChangeStreamMetrics.java | 36 +
.../action/DetectNewPartitionsAction.java | 228 +++++-
.../action/ReadChangeStreamPartitionAction.java | 53 +-
.../changestreams/dao/MetadataTableDao.java | 162 +++-
.../reconciler/PartitionReconciler.java | 153 ++++
.../changestreams/reconciler/package-info.java | 15 +-
.../ReadChangeStreamPartitionProgressTracker.java | 3 +-
.../changestreams/restriction/StreamProgress.java | 32 +-
.../changestreams/restriction/PartitionMode.java | 35 -
.../restriction/PartitionPosition.java | 90 ---
.../restriction/PartitionRestriction.java | 141 ----
.../restriction/PartitionRestrictionClaimer.java | 136 ----
.../restriction/PartitionRestrictionMetadata.java | 106 ---
.../PartitionRestrictionProgressChecker.java | 96 ---
.../restriction/PartitionRestrictionSplitter.java | 128 ---
.../restriction/PartitionRestrictionTracker.java | 159 ----
.../ReadChangeStreamPartitionRangeTracker.java | 2 +-
.../restriction/TimestampRangeTracker.java | 24 +
.../gcp/bigquery/BeamRowToStorageApiProtoTest.java | 2 +-
...StorageWriteApiSchemaTransformProviderTest.java | 221 ++++--
.../changestreams/ByteStringRangeHelperTest.java | 108 +++
.../ChangeStreamContinuationTokenHelperTest.java | 68 ++
.../action/DetectNewPartitionsActionTest.java | 233 +++++-
.../ReadChangeStreamPartitionActionTest.java | 138 +++-
.../changestreams/dao/MetadataTableDaoTest.java | 133 +++-
.../reconciler/PartitionReconcilerTest.java | 195 +++++
...adChangeStreamPartitionProgressTrackerTest.java | 19 +
.../restriction/PartitionPositionTest.java | 70 --
.../PartitionRestrictionClaimerTest.java | 272 -------
.../PartitionRestrictionProgressCheckerTest.java | 230 ------
.../PartitionRestrictionSplitterTest.java | 168 ----
.../restriction/PartitionRestrictionTest.java | 78 --
.../PartitionRestrictionTrackerTest.java | 859 ---------------------
.../util/PartitionPositionGenerator.java | 54 --
.../sdk/io/hbase/HBaseCoderProviderRegistrar.java | 1 +
.../java/org/apache/beam/sdk/io/hbase/HBaseIO.java | 221 ++++++
.../beam/sdk/io/hbase/HBaseRowMutationsCoder.java | 150 ++++
.../beam/sdk/io/hbase/HBaseSharedConnection.java | 153 ++++
.../io/hbase/HBaseCoderProviderRegistrarTest.java | 6 +
.../sdk/io/hbase/HBaseSharedConnectionTest.java | 100 +++
.../sdk/io/hbase/HbaseIOWriteRowMutationsTest.java | 214 +++++
.../beam/sdk/io/hbase/RowMutationsCoderTest.java | 153 ++++
.../apache/beam/sdk/io/hbase/utils/HashUtils.java | 151 ++++
.../beam/sdk/io/hbase/utils/TestConstants.java | 37 +
.../beam/sdk/io/hbase/utils/TestHBaseUtils.java | 73 ++
.../java/org/apache/beam/sdk/io/jms/JmsIO.java | 3 +-
.../io/kafka/KafkaReadSchemaTransformProvider.java | 76 ++
sdks/python/apache_beam/dataframe/convert_test.py | 20 +
sdks/python/apache_beam/dataframe/frames.py | 20 +
.../apache_beam/examples/inference/README.md | 12 +-
.../sklearn_examples_requirements.txt} | 7 +-
.../sklearn_japanese_housing_regression.py | 7 +
.../inference/sklearn_mnist_classification.py | 7 +
.../inference/tensorflow_imagenet_segmentation.py | 6 +-
.../examples/ml-orchestration/tfx/requirements.txt | 2 +-
.../io/external/xlang_bigqueryio_it_test.py | 239 ++++++
sdks/python/apache_beam/io/gcp/bigquery.py | 213 ++++-
sdks/python/apache_beam/io/gcp/bigquery_tools.py | 119 +++
.../apache_beam/io/gcp/bigquery_tools_test.py | 238 ++++++
.../python/apache_beam/options/pipeline_options.py | 12 +-
.../apache_beam/options/pipeline_options_test.py | 4 +-
.../apache_beam/runners/dataflow/internal/names.py | 4 +-
.../apache-beam-jupyterlab-sidepanel/yarn.lock | 80 +-
sdks/python/apache_beam/transforms/combiners.py | 20 +-
.../apache_beam/transforms/combiners_test.py | 14 +
sdks/python/apache_beam/transforms/external.py | 30 +
.../python/apache_beam/transforms/external_test.py | 43 ++
sdks/python/apache_beam/yaml/yaml_provider.py | 16 -
sdks/python/apache_beam/yaml/yaml_transform.py | 15 +-
.../python/apache_beam/yaml/yaml_transform_test.py | 61 ++
sdks/python/container/Dockerfile | 6 +-
sdks/python/container/boot.go | 62 +-
sdks/python/container/common.gradle | 2 +
sdks/python/gen_protos.py | 60 +-
sdks/python/pytest.ini | 1 +
sdks/python/setup.py | 11 +-
sdks/python/test-suites/dataflow/common.gradle | 24 +
sdks/python/test-suites/direct/common.gradle | 25 +
sdks/python/test-suites/tox/py38/build.gradle | 4 +
sdks/python/test-suites/xlang/build.gradle | 62 ++
sdks/python/tox.ini | 14 +-
sdks/typescript/container/Dockerfile | 4 +-
sdks/typescript/container/boot.go | 39 +-
sdks/typescript/container/build.gradle | 2 +
sdks/typescript/package-lock.json | 4 +-
.../apache_beam/runners/portable_runner/runner.ts | 4 +-
settings.gradle.kts | 1 +
.../site/content/en/contribute/release-guide.md | 18 +-
.../site/content/en/documentation/ml/about-ml.md | 23 +-
.../en/documentation/ml/inference-overview.md | 4 +-
.../en/documentation/ml/side-input-updates.md | 143 ++++
.../content/en/documentation/programming-guide.md | 7 +-
.../documentation/sdks/python-machine-learning.md | 42 +-
.../content/en/documentation/sdks/typescript.md | 146 ++++
website/www/site/content/en/get-started/_index.md | 3 +-
.../content/en/get-started/quickstart/python.md | 2 +-
.../en/get-started/quickstart/typescript.md | 187 +++++
website/www/site/data/pipelines.yaml | 3 +
.../partials/section-menu/en/documentation.html | 1 +
.../partials/section-menu/en/get-started.html | 1 +
.../layouts/partials/section-menu/en/sdks.html | 12 +
.../images/logos/sdks/typescript_pipelines.png | Bin 0 -> 4864 bytes
322 files changed, 13509 insertions(+), 7233 deletions(-)
create mode 100644
.test-infra/jenkins/job_PostCommit_Python_CrossLanguage_Gcp_Dataflow.groovy
create mode 100644
.test-infra/jenkins/job_PostCommit_Python_CrossLanguage_Gcp_Direct.groovy
delete mode 100644
examples/notebooks/beam-ml/run_inference_tensorflow_hub.ipynb
create mode 100644
examples/notebooks/beam-ml/run_inference_with_tensorflow_hub.ipynb
copy playground/backend/{internal/fs_tool/go_fs.go =>
cmd/server/build_info.go} (64%)
copy
playground/{frontend/playground_components/test/src/common/example_cache.dart
=> backend/containers/git-functions.gradle} (52%)
copy playground/{frontend/playground_components/lib/src/locator.dart =>
backend/internal/fs_tool/java_testdata/HasIncorrectMain.java} (74%)
copy playground/{frontend/playground_components/lib/src/locator.dart =>
backend/internal/fs_tool/java_testdata/HasMainTest1.java} (74%)
copy playground/{frontend/playground_components/lib/src/locator.dart =>
backend/internal/fs_tool/java_testdata/HasMainTest2.java} (74%)
copy playground/{frontend/playground_components/lib/src/locator.dart =>
backend/internal/fs_tool/java_testdata/HasNoMain.java} (74%)
copy playground/{frontend/playground_components/lib/src/locator.dart =>
backend/internal/utils/test_data/JavaFileName.java} (74%)
copy {sdks/python/apache_beam/examples =>
playground/backend/internal/utils/test_data}/wordcount.py (100%)
mode change 100644 => 100755 playground/backend/new_scio_project.sh
delete mode 100644 playground/frontend/lib/config.example.dart
copy playground/frontend/{playground_components/lib/src/locator.dart =>
lib/config.g.dart} (71%)
create mode 100644
playground/frontend/lib/modules/shortcuts/components/shortcuts_dialog.dart
delete mode 100644
playground/frontend/lib/modules/shortcuts/components/shortcuts_modal.dart
copy playground/frontend/playground_components/lib/src/{locator.dart =>
build_metadata.g.dart} (68%)
create mode 100644
playground/frontend/playground_components/lib/src/constants/backend_urls.dart
create mode 100644
playground/frontend/playground_components/lib/src/controllers/build_metadata.dart
copy playground/frontend/playground_components/lib/src/models/{shortcut.dart
=> component_version.dart} (52%)
create mode 100644
playground/frontend/playground_components/lib/src/repositories/backend_urls.dart
copy
playground/frontend/{playground_components_dev/lib/src/examples/python/aggregation_mean.dart
=>
playground_components/lib/src/repositories/get_metadata_response_grpc_extension.dart}
(57%)
copy playground/frontend/playground_components/lib/src/{locator.dart =>
util/async.dart} (74%)
copy playground/frontend/playground_components/lib/src/{locator.dart =>
util/iterable.dart} (74%)
copy playground/frontend/playground_components/lib/src/{models/shortcut.dart
=> widgets/close_button.dart} (53%)
copy
playground/frontend/playground_components/lib/src/widgets/{shortcuts_manager.dart
=> dialog.dart} (50%)
create mode 100644
playground/frontend/playground_components/lib/src/widgets/versions/versions.dart
create mode 100644
playground/frontend/playground_components/test/src/models/shortcut_test.dart
create mode 100755
playground/frontend/playground_components/tools/generate_build_metadata.sh
copy sdks/python/apache_beam/examples/ml-orchestration/tfx/requirements.txt =>
playground/infrastructure/constants.py (73%)
create mode 100644 sdks/go/container/tools/logging.go
create mode 100644 sdks/go/container/tools/logging_test.go
copy sdks/go/{pkg/beam/provision => container/tools}/provision.go (83%)
create mode 100644 sdks/go/container/tools/provision_test.go
create mode 100644
sdks/go/examples/slowly_updating_side_input/slowly_updating_side_input.go
delete mode 100644 sdks/go/pkg/beam/core/runtime/harness/session.go
delete mode 100644 sdks/go/pkg/beam/core/runtime/harness/session.proto
delete mode 100644 sdks/go/pkg/beam/core/runtime/harness/session/session.pb.go
create mode 100644 sdks/go/pkg/beam/io/fileio/example_test.go
create mode 100644 sdks/go/pkg/beam/io/fileio/file.go
create mode 100644 sdks/go/pkg/beam/io/fileio/file_test.go
create mode 100644 sdks/go/pkg/beam/io/fileio/gzip.go
create mode 100644 sdks/go/pkg/beam/io/fileio/helper_test.go
create mode 100644 sdks/go/pkg/beam/io/fileio/match.go
create mode 100644 sdks/go/pkg/beam/io/fileio/match_test.go
create mode 100644 sdks/go/pkg/beam/io/fileio/read.go
create mode 100644 sdks/go/pkg/beam/io/fileio/read_test.go
create mode 100644 sdks/go/pkg/beam/transforms/periodic/periodic.go
create mode 100644 sdks/go/pkg/beam/transforms/periodic/periodic_test.go
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/ChangeStreamContinuationTokenHelper.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/reconciler/PartitionReconciler.java
copy playground/frontend/playground_components/lib/src/locator.dart =>
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/reconciler/package-info.java
(73%)
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionMode.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionPosition.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestriction.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionClaimer.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionMetadata.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionProgressChecker.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionSplitter.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionTracker.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/ChangeStreamContinuationTokenHelperTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/reconciler/PartitionReconcilerTest.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionPositionTest.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionClaimerTest.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionProgressCheckerTest.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionSplitterTest.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionTest.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/restriction/PartitionRestrictionTrackerTest.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/util/PartitionPositionGenerator.java
create mode 100644
sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseRowMutationsCoder.java
create mode 100644
sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseSharedConnection.java
create mode 100644
sdks/java/io/hbase/src/test/java/org/apache/beam/sdk/io/hbase/HBaseSharedConnectionTest.java
create mode 100644
sdks/java/io/hbase/src/test/java/org/apache/beam/sdk/io/hbase/HbaseIOWriteRowMutationsTest.java
create mode 100644
sdks/java/io/hbase/src/test/java/org/apache/beam/sdk/io/hbase/RowMutationsCoderTest.java
create mode 100644
sdks/java/io/hbase/src/test/java/org/apache/beam/sdk/io/hbase/utils/HashUtils.java
create mode 100644
sdks/java/io/hbase/src/test/java/org/apache/beam/sdk/io/hbase/utils/TestConstants.java
create mode 100644
sdks/java/io/hbase/src/test/java/org/apache/beam/sdk/io/hbase/utils/TestHBaseUtils.java
copy sdks/python/apache_beam/examples/{ml-orchestration/tfx/requirements.txt
=> inference/sklearn_examples_requirements.txt} (84%)
create mode 100644
sdks/python/apache_beam/io/external/xlang_bigqueryio_it_test.py
create mode 100644 sdks/python/test-suites/xlang/build.gradle
create mode 100644
website/www/site/content/en/documentation/ml/side-input-updates.md
create mode 100644 website/www/site/content/en/documentation/sdks/typescript.md
create mode 100644
website/www/site/content/en/get-started/quickstart/typescript.md
create mode 100644
website/www/site/static/images/logos/sdks/typescript_pipelines.png
