This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.16.0
in repository https://gitbox.apache.org/repos/asf/beam.git
omit 1068d04a2ab Bump github.com/testcontainers/testcontainers-go in /sdks
add d50924d2ae9 [Spark Dataset runner] Fix collection encoder bug that may
lead to corrupted data due to naive usage of unsafe storage (fixes #25296).
(#25297)
add 72781cb089a [Spark Dataset runner] Break linage of dataset to reduce
Spark planning overhead in case of large complex query plans (relates to #24710
and #23845) (#25187)
add 86480f33397 Fix flaky test due to create bigquery dataset conflict
(#25266)
add b3be8b65aff 24472 Implement FileWriteSchemaTransformProvider (#24806)
add 0441d9d5823 Embed ML video to docs (#25302)
add d68b02804a5 skip automated expansion test (#25304)
add ed699f4504f Add banner highlighting beam ml (#25306)
add b1c9d8aec07 Optimize to use cached output receiver instead of creating
one on DoFn invocation #21250 (#25245)
add c65e777d6c6 Optimize PGBK table to only update cache when there is a
large enough size change. #21250 (#25219)
add 679d30256c6 Swap setting a context from being on the hot path when we
emit elements to only be done during bundle creation and teardown #21250
(#25291)
add 4e6a3e41aba Basic SchemaTransform implementation for SQLTransform.
(#25177)
add 16cb63be7e0 issue24170 google colab link added (#24820)
add aee2c844ad8 Clarify llm download/loading instructions (#25145)
add bb582d8db12 Allow for setMaxRetryJobs in BigQueryIO to be configurable
(#25224)
add fd4ea9b6bce delete file used in internal testing (#25339)
add bbe27e96de0 Don't discard output column names
add 9178456b38f Merge pull request #25290: Don't discard output column
names
add cd8eb0c9c68 Fix output timestamp for multi output receiver in
FnApiDoFnRunner #25344 (#25349)
add dc632029f0c Change UnboundedScheduledExecutorService to avoid creating
threads when (#25300)
add d20d0b01c3c Upgrading spring-expression to latest patch version
(#25348)
add 2a350761d09 Better batching for higher fixed costs.
add d6d88808aa4 Add an option to get the old behavior.
add 37112d65578 lint
add 46a5e77232d Merge pull request #25351 Better batching for higher fixed
costs.
add bf5114bb2fc [BEAM-12164] Enforced only positive state transitions from
CREATED -> SCHEDULED -> RUNNING -> FINISHED for the change stream metadata
table. (#25311)
add a9e80d2981c Added Role-based access control integration tests for
Spanner Change Streams (#25246)
add 88da381cc33 Fail the pipeline when a mismatched Python or Beam version
is detected. (#25313)
add 063eea63345 [Spark runner] Removal of Spark 2 runner support (closes
#25259) (#25263)
add 0bec597aabe Add TensorRT runinference example for Text Classification
(#25226)
add b90ed151467 Bump cryptography from 39.0.0 to 39.0.1 in
/sdks/python/container/py39 (#25377)
add e84a3e8e001 Playground Frontend Test workflow (#24728) (#25254)
add 35af801a3d6 Simpify composite literal in metrics_test.go (#25384)
add 338ddec28c8 Replace use of deprecated strings.Title function (#25385)
add e967d6ab74d Minor updates to typescript generation.
add bf09a5ed4a5 Merge pull request #25260 Minor updates to typescript
generation.
add 287ddf6aa79 Explicitly set mongo container version for testing (#25369)
add 69ddf44e67d Update upper bound for numpy. (#24725)
add 2965ef2095d Support batching as config in RunInference (sklearn and
pytorch) (#25370)
add 9fcb3a5b48a Add a note on increased lower bounds. (#25389)
add ea1625ab5f3 Support samza portable UDF metrics. (#25265)
add 513f4b003b1 Adding support for @SchemaFieldDescription annotation that
allows ann… (#25268)
add d36f34df570 Adding SpannerIO.readChangeStreams support for
SchemaTransform (#24999)
add 2b15208feb8 Bump google.golang.org/api from 0.108.0 to 0.109.0 in
/sdks (#25250)
add 47353efb9d5 Add min and max batch size args to model handler (#25395)
add 840ea6e8bc0 Beam/sdks/io/gcp/java/healcare/hl7v2 io read (#25056)
add a96138fa3c2 Bump github.com/testcontainers/testcontainers-go in /sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (1068d04a2ab)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.16.0
(a96138fa3c2)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/playground_frontend_test.yml | 91 ++++
...ommit_CrossLanguageValidatesRunner_Spark.groovy | 51 ---
.../job_PostCommit_Java_Examples_Spark.groovy | 1 -
...PortableValidatesRunner_Spark2_Streaming.groovy | 43 --
...Java_PortableValidatesRunner_Spark_Batch.groovy | 2 -
...ob_PostCommit_Java_ValidatesRunner_Spark.groovy | 1 -
...ValidatesRunner_SparkStructuredStreaming.groovy | 1 -
.test-infra/tools/stale_bq_datasets_cleaner.sh | 2 +-
CHANGES.md | 8 +-
build.gradle.kts | 6 -
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 5 +-
playground/backend/containers/java/Dockerfile | 2 +-
playground/frontend/README.md | 11 +-
.../frontend/assets/{github.svg => colab.svg} | 10 +-
playground/frontend/assets/github.svg | 4 +-
playground/frontend/assets/translations/en.yaml | 3 +
playground/frontend/build.gradle | 5 +-
.../lib/components/link_button/colab_button.dart | 31 +-
.../lib/components/link_button/dataset_button.dart | 36 +-
.../lib/components/link_button/github_button.dart | 29 +-
.../lib/components/link_button/link_button.dart | 63 +++
playground/frontend/lib/config.example.dart | 15 +-
playground/frontend/lib/l10n/app_en.arb | 4 -
.../description_popover/description_popover.dart | 30 +-
.../examples/components/example_actions.dart | 52 +++
.../widgets/embedded_actions.dart | 58 ++-
playground/frontend/lib/src/assets/assets.gen.dart | 2 +
.../lib/playground_components.dart | 1 +
.../lib/src/models/example.dart | 6 +-
.../lib/src/models/example_base.dart | 6 +-
.../example_client/grpc_example_client.dart | 5 +-
.../src/main/scripts/build_release_candidate.sh | 2 +
.../runtime/SamzaMetricsBundleProgressHandler.java | 10 +-
runners/spark/2/build.gradle | 33 --
runners/spark/2/job-server/build.gradle | 31 --
runners/spark/2/job-server/container/build.gradle | 27 --
.../spark/structuredstreaming/Constants.java | 25 --
.../SparkStructuredStreamingPipelineOptions.java | 42 --
.../SparkStructuredStreamingPipelineResult.java | 134 ------
.../SparkStructuredStreamingRunner.java | 228 ----------
.../SparkStructuredStreamingRunnerRegistrar.java | 54 ---
.../aggregators/AggregatorsAccumulator.java | 73 ---
.../aggregators/NamedAggregators.java | 113 -----
.../aggregators/NamedAggregatorsAccumulator.java | 63 ---
.../aggregators/package-info.java | 20 -
.../structuredstreaming/examples/WordCount.java | 132 ------
.../metrics/AggregatorMetric.java | 74 ---
.../metrics/AggregatorMetricSource.java | 49 --
.../structuredstreaming/metrics/BeamMetricSet.java | 60 ---
.../metrics/MetricsAccumulator.java | 76 ----
.../MetricsContainerStepMapAccumulator.java | 65 ---
.../metrics/SparkBeamMetric.java | 107 -----
.../metrics/SparkBeamMetricSource.java | 48 --
.../metrics/SparkMetricsContainerStepMap.java | 43 --
.../metrics/WithMetricsSupport.java | 91 ----
.../structuredstreaming/metrics/package-info.java | 20 -
.../metrics/sink/CodahaleCsvSink.java | 86 ----
.../metrics/sink/CodahaleGraphiteSink.java | 89 ----
.../metrics/sink/package-info.java | 20 -
.../spark/structuredstreaming/package-info.java | 20 -
.../translation/AbstractTranslationContext.java | 235 ----------
.../translation/PipelineTranslator.java | 202 ---------
.../translation/SparkSessionFactory.java | 71 ---
.../translation/SparkTransformOverrides.java | 56 ---
.../translation/TranslationContext.java | 34 --
.../translation/batch/AggregatorCombiner.java | 270 -----------
.../batch/CombinePerKeyTranslatorBatch.java | 114 -----
.../CreatePCollectionViewTranslatorBatch.java | 60 ---
.../translation/batch/DatasetSourceBatch.java | 160 -------
.../translation/batch/DoFnFunction.java | 164 -------
.../translation/batch/DoFnRunnerWithMetrics.java | 104 -----
.../translation/batch/FlattenTranslatorBatch.java | 65 ---
.../batch/GroupByKeyTranslatorBatch.java | 90 ----
.../translation/batch/ImpulseTranslatorBatch.java | 49 --
.../translation/batch/ParDoTranslatorBatch.java | 257 -----------
.../translation/batch/PipelineTranslatorBatch.java | 101 -----
.../translation/batch/ProcessContext.java | 138 ------
.../batch/ReadSourceTranslatorBatch.java | 88 ----
.../batch/ReshuffleTranslatorBatch.java | 30 --
.../batch/WindowAssignTranslatorBatch.java | 61 ---
.../GroupAlsoByWindowViaOutputBufferFn.java | 173 --------
.../batch/functions/NoOpStepContext.java | 36 --
.../batch/functions/SparkSideInputReader.java | 186 --------
.../translation/batch/functions/package-info.java | 20 -
.../translation/batch/package-info.java | 20 -
.../translation/helpers/CoderHelpers.java | 63 ---
.../translation/helpers/EncoderFactory.java | 49 --
.../translation/helpers/EncoderHelpers.java | 71 ---
.../translation/helpers/MultiOutputCoder.java | 84 ----
.../translation/helpers/RowHelpers.java | 75 ----
.../translation/helpers/SchemaHelpers.java | 39 --
.../translation/helpers/SideInputBroadcast.java | 49 --
.../translation/helpers/WindowingHelpers.java | 82 ----
.../translation/helpers/package-info.java | 20 -
.../translation/package-info.java | 20 -
.../streaming/DatasetSourceStreaming.java | 260 -----------
.../streaming/PipelineTranslatorStreaming.java | 93 ----
.../streaming/ReadSourceTranslatorStreaming.java | 87 ----
.../translation/streaming/package-info.java | 20 -
.../translation/utils/CachedSideInputReader.java | 93 ----
.../translation/utils/SideInputStorage.java | 107 -----
.../translation/utils/package-info.java | 20 -
.../structuredstreaming/SparkSessionRule.java | 88 ----
...parkStructuredStreamingRunnerRegistrarTest.java | 70 ---
.../StructuredStreamingPipelineStateTest.java | 225 ----------
.../aggregators/metrics/sink/InMemoryMetrics.java | 80 ----
.../metrics/sink/SparkMetricsSinkTest.java | 73 ---
.../metrics/SparkBeamMetricTest.java | 59 ---
.../translation/batch/CombineTest.java | 186 --------
.../translation/batch/ComplexSourceTest.java | 86 ----
.../translation/batch/FlattenTest.java | 59 ---
.../translation/batch/GroupByKeyTest.java | 124 ------
.../translation/batch/ParDoTest.java | 153 -------
.../translation/batch/SimpleSourceTest.java | 53 ---
.../translation/batch/WindowAssignTest.java | 69 ---
.../translation/helpers/EncoderHelpersTest.java | 98 ----
.../translation/streaming/SimpleSourceTest.java | 57 ---
.../utils/SerializationDebugger.java | 115 -----
.../structuredstreaming/utils/package-info.java | 20 -
.../translation/EvaluationContext.java | 10 +-
.../translation/PipelineTranslator.java | 97 +++-
.../translation/TransformTranslator.java | 13 +-
.../batch/CombineGloballyTranslatorBatch.java | 4 +
.../batch/CombineGroupedValuesTranslatorBatch.java | 4 +
.../batch/CombinePerKeyTranslatorBatch.java | 4 +
.../translation/batch/FlattenTranslatorBatch.java | 4 +
.../batch/GroupByKeyTranslatorBatch.java | 5 +-
.../translation/batch/ImpulseTranslatorBatch.java | 4 +
.../translation/batch/ParDoTranslatorBatch.java | 6 +-
.../batch/ReadSourceTranslatorBatch.java | 7 +-
.../batch/ReshuffleTranslatorBatch.java | 8 +
.../batch/WindowAssignTranslatorBatch.java | 4 +
.../translation/helpers/EncoderHelpers.java | 19 +-
runners/spark/spark_runner.gradle | 16 +-
.../spark/translation/SparkContextFactory.java | 8 +-
.../SparkStreamingPortablePipelineTranslator.java | 4 +-
.../spark/translation/TransformTranslator.java | 11 +-
.../streaming/StreamingTransformTranslator.java | 4 +-
.../beam/runners/spark/util/SparkCompat.java | 117 -----
sdks/go.mod | 2 +-
sdks/go.sum | 4 +-
sdks/go/cmd/specialize/main.go | 5 +-
.../go/pkg/beam/core/runtime/xlangx/expand_test.go | 39 --
.../runners/dataflow/dataflowlib/metrics_test.go | 2 +-
sdks/go/pkg/beam/util/shimx/generate.go | 5 +-
.../sdk/schemas/FieldValueTypeInformation.java | 18 +
.../annotations/SchemaFieldDescription.java | 62 +++
.../sdk/schemas/utils/StaticSchemaInference.java | 11 +-
.../beam/sdk/transforms/DoFnOutputReceivers.java | 2 +-
.../util/UnboundedScheduledExecutorService.java | 28 +-
.../beam/sdk/schemas/AutoValueSchemaTest.java | 32 ++
.../beam/sdk/schemas/JavaBeanSchemaTest.java | 11 +
.../beam/sdk/schemas/JavaFieldSchemaTest.java | 9 +
.../beam/sdk/schemas/utils/TestJavaBeans.java | 23 +
.../apache/beam/sdk/schemas/utils/TestPOJOs.java | 39 ++
.../UnboundedScheduledExecutorServiceTest.java | 54 +++
.../extensions/sql/expansion-service/build.gradle | 1 +
.../SqlTransformSchemaTransformProvider.java | 232 ++++++++++
.../beam/sdk/extensions/sql/SqlTransform.java | 3 +-
.../extensions/sql/impl/planner/BeamRuleSets.java | 4 -
.../sql/impl/rel/AbstractBeamCalcRel.java | 10 +
.../sql/impl/rule/LogicalCalcMergeRule.java | 42 --
.../extensions/sql/BeamSqlMultipleSchemasTest.java | 21 +
.../sql/impl/rule/IOPushDownRuleTest.java | 2 -
...tTableProviderWithFilterAndProjectPushDown.java | 2 -
.../test/TestTableProviderWithFilterPushDown.java | 2 -
.../test/TestTableProviderWithProjectPushDown.java | 2 -
.../sql/zetasql/ZetaSQLQueryPlanner.java | 15 +-
.../jmh/PrecombineGroupingTableBenchmark.java | 87 ++--
.../java/org/apache/beam/fn/harness/Caches.java | 43 +-
.../apache/beam/fn/harness/FnApiDoFnRunner.java | 309 ++++++++++++-
.../beam/fn/harness/PrecombineGroupingTable.java | 10 +-
.../fn/harness/control/ExecutionStateSampler.java | 109 ++++-
.../fn/harness/control/ProcessBundleHandler.java | 57 +--
.../harness/data/PCollectionConsumerRegistry.java | 74 +--
.../harness/data/PTransformFunctionRegistry.java | 26 +-
.../fn/harness/PrecombineGroupingTableTest.java | 33 +-
.../harness/control/ExecutionStateSamplerTest.java | 144 ++++++
.../harness/control/ProcessBundleHandlerTest.java | 11 +-
.../data/PCollectionConsumerRegistryTest.java | 145 +++---
.../data/PTransformFunctionRegistryTest.java | 81 ++--
.../beam/sdk/io/common/SchemaAwareJavaBeans.java | 494 +++++++++++++++++++++
.../sdk/io/common/SchemaAwareJavaBeansTest.java | 175 ++++++++
.../OWNERS | 3 +
.../build.gradle | 20 +-
.../AvroWriteSchemaTransformFormatProvider.java | 88 ++++
.../CsvWriteSchemaTransformFormatProvider.java | 29 +-
.../FileWriteSchemaTransformConfiguration.java | 66 +--
.../FileWriteSchemaTransformFormatProvider.java | 16 +-
.../FileWriteSchemaTransformFormatProviders.java | 148 ++++++
.../FileWriteSchemaTransformProvider.java | 191 ++++++++
.../JsonWriteSchemaTransformFormatProvider.java | 93 ++++
.../ParquetWriteSchemaTransformFormatProvider.java | 117 +++++
.../io/fileschematransform/XmlDateTimeAdapter.java | 34 +-
.../sdk/io/fileschematransform/XmlRowAdapter.java | 93 ++++
.../sdk/io/fileschematransform/XmlRowValue.java | 268 +++++++++++
.../XmlWriteSchemaTransformFormatProvider.java | 115 +++++
.../sdk/io/fileschematransform/package-info.java | 0
...FileWriteSchemaTransformFormatProviderTest.java | 104 +++++
...FileWriteSchemaTransformFormatProviderTest.java | 434 ++++++++++++++++++
...WriteSchemaTransformFormatProviderTestData.java | 231 ++++++++++
...ileWriteSchemaTransformFormatProvidersTest.java | 17 +-
.../FileWriteSchemaTransformProviderTest.java | 127 ++++++
...FileWriteSchemaTransformFormatProviderTest.java | 104 +++++
...FileWriteSchemaTransformFormatProviderTest.java | 112 +++++
.../XmlDateTimeAdapterTest.java} | 32 +-
...FileWriteSchemaTransformFormatProviderTest.java | 120 +++++
.../io/fileschematransform/XmlRowAdapterTest.java | 190 ++++++++
.../io/fileschematransform/XmlRowValueTest.java | 391 ++++++++++++++++
.../FileWriteSchemaTransformFormatProviders.java | 124 ------
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 12 +-
.../apache/beam/sdk/io/gcp/healthcare/HL7v2IO.java | 272 +++++++++---
.../beam/sdk/io/gcp/healthcare/HL7v2Message.java | 26 ++
.../sdk/io/gcp/healthcare/HL7v2ReadParameter.java | 85 ++++
.../sdk/io/gcp/healthcare/HL7v2ReadResponse.java | 96 ++++
.../io/gcp/healthcare/HL7v2ReadResponseCoder.java | 58 +++
.../beam/sdk/io/gcp/spanner/ReadSpannerSchema.java | 37 +-
.../beam/sdk/io/gcp/spanner/SpannerSchema.java | 8 +-
...erChangestreamsReadSchemaTransformProvider.java | 394 ++++++++++++++++
.../action/DetectNewPartitionsAction.java | 2 +-
.../action/QueryChangeStreamAction.java | 2 +-
.../changestreams/dao/PartitionMetadataDao.java | 85 +++-
.../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 14 +
.../sdk/io/gcp/healthcare/HL7v2IOReadWriteIT.java | 84 ++++
.../beam/sdk/io/gcp/healthcare/HL7v2IOTest.java | 41 +-
.../dao/PartitionMetadataDaoTest.java | 108 ++++-
.../it/ChangeStreamTestPipelineOptions.java | 8 +-
.../changestreams/it/IntegrationTestEnv.java | 62 ++-
.../changestreams/it/SpannerChangeStreamIT.java | 40 +-
.../it/SpannerChangeStreamsSchemaTransformIT.java | 262 +++++++++++
sdks/java/io/kafka/build.gradle | 2 +-
sdks/java/testing/watermarks/build.gradle | 19 -
sdks/python/apache_beam/coders/row_coder_test.py | 3 +-
.../inference/large_language_modeling/main.py | 4 +-
.../inference/pytorch_image_classification.py | 10 +-
.../inference/tensorrt_text_classification.py | 126 ++++++
.../io/gcp/big_query_query_to_table_it_test.py | 6 +-
.../apache_beam/io/gcp/bigquery_file_loads_test.py | 8 +-
.../apache_beam/io/gcp/bigquery_read_it_test.py | 10 +-
sdks/python/apache_beam/io/gcp/bigquery_test.py | 14 +-
.../apache_beam/io/gcp/bigquery_write_it_test.py | 8 +-
sdks/python/apache_beam/io/gcp/tests/utils.py | 6 +-
.../apache_beam/ml/inference/pytorch_inference.py | 24 +-
.../ml/inference/pytorch_inference_test.py | 92 ++++
.../apache_beam/ml/inference/sklearn_inference.py | 24 +-
.../ml/inference/sklearn_inference_test.py | 68 +++
.../apache_beam/ml/inference/tensorrt_inference.py | 11 +-
.../python/apache_beam/options/pipeline_options.py | 5 +-
sdks/python/apache_beam/pipeline_test.py | 16 +
.../runners/portability/spark_runner.py | 4 +-
.../portability/spark_uber_jar_job_server.py | 4 +-
.../apache_beam/runners/worker/bundle_processor.py | 23 +
sdks/python/apache_beam/transforms/environments.py | 9 +-
sdks/python/apache_beam/transforms/util.py | 40 +-
sdks/python/apache_beam/transforms/util_test.py | 18 +-
sdks/python/build-requirements.txt | 2 +-
.../container/py310/base_image_requirements.txt | 2 +-
.../container/py38/base_image_requirements.txt | 2 +-
.../container/py39/base_image_requirements.txt | 4 +-
sdks/python/setup.py | 15 +-
sdks/python/test-suites/direct/common.gradle | 3 +-
sdks/python/test-suites/tox/py38/build.gradle | 26 +-
sdks/python/tox.ini | 15 +-
sdks/typescript/package.json | 1 +
settings.gradle.kts | 5 +-
website/www/site/assets/scss/_banner.sass | 43 ++
website/www/site/assets/scss/_hero.scss | 3 +-
website/www/site/assets/scss/_navbar-desktop.scss | 3 +-
website/www/site/assets/scss/main.scss | 1 +
.../en/documentation/ml/large-language-modeling.md | 26 +-
.../site/content/en/documentation/ml/overview.md | 5 +-
.../en/documentation/ml/tensorrt-runinference.md | 150 +++++++
.../site/content/en/documentation/runners/spark.md | 2 +-
.../get-started/resources/videos-and-podcasts.md | 7 +
website/www/site/layouts/partials/header.html | 6 +
.../partials/section-menu/en/documentation.html | 1 +
website/www/site/static/images/banner_desktop.jpg | Bin 0 -> 111671 bytes
website/www/site/static/images/banner_mobile.jpg | Bin 0 -> 59518 bytes
278 files changed, 7936 insertions(+), 8782 deletions(-)
create mode 100644 .github/workflows/playground_frontend_test.yml
delete mode 100644
.test-infra/jenkins/job_PostCommit_CrossLanguageValidatesRunner_Spark.groovy
delete mode 100644
.test-infra/jenkins/job_PostCommit_Java_PortableValidatesRunner_Spark2_Streaming.groovy
copy playground/frontend/assets/{github.svg => colab.svg} (53%)
rename
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/KVHelpers.java
=> playground/frontend/lib/components/link_button/colab_button.dart (58%)
rename
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/utils/ScalaInterop.java
=> playground/frontend/lib/components/link_button/dataset_button.dart (50%)
rename
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/metrics/sink/InMemoryMetricsSinkRule.java
=> playground/frontend/lib/components/link_button/github_button.dart (59%)
create mode 100644
playground/frontend/lib/components/link_button/link_button.dart
create mode 100644
playground/frontend/lib/modules/examples/components/example_actions.dart
delete mode 100644 runners/spark/2/build.gradle
delete mode 100644 runners/spark/2/job-server/build.gradle
delete mode 100644 runners/spark/2/job-server/container/build.gradle
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/Constants.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingPipelineOptions.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingPipelineResult.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingRunner.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingRunnerRegistrar.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/AggregatorsAccumulator.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/NamedAggregators.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/NamedAggregatorsAccumulator.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/package-info.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/examples/WordCount.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/AggregatorMetric.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/AggregatorMetricSource.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/BeamMetricSet.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/MetricsAccumulator.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/MetricsContainerStepMapAccumulator.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkBeamMetric.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkBeamMetricSource.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkMetricsContainerStepMap.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/WithMetricsSupport.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/package-info.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/sink/CodahaleCsvSink.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/sink/CodahaleGraphiteSink.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/sink/package-info.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/package-info.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/AbstractTranslationContext.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/PipelineTranslator.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/SparkSessionFactory.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/SparkTransformOverrides.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/TranslationContext.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/AggregatorCombiner.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CombinePerKeyTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CreatePCollectionViewTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DatasetSourceBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DoFnFunction.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DoFnRunnerWithMetrics.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/FlattenTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/GroupByKeyTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ImpulseTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ParDoTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/PipelineTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ProcessContext.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ReadSourceTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ReshuffleTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/WindowAssignTranslatorBatch.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/functions/GroupAlsoByWindowViaOutputBufferFn.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/functions/NoOpStepContext.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/functions/SparkSideInputReader.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/functions/package-info.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/package-info.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/CoderHelpers.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/EncoderFactory.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/EncoderHelpers.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/MultiOutputCoder.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/RowHelpers.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/SchemaHelpers.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/SideInputBroadcast.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/WindowingHelpers.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/package-info.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/package-info.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/DatasetSourceStreaming.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/PipelineTranslatorStreaming.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/ReadSourceTranslatorStreaming.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/package-info.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/utils/CachedSideInputReader.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/utils/SideInputStorage.java
delete mode 100644
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/utils/package-info.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/SparkSessionRule.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingRunnerRegistrarTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/StructuredStreamingPipelineStateTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/metrics/sink/InMemoryMetrics.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/metrics/sink/SparkMetricsSinkTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkBeamMetricTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CombineTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ComplexSourceTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/FlattenTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/GroupByKeyTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ParDoTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/SimpleSourceTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/WindowAssignTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/EncoderHelpersTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/SimpleSourceTest.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/utils/SerializationDebugger.java
delete mode 100644
runners/spark/2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/utils/package-info.java
delete mode 100644
runners/spark/src/main/java/org/apache/beam/runners/spark/util/SparkCompat.java
delete mode 100644 sdks/go/pkg/beam/core/runtime/xlangx/expand_test.go
create mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/annotations/SchemaFieldDescription.java
create mode 100644
sdks/java/extensions/sql/expansion-service/src/main/java/org/apache/beam/sdk/extensions/sql/expansion/SqlTransformSchemaTransformProvider.java
delete mode 100644
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/LogicalCalcMergeRule.java
create mode 100644
sdks/java/io/common/src/test/java/org/apache/beam/sdk/io/common/SchemaAwareJavaBeans.java
create mode 100644
sdks/java/io/common/src/test/java/org/apache/beam/sdk/io/common/SchemaAwareJavaBeansTest.java
rename sdks/java/io/{fileschematransform => file-schema-transform}/OWNERS (55%)
rename sdks/java/io/{fileschematransform =>
file-schema-transform}/build.gradle (70%)
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/AvroWriteSchemaTransformFormatProvider.java
rename
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/TransformTranslator.java
=>
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/CsvWriteSchemaTransformFormatProvider.java
(50%)
rename sdks/java/io/{fileschematransform =>
file-schema-transform}/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformConfiguration.java
(77%)
rename sdks/java/io/{fileschematransform =>
file-schema-transform}/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProvider.java
(66%)
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProviders.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformProvider.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/JsonWriteSchemaTransformFormatProvider.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/ParquetWriteSchemaTransformFormatProvider.java
rename
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/CompositeSource.java
=>
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlDateTimeAdapter.java
(50%)
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlRowAdapter.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlRowValue.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlWriteSchemaTransformFormatProvider.java
rename sdks/java/io/{fileschematransform =>
file-schema-transform}/src/main/java/org/apache/beam/sdk/io/fileschematransform/package-info.java
(100%)
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/AvroFileWriteSchemaTransformFormatProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProviderTestData.java
copy sdks/java/io/{fileschematransform =>
file-schema-transform}/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProvidersTest.java
(64%)
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/JsonFileWriteSchemaTransformFormatProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/ParquetFileWriteSchemaTransformFormatProviderTest.java
rename
sdks/java/io/{fileschematransform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProvidersTest.java
=>
file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/XmlDateTimeAdapterTest.java}
(56%)
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/XmlFileWriteSchemaTransformFormatProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/XmlRowAdapterTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/XmlRowValueTest.java
delete mode 100644
sdks/java/io/fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProviders.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2ReadParameter.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2ReadResponse.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2ReadResponseCoder.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/SpannerChangestreamsReadSchemaTransformProvider.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/it/SpannerChangeStreamsSchemaTransformIT.java
create mode 100644
sdks/python/apache_beam/examples/inference/tensorrt_text_classification.py
create mode 100644 website/www/site/assets/scss/_banner.sass
create mode 100644
website/www/site/content/en/documentation/ml/tensorrt-runinference.md
create mode 100644 website/www/site/static/images/banner_desktop.jpg
create mode 100644 website/www/site/static/images/banner_mobile.jpg
