This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/cloud.google.com/go/bigquery-1.42.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard c4641056533 Bump cloud.google.com/go/bigquery from 1.41.0 to 1.42.0 in
/sdks
add cf790b50cc5 Do not use .get() on ValueProvider during pipeline creation
add 6b7d8b19299 Merge pull request #23294: SpannerIO - Do not use .get()
on ValueProvider during pipeline creation
add 762edd7f3a6 Improved pipeline translation in
SparkStructuredStreamingRunner (#22446)
add a6cda1370b3 use avro DataFileReader to read avro container files
add 483a0c95734 Merge pull request #23214: Use avro DataFileReader to read
avro container files
add 5af5a1fb201 Change google_cloud_bigdataoss_version to 2.2.8. (#23300)
add 0cefd123ca9 Fix Nexmark default log level
add 19f5e62c61d Merge pull request #23337: Fix Nexmark default log level
add 90739533a8c Bump cloud.google.com/go/storage from 1.26.0 to 1.27.0 in
/sdks (#23336)
add 5a0fa4a1514 Bump cloud.google.com/go/bigquery from 1.41.0 to 1.42.0 in
/sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (c4641056533)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/cloud.google.com/go/bigquery-1.42.0
(5a0fa4a1514)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
CHANGES.md | 1 +
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +-
.../translation/helpers/EncoderFactory.java | 12 +-
.../utils/{package-info.java => ScalaInterop.java} | 24 +-
.../spark/structuredstreaming/Constants.java | 25 -
.../SparkStructuredStreamingRunner.java | 51 +-
.../io/BoundedDatasetFactory.java | 324 +++++++++++
.../structuredstreaming}/io/package-info.java | 2 +-
.../metrics/WithMetricsSupport.java | 2 +-
.../translation/AbstractTranslationContext.java | 235 --------
.../translation/PipelineTranslator.java | 57 +-
.../translation/TransformTranslator.java | 198 ++++++-
.../translation/TranslationContext.java | 124 ++++-
.../translation/batch/AggregatorCombiner.java | 270 ----------
.../translation/batch/Aggregators.java | 591 +++++++++++++++++++++
.../batch/CombineGloballyTranslatorBatch.java | 121 +++++
.../batch/CombinePerKeyTranslatorBatch.java | 181 ++++---
.../CreatePCollectionViewTranslatorBatch.java | 24 +-
.../translation/batch/DatasetSourceBatch.java | 240 ---------
.../translation/batch/DoFnFunction.java | 164 ------
.../batch/DoFnMapPartitionsFactory.java | 224 ++++++++
.../translation/batch/FlattenTranslatorBatch.java | 60 +--
.../translation/batch/GroupByKeyHelpers.java | 106 ++++
.../batch/GroupByKeyTranslatorBatch.java | 298 +++++++++--
.../translation/batch/ImpulseTranslatorBatch.java | 26 +-
.../translation/batch/ParDoTranslatorBatch.java | 315 ++++++-----
.../translation/batch/PipelineTranslatorBatch.java | 28 +-
.../translation/batch/ProcessContext.java | 138 -----
.../batch/ReadSourceTranslatorBatch.java | 76 +--
.../batch/ReshuffleTranslatorBatch.java | 30 --
.../batch/WindowAssignTranslatorBatch.java | 90 +++-
.../translation/helpers/CoderHelpers.java | 10 +-
.../translation/helpers/EncoderFactory.java | 71 ++-
.../translation/helpers/EncoderHelpers.java | 546 ++++++++++++++++++-
.../translation/helpers/KVHelpers.java | 31 --
.../translation/helpers/MultiOutputCoder.java | 84 ---
.../translation/helpers/RowHelpers.java | 75 ---
.../translation/helpers/SchemaHelpers.java | 39 --
.../translation/helpers/WindowingHelpers.java | 82 ---
.../streaming/DatasetSourceStreaming.java | 25 -
.../streaming/PipelineTranslatorStreaming.java | 93 ----
.../streaming/ReadSourceTranslatorStreaming.java | 87 ---
.../translation/streaming/package-info.java | 20 -
.../translation/utils/ScalaInterop.java | 114 ++++
.../aggregators/metrics/sink/InMemoryMetrics.java | 2 +-
.../translation/batch/AggregatorsTest.java | 370 +++++++++++++
.../translation/batch/CombineGloballyTest.java} | 129 ++---
.../{CombineTest.java => CombinePerKeyTest.java} | 92 ++--
.../translation/batch/ComplexSourceTest.java | 15 +-
.../translation/batch/FlattenTest.java | 12 +-
.../translation/batch/GroupByKeyTest.java | 152 ++++--
.../translation/batch/ParDoTest.java | 54 +-
.../translation/batch/SimpleSourceTest.java | 12 +-
.../translation/batch/WindowAssignTest.java | 12 +-
.../translation/helpers/EncoderHelpersTest.java | 210 +++++++-
.../runners/spark/SparkCommonPipelineOptions.java | 6 +
.../beam/runners/spark/SparkPipelineOptions.java | 6 -
sdks/go.mod | 6 +-
sdks/go.sum | 14 +-
.../java/org/apache/beam/sdk/io/AvroSource.java | 343 +++---------
.../org/apache/beam/sdk/io/AvroSourceTest.java | 166 ------
.../beam/sdk/io/gcp/spanner/BatchSpannerRead.java | 6 +-
.../beam/sdk/io/gcp/spanner/SpannerConfig.java | 13 +-
.../nexmark/src/main/resources/log4j.properties | 2 +-
64 files changed, 4087 insertions(+), 2851 deletions(-)
copy
runners/spark/2/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/utils/{package-info.java
=> ScalaInterop.java} (62%)
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/Constants.java
create mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/io/BoundedDatasetFactory.java
copy runners/spark/{src/main/java/org/apache/beam/runners/spark =>
3/src/main/java/org/apache/beam/runners/spark/structuredstreaming}/io/package-info.java
(93%)
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/AbstractTranslationContext.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/AggregatorCombiner.java
create mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/Aggregators.java
create mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CombineGloballyTranslatorBatch.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DatasetSourceBatch.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DoFnFunction.java
create mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DoFnMapPartitionsFactory.java
create mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/GroupByKeyHelpers.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ProcessContext.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ReshuffleTranslatorBatch.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/KVHelpers.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/MultiOutputCoder.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/RowHelpers.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/SchemaHelpers.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/WindowingHelpers.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/DatasetSourceStreaming.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/PipelineTranslatorStreaming.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/ReadSourceTranslatorStreaming.java
delete mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/package-info.java
create mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/utils/ScalaInterop.java
create mode 100644
runners/spark/3/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/AggregatorsTest.java
copy
runners/spark/{2/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CombineTest.java
=>
3/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CombineGloballyTest.java}
(56%)
rename
runners/spark/3/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/{CombineTest.java
=> CombinePerKeyTest.java} (71%)
