This is an automated email from the ASF dual-hosted git repository.
echauchot pushed a change to branch spark-runner_structured-streaming
in repository https://gitbox.apache.org/repos/asf/beam.git.
discard 014e695 Fix combiner using KV as input, use binary encoders in place
of accumulatorEncoder and outputEncoder, use helpers, spotless
new dbb9938 Fix combiner using KV as input, use binary encoders in place
of accumulatorEncoder and outputEncoder, use helpers, spotless
new a6fc04e Add combinePerKey and CombineGlobally tests
new 15ea094 Introduce RowHelpers
new 463664a Add CombineGlobally translation to avoid translating
Combine.perKey as a composite transform based on Combine.PerKey (which uses low
perf GBK)
new 32f08b0 Cleaning
new aa797b3 Get back to classes in translators resolution because URNs
cannot translate Combine.Globally
new d4d2e7e Fix various type checking issues in Combine.Globally
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (014e695)
\
N -- N -- N refs/heads/spark-runner_structured-streaming
(d4d2e7e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 19429 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
...mbiner.java => AggregatorCombinerGlobally.java} | 12 ++--
...ch.java => CombineGloballyTranslatorBatch.java} | 59 ++++++++++--------
.../batch/CombinePerKeyTranslatorBatch.java | 6 +-
.../translation/batch/DatasetSourceBatch.java | 3 +-
.../translation/batch/DoFnFunction.java | 2 -
.../translation/batch/PipelineTranslatorBatch.java | 39 ++++++------
.../batch/ReadSourceTranslatorBatch.java | 4 +-
...Combiner.java => AggregatorCombinerPerKey.java} | 4 +-
.../translation/helpers/EncoderHelpers.java | 2 +-
.../{WindowingHelpers.java => RowHelpers.java} | 70 ++++++++++------------
.../translation/helpers/WindowingHelpers.java | 54 -----------------
.../streaming/DatasetSourceStreaming.java | 1 +
.../translation/batch/CombineTest.java | 69 +++++++++++++--------
.../translation/batch/FlattenTest.java | 2 +-
14 files changed, 148 insertions(+), 179 deletions(-)
copy
runners/spark-structured-streaming/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/{functions/AggregatorCombiner.java
=> AggregatorCombinerGlobally.java} (85%)
copy
runners/spark-structured-streaming/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/{CombinePerKeyTranslatorBatch.java
=> CombineGloballyTranslatorBatch.java} (51%)
rename
runners/spark-structured-streaming/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/functions/{AggregatorCombiner.java
=> AggregatorCombinerPerKey.java} (93%)
copy
runners/spark-structured-streaming/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/{WindowingHelpers.java
=> RowHelpers.java} (70%)
