This is an automated email from the ASF dual-hosted git repository.
echauchot pushed a change to branch spark-runner_structured-streaming
in repository https://gitbox.apache.org/repos/asf/beam.git.
omit eae506b [BEAM-8470] Fix wrong expected results in
CombineTest.testBinaryCombineWithSlidingWindows
omit ef0e870 [BEAM-8470] Fix tests: use correct
SparkStructuredStreamingPipelineOptions, set testMode to true. Some renaming
omit 6826025 [BEAM-8470] Add a test to test combine translation on
binaryCombineFn with sliding windows
omit f668e06 [temp] nexmark run in new spark runner
omit 7ee091e [temp] debug pardo print input and output in both spark
runners and put spark in debug mode
add c4f2575 [BEAM-8470] Add a test to test combine translation on
binaryCombineFn with sliding windows
add 2e048f9 [BEAM-8470] Fix tests: use correct
SparkStructuredStreamingPipelineOptions, set testMode to true. Some renaming
add fba2109 [BEAM-8470] Fix wrong expected results in
CombineTest.testBinaryCombineWithSlidingWindows
add a7d67b2 [BEAM-8470] Add disclaimers about this runner being
experimental
add 4a2c59d [BEAM-8470] Fix: create an empty accumulator in
combine.mergeAccumulators, because this method modifies its first input
accumulator. Decrease memory usage by storing only accumulator and timestamp in
the combine.merge map
add d949942 [BEAM-8470] Apply spotless
add 5a059d1 [BEAM-8470] Add a countPerElement test with sliding windows
add 2e123ac [BEAM-8470] Fix the output timestamps of combine: timestamps
must be merged to a window before combining
add 3506d6d [BEAM-8470] set log level to info to avoid resource
consumption in production mode
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (eae506b)
\
N -- N -- N refs/heads/spark-runner_structured-streaming
(3506d6d)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../SparkStructuredStreamingRunner.java | 19 +++--
.../translation/batch/AggregatorCombiner.java | 39 ++++++----
.../translation/batch/ParDoTranslatorBatch.java | 43 +----------
.../spark/translation/TransformTranslator.java | 21 +-----
runners/spark/src/main/resources/log4j.properties | 4 +-
.../metrics/sink/SparkMetricsSinkTest.java | 4 +-
.../metrics/MetricsPusherTest.java | 6 +-
.../translation/batch/CombineTest.java | 84 +++++++++++++++-------
.../translation/batch/ComplexSourceTest.java | 4 +-
.../translation/batch/FlattenTest.java | 4 +-
.../translation/batch/GroupByKeyTest.java | 7 +-
.../translation/batch/ParDoTest.java | 19 +++--
.../translation/batch/SimpleSourceTest.java | 4 +-
.../translation/batch/WindowAssignTest.java | 7 +-
.../translation/streaming/SimpleSourceTest.java | 4 +-
.../apache/beam/sdk/nexmark/NexmarkLauncher.java | 2 -
.../apache/beam/sdk/nexmark/queries/Query5.java | 2 +-
.../nexmark/src/main/resources/log4j.properties | 2 +-
18 files changed, 138 insertions(+), 137 deletions(-)
