This is an automated email from the ASF dual-hosted git repository.
mmack pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 9e044312424 Reorganizing the ML content (#25700)
add 7adcb417f8e [Spark Dataset runner] Implement producer-consumer fusion
for ParDos (closes #25761) (#25762)
No new revisions were added by this update.
Summary of changes:
.../translation/PipelineTranslator.java | 79 ++++--
.../translation/TransformTranslator.java | 7 +
.../batch/DoFnPartitionIteratorFactory.java | 131 ++-------
.../translation/batch/DoFnRunnerFactory.java | 296 +++++++++++++++++++++
.../translation/batch/DoFnRunnerWithMetrics.java | 19 +-
.../translation/batch/ParDoTranslatorBatch.java | 64 ++++-
6 files changed, 451 insertions(+), 145 deletions(-)
create mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DoFnRunnerFactory.java
