This is an automated email from the ASF dual-hosted git repository.
dmvk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from b6ca2ab [BEAM-9723] Add DLP integration transforms (#11566)
add 0c45136 [BEAM-8850,BEAM-8849] FlinkBatchTranslationContext respects
input parameters during lookups. Add support for a multiple translation
variants for a single urn.
add 3c4536d [BEAM-8848] Flink Batch Runner: optimize GBK translation for
non-merging windows.
add 23a1fb7 [BEAM-8848] Code review.
add ccc2fa4 Merge pull request #11874: Flink Batch Runner: optimize GBK
translation for non-merging windows.
No new revisions were added by this update.
Summary of changes:
.../flink/FlinkBatchPipelineTranslator.java | 15 ++-
.../flink/FlinkBatchTransformTranslators.java | 124 +++++++++++++++++----
.../flink/FlinkBatchTranslationContext.java | 61 +++++-----
.../beam/runners/flink/FlinkPipelineOptions.java | 7 ++
...ction.java => FlinkExplodeWindowsFunction.java} | 25 ++---
.../functions/FlinkNonMergingReduceFunction.java | 112 +++++++++++++++++++
...KeySelector.java => WindowedKvKeySelector.java} | 16 ++-
.../utils/CountingPipelineVisitor.java} | 32 +++---
.../translation/utils/LookupPipelineVisitor.java | 98 ++++++++++++++++
.../flink/batch/NonMergingGroupByKeyTest.java | 80 +++++++++++++
.../shortcodes/flink_java_pipeline_options.html | 5 +
.../shortcodes/flink_python_pipeline_options.html | 5 +
12 files changed, 493 insertions(+), 87 deletions(-)
copy
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/functions/{FlinkIdentityFunction.java
=> FlinkExplodeWindowsFunction.java} (59%)
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkNonMergingReduceFunction.java
copy
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/types/{KvKeySelector.java
=> WindowedKvKeySelector.java} (68%)
copy
runners/{google-cloud-dataflow-java/src/test/java/org/apache/beam/runners/dataflow/RecordingPipelineVisitor.java
=>
flink/src/main/java/org/apache/beam/runners/flink/translation/utils/CountingPipelineVisitor.java}
(59%)
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/utils/LookupPipelineVisitor.java
create mode 100644
runners/flink/src/test/java/org/apache/beam/runners/flink/batch/NonMergingGroupByKeyTest.java
