This is an automated email from the ASF dual-hosted git repository.
kenn pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 673da546c14 [#30083][prism] Factor out hold tracking to dedicated
structures (#31105)
add 08fc33b26cc Touch trigger files (add some comments as well)
add 7ade06f6f77 Add Redistribute to Beam model protos
add f49ac7d10a0 Add Redistribute transform to Java SDK
add 18ebc454928 Add direct Dataflow translation of Redistribute transform
add 21e3fa1eedf Add Redistribute translation to Spark runner
add 8f1d3da4399 Add Redistribute translation to Samza runner
add bdfc7812fdf Add Redistribute translation to FlinkRunner
new 6fa6a28b397 Merge pull request #30545: Add Redistribute transform to
model, Java SDK, and most active runners
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
..._PostCommit_Java_ValidatesRunner_Dataflow.json} | 0
...t_Java_ValidatesRunner_Dataflow_Streaming.json} | 0
...eam_PostCommit_Java_ValidatesRunner_Flink.json} | 0
...eam_PostCommit_Java_ValidatesRunner_Samza.json} | 0
...eam_PostCommit_Java_ValidatesRunner_Spark.json} | 0
.../beam/model/pipeline/v1/beam_runner_api.proto | 8 +
runners/flink/flink_runner.gradle | 1 +
runners/flink/job-server/flink_job_server.gradle | 2 +
.../FlinkBatchPortablePipelineTranslator.java | 17 +-
.../flink/FlinkBatchTransformTranslators.java | 78 ++++++
.../FlinkStreamingPortablePipelineTranslator.java | 22 ++
.../flink/FlinkStreamingTransformTranslators.java | 37 +++
.../dataflow/DataflowPipelineTranslator.java | 37 +++
.../beam/runners/dataflow/util/PropertyNames.java | 1 +
.../translation/RedistributeByKeyTranslator.java | 64 +++++
.../samza/translation/ReshuffleTranslator.java | 14 +-
.../samza/translation/SamzaPipelineTranslator.java | 1 +
.../spark/translation/TransformTranslator.java | 66 +++++
.../streaming/StreamingTransformTranslator.java | 70 +++++
.../apache/beam/sdk/transforms/Redistribute.java | 300 +++++++++++++++++++++
.../util/construction/PTransformTranslation.java | 8 +
.../{ReshuffleTest.java => RedistributeTest.java} | 155 +++++------
22 files changed, 784 insertions(+), 97 deletions(-)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Dataflow.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Flink.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Samza.json} (100%)
copy .github/trigger_files/{beam_PostCommit_Java_PVR_Spark3_Streaming.json =>
beam_PostCommit_Java_ValidatesRunner_Spark.json} (100%)
create mode 100644
runners/samza/src/main/java/org/apache/beam/runners/samza/translation/RedistributeByKeyTranslator.java
create mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Redistribute.java
copy
sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/{ReshuffleTest.java
=> RedistributeTest.java} (74%)
