This is an automated email from the ASF dual-hosted git repository.
xushiyan pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
from e654914b3fea perf: Support mini-batch access to the MDT index for
bucket assign fu… (#17867)
add 30029e37017f feat: support disruptor-queue buffer for Flink writers
(#17864)
No new revisions were added by this update.
Summary of changes:
.../common/util/queue/DisruptorMessageQueue.java | 14 +-
.../apache/hudi/configuration/FlinkOptions.java | 39 +++-
.../hudi/sink/append/AppendWriteFunction.java | 3 +-
...a => AppendWriteFunctionWithBIMBufferSort.java} | 59 +++--
...AppendWriteFunctionWithDisruptorBufferSort.java | 236 ++++++++++++++++++++
.../hudi/sink/append/AppendWriteFunctions.java | 66 +++++-
.../org/apache/hudi/sink/buffer/BufferType.java | 22 +-
.../hudi/sink/buffer/MemorySegmentPoolFactory.java | 20 +-
.../ITTestAppendWriteFunctionWithBufferSort.java | 244 ++++++++++++---------
.../TestAppendWriteFunctionWithBufferSort.java | 75 +++++++
10 files changed, 618 insertions(+), 160 deletions(-)
rename
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/append/{AppendWriteFunctionWithBufferSort.java
=> AppendWriteFunctionWithBIMBufferSort.java} (79%)
create mode 100644
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/append/AppendWriteFunctionWithDisruptorBufferSort.java
copy
hudi-common/src/main/java/org/apache/hudi/common/bloom/BloomFilterTypeCode.java
=>
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/buffer/BufferType.java
(54%)
create mode 100644
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/append/TestAppendWriteFunctionWithBufferSort.java
