voonhous commented on PR #19962: URL: https://github.com/apache/hudi/pull/19962#issuecomment-5683552610
What replaced it: since #2553, Flink deletes are not a separate key-list action. They are ordinary records in the streaming upsert: 1. `RecordConverter` turns a Flink row with `RowKind.DELETE` into a `HoodieFlinkRecord` whose operation is `HoodieOperation.DELETE` ([RecordConverter.java#L41](https://github.com/apache/hudi/blob/7f7f16f7dd7936bb68b28d095d1ac37b2d725b08/hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/transform/RecordConverter.java#L41)). 2. With the default Flink-state index, `BucketAssignFunction` looks up the record's location in state, which takes the place of the helper's `tagLocation` step. With a global index it also emits a DELETE for the old partition when a record changes partition ([BucketAssignFunction.java#L212-L231](https://github.com/apache/hudi/blob/7f7f16f7dd7936bb68b28d095d1ac37b2d725b08/hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/BucketAssignFunction.java#L212-L231)). Bucket-index tables skip the lookup and hash to the bucket instead. 3. `StreamWriteFunction` buffers deletes together with inserts and updates, and routes the `DELETE` operation to `writeClient.upsert(records, bucketInfo, instantTime)` ([StreamWriteFunction.java#L245-L249](https://github.com/apache/hudi/blob/7f7f16f7dd7936bb68b28d095d1ac37b2d725b08/hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteFunction.java#L245-L249)). 4. The write handles drop the row because `HoodieFlinkRecord.checkIsDelete` returns true for a `DELETE` operation ([HoodieFlinkRecord.java#L254-L257](https://github.com/apache/hudi/blob/7f7f16f7dd7936bb68b28d095d1ac37b2d725b08/hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/model/HoodieFlinkRecord.java#L254-L257)). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
