prashantwason opened a new pull request, #18088:
URL: https://github.com/apache/hudi/pull/18088
### Describe the issue this Pull Request addresses
This PR adds support for the DELETE write operation type in
HoodieDeltaStreamer's StreamSync.
Previously, DeltaStreamer supported INSERT, UPSERT, BULK_INSERT,
INSERT_OVERWRITE, INSERT_OVERWRITE_TABLE,
and DELETE_PARTITION operations, but lacked support for the basic DELETE
operation which deletes
records by their keys.
### Summary and Changelog
Adds DELETE operation support to HoodieDeltaStreamer, enabling users to
perform delete operations
using record keys through the streaming ingestion pipeline.
**Changes:**
- Added `DELETE` case in `StreamSync.writeToSink()` switch statement that
extracts HoodieKeys from
records and calls `writeClient.delete()`
- Added `testDelete()` parameterized test method to validate DELETE
operation with both AVRO and SPARK record types
- Modified `testDeltaStreamerWithSpecifiedOperation()` to handle DELETE
operation assertions appropriately
### Impact
- New user-facing feature: Users can now specify `--op DELETE` when running
HoodieDeltaStreamer to perform delete operations
- No breaking changes to existing functionality
- No public API changes
### Risk Level
low - This is an additive change that follows the existing pattern for other
write operations.
The DELETE operation already exists in the write client and is well-tested;
this PR simply exposes
it through the DeltaStreamer interface.
### Documentation Update
none - The DELETE operation type is already documented as part of
WriteOperationType enum.
No new configs are added.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
