nada-attia opened a new issue, #18107:
URL: https://github.com/apache/hudi/issues/18107
### Feature Description
**What the feature achieves:**
Adds touch partition support in HiveSyncTool that allows partitions with
unchanged storage locations to be re-registered in HMS. This is gated by the
META_SYNC_CONDITIONAL_SYNC configuration—TOUCH
events are only generated when conditional sync is enabled. The feature
also adds metaconf configuration when creating HMSClient to ensure proper
client initialization.
**Why this feature is needed:**
During Hive sync operations, partitions that already exist with the correct
location currently have no way to signal they've been "touched". This is a
useful event which may be needed for data freshness monitoring. By introducing
TOUCH events (only when conditional sync is enabled), we can perform
lightweight re-registration for unchanged partitions signaling that a commit
has written new data to existing partitions.
### User Experience
**How users will use this feature:**
- Configuration changes needed
- they will need to pass `hoodie.datasource.meta_sync.condition.sync=true`
### Hudi RFC Requirements
**RFC PR link:** (if applicable)
**Why RFC is/isn't needed:**
- Does this change public interfaces/APIs? (Yes/No)
- Does this change storage format? (Yes/No)
- Justification:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
