danny0405 opened a new issue, #20403: URL: https://github.com/apache/hudi/issues/20403
### Task Description Extend DefaultSource.createSink to collect spark.hoodie.* session settings using the existing DataSourceOptionsHelper normalization used by batch createRelation. Capture the effective flow settings when constructing the sink; pass the same canonical options to multi-writer validation, canSkipBatch, and the writer. Explicit writer/storage options take precedence over session settings; session settings precede defaults. Preserve immutable table identity and do not forward bare hoodie.* session keys. Keep operation/tuning out of new SDP table properties; report conflicting legacy storage options clearly. This intentionally changes direct writeStream behavior for ambient spark.hoodie.* settings and needs compatibility documentation. The batch-path fix in closed #18649 / #18650 is related prior work; this task covers the remaining streaming path. ### Acceptance criteria - [ ] Separate flow definitions using SET ...operation=insert and upsert receive the correct operations without cross-flow leakage. - [ ] Explicit options override session settings and the resolved operation agrees between deduplication and writing. - [ ] Tests cover no session settings, conflicting settings, normalization, and table-identity validation. - [ ] Direct Structured Streaming regression tests and user-facing compatibility documentation are included. - [ ] The CLI/Connect harness demonstrates per-flow SET propagation. ### Task Type Code improvement/refactoring ### Related Issues Parent feature issue: #20401 RFC: #19997 (under review; implementation must follow the agreed contract). This task can start independently; coordinate integration with the sibling tasks under #20401. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
