DanielLeens commented on issue #12497: URL: https://github.com/apache/seatunnel/issues/12497#issuecomment-5869558936
Classification: D / Connector-V2 DuckLake bulk-write design. The small-file observation is credible, and #12498 is the exact active implementation path. I checked its current boundary: the default JDBC behavior remains unchanged, while the opt-in path adds a public ducklake_bulk_write option, buffers one batch in a connection-local temporary table, and transfers it with one INSERT SELECT. It deliberately rejects XA, retries, upserts, custom SQL, multi-table routing, automatic target creation, and non-insert rows. This is a new connector delivery contract, not a small local optimization. The real DuckLake check is opt-in and upstream Build is still in progress, so neither the 100-row file-count experiment nor the local recovery smoke makes it a completed feature or an exactly-once implementation. Keep the contract append-only and explicit about replay duplicates, check every writer connection has the attached target, and retain tests for failed transfers, stage cleanup, schema types, and identifier components. The next gate is green upstream CI and review of the option, lifecycle, and failure semantics. No label or assignment change in this pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
