ekas-7 commented on issue #871: URL: https://github.com/apache/wayang/issues/871#issuecomment-6774051721
Hi @AliRana30, @Kaileshwar16, Thanks for chiming in. Regarding generic JDBC vs. a dedicated module (@Kaileshwar16): The rationale matches why Wayang maintains dedicated modules for Trino, Presto, and BigQuery rather than relying on wayang-jdbc-template alone: Engine semantics: ClickHouse DDL requires explicit storage engines (e.g., CREATE TABLE ... ENGINE = MergeTree ORDER BY tuple() AS SELECT ...). Generic JDBC does not support this syntax pattern for table sinks without engine-specific overrides. Type system and dialect boundaries: Dedicated mapping ensures types like UInt64, LowCardinality, and Nullable translate correctly to Wayang streams/RDD channels, alongside ClickHouse-specific quoting rules and join nullability semantics. Targeted cost modeling & pushdown optimization: A distinct platform identity is required to configure dialect-specific execution costs and accurately steer plan optimization toward column-oriented storage. The implementation semantics are already isolated and scoped to avoid cross-module churn—it follows the established platform template without modifying wayang-jdbc-template. Next steps (@AliRana30): I already have the architectural plan and initial implementation underway locally following the phased PR sequence outlined above. To keep the review process tight and avoid coordination overhead, I will be driving the implementation of PR 1 through completion myself. I'll push the draft for PR 1 shortly so everyone can review the concrete code against the proposal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
