GitHub user OjashKush created a discussion: Native SQL DDL support for Hudi table creation across engines (Trino, Flink SQL etc.)
## Problem Today, table creation via `CREATE TABLE` DDL is fully supported only through Spark SQL (via Hudi's Spark SQL extensions). Other engines — Trino, Hive, etc. — either don't support creating Hudi tables via SQL at all, or only have partial/inconsistent support (e.g. querying existing tables works, but creating new ones doesn't, or the config keys/syntax diverge from what Spark SQL expects). In practice, this means anyone creating a new Hudi table today has to go through Spark SQL, regardless of which engine they'll actually read/write with day-to-day. ## Motivation - **Multi-engine adoption**: Hudi's core value proposition is being a multi-engine format (Flink, Trino, Presto, Hive, Spark). Table creation shouldn't be gated behind a single engine. - **Spark as a mandatory bootstrap step**: Teams that are Flink-first or Trino-first for their actual workloads still have to stand up Spark just to create a table. That's an unnecessary dependency and operational overhead. - **Consistency of schema/config handling**: Each engine currently has its own (or no) way of expressing Hudi-specific table properties — record key, partition path, precombine field, etc. — at creation time. This creates room for subtle misconfiguration when a table created in one context is later used from another engine. - **Accessibility for analysts**: Analysts frequently need to create ad-hoc tables and are SQL-heavy by default. Analysts and other less Spark-fluent users are often intimidated by Spark SQL / Spark session setup, but still need to create Hudi tables to get started. Trino has no CREATE TABLE path for Hudi today, so when analysts need a quick ad-hoc table, the practical workaround is to create it in Iceberg instead. Supporting CREATE TABLE natively in Trino would close that gap directly — no new tooling for analysts to learn, and no reason to default to Iceberg just to get a table created. ## Ask Opening this up for discussion: - Is there existing work or interest in closing this gap for specific engines (e.g. Flink SQL `CREATE TABLE` support, Trino DDL)? - What's the right shared abstraction here - should DDL translate through a common catalog/metaclient path, so config handling stays consistent across engines rather than each engine reinventing its own `CREATE TABLE` handling? - Given the breadth of per-engine catalog/DDL work involved, should this be scoped as a formal RFC? cc @vinothchandar - flagging this as a discussion per your suggestion. Curious to hear from others who've hit this gap too. GitHub link: https://github.com/apache/hudi/discussions/19484 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
