GitHub user OjashKush added a comment to the discussion: Native SQL DDL support for Hudi table creation across engines (Trino, Presto etc.)
@voonhous Started on the first bucket in your split (`register_table`/`unregister_table` plus empty `CREATE`/`DROP`). Three findings before I go further, two of which change premises above. **1. `HiveSyncTool` / `HiveSchemaUtil` don't link in the Trino runtime.** `hudi-hive-sync` is compile-scope on `hudi-trino` as you said, but the Hive classes those two need aren't reachable: `HiveSyncTool` and `HiveSyncConfig` pull `org.apache.hadoop.hive.conf.HiveConf`, and `HiveSchemaUtil` needs `org.apache.hadoop.hive.metastore.api.FieldSchema` (plus `HiveSyncConfig`). Neither class is present in any of the 479 jars on the module's compile classpath (`dependency:build-classpath -Dmdep.includeScope=compile`) — no `hive-exec`, no `hive-metastore`. The only Hive-thrift artifact is `io.trino.hive:hive-thrift`, whose classes are relocated to `io.trino.hive.thrift.metastore.*`. Today the module only touches `hudi-hive-sync` for the hadoop-free `PartitionValueExtractor` impls in `HudiBackgroundSplitLoader`. So the catalog side would have to go through `io.trino.metastore.HiveMetastore.createTable(...)` instead; `ResourceHudiTablesInitializer.createTable` in test code is already a working blueprint for the descriptor. Flagging it because cross-engine identity then has to come from *matching* what `HiveSyncTool` emits rather than calling it, which is a real correctness burden. Happy to be told I'm missing a route here. **2. The RFC-103 read gap looks closed.** `HudiTrinoReaderContext.getFileRecordIterator` builds a fresh page source on demand for native parquet delta-log files with pushdown disabled, and its javadoc notes classic Avro log blocks never reach it (they deserialize inline). `getFileFormatUtils` covers PARQUET and HFILE, throwing only for other formats; the separate variant that still throws is inline blocks (`inlinefs://`, rejected in `getFileRecordIterator`). MOR also has v6/v8 fixtures plus merge-mode and payload tests. So the read-path reason for restricting to COW appears to be gone. Planning to support both COW and MOR, but still pin `hoodie.table.version` explicitly rather than inheriting `HoodieTableVersion.current()` (currently TEN) — for a different reason than your original note: otherwise a future `current()` bump silently changes what the connector emits, with no diff to review. Will add a test asserting the pinned value so a bump fails loudly instead. **3. One actual question — managed vs external for `createTable`.** `register_table` is clearly EXTERNAL. Less sure about `createTable`: the fixtures use `EXTERNAL_TABLE` + `EXTERNAL=TRUE`, but if `DROP TABLE` should clean up a `.hoodie` directory the connector itself created, managed seems closer to right. Do you have a preference, or a precedent from another connector you'd point at? Proceeding with the property allowlist, the Trino→Avro type mapping, and metaclient init in the meantime, since none of those depend on the above. GitHub link: https://github.com/apache/hudi/discussions/19484#discussioncomment-18092573 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
