voonhous opened a new issue, #20670:
URL: https://github.com/apache/hudi/issues/20670

   ### Problem
   
   The Trino connector never relates `HoodieSchema` to Trino types in one 
place. Column types come from the Hive type strings in the metastore 
(`HoodieSchema` to Hive type through Hive sync, then Hive type to Trino type 
through Trino's Hive translation), while the merge path works on records typed 
by `HoodieSchema` that hold Trino objects.
   
   Code that needs both sides has to know both hops. Two current examples:
   
   - #20648: the merge buffer's spill serializer has to know that UUID and 
VECTOR arrive as `varbinary` and BLOB and VARIANT as rows, because that is how 
Hive sync declares them.
   - #20012: `CREATE TABLE` adds its own Trino to `HoodieSchema` conversion and 
rejects UUID and TIME only because the metastore has no such types.
   
   ### Proposal
   
   Add `TrinoTypeMapper` to `hudi-trino`: a plain class that maps each 
`HoodieSchemaType` to exactly one Trino type and rejects the types Trino cannot 
hold. It is the Trino entry of the type plane in RFC-104 (#18661, Workstream 
D), introduced without the shared interface so that connector work can start 
using it; implementing the interface later is a hook-up.
   
   Scope of the first PR:
   
   - `toTypeSystem(HoodieSchema)` only, with tests for every `HoodieSchemaType`.
   - No caller in the connector. Column types still come from the metastore.
   
   Follow-ups, each on its own:
   
   - The reverse direction (Trino type to `HoodieSchema`), folding in the 
converter from #20012.
   - Moving the connector's column types onto the mapper. This changes what 
Trino reports for UUID, TIME, TIMESTAMP and VECTOR columns, so it needs gating.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to