I don’t think tying catalog behavior to engine capabilities is a good 
direction. A catalog should remain engine-agnostic and focus purely on metadata 
management and discovery, not on the execution capabilities of specific query 
engines.

Hive Metastore is intentionally designed as a neutral metadata service. It 
exposes table definitions, while each engine (e.g., Apache Spark, Trino, etc.) 
decides whether it can actually process those tables based on its configured 
connectors or format support. Introducing capability negotiation would 
effectively couple the catalog to specific engines and their runtime 
configuration, which breaks that separation of concerns and makes the catalog 
responsible for execution-layer logic.

If a particular engine does not support a given format or catalog (for example, 
it does not have the appropriate client/connector installed), the cleaner 
solution is access control, not metadata filtering. In practice, permissions 
can simply be removed for users of that engine on catalogs or tables they are 
not expected to query.

Keeping the catalog engine-agnostic preserves interoperability and avoids 
embedding engine-specific behavior into the metadata layer.

Reply via email to