GitHub user vaibhavk1992 added a comment to the discussion: [DISCUSS] New
Agentic lakehouse module & built-in AI agent gateway
+1 for this idea.
Here are some of the areas which I feel we may need to address.
**Time-travel + Hudi-native features:**
1. Will the agent have Hudi-specific SQL knowledge baked in — e.g., generating
time-travel queries
2. Understanding COW vs MOR read semantics, and correctly constructing
partition-pruned queries? A generic text-to-SQL model
3. Will generate standard SQL and miss these entirely, which is both a
correctness and a cost risk.
**XTable / cross-format:**
1. Many Hudi deployments sync tables to Iceberg/Delta (via XTable or Hudi's
own sync). When the agent gateway queries via
Trino, which representation does it target — the Hudi native path or the
synced format? The two can be minutes out of sync, so the choice affects data
freshness guarantees.
**Cost control:**
1. Is there a planned query budget or cost-cap mechanism? An agentic loop that
misunderstands a question could issue multiple full-table scans before giving
up. For petabyte-scale tables this is a real risk. Even a simple 'max bytes
scanned per session' guard would help.
**Hallucination / verification:**
1. Given the ~70% text-to-SQL accuracy baseline, is there a planned
verification step before results are returned — e.g. showing the generated SQL
to the user for confirmation, or running a lightweight sanity check (row count
plausibility)? For financial/operational analytics, a wrong number that
looks right is the worst failure mode.
GitHub link:
https://github.com/apache/hudi/discussions/19264#discussioncomment-17631796
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
