GitHub user allenjiji added a comment to the discussion: Agent using MCP cannot properly compute metrics
Yes, I’ve seen exactly this, and I don’t think it’s a prompting problem. Disclosure: I work on 0sql, which exists because of this failure. The agent hallucinates a "count" column because it was asked to write an aggregation over raw columns. There is no such column, so it invents one. Asking it to inspect the dataset first helps, but it still has to figure out the aggregation, grain, and joins every time. Sooner or later, it gets something wrong in a way that executes successfully but returns an incorrect result. Letting the agent write custom SQL expressions, which is the newer fix, makes this worse rather than better. What has worked for us is to not let the model write SQL at all. The agent picks named measures and dimensions from a catalogue, for example, "order count" by "category" by "day", and sends that as a small spec. A planner over a governed model generates the SQL, with the aggregations and joins already defined correctly. If the agent names a field that doesn't exist, it gets a refusal with the nearest valid fields instead of a query containing a made-up column. Counting rows per category is simply a measure called "row count", defined once in the model, so the agent never needs to know it's COUNT(*). For Superset, the planned SQL would go into a virtual dataset or SQL Lab rather than through the chart API's metric schema, so this isn't a drop-in fix for the MCP server. But if the goal is an agent that aggregates reliably, moving SQL generation out of the model is the fix. Docs: https://0sql.io/ GitHub link: https://github.com/apache/superset/discussions/40920#discussioncomment-18947298 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
