mistercrunch commented on PR #32649: URL: https://github.com/apache/superset/pull/32649#issuecomment-3050552831
I'll give you my take, but curious to hear from others too. A few structured thoughts: --- ### 1. If in Superset, LLM config should be modular and first-class What we need here goes way beyond a single config block in `superset_config`. The idea of one global LLM setup doesn't scale—different features may need different models, token limits, temps, prompt chains, etc. Think OpenAI for one use case, Anthropic or local Llama for another. Feels like LLM configs should be treated more like database connections: centrally defined and then mapped to features. We'd need: - a registry of LLM configs (maybe via an admin UI or file), - a mapping system that assigns features to specific LLMs, - prompt templates that are overridable, probably Jinja-based, and customizable per org/use case. --- ### 2. Feature flags aren’t the right control plane The current pattern of scattering feature flags for every AI experiment isn't going to scale. In theory, FFs were designed for progressive rollouts of “forever features,” but in practice they've already become a de facto config framework. What we probably need is something more intentional: a centralized `AI_FEATURE_CONFIG` layer that lets you: - toggle individual AI-powered features, - assign them to specific LLM configs, - override/extend prompts, - control behavior more cleanly and scalably. --- ### 3. Superset already has a huge product surface We have a lot of ground to cover already, and anything we land here we’re likely stuck supporting for the next decade. Every new AI integration adds cognitive and maintenance load in a fast-moving space with fragile dependencies and shifting UX norms. That’s a real risk, especially when the ecosystem and design patterns are changing daily --- ### 4. The right place for this is the extension framework This is exactly the kind of thing that should live in our (in-dev) extension framework. That would let us: - avoid overloading core Superset, - validate the modular design needed for the AI wave, - and create space for orgs to experiment and customize safely. If Superset becomes more like VSCode—with rich, modular extensions—this type of feature becomes additive instead of burdensome. --- ### 5. The way I think about it: **MCP is the path forward** MCP is the cleanest way to support AI in a scalable, deterministic way. Instead of trying to bolt LLMs *into* Superset, we focus on exposing a solid API that external agents can use. The flow becomes: > “Hey LLM, I’m looking at the ‘Sales’ dashboard in Superset—what’s up with international orders in Canada?” And the LLM uses MCP calls to introspect metadata, generate SQL, request charts, etc. Superset stays composable, auditable, and predictable—and the fuzzy magic stays outside the boundary. --- ### 6. A backend-up abstraction could work well I’d be happy to help shape a backend-first approach - ideally through an extension: - A `superset-extension-langchain` could expose LLM configs and register new REST endpoints, - Extensions in the frontend can use those endpoints to build UX, - And we define the safe, pluggable backend pieces: RBAC, prompt templating, async support, etc. That gives us a clean, layered system—one that avoids LangChain/LLM lock-in, and gives orgs real flexibility. --- ### 7. AI testing is still shaky Worth calling out: we don’t have great patterns yet for testing these kinds of flows. Even small prompt tweaks or model changes can create huge diffs in output. The more this stuff ends up in core, the more brittle our test matrix becomes. --- **TL;DR**: This is a super cool POC, and the use case is legit—but we’re not ready to enshrine this in core. Let’s use this as a proof point for the extension framework, build modular LLM abstractions, and anchor our long-term AI strategy around MCP. That path gives us power *and* flexibility, without locking us into early decisions we’ll regret. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
