I think the Python POC is MUCH cleaner! Nice work! Cheers,
Adam On Wed, Nov 5, 2025 at 1:05 AM Yufei Gu <[email protected]> wrote: > Indeed, in our last two discussions on the Spark Client and NonSQL > implementation, we decided to include both in the main repo. > > To echo the previous discussion with Adam, I’ve also created a Python-based > MCP Server POC here: https://github.com/apache/polaris/pull/2980. It uses > `uv` as the package manager and is built on the fastMCP library (Apache > License 2.0). My manual test against previous use cases worked well. It > does feel noticeably simpler than the Java version. Maybe we should go with > the Python one. Thoughts? > > Yufei > > > On Tue, Nov 4, 2025 at 4:25 AM Robert Stupp <[email protected]> wrote: > > > Hi all, > > > > The discussion about one repo per tool or one tools repo sounds quite > > familiar to me. Haven't we had this discussion before and > > intentionally went with the single repo approach? > > > > Robert > > > > > > On Tue, Nov 4, 2025 at 7:41 AM Jean-Baptiste Onofré <[email protected]> > > wrote: > > > > > > Hi > > > > > > Repo and release are two different things: you can have one repo with > > > submodules released independently. > > > > > > If MCP server goes in polaris main repo, than other tools could be > > > there too (console, catalog migrator, etc). But I don't think it's a > > > good idea due to the LICENSE/NOTICE overhead and also clear separation > > > of concerns. > > > > > > My preference is still about polaris-tool or a each "tool" in its own > > > repository (polaris-mcp, polaris-console, ...). > > > For instance, it's what I'm doing in Karaf (you have karaf, > > > karaf-cellar, karaf-decanter, karaf-cave, karaf-minho, all related to > > > karaf but different tools/purposes, with different version and release > > > cycle). Camel is also doing the same (camel-core, camel-spring-boot, > > > camel-quarkus, camel-karaf, ...). > > > > > > Regards > > > JB > > > > > > On Mon, Nov 3, 2025 at 10:57 PM Yufei Gu <[email protected]> wrote: > > > > > > > > Thanks a lot for the summary, Adam! This captures most of the urgent > > items > > > > we need to move forward. > > > > > > > > One idea to consider is creating a separate project under Polaris > > dedicated > > > > to the MCP server. Managing multiple tools in a single > > repo(Polaris-tool) > > > > can make releases quite challenging, since: > > > > • Their release cadences differ. > > > > • License requirements may vary. > > > > • They could end up blocking each other’s releases, reducing > > flexibility. > > > > > > > > Having an independent project would make development and release > > management > > > > much cleaner and more agile. > > > > > > > > Alternatively, placing the MCP Server as a client module within the > > main > > > > Polaris repo could also simplify the release process, we wouldn’t > need > > to > > > > maintain complex mapping tables between MCP Server and Polaris > > versions. > > > > It’s not just about REST API dependencies; in fact, the REST clients > > are > > > > relatively trivial compared to the context coupling an MCP server > > requires, > > > > which may naturally align with a Polaris release. > > > > > > > > Yufei > > > > > > > > > > > > On Mon, Nov 3, 2025 at 12:46 PM Adam Christian < > > > > [email protected]> wrote: > > > > > > > > > Just so that folks are aware, Yufei & I just chatted and here's our > > > > > updates: > > > > > > > > > > 1. Python POC*: *While Yufei's POC is in Java, there is a richer > > ecosystem > > > > > in Python for AI, ML, & data science tools. Yufei is going to > drive a > > > > > Python POC to see whether we can leverage some of the larger > > ecosystem and > > > > > whether that makes the code easier to maintain. > > > > > 2. Repository: We chatted, but did not get to a solid consensus. > > There were > > > > > some concerns Yufei mentioned about Polaris-Tools: > > > > > 2a. Polaris-Tools has multiple tools so semantic versioning is > > not as > > > > > indicative of where the tool is. For example, folks have leveraged > > the > > > > > Iceberg Catalog Migrator for a while, but the MCP Server would be > > new. Both > > > > > of these would be released under the same version, so it does not > > > > > necessarily indicate how stable the particular tool is. A remedy to > > that > > > > > would be to have a separate repository. I think this is going to be > > true > > > > > for all of our various tools, so we will just need to figure out > how > > to > > > > > handle that. > > > > > 2b. We will have to ensure that all of the licensing checks > work > > > > > properly for the release of Polaris-Tools. Obviously, this is true > > whenever > > > > > we decide to release Polaris-Tools, but it might incur additional > > overhead. > > > > > 3. Next Steps: Yufei is going to drive the Python POC and, then, > as a > > > > > community, we can chat about which way to go around the large open > > > > > discussions. > > > > > > > > > > Yufei can correct me if I misspoke on any of this stuff. > > > > > > > > > > Go community! > > > > > > > > > > Adam > > > > > > > > > > On Mon, Nov 3, 2025 at 12:59 PM Dmitri Bourlatchkov < > > [email protected]> > > > > > wrote: > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > It's an interesting proposal. Thanks for starting a doc on it! > > > > > > > > > > > > Re: code location, I tend to agree with Eric's proposal to put it > > into > > > > > > polaris-tools. > > > > > > > > > > > > I suppose the MCP server evolution does not have to follow the > > Server > > > > > > evolution step by step. In fact, I'd expect the MCP server to be > > > > > revisioned > > > > > > out-of-sync with the Polaris Server to accommodate features on > the > > AI > > > > > > (agent) side. It only needs to consume the REST APIs from the > > server, and > > > > > > those APIs are fairly stable to allow for independent releases. > > > > > > > > > > > > Cheers, > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 30, 2025 at 8:34 PM Eric Maynard < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > This is a pretty cool project! > > > > > > > > > > > > > > I recall that originally the idea behind polaris-tools was that > > things > > > > > > > which are in the Polaris “ecosystem” could start there and > > potentially > > > > > be > > > > > > > folded into the core project if they wind up being really > useful > > or > > > > > > widely > > > > > > > used. This hasn’t happened yet but I feel like we shouldn’t be > > afraid > > > > > to > > > > > > > put things in tools and move them later. > > > > > > > > > > > > > > I do view the MCP server as a kind of client to Polaris, and I > > agree > > > > > that > > > > > > > running it locally is probably the easiest thing for now. > > > > > > > > > > > > > > —EM > > > > > > > > > > > > > > On Thu, Oct 30, 2025 at 3:58 PM Yufei Gu <[email protected] > > > > wrote: > > > > > > > > > > > > > > > Hi Adam, thanks for chiming in! > > > > > > > > > > > > > > > > I’m okay with either polaris (main) or polaris-tools. The POC > > > > > currently > > > > > > > > lives in the main repo, but it’s a standalone JAR, it neither > > depends > > > > > > on > > > > > > > > other modules nor has modules depending on it. Another option > > is to > > > > > > > > consider it as a client. We could put it along with the CLI > > client > > > > > > under > > > > > > > > "/client". These are all minor points. I will move it to the > > tool > > > > > repo > > > > > > if > > > > > > > > people feel strongly about it. > > > > > > > > > > > > > > > > It's a good idea to have an Iceberg MCP server. We might > think > > of it > > > > > > as a > > > > > > > > separate effort. > > > > > > > > > > > > > > > > It's a POC, we will definitely add more context. And context > > > > > > engineering > > > > > > > > isn't just a one-time effort. It's a continuing project. I'm > > still > > > > > > > > trying to figure out a better way to do so, a better way > means > > not > > > > > too > > > > > > > much > > > > > > > > also not too less and more accurate. With that said, a remote > > MCP > > > > > > server > > > > > > > is > > > > > > > > probably a better option, as the context update would be > > centralized. > > > > > > > > > > > > > > > > For availability and rolling upgrades, it’s fine that the MCP > > server > > > > > is > > > > > > > > local for now. We can address those concerns when we build > out > > the > > > > > > remote > > > > > > > > MCP server. > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 30, 2025 at 7:44 AM Adam Christian < > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > This is fantastic, Yufei! I love this idea and the demo is > > > > > wonderful. > > > > > > > > > > > > > > > > > > A few thoughts: > > > > > > > > > > > > > > > > > > 1. I agree with your assessment that an MCP Server would > > > > > > complement > > > > > > > > > Polaris' other interfaces - REST, the python client, & > the > > > > > future > > > > > > > UI. > > > > > > > > > This > > > > > > > > > adds to the growing Polaris ecosystem and I know people > > will > > > > > find > > > > > > > this > > > > > > > > > helpful. > > > > > > > > > 2. I agree with your approach of building an MCP Server > > rather > > > > > > than > > > > > > > > > using some sort of RAG approach (adding it to the chat > > context). > > > > > > > It's > > > > > > > > a > > > > > > > > > seamless sort of integration that we see other vendors > > > > > leveraging > > > > > > as > > > > > > > > > well. > > > > > > > > > 3. I am unsure of whether this work should be integrated > > into > > > > > the > > > > > > > > > Polaris repository or the Polaris-Tools repository. I > > don't know > > > > > > how > > > > > > > > we > > > > > > > > > as > > > > > > > > > a community make the distinction. I know that the UI is > > going to > > > > > > be > > > > > > > > put > > > > > > > > > into the Polaris-Tools repository. Here's my > > understanding so > > > > > far, > > > > > > > but > > > > > > > > > I'd > > > > > > > > > love to chat about it: > > > > > > > > > 1. Polaris-Tools Pros: > > > > > > > > > 1. We don't want any hard dependencies between the > > core > > > > > > > Polaris > > > > > > > > > codebase and the MCP Server. You have done a > > phenomenal > > > > > job > > > > > > > > > at doing this > > > > > > > > > by making the MCP Server be a shim between the > REST > > API > > > > > and > > > > > > > the > > > > > > > > AI > > > > > > > > > Application (Claude Desktop, 5ire, etc). > > > > > > > > > 2. This work might not be applicable for core use > > cases, > > > > > so > > > > > > it > > > > > > > > > would be an additional series of packages that > > would be > > > > > > > > > unused by users and > > > > > > > > > cause overhead for building and maintaining. > > > > > > > > > 2. Polaris Pros: > > > > > > > > > 1. It can leverage common infra (Gradle, etc). > > > > > > > > > 2. It'll be faster to develop. > > > > > > > > > 3. It'll be easier to keep in sync with the REST > > APIs > > > > > > > (although > > > > > > > > > that applies to the UI as well). > > > > > > > > > 4. I'm wondering if we should create an Iceberg > > Catalog MCP > > > > > > > Server > > > > > > > > > and a Polaris MCP Server. A lot of the tools that you > > created > > > > > > would > > > > > > > > > benefit > > > > > > > > > the wider Iceberg Catalog ecosystem and are not > > > > > Polaris-specific. > > > > > > > For > > > > > > > > > example, the polaris-namespace-request and the > > > > > > polaris-table-request > > > > > > > > are > > > > > > > > > just Iceberg Catalog operations. What are your thoughts > > there? > > > > > > > > > 5. For some of the issues that you have observed with > > setting > > > > > > > > > privileges, I believe that we might be able to improve > the > > > > > > accuracy > > > > > > > by > > > > > > > > > updating the tools' descriptions. You & I can chat about > > this > > > > > > > offline, > > > > > > > > > but > > > > > > > > > from my analysis of the code, we can make those > > descriptions > > > > > much > > > > > > > > richer > > > > > > > > > and that'll increase accuracy without too much > additional > > work. > > > > > > I've > > > > > > > > > seen > > > > > > > > > this in other MCP Servers that I've built. But, we can > > handle > > > > > this > > > > > > > > after > > > > > > > > > we've released the first version. It's a small change. > > > > > > > > > 6. I know this is early days, but have you thought about > > what > > > > > sort > > > > > > > of > > > > > > > > > availability you would want with the MCP Server? Like, > > Polaris > > > > > > tries > > > > > > > > to > > > > > > > > > support rolling upgrades [1]. Is that your intention > here? > > > > > > > > > > > > > > > > > > Thanks for doing all this work. I'm excited about where > this > > is > > > > > > going! > > > > > > > > > > > > > > > > > > [1] - > > https://polaris.apache.org/in-dev/unreleased/evolution/ > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > > > Adam > > > > > > > > > > > > > > > > > > On Tue, Oct 28, 2025 at 4:30 PM Yufei Gu < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > I’d like to propose the Polaris MCP Server, which lets > > users and > > > > > AI > > > > > > > > > agents > > > > > > > > > > interact with Polaris using natural language instead of > > REST or > > > > > > CLI. > > > > > > > > > > Example: “Show me all tables in prod.sales updated in the > > last 7 > > > > > > > days.” > > > > > > > > > > > > > > > > > > > > It runs locally, connects to Polaris via REST, and > > requires no > > > > > > > backend > > > > > > > > > > change. Privileges are limited to the authenticated > > Polaris user. > > > > > > > > > Supports > > > > > > > > > > OAuth2 client credentials and direct bearer tokens. > > > > > > > > > > > > > > > > > > > > This bridges Polaris with LLM-based tools like Claude and > > Cursor, > > > > > > > > laying > > > > > > > > > > the groundwork for agentic catalog operations. > > > > > > > > > > > > > > > > > > > > Design Doc: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1yA_d3netDzgicn0z5HEGZIIOHcCgTJC1eKx_DTZPXps/edit?usp=sharing > > > > > > > > > > A Java-based, working POC: > > > > > > > https://github.com/apache/polaris/pull/2803 > > > > > > > > . > > > > > > > > > I > > > > > > > > > > demoed it at this event https://luma.com/pxikwty3. > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
