Indeed, in our last two discussions on the Spark Client and NonSQL implementation, we decided to include both in the main repo.
To echo the previous discussion with Adam, I’ve also created a Python-based MCP Server POC here: https://github.com/apache/polaris/pull/2980. It uses `uv` as the package manager and is built on the fastMCP library (Apache License 2.0). My manual test against previous use cases worked well. It does feel noticeably simpler than the Java version. Maybe we should go with the Python one. Thoughts? Yufei On Tue, Nov 4, 2025 at 4:25 AM Robert Stupp <[email protected]> wrote: > Hi all, > > The discussion about one repo per tool or one tools repo sounds quite > familiar to me. Haven't we had this discussion before and > intentionally went with the single repo approach? > > Robert > > > On Tue, Nov 4, 2025 at 7:41 AM Jean-Baptiste Onofré <[email protected]> > wrote: > > > > Hi > > > > Repo and release are two different things: you can have one repo with > > submodules released independently. > > > > If MCP server goes in polaris main repo, than other tools could be > > there too (console, catalog migrator, etc). But I don't think it's a > > good idea due to the LICENSE/NOTICE overhead and also clear separation > > of concerns. > > > > My preference is still about polaris-tool or a each "tool" in its own > > repository (polaris-mcp, polaris-console, ...). > > For instance, it's what I'm doing in Karaf (you have karaf, > > karaf-cellar, karaf-decanter, karaf-cave, karaf-minho, all related to > > karaf but different tools/purposes, with different version and release > > cycle). Camel is also doing the same (camel-core, camel-spring-boot, > > camel-quarkus, camel-karaf, ...). > > > > Regards > > JB > > > > On Mon, Nov 3, 2025 at 10:57 PM Yufei Gu <[email protected]> wrote: > > > > > > Thanks a lot for the summary, Adam! This captures most of the urgent > items > > > we need to move forward. > > > > > > One idea to consider is creating a separate project under Polaris > dedicated > > > to the MCP server. Managing multiple tools in a single > repo(Polaris-tool) > > > can make releases quite challenging, since: > > > • Their release cadences differ. > > > • License requirements may vary. > > > • They could end up blocking each other’s releases, reducing > flexibility. > > > > > > Having an independent project would make development and release > management > > > much cleaner and more agile. > > > > > > Alternatively, placing the MCP Server as a client module within the > main > > > Polaris repo could also simplify the release process, we wouldn’t need > to > > > maintain complex mapping tables between MCP Server and Polaris > versions. > > > It’s not just about REST API dependencies; in fact, the REST clients > are > > > relatively trivial compared to the context coupling an MCP server > requires, > > > which may naturally align with a Polaris release. > > > > > > Yufei > > > > > > > > > On Mon, Nov 3, 2025 at 12:46 PM Adam Christian < > > > [email protected]> wrote: > > > > > > > Just so that folks are aware, Yufei & I just chatted and here's our > > > > updates: > > > > > > > > 1. Python POC*: *While Yufei's POC is in Java, there is a richer > ecosystem > > > > in Python for AI, ML, & data science tools. Yufei is going to drive a > > > > Python POC to see whether we can leverage some of the larger > ecosystem and > > > > whether that makes the code easier to maintain. > > > > 2. Repository: We chatted, but did not get to a solid consensus. > There were > > > > some concerns Yufei mentioned about Polaris-Tools: > > > > 2a. Polaris-Tools has multiple tools so semantic versioning is > not as > > > > indicative of where the tool is. For example, folks have leveraged > the > > > > Iceberg Catalog Migrator for a while, but the MCP Server would be > new. Both > > > > of these would be released under the same version, so it does not > > > > necessarily indicate how stable the particular tool is. A remedy to > that > > > > would be to have a separate repository. I think this is going to be > true > > > > for all of our various tools, so we will just need to figure out how > to > > > > handle that. > > > > 2b. We will have to ensure that all of the licensing checks work > > > > properly for the release of Polaris-Tools. Obviously, this is true > whenever > > > > we decide to release Polaris-Tools, but it might incur additional > overhead. > > > > 3. Next Steps: Yufei is going to drive the Python POC and, then, as a > > > > community, we can chat about which way to go around the large open > > > > discussions. > > > > > > > > Yufei can correct me if I misspoke on any of this stuff. > > > > > > > > Go community! > > > > > > > > Adam > > > > > > > > On Mon, Nov 3, 2025 at 12:59 PM Dmitri Bourlatchkov < > [email protected]> > > > > wrote: > > > > > > > > > Hi Yufei, > > > > > > > > > > It's an interesting proposal. Thanks for starting a doc on it! > > > > > > > > > > Re: code location, I tend to agree with Eric's proposal to put it > into > > > > > polaris-tools. > > > > > > > > > > I suppose the MCP server evolution does not have to follow the > Server > > > > > evolution step by step. In fact, I'd expect the MCP server to be > > > > revisioned > > > > > out-of-sync with the Polaris Server to accommodate features on the > AI > > > > > (agent) side. It only needs to consume the REST APIs from the > server, and > > > > > those APIs are fairly stable to allow for independent releases. > > > > > > > > > > Cheers, > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > On Thu, Oct 30, 2025 at 8:34 PM Eric Maynard < > [email protected]> > > > > > wrote: > > > > > > > > > > > This is a pretty cool project! > > > > > > > > > > > > I recall that originally the idea behind polaris-tools was that > things > > > > > > which are in the Polaris “ecosystem” could start there and > potentially > > > > be > > > > > > folded into the core project if they wind up being really useful > or > > > > > widely > > > > > > used. This hasn’t happened yet but I feel like we shouldn’t be > afraid > > > > to > > > > > > put things in tools and move them later. > > > > > > > > > > > > I do view the MCP server as a kind of client to Polaris, and I > agree > > > > that > > > > > > running it locally is probably the easiest thing for now. > > > > > > > > > > > > —EM > > > > > > > > > > > > On Thu, Oct 30, 2025 at 3:58 PM Yufei Gu <[email protected]> > wrote: > > > > > > > > > > > > > Hi Adam, thanks for chiming in! > > > > > > > > > > > > > > I’m okay with either polaris (main) or polaris-tools. The POC > > > > currently > > > > > > > lives in the main repo, but it’s a standalone JAR, it neither > depends > > > > > on > > > > > > > other modules nor has modules depending on it. Another option > is to > > > > > > > consider it as a client. We could put it along with the CLI > client > > > > > under > > > > > > > "/client". These are all minor points. I will move it to the > tool > > > > repo > > > > > if > > > > > > > people feel strongly about it. > > > > > > > > > > > > > > It's a good idea to have an Iceberg MCP server. We might think > of it > > > > > as a > > > > > > > separate effort. > > > > > > > > > > > > > > It's a POC, we will definitely add more context. And context > > > > > engineering > > > > > > > isn't just a one-time effort. It's a continuing project. I'm > still > > > > > > > trying to figure out a better way to do so, a better way means > not > > > > too > > > > > > much > > > > > > > also not too less and more accurate. With that said, a remote > MCP > > > > > server > > > > > > is > > > > > > > probably a better option, as the context update would be > centralized. > > > > > > > > > > > > > > For availability and rolling upgrades, it’s fine that the MCP > server > > > > is > > > > > > > local for now. We can address those concerns when we build out > the > > > > > remote > > > > > > > MCP server. > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 30, 2025 at 7:44 AM Adam Christian < > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > This is fantastic, Yufei! I love this idea and the demo is > > > > wonderful. > > > > > > > > > > > > > > > > A few thoughts: > > > > > > > > > > > > > > > > 1. I agree with your assessment that an MCP Server would > > > > > complement > > > > > > > > Polaris' other interfaces - REST, the python client, & the > > > > future > > > > > > UI. > > > > > > > > This > > > > > > > > adds to the growing Polaris ecosystem and I know people > will > > > > find > > > > > > this > > > > > > > > helpful. > > > > > > > > 2. I agree with your approach of building an MCP Server > rather > > > > > than > > > > > > > > using some sort of RAG approach (adding it to the chat > context). > > > > > > It's > > > > > > > a > > > > > > > > seamless sort of integration that we see other vendors > > > > leveraging > > > > > as > > > > > > > > well. > > > > > > > > 3. I am unsure of whether this work should be integrated > into > > > > the > > > > > > > > Polaris repository or the Polaris-Tools repository. I > don't know > > > > > how > > > > > > > we > > > > > > > > as > > > > > > > > a community make the distinction. I know that the UI is > going to > > > > > be > > > > > > > put > > > > > > > > into the Polaris-Tools repository. Here's my > understanding so > > > > far, > > > > > > but > > > > > > > > I'd > > > > > > > > love to chat about it: > > > > > > > > 1. Polaris-Tools Pros: > > > > > > > > 1. We don't want any hard dependencies between the > core > > > > > > Polaris > > > > > > > > codebase and the MCP Server. You have done a > phenomenal > > > > job > > > > > > > > at doing this > > > > > > > > by making the MCP Server be a shim between the REST > API > > > > and > > > > > > the > > > > > > > AI > > > > > > > > Application (Claude Desktop, 5ire, etc). > > > > > > > > 2. This work might not be applicable for core use > cases, > > > > so > > > > > it > > > > > > > > would be an additional series of packages that > would be > > > > > > > > unused by users and > > > > > > > > cause overhead for building and maintaining. > > > > > > > > 2. Polaris Pros: > > > > > > > > 1. It can leverage common infra (Gradle, etc). > > > > > > > > 2. It'll be faster to develop. > > > > > > > > 3. It'll be easier to keep in sync with the REST > APIs > > > > > > (although > > > > > > > > that applies to the UI as well). > > > > > > > > 4. I'm wondering if we should create an Iceberg > Catalog MCP > > > > > > Server > > > > > > > > and a Polaris MCP Server. A lot of the tools that you > created > > > > > would > > > > > > > > benefit > > > > > > > > the wider Iceberg Catalog ecosystem and are not > > > > Polaris-specific. > > > > > > For > > > > > > > > example, the polaris-namespace-request and the > > > > > polaris-table-request > > > > > > > are > > > > > > > > just Iceberg Catalog operations. What are your thoughts > there? > > > > > > > > 5. For some of the issues that you have observed with > setting > > > > > > > > privileges, I believe that we might be able to improve the > > > > > accuracy > > > > > > by > > > > > > > > updating the tools' descriptions. You & I can chat about > this > > > > > > offline, > > > > > > > > but > > > > > > > > from my analysis of the code, we can make those > descriptions > > > > much > > > > > > > richer > > > > > > > > and that'll increase accuracy without too much additional > work. > > > > > I've > > > > > > > > seen > > > > > > > > this in other MCP Servers that I've built. But, we can > handle > > > > this > > > > > > > after > > > > > > > > we've released the first version. It's a small change. > > > > > > > > 6. I know this is early days, but have you thought about > what > > > > sort > > > > > > of > > > > > > > > availability you would want with the MCP Server? Like, > Polaris > > > > > tries > > > > > > > to > > > > > > > > support rolling upgrades [1]. Is that your intention here? > > > > > > > > > > > > > > > > Thanks for doing all this work. I'm excited about where this > is > > > > > going! > > > > > > > > > > > > > > > > [1] - > https://polaris.apache.org/in-dev/unreleased/evolution/ > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > Adam > > > > > > > > > > > > > > > > On Tue, Oct 28, 2025 at 4:30 PM Yufei Gu < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > I’d like to propose the Polaris MCP Server, which lets > users and > > > > AI > > > > > > > > agents > > > > > > > > > interact with Polaris using natural language instead of > REST or > > > > > CLI. > > > > > > > > > Example: “Show me all tables in prod.sales updated in the > last 7 > > > > > > days.” > > > > > > > > > > > > > > > > > > It runs locally, connects to Polaris via REST, and > requires no > > > > > > backend > > > > > > > > > change. Privileges are limited to the authenticated > Polaris user. > > > > > > > > Supports > > > > > > > > > OAuth2 client credentials and direct bearer tokens. > > > > > > > > > > > > > > > > > > This bridges Polaris with LLM-based tools like Claude and > Cursor, > > > > > > > laying > > > > > > > > > the groundwork for agentic catalog operations. > > > > > > > > > > > > > > > > > > Design Doc: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1yA_d3netDzgicn0z5HEGZIIOHcCgTJC1eKx_DTZPXps/edit?usp=sharing > > > > > > > > > A Java-based, working POC: > > > > > > https://github.com/apache/polaris/pull/2803 > > > > > > > . > > > > > > > > I > > > > > > > > > demoed it at this event https://luma.com/pxikwty3. > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
