Thanks Adam and Dmitri for chiming in with suggestions! Tests will be added when it's ready for review.
It looks like more people lean toward moving the MCP Server to the Polaris-Tools repo. I’ll move it there if there are no objections. Yufei On Wed, Nov 5, 2025 at 8:50 AM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Yufei, > > The python MCP impl. does look simpler than the java impl. Thanks for > making it! > > I wonder about testing, though. For a POC manual testing is sufficient, of > course, but when the code is merged and will need to be maintained, having > automated tests is essential. I hope they can be done using the server > docker image and some pre-constructed requests (to avoid running a full AI > agent in tests). WDYT? > > Thanks, > Dmitri. > > On Wed, Nov 5, 2025 at 1:05 AM Yufei Gu <[email protected]> wrote: > > > Indeed, in our last two discussions on the Spark Client and NonSQL > > implementation, we decided to include both in the main repo. > > > > To echo the previous discussion with Adam, I’ve also created a > Python-based > > MCP Server POC here: https://github.com/apache/polaris/pull/2980. It > uses > > `uv` as the package manager and is built on the fastMCP library (Apache > > License 2.0). My manual test against previous use cases worked well. It > > does feel noticeably simpler than the Java version. Maybe we should go > with > > the Python one. Thoughts? > > > > Yufei > > > > > > On Tue, Nov 4, 2025 at 4:25 AM Robert Stupp <[email protected]> wrote: > > > > > Hi all, > > > > > > The discussion about one repo per tool or one tools repo sounds quite > > > familiar to me. Haven't we had this discussion before and > > > intentionally went with the single repo approach? > > > > > > Robert > > > > > > > > > On Tue, Nov 4, 2025 at 7:41 AM Jean-Baptiste Onofré <[email protected]> > > > wrote: > > > > > > > > Hi > > > > > > > > Repo and release are two different things: you can have one repo with > > > > submodules released independently. > > > > > > > > If MCP server goes in polaris main repo, than other tools could be > > > > there too (console, catalog migrator, etc). But I don't think it's a > > > > good idea due to the LICENSE/NOTICE overhead and also clear > separation > > > > of concerns. > > > > > > > > My preference is still about polaris-tool or a each "tool" in its own > > > > repository (polaris-mcp, polaris-console, ...). > > > > For instance, it's what I'm doing in Karaf (you have karaf, > > > > karaf-cellar, karaf-decanter, karaf-cave, karaf-minho, all related to > > > > karaf but different tools/purposes, with different version and > release > > > > cycle). Camel is also doing the same (camel-core, camel-spring-boot, > > > > camel-quarkus, camel-karaf, ...). > > > > > > > > Regards > > > > JB > > > > > > > > On Mon, Nov 3, 2025 at 10:57 PM Yufei Gu <[email protected]> > wrote: > > > > > > > > > > Thanks a lot for the summary, Adam! This captures most of the > urgent > > > items > > > > > we need to move forward. > > > > > > > > > > One idea to consider is creating a separate project under Polaris > > > dedicated > > > > > to the MCP server. Managing multiple tools in a single > > > repo(Polaris-tool) > > > > > can make releases quite challenging, since: > > > > > • Their release cadences differ. > > > > > • License requirements may vary. > > > > > • They could end up blocking each other’s releases, reducing > > > flexibility. > > > > > > > > > > Having an independent project would make development and release > > > management > > > > > much cleaner and more agile. > > > > > > > > > > Alternatively, placing the MCP Server as a client module within the > > > main > > > > > Polaris repo could also simplify the release process, we wouldn’t > > need > > > to > > > > > maintain complex mapping tables between MCP Server and Polaris > > > versions. > > > > > It’s not just about REST API dependencies; in fact, the REST > clients > > > are > > > > > relatively trivial compared to the context coupling an MCP server > > > requires, > > > > > which may naturally align with a Polaris release. > > > > > > > > > > Yufei > > > > > > > > > > > > > > > On Mon, Nov 3, 2025 at 12:46 PM Adam Christian < > > > > > [email protected]> wrote: > > > > > > > > > > > Just so that folks are aware, Yufei & I just chatted and here's > our > > > > > > updates: > > > > > > > > > > > > 1. Python POC*: *While Yufei's POC is in Java, there is a richer > > > ecosystem > > > > > > in Python for AI, ML, & data science tools. Yufei is going to > > drive a > > > > > > Python POC to see whether we can leverage some of the larger > > > ecosystem and > > > > > > whether that makes the code easier to maintain. > > > > > > 2. Repository: We chatted, but did not get to a solid consensus. > > > There were > > > > > > some concerns Yufei mentioned about Polaris-Tools: > > > > > > 2a. Polaris-Tools has multiple tools so semantic versioning > is > > > not as > > > > > > indicative of where the tool is. For example, folks have > leveraged > > > the > > > > > > Iceberg Catalog Migrator for a while, but the MCP Server would be > > > new. Both > > > > > > of these would be released under the same version, so it does not > > > > > > necessarily indicate how stable the particular tool is. A remedy > to > > > that > > > > > > would be to have a separate repository. I think this is going to > be > > > true > > > > > > for all of our various tools, so we will just need to figure out > > how > > > to > > > > > > handle that. > > > > > > 2b. We will have to ensure that all of the licensing checks > > work > > > > > > properly for the release of Polaris-Tools. Obviously, this is > true > > > whenever > > > > > > we decide to release Polaris-Tools, but it might incur additional > > > overhead. > > > > > > 3. Next Steps: Yufei is going to drive the Python POC and, then, > > as a > > > > > > community, we can chat about which way to go around the large > open > > > > > > discussions. > > > > > > > > > > > > Yufei can correct me if I misspoke on any of this stuff. > > > > > > > > > > > > Go community! > > > > > > > > > > > > Adam > > > > > > > > > > > > On Mon, Nov 3, 2025 at 12:59 PM Dmitri Bourlatchkov < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > > > It's an interesting proposal. Thanks for starting a doc on it! > > > > > > > > > > > > > > Re: code location, I tend to agree with Eric's proposal to put > it > > > into > > > > > > > polaris-tools. > > > > > > > > > > > > > > I suppose the MCP server evolution does not have to follow the > > > Server > > > > > > > evolution step by step. In fact, I'd expect the MCP server to > be > > > > > > revisioned > > > > > > > out-of-sync with the Polaris Server to accommodate features on > > the > > > AI > > > > > > > (agent) side. It only needs to consume the REST APIs from the > > > server, and > > > > > > > those APIs are fairly stable to allow for independent releases. > > > > > > > > > > > > > > Cheers, > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 30, 2025 at 8:34 PM Eric Maynard < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > This is a pretty cool project! > > > > > > > > > > > > > > > > I recall that originally the idea behind polaris-tools was > that > > > things > > > > > > > > which are in the Polaris “ecosystem” could start there and > > > potentially > > > > > > be > > > > > > > > folded into the core project if they wind up being really > > useful > > > or > > > > > > > widely > > > > > > > > used. This hasn’t happened yet but I feel like we shouldn’t > be > > > afraid > > > > > > to > > > > > > > > put things in tools and move them later. > > > > > > > > > > > > > > > > I do view the MCP server as a kind of client to Polaris, and > I > > > agree > > > > > > that > > > > > > > > running it locally is probably the easiest thing for now. > > > > > > > > > > > > > > > > —EM > > > > > > > > > > > > > > > > On Thu, Oct 30, 2025 at 3:58 PM Yufei Gu < > [email protected] > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi Adam, thanks for chiming in! > > > > > > > > > > > > > > > > > > I’m okay with either polaris (main) or polaris-tools. The > POC > > > > > > currently > > > > > > > > > lives in the main repo, but it’s a standalone JAR, it > neither > > > depends > > > > > > > on > > > > > > > > > other modules nor has modules depending on it. Another > option > > > is to > > > > > > > > > consider it as a client. We could put it along with the CLI > > > client > > > > > > > under > > > > > > > > > "/client". These are all minor points. I will move it to > the > > > tool > > > > > > repo > > > > > > > if > > > > > > > > > people feel strongly about it. > > > > > > > > > > > > > > > > > > It's a good idea to have an Iceberg MCP server. We might > > think > > > of it > > > > > > > as a > > > > > > > > > separate effort. > > > > > > > > > > > > > > > > > > It's a POC, we will definitely add more context. And > context > > > > > > > engineering > > > > > > > > > isn't just a one-time effort. It's a continuing project. > I'm > > > still > > > > > > > > > trying to figure out a better way to do so, a better way > > means > > > not > > > > > > too > > > > > > > > much > > > > > > > > > also not too less and more accurate. With that said, a > remote > > > MCP > > > > > > > server > > > > > > > > is > > > > > > > > > probably a better option, as the context update would be > > > centralized. > > > > > > > > > > > > > > > > > > For availability and rolling upgrades, it’s fine that the > MCP > > > server > > > > > > is > > > > > > > > > local for now. We can address those concerns when we build > > out > > > the > > > > > > > remote > > > > > > > > > MCP server. > > > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 30, 2025 at 7:44 AM Adam Christian < > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > This is fantastic, Yufei! I love this idea and the demo > is > > > > > > wonderful. > > > > > > > > > > > > > > > > > > > > A few thoughts: > > > > > > > > > > > > > > > > > > > > 1. I agree with your assessment that an MCP Server > would > > > > > > > complement > > > > > > > > > > Polaris' other interfaces - REST, the python client, & > > the > > > > > > future > > > > > > > > UI. > > > > > > > > > > This > > > > > > > > > > adds to the growing Polaris ecosystem and I know > people > > > will > > > > > > find > > > > > > > > this > > > > > > > > > > helpful. > > > > > > > > > > 2. I agree with your approach of building an MCP > Server > > > rather > > > > > > > than > > > > > > > > > > using some sort of RAG approach (adding it to the chat > > > context). > > > > > > > > It's > > > > > > > > > a > > > > > > > > > > seamless sort of integration that we see other vendors > > > > > > leveraging > > > > > > > as > > > > > > > > > > well. > > > > > > > > > > 3. I am unsure of whether this work should be > integrated > > > into > > > > > > the > > > > > > > > > > Polaris repository or the Polaris-Tools repository. I > > > don't know > > > > > > > how > > > > > > > > > we > > > > > > > > > > as > > > > > > > > > > a community make the distinction. I know that the UI > is > > > going to > > > > > > > be > > > > > > > > > put > > > > > > > > > > into the Polaris-Tools repository. Here's my > > > understanding so > > > > > > far, > > > > > > > > but > > > > > > > > > > I'd > > > > > > > > > > love to chat about it: > > > > > > > > > > 1. Polaris-Tools Pros: > > > > > > > > > > 1. We don't want any hard dependencies between > the > > > core > > > > > > > > Polaris > > > > > > > > > > codebase and the MCP Server. You have done a > > > phenomenal > > > > > > job > > > > > > > > > > at doing this > > > > > > > > > > by making the MCP Server be a shim between the > > REST > > > API > > > > > > and > > > > > > > > the > > > > > > > > > AI > > > > > > > > > > Application (Claude Desktop, 5ire, etc). > > > > > > > > > > 2. This work might not be applicable for core > use > > > cases, > > > > > > so > > > > > > > it > > > > > > > > > > would be an additional series of packages that > > > would be > > > > > > > > > > unused by users and > > > > > > > > > > cause overhead for building and maintaining. > > > > > > > > > > 2. Polaris Pros: > > > > > > > > > > 1. It can leverage common infra (Gradle, etc). > > > > > > > > > > 2. It'll be faster to develop. > > > > > > > > > > 3. It'll be easier to keep in sync with the REST > > > APIs > > > > > > > > (although > > > > > > > > > > that applies to the UI as well). > > > > > > > > > > 4. I'm wondering if we should create an Iceberg > > > Catalog MCP > > > > > > > > Server > > > > > > > > > > and a Polaris MCP Server. A lot of the tools that you > > > created > > > > > > > would > > > > > > > > > > benefit > > > > > > > > > > the wider Iceberg Catalog ecosystem and are not > > > > > > Polaris-specific. > > > > > > > > For > > > > > > > > > > example, the polaris-namespace-request and the > > > > > > > polaris-table-request > > > > > > > > > are > > > > > > > > > > just Iceberg Catalog operations. What are your > thoughts > > > there? > > > > > > > > > > 5. For some of the issues that you have observed with > > > setting > > > > > > > > > > privileges, I believe that we might be able to improve > > the > > > > > > > accuracy > > > > > > > > by > > > > > > > > > > updating the tools' descriptions. You & I can chat > about > > > this > > > > > > > > offline, > > > > > > > > > > but > > > > > > > > > > from my analysis of the code, we can make those > > > descriptions > > > > > > much > > > > > > > > > richer > > > > > > > > > > and that'll increase accuracy without too much > > additional > > > work. > > > > > > > I've > > > > > > > > > > seen > > > > > > > > > > this in other MCP Servers that I've built. But, we can > > > handle > > > > > > this > > > > > > > > > after > > > > > > > > > > we've released the first version. It's a small change. > > > > > > > > > > 6. I know this is early days, but have you thought > about > > > what > > > > > > sort > > > > > > > > of > > > > > > > > > > availability you would want with the MCP Server? Like, > > > Polaris > > > > > > > tries > > > > > > > > > to > > > > > > > > > > support rolling upgrades [1]. Is that your intention > > here? > > > > > > > > > > > > > > > > > > > > Thanks for doing all this work. I'm excited about where > > this > > > is > > > > > > > going! > > > > > > > > > > > > > > > > > > > > [1] - > > > https://polaris.apache.org/in-dev/unreleased/evolution/ > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > > > > > Adam > > > > > > > > > > > > > > > > > > > > On Tue, Oct 28, 2025 at 4:30 PM Yufei Gu < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > I’d like to propose the Polaris MCP Server, which lets > > > users and > > > > > > AI > > > > > > > > > > agents > > > > > > > > > > > interact with Polaris using natural language instead of > > > REST or > > > > > > > CLI. > > > > > > > > > > > Example: “Show me all tables in prod.sales updated in > the > > > last 7 > > > > > > > > days.” > > > > > > > > > > > > > > > > > > > > > > It runs locally, connects to Polaris via REST, and > > > requires no > > > > > > > > backend > > > > > > > > > > > change. Privileges are limited to the authenticated > > > Polaris user. > > > > > > > > > > Supports > > > > > > > > > > > OAuth2 client credentials and direct bearer tokens. > > > > > > > > > > > > > > > > > > > > > > This bridges Polaris with LLM-based tools like Claude > and > > > Cursor, > > > > > > > > > laying > > > > > > > > > > > the groundwork for agentic catalog operations. > > > > > > > > > > > > > > > > > > > > > > Design Doc: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1yA_d3netDzgicn0z5HEGZIIOHcCgTJC1eKx_DTZPXps/edit?usp=sharing > > > > > > > > > > > A Java-based, working POC: > > > > > > > > https://github.com/apache/polaris/pull/2803 > > > > > > > > > . > > > > > > > > > > I > > > > > > > > > > > demoed it at this event https://luma.com/pxikwty3. > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
