Thanks a lot for the summary, Adam! This captures most of the urgent items we need to move forward.
One idea to consider is creating a separate project under Polaris dedicated to the MCP server. Managing multiple tools in a single repo(Polaris-tool) can make releases quite challenging, since: • Their release cadences differ. • License requirements may vary. • They could end up blocking each other’s releases, reducing flexibility. Having an independent project would make development and release management much cleaner and more agile. Alternatively, placing the MCP Server as a client module within the main Polaris repo could also simplify the release process, we wouldn’t need to maintain complex mapping tables between MCP Server and Polaris versions. It’s not just about REST API dependencies; in fact, the REST clients are relatively trivial compared to the context coupling an MCP server requires, which may naturally align with a Polaris release. Yufei On Mon, Nov 3, 2025 at 12:46 PM Adam Christian < [email protected]> wrote: > Just so that folks are aware, Yufei & I just chatted and here's our > updates: > > 1. Python POC*: *While Yufei's POC is in Java, there is a richer ecosystem > in Python for AI, ML, & data science tools. Yufei is going to drive a > Python POC to see whether we can leverage some of the larger ecosystem and > whether that makes the code easier to maintain. > 2. Repository: We chatted, but did not get to a solid consensus. There were > some concerns Yufei mentioned about Polaris-Tools: > 2a. Polaris-Tools has multiple tools so semantic versioning is not as > indicative of where the tool is. For example, folks have leveraged the > Iceberg Catalog Migrator for a while, but the MCP Server would be new. Both > of these would be released under the same version, so it does not > necessarily indicate how stable the particular tool is. A remedy to that > would be to have a separate repository. I think this is going to be true > for all of our various tools, so we will just need to figure out how to > handle that. > 2b. We will have to ensure that all of the licensing checks work > properly for the release of Polaris-Tools. Obviously, this is true whenever > we decide to release Polaris-Tools, but it might incur additional overhead. > 3. Next Steps: Yufei is going to drive the Python POC and, then, as a > community, we can chat about which way to go around the large open > discussions. > > Yufei can correct me if I misspoke on any of this stuff. > > Go community! > > Adam > > On Mon, Nov 3, 2025 at 12:59 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Yufei, > > > > It's an interesting proposal. Thanks for starting a doc on it! > > > > Re: code location, I tend to agree with Eric's proposal to put it into > > polaris-tools. > > > > I suppose the MCP server evolution does not have to follow the Server > > evolution step by step. In fact, I'd expect the MCP server to be > revisioned > > out-of-sync with the Polaris Server to accommodate features on the AI > > (agent) side. It only needs to consume the REST APIs from the server, and > > those APIs are fairly stable to allow for independent releases. > > > > Cheers, > > Dmitri. > > > > > > > > On Thu, Oct 30, 2025 at 8:34 PM Eric Maynard <[email protected]> > > wrote: > > > > > This is a pretty cool project! > > > > > > I recall that originally the idea behind polaris-tools was that things > > > which are in the Polaris “ecosystem” could start there and potentially > be > > > folded into the core project if they wind up being really useful or > > widely > > > used. This hasn’t happened yet but I feel like we shouldn’t be afraid > to > > > put things in tools and move them later. > > > > > > I do view the MCP server as a kind of client to Polaris, and I agree > that > > > running it locally is probably the easiest thing for now. > > > > > > —EM > > > > > > On Thu, Oct 30, 2025 at 3:58 PM Yufei Gu <[email protected]> wrote: > > > > > > > Hi Adam, thanks for chiming in! > > > > > > > > I’m okay with either polaris (main) or polaris-tools. The POC > currently > > > > lives in the main repo, but it’s a standalone JAR, it neither depends > > on > > > > other modules nor has modules depending on it. Another option is to > > > > consider it as a client. We could put it along with the CLI client > > under > > > > "/client". These are all minor points. I will move it to the tool > repo > > if > > > > people feel strongly about it. > > > > > > > > It's a good idea to have an Iceberg MCP server. We might think of it > > as a > > > > separate effort. > > > > > > > > It's a POC, we will definitely add more context. And context > > engineering > > > > isn't just a one-time effort. It's a continuing project. I'm still > > > > trying to figure out a better way to do so, a better way means not > too > > > much > > > > also not too less and more accurate. With that said, a remote MCP > > server > > > is > > > > probably a better option, as the context update would be centralized. > > > > > > > > For availability and rolling upgrades, it’s fine that the MCP server > is > > > > local for now. We can address those concerns when we build out the > > remote > > > > MCP server. > > > > > > > > Yufei > > > > > > > > > > > > On Thu, Oct 30, 2025 at 7:44 AM Adam Christian < > > > > [email protected]> wrote: > > > > > > > > > This is fantastic, Yufei! I love this idea and the demo is > wonderful. > > > > > > > > > > A few thoughts: > > > > > > > > > > 1. I agree with your assessment that an MCP Server would > > complement > > > > > Polaris' other interfaces - REST, the python client, & the > future > > > UI. > > > > > This > > > > > adds to the growing Polaris ecosystem and I know people will > find > > > this > > > > > helpful. > > > > > 2. I agree with your approach of building an MCP Server rather > > than > > > > > using some sort of RAG approach (adding it to the chat context). > > > It's > > > > a > > > > > seamless sort of integration that we see other vendors > leveraging > > as > > > > > well. > > > > > 3. I am unsure of whether this work should be integrated into > the > > > > > Polaris repository or the Polaris-Tools repository. I don't know > > how > > > > we > > > > > as > > > > > a community make the distinction. I know that the UI is going to > > be > > > > put > > > > > into the Polaris-Tools repository. Here's my understanding so > far, > > > but > > > > > I'd > > > > > love to chat about it: > > > > > 1. Polaris-Tools Pros: > > > > > 1. We don't want any hard dependencies between the core > > > Polaris > > > > > codebase and the MCP Server. You have done a phenomenal > job > > > > > at doing this > > > > > by making the MCP Server be a shim between the REST API > and > > > the > > > > AI > > > > > Application (Claude Desktop, 5ire, etc). > > > > > 2. This work might not be applicable for core use cases, > so > > it > > > > > would be an additional series of packages that would be > > > > > unused by users and > > > > > cause overhead for building and maintaining. > > > > > 2. Polaris Pros: > > > > > 1. It can leverage common infra (Gradle, etc). > > > > > 2. It'll be faster to develop. > > > > > 3. It'll be easier to keep in sync with the REST APIs > > > (although > > > > > that applies to the UI as well). > > > > > 4. I'm wondering if we should create an Iceberg Catalog MCP > > > Server > > > > > and a Polaris MCP Server. A lot of the tools that you created > > would > > > > > benefit > > > > > the wider Iceberg Catalog ecosystem and are not > Polaris-specific. > > > For > > > > > example, the polaris-namespace-request and the > > polaris-table-request > > > > are > > > > > just Iceberg Catalog operations. What are your thoughts there? > > > > > 5. For some of the issues that you have observed with setting > > > > > privileges, I believe that we might be able to improve the > > accuracy > > > by > > > > > updating the tools' descriptions. You & I can chat about this > > > offline, > > > > > but > > > > > from my analysis of the code, we can make those descriptions > much > > > > richer > > > > > and that'll increase accuracy without too much additional work. > > I've > > > > > seen > > > > > this in other MCP Servers that I've built. But, we can handle > this > > > > after > > > > > we've released the first version. It's a small change. > > > > > 6. I know this is early days, but have you thought about what > sort > > > of > > > > > availability you would want with the MCP Server? Like, Polaris > > tries > > > > to > > > > > support rolling upgrades [1]. Is that your intention here? > > > > > > > > > > Thanks for doing all this work. I'm excited about where this is > > going! > > > > > > > > > > [1] - https://polaris.apache.org/in-dev/unreleased/evolution/ > > > > > > > > > > Cheers, > > > > > > > > > > Adam > > > > > > > > > > On Tue, Oct 28, 2025 at 4:30 PM Yufei Gu <[email protected]> > > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > I’d like to propose the Polaris MCP Server, which lets users and > AI > > > > > agents > > > > > > interact with Polaris using natural language instead of REST or > > CLI. > > > > > > Example: “Show me all tables in prod.sales updated in the last 7 > > > days.” > > > > > > > > > > > > It runs locally, connects to Polaris via REST, and requires no > > > backend > > > > > > change. Privileges are limited to the authenticated Polaris user. > > > > > Supports > > > > > > OAuth2 client credentials and direct bearer tokens. > > > > > > > > > > > > This bridges Polaris with LLM-based tools like Claude and Cursor, > > > > laying > > > > > > the groundwork for agentic catalog operations. > > > > > > > > > > > > Design Doc: > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1yA_d3netDzgicn0z5HEGZIIOHcCgTJC1eKx_DTZPXps/edit?usp=sharing > > > > > > A Java-based, working POC: > > > https://github.com/apache/polaris/pull/2803 > > > > . > > > > > I > > > > > > demoed it at this event https://luma.com/pxikwty3. > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > >
