Hi Greg, Thank you, this genuinely means a lot. I am glad to accept your offer to Champion the effort, and I appreciate you reaching out to line up mentors. Coming from someone with your history and leadership at the Foundation, that is exactly the kind of guidance I was hoping to find here.
The repository is public so you and prospective mentors can read the code before any vote: https://github.com/vaquarkhan/aegis-mcp-gateway. It builds green on JDK 21 with the full test suite passing, and the design-conformance matrix marks each claim 'Done' or 'Roadmap' so the maturity line stays explicit. To be honest: four adapters (Flink, Kafka, Spark, Iceberg) are in-depth, and the rest are deliberately thin. I am completely open to trimming or quarantining the thin set if the community would rather see a smaller, sharper initial contribution during the bootstrap phase. A few responses to your points: *On your own MCP build*: I would be happy to help on a timeline that works for both of us. The gateway core is engine-agnostic, and adapters plug in through a small service provider interface, so bringing up a new MCP surface is mostly mapping tool definitions and backend calls. The core applies the shared governance chain uniformly: identity, scope, policy, approvals, egress defense, rate limiting, redaction, and audit. The repo has a getting-started guide, runnable examples, and an adapters guide to start from. On the Foundation's existing MCP servers: I would welcome composing with those rather than replacing them. Where an engine or server already ships an MCP surface, the intent is for the gateway to sit in front of it for governance rather than reimplement it. If you point me at the ones you have in mind, I will take a look and share a short write-up on how they might fit. Since you are stepping in as Champion, I would love to defer to your experience on how best to proceed. Please let me know how you would like to handle mentor recruitment, and what you need from me regarding next steps for IP clearance, naming, or codebase scope before we eventually move toward a formal vote. Thanks again for stepping in. Best regards, Viquar Khan https://www.linkedin.com/in/vaquar-khan-b695577/ On Thu, Aug 6, 2026 at 1:37 PM Greg Stein <[email protected]> wrote: > This is great stuff, Viquar! ... I'd be happy to Champion this effort > through the Incubator. I already have a need to build an MCP from > scratch/zero, and would like to use exactly this kind of framework to build > that. > > We have a few existing MCP servers at the Foundation today, which might be > able to leverage this framework, too. > > I'll poke some people to get some additional Mentors. > > My background: 28 years at Apache, current Director, past Chairman, initial > Incubator member, current Apache RAI participant, past Infra, Subversion, > HTTPD, and other projects. > > Cheers, > -g > > > On Tue, Aug 4, 2026 at 12:18 AM vaquar khan <[email protected]> > wrote: > > > Hello everyone, > > I would like to propose a project called Aegis MCP Governance Gateway for > > the Apache Incubator, and start a discussion. It is a working reference > > implementation, and I am bringing it here mainly because it needs a > > community rather than a single author, which I state plainly in the > > proposal. > > *In short: *when an AI agent needs to operate data systems like Flink, > > Kafka, Spark, or Iceberg, it usually drives each engine control plane > > directly or connects to a pile of separate MCP servers, each with its own > > security posture, no shared audit, and no common policy. An MCP server > that > > can reach a control plane is a privileged client, so shipping one safely > > means shipping an access-control product. Aegis puts one governed control > > plane in front of all of those engines, so every agent action runs > through > > a single fail-closed chain of identity, scope, policy, approval for > > destructive actions, egress and request-forgery defense, rate limiting, > > output redaction, and a tamper-evident audit trail. Engines plug in > through > > a small service provider interface and contribute no policy. > > Current state, stated honestly. There is a working Java 21 build: a > gateway > > core, four in-depth adapters (Flink, Kafka, Spark, Iceberg), thinner > > adapters for other Apache systems, and a runnable distribution. It builds > > cleanly with 309 tests passing across all modules. A published > > design-conformance matrix maps every design claim to Done or Roadmap. > > Several capabilities ship as roadmap items and are not sold as done: > > externalized high-availability state, CIMD and SPIFFE identity, a full > > Cedar runtime, vault-backed credentials, and an OpenTelemetry SDK bridge. > > Parts of the code were developed with AI-assisted tools under my > direction > > and review, which I disclose up front for IP clearance. > > *Biggest risk: *The project is currently a single contributor. Growing a > > diverse, multi-organization community is the main reason I am bringing it > > here. I am currently in active conversations with engineers in the > > ecosystem and expect to add a few of them as initial committers in the > next > > few days, prior to a formal vote > > What I am looking for now: a Champion (an ASF Member) and mentors (three > or > > more IPMC members), ideally including people from the Flink, Kafka, > Spark, > > or Iceberg communities. The full proposal follows, and the reference > > implementation is at https://github.com/vaquarkhan/aegis-mcp-gateway. I > > welcome feedback on scope, name (a name search is still to be run), > risks, > > and anything I have missed. > > Thank you, > > Viquar Khan > > https://www.linkedin.com/in/vaquar-khan-b695577/ > > > > Apache Incubator Proposal: Aegis MCP Governance Gateway > > This is the proposal for entry into the Apache Incubator, written for > > discussion on the Incubator general list. Aegis is a working codename; a > > name search will be run and the ASF trademark process followed if the > > project is accepted. The document follows the ASF Incubator proposal > > template and states the maturity of the code honestly, including what is > > implemented and what is deferred to the roadmap. > > Field > > Value > > Proposal name > > Aegis MCP Governance Gateway (codename) > > Proposed short name > > aegis (subject to name search) > > Proposal date > > to be set at submission > > Sponsor > > Apache Incubator PMC > > Champion > > to be identified (ASF Member) > > Mentors > > to be recruited (three or more IPMC members) > > Initial code > > Java 21 multi-module reference implementation, 36 Maven modules, 309 > tests > > passing > > Reference repository > > https://github.com/vaquarkhan/aegis-mcp-gateway (private for now, access > > on > > request) > > License > > Apache License 2.0 > > Discussion list > > general at incubator.apache.org > > Abstract > > Aegis MCP Governance Gateway is a vendor-neutral, secure-by-default Model > > Context Protocol (MCP) gateway that puts one governed, auditable control > > plane in front of many Apache data and streaming engines, so an AI agent > > never talks to Flink, Kafka, Spark, or Iceberg directly. Every agent > action > > passes one fail-closed governance chain: secure-by-default tool exposure, > > per-caller authentication and scope, a policy decision point, approval > for > > destructive operations, egress and server-side request forgery defense, > > rate limiting, circuit breaking, prompt-injection screening, output > > bounding and redaction, tool-catalog integrity, and a tamper-evident > > hash-chained audit trail. Engines plug in through a small service > provider > > interface and contribute only tool definitions and backend calls. > > Proposal > > We propose to develop, under Apache governance, a Layer 7 policy MCP > > gateway plus a family of engine adapters. A proxy forwards bytes and a > > router dispatches by capability, but neither is a trust boundary. A > gateway > > is: it knows the tool, the principal, the budget, and the policy for > every > > call, and it can deny, redact, gate, and record. The gateway core is > > stateless at the protocol edge so it can scale behind an ordinary load > > balancer once shared governance state is externalized, and it is designed > > to align with OAuth 2.1, SPIFFE, SLSA, OpenTelemetry, and the NIST AI > Risk > > Management Framework and ISO/IEC 42001. > > The initial contribution is a working Java reference implementation: a > > reusable gateway core, four in-depth engine adapters (Flink, Kafka, > Spark, > > Iceberg), and a further set of thinner adapters for the wider Apache data > > platform. During incubation the priorities are community building, IP > > clearance, and promoting the roadmap items to full implementations. > > Background > > The Model Context Protocol has become a common way for AI agents to reach > > tools and data. As organizations connect agents to distributed Apache > data > > estates, an agent that needs Flink, Kafka, Spark, and Iceberg must either > > drive each engine control plane directly or connect to a sprawl of > > independent, mostly read-only MCP servers, each with its own security > > posture, no shared audit, and no unified policy. > > An MCP server that can reach a control plane (a JobManager REST API, a > > Kafka AdminClient, a SQL gateway, an Iceberg catalog) is a privileged > > client. Shipping one safely is effectively shipping an access-control > > product: authentication, authorization, approval for destructive actions, > > redaction of secrets and personal data, rate and cost limits, and > forensic > > audit. Many existing options lack these controls and carry > > single-maintainer sustainability risk. A single governed gateway, > developed > > in the open under neutral governance, gives the ecosystem a shared > standard > > for how agents operate Apache systems safely. > > Rationale > > A single governed gateway resolves the problems of the decentralized > > approach: > > > > - Security. One place enforces identity, authorization, approval, > > redaction, egress, and audit, instead of each server reimplementing or > > omitting them. > > - Maintainability. Adding an engine is a small adapter, not a new > > security model. There is one chain to review and harden, so the > security > > review surface stays flat as engines are added. > > - Governance. One control plane for policy, budget, and provenance, > with > > a stable deny taxonomy so operators can alert on codes such as > > POLICY_DENIED or APPROVAL_REQUIRED across every engine with one rule. > > > > Doing this under Apache governance keeps it neutral across vendors and > > engines rather than leaving agent access to fragmented single-vendor > > servers. > > Initial goals > > > > 1. Establish the project and practice the Apache Way: transparency, > > consensus, meritocracy, and community over code. > > 2. Complete IP clearance for the existing code and land it under > > Apache-2.0 with correct LICENSE and NOTICE files. > > 3. Publish and stabilize the adapter service provider interface so > > per-engine adapters are small, reviewable contributions. > > 4. Promote the roadmap items to full implementations: externalized > > high-availability state, CIMD and SPIFFE identity, a full Cedar > runtime, > > vault-backed credential exchange, and an OpenTelemetry SDK bridge. > > 5. Produce the first Apache source releases and grow a diverse, > > multi-organization contributor base. > > > > Current status > > Aegis is a new codebase with a working multi-module reference > > implementation. It is not yet a community. The design, threat model, > > low-level design, and standards research are complete and public. The > code > > builds green and is tested. A published design-conformance matrix maps > > every design claim to Done or Roadmap, so the maturity line is explicit. > > Verified build and test state > > > > - 36 Maven modules: a gateway core, adapters for 33 Apache engines > (four > > in-depth today, Flink, Kafka, Spark, and Iceberg, and 29 thinner > > integrations covering Cassandra, HBase, Hive, Pulsar, Airflow, NiFi, > > Druid, > > Pinot, Doris, Impala, Calcite, Hudi, Paimon, Kudu, Ignite, BookKeeper, > > Ozone, Hadoop, ZooKeeper, Solr, Superset, Arrow Flight, Beam, Storm, > > Flume, > > ActiveMQ, CouchDB, Ranger, and Atlas), and a shaded runnable > > distribution. > > These are engine adapters the gateway supports, which is distinct from > > external adopters. > > - Builds with JDK 21 and runs on JDK 17 or newer with Maven 3.9 or > > newer. > > - 309 unit and integration tests across all modules, all passing, zero > > failures and zero errors on a clean build. > > > > Implemented in 0.1.0 > > > > - The governance chain with ordered steps and first-denial-wins > > semantics. > > - Identity: shared bearer and a hashed multi-caller token file with > > per-caller scope, and an OAuth 2.1 resource-server mode with JWKS > > validation. Constant-time secret comparison and fail-closed startup > > validation. > > - Approvals: HMAC-SHA256 signed, TTL-bounded, single-use, > > tool-and-scope-bound tokens with replay rejection. > > - Policy: a builtin decision point, a live fail-closed OPA HTTP > decision > > point, and a Cedar-style deny-file or HTTP delegate. > > - Per-caller outbound credential pass-through so a downstream engine > can > > see a per-caller Authorization value when one was bound at admission. > > - Egress and server-side request forgery defense: allow list plus > > unconditional deny of cloud metadata and non-routable ranges, > including > > numeric IP encodings and resolved addresses, with redirects disabled > on > > outbound clients. > > - Rate limiting, circuit breaking, per-tool timeout, read-only SQL > > guard, identifier validation, output bounding, and > data-loss-prevention > > redaction. > > - Tool-catalog integrity with durable pins and a fail-closed switch, a > > tamper-evident hash-chained audit, Prometheus metrics, and > GenAI-shaped > > span log lines. > > - Two transports: stdio and authenticated Streamable HTTP with TLS, > plus > > health, readiness, and metrics endpoints. A CycloneDX software bill of > > materials is produced at build time. > > > > Deferred to the roadmap (honestly labeled, not sold as implemented) > > > > - CIMD and SPIFFE authentication modes (they refuse to start today). > > - A full Cedar policy runtime, and vault-backed or token-exchange > > credential resolution. > > - Externalized high-availability state (shared nonce, rate, and audit > > stores). Until then, run a single replica; multi-replica replay > > protection > > and rate limits require the shared store. > > - The MCP Tasks extension for long-running jobs, an OpenTelemetry SDK > > bridge, a full Merkle and signature-based reconciliation proof, and > true > > semantic routing and caching. > > > > These are tracked in the roadmap and the design-conformance matrix that > > ship with the code. > > Meritocracy, community, core developers, alignment > > The project will operate meritocratically from day one, with technical > > decisions on the dev list and committership earned through contribution. > > The community is currently centered on the initial author, Viquar Khan, > who > > has around 23 years in software, has worked in the open since 2013, and > has > > contributed to Apache Spark, Apache Iceberg, Apache Kafka, and other > > open-source projects. The author also maintains MCP-related open-source > > packages that have together been downloaded on the order of half a > million > > times, so there is an existing user base to draw contributors from. > Growing > > a committer base across the Flink, Kafka, Spark, and Iceberg communities > > and across employers is the primary goal of incubation. Aegis consumes > > public engine interfaces only and does not fork or modify any engine > core. > > Known risks > > Orphaned products > > Low to moderate. Agentic access to data systems is a growing need and the > > governance gap is real, so demand is unlikely to vanish. The initial > author > > has maintained open-source projects since 2013 rather than abandoning > them, > > and the MCP-related packages already have a real user base, which lowers > > the abandonment risk. Mitigation: build a multi-organization community, > > keep the core small and standards aligned, and keep adapters independent > so > > the project survives the departure of any single contributor. > > Inexperience with open source > > Low. This is not the situation here. The initial author has about 23 > years > > in software, has worked in open source since 2013, and has contributed to > > Apache Spark, Apache Iceberg, Apache Kafka, and other projects, so the > > norms of public development, code review, and releases are familiar. > > Mentors will still help with the ASF-specific mechanics of voting, board > > reports, and IP clearance, which are process details rather than a gap in > > open-source experience. > > Homogenous developers > > This is the real risk and we state it plainly: the podling has one > > committer today. However, we are in active discussions with potential > > co-committers and expect to formally add them to this proposal in the > > coming days. What further reduces this risk is that the author is > already > > embedded in the exact communities this project draws from (Spark, > Iceberg, > > Kafka) and has an existing user base for the MCP packages, so there is a > > concrete pool to recruit from rather than a cold start. Mitigation: > recruit > > committers from more than one organization before graduation, shape the > > adapter contract so per-engine contributions are natural, and use > > good-first-issue and adapter-sized work to onboard newcomers. Growing the > > committer base is the main reason we are here. > > Reliance on salaried developers > > Low current reliance; the work to date is largely independent. The > project > > will attract contributors with varied motivations and employers so no > > single company can stall it. > > Relationships with other Apache products > > Aegis consumes the public interfaces of Flink, Kafka, Spark, Iceberg, and > > other Apache data systems and does not fork or modify them. We will > > coordinate on the relevant dev lists and will not duplicate or compete > with > > an engine that ships its own MCP surface; where an engine has one, Aegis > > composes with it. Aegis is not proposed for any engine core. > > Excessive fascination with the Apache brand > > The project seeks Apache governance for its community model and > neutrality, > > not for the brand. Until and unless accepted, it uses a neutral name and > > does not use the Apache marks or the feather. The breadth of engine > > adapters reflects a common governance surface, not an attempt to claim > the > > ecosystem; only four adapters are in-depth today and the rest are > > explicitly thin. > > Documentation > > Public documentation exists and will be maintained in the repository: an > > end-to-end design with architecture decision records, a low-level design > > with architecture, sequence, and state diagrams, a threat and governance > > model, a design-conformance matrix, standards research, and operations > and > > adapter guides. > > Initial source > > The initial source is the Java multi-module gateway described above, > > currently in a private repository at > > https://github.com/vaquarkhan/aegis-mcp-gateway under the initial > author. > > Access is available on request to anyone reviewing this proposal, and the > > author will make it public before the vote if the community prefers to > read > > the code first. It includes the gateway core, the four in-depth adapters, > > the thinner adapters, the shaded distribution, runnable examples, and the > > documentation set. On acceptance it will be donated under a Software > Grant > > and made public. > > Source and intellectual property submission plan > > The initial author will submit the code under a Software Grant Agreement > > and complete IP clearance early in incubation, with ICLAs from every > > committer. Every source file will carry the standard Apache-2.0 header > with > > correct LICENSE and NOTICE files; any premature ASF-ownership wording > will > > be corrected to the standard header at donation time. > > Transparency on tooling: parts of the codebase were developed with > > AI-assisted coding tools under the author's direction and review. The > > author asserts authorship and the right to contribute the work, and will > > confirm provenance and authorship during IP clearance so the grant is > > clean. We raise this proactively so the IPMC can review it rather than > > discover it. > > External dependencies > > Runtime dependencies are Apache-2.0 or compatible, including the MCP Java > > SDK, a servlet container for the HTTP transport, SLF4J, Logback, and > Nimbus > > for JOSE and JWT, with JUnit for tests. Per-engine adapters add that > engine > > to the official client, for example the Kafka clients library. A full > > dependency review and a CycloneDX software bill of materials are > produced. > > No GPL or ASF category X dependency is introduced into the core. > > Cryptography > > Aegis uses standard, library-based cryptography: HMAC-SHA256 for approval > > tokens, TLS for the HTTP transport, and JSON Web Token validation for > > OAuth. Signature-based integrity is an optional gate. The project will > > comply with ASF export notification requirements. > > Required resources > > Mailing lists > > > > - [email protected] > > - [email protected] > > - [email protected] > > > > Source control > > > > - A Git repository under the ASF (gitbox with a GitHub mirror). > > > > Issue tracking > > > > - ASF Jira or GitHub issues per current ASF practice. > > > > Other > > > > - Website and wiki space, and continuous integration through ASF > > infrastructure. > > > > Initial committers > > > > - Viquar Khan *(Note: We are in active conversations with potential > > contributors and expect to add a few additional initial committers > here > > in > > the next few days before moving to a formal vote.)* > > > > - To be recruited before and during incubation: committers from the > > Flink, Kafka, Spark, and Iceberg communities and from additional > > organizations. Adapter-sized work is designed to make these > > contributions > > natural. > > > > Affiliations > > To be completed as committers are confirmed. Diversity of affiliation is > an > > explicit goal; the current single-affiliation status is the main risk > being > > addressed. > > Sponsors > > Champion > > To be identified. An ASF Member Champion will help finalize this > proposal, > > recruit mentors, and shepherd the discussion and vote on the Incubator > > general list. > > Nominated mentors > > Three or more IPMC members to be recruited, ideally including > participants > > from the Flink, Kafka, and Iceberg communities. > > Sponsoring entity > > The Apache Incubator PMC. > > Roadmap during incubation > > Phase > > Focus > > Bootstrap > > IP clearance, name search, infra, initial committers, first dev-list > > decisions > > 0.2 > > Externalized HA state, vault-backed credentials, OAuth multi-issuer, OTel > > SDK bridge, supply-chain CI (enforcer, CVE scan, signing) > > 0.3 > > CIMD and SPIFFE identity, full Cedar runtime, MCP Tasks, signature-based > > reconciliation proof, semantic routing and caching > > Graduation readiness > > Diverse community, cadence of Apache releases, healthy governance > > References > > > > - Apache Incubator proposal guide: > > https://incubator.apache.org/guides/proposal.html > > - Apache Incubator Cookbook: https://incubator.apache.org/cookbook/ > > - The Apache Way: https://www.apache.org/theapacheway/ > > - Model Context Protocol: https://modelcontextprotocol.io/ > > >
