Hi All, Re: the shating API PR [5446], I'm repeating my GH comment here for visibility.
I'd like the API to be neutral from the authentication perspective. This is how we handle the IRC spec in Polaris. The API declares that clients need to authenticate, but specific details about principals, clients and secrets are delegated to the server. I think this is a sound approach for the sharing API too. Specific details related to native Polaris principals (ConsumerCredential, etc.) can probably remain in the Management API. WDYT? Thanks, Dmitri. [5446] https://github.com/apache/polaris/pull/5446 On Fri, Oct 2, 2026 at 12:50 AM Jean-Baptiste Onofré <[email protected]> wrote: > Hi > > As I said during the community meeting, let's focus on the API PR > first. We will build from there. > > Regards > JB > > On Fri, Oct 2, 2026 at 3:18 AM Dennis Huo <[email protected]> wrote: > > > > +1, thanks for the discussion everyone! > > > > Yes, I meant to flip it to "ready to review" earlier; flipped it now. I'm > > still working through the existing set of comments and may need to come > > back to it in the next few days before getting proper replies, but feel > > free to add additional comments. > > > > Recapping the community sync discussion re: Dmitri's last message, I'm > > fully aligned on building the sharing on top of existing authz > mechanisms, > > instead of being a new mechanism, and indeed external IdP is intended to > be > > a first-class supported use case. > > > > The latest distinction is that the ExternalConsumer isn't fundamentally > > identified by any internal PolarisPrincipal, but rather that it serves > as a > > wrapper that references some identity, whether that identity is defined > in > > an external IdP without any latent in-Polaris representation or if it's a > > hidden PolarisPrincipal (for the Polaris-standalone RBAC use case). > > > > The sharing endpoint via ExternalConsumer is the only layer of > indirection > > at the API entry-point, but then once the set of activePrincipalRoles is > > resolved, the underlying authz engine doesn't behave any differently from > > normal Polaris authz. > > > > On Thu, Oct 1, 2026 at 5:58 PM Yufei Gu <[email protected]> wrote: > > > > > Thanks everyone for the discussion! > > > > > > As a next step discussed in today's community sync, we will start > reviewing > > > the spec change. > > > > > > Hi Dennis, is this PR ready for review? > > > https://github.com/apache/polaris/pull/5446 > > > > > > Yufei > > > > > > > > > On Sat, Sep 26, 2026 at 12:23 PM Travis Bowen <[email protected]> > > > wrote: > > > > > > > Thanks for this discussion Dennis! > > > > > > > > Wanted to weigh in with my thoughts and questions. > > > > > > > > > > > > > > Do you prefer /api/shares-management/v1/ > > > > > instead of adding to /api/management/v1? > > > > > > > > > > > > While I can see the argument for a share specific management endpoint > > > there > > > > doesn't seem to be a clear obvious choice from what I can see - I > lean > > > > towards /api/management as being the central place for all the > > > management - > > > > share or not. As long as we're careful not to leak share types or non > > > share > > > > types into the wrong api (which could be helped by having separate > yamls) > > > > then it feels reasonable to have all under the management api. > > > > > > > > Also went through the doc and PR and made several comments as well. > > > > > > > > On Fri, Sep 18, 2026 at 9:54 AM Dmitri Bourlatchkov < > [email protected]> > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > Prithvi mentioned ExternalConsumer and principals. I'd like to > > > > > reinforce the need to further discuss "consumers" and principals > and > > > how > > > > > they relate to IdP and Polaris Authentication. > > > > > > > > > > From my POV, share access control should not be a new mechanism but > > > > should > > > > > build on top of the existing extensible authentication / > authorization > > > > > framework. Supporting external IdP for shares would be a valuable > > > feature > > > > > IMHO. > > > > > > > > > > Side note: I'm posting comments on PR 5446 too. > > > > > > > > > > Cheers, > > > > > Dmitri. > > > > > > > > > > On Fri, Sep 4, 2026 at 3:30 PM Prithvi S < > [email protected]> > > > > > wrote: > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > Thank you for reviving this, and for the concrete spec in PR > 5446.. > > > the > > > > > > journeys doc was especially helpful for seeing the end-to-end > flow. > > > > > > > > > > > > I agree this is worth doing, and that v1 is more than a tutorial > on > > > > > today's > > > > > > RBAC. A partner principal on /api/catalog is already possible; > what > > > > seems > > > > > > valuable is a first-class share (enumerated members, a restricted > > > > > consumer, > > > > > > a listing as the binding and audit unit) plus a segregated, > read-only > > > > > > Iceberg REST surface with credential vending. Stock Iceberg REST > as > > > the > > > > > > consumer protocol, and keeping Flight and Polaris-to-Polaris > > > federation > > > > > out > > > > > > of v1, all sound right to me. > > > > > > > > > > > > I don't have a strong view yet on the URL layout (management vs > > > shares > > > > > > prefix, nested vs top-level, prefix keyed by share vs listing) > and am > > > > > happy > > > > > > to follow whatever the community prefers there. > > > > > > > > > > > > A few things I wanted to check on the spec itself, several of > these > > > > echo > > > > > > points already raised on the list: > > > > > > > > > > > > 1. Membership as source of truth. If a hidden catalog role is > > > > > independently > > > > > > grantable, a share could be widened without going through the > share > > > > APIs. > > > > > > Would it make sense to say the data plane authorizes against > > > > membership + > > > > > > listing, and that any backing roles are not user-visible / not > > > > > > independently grantable? > > > > > > 2. ExternalConsumer and principals. The Aug 7 recap already > noted we > > > > > should > > > > > > not assume a PrincipalEntity behind every ExternalConsumer. The > > > > journeys > > > > > > still describe granting the consumer a share principal role, I > assume > > > > > that > > > > > > is an implementation sketch for v1 client_id/secret, not the > public > > > > > model? > > > > > > 3. Listing expiration. JB's original note was "share these tables > > > with > > > > > this > > > > > > consumer until this date." Credential expiry is a bit different. > > > Would > > > > a > > > > > > listing-level expiration (and maybe ACTIVE/SUSPENDED) be in > scope for > > > > v1? > > > > > > 4. Time-travel. As raised in the community sync, sharing via > Iceberg > > > > REST > > > > > > means historical snapshots remain readable. Totally fine for v1 > to > > > > > support > > > > > > only full history, would an explicit field on the share help so a > > > later > > > > > > CURRENT_ONLY option is not a breaking surprise? > > > > > > 5. Consumer data plane. The current file is control-plane only, > which > > > > is > > > > > a > > > > > > good first step. It might help to list the Iceberg routes that > are > > > > > mounted, > > > > > > that writes 404, that non-members 404, and that /config does not > > > return > > > > > > provider catalog properties, so implementations don't diverge. > > > > > > > > > > > > Two smaller notes, if useful: in the J1 journey the consumer > still > > > > posts > > > > > to > > > > > > /api/catalog/v1/oauth/tokens, if partners are not supposed to > touch > > > the > > > > > > internal catalog API, should the default token endpoint live on > the > > > > > shares > > > > > > surface (or an external IdP)? And on Dmitri's point about share > > > > operators > > > > > > vs catalog operators, a SHARE_MANAGE privilege on the bound > catalog > > > > > > (independent of CATALOG_MANAGE_ACCESS) would make that work on > any > > > URL > > > > > > prefix. > > > > > > > > > > > > Happy to review the next revision, and thanks again to both of > you > > > for > > > > > > driving this. > > > > > > > > > > > > Regards, > > > > > > Prithvi S > > > > > > > > > > > > On Fri, Sep 4, 2026 at 4:38 PM Dennis Huo <[email protected]> > wrote: > > > > > > > > > > > > > As discussed live, looks like we both might have some drafts > we can > > > > > > compare > > > > > > > and choose/merge from :) > > > > > > > > > > > > > > Here's my current draft spec PR; it opts for my preferred > syntax > > > > > choices > > > > > > > for now (/api/management/ for share-management, /api/shares/ > for > > > > > consumer > > > > > > > data-plane): https://github.com/apache/polaris/pull/5446 > > > > > > > > > > > > > > Since we were talking about preferring mdfiles for design > iteration > > > > > > here's > > > > > > > a companion design-addendum which just enumerates what the > > > end-to-end > > > > > > user > > > > > > > journeys look like syntactically for each combination of design > > > > > choices: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/dennishuo/dhuo-public-provenance/blob/main/polaris-sharing/design/shares-journeys.md > > > > > > > > > > > > > > Please let me know if anyone prefers a Google Doc version of > the > > > > > > > design-addendum for easier commenting, or if the > design-addendum > > > > itself > > > > > > > should also be modeled as a PR on some free-floating branch > just to > > > > > > capture > > > > > > > comments. > > > > > > > > > > > > > > > > > > > > > On Thu, Sep 3, 2026 at 7:22 AM Jean-Baptiste Onofré < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > I am resuming work on this proposal. > > > > > > > > > > > > > > > > Regarding our recent discussion, I will move forward with > > > opening a > > > > > > > > draft PR for the /api/shares/v1 endpoint to help illustrate > the > > > > > > > > design. > > > > > > > > > > > > > > > > I will update this thread once the PR is ready. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > JB > > > > > > > > > > > > > > > > On Fri, Aug 7, 2026 at 5:04 PM Dmitri Bourlatchkov < > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > Hi Dennis, > > > > > > > > > > > > > > > > > > Thanks for the recap! > > > > > > > > > > > > > > > > > > I was thinking that share management is probably distinct > from > > > > > > catalog > > > > > > > > > management. A user who already has a realm+catalog may > have an > > > > > option > > > > > > > to > > > > > > > > > manage shares, but not the catalog itself. Would it make > sense > > > to > > > > > > > > separate > > > > > > > > > the share management API from the general Polaris > management > > > > API? I > > > > > > > mean > > > > > > > > > using a different prefix for URIs. > > > > > > > > > > > > > > > > > > For example: /api/shares/v1/ > > > > > > > > > > > > > > > > > > Obviously clients should not make any assumptions about the > > > base > > > > > path > > > > > > > > > (/api/shares), which could be different in different > > > deployments > > > > > > (e.g. > > > > > > > > > going through a gateway). Clients must be able to take the > base > > > > URI > > > > > > as > > > > > > > a > > > > > > > > > config option. Path segments after /v1/ will be specified > by > > > > > Polaris > > > > > > > > (Open > > > > > > > > > API yaml). > > > > > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > On Fri, Aug 7, 2026 at 3:52 AM Dennis Huo <[email protected] > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Just to recap some of the discussion from today's > community > > > > sync, > > > > > > > > Dmitri, > > > > > > > > > > Robert, and Alex helped raise a few considerations to > roll > > > into > > > > > the > > > > > > > > next > > > > > > > > > > design/prototyping phase: > > > > > > > > > > > > > > > > > > > > 1. Given the way sharing is packaged as a separate > thing, it > > > > may > > > > > > not > > > > > > > be > > > > > > > > > > obvious to users that historical snapshots are equally > > > > available > > > > > to > > > > > > > > > > consumers as the latest snapshot (by virtue of sharing > via > > > the > > > > > > > > standard IRC > > > > > > > > > > protocol) - we should make sure to document that fact > well, > > > > > > including > > > > > > > > the > > > > > > > > > > pitfall that deleting rows by updating a table doesn't > > > > > necessarily > > > > > > > mean > > > > > > > > > > those rows become inaccessible to consumers > > > > > > > > > > 2. Make sure not to hard-code assumptions about the > existence > > > > of > > > > > an > > > > > > > > > > internal PrincipalEntity backing an ExternalConsumer - > even > > > if > > > > > the > > > > > > > > first > > > > > > > > > > MVP implementation uses a nested/hidden Principal of some > > > type > > > > > that > > > > > > > > holds a > > > > > > > > > > Polaris-managed client_id/client_secret, the 3rd-party > IdP > > > use > > > > > > cases > > > > > > > > > > require "implicit" Principals resolved via the oauth > token. > > > In > > > > > such > > > > > > > > cases > > > > > > > > > > the ExternalConsumer contains the metadata coordinating > the > > > > > > three-way > > > > > > > > > > handshake between the data-provider, data-consumer, and > IdP, > > > > but > > > > > > > there > > > > > > > > is > > > > > > > > > > no PrincipalEntity underlying it. > > > > > > > > > > 3. Longer-term, we may want to tackle "on-behalf-of" > > > semantics > > > > - > > > > > > > > initially, > > > > > > > > > > the way an ExternalConsumer works is that the > finer-grained > > > > > > consumer > > > > > > > > users > > > > > > > > > > share a single ExternalConsumer identity, just like how a > > > > > FEDERATED > > > > > > > > catalog > > > > > > > > > > shares a single service principal in its > ConnectionConfig. > > > But > > > > in > > > > > > > some > > > > > > > > > > cases the consumer may want some end-to-end attribution > of > > > > > > > fine-grained > > > > > > > > > > users, like "on-behalf-of: bob @ consumercompany.com" > when > > > the > > > > > > > > > > ExternalConsumer implicit principal is something like > > > > > > > > > > "externalconsumer$consumercompany" > > > > > > > > > > 4. We can iterate more rapidly on a draft spec PR and > also > > > > expand > > > > > > on > > > > > > > > the > > > > > > > > > > detailed technical design > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > Dennis > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2026 at 6:12 PM Dennis Huo < > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Thanks JB for reviving this thread, and belated thanks > to > > > all > > > > > > > > reviewers > > > > > > > > > > > who have left feedback in the doc or this thread so > far! > > > And > > > > > > > > apologies > > > > > > > > > > for > > > > > > > > > > > the silence on this, it's still been on my radar but I > got > > > > > > > > distracted by > > > > > > > > > > > some other work for awhile. > > > > > > > > > > > > > > > > > > > > > > Echoing JB's summary, the proposal is indeed that we're > > > > mostly > > > > > > just > > > > > > > > > > > reusing existing Authz/RBAC machinery, and the > translation > > > > > layer > > > > > > to > > > > > > > > the > > > > > > > > > > new > > > > > > > > > > > sharing concepts sits *on top of* the existing authz > engine > > > > as > > > > > > much > > > > > > > > as > > > > > > > > > > > possible. > > > > > > > > > > > > > > > > > > > > > > So even if we're adding layers of indirection on > > > > > > > > ExternalConsumer/Listing > > > > > > > > > > > -> Principal/PrincipalRole/CatalogRole+grants, the > parts > > > that > > > > > end > > > > > > > up > > > > > > > > > > doing > > > > > > > > > > > the underlying authz just reuse existing Authorizer > logic. > > > > > > > > > > > > > > > > > > > > > > Based on some other offline discussion with Dmitri, I > > > > realized > > > > > > why > > > > > > > > the > > > > > > > > > > > discussion around "requirements" may have been > confusing: > > > > > > > > > > > > > > > > > > > > > > 1. My attempts to shorten the doc inadvertently cut out > > > some > > > > of > > > > > > the > > > > > > > > more > > > > > > > > > > > "abstract requirements" formulations > > > > > > > > > > > 2. The project in itself is somewhat more of a "form > *as* > > > > > > function" > > > > > > > > > > > feature than *strictly* functional in nature -- for > > > example, > > > > > many > > > > > > > > > > correctly > > > > > > > > > > > pointed out that "sharing" via the existing > RBAC/Principals > > > > > > syntax > > > > > > > is > > > > > > > > > > > already *possible* just by letting a cross-org > "consumer" > > > be > > > > a > > > > > > > > > > first-class > > > > > > > > > > > principal and access your own company's main Polaris > > > catalog > > > > > > > > endpoints. > > > > > > > > > > > > > > > > > > > > > > For (1) I added this to the Requirements section in > the doc > > > > to > > > > > > > better > > > > > > > > > > > clarify the abstract form of the data model element: > > > > > > > > > > > > > > > > > > > > > > *A. Sharing Container*: Container concept for holding > > > things > > > > > you > > > > > > > > want to > > > > > > > > > > > share - “ShareEntity” > > > > > > > > > > > *B. Consumer Identity*: Special flavor of > principal-like > > > > entity > > > > > > > > > > > representing a consumer relationship - > “ExternalConsumer” > > > > > > > > > > > *C. Map Containers to Consumers*: A way to specify > letting > > > a > > > > > > given > > > > > > > > > > > consumer access a given sharing container - “Listing” > > > > > > > > > > > *D. Consumer Endpoint*: A segregated API access-point > for > > > > > > > consumers - > > > > > > > > > > > “EndpointConfig” > > > > > > > > > > > *E. Objects in Containers*: A way to specify which > > > in-catalog > > > > > > > objects > > > > > > > > > > > belong in the big sharing container - “ShareMembership” > > > > > > > > > > > > > > > > > > > > > > This still bakes in a bit of implementation detail > since it > > > > > begs > > > > > > > the > > > > > > > > > > > question why not just let core RBAC/grants handle (C) > and > > > > (E). > > > > > To > > > > > > > > that > > > > > > > > > > end, > > > > > > > > > > > and addressing problem (2) of where the "form *is* > > > > function", I > > > > > > > > added a > > > > > > > > > > > "Concept Mapping" tab: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1Y0yQi5iWbmuTHPkFiIs7WjIiC3EXJTl1PzZ-wtoRnZ0/edit?tab=t.rms5bxy0jntc#heading=h.wuh830gzomt0 > > > > > > > > > > > > > > > > > > > > > > I think the details of more advanced features can > still be > > > > > hashed > > > > > > > out > > > > > > > > > > > incrementally without blocking work on an MVP (and > > > certainly > > > > > the > > > > > > > MVP > > > > > > > > > > scope > > > > > > > > > > > could be trimmed so we can launch-and-iterate), so as > long > > > as > > > > > > > there's > > > > > > > > > > > community alignment on the general direction I propose > we > > > > could > > > > > > > move > > > > > > > > > > > forward on basic building blocks. Or if building blocks > > > > > > themselves > > > > > > > > are > > > > > > > > > > > controversial we could still proceed with some > prototyping > > > > > across > > > > > > > > > > different > > > > > > > > > > > options. > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > Dennis > > > > > > > > > > > > > > > > > > > > > > On Tue, Jul 21, 2026 at 10:44 PM Jean-Baptiste Onofré < > > > > > > > > [email protected]> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > >> Hi everyone, > > > > > > > > > > >> > > > > > > > > > > >> I would like to resume the discussion about Data > Sharing > > > in > > > > > > > Polaris > > > > > > > > > > >> and propose focusing on three major items to help us > move > > > > > > forward. > > > > > > > > > > >> > > > > > > > > > > >> 1. What is data sharing? > > > > > > > > > > >> In a nutshell, it is an open protocol proposal > > > allowing > > > > > > > > > > >> organizations to share live data without copying or > > > > > replication. > > > > > > > The > > > > > > > > > > >> main goal is zero-copy sharing achieved through shared > > > > > metadata > > > > > > > and > > > > > > > > > > >> authorization. > > > > > > > > > > >> > > > > > > > > > > >> On Polaris, this would mean: > > > > > > > > > > >> a. The consumer is authenticated as usual. > > > > > > > > > > >> b. The Polaris server provides the consumer with > direct > > > > access > > > > > > to > > > > > > > > the > > > > > > > > > > >> data files (e.g., Parquet, as specified in the > > > > metadata.json). > > > > > > > > > > >> c. The consumer reads the data files directly using > their > > > > > > > preferred > > > > > > > > > > >> tool (e.g., a query engine). > > > > > > > > > > >> > > > > > > > > > > >> While Polaris already has most of the necessary > underlying > > > > > > > > > > >> capabilities, we need clear branding/packaging around > this > > > > use > > > > > > > case. > > > > > > > > > > >> Specifically, this means: > > > > > > > > > > >> a. A share is a dedicated catalog in the Polaris > server, > > > or > > > > a > > > > > > > > catalog > > > > > > > > > > >> role scoped to specific namespaces. > > > > > > > > > > >> b. A recipient is represented by a principal and a > > > principal > > > > > > role. > > > > > > > > > > >> c. A consumer profile consists of the catalog URI and > the > > > > > OAuth2 > > > > > > > > > > >> client_id/client_secret. > > > > > > > > > > >> d. We use credential vending (temporary STS > credentials) > > > for > > > > > > > > consumer > > > > > > > > > > >> access to the data files. > > > > > > > > > > >> e. We use the Polaris IRC layer to expose the > metadata and > > > > the > > > > > > > file > > > > > > > > > > >> locations/access. > > > > > > > > > > >> f. We can also support Polaris-to-Polaris sharing via > > > > > > Federation. > > > > > > > > > > >> > > > > > > > > > > >> Concretely: > > > > > > > > > > >> > > > > > > > > > > >> - On the provider side, we use the standard Polaris > RBAC > > > > > model > > > > > > > > > > >> (assigning grants to catalog roles, and assigning > those > > > > > catalog > > > > > > > > roles > > > > > > > > > > >> to principal roles): > > > > > > > > > > >> - Create a dedicated principal with its own > > > > > > > > > > client_id/client_secret. > > > > > > > > > > >> - Create a catalog role with TABLE_READ_DATA, > > > > > TABLE_LIST, > > > > > > > and > > > > > > > > > > >> NAMESPACE_LIST grants on the namespaces to be shared > (no > > > > write > > > > > > > > > > >> grants). > > > > > > > > > > >> - Create a principal role per destination, > attach > > > the > > > > > > > catalog > > > > > > > > > > >> role, and assign the principal. > > > > > > > > > > >> - The destination engine uses loadTable with the > > > > > > > > > > >> "X-Iceberg-Access-Delegation: vended-credentials" > header. > > > > > > Polaris > > > > > > > > then > > > > > > > > > > >> emits temporary, read-only storage credentials, > allowing > > > the > > > > > > > engine > > > > > > > > to > > > > > > > > > > >> read the data files directly from the owner's bucket > > > without > > > > > > > > accessing > > > > > > > > > > >> the rest of the bucket. This achieves zero-copy > sharing. > > > > > > > > > > >> - On the recipient/consumer side, no special > > > configuration > > > > > is > > > > > > > > > > >> needed; any IRC client can use it. > > > > > > > > > > >> > > > > > > > > > > >> 2. What is data sharing not? > > > > > > > > > > >> I previously considered adding an Arrow Flight > > > endpoint > > > > to > > > > > > > > > > >> Polaris, but I no longer think it is a good idea for > the > > > > > initial > > > > > > > > > > >> phase. An Arrow Flight endpoint would serve the data > > > > directly, > > > > > > > which > > > > > > > > > > >> would require Polaris to read the data and expose it. > I am > > > > > > against > > > > > > > > > > >> embedding a query engine in Polaris, as this tight > > > coupling > > > > > > feels > > > > > > > > like > > > > > > > > > > >> an anti-pattern. While we could use a delegation > service > > > to > > > > > > serve > > > > > > > > the > > > > > > > > > > >> Flight endpoint, I think it should be considered for a > > > later > > > > > > > phase, > > > > > > > > if > > > > > > > > > > >> at all, since a Flight endpoint might be better > suited as > > > a > > > > > > > consumer > > > > > > > > > > >> responsibility and could encourage data copying. > > > > > > > > > > >> > > > > > > > > > > >> The first phase should focus on packaging and > branding the > > > > > > > > > > >> capabilities we already have. > > > > > > > > > > >> > > > > > > > > > > >> 3. What could the Data Sharing "package" look like in > > > > > Polaris? > > > > > > > > > > >> To make Data Sharing obvious and easy to use, we > > > should > > > > > > > > introduce > > > > > > > > > > >> the missing concept of a "share"—a named definition > that > > > > users > > > > > > can > > > > > > > > > > >> list, discover, and define. The goal is to > materialize the > > > > > > user's > > > > > > > > > > >> intent ("I want to share these tables with this > consumer > > > > until > > > > > > > this > > > > > > > > > > >> date") into standard Polaris entities (principal, > > > principal > > > > > > role, > > > > > > > > > > >> catalog role, and grants). > > > > > > > > > > >> > > > > > > > > > > >> Therefore, I propose adding a /shares endpoint to the > > > > Polaris > > > > > > > > > > >> Management API to package and materialize these share > > > > > > definitions > > > > > > > > into > > > > > > > > > > >> Polaris entities. This would provide a usable, > > > first-version > > > > > > > package > > > > > > > > > > >> for Polaris Data Sharing. > > > > > > > > > > >> > > > > > > > > > > >> I hope this addresses Dmitri's questions and provides > a > > > more > > > > > > > > concrete > > > > > > > > > > >> scenario for Polaris Data Sharing. > > > > > > > > > > >> > > > > > > > > > > >> Thoughts? > > > > > > > > > > >> > > > > > > > > > > >> Regards, > > > > > > > > > > >> JB > > > > > > > > > > >> > > > > > > > > > > >> On Thu, May 28, 2026 at 7:46 AM Dennis Huo < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > >> > > > > > > > > > > > >> > Hi All, > > > > > > > > > > >> > > > > > > > > > > > >> > One big emerging enterprise use case coming up as > more > > > > > people > > > > > > > > > > >> consolidate > > > > > > > > > > >> > Data Lakehouses and Catalogs is something commonly > known > > > > as > > > > > > > "Data > > > > > > > > > > >> Sharing", > > > > > > > > > > >> > an more specifically over the course of adoption of > open > > > > > table > > > > > > > > formats > > > > > > > > > > >> > "Open Data Sharing". > > > > > > > > > > >> > > > > > > > > > > > >> > Examples of existing managed service providers' Data > > > > Sharing > > > > > > > > features: > > > > > > > > > > >> > > > > > > > > > > > >> > https://www.databricks.com/product/delta-sharing > > > > > > > > > > >> > > > > > https://docs.snowflake.com/en/user-guide/data-sharing-intro > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > https://docs.aws.amazon.com/redshift/latest/dg/datashare-overview.html > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > https://docs.cloud.google.com/bigquery/docs/analytics-hub-introduction > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://learn.microsoft.com/en-us/fabric/governance/external-data-sharing-overview > > > > > > > > > > >> > > > > > > > > > > > >> > The basic idea is that when you share data between > > > > different > > > > > > > > > > companies, > > > > > > > > > > >> you > > > > > > > > > > >> > need a first-class governance/management layer and > extra > > > > > > > > > > >> bells-and-whistles > > > > > > > > > > >> > that are distinct from just the basic capabilities > of > > > RBAC > > > > > or > > > > > > > > > > >> generalized > > > > > > > > > > >> > access-control (i.e. if you're sharing across > > > > > > > partially-untrusted > > > > > > > > org > > > > > > > > > > >> > boundaries, you don't just let the consumer > organization > > > > log > > > > > > > into > > > > > > > > your > > > > > > > > > > >> > datalake like one of your own employees). > > > > > > > > > > >> > > > > > > > > > > > >> > JB and I put together this high-level proposal for > > > > > supporting > > > > > > > Open > > > > > > > > > > >> Sharing > > > > > > > > > > >> > in Polaris: > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1Y0yQi5iWbmuTHPkFiIs7WjIiC3EXJTl1PzZ-wtoRnZ0/edit?usp=sharing > > > > > > > > > > >> > > > > > > > > > > > >> > Tentatively, it means adding ~5 logical data model > > > > > constructs, > > > > > > > > some of > > > > > > > > > > >> > which may be a first-class PolarisEntity type, > others > > > > > subtypes > > > > > > > of > > > > > > > > > > >> existing > > > > > > > > > > >> > entities, and others just a nested construct: > > > > > > > > > > >> > > > > > > > > > > > >> > - ShareEntity (would behave similarly to a > Catalog) > > > > > > > > > > >> > - ExternalConsumer (mostly inherits from > Principal) > > > > > > > > > > >> > - Listing (Similar to a "role grant" but has > > > different > > > > > > > > metadata) > > > > > > > > > > >> > - EndpointConfig (nested config under Listing) > > > > > > > > > > >> > - ShareMembership (Similar to a "securable > grant" but > > > > > > > different > > > > > > > > > > >> metadata) > > > > > > > > > > >> > > > > > > > > > > > >> > Feedback/comments welcome! I'll also bring it up for > > > live > > > > > > > > discussion > > > > > > > > > > if > > > > > > > > > > >> > there's time in the community sync. > > > > > > > > > > >> > > > > > > > > > > > >> > Cheers, > > > > > > > > > > >> > Dennis > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
