Re: [DISCUSS] - When to remove EclipseLink?

2025-09-18 Thread Prashant Singh
+1 to remove Eclipselink on 1.3 or later ! Best, Prashant Singh On Thu, Sep 18, 2025 at 7:08 AM Dmitri Bourlatchkov wrote: > Doc updates: https://github.com/apache/polaris/pull/2605 > > On Thu, Sep 18, 2025 at 9:55 AM Dmitri Bourlatchkov > wrote: > > > EL is officially d

Re: DISCUSS: 1.2 release

2025-09-16 Thread Prashant Singh
e I also added an iceberg version bump to 1.10 in this track, as our tests were failing with dependabot update, it will be nice to include this as well ! https://github.com/apache/polaris/pull/2586 Best, Prashant Singh On Tue, Sep 16, 2025 at 9:53 AM Alexandre Dutra wrote: > Hi Prashant,

Re: DISCUSS: 1.2 release

2025-09-15 Thread Prashant Singh
onable to me ! Best, Prashant Singh On Mon, Sep 15, 2025 at 2:42 PM Dmitri Bourlatchkov wrote: > Thanks for volunteering to drive the 1.2 release, Prashant! > > I believe we previously discussed using the "release train" model, so > whatever is committed by the branch

DISCUSS: 1.2 release

2025-09-15 Thread Prashant Singh
volunteer as 1.2 release manager, please let me know your thoughts ! Best, Prashant Singh

Re: [PROPOSAL] Add Data Lake operational metrics to Polaris

2025-09-05 Thread Prashant Singh
they get that report, to which i suggested them to write their own custom metric reporter which rather than hitting the /report endpoint of Polaris it just dumps data to a DB which their downstream maintainer services can use. Looking forward to discussing this more ! Best, Prashant Singh On Fri

Re: [PROPOSAL] Add Data Lake operational metrics to Polaris

2025-09-04 Thread Prashant Singh
he/iceberg/blob/main/core/src/main/java/org/apache/iceberg/metrics/CommitMetrics.java> are already available but at this point we don't persist them and hence they are lost, there has been a request for this in Polaris slack too ! My recommendations would start from here ! Best, Prashant Sin

Re: [RELEASE] 1.1.0-incubating release preparation

2025-08-22 Thread Prashant Singh
cid=C084QSKD6S2 > [2341] https://github.com/apache/polaris/pull/2341 > > Thanks, > Dmitri. > > On Thu, Aug 21, 2025 at 8:13 PM Prashant Singh > wrote: > > > Hey All, > > I would also request to consider getting credentials vending PR in, > mostly > >

Re: [DISCUSS] Feature flag for credential refresh config in REST responses

2025-08-21 Thread Prashant Singh
Thank you so much for the context and pointers Dmitri ! Then from my POV I don't think we would require this flag. Will wait to hear other folks' thoughts on this, then ! Best, Prashant Singh On Thu, Aug 21, 2025 at 5:19 PM Dmitri Bourlatchkov wrote: > Hi Prashant, > >

Re: [RELEASE] 1.1.0-incubating release preparation

2025-08-21 Thread Prashant Singh
Hey All, I would also request to consider getting credentials vending PR in, mostly because a lot of users have raised this issue, for long running jobs and are eagerly waiting for it ! Best, Prashant Singh On Thu, Aug 21, 2025 at 4:53 PM Dmitri Bourlatchkov wrote: > Hi All, > > Just

Re: [DISCUSS] Feature flag for credential refresh config in REST responses

2025-08-21 Thread Prashant Singh
same time not require it to come back to the server to get new creds. Best, Prashant Singh On Thu, Aug 21, 2025 at 5:00 PM Dmitri Bourlatchkov wrote: > Hi All, > > PR [2341] enables sending endpoints for credential refresh to Iceberg REST > Catalog clients. > > Currently, t

Re: Re: [DISCUSS] S3 Credential vending without STS

2025-08-14 Thread Prashant Singh
e pre-signed url (https://amzn-s3-demo-bucket.s3.amazonaws.com/ object.txt?AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&Signature= vjbyNxybdZaMmLa%2ByT372YEAiv4%3D&Expires=1741978496) Looking forward to the design doc / proposal for Polaris. Best, Prashant Singh On Tue, Aug 5, 2025 at 6:23 AM Robert

Re: [MEETUPS] Proposed Seattle Meetup

2025-08-13 Thread Prashant Singh
Thank you for organizing this Danica ! +1 for the meetup :) and having some guidelines around, iceberg community is a good example. Best, Prashant Singh On Wed, Aug 13, 2025 at 8:13 AM Alex Merced wrote: > I think this meetup would be a great idea, I'll ask around if we have > a

Re: [PROPOSAL] Release Apache Polaris 1.0.1-incubating

2025-07-22 Thread Prashant Singh
+1 to 1.0.1 with just helm fix ! On Tue, Jul 22, 2025 at 7:07 AM Dmitri Bourlatchkov wrote: > A 1.0.1 (patch) release from the 1.0.x branch with just helm fixes SGTM. > > Cheers, > Dmitri. > > > On Tue, Jul 22, 2025 at 9:24 AM Jean-Baptiste Onofré > wrote: > > > That's a good call. > > > > My i

Re: [ANNOUNCE] Apache Polaris 1.0.0-incubating

2025-07-10 Thread Prashant Singh
Awesome news ! Thank you Yufei for driving the release :) ! Thanks a ton to the Polaris community for your participation and contribution; it wouldn't have been possible without you ! Best, Prashant Singh On Thu, Jul 10, 2025 at 8:59 AM Dmitri Bourlatchkov wrote: > Great news! Th

Re: [VOTE] Release Apache Polaris 1.0.0-incubating (RC6)

2025-07-02 Thread Prashant Singh
+1 (non-binding) * Checksums * Signatures * tested Postgres Thank you Yufei, Best, Prashant Singh On Wed, Jul 2, 2025 at 3:32 PM Russell Spitzer wrote: > +1 (binding) > > -Verified Checksums and Sigs > -Verified all build and test again > -Rechecked the license fixes we did i

Re: [VOTE] Release Apache Polaris 1.0.0-incubating (RC5)

2025-07-01 Thread Prashant Singh
+1 (non binding) I verified the following: - Source tarball (shasum and GPG signature) - Build and test - Verified server binary distribution - basic sanity testing with JDBC persistence layer Thank you Yufei ! Bests, Prashant On Tue, Jul 1, 2025 at 9:47 PM Jean-Baptiste Onofré wrote: > +1 (b

Proposal: Secure Views for FGAC Dynamic Enforcement (based on prior Access Decision Exchange work)

2025-06-25 Thread Prashant Singh
rg views and expects further enhancements via Iceberg Expressions expansion and Iceberg UDF's.* Best, Prashant Singh & Russell Spitzer

Re: Proposal: Secure Views for FGAC Dynamic Enforcement (based on prior Access Decision Exchange work)

2025-06-25 Thread Prashant Singh
Apologies for the broken link to the proposal : here you go [OSS] Secure Views for dynamic policy enforcement. <https://docs.google.com/document/d/1AJicez7xPhzwKXenGZ19h0hngxrwAg3rSajDV1v0x-s/edit?usp=sharing> Best, Prashant Singh On Wed, Jun 25, 2025 at 11:06 AM Prashant Singh wrote:

Re: [DISCUSS] Prepare for 1.0 Release

2025-06-20 Thread Prashant Singh
led, I cherry-picked the feature to 1.0, it's a clean cherry-pick ! I believe it will strongly help in Apache Polaris adoption, and sincerely thank everyone who participated ! Best, Prashant Singh On Fri, Jun 20, 2025 at 9:13 AM Dmitri Bourlatchkov wrote: > Hi All, > > Posting here for

Re: Context-Aware Functions for Apache Polaris

2025-06-17 Thread Prashant Singh
c they can very well be called context variables instead. Re: a few common, constant-foldable expressions in FGAC policies, and go from there. Precisely, let me share the doc (by next week) which we have been interating on internally for a while now, I hope all this makes sense once shared. Best, Prashan

Re: Adding Support for Apache Hudi within Polaris

2025-06-14 Thread Prashant Singh
this change ! Best, Prashant Singh On Fri, Jun 13, 2025 at 10:48 PM Jean-Baptiste Onofré wrote: > Hi Rahil > > It's interesting, thanks for that. As a first step, the approach > corresponds to the one used for Delta. > > Long term, I would love to see support on other tab

Re: [DISCUSS] Skip 0.10.0-beta-incubating release to directly release 1.0.0-incubating ?

2025-06-06 Thread Prashant Singh
+1 on skipping 0.10 since we are very close to closing all 1.0 blockers. Best, Prashant Singh On Fri, Jun 6, 2025 at 3:57 PM Yufei Gu wrote: > +1 on skipping 0.10 and releasing 1.0 directly given the current status. We > don't have to spend more time on 0.10.0-beta, and 0.10.0

Re: Context-Aware Functions for Apache Polaris

2025-06-06 Thread Prashant Singh
hat I am working on a proposal doc for incorporating these thoughts for Polaris, I will share with the community *soon* and would love to get all of your feedback ! Best, Prashant Singh On Fri, Jun 6, 2025 at 2:48 PM Eric Maynard wrote: > It seems to me that the *easiest* to start with wo

Re: [DISCUSS] Disable renovatebot on release branches

2025-06-02 Thread Prashant Singh
+1, Agree with running this only on main On Mon, Jun 2, 2025 at 11:22 AM Eric Maynard wrote: > +1, great idea. This process change will reduce noise in the repo as we add > more releases, and it also aligns more closely with the idea of a release > as a stable snapshot of the code at some point

Re: [DISCUSS] PR #1724 – Directory & package refactor for persistence modules

2025-06-02 Thread Prashant Singh
The refactor makes sense to me as well, Thanks Yufei ! persistence/ ├── eclipselink/ ├── relational-jdbc/ ├── mongodb/ └── .../ Best, Prashant Singh On Mon, Jun 2, 2025 at 1:23 AM Pierre Laporte wrote: > Could you add a README file at the root of the `persistence/` directory so > that

Re: Context-Aware Functions for Apache Polaris

2025-05-20 Thread Prashant Singh
> Let's work together on a proper design. For sure ! will be reaching out soon for this ! Best, Prashant Singh

Re: Context-Aware Functions for Apache Polaris

2025-05-20 Thread Prashant Singh
world to unify spark's current_user() to is_prinicpal() in polaris. I hope this answers your concern. I am totally open to any recommendation and work rough edges, let's solve this problem, together as a community ! Best, Prashant Singh On Tue, May 20, 2025 at 9:06 AM Robert Stupp wrote:

Re: Context-Aware Functions for Apache Polaris

2025-05-20 Thread Prashant Singh
is ANALYST i will never get back view but my IS_MEMBER() is true so i think i should be able to read the view this helps expressing this more verbose and get away from grants This is infact a feature in UNity [1] https://docs.databricks.com/aws/en/views/dynamic Happy to discuss more ! Best, P

Re: Context-Aware Functions for Apache Polaris

2025-05-19 Thread Prashant Singh
Thank you so much for driving this ! Please Let me know your thoughts. Best, Prashant Singh On Mon, May 19, 2025 at 10:45 PM Ajantha Bhat wrote: > Hi Prashant, > > It seems to me that what you are trying to do is to store scalar UDF in the > polaris catalog. > Iceberg is t

Re: Context-Aware Functions for Apache Polaris

2025-05-19 Thread Prashant Singh
an engine only when things like identity is involved, we need to do this at catalog level to have uniform enforcement. Please let me know your further thoughts. Best, Prashant Singh On Mon, May 19, 2025 at 12:21 PM Jean-Baptiste Onofré wrote: > Hi Prashant > > Thanks for the propos

Re: Context-Aware Functions for Apache Polaris

2025-05-19 Thread Prashant Singh
rather than just ACL (which are conjunctions) for ex if we just complicate the view def with more predicated for ex disjunction select * from ns1.layer1_table where (condition1) OR (is_principal_role('ANALYST')) I would love to get your further feedback, considering the above. Best, Prashant

Re: Context-Aware Functions for Apache Polaris

2025-05-19 Thread Prashant Singh
n anything in Polaris all it matter when the view was asked what was the caller's identity and hence get the appropriate view definition. A trusted engine should just focus on executing the view and not care about identity resolution. Best, Prashant Singh On Mon, May 19, 2025 at 10:40 AM

Context-Aware Functions for Apache Polaris

2025-05-19 Thread Prashant Singh
sql-reference/functions-context> Databricks Unity Catalog offers a similar mechanism called *dynamic views*: https://docs.databricks.com/aws/en/views/dynamic *Next steps* If the community is interested, we can discuss API surface, engine implications, and a roadmap for merging. Eager to hear your feedback! Best, Prashant Singh

Re: GitHub pull-requests

2025-05-19 Thread Prashant Singh
+1 to the suggestion ! On Mon, May 19, 2025 at 7:39 AM Dmitri Bourlatchkov wrote: > Great suggestions, Robert! Thanks for writing them down. > > Cheers, > Dmitri. > > On Mon, May 19, 2025 at 8:34 AM Robert Stupp wrote: > > > Hi all, > > > > Looking a bit ahead with respect to releases and (semi

Re: [DISCUSS] Deprecate Eclipse Link and make Relational JDBC as default

2025-05-15 Thread Prashant Singh
and forgot to reply here :) > > > > > > > > Yes agree to deprecate eclipselink it makes sense to me and promote > > > > "JDBC" instead. > > > > > > > > That said, as said in the PR, I think we should keep the eclipselink > >

Re: [ANNOUNCE] Relational JDBC the new default for persistence

2025-05-14 Thread Prashant Singh
295 | 295 | 0 | 47 | 11 | 50 | 88 Unfortunately our mail client does not allow adding zip attachments, please let me know if you need detailed gatling reports, happy to send it over the slack ! Best, Prashant Singh On

Re: Discussion: Re-evaluating Realm Modeling in Polaris

2025-05-13 Thread Prashant Singh
routed to 1 realm still and then we can diff instance running diff realms ? Best, Prashant Singh On Tue, Apr 22, 2025 at 9:29 AM Yufei Gu wrote: > To clarify, Polaris *does* support multi-tenancy. What’s currently limited > with Option 1 is *account-level* multi-tenancy specifically

[ANNOUNCE] Relational JDBC the new default for persistence

2025-05-13 Thread Prashant Singh
olaris.apache.org/in-dev/unreleased/metastores/>. Cheers, Prashant Singh

Re: [VOTE] Release Apache Polaris 0.10.0-beta-incubating (rc2)

2025-05-06 Thread Prashant Singh
+1 1. Verified checksums, signature, artifacts 2. did manual testing with admin and server, works well ! Best, Prashant Singh On Tue, May 6, 2025 at 7:52 AM Jean-Baptiste Onofré wrote: > +1 (binding) > > I checked: > - checksum and signature are good on the source distributio

[DISCUSS] Deprecate Eclipse Link and make Relational JDBC as default

2025-05-02 Thread Prashant Singh
Hi all, I’d like to get your thoughts on deprecating EclipseLink and making JDBC the default for our persistence layer. Our current EclipseLink setup mandates execution within a transaction, which has introduced several issues — notably, an improper implementation of CAS (compare-and-swap) semant

Re: Discussion: Re-evaluating Realm Modeling in Polaris

2025-04-21 Thread Prashant Singh
ators to choose between options > 1, > > > 2, 3 > > > > > > > according to their specific deployment concerns. > > > > > > > > > > > > > > This means that the primary key has to include the realm ID, > > > because &g

Re: [DISCUSS] Rollback REPLACE commit on conflicts

2025-04-18 Thread Prashant Singh
a work ! Best, Prashant Singh On Fri, Apr 18, 2025 at 8:56 AM Dmitri Bourlatchkov wrote: > Hi Prashant, > > Sorry for the delayed reply and apologies if I missed some relevant > discussion. > > As I understand the catalog could remove snapshots that come in-between > previous and

Discussion: Re-evaluating Realm Modeling in Polaris

2025-04-14 Thread Prashant Singh
options. Best regards, Prashant Singh

Re: [Discuss] Change EntityTable properties and internal properties from TEXT to JSONB

2025-04-01 Thread Prashant Singh
et me know what your thoughts are ? Best, Prashant Singh On Sat, Mar 29, 2025 at 1:16 AM Eric Maynard wrote: > I'm generally a +1 on using JSONB within Postgres. > > However I am in agreement with Dennis that we should avoid his item (2) > above. If the application will need to

[DISCUSS] Rollback REPLACE commit on conflicts

2025-03-31 Thread Prashant Singh
https://docs.google.com/document/d/1pSqxf5A59J062j9VFF5rcCpbW9vdTbBKTmjps80D-B0/edit?tab=t.0#heading=h.fn6jmpw6phpn [2] https://github.com/apache/iceberg/pull/5888 [3] https://github.com/apache/polaris/pull/1285 Best, Prashant Singh

Re: [ANNOUNCE] Welcome Dmitri Bourlatchkov, Dennis Huo and Yufei Gu to the Apache Polaris PPMC

2025-03-26 Thread Prashant Singh
Congratulations folks, well deserved ! Best, Prashant Singh On Wed, Mar 26, 2025 at 5:02 PM yun zou wrote: > Congratulations! > > Best Regards, > Yun > > On Wed, Mar 26, 2025 at 4:28 PM Michael Collado > wrote: > > > Wow! Awesome news. Congrats folks! > > &

Re: [VOTE] REST API changes for Catalog Federation - addition of ConnnectionConfigInto to ExternalCatalog

2025-03-25 Thread Prashant Singh
+1 (non-binding) Best, Prashant On Tue, Mar 25, 2025 at 7:01 AM Dmitri Bourlatchkov wrote: > +1 LGTM. > > Cheers, > Dmitri. > > On Tue, Mar 25, 2025 at 12:30 AM Dennis Huo wrote: > > > Hello all, > > > > We've had some productive discussion in various places on the mailing > list, > > in the g

Re: [Discuss] Change EntityTable properties and internal properties from TEXT to JSONB

2025-03-21 Thread Prashant Singh
patterns. Please share your thoughts on this. Best regards, Prashant Singh [1] https://github.com/apache/polaris/blob/main/polaris-core/src/main/java/org/apache/polaris/core/persistence/AtomicOperationMetaStoreManager.java [2] https://github.com/apache/polaris/blob/main/polaris-core/src/main/

[Discuss] Change EntityTable properties and internal properties from TEXT to JSONB

2025-03-20 Thread Prashant Singh
Hey folks, I am presently working on implementing non transactional postgres implementation using jdbc. in the course of that i noticed in the entity table that the column type of the properties / internal properties column is TEXT and not JSONB, I dug a bit into the history of this, seems like we

Re: Polaris benchmarks proposal

2025-03-19 Thread Prashant Singh
Thank you so much for the benchmarks ! +1, having benchmark results committed, it will help catch any degradation / correctness issue that can creep in ! equivalent to golden files of tpc-ds / tpc-h in spark repo. Best, Prashant Sungh On Wed, Mar 19, 2025 at 8:53 AM Russell Spitzer wrote: > I t