Hi, I would like to clarify why I do not see the concerns about the release process being intrinsically different from a regular release:
1. Release Execution: This was a semi-automatic release, not fully automatic. The process was initiated by a release manager (Pierre) who executed the script; it was not triggered by a simple button click on GitHub Actions. 2. Key Management: The script uses a key that is published in the KEYS file, available at: https://dist.apache.org/repos/dist/release/incubator/polaris/KEYS. The corresponding private key and password are securely stored as GitHub secrets, a practice also adopted by other Apache projects ( https://github.com/apache/iceberg/tree/main/dev for instance). I don't think the infra has to review all Apache release scripts (not really different from a mvn release:prepare/perform for instance). 3. Verification Process: The verification steps remain identical to previous releases. Voters can still verify the signature, checksum, and other artifacts. The documentation on the website is still valid. I do not see the key not belonging to the release manager as an issue, given that the key is publicly registered and securely managed. As a side note, Polaris is also volunteering for the Automated TLP Releases (ATR) program. Regards, JB On Wed, Dec 10, 2025 at 10:26 AM Jarek Potiuk <[email protected]> wrote: > Hello here, > > Just lurking around here and JBO asked me to take a look. So I did :). JBO > - I hope you will not regret it, but for me it is: -1 currently. Mostly > because I believe that the automation setup - even if very mature and well > designed - is not done in an ASF / INFRA compliant way, and because I > think the verification process is not well communicated (and we do not know > if it has been followed). > I might be mistaken of course - and it could be that I could not find the > necessary information - so I might change the vote once I got more > feedback. Sorry if I missed something obvious, I tried to be as diligent as > possible to find all the information - this is my first message here, so I > might have missed **something**, > > Automation of signing the release is only allowed when infra reviews your > automation and when the build reproducibility is part of the verification > done by the (P)PMC. This is clearly described in the Automated Release > Signing step of release signing policy by infra [1]. > I think while the signing process is technically sound, those conditions > are not met. > > It's the first time for me to look at that as I am a new Incubator mentor, > but I did my homework and tried to find all the answers - including reading > the automation process thread [5] and the process proposal [6] that I found > after JBO's comment. > > As a side comment - this is a fantastic job and way to set it up - with > devlist discussion, proposal, prototypes etc. You definitely rock when it > comes to maturity of your automation, process and description of what you > have done. This was a fantastic read. > > However I think some crucial steps are missing in the "approval" and > "communication" parts from [1]: > > 1) There must be a JIRA ticket in Infra where they reviewed the > signing process and agreed to it and provided you with the key (and you > never had access to the key). I actually looked at JIRA and tried to find > the Polaris ticket about it and could not - I saw quite a few "setup" > tickets - including Dockerhub secrets creation - but I could not see the > key signing ticket (maybe I missed it and you can refer to it here). From > the policy: > > The project must request a signing key through an Infra Jira ticket, and > Infra will provide a signing key for the project: > > 2) There must be a step (and it has to be followed) in the release process > where (P)PMC member giving +1 builds the artifacts locally and gets a > binary identical result as the one that is automatically signed - i.e. > reproducibility check. It does not have to be binary reproducibility - in > case package contains only sources and no compiled code, it's enough to > compare if sources are identical, some binary properties like timestamps, > permissions are not necessary (but binary reproducibility check is > generally easier if your process is binary-build-reproducible). This is > explained in the incubator release checklist [2] as "Do the contents of the > release match with what's tagged in version control?". I tried to find such > process and find out what level of reproducibility Polaris has and how you > verify it - all I found was the (very nice) description of the release > process > > https://github.com/apache/polaris/tree/apache-polaris-1.3.0-incubating-rc2/releasey > and the release scipts mentioned in the doc, but I could not find a step > describing the process of verification - so I could not find how you verify > reproducibility. The reproducibility / verification scripts are mentioned > in [7] as an idea of "Release verification script" and reproducibility is > even mentioned there. But again - this is my first time here, so I could > have missed it. > > I looked at the release email - and I cannot see the verification process > referred to. So as someone who would give +1 - I am not sure if I know the > process to follow. And I have no idea what any of the people giving +1 did. > > Also - this is actually required - each of the people casting their vote > should specify what they did - which of the checks they did (signatures, > checksums, licences, reproducibility). I see only +1s without explaining > what has been done which is considered generally a bad practice (I cannot > find a reference here - but maybe some other mentors might help heere > > Example of similar processes we do in Airflow [3] -> this is where the > release verification "steps" are described. And when we vote, we mention > the steps we have done - example here [4] (usually we do all of them as PMC > members). There you can see: > > * link to the release verification process > * each PMC member voting explicitly states what they have done > * reproducibility is specifically checked (even if we do not have yet > automation of releases - we do it to verify Release Manager's work). > > And just to explain why this is needed, so that you do not think it's some > "cargo cult". It's all but that. > > This is needed from the legal standpoint. Voting on the release is a legal > act of Foundation - which means that the three (P)PMC members voting on it, > make the Foundation potentially liable in case someone has a legal > complaint - for example harm done by using the software. That's why it's > super important that we follow the process and confirm it. Reproducibility > check in this case is a proof that this release has been prepared from the > source code that is stored in our git repository - otherwise it is possible > that release manager (if the build is prepared manually) or malicious actor > who broke into the machine/system that build the artifacts (in case of > automated process) has not injected anything. And it's not > theoretical threat - this is exactly what happened in the infamous xz case > [5] where maintainer got release manager status turned out to be a > malicious actor and injected backdoor that almost made it to major > distributions because they used the "official" packaged artifact published > by the project and it contained malicious modification of the build process > that was not in the source code. > > Since this is PPMC and not yet TLP, possibly not all those requirements > have to be met, but that will not qualify this release as "ASF cm > > [1] Infra Signing Policy > https://infra.apache.org/release-signing.html#automated-release-signing > [2] Icubator Release Checklist > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > [3] Airflow's Release verification process: > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-pmc-members > > [4] Airflow Voting mail example with +1s and stating the actions: > https://lists.apache.org/thread/crsg4bxhovyyz3gjorv5dcm85d7ytm5x > [5] XZ Utils backdoor: https://en.wikipedia.org/wiki/XZ_Utils_backdoor > [6] Polaris devlist discussion on automated releases: > https://lists.apache.org/thread/7tlmzwpdc9m4p4dx79jrydxs3x60m875 > [7] Polaris release automation process: > > https://docs.google.com/document/d/1MpovNXHFxqC1rxw_WhekQujDSj_zOAl4AN2QyFkX23k/edit?tab=t.0#heading=h.itmh6zawe02p > > J. > > > > On Wed, Dec 10, 2025, 05:04 Jean-Baptiste Onofré <[email protected]> wrote: > > > Hi > > > > Yes the key is in GitHub Secret. > > As other ASF projects, the podling uses several GitHub Secrets to > automate > > release. > > It has been described in the podling dev mailing list and also the > > semi-automated script was on a PR and reviewed. > > > > Regards > > JB > > > > Le mar. 9 déc. 2025 à 21:53, Ryan Blue <[email protected]> a écrit : > > > > > +0 > > > > > > More information is needed for me to update to +1. > > > > > > I see this signature: > > > > > > ``` > > > [blue@dev tmp]$ gpg --verify > apache-polaris-1.3.0-incubating.tar.gz.asc > > > gpg: assuming signed data in 'apache-polaris-1.3.0-incubating.tar.gz' > > > gpg: Signature made Tue 25 Nov 2025 01:08:54 AM PST > > > gpg: using RSA key > > 6A6532EAD1AE4441ACE054870E971D601C4AD16F > > > gpg: Good signature from "Apache Polaris <[email protected]>" > > > [unknown] > > > gpg: WARNING: This key is not certified with a trusted signature! > > > gpg: There is no indication that the signature belongs to the > > > owner. > > > Primary key fingerprint: 6A65 32EA D1AE 4441 ACE0 5487 0E97 1D60 1C4A > > D16F > > > ``` > > > > > > Since that is not a release manager, this must have been produced by > the > > > release automation scripts that were discussed on the dev list. I took > a > > > quick look, but I don't see how the private key is protected. The > release > > > guide covers manual releases. I'm assuming that this is stored as a > > github > > > secret and is only accessible in a workflow that authorized users have > > > access to? (I see that it must be run by a committer from the thread on > > the > > > dev list.) > > > > > > I'll change to +1 if someone can let me know how the keys are managed. > > > > > > On Tue, Dec 9, 2025 at 9:27 AM Jean-Baptiste Onofré <[email protected]> > > > wrote: > > > > > > > Dear IPMC members, > > > > > > > > This is a gentle reminder that the Apache Polaris 1.3.0-incubating > > (RC2) > > > > vote still requires a third binding vote to pass. > > > > > > > > Thank you, > > > > > > > > Regards > > > > JB > > > > > > > > On Mon, Dec 1, 2025 at 9:35 AM Pierre Laporte <[email protected] > > > > > > wrote: > > > > > > > > > Hello everyone, > > > > > > > > > > The Apache Polaris community has voted and approved the release of > > > Apache > > > > > Polaris 1.3.0-incubating (RC2). We now kindly request the IPMC > > members > > > > > review and vote for this release. > > > > > > > > > > Polaris community vote thread: > > > > > * https://lists.apache.org/thread/fw8xhobpnoy3mvvw8hxd3r7kw5of4kos > > > > > > > > > > Vote result thread: > > > > > * https://lists.apache.org/thread/xjwb04c33oo387g5gjdx674bw7t9bhz2 > > > > > > > > > > This corresponds to the tag: apache-polaris-1.3.0-incubating-rc2 > > > > > * > > > > > > > > > > > > > > > > > > > > https://github.com/apache/polaris/commits/apache-polaris-1.3.0-incubating-rc2 > > > > > * > > > > > > > > > > > > > > > > > > > > https://github.com/apache/polaris/tree/308134d6440f8167afd563a885187e238c21048a > > > > > > > > > > The release tarball, signature, and checksums are here: > > > > > * > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/polaris/1.3.0-incubating > > > > > > > > > > Helm charts are available on: > > > > > * > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/polaris/helm-chart/1.3.0-incubating/ > > > > > NB: you have to build the Docker images locally in order to test > Helm > > > > > charts. > > > > > > > > > > You can find the KEYS file here: > > > > > * https://downloads.apache.org/incubator/polaris/KEYS > > > > > > > > > > Convenience binary artifacts are staged on Nexus. The Maven > > repository > > > > URL > > > > > is: > > > > > * > > > > > > https://repository.apache.org/content/repositories/orgapachepolaris-1046 > > > > > > > > > > Please download, verify and test. > > > > > > > > > > Please vote in the next 72 hours. > > > > > > > > > > [ ] +1 approve > > > > > [ ] +0 no opinion > > > > > [ ] -1 disapprove with the reason > > > > > > > > > > To learn more about apache Polaris, please see > > > > https://polaris.apache.org/ > > > > > > > > > > Thanks, > > > > > > > > > > -- > > > > > > > > > > Pierre > > > > > > > > > > > > > > >
