Hi Arnold,
Good questions/feedback, thank you. I want to leverage your insight here
to improve the release process. I'll go into detail for posterity. I
imagine given your tenure on the project some of this will be review.
Here's specifically what I've asked folks to do while voting on a rc
(release candidate):
1. verify rc checksums and signatures
2. build Fineract from rc src artifact
3. run Fineract from the compiled rc bin artifact
This is detailed in our docs
<https://fineract.apache.org/docs/1.13.0/#_artifact_verification>. It
stems from ASF release approval policy
<https://www.apache.org/legal/release-policy.html#release-approval>:
/Before casting +1 binding votes, individuals are REQUIRED to
download all signed source code packages onto their own hardware,
verify that they meet all requirements of ASF policy on releases as
described below, validate all cryptographic signatures, compile as
provided, and test the result on their own platform./
We started formalizing all this about 7 months ago --Â for 1.10.1rc
<https://lists.apache.org/thread/cs5t0ho39ktg2jxr74dqd7fg9nyfcz3k>Â because
I had the exact same question you did: What does /test/Â mean in this
context?
First can we agree that verifying rc checksums and signatures and
building rc src artifact are all straightforward requirements?
If so, then we're back to the ambiguous: /test/. I read /test/ as: *Make
sure the binary rc artifact works*, for you, as you understand it
should. To be painfully explicit: Anticipate someone else will download
and run the binary once/if it is promoted to a release and do yourself
what they'll do, taking into account known issues. Not: Repeat what we
do in CI. We're not expecting voters do any functional testing against
the rc.
Hence, the new-ish "Tested: YES/NO/PARTIAL" section in rc vote emails.
*It's my interpretation of what the ASF says its projects must do*. And
I agree with James: "Tested" is not the right word here. "Verified"
would be better. I'll change that.
But I want to change more.
*Overall I think our release process is inefficient and robust*.
Especially when we all vote. It's one thing for a release manager to go
through the steps and it's not hard once you get the hang of it, but I
don't love the parts of the rc vote beyond everyone just sending their
+1/-1. *I'm confident we can make it more efficient and just as robust*,
still meeting ASF policy with less of, e.g., the manual rc verification
we now do and just a more sensible release process overall. Maybe with
some simple automation, but maybe not just that.
I want to have a meeting to gather ideas to streamline releases, because
y'all know things I don't and policy affects us all.
I'm not looking forward to onboarding the next release manager. I don't
want them to have to just repeat what I've done. I'd rather hand them
something wonderfully improved and say "hey look how easy we made your
job! just click RELEASE and you're good" or something like that. Heck,
maybe we won't need a release manager at all if we judo this creatively.
Plenty of projects ship robust releases far more often than we do.
Thoughts? Wanna meet up?
-Adam
On 10/20/25 12:16, Arnold Galovics wrote:
+1 from me too, but when people say "tested", do we know what kind of
testing is supposed to happen? I feel like we just have this feeling
of the package being tested, but in reality it cannot be considered
actual functional verification (if that's what's supposed to happen).
In my mind, functional testing is part of the pipeline and every
commit making into the develop branch has to pass all the tests, it's
a quality gate. Therefore by logic, whatever commit we pick for
releasing is passing the quality gates hence no need for manual
testing. What am I missing?
( original "[FINERACT] [VOTE] 🗳 1.13.0 for release" thread
<https://lists.apache.org/thread/bm9cynvrvvb1mvp7pc05v1zy5zo4h0hs> )
( original "release process improvement" thread
<https://lists.apache.org/thread/f99mhvxg9l3wz17thmwonvgn6vyz05vq>Â )
