Hello Bertil, Thanks for the update! That sounds good from my side - I'm not sure I've seen other projects also do the upload to dist.apache.org from CI, Infra might have more background on whether/how to best achieve that.
Kind regards, Arnout On Wed, Mar 20, 2024 at 10:56 PM Bertil Chapuis <bchap...@gmail.com> wrote: > Hello Arnout, > > The workflow and the instructions have now been updated and they ensure > that our builds are bit-by-bit reproducible. Building the release on linux > or on a mac from the release tag with OpenJDK 17 and maven wrapper produces > the same tar.gz files as the one produced by the GitHub Actions workflow. > > The release instructions [2] describe the overall process. The idea > consists in tagging the repository with candidate tags (e.g. v0.7.3-rc1), > until one gets approved. Then, a tag for the release is create (e.g. > v0.7.3) on the hash of the latest candidate. Some information are also > included in the release instructions to reproduce the builds locally with > docker. > > The GitHub secrets required by the pipeline are listed below and in the > workflow [3]: > - GPG_KEY_ID (tested) > - GPG_PRIVATE_KEY (tested) > - GPG_PASSPHRASE (tested) > - GITHUB_TOKEN (tested) > - APACHE_USERNAME (not tested) > - APACHE_PASSWORD (not tested) > - NEXUS_USERNAME (not tested) > - NEXUS_PASSWORD (not tested) > > As the signing, hashing, and publication of release candidates works well > on GitHub[4], I’m pretty confident that the publication on the dist Apache > server will be relatively easy once the secrets are set. I’m a bit less > confident with the publication on nexus (limited experience), but our > pipeline used to work with maven central in the past. In both cases, we > will first ensure that the automation of the process works well for release > candidates by enabling the publication on dev and staging. Once the process > gets stable and well tested, we will replicate it on dist and releases. > > I hope these improvements make things clearer. Please let me know if you > need further information. I’d also be interested in knowing if smoke tests > are possible for the publication on the apache server and on nexus. > > Best regards, > > Bertil > > [1] https://github.com/apache/incubator-baremaps/pull/844 > [2] > https://github.com/apache/incubator-baremaps/blob/f34e4527d9e769ca7e21d6d6534aa07f137ab3a4/RELEASE.md > [3] > https://github.com/apache/incubator-baremaps/blob/f34e4527d9e769ca7e21d6d6534aa07f137ab3a4/.github/workflows/candidate.yml > [4] https://github.com/apache/incubator-baremaps/releases > > > > On 15 Mar 2024, at 15:02, Bertil Chapuis <bchap...@gmail.com> wrote: > > > > Hi Arnout, > > > > Thanks a lot for your answer. The old process is documented and the new > one has not yet been finalised. That being said, I haven’t checked if it is > bit-by-bit reproducible. I will try to see what it means in our case and > come back to you once we have more information. > > > > Best regards, > > > > Bertil > > > >> On 15 Mar 2024, at 14:12, Arnout Engelen <enge...@apache.org> wrote: > >> > >> Hi Bertil, baremaps PPMC, > >> > >> Thanks for checking! That sounds pretty good already. > >> > >> Part of the challenge in releasing from CI is that CI systems are > >> notoriously hard to secure, and an undetected supply-chain attack > >> could lead to publishing artifacts with injected malware. For that > >> reason I'm sure you've seen that we require that you make your build > >> bit-by-bit reproducible, and include steps in your release process to > >> make sure you reproduce the build on independent hardware before > >> promoting your release. Have you started documenting your release > >> procedure yet? Have you included reproducing the artifacts as a step? > >> > >> > >> Kind regards, > >> > >> Arnout > >> > >> > >> On Fri, Mar 15, 2024 at 11:04 AM Bertil Chapuis <bchap...@gmail.com> > wrote: > >>> > >>> Hello Apache Security Team, > >>> > >>> We are currently trying to automate the release process of Apache > Baremaps (incubating) [1]. As highlighted in the documentation, it seems > possible to get github secrets to sign artifacts [2]. Other projects are > also using a nexus username and password to publish maven snapshots and > releases [3, 4]. > >>> > >>> To do so, we drafted two release workflows on Github Actions. > >>> - The first one [5] publishes a pre release on GitHub. The source and > binary artifacts are signed and hashed. This workflow is working currently > works with a test key set as a secret in our CI. > >>> - The second one [6] tries to publish snapshot artifacts on Nexus. > Later on, the intent is also to automate the publication of release > artifacts. This workflow currently fails with a 401 Unauthorized error. > >>> > >>> The INFRA Team asked for a review of the workflow by the security team > before setting the following secrets in the CI. > >>> - NEXUS_USERNAME > >>> - NEXUS_PASSWORD > >>> - GPG_KEY_ID > >>> - GPG_PASSPHRASE > >>> - GPG_PRIVATE_KEY > >>> > >>> Thanks a lot for your help, > >>> > >>> Bertil Chapuis > >>> > >>> [1] https://github.com/apache/incubator-baremaps/issues/752 > >>> [2] > https://infra.apache.org/release-signing.html#automated-release-signing > >>> [3] > https://github.com/apache/drill/blob/26f4d30dbefcc09a7dfe05576d3f9c7b45d822a0/.github/workflows/publish-snapshot.yml#L42 > >>> [4] https://infra.apache.org/publishing-maven-artifacts.html > >>> [5] > https://github.com/apache/incubator-baremaps/blob/293da521086c402ce78be931a8e90ecb50e58e7e/.github/workflows/release.yml > >>> [6] > https://github.com/apache/incubator-baremaps/blob/293da521086c402ce78be931a8e90ecb50e58e7e/.github/workflows/publish.yml > >> > >> > >> > >> -- > >> Arnout Engelen > >> ASF Security Response > >> Apache Pekko PPMC member > >> ASF Member > >> NixOS Committer > >> Independent Open Source consultant > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@baremaps.apache.org > >> For additional commands, e-mail: dev-h...@baremaps.apache.org > >> > > > > -- Arnout Engelen ASF Security Response Apache Pekko PPMC member ASF Member NixOS Committer Independent Open Source consultant
