Hi all,
FYI there is the ASF infra roundtable soon. One of the subjects for this
session is GitHub Actions. It could be worth passing by:
December 6th, 2023 at 1700 UTC on the #Roundtablechannel on Slack.
For information about theroundtables, and about how to join,
see:https://infra.apache.org/roundtable.html
<https://infra.apache.org/roundtable.html>
Best
Etienne
Le 24/11/2023 à 14:16, Maximilian Michels a écrit :
Thanks for reviving the efforts here Matthias! +1 for the transition
to GitHub Actions.
As for ASF Infra Jenkins, it works fine. Jenkins is extremely
feature-rich. Not sure about the spare capacity though. I know that
for Apache Beam, Google donated a bunch of servers to get additional
build capacity.
-Max
On Thu, Nov 23, 2023 at 10:30 AM Matthias Pohl
<matthias.p...@aiven.io.invalid> wrote:
Btw. even though we've been focusing on GitHub Actions with this FLIP, I'm
curious whether somebody has experience with Apache Infra's Jenkins
deployment. The discussion I found about Jenkins [1] is quite out-dated
(2014). I haven't worked with it myself but could imagine that there are
some features provided through plugins which are missing in GitHub Actions.
[1]https://lists.apache.org/thread/vs81xdhn3q777r7x9k7wd4dyl9kvoqn4
On Tue, Nov 21, 2023 at 4:19 PM Matthias Pohl<matthias.p...@aiven.io>
wrote:
That's a valid point. I updated the FLIP accordingly:
Currently, the secrets (e.g. for S3 access tokens) are maintained by
certain PMC members with access to the corresponding configuration in the
Azure CI project. This responsibility will be moved to Apache Infra. They
are in charge of handling secrets in the Apache organization. As a
consequence, updating secrets is becoming a bit more complicated. This can
be still considered an improvement from a legal standpoint because the
responsibility is transferred from an individual company (i.e. Ververica
who's the maintainer of the Azure CI project) to the Apache Foundation.
On Tue, Nov 21, 2023 at 3:37 PM Martijn Visser<martijnvis...@apache.org>
wrote:
Hi Matthias,
Thanks for the write-up and for the efforts on this. I really hope
that we can move away from Azure towards GHA for a better integration
as well (directly seeing if a PR can be merged due to CI passing for
example).
The one thing I'm missing in the FLIP is how we would setup the
secrets for the nightly runs (for the S3 tests, potential tests with
external services etc). My guess is we need to provide the secret to
ASF Infra and then we would be able to refer to them in a pipeline?
Best regards,
Martijn
On Tue, Nov 21, 2023 at 3:05 PM Matthias Pohl
<matthias.p...@aiven.io.invalid> wrote:
I realized that I mixed up FLIP IDs. FLIP-395 is already reserved [1]. I
switched to FLIP-396 [2] for the sake of consistency. 8)
[1]https://lists.apache.org/thread/wjd3nbvg6nt93lb0sd52f0lzls6559tv
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-396%3A+Migration+to+GitHub+Actions
On Tue, Nov 21, 2023 at 2:58 PM Matthias Pohl<matthias.p...@aiven.io>
wrote:
Hi everyone,
The Flink community discussed migrating from Azure CI to GitHub
Actions
quite some time ago [1]. The efforts around that stalled due to
limitations
around self-hosted runner support from Apache Infra’s side. There
were some
recent developments on that topic. Apache Infra is experimenting with
ephemeral runners now which might enable us to move ahead with GitHub
Actions.
The goal is to join the trial phase for ephemeral runners and
experiment
with our CI workflows in terms of stability and performance. At the
end we
can decide whether we want to abandon Azure CI and move to GitHub
Actions
or stick to the former one.
Nico Weidner and Chesnay laid the groundwork on this topic in the
past. I
picked up the work they did and continued experimenting with it in my
own
fork XComp/flink [2] the past few weeks. The workflows are in a state
where
I think that we start moving the relevant code into Flink’s
repository.
Example runs for the basic workflow [3] and the extended (nightly)
workflow
[4] are provided.
This will bring a few more changes to the Flink contributors. That is
why
I wanted to bring this discussion to the mailing list first. I did a
write
up on (hopefully) all related topics in FLIP-395 [5].
I’m looking forward to your feedback.
Matthias
[1]https://lists.apache.org/thread/vcyx2nx0mhklqwm827vgykv8pc54gg3k
[2]https://github.com/XComp/flink/actions
[3]https://github.com/XComp/flink/actions/runs/6926309782
[4]https://github.com/XComp/flink/actions/runs/6927443941
[5]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-395%3A+Migration+to+GitHub+Actions
--
[image: Aiven]<https://www.aiven.io>
*Matthias Pohl*
Opensource Software Engineer, *Aiven*
matthias.p...@aiven.io <i...@aiven.io> | +49 170 9869525
aiven.io<https://www.aiven.io> |
<https://www.facebook.com/aivencloud>
<https://www.linkedin.com/company/aiven/> <
https://twitter.com/aiven_io>
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
Amtsgericht Charlottenburg, HRB 209739 B
