Thank you, Volker! As always, excellent points.
I have replied at the Okular repository MR. For those who are
interested, here's my response:
https://invent.kde.org/graphics/okular/-/merge_requests/1030#note_1082785
Cheers,
Joseph
On 21.11.24 4:50 PM, Volker Krause wrote:
On Mittwoch, 20. November 2024 15:06:18 Mitteleuropäische Normalzeit Joseph P.
De Veaugh-Geiss wrote:
Thanks for weighing in, Albert! Anyone else?
Sorry for the late reply, I first wanted to get a better understanding of what
this actually does, yesterday's presentation certainly helped with that.
There's basically two use-cases you mentioned here:
(1) Embodied carbon of a software release in KDE
My assumption is that measuring a single CI pipeline is just a tiny fraction
of this. You'd need to sum all CI pipelines run during work on a release, the
cost of local development and the cost of the dependency chain (or rather a
certain share of that, for shared components). With the latter practically
impossible to determine while at the same time presumably the dominating part
I'm not convinced you'd get to a meaningful result here. And ignoring the
dependency chain makes it trivial to game the numbers.
(2) Optimizing CI operations to reduce energy use
That's IMHO the more interesting goal. However there's probably bigger things
to look into first, such as identifying unnecessary pipeline runs (a work
branch push/MR creation currently triggers two pipelines which are identical
in most cases for example).
We'd probably also want a KDE-wide monitoring for CI state and CI cost then,
to identify the projects/pipelines with the biggest impact. And then we get to
look at optimizing individual pipelines. Any metric would likely do as a first
approximation there (CPU load, runtime), before looking at energy specifically
becomes relevant at that level.
Such optimizations would not only benefit energy use but also general CI
performance/turn around times (and thus contributor efficiency) and potentially
infrastructure cost.
For energy optimizations specifically there might be other/additional
approaches as well, as we run our own infrastructure. We could for example
look at the overall utilization over time and consider moving larger workloads
caused by automated operations (translation runs, release automation, etc) to
times or locations with a better energy mix. Assuming there's anything to gain
there at all, AFAIK we already use data centers that claim to use 100%
renewable energy anyway.
Regards,
Volker
On 19.11.24 10:16 PM, Albert Astals Cid wrote:
El divendres, 15 de novembre del 2024, a les 13:10:22 (Hora estàndard del
Centre d’Europa), Joseph P. De Veaugh-Geiss va escriure:
Anyone have opinions on this? Good idea? Bad idea?
I think it's a reasonable idea, but who is going to do the work?
I understand that with sysadmin support Arne from Green Coding Solutions
would help set up the tool according to KDE's requirements.
See here for KDE's requirements:
https://invent.kde.org/graphics/okular/-/merge_requests/1030#note_995641
And Arne's reply:
https://invent.kde.org/graphics/okular/-/merge_requests/1030#note_1031692
Perhaps we can discuss it at the meetup tonight.
Cheers,
Joseph
Cheers,
Albert
On 13.11.24 12:33 PM, Joseph P. De Veaugh-Geiss wrote:
Hi! I'd like to ask the KDE community to give feedback on a possible eco
initiative within KDE. The Open Source company Green Coding Solutions
has a new tool called Eco-CI to estimate the carbon footprint of the CI/
CD pipeline. Is there support for adopting this tool within the KDE
community (e.g., for Okular, as proposed)?
FYI if you are interested in talking with the developers directly, join
the KDE Eco online meetup next week (see below) where Arne and Didi will
be presenting.
As I see it, benefits of integrating Eco-CI into the development
pipeline include:
- Maximal transparency about the embodied carbon of a software
releases in KDE.
- Opportunities for data-driven improvement.
- Quantifiable differences for any changes made to the development
process.
- KDE will be a leader in providing data about the carbon footprint
of
our software development.
Drawbacks could include:
- KDE may be the only organization providing such data and it could
give negative impressions abut our environmental impact (to note: I
don't think this is necessarily a good argument, but I can see it being
one).
- Discouraging use of the CI/CD pipeline could negatively impact
software quality.
Other benefits? Drawbacks? Do you support the idea?
More info can be found about Green Coding Solutions and the Eco-CI tool
at this MR in the Okular repository: https://invent.kde.org/graphics/
okular/-/merge_requests/1030
"""
[Eco-CI] integrates into the GitLab CI/CD pipeline and estimates the
energy and CO2 consumption of the pipeline by utilizing a Machine
Learning model trained on real server energy data from SPECpower
The tool creates awareness of the energy cost and carbon emissions of
CI/CD pipelines and empowers developers to create action for more
sustainability.
"""
If you want to discuss with the Eco-CI developers directly, join us next
Wednesday 20 November 19h CET (see below for details).
Cheers,
Joseph
-------- Forwarded Message --------
Subject: online meetup Wed. 20 Nov. 18h UTC | Green Coding Solutions:
"Eco-CI" and "Powerletrics"
Date: Tue, 5 Nov 2024 12:40:14 +0100
From: Joseph P. De Veaugh-Geiss <[email protected]>
To: [email protected]
After a short three-month hiatus, the next KDE Eco online meetup will be
Wednesday *20 November* 18-19h UTC! I am excited to announce that Green
Coding Solutions will present two of their new measurement tools:
- "Eco-CI" to measure energy consumption of CI/CD pipeline
- "Powerletrics" Linux kernel extension for power usage estimations
Eco-CI has been suggested for integration into the Okular's CI/CD
pipeline. For discussion, see: https://invent.kde.org/graphics/okular/-/
merge_requests/1030
See below for more details.
Minutes from past meetups can be found here: https://invent.kde.org/
teams/eco/opt-green/-/tree/master/community-meetups
_Overview_
*When*: Wed. 20 November 18-19h UTC
*Where*: https://meet.kde.org/b/jos-l59-2i1-9yt
*Topic*: Green Coding Solutions tools "Eco-CI" (CI/CD pipeline) and
"Powerletrics" (Linux kernel extension)
*Pad*: Further ideas are collected at this pad, please add ideas of your
own:
https://collaborate.kde.org/s/cactBt4frrfTjbW
*Details*:
Eco-CI [0] is an open source GitHub / GitLab Plugin that estimates the
energy and carbon emissions of a CI/CD workload. It hooks into the
pipeline and will print a summary directly in the logs or as a
downloadable artifact. Arne, one of the core developers, will present
the tool and show some numbers from repos, including their average
emissions, to get a feel for the importance of the topic. A discussion
of integrating Eco-CI into Okular's GitLab repository will follow the
presentation.
Powerletrics [1] is a kernel extension that brings per process power
usage estimations to Linux. It is modelled after the MacOS tool
powermetrics which developers can use to gain insights into their
environmental impact of their code. With modern ICT infrastructure using
more and more resources, it is important that there are easy tools for
monitoring and optimisation available so that we don’t waste precious
resources.
[0] https://github.com/green-coding-solutions/eco-ci-energy-estimation
[1] https://github.com/green-kernel/powerletrics
--
Joseph P. De Veaugh-Geiss (he/him)
KDE Eco Project & Community Manager
OpenPGP: 8FC5 4178 DC44 AD55 08E7 DF57 453E 5746 59A6 C06F
Matrix: @joseph:kde.org
Generally available Monday-Thursday from 10-16h CET/CEST. Outside of
these times it may take a little longer for me to respond.
KDE Eco: Building Energy-Efficient Free Software!
Website: https://eco.kde.org
Mastodon: @[email protected]
