Also - all past images (turns that we have  > 100) have been patched. I
have not rebuilt all the images, I just added extra layers on top - with
installing the new GPG key from Oracle AND removing the mysql repository.
This should prevent us from having to redo the same in 2025.

I also  made it quite clear to MySQL/Oracle that their policy is flawed. It
caused for example that their own image `mysql:8.0.35-debian` released 25
days ago became unusable (which basically means it has an expiry date of 24
days).
They will likely have to rebuild multiple hundreds of their own images now.

This is the comment I added to their issues (I underestimated the number of
images we have. We have more than 2x more  :D)

------

Thanks for the fix. I think however the policy of Oracle/MySQL to have
expiry date for your software is deeply flawed.

We had to manually fix all ~50 images we released in the past for Apache
Airflow because of the expiry date.

Nobody else does it. Postgres, MariaDB, even MsSQL put no expiry date on
the keys that are used to sign repos.

By putting expiry key on your apt repository you basically put an expiry
date on your software and this expiry date gets shorter and shorter.

A good example of that are your own images that are affected. We had a user
asking us for help in Airflow repo
https://github.com/apache/airflow/issues/36231#issuecomment-1858419966
`
to help fix the same issue with  `mysql:8.0.35-debian` image of yours and
we sent them to your support (as well, you should deal with your own
problems).

This image was released just 25 days ago. And due to the flawed policy of
having an expiry date on your key, effectively the lifetime of this image
was 24 days. Not much. And likely you have a number of those images
(similarly as what we had 50 of ours).  Now I guess you need to
retroactively rebuild/patch your images - which is something the flawed
policy of yours made us to get 36 hours of scrambling and  and answering
support issues of our users (which we did despite our team is made of
volunteers, not paid staff as is in the case of MySQL/Oracle).

We kinda lost faith in Oracle being a good steward of MySQL apt repos and
we decided in Apache Airflow in accelerated discussion and (currently
running) lazy consensus, to switch to MariaDB clients for all our future
releases (including the 2.8.0 release that was actually delayed by at least
2 days because of this bug).

Lazy consensus thread here:
https://lists.apache.org/list.html?dev@airflow.apache.org

I hope - for the sake of your users losing days due to such issues, you
will reconsider your policies around signing your APT repos.


On Fri, Dec 15, 2023 at 4:47 PM Jarek Potiuk <ja...@potiuk.com> wrote:

> BTW. Big shoutout to Andrey who implemented the easy switch between the
> clients, a few months ago (for ARM case) in the way that we could implement
> the switch **really** quickly.  Kudos for anticipating the need :D
>
> On Fri, Dec 15, 2023 at 4:45 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
>> Hello everyone,
>>
>> This is a bit hurried, but the circumstances are unusual.
>>
>> Even if we use MySQL and not MariaDB as our "production ready" database
>> we've lost a bit of faith in Oracle being a good steward of MySQL apt repos.
>>
>> More details can be found in:
>> https://lists.apache.org/thread/dy3fgwhoc5m2f3tpqkwqcytrs6nhjlc4
>>
>> Oracle eventually replaced their keys with new ones after ~24 hours of
>> breaking everyone who tried to install MySQL + breaking workflows of
>> everyone who wanted to extend any of our images (which still need fixes as
>> we only have workarounds for now) .
>>
>> I am calling for a consensus on:
>>
>> 1) switching our Intel/X86_64 to mariadb client (same as ARM images)
>> 2) allowing the users to build their image to use mysql client
>> 3) adding the new key but also removing mysql APT repository from the
>> image to remove 2025 drama again
>> 4) applying point 3) to all released images
>>
>> Anticipating (after the short discussion) that consensus will be reached,
>> I am calling for it.
>>
>> We proceed with completing and merging
>> https://github.com/apache/airflow/pull/36243 that implements points 1)
>> 2) 3) (and later with point 4) by updating the images). Also Andrey
>> improved our code to import the keys in
>> https://github.com/apache/airflow/pull/36244. This will avoid some
>> deprecation warnings and make our key-importing code more DRY.
>>
>> We will also release 2.8.0rc4 with these two cherry-picked shortly.
>>
>> No need to +1 this email, but if you think it's wrong, please respond
>> with reasoning.
>>
>> J.
>>
>>
>>

Reply via email to