Thanks for your feedback, Ankit! Great point, we should understand whether the key is specifically empty or not set, because this affects whether we should generate one. I think we can introduce a specific indication for making it intentionally (explicitly) empty, either in the config or in the environment variable (for us to try to generate or fail) as a non-empty special value, such as `NOT_ENCRYPTED` or `NOT_PROVIDED`, so that accidental misconfigurations do not happen. Anyone who wants to have an unencrypted environment with a big warning for some reasons can leverage it. We can document the empty state of the Fernet key clearly for the non-encrypted case (with a huge warning) and make the default behaviour more secure. We can fall back to generating a key and failing if generation is not possible. We will put a bit of migration effort who specifically keep it empty, but we can convert our examples and code to provide easier migration. I assume lots of use cases have a generated Fernet key, so I hope the impact radius is small.
Of course, up to discussion. This default behaviour could prevent security issues if users accidentally forget to set a Fernet key in a new environment, which is especially concerning in production. At the same time, Airflow administrators can always access the configured key from the operating system or from a secret manager, or they are the ones who generate it. Because of this, it might be worth making a breaking change. &TLDR It may not be the same situation, but we already follow similar approaches in other cases, sometimes even when they are not related to security. For example, if the community agrees on a rule such as not using caplog in tests, we ask Committers/PMCs to add a specific label when an exceptional case is required. Another example is the recent change Jarek made to the provider release process to make the Release Manager’s work easier for cross-version management. We tend to use Selective Checks and warn people, but still allow them to proceed in special cases, rather than accepting everything by default. I think in this situation as well, we should define a special value for handling an empty case. Bugra Ozturk On Wed, 19 Nov 2025, 10:35 Ankit Chaurasia, <[email protected]> wrote: > Hi all, > > I support the proposed approach. It improves security by default and makes > disabling encryption an explicit option. > > I noticed a few things: In get_fernet() > (airflow-core/src/airflow/models/crypto.py > < > https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/models/crypto.py#L97 > >), > we need to distinguish: > - Not specified: Key missing from all sources (env vars, config file, > defaults) > - Explicitly empty: Key exists but is "" (e.g., fernet_key = in config or > `AIRFLOW__CORE__FERNET_KEY="") > > Currently, conf.get("core", "FERNET_KEY") may return "" in both cases if > the default template expands to empty. We should check config sources > directly, such as using conf.has_option() or inspecting the lookup > sequence, to differentiate these cases. > > For the unspecified case, we need to handle key generation, either in > get_fernet() or during initialisation of the configuration. > > Thanks for driving this forward. > > Best regards, > > Ankit Chaurasia > > > > > > > On Tue, Nov 18, 2025 at 1:18 AM Buğra Öztürk <[email protected]> > wrote: > > > Hey all, > > > > It would be great to collect more minds on this. > > Otherwise, I will move forward with a lazy consensus in the following > days. > > It is important to have some level of changes including warnings, please > > let us know your thoughts. :) > > > > Bugra Ozturk > > > > On Wed, 12 Nov 2025, 09:53 Jarek Potiuk, <[email protected]> wrote: > > > > > Yep. Also to add - a little bit on `whys`. > > > > > > The root cause was that we had some inconsistency in behaviour of code > > > duplication that Jens tried to sort out and it broke our tests. And > > > for me the consistent approach described by Buğra is something we can > > > nicely explain to users. Currently the behaviour of a few border > > > conditions is not specified in the docs: > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/security/secrets/fernet.html > > > > > > Essentially, the proposal is that you can only disable fernet > > > encryption by specifying an empty key - explicitly (but also warn). We > > > believe it's a rather common producti case that airflow.cfg is either > > > explicitly provided or the configuration folder is read-only. And > > > there we could fall-back to no-encryption even if default behaviour is > > > "generate unique key by default" - so we propose to opt for explicit > > > failure, to avoid surprises and people not being aware that their > > > sensitive data is **not** encrypted. > > > > > > The reason we raise it here is that this is a bit of a behaviour > > > change. It's a "security hardening" and not intentional behaviour so > > > we are **good** with SemVer and could do it in 3.2, however maybe > > > there are some considerations that we have not thought about so wanted > > > to get wider feedback. > > > > > > J. > > > > > > On Wed, Nov 12, 2025 at 1:37 AM Buğra Öztürk <[email protected]> > > > wrote: > > > > > > > > Hello, > > > > > > > > Recent work on cleaning and standardising Fernet Key parsing > introduced > > > > some breaking changes. For example, empty Fernet keys ('') were > raising > > > > exceptions, especially in integration/end-to-end tests. Previously, > > even > > > > though comments were specific to certain use cases, empty Fernet keys > > > were > > > > allowed and handled by NullFernet. This approach makes the system > > > > unencrypted, similar to having just an empty password while logging > in > > > to a > > > > system. We have reverted the breaking changes to enable room for > > > discussion. > > > > > > > > We should have a generic, agreed-upon approach for Fernet key > handling. > > > > Where appropriate, big warnings and documentation should be added or > > > > updated. For example, our Helm chart generates a Fernet key for > > > deployment > > > > if not provided (empty), whereas Docker Compose does not. > > > > > > > > There is also a PR related to providing a Fernet key for integration > > > tests, > > > > where tests should always have a key to run reliably. This may not > > cover > > > > all cases, but it’s the first step toward making Fernet handling > > > resilient > > > > and consistent. > > > > > > > > Below is the approach discussed with a small group of people (Jarek > > > Potiuk, > > > > Jens Scheffler and me): > > > > > > > > Proposed Fernet Key parsing approach: > > > > > > > > > > > > - > > > > > > > > Specified but empty ('')* →* use NullFernet (log a big warning > that > > > it’s > > > > unencrypted). > > > > - > > > > > > > > Not specified* →* attempt to generate and persist; fail if unable > > > (e.g., > > > > due to permissions). > > > > - > > > > > > > > Specified and non-empty* →* attempt to use it; fail if format is > > > invalid. > > > > > > > > > > > > It would be great to collect any feedback or ideas. You can find some > > > > useful links below. > > > > Which part of the code mentioned: > > > > > > > > > > https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/models/crypto.py > > > > Adding Fernet Key as default (followed by Amogh Desai with a good > idea > > of > > > > generating the fernet dynamically in the code for integration tests, > > > which > > > > will be updated): https://github.com/apache/airflow/pull/58112 > > > > > > > > > > > > Kind regards, > > > > > > > > -- > > > > Bugra Ozturk > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > >
