Hey fellow Airflowers,

I'm cancelling this vote because of the above issue and will create an rc2
soon.

Thanks Jens and Jarek for testing & voting on this release candidate

- Ephraim

On Sun, 7 Dec 2025 at 19:13, Jarek Potiuk <[email protected]> wrote:

> I checked the same scenario and got the same error.
>
> I also think we should handle the different token better from 3.1.3,
> because it is basically guaranteed to happen for everyone who migrates.
> The "Internal Error" displayed will flood support people who manage
> Airflow..
>
> *Airflow/Airflow Core: -1 (binding).*
>   - SVN
>   - Reproducible package build
>   - Licenses
>   - Signatures
>   - Checksums
>   - Install in Docker
>   - Source tarball
> *Task SDK: +1 (binding), checked:*
>   - SVN
>   - Reproducible package build
>   - Licenses
>   - Signatures
>   - Checksums
>   - Install in Docker
>
> Also I see Jens fixed some small issues with breeze release-management
> check-release command -> I also added fix on top of that:
> https://github.com/apache/airflow/pull/59164
>
> J,.
>
>
>
> On Sun, Dec 7, 2025 at 5:04 PM Jens Scheffler <[email protected]> wrote:
>
> > Hi Ephraim,
> >
> > thanks for preparing the release! Unfortunately in a simple test I got a
> > negative result, see details below:
> >
> > Airflow/Airflow Core: -1 (binding), technical checks as PMC:
> >   - SVN
> >   - Reproducible package build
> >   - Licenses
> >   - Signatures
> >   - Checksums
> >   - Install in Docker
> >   - Source tarball
> > Task SDK: +1 (binding), checked:
> >   - SVN
> >   - Reproducible package build
> >   - Licenses
> >   - Signatures
> >   - Checksums
> >   - Install in Docker
> >
> > So technically all looking good, started via breeze with command
> >
> > breeze start-airflow --python 3.12 --load-example-dags --backend
> > postgres --executor EdgeExecutor --answer y --use-airflow-version 3.1.3
> > --use-distributions-from-dist
> >
> > And opening browser (as of breeze requiring a new session) expected a
> > login page but received an "INTERNAL SERVER ERROR". Some fix in JWT
> > token handling seems to generate an exception instead of forwarding to
> > login page. In view this is not acceptable as we must expect invalid
> > sessions from users and need to redirect to login page. Exception seen
> > in breeze tmux panel for API server:
> >
> > 2025-12-07T15:56:23.562782Z [error    ] JWT token is not valid:
> > Signature verification failed
> > [airflow.api_fastapi.auth.managers.base_auth_manager]
> > loc=base_auth_manager.py:107
> > INFO:     172.18.0.1:34366 - "GET / HTTP/1.1" 500 Internal Server Error
> > ERROR:    Exception in ASGI application
> >    + Exception Group Traceback (most recent call last):
> >    |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/_utils.py", line 79,
> > in collapse_excgroups
> >    |     yield
> >    |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/base.py",
> > line 183, in __call__
> >    |     async with anyio.create_task_group() as task_group:
> >    |                ^^^^^^^^^^^^^^^^^^^^^^^^^
> >    |   File
> > "/usr/python/lib/python3.12/site-packages/anyio/_backends/_asyncio.py",
> > line 783, in __aexit__
> >    |     raise BaseExceptionGroup(
> >    | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
> >    +-+---------------- 1 ----------------
> >      | Traceback (most recent call last):
> >      |   File
> >
> "/usr/python/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py",
> >
> > line 409, in run_asgi
> >      |     result = await app(  # type: ignore[func-returns-value]
> >      |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >      |   File
> > "/usr/python/lib/python3.12/site-packages/fastapi/applications.py", line
> > 1082, in __call__
> >      |     await super().__call__(scope, receive, send)
> >      |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/applications.py",
> > line 113, in __call__
> >      |     await self.middleware_stack(scope, receive, send)
> >      |   File
> >
> "/usr/python/lib/python3.12/site-packages/starlette/middleware/errors.py",
> > line 186, in __call__
> >      |     raise exc
> >      |   File
> >
> "/usr/python/lib/python3.12/site-packages/starlette/middleware/errors.py",
> > line 164, in __call__
> >      |     await self.app(scope, receive, _send)
> >      |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/gzip.py",
> > line 29, in __call__
> >      |     await responder(scope, receive, send)
> >      |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/gzip.py",
> > line 130, in __call__
> >      |     await super().__call__(scope, receive, send)
> >      |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/gzip.py",
> > line 46, in __call__
> >      |     await self.app(scope, receive, self.send_with_compression)
> >      |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/cors.py",
> > line 85, in __call__
> >      |     await self.app(scope, receive, send)
> >      |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/base.py",
> > line 182, in __call__
> >      |     with recv_stream, send_stream, collapse_excgroups():
> >      |                                    ^^^^^^^^^^^^^^^^^^^^
> >      |   File "/usr/python/lib/python3.12/contextlib.py", line 158, in
> > __exit__
> >      |     self.gen.throw(value)
> >      |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/_utils.py", line 85,
> > in collapse_excgroups
> >      |     raise exc
> >      |   File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/base.py",
> > line 184, in __call__
> >      |     response = await self.dispatch_func(request, call_next)
> >      |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >      |   File
> >
> "/usr/python/lib/python3.12/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py",
> >
> > line 45, in dispatch
> >      |     new_user = await self._refresh_user(current_token)
> >      |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >      |   File
> >
> "/usr/python/lib/python3.12/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py",
> >
> > line 67, in _refresh_user
> >      |     user = await resolve_user_from_token(current_token)
> >      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >      |   File
> >
> "/usr/python/lib/python3.12/site-packages/airflow/api_fastapi/core_api/security.py",
> >
> > line 103, in resolve_user_from_token
> >      |     raise HTTPException(status_code=status.HTTP_403_FORBIDDEN,
> > detail="Invalid JWT token")
> >      | fastapi.exceptions.HTTPException: 403: Invalid JWT token
> >      +------------------------------------
> >
> > During handling of the above exception, another exception occurred:
> >
> > Traceback (most recent call last):
> >    File
> >
> "/usr/python/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py",
> >
> > line 409, in run_asgi
> >      result = await app(  # type: ignore[func-returns-value]
> >               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >    File
> > "/usr/python/lib/python3.12/site-packages/fastapi/applications.py", line
> > 1082, in __call__
> >      await super().__call__(scope, receive, send)
> >    File
> > "/usr/python/lib/python3.12/site-packages/starlette/applications.py",
> > line 113, in __call__
> >      await self.middleware_stack(scope, receive, send)
> >    File
> >
> "/usr/python/lib/python3.12/site-packages/starlette/middleware/errors.py",
> > line 186, in __call__
> >      raise exc
> >    File
> >
> "/usr/python/lib/python3.12/site-packages/starlette/middleware/errors.py",
> > line 164, in __call__
> >      await self.app(scope, receive, _send)
> >    File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/gzip.py",
> > line 29, in __call__
> >      await responder(scope, receive, send)
> >    File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/gzip.py",
> > line 130, in __call__
> >      await super().__call__(scope, receive, send)
> >    File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/gzip.py",
> > line 46, in __call__
> >      await self.app(scope, receive, self.send_with_compression)
> >    File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/cors.py",
> > line 85, in __call__
> >      await self.app(scope, receive, send)
> >    File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/base.py",
> > line 182, in __call__
> >      with recv_stream, send_stream, collapse_excgroups():
> >                                     ^^^^^^^^^^^^^^^^^^^^
> >    File "/usr/python/lib/python3.12/contextlib.py", line 158, in __exit__
> >      self.gen.throw(value)
> >    File "/usr/python/lib/python3.12/site-packages/starlette/_utils.py",
> > line 85, in collapse_excgroups
> >      raise exc
> >    File
> > "/usr/python/lib/python3.12/site-packages/starlette/middleware/base.py",
> > line 184, in __call__
> >      response = await self.dispatch_func(request, call_next)
> >                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >    File
> >
> "/usr/python/lib/python3.12/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py",
> >
> > line 45, in dispatch
> >      new_user = await self._refresh_user(current_token)
> >                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >    File
> >
> "/usr/python/lib/python3.12/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py",
> >
> > line 67, in _refresh_user
> >      user = await resolve_user_from_token(current_token)
> >             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >    File
> >
> "/usr/python/lib/python3.12/site-packages/airflow/api_fastapi/core_api/security.py",
> >
> > line 103, in resolve_user_from_token
> >      raise HTTPException(status_code=status.HTTP_403_FORBIDDEN,
> > detail="Invalid JWT token")
> > fastapi.exceptions.HTTPException: 403: Invalid JWT token
> >
> > Clearing all local browser session tokens helped though but this we can
> > not release.
> >
> >
> > How to reproduce:
> >
> > Start Airflow via breeze as described above
> >
> > Log in (either with a fresh borwser or clear the cookies/sotrages and log
> > in
> >
> > Stop Airflow and run "breeze down"
> >
> > Start Airflow again and attempt to open a page. Will raise 500 INTERNAL
> > SERVER ERROR
> >
> > Tested both with Firefox and Chromium on Ubuntu Linux
> >
> > Jens
> >
> > On 12/4/25 09:52, Ephraim Anierobi wrote:
> > > Hey fellow Airflowers,
> > >
> > > The release candidates for Apache Airflow 3.1.4rc1 and Task SDK
> 1.1.4rc1
> > > are now available for testing!
> > >
> > > This email is calling for a vote on the release, which will last at
> least
> > > 72 hours, from Thursday, December 4, 2025 at 9:00 am UTC
> > > until Monday, December 8, 2025 at 4:00 pm UTC
> > > <
> >
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=8&iso=20251208T1600&p1=1440
> > >,
> > > and until 3 binding +1 votes have been received.
> > >
> > > Status of testing of the release is kept in
> > > https://github.com/apache/airflow/issues/59033
> > >
> > > Consider this my +1 binding vote.
> > >
> > > Airflow 3.1.4rc1 is available at:
> > > https://dist.apache.org/repos/dist/dev/airflow/3.1.4rc1/
> > >
> > > "apache-airflow" Meta package:
> > > - *apache-airflow-3.1.4-source.tar.gz* is a source release that comes
> > with
> > > INSTALL instructions.
> > > - *apache-airflow-3.1.4.tar.gz* is the binary Python "sdist" release.
> > > - *apache_airflow-3.1.4-py3-none-any.whl* is the binary Python wheel
> > > "binary" release.
> > >
> > > "apache-airflow-core" package:
> > > - *apache_airflow_core-3.1.4.tar.gz* is the binary Python "sdist"
> > release.
> > > - *apache_airflow_core-3.1.4-py3-none-any.whl* is the binary Python
> wheel
> > > "binary" release.
> > >
> > > Task SDK 1.1.4 is available at:
> > > https://dist.apache.org/repos/dist/dev/airflow/task-sdk/1.1.4rc1/
> > >
> > > "apache-airflow-task-sdk" package:
> > > - *apache_airflow_task_sdk-1.1.4.tar.gz* is the binary Python "sdist"
> > > release.
> > > - *apache_airflow_task_sdk-1.1.4-py3-none-any.whl* is the binary Python
> > > wheel "binary" release.
> > >
> > > Public keys are available at:
> > > https://dist.apache.org/repos/dist/release/airflow/KEYS
> > >
> > > Please vote accordingly:
> > >
> > > [ ] +1 approve
> > > [ ] +0 no opinion
> > > [ ] -1 disapprove with the reason
> > >
> > > Only votes from PMC members are binding, but all members of the
> community
> > > are encouraged to test the release and vote with "(non-binding)".
> > >
> > > The test procedure for PMC members is described in:
> > >
> >
> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-pmc-members
> > >
> > > The test procedure for contributors and members of the community who
> > would
> > > like to test this RC is described in:
> > >
> >
> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-contributors
> > >
> > > Please note that the version number excludes the 'rcX' string, so it's
> > now
> > > simply 3.1.4 for Airflow package and 1.1.4 for Task SDK. This will
> allow
> > us
> > > to rename the artifact without modifying
> > > the artifact checksums when we actually release.
> > >
> > > Docs (for preview):
> > > https://airflow.staged.apache.org/docs/apache-airflow/3.1.4
> > >
> > > Release Notes:
> > > - https://github.com/apache/airflow/blob/3.1.4rc1/RELEASE_NOTES.rst
> > > -
> > >
> >
> https://airflow.staged.apache.org/docs/apache-airflow/3.1.4/release_notes.html
> > > (Rendered HTML)
> > >
> > > Testing Instructions using PyPI:
> > > You can build a virtualenv that installs this and other required
> packages
> > > (e.g. task sdk), like this:
> > >
> > > uv venv
> > > uv pip install -U \
> > >    apache-airflow==3.1.4rc1 \
> > >    apache-airflow-core==3.1.4rc1 \
> > >    apache-airflow-task-sdk==1.1.4rc1
> > >
> > > Constraints files are at
> > > https://github.com/apache/airflow/tree/constraints-3.1.4rc1
> > >
> > > Cheers,
> > > Ephraim
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [email protected]
> > For additional commands, e-mail: [email protected]
> >
> >
>

Reply via email to