razoreater opened a new issue, #40573:
URL: https://github.com/apache/airflow/issues/40573
### Official Helm Chart version
1.12.0
### Apache Airflow version
2.7.1
### Kubernetes Version
v1.27.13+048520e
### Helm Chart configuration
Following things are configured in the values.yaml file:
` createUserJob:
useHelmHooks: false
applyCustomEnv: false
migrateDatabaseJob:
useHelmHooks: false
applyCustomEnv: false`
` postgresql:
enabled: false`
### Docker Image customizations
_No response_
### What happened
We were still running a setup based on chart 1.10.0 and started an upgrade
cycle to upgrade our Airflow installation.
The upgrade to 2.7.1 with chart 1.11.0 worked like a charm. However when we
want to upgrade to a version deployed through chart version 1.12.0, we receive
the following error when the **DB migration** runs:
```
[2024-07-02T13:59:36.856+0000] {migration.py:216} INFO - Context impl
MSSQLImpl.
[2024-07-02T13:59:36.858+0000] {migration.py:219} INFO - Will assume
transactional DDL.
[2024-07-02T13:59:36.965+0000] {db.py:1616} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl MSSQLImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 405de8318b3a ->
375a816bbbf4, add new field 'clear_number' to dagrun
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1910, in _execute_context
self.dialect.do_execute(
File
"/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 736, in do_execute
cursor.execute(statement, parameters)
pyodbc.IntegrityError: ('23000', "[23000] [Microsoft][ODBC Driver 18 for SQL
Server][SQL Server]Cannot insert the value NULL into column 'clear_number',
table 'SG_airflow_DV.dbo.dag_run'; column does not allow nulls. UPDATE fails.
(515) (SQLExecDirectW)")
The above exception was the direct cause of the following exception:`
```
When looking up the **table** information, the 'clear_number' column doesn't
seem to be there:
```
USE [SG_airflow_DV]
GO
/****** Object: Table [dbo].[dag_run] Script Date: 3/07/2024 8:58:32
******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[dag_run](
[id] [int] IDENTITY(1,1) NOT NULL,
[dag_id] [varchar](250) NOT NULL,
[queued_at] [datetime2](6) NULL,
[execution_date] [datetime2](6) NOT NULL,
[start_date] [datetime2](6) NULL,
[end_date] [datetime2](6) NULL,
[state] [varchar](50) NULL,
[run_id] [varchar](250) NOT NULL,
[creating_job_id] [int] NULL,
[external_trigger] [bit] NULL,
[run_type] [varchar](50) NOT NULL,
[conf] [varbinary](max) NULL,
[data_interval_start] [datetime2](6) NULL,
[data_interval_end] [datetime2](6) NULL,
[last_scheduling_decision] [datetime2](6) NULL,
[dag_hash] [varchar](32) NULL,
[log_template_id] [int] NULL,
[updated_at] [datetime2](6) NULL,
CONSTRAINT [dag_run_pkey] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF)
ON [PRIMARY],
CONSTRAINT [dag_run_dag_id_execution_date_key] UNIQUE NONCLUSTERED
(
[dag_id] ASC,
[execution_date] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF)
ON [PRIMARY],
CONSTRAINT [dag_run_dag_id_run_id_key] UNIQUE NONCLUSTERED
(
[dag_id] ASC,
[run_id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF)
ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
ALTER TABLE [dbo].[dag_run] WITH CHECK ADD CONSTRAINT
[task_instance_log_template_id_fkey] FOREIGN KEY([log_template_id])
REFERENCES [dbo].[log_template] ([id])
GO
ALTER TABLE [dbo].[dag_run] CHECK CONSTRAINT
[task_instance_log_template_id_fkey]
GO
```
### What you think should happen instead
The DB Migration completes without errors and we can continue with our
upgrade path.
### How to reproduce
We started with a MySQL database and a config based on chart 1.10.0.
This all runs in a Openshift 4 environment, we deploy through ArgoCD.
First upgrade to chart 1.11.0, this should work fine.
Then try to upgrade to version 1.12.0, the migration should fail.
### Anything else
This older issue seems to deal with the same column, these might be related:
https://github.com/apache/airflow/pull/34344
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
