rawwar opened a new issue, #57608:
URL: https://github.com/apache/airflow/issues/57608
### Apache Airflow version
3.1.1
### If "Other Airflow 2/3 version" selected, which one?
_No response_
### What happened?
While migrating from Airflow 2 to 3, migration failed in
0049_3_0_0_remove_pickled_data_from_xcom_table with below error:
```
sqlalchemy.exc.DataError: (psycopg2.errors.InvalidTextRepresentation)
invalid input syntax for type json
DETAIL: Token "nan" is invalid.
CONTEXT: JSON data, line 1: ...<redact_string>"nan...
[SQL:
ALTER TABLE xcom
ALTER COLUMN value TYPE JSONB
USING CASE
WHEN value IS NOT NULL THEN CAST(CONVERT_FROM(value, 'UTF8')
AS JSONB)
ELSE NULL
END
]
```
### What you think should happen instead?
when XCOM data has a string NaN, the migration will fail due to the way we
are converting null values.
example xcom data:
```
{"craft": "Tiangong", "name": "Ye GuangfuNaN"}
```
### How to reproduce
Have an Airflow instance on 2.x. run below dag
complete dag to reproduce issue:
```
from airflow import Dataset
from airflow.decorators import dag, task
from pendulum import datetime
import requests
# Define the basic parameters of the DAG, like schedule and start_date
@dag(
start_date=datetime(2024, 1, 1),
schedule="@daily",
catchup=False,
doc_md=__doc__,
default_args={"owner": "Astro", "retries": 3},
tags=["example"],
)
def example_astronauts():
@task
def get_astronauts(**context) -> list[dict]:
number_of_people_in_space = 1
list_of_people_in_space = [
{"craft": "Tiangong", "name": "Ye GuangfuNaN"},
]
context["ti"].xcom_push(
key="number_of_people_in_space", value=number_of_people_in_space
)
return list_of_people_in_space
@task
def print_astronaut_craft(greeting: str, person_in_space: dict) -> None:
"""
This task creates a print statement with the name of an
Astronaut in space and the craft they are flying on from
the API request results of the previous task, along with a
greeting which is hard-coded in this example.
"""
craft = person_in_space["craft"]
name = person_in_space["name"]
print(f"{name} is currently in space flying on the {craft}!
{greeting}")
print_astronaut_craft.partial(greeting="Hello! :)").expand(
person_in_space=get_astronauts() # Define dependencies using
TaskFlow API syntax
)
# Instantiate the DAG
example_astronauts()
```
### Operating System
mac
### Versions of Apache Airflow Providers
_No response_
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
