Taragolis commented on issue #34673: URL: https://github.com/apache/airflow/issues/34673#issuecomment-1740138056
This about [Amazon system tests](https://github.com/apache/airflow/blob/main/tests/system/providers/amazon/README.md), we do not run any system tests into the Airflow CI this tests intend to run by owners of the providers or as examples for the documentation. AFAIK some of the Amazon system test run into Amazon CI on actual AWS environments and provide some nice [Dashboard](https://aws-mwaa.github.io/open-source/system-tests/dashboard.html) As I can see, this task return the datetime, however timezone which returned by `boto3` use dateutil `tzinfo` implementation, which can not be serialized https://github.com/apache/airflow/blob/84e60367816f42eaba7f398fb8027dc89173edde/tests/system/providers/amazon/aws/example_dynamodb_to_s3.py#L85-L93 and the deserialise error happen int this task https://github.com/apache/airflow/blob/84e60367816f42eaba7f398fb8027dc89173edde/tests/system/providers/amazon/aws/example_dynamodb_to_s3.py#L165-L174 It works before, because previous (incorrect) implementation return `UTC` when we call `tzname()` during serialisation. --- Simple snippet for show what is going wrong ```python from dateutil.tz import tzutc from datetime import datetime from airflow.serialization.serializers.datetime import serialize, deserialize sample_data = datetime(2021, 1, 1, tzinfo=tzutc()) ser_data = serialize(sample_data) de_data = deserialize(classname=ser_data[1], version=ser_data[2], data=ser_data[0]) # Error happen here ``` --- This might be fixed in system tests if convert it to Pendulum Timezone by use `convert_to_utc`, at least it works locally. cc: @ferruzzi ```python from datetime import datetime from airflow.utils.timezone import convert_to_utc from airflow.serialization.serializers.datetime import serialize, deserialize from zoneinfo import ZoneInfo sample_data = datetime(2021, 1, 1, tzinfo=ZoneInfo("UTC")) sample_data = convert_to_utc(sample_data) # <- This will convert tz to pendulum timezone ser_data = serialize(sample_data) de_data = deserialize(classname=ser_data[1], version=ser_data[2], data=ser_data[0]) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
