amoghrajesh opened a new pull request, #46961:
URL: https://github.com/apache/airflow/pull/46961
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
closes: #46754
#45294 added a new key "start_date" to the context, but context shouldnt
have that.
(https://github.com/apache/airflow/blob/v2-10-test/airflow/utils/context.py)
This was leading to the checks in `determine_kwargs(self.python_callable,
self.op_args, context)` failing because the context keys were clashing with the
example dag callable keys.
DAG used to test:
```
# This compares the datetime objects without timezones
# do something with time with both datetime and pendulum
# dags can do same thing
# give it a start date in the past
# proposed test plan:
# dag1:
# - pendulum start time in the past
# - standard datetime in the future as a user macro
# - task that calculates the difference between the two and prints it
# dag2:
# - standard start time in the past
# - pendulum datetime in the future as a user macro
# - task that calculates the difference between the two and prints it
# (future) outer dag:
# - run both and compare that they do the same things
# templated fields to calculate how long ago the start date is
from airflow.models.dag import DAG
from airflow.providers.standard.operators.python import PythonOperator
# from pendulum import datetime, timezone
import pendulum
import datetime
import pytz
etc_gmt13 = 'Etc/GMT-13'
chuuk = 'Pacific/Chuuk'
start_date = datetime.datetime(2021, 11, 25)
def subtractor(start_date, future_date):
print(type(start_date), type(future_date)) # both strings, now find a
way to compare
if "T" in start_date or "T" in future_date:
start_date = start_date.replace("T", "")
future_date = future_date.replace("T", "")
if "00:00:00" in start_date or "00:00:00" in future_date:
start_date = start_date.replace("00:00:00", "")
future_date = future_date.replace("00:00:00", "")
if "+00:00" in start_date or "+00:00" in future_date:
start_date = start_date.replace("+00:00", "")
future_date = future_date.replace("+00:00", "")
print(start_date, future_date) # 2021-11-25 00:00:00+10:00 2135-11-25
00:00:00+13:00
for i in start_date:
if "-" == i:
start_date = start_date.replace("-", "_")
for i in future_date:
if "-" == i:
future_date = future_date.replace("-", "_")
print(
f"The newly filtered start date is {start_date} and the newly
filtered future date is {future_date}")
# 2021_11_25 +10:00, 2135_11_25 +13:00
start_date_obj = datetime.datetime.strptime(start_date.strip(),
"%Y_%m_%d")
future_date_obj = datetime.datetime.strptime(future_date.strip(),
"%Y_%m_%d")
print(f"The start date object is: {start_date_obj}")
print(f"The future date object is: {future_date_obj}")
# now that the filtering has commenced
# compare the 2 datetimes to ensure the result is the same across the 2
dags
# the actual comparison
the_difference = future_date_obj - start_date_obj
print(f"This is the difference between the 2 dates in days:
{the_difference}")
print(f"This is the datatype of the difference betweeen the 2 datetime
objects: {type(the_difference)}")
default_args = {
"owner": "airflow",
}
with DAG(
dag_id="compare_pendulum_datetime2",
default_args=default_args,
# datetime start date, pendulum future date as user defined macro
start_date=start_date,
schedule=None,
catchup=False,
user_defined_macros={
"future_date": pendulum.datetime(2135, 11, 25)
},
tags=['core'],
) as dag:
t1 = PythonOperator(
task_id="subtract_future_and_start_date",
python_callable=subtractor,
op_args=[str(start_date), '{{ future_date }}']
)
t1
```
```
{"timestamp":"2025-02-21T12:31:17.122121Z","level":"info","event":"The start
date object is: 2021-11-25 00:00:00","chan":"stdout","logger":"task"}
{"timestamp":"2025-02-21T12:31:17.122194Z","level":"info","event":"The
future date object is: 2135-11-25 00:00:00","chan":"stdout","logger":"task"}
{"timestamp":"2025-02-21T12:31:17.122272Z","level":"info","event":"This is
the difference between the 2 dates in days: 41637 days,
0:00:00","chan":"stdout","logger":"task"}
```
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
