imamdigmi commented on issue #18312:
URL: https://github.com/apache/airflow/issues/18312#issuecomment-922277685
Thanks for answering @uranusjr . After some exploring, I'm guessing this
problem has to do with schedule intervals. I want to share the DAG, but
unfortunately I can't share the DAG code that I created due to privacy reasons
from my company, but I'm happy to explain about the DAG I made, this is the
schedule interval I use (but mostly we use `@daily`):
1. `@daily`
1. `@hourly`
1. `0 */3 * * *`
1. `0 8 * * *`
1. `0 7 * * *`
1. `0 11 * * *`
1. `0 21 * * *`
1. `30 12 * * *`
1. `0 12 * * *`
1. `0 13 * * *`
These DAGs are generated dynamically with a custom generator that generates
hundreds of DAG Objects with configurable schedule intervals, and here is the
interval map that I made to support that:
```py
def interval_map(alias):
"""Translate the initials of the schedule interval."""
import string
from datetime import timedelta # noqa: F401
res = {"alias": alias, "interval": None}
default = {
"1h": "@hourly",
"1d": "@daily",
"1w": "@weekly",
"1m": "@monthly",
"1q": "@quarterly",
"1y": "@yearly",
}
if alias is None:
res["alias"] = "manual"
res["interval"] = None
return res
alias = alias.translate(str.maketrans("", "", string.whitespace)).lower()
_number, _type = alias[:-1], alias[-1:] # noqa: F841
if alias == "o":
res["alias"] = "once"
res["interval"] = "@once"
return res
if intrv := default.get(alias):
res["alias"] = alias
res["interval"] = intrv
return res
if "h" in alias:
res["alias"] = alias
res["interval"] = f"0 */{_number} * * *"
return res
if "d" in alias:
res["alias"] = alias
res["interval"] = f"0 0 */{_number} * *"
return res
raise ValueError("Unrecognized interval type")
```
then if we use the above function with:
```py
print(interval_map('4h'))
print(interval_map('1w'))
print(interval_map('2d'))
print(interval_map('1d'))
print(interval_map('o'))
print(interval_map(None))
print(interval_map('1q'))
print(interval_map('1m'))
print(interval_map('1y'))
```
the result is:
```
{'alias': '4h', 'interval': '0 */4 * * *'}
{'alias': '1w', 'interval': '@weekly'}
{'alias': '2d', 'interval': '0 0 */2 * *'}
{'alias': '1d', 'interval': '@daily'}
{'alias': 'once', 'interval': '@once'}
{'alias': 'manual', 'interval': None}
{'alias': '1q', 'interval': '@quarterly'}
{'alias': '1m', 'interval': '@monthly'}
{'alias': '1y', 'interval': '@yearly'}
```
however I haven't found any hint based on the code in this line:
https://github.com/apache/airflow/blob/v2-1-stable/airflow/models/dag.py#L492
I hope my explanation can help you find the problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
