[
https://issues.apache.org/jira/browse/AIRFLOW-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16639242#comment-16639242
]
ASF GitHub Bot commented on AIRFLOW-1837:
-----------------------------------------
dima-asana opened a new pull request #4000: [AIRFLOW-1837] Respect task
start_date when different from dag's
URL: https://github.com/apache/incubator-airflow/pull/4000
Currently task instances get created and scheduled based on the DAG's
start date rather than their own. This commit adds a check before
creating a task instance to see that the start date is not after
the execution date.
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-1837
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Currently task instances get created and scheduled based on the DAG's
start date rather than their own. This PR adds a check before
creating a task instance to see that the start date is not after
the execution date.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
SchedulerJobTest.test_scheduler_task_start_date
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [N/A] In case of new functionality, my PR adds documentation that
describes how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Differing start_dates on tasks not respected by scheduler.
> ----------------------------------------------------------
>
> Key: AIRFLOW-1837
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1837
> Project: Apache Airflow
> Issue Type: Bug
> Affects Versions: 1.9.0
> Reporter: Ash Berlin-Taylor
> Priority: Major
> Attachments: everything_ran.png
>
>
> It it possible to specify start_date directly on tasks in dag, as well as on
> the DAG. This is correctly handled when creating dag runs, but it is
> seemingly ignored when scheduling tasks.
> Given this example:
> {code}
> dag_args = {
> "start_date": datetime(2017, 9, 4),
> }
> dag = DAG(
> "my-dag",
> default_args=dag_args,
> schedule_interval="0 0 * * Mon",
> )
> # ...
> with dag:
> op = PythonOperator(
> python_callable=fetcher.run,
> task_id="fetch_all_respondents",
> provide_context=True,
> # The "unfiltered" API calls are a lot quicker, so lets put them
> # ahead of any other filtered job in the queue.
> priority_weight=10,
> start_date=datetime(2014, 9, 1),
> )
> op = PythonOperator(
> python_callable=fetcher.run,
> task_id="fetch_by_demographics",
> op_kwargs={
> 'demo_names': demo_names,
> },
> provide_context=True,
> priority_weight=5,
> )
> {code}
> I only want the fetch_all_respondents tasks to run for 2014..2017, and then
> from September 2017 I also want the fetch_by_demographics task to run.
> However right now both tasks are being scheduled from 2014-09-01.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
