[jira] [Created] (AIRFLOW-500) github enterprise auth teams are implemented naively
Michael Lyons created AIRFLOW-500: - Summary: github enterprise auth teams are implemented naively Key: AIRFLOW-500 URL: https://issues.apache.org/jira/browse/AIRFLOW-500 Project: Apache Airflow Issue Type: Improvement Components: security Reporter: Michael Lyons Assignee: Michael Lyons https://github.com/apache/incubator-airflow/commit/4796245be517aed06df21a85c93a2b86a7f31939#commitcomment-18699757 the summary of the message from github security is that the team string isn't unique across an organization, and you should use the long id instead. not pretty for the config, but it is secure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-497) Release plans & info
[ https://issues.apache.org/jira/browse/AIRFLOW-497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478185#comment-15478185 ] Arthur Wiedmer commented on AIRFLOW-497: Hi Alexander, I think I can leave a quick update here. While the committers and various contributors have worked on several improvements, we have been blocked on navigating our first apache release (a decent amount of contributors are new to this process and it takes a little getting used to). The main issues that the next release will address are licensing issues, stripping out components that were not compatible with the Apache License as well as a few bug fixes. We hope to be able to release more often in the future once we document the release process internally and make sure we are starting with the right base to be a successful project under the Apache umbrella. A general idea of the improvement roadmap can be found on the wiki : https://cwiki.apache.org/confluence/display/AIRFLOW/Roadmap Feel free to ping the dev mailing list also if you have more questions or want to start a conversation about releases. Best, Arthur > Release plans & info > > > Key: AIRFLOW-497 > URL: https://issues.apache.org/jira/browse/AIRFLOW-497 > Project: Apache Airflow > Issue Type: Wish > Components: core, docs >Reporter: Alexander Kachkaev >Priority: Minor > Labels: build, newbie, release > > I did a couple of experiments with airflow several months ago and returned to > explore it properly this week. After a few days of quite intensive reading > and hacking it still remains unclear to me what's going on with the project > ATM. > The latest release is 1.7.1.3, which dates back to 2016-06-13 (three months > from now). The docs on pythonhosted sometimes refer to 1.8 and git blame > reveals that these mentionings have been there since at least April 2016. > JIRA's dashboard has references to versions 1.8 and 2.0, but those only > contain lists with issues - no deadline etc. > I imagine that core developers have a clear picture about the situation and > it is probably possible to figure things out from the mailing list and > gitter, However, it would be good to see roadmap etc. in a slightly more > accessible way. > More frequent releases will help a lot as well. I'm seeing some issues when > running 1.7.1.3 via docker-airflow / celery, but it's totally unclear whether > these still exist on airflow's master branch or even something's wrong with > the docker wrapper I'm using. Opening an issue in JIRA seems somewhat stupid > in this situation. > Could anyone please increase the clarity of meta? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-494) Add per-operator success/failure metrics.
[ https://issues.apache.org/jira/browse/AIRFLOW-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477690#comment-15477690 ] ASF subversion and git services commented on AIRFLOW-494: - Commit daa326cb4dc5e367182f344a957b979952731c73 in incubator-airflow's branch refs/heads/master from [~aoen] [ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=daa326c ] [AIRFLOW-494] Add per-operator success/failure metrics Adds metrics for success/failure rates of each operator, that way when we e.g. do a new release we will have some signal if there is a regression in an operator. It will also be useful if e.g. a user wants to upgrade their infrastructure and make sure that all of the operators still work as expected. Testing Done: - Local staging and make sure that several operators successes/failures were accurately reflected Closes #1785 from aoen/ddavydov/add_per_operator_s uccess_fail_metrics > Add per-operator success/failure metrics. > - > > Key: AIRFLOW-494 > URL: https://issues.apache.org/jira/browse/AIRFLOW-494 > Project: Apache Airflow > Issue Type: Improvement > Components: rics >Reporter: Dan Davydov >Assignee: Dan Davydov >Priority: Minor > > It would be good to have metrics for success/failure rates of each operator, > that way when we e.g. do a new release we will have some signal if there is a > regression in an operator. It will also be useful if e.g. a user wants to > upgrade their infrastructure and make sure that all of the operators still > work as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-airflow git commit: [AIRFLOW-494] Add per-operator success/failure metrics
Repository: incubator-airflow
Updated Branches:
refs/heads/master 3a1be4aac -> daa326cb4
[AIRFLOW-494] Add per-operator success/failure metrics
Adds metrics for success/failure rates of each operator, that way
when we e.g. do a new release we will have some
signal if there is a regression in an operator. It
will also be useful if e.g. a user wants to
upgrade their infrastructure and make sure that
all of the operators still work as expected.
Testing Done:
- Local staging and make sure that several
operators successes/failures were accurately
reflected
Closes #1785 from aoen/ddavydov/add_per_operator_s
uccess_fail_metrics
Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/daa326cb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/daa326cb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/daa326cb
Branch: refs/heads/master
Commit: daa326cb4dc5e367182f344a957b979952731c73
Parents: 3a1be4a
Author: Dan Davydov
Authored: Fri Sep 9 10:37:28 2016 -0700
Committer: Dan Davydov
Committed: Fri Sep 9 10:37:32 2016 -0700
--
airflow/models.py | 3 +++
1 file changed, 3 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/daa326cb/airflow/models.py
--
diff --git a/airflow/models.py b/airflow/models.py
index 64727d6..15bbc30 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -1268,6 +1268,8 @@ class TaskInstance(Base):
self.xcom_push(key=XCOM_RETURN_KEY, value=result)
task_copy.post_execute(context=context)
+Stats.incr('operator_successes_{}'.format(
+self.task.__class__.__name__), 1, 1)
self.state = State.SUCCESS
except AirflowSkipException:
self.state = State.SKIPPED
@@ -1307,6 +1309,7 @@ class TaskInstance(Base):
session = settings.Session()
self.end_date = datetime.now()
self.set_duration()
+Stats.incr('operator_failures_{}'.format(task.__class__.__name__), 1,
1)
if not test_mode:
session.add(Log(State.FAILED, self))
[jira] [Commented] (AIRFLOW-85) Create DAGs UI
[
https://issues.apache.org/jira/browse/AIRFLOW-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477580#comment-15477580
]
Chris Riccomini commented on AIRFLOW-85:
No one is working on this on our end. If you would like to take it over, that
would be fantastic.
> Create DAGs UI
> --
>
> Key: AIRFLOW-85
> URL: https://issues.apache.org/jira/browse/AIRFLOW-85
> Project: Apache Airflow
> Issue Type: Bug
> Components: security, ui
>Reporter: Chris Riccomini
>
> Airflow currently provides only an {{/admin}} UI interface for the webapp.
> This UI provides three distinct roles:
> * Admin
> * Data profiler
> * None
> In addition, Airflow currently provides the ability to log in, either via a
> secure proxy front-end, or via LDAP/Kerberos, within the webapp.
> We run Airflow with LDAP authentication enabled. This helps us control access
> to the UI. However, there is insufficient granularity within the UI. We would
> like to be able to grant users the ability to:
> # View their DAGs, but no one else's.
> # Control their DAGs, but no one else's.
> This is not possible right now. You can take away the ability to access the
> connections and data profiling tabs, but users can still see all DAGs, as
> well as control the state of the DB by clearing any DAG status, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (AIRFLOW-499) Add in capabilities to use atlassian tools with Airflow
Tony Doran created AIRFLOW-499: -- Summary: Add in capabilities to use atlassian tools with Airflow Key: AIRFLOW-499 URL: https://issues.apache.org/jira/browse/AIRFLOW-499 Project: Apache Airflow Issue Type: New Feature Reporter: Tony Doran -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-401) scheduler gets stuck without a trace
[ https://issues.apache.org/jira/browse/AIRFLOW-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477516#comment-15477516 ] Maciej Bryński commented on AIRFLOW-401: I will try this. In the meantime I found not documented min_file_process_interval option. That's solved many of my problems but trigger new. How can I set up HA Scheduler ? Having more than one instance triggers duplicates of DagRuns. > scheduler gets stuck without a trace > > > Key: AIRFLOW-401 > URL: https://issues.apache.org/jira/browse/AIRFLOW-401 > Project: Apache Airflow > Issue Type: Bug > Components: executor, scheduler >Affects Versions: Airflow 1.7.1.3 >Reporter: Nadeem Ahmed Nazeer >Assignee: Bolke de Bruin >Priority: Minor > Attachments: Dag_code.txt, schduler_cpu100%.png, scheduler_stuck.png, > scheduler_stuck_7hours.png > > > The scheduler gets stuck without a trace or error. When this happens, the CPU > usage of scheduler service is at 100%. No jobs get submitted and everything > comes to a halt. Looks it goes into some kind of infinite loop. > The only way I could make it run again is by manually restarting the > scheduler service. But again, after running some tasks it gets stuck. I've > tried with both Celery and Local executors but same issue occurs. I am using > the -n 3 parameter while starting scheduler. > Scheduler configs, > job_heartbeat_sec = 5 > scheduler_heartbeat_sec = 5 > executor = LocalExecutor > parallelism = 32 > Please help. I would be happy to provide any other information needed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (AIRFLOW-498) Fix GCP Dataflow Hook
[ https://issues.apache.org/jira/browse/AIRFLOW-498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-498 started by Julian V. Modesto. - > Fix GCP Dataflow Hook > - > > Key: AIRFLOW-498 > URL: https://issues.apache.org/jira/browse/AIRFLOW-498 > Project: Apache Airflow > Issue Type: Bug > Components: gcp >Reporter: Julian V. Modesto >Assignee: Julian V. Modesto > Labels: gcp > Fix For: Airflow 1.8 > > > The GCP Dataflow hook added in AIRFLOW-24 contains a hard-coded GCP project > ID here: > https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-498) Fix GCP Dataflow Hook
[ https://issues.apache.org/jira/browse/AIRFLOW-498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian V. Modesto updated AIRFLOW-498: -- External issue URL: (was: https://github.com/apache/incubator-airflow/pull/1786) > Fix GCP Dataflow Hook > - > > Key: AIRFLOW-498 > URL: https://issues.apache.org/jira/browse/AIRFLOW-498 > Project: Apache Airflow > Issue Type: Bug > Components: gcp >Reporter: Julian V. Modesto >Assignee: Julian V. Modesto > Labels: gcp > Fix For: Airflow 1.8 > > > The GCP Dataflow hook added in AIRFLOW-24 contains a hard-coded GCP project > ID here: > https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-498) Fix GCP Dataflow Hook
[ https://issues.apache.org/jira/browse/AIRFLOW-498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian V. Modesto updated AIRFLOW-498: -- External issue URL: https://github.com/apache/incubator-airflow/pull/1786 > Fix GCP Dataflow Hook > - > > Key: AIRFLOW-498 > URL: https://issues.apache.org/jira/browse/AIRFLOW-498 > Project: Apache Airflow > Issue Type: Bug > Components: gcp >Reporter: Julian V. Modesto >Assignee: Julian V. Modesto > Labels: gcp > Fix For: Airflow 1.8 > > > The GCP Dataflow hook added in AIRFLOW-24 contains a hard-coded GCP project > ID here: > https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-498) Fix GCP Dataflow Hook
[ https://issues.apache.org/jira/browse/AIRFLOW-498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian V. Modesto updated AIRFLOW-498: -- Description: The GCP Dataflow hook added in AIRFLOW-24 contains a hard-coded GCP project ID here: https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142 (was: The GCP Dataflow hook added in [AIRFLOW-24](https://issues.apache.org/jira/browse/AIRFLOW-24) contains a hard-coded GCP project ID here: https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142) > Fix GCP Dataflow Hook > - > > Key: AIRFLOW-498 > URL: https://issues.apache.org/jira/browse/AIRFLOW-498 > Project: Apache Airflow > Issue Type: Bug > Components: gcp >Reporter: Julian V. Modesto > Labels: gcp > Fix For: Airflow 1.8 > > > The GCP Dataflow hook added in AIRFLOW-24 contains a hard-coded GCP project > ID here: > https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-498) Fix GCP Dataflow Hook
[ https://issues.apache.org/jira/browse/AIRFLOW-498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian V. Modesto updated AIRFLOW-498: -- Description: The GCP Dataflow hook added in [AIRFLOW-24](https://issues.apache.org/jira/browse/AIRFLOW-24) contains a hard-coded GCP project ID here: https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142 (was: Hi, I'm using Airflow 1.7.3, but I added the hook and operator from https://issues.apache.org/jira/browse/AIRFLOW-24 to my plugins, but the Dataflow hook doesn't work. For example, one bug is that a GCP project ID is hardcoded here: https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142 I was able to fix the operator, so I'd be happy to compare my fixes and open a PR to fix this bug.) > Fix GCP Dataflow Hook > - > > Key: AIRFLOW-498 > URL: https://issues.apache.org/jira/browse/AIRFLOW-498 > Project: Apache Airflow > Issue Type: Bug > Components: gcp >Reporter: Julian V. Modesto > Labels: gcp > Fix For: Airflow 1.8 > > > The GCP Dataflow hook added in > [AIRFLOW-24](https://issues.apache.org/jira/browse/AIRFLOW-24) contains a > hard-coded GCP project ID here: > https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-498) Fix GCP Dataflow Hook
Julian Victor Modesto created AIRFLOW-498: - Summary: Fix GCP Dataflow Hook Key: AIRFLOW-498 URL: https://issues.apache.org/jira/browse/AIRFLOW-498 Project: Apache Airflow Issue Type: Bug Components: gcp Reporter: Julian Victor Modesto Fix For: Airflow 1.8 Hi, I'm using Airflow 1.7.3, but I added the hook and operator from https://issues.apache.org/jira/browse/AIRFLOW-24 to my plugins, but the Dataflow hook doesn't work. For example, one bug is that a GCP project ID is hardcoded here: https://github.com/apache/incubator-airflow/pull/1648/files#diff-074de2df4632aa336129e3b30cbd63adR142 I was able to fix the operator, so I'd be happy to compare my fixes and open a PR to fix this bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-497) Release plans & info
Alexander Kachkaev created AIRFLOW-497: -- Summary: Release plans & info Key: AIRFLOW-497 URL: https://issues.apache.org/jira/browse/AIRFLOW-497 Project: Apache Airflow Issue Type: Wish Components: core, docs Reporter: Alexander Kachkaev Priority: Minor I did a couple of experiments with airflow several months ago and returned to explore it properly this week. After a few days of quite intensive reading and hacking it still remains unclear to me what's going on with the project ATM. The latest release is 1.7.1.3, which dates back to 2016-06-13 (three months from now). The docs on pythonhosted sometimes refer to 1.8 and git blame reveals that these mentionings have been there since at least April 2016. JIRA's dashboard has references to versions 1.8 and 2.0, but those only contain lists with issues - no deadline etc. I imagine that core developers have a clear picture about the situation and it is probably possible to figure things out from the mailing list and gitter, However, it would be good to see roadmap etc. in a slightly more accessible way. More frequent releases will help a lot as well. I'm seeing some issues when running 1.7.1.3 via docker-airflow / celery, but it's totally unclear whether these still exist on airflow's master branch or even something's wrong with the docker wrapper I'm using. Opening an issue in JIRA seems somewhat stupid in this situation. Could anyone please increase the clarity of meta? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (AIRFLOW-78) airflow clear leaves dag_runs
[ https://issues.apache.org/jira/browse/AIRFLOW-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin reopened AIRFLOW-78: --- Change was reverted due to regressions monitored and tasks not being scheduled when max_dag_runs was reached. > airflow clear leaves dag_runs > - > > Key: AIRFLOW-78 > URL: https://issues.apache.org/jira/browse/AIRFLOW-78 > Project: Apache Airflow > Issue Type: Wish > Components: cli >Affects Versions: Airflow 1.6.2 >Reporter: Adrian Bridgett >Assignee: Norman Mu >Priority: Minor > > (moved from https://github.com/apache/incubator-airflow/issues/829) > "airflow clear -c -d -s 2016-01-03 dagid" doesn't clear the dagrun, it sets > it to running instead (apparently since this is often used to re-run jobs). > However this then breaks max_active_runs=1 (I have to stop the scheduler, > then airflow clear, psql to delete the dagrun, then start the scheduler). > This problem was probably seen on an Airflow 1.6.x install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-78) airflow clear leaves dag_runs
[ https://issues.apache.org/jira/browse/AIRFLOW-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15476471#comment-15476471 ] ASF subversion and git services commented on AIRFLOW-78: Commit 3a1be4aacf31ee33d6128e5d5fa563a7625c7c62 in incubator-airflow's branch refs/heads/master from [~bolke] [ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=3a1be4a ] Revert "[AIRFLOW-78] airflow clear leaves dag_runs" This reverts commit 197c9050ef3a142c18aa97819da48ee8cadbf8d8. Regressions were observed and tasks were not scheduled in case of max_dag_runs reached. > airflow clear leaves dag_runs > - > > Key: AIRFLOW-78 > URL: https://issues.apache.org/jira/browse/AIRFLOW-78 > Project: Apache Airflow > Issue Type: Wish > Components: cli >Affects Versions: Airflow 1.6.2 >Reporter: Adrian Bridgett >Assignee: Norman Mu >Priority: Minor > > (moved from https://github.com/apache/incubator-airflow/issues/829) > "airflow clear -c -d -s 2016-01-03 dagid" doesn't clear the dagrun, it sets > it to running instead (apparently since this is often used to re-run jobs). > However this then breaks max_active_runs=1 (I have to stop the scheduler, > then airflow clear, psql to delete the dagrun, then start the scheduler). > This problem was probably seen on an Airflow 1.6.x install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-airflow git commit: Revert "[AIRFLOW-78] airflow clear leaves dag_runs"
Repository: incubator-airflow
Updated Branches:
refs/heads/master 32f3c1c5d -> 3a1be4aac
Revert "[AIRFLOW-78] airflow clear leaves dag_runs"
This reverts commit 197c9050ef3a142c18aa97819da48ee8cadbf8d8.
Regressions were observed and tasks were not scheduled in case of
max_dag_runs reached.
Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/3a1be4aa
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/3a1be4aa
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/3a1be4aa
Branch: refs/heads/master
Commit: 3a1be4aacf31ee33d6128e5d5fa563a7625c7c62
Parents: 32f3c1c
Author: Bolke de Bruin
Authored: Fri Sep 9 11:34:46 2016 +0200
Committer: Bolke de Bruin
Committed: Fri Sep 9 11:34:46 2016 +0200
--
airflow/jobs.py | 4
airflow/models.py | 9 -
tests/jobs.py | 18 --
3 files changed, 4 insertions(+), 27 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/3a1be4aa/airflow/jobs.py
--
diff --git a/airflow/jobs.py b/airflow/jobs.py
index 075d2fb..bae1168 100644
--- a/airflow/jobs.py
+++ b/airflow/jobs.py
@@ -1073,10 +1073,6 @@ class SchedulerJob(BaseJob):
"""
for dag in dags:
dag = dagbag.get_dag(dag.dag_id)
-if dag.reached_max_runs:
-self.logger.info("Not processing DAG {} since its max runs has
been reached"
-.format(dag.dag_id))
-continue
if dag.is_paused:
self.logger.info("Not processing DAG {} since it's paused"
.format(dag.dag_id))
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/3a1be4aa/airflow/models.py
--
diff --git a/airflow/models.py b/airflow/models.py
index b352b43..64727d6 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -2786,15 +2786,6 @@ class DAG(BaseDag, LoggingMixin):
l += task.subdag.subdags
return l
-@property
-def reached_max_runs(self):
-active_runs = DagRun.find(
-dag_id=self.dag_id,
-state=State.RUNNING,
-external_trigger=False
-)
-return len(active_runs) >= self.max_active_runs
-
def resolve_template_files(self):
for t in self.tasks:
t.resolve_template_files()
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/3a1be4aa/tests/jobs.py
--
diff --git a/tests/jobs.py b/tests/jobs.py
index ae94d98..af7ad61 100644
--- a/tests/jobs.py
+++ b/tests/jobs.py
@@ -20,7 +20,6 @@ from __future__ import unicode_literals
import datetime
import logging
import os
-import time
import unittest
from airflow import AirflowException, settings
@@ -620,23 +619,14 @@ class SchedulerJobTest(unittest.TestCase):
session.commit()
session.close()
-scheduler = SchedulerJob(dag.dag_id,
-run_duration=1)
+scheduler = SchedulerJob()
+dag.clear()
dr = scheduler.create_dag_run(dag)
self.assertIsNotNone(dr)
-dr2 = scheduler.create_dag_run(dag)
-self.assertIsNone(dr2)
-
-dag.clear()
-
-dag.max_active_runs = 0
-scheduler.run()
-
-session = settings.Session()
-self.assertEqual(
-len(session.query(TI).filter(TI.dag_id == dag.dag_id).all()), 0)
+dr = scheduler.create_dag_run(dag)
+self.assertIsNone(dr)
def test_scheduler_fail_dagrun_timeout(self):
"""
