[jira] [Commented] (AIRFLOW-2858) Make tox.ini indentation and whitespace consistent
[
https://issues.apache.org/jira/browse/AIRFLOW-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569764#comment-16569764
]
ASF GitHub Bot commented on AIRFLOW-2858:
-
tedmiston closed pull request #3705: [AIRFLOW-2858] Make tox.ini indentation
and whitespace consistent
URL: https://github.com/apache/incubator-airflow/pull/3705
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/tox.ini b/tox.ini
index a20d2c5546..b9c386afb1 100644
--- a/tox.ini
+++ b/tox.ini
@@ -18,7 +18,7 @@
[tox]
envlist = flake8,{py27,py35}-backend_{mysql,sqlite,postgres}
-skipsdist=True
+skipsdist = True
[global]
wheel_dir = {homedir}/.wheelhouse
@@ -36,19 +36,19 @@ deps =
coveralls
basepython =
- py27: python2.7
- py35: python3.5
+py27: python2.7
+py35: python3.5
setenv =
- HADOOP_DISTRO=cdh
- HADOOP_HOME=/tmp/hadoop-cdh
- HADOOP_OPTS=-D/tmp/krb5.conf
- MINICLUSTER_HOME=/tmp/minicluster-1.1-SNAPSHOT
- HIVE_HOME=/tmp/hive
- backend_mysql: AIRFLOW__CORE__SQL_ALCHEMY_CONN=mysql://root@localhost/airflow
- backend_postgres:
AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql+psycopg2://postgres@localhost/airflow
- backend_sqlite:
AIRFLOW__CORE__SQL_ALCHEMY_CONN=sqlite:///{homedir}/airflow/airflow.db
- backend_sqlite: AIRFLOW__CORE__EXECUTOR=SequentialExecutor
+HADOOP_DISTRO = cdh
+HADOOP_HOME = /tmp/hadoop-cdh
+HADOOP_OPTS = -D/tmp/krb5.conf
+MINICLUSTER_HOME = /tmp/minicluster-1.1-SNAPSHOT
+HIVE_HOME = /tmp/hive
+backend_mysql: AIRFLOW__CORE__SQL_ALCHEMY_CONN =
mysql://root@localhost/airflow
+backend_postgres: AIRFLOW__CORE__SQL_ALCHEMY_CONN =
postgresql+psycopg2://postgres@localhost/airflow
+backend_sqlite: AIRFLOW__CORE__SQL_ALCHEMY_CONN =
sqlite:///{homedir}/airflow/airflow.db
+backend_sqlite: AIRFLOW__CORE__EXECUTOR = SequentialExecutor
passenv =
HOME
@@ -67,15 +67,15 @@ passenv =
SLUGIFY_USES_TEXT_UNIDECODE
commands =
- pip wheel -w {homedir}/.wheelhouse -f {homedir}/.wheelhouse -e .[devel_ci]
- pip install --find-links={homedir}/.wheelhouse --no-index -e .[devel_ci]
- sudo {toxinidir}/scripts/ci/setup_kdc.sh
- {toxinidir}/scripts/ci/setup_env.sh
- {toxinidir}/scripts/ci/ldap.sh
- {toxinidir}/scripts/ci/load_fixtures.sh
- {toxinidir}/scripts/ci/load_data.sh
- {toxinidir}/scripts/ci/run_tests.sh []
- {toxinidir}/scripts/ci/check-license.sh
+pip wheel -w {homedir}/.wheelhouse -f {homedir}/.wheelhouse -e .[devel_ci]
+pip install --find-links={homedir}/.wheelhouse --no-index -e .[devel_ci]
+sudo {toxinidir}/scripts/ci/setup_kdc.sh
+{toxinidir}/scripts/ci/setup_env.sh
+{toxinidir}/scripts/ci/ldap.sh
+{toxinidir}/scripts/ci/load_fixtures.sh
+{toxinidir}/scripts/ci/load_data.sh
+{toxinidir}/scripts/ci/run_tests.sh []
+{toxinidir}/scripts/ci/check-license.sh
[testenv:flake8]
basepython = python3
@@ -85,4 +85,3 @@ deps =
commands =
{toxinidir}/scripts/ci/flake8_diff.sh
-
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Make tox.ini indentation and whitespace consistent
> --
>
> Key: AIRFLOW-2858
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2858
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ci, tests
>Reporter: Taylor Edmiston
>Assignee: Taylor Edmiston
>Priority: Minor
>
> The tox.ini file currently mixes multiple indentation and whitespace styles.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] tedmiston commented on issue #3705: [AIRFLOW-2858] Make tox.ini indentation and whitespace consistent
tedmiston commented on issue #3705: [AIRFLOW-2858] Make tox.ini indentation and whitespace consistent URL: https://github.com/apache/incubator-airflow/pull/3705#issuecomment-410596936 The build failures on Python 3.5 seem random / unrelated. Triggering a rebuild to see if it passes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2858) Make tox.ini indentation and whitespace consistent
[
https://issues.apache.org/jira/browse/AIRFLOW-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569765#comment-16569765
]
ASF GitHub Bot commented on AIRFLOW-2858:
-
tedmiston opened a new pull request #3705: [AIRFLOW-2858] Make tox.ini
indentation and whitespace consistent
URL: https://github.com/apache/incubator-airflow/pull/3705
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2858
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
The tox.ini file currently mixes multiple indentation and whitespace styles.
This PR makes tox.ini consistently use 4-space indentation to match Python
code and the [examples in the official Tox
docs](https://tox.readthedocs.io/en/latest/example/basic.html).
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Verified by successful run of tox command on CI.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Make tox.ini indentation and whitespace consistent
> --
>
> Key: AIRFLOW-2858
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2858
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ci, tests
>Reporter: Taylor Edmiston
>Assignee: Taylor Edmiston
>Priority: Minor
>
> The tox.ini file currently mixes multiple indentation and whitespace styles.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] tedmiston closed pull request #3705: [AIRFLOW-2858] Make tox.ini indentation and whitespace consistent
tedmiston closed pull request #3705: [AIRFLOW-2858] Make tox.ini indentation
and whitespace consistent
URL: https://github.com/apache/incubator-airflow/pull/3705
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/tox.ini b/tox.ini
index a20d2c5546..b9c386afb1 100644
--- a/tox.ini
+++ b/tox.ini
@@ -18,7 +18,7 @@
[tox]
envlist = flake8,{py27,py35}-backend_{mysql,sqlite,postgres}
-skipsdist=True
+skipsdist = True
[global]
wheel_dir = {homedir}/.wheelhouse
@@ -36,19 +36,19 @@ deps =
coveralls
basepython =
- py27: python2.7
- py35: python3.5
+py27: python2.7
+py35: python3.5
setenv =
- HADOOP_DISTRO=cdh
- HADOOP_HOME=/tmp/hadoop-cdh
- HADOOP_OPTS=-D/tmp/krb5.conf
- MINICLUSTER_HOME=/tmp/minicluster-1.1-SNAPSHOT
- HIVE_HOME=/tmp/hive
- backend_mysql: AIRFLOW__CORE__SQL_ALCHEMY_CONN=mysql://root@localhost/airflow
- backend_postgres:
AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql+psycopg2://postgres@localhost/airflow
- backend_sqlite:
AIRFLOW__CORE__SQL_ALCHEMY_CONN=sqlite:///{homedir}/airflow/airflow.db
- backend_sqlite: AIRFLOW__CORE__EXECUTOR=SequentialExecutor
+HADOOP_DISTRO = cdh
+HADOOP_HOME = /tmp/hadoop-cdh
+HADOOP_OPTS = -D/tmp/krb5.conf
+MINICLUSTER_HOME = /tmp/minicluster-1.1-SNAPSHOT
+HIVE_HOME = /tmp/hive
+backend_mysql: AIRFLOW__CORE__SQL_ALCHEMY_CONN =
mysql://root@localhost/airflow
+backend_postgres: AIRFLOW__CORE__SQL_ALCHEMY_CONN =
postgresql+psycopg2://postgres@localhost/airflow
+backend_sqlite: AIRFLOW__CORE__SQL_ALCHEMY_CONN =
sqlite:///{homedir}/airflow/airflow.db
+backend_sqlite: AIRFLOW__CORE__EXECUTOR = SequentialExecutor
passenv =
HOME
@@ -67,15 +67,15 @@ passenv =
SLUGIFY_USES_TEXT_UNIDECODE
commands =
- pip wheel -w {homedir}/.wheelhouse -f {homedir}/.wheelhouse -e .[devel_ci]
- pip install --find-links={homedir}/.wheelhouse --no-index -e .[devel_ci]
- sudo {toxinidir}/scripts/ci/setup_kdc.sh
- {toxinidir}/scripts/ci/setup_env.sh
- {toxinidir}/scripts/ci/ldap.sh
- {toxinidir}/scripts/ci/load_fixtures.sh
- {toxinidir}/scripts/ci/load_data.sh
- {toxinidir}/scripts/ci/run_tests.sh []
- {toxinidir}/scripts/ci/check-license.sh
+pip wheel -w {homedir}/.wheelhouse -f {homedir}/.wheelhouse -e .[devel_ci]
+pip install --find-links={homedir}/.wheelhouse --no-index -e .[devel_ci]
+sudo {toxinidir}/scripts/ci/setup_kdc.sh
+{toxinidir}/scripts/ci/setup_env.sh
+{toxinidir}/scripts/ci/ldap.sh
+{toxinidir}/scripts/ci/load_fixtures.sh
+{toxinidir}/scripts/ci/load_data.sh
+{toxinidir}/scripts/ci/run_tests.sh []
+{toxinidir}/scripts/ci/check-license.sh
[testenv:flake8]
basepython = python3
@@ -85,4 +85,3 @@ deps =
commands =
{toxinidir}/scripts/ci/flake8_diff.sh
-
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] tedmiston opened a new pull request #3705: [AIRFLOW-2858] Make tox.ini indentation and whitespace consistent
tedmiston opened a new pull request #3705: [AIRFLOW-2858] Make tox.ini
indentation and whitespace consistent
URL: https://github.com/apache/incubator-airflow/pull/3705
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2858
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
The tox.ini file currently mixes multiple indentation and whitespace styles.
This PR makes tox.ini consistently use 4-space indentation to match Python
code and the [examples in the official Tox
docs](https://tox.readthedocs.io/en/latest/example/basic.html).
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Verified by successful run of tox command on CI.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Closed] (AIRFLOW-2788) pymysql.err.ProgrammingError insert into log mysql table error
[
https://issues.apache.org/jira/browse/AIRFLOW-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eran Levy closed AIRFLOW-2788.
--
Resolution: Fixed
Assignee: Eran Levy
see last comment re compatibility issues
> pymysql.err.ProgrammingError insert into log mysql table error
> --
>
> Key: AIRFLOW-2788
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2788
> Project: Apache Airflow
> Issue Type: Bug
> Components: webapp
>Affects Versions: 1.9.0
> Environment: macos
>Reporter: Eran Levy
>Assignee: Eran Levy
>Priority: Major
> Fix For: 1.9.0
>
>
> Im using airflow 1.9.0 on macos, installed using:
> pip install apache-airflow
> Configured MySQL 5.7 as my metadata backend database server with pymysql.
> airflow initdb bootstrapped the database tables successfully.
> Error raised only when used mysql - didnt encounter any error while using the
> default SQLite database.
> Reproduce steps:
> 1. Run the airflow webserver
> 2. Open the UI and click "DAGs" in the top navigation menu
> 3. Click one of the available dags - example dags or your own
> 4. You will see an "Ooops" page with the following error:
> {code:java}
> [2018-07-23 13:53:35,815] ERROR in app: Exception on /admin/airflow/tree [GET]
> Traceback (most recent call last):
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1988, in wsgi_app
> response = self.full_dispatch_request()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1641, in full_dispatch_request
> rv = self.handle_user_exception(e)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1544, in handle_user_exception
> reraise(exc_type, exc_value, tb)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1639, in full_dispatch_request
> rv = self.dispatch_request()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1625, in dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask_admin/base.py",
> line 69, in inner
> return self._run_view(f, *args, **kwargs)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask_admin/base.py",
> line 368, in _run_view
> return fn(self, *args, **kwargs)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask_login.py", line
> 755, in decorated_view
> return func(*args, **kwargs)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/airflow/www/utils.py",
> line 356, in view_func
> return f(*args, **kwargs)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/airflow/www/utils.py",
> line 260, in wrapper
> session.commit()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 906, in commit
> self.transaction.commit()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 461, in commit
> self._prepare_impl()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 441, in _prepare_impl
> self.session.flush()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 2171, in flush
> self._flush(objects)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 2291, in _flush
> transaction.rollback(_capture_exception=True)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
> line 66, in __exit__
> compat.reraise(exc_type, exc_value, exc_tb)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 2255, in _flush
> flush_context.execute()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py",
> line 389, in execute
> rec.execute(self)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py",
> line 548, in execute
> uow
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py",
> line 181, in save_obj
> mapper, table, insert)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py",
> line 835, in _emit_insert_statements
> execute(statement, params)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 945, in execute
> return meth(self, multiparams, params)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/sql/elements.py",
[jira] [Commented] (AIRFLOW-2788) pymysql.err.ProgrammingError insert into log mysql table error
[
https://issues.apache.org/jira/browse/AIRFLOW-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569729#comment-16569729
]
Eran Levy commented on AIRFLOW-2788:
Solved. Use python >= 3.4, if you are using pymysql DBAPI,
airflow=1.9 with python=2.7 and pymysql=0.9.2 got compatibility issues and
causes the metadata CRUD statements to break.
> pymysql.err.ProgrammingError insert into log mysql table error
> --
>
> Key: AIRFLOW-2788
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2788
> Project: Apache Airflow
> Issue Type: Bug
> Components: webapp
>Affects Versions: 1.9.0
> Environment: macos
>Reporter: Eran Levy
>Priority: Major
> Fix For: 1.9.0
>
>
> Im using airflow 1.9.0 on macos, installed using:
> pip install apache-airflow
> Configured MySQL 5.7 as my metadata backend database server with pymysql.
> airflow initdb bootstrapped the database tables successfully.
> Error raised only when used mysql - didnt encounter any error while using the
> default SQLite database.
> Reproduce steps:
> 1. Run the airflow webserver
> 2. Open the UI and click "DAGs" in the top navigation menu
> 3. Click one of the available dags - example dags or your own
> 4. You will see an "Ooops" page with the following error:
> {code:java}
> [2018-07-23 13:53:35,815] ERROR in app: Exception on /admin/airflow/tree [GET]
> Traceback (most recent call last):
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1988, in wsgi_app
> response = self.full_dispatch_request()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1641, in full_dispatch_request
> rv = self.handle_user_exception(e)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1544, in handle_user_exception
> reraise(exc_type, exc_value, tb)
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1639, in full_dispatch_request
> rv = self.dispatch_request()
> File "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask/app.py",
> line 1625, in dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask_admin/base.py",
> line 69, in inner
> return self._run_view(f, *args, **kwargs)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask_admin/base.py",
> line 368, in _run_view
> return fn(self, *args, **kwargs)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/flask_login.py", line
> 755, in decorated_view
> return func(*args, **kwargs)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/airflow/www/utils.py",
> line 356, in view_func
> return f(*args, **kwargs)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/airflow/www/utils.py",
> line 260, in wrapper
> session.commit()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 906, in commit
> self.transaction.commit()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 461, in commit
> self._prepare_impl()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 441, in _prepare_impl
> self.session.flush()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 2171, in flush
> self._flush(objects)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 2291, in _flush
> transaction.rollback(_capture_exception=True)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
> line 66, in __exit__
> compat.reraise(exc_type, exc_value, exc_tb)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 2255, in _flush
> flush_context.execute()
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py",
> line 389, in execute
> rec.execute(self)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py",
> line 548, in execute
> uow
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py",
> line 181, in save_obj
> mapper, table, insert)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py",
> line 835, in _emit_insert_statements
> execute(statement, params)
> File
> "/Users/eranlevy/anaconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 945, in execute
> return meth(self,
[GitHub] codecov-io edited a comment on issue #3692: [AIRFLOW-2847] Remove legacy imports support for plugins for 2.0 deprecation.
codecov-io edited a comment on issue #3692: [AIRFLOW-2847] Remove legacy imports support for plugins for 2.0 deprecation. URL: https://github.com/apache/incubator-airflow/pull/3692#issuecomment-410581422 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=h1) Report > Merging [#3692](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/dc64649d98f9c6ddb508b49b52a61fc7799e4343?src=pr=desc) will **increase** coverage by `0.09%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3692 +/- ## == + Coverage 77.56% 77.66% +0.09% == Files 204 204 Lines 1576615737 -29 == - Hits122291 -7 + Misses 3537 3515 -22 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3692/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvX19pbml0X18ucHk=) | `83.33% <ø> (+22.22%)` | :arrow_up: | | [airflow/hooks/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3692/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9fX2luaXRfXy5weQ==) | `100% <ø> (+41.17%)` | :arrow_up: | | [airflow/macros/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3692/diff?src=pr=tree#diff-YWlyZmxvdy9tYWNyb3MvX19pbml0X18ucHk=) | `100% <ø> (+18.51%)` | :arrow_up: | | [airflow/sensors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3692/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL19faW5pdF9fLnB5) | `81.81% <ø> (+22.99%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=footer). Last update [dc64649...7031e25](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #3692: [AIRFLOW-2847] Remove legacy imports support for plugins for 2.0 deprecation.
codecov-io commented on issue #3692: [AIRFLOW-2847] Remove legacy imports support for plugins for 2.0 deprecation. URL: https://github.com/apache/incubator-airflow/pull/3692#issuecomment-410581422 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=h1) Report > Merging [#3692](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/dc64649d98f9c6ddb508b49b52a61fc7799e4343?src=pr=desc) will **increase** coverage by `0.09%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3692 +/- ## == + Coverage 77.56% 77.66% +0.09% == Files 204 204 Lines 1576615737 -29 == - Hits122291 -7 + Misses 3537 3515 -22 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/hooks/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3692/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9fX2luaXRfXy5weQ==) | `100% <ø> (+41.17%)` | :arrow_up: | | [airflow/operators/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3692/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvX19pbml0X18ucHk=) | `83.33% <ø> (+22.22%)` | :arrow_up: | | [airflow/macros/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3692/diff?src=pr=tree#diff-YWlyZmxvdy9tYWNyb3MvX19pbml0X18ucHk=) | `100% <ø> (+18.51%)` | :arrow_up: | | [airflow/sensors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3692/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL19faW5pdF9fLnB5) | `81.81% <ø> (+22.99%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=footer). Last update [dc64649...7031e25](https://codecov.io/gh/apache/incubator-airflow/pull/3692?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #3704: [AIRFLOW-XXX] Add how to run the docs server to docs
codecov-io commented on issue #3704: [AIRFLOW-XXX] Add how to run the docs server to docs URL: https://github.com/apache/incubator-airflow/pull/3704#issuecomment-410581096 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3704?src=pr=h1) Report > Merging [#3704](https://codecov.io/gh/apache/incubator-airflow/pull/3704?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/dc64649d98f9c6ddb508b49b52a61fc7799e4343?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3704?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3704 +/- ## == - Coverage 77.56% 77.55% -0.01% == Files 204 204 Lines 1576615766 == - Hits1222912228 -1 - Misses 3537 3538 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/3704?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3704/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `88.54% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3704?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3704?src=pr=footer). Last update [dc64649...6915b40](https://codecov.io/gh/apache/incubator-airflow/pull/3704?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2858) Make tox.ini indentation and whitespace consistent
[
https://issues.apache.org/jira/browse/AIRFLOW-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569694#comment-16569694
]
ASF GitHub Bot commented on AIRFLOW-2858:
-
tedmiston opened a new pull request #3705: [AIRFLOW-2858] Make tox.ini
indentation and whitespace consistent
URL: https://github.com/apache/incubator-airflow/pull/3705
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2858
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
The tox.ini file currently mixes multiple indentation and whitespace styles.
This PR makes tox.ini consistently use 4-space indentation to match Python
code and the [examples in the official Tox
docs](https://tox.readthedocs.io/en/latest/example/basic.html).
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Verified by successful run of tox command on CI.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Make tox.ini indentation and whitespace consistent
> --
>
> Key: AIRFLOW-2858
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2858
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ci, tests
>Reporter: Taylor Edmiston
>Assignee: Taylor Edmiston
>Priority: Minor
>
> The tox.ini file currently mixes multiple indentation and whitespace styles.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] tedmiston opened a new pull request #3705: [AIRFLOW-2858] Make tox.ini indentation and whitespace consistent
tedmiston opened a new pull request #3705: [AIRFLOW-2858] Make tox.ini
indentation and whitespace consistent
URL: https://github.com/apache/incubator-airflow/pull/3705
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2858
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
The tox.ini file currently mixes multiple indentation and whitespace styles.
This PR makes tox.ini consistently use 4-space indentation to match Python
code and the [examples in the official Tox
docs](https://tox.readthedocs.io/en/latest/example/basic.html).
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Verified by successful run of tox command on CI.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-2858) Make tox.ini indentation and whitespace consistent
Taylor Edmiston created AIRFLOW-2858: Summary: Make tox.ini indentation and whitespace consistent Key: AIRFLOW-2858 URL: https://issues.apache.org/jira/browse/AIRFLOW-2858 Project: Apache Airflow Issue Type: Improvement Components: ci, tests Reporter: Taylor Edmiston Assignee: Taylor Edmiston The tox.ini file currently mixes multiple indentation and whitespace styles. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2857) Fix verify_gpl_dependency for Read the Docs
[ https://issues.apache.org/jira/browse/AIRFLOW-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taylor Edmiston updated AIRFLOW-2857: - Component/s: docs ci > Fix verify_gpl_dependency for Read the Docs > --- > > Key: AIRFLOW-2857 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2857 > Project: Apache Airflow > Issue Type: Bug > Components: ci, docs >Reporter: Taylor Edmiston >Assignee: Taylor Edmiston >Priority: Major > > Per the mailing list thread "Readthedocs is Broken" on Aug 5, 2018, the Read > the Docs build process is currently broken per the recent addition of > [https://github.com/apache/incubator-airflow/pull/3660|https://github.com/apache/incubator-airflow/pull/3660.]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] codecov-io edited a comment on issue #3703: [AIRFLOW-2857] Fix verify_gpl_dependency for Read the Docs
codecov-io edited a comment on issue #3703: [AIRFLOW-2857] Fix verify_gpl_dependency for Read the Docs URL: https://github.com/apache/incubator-airflow/pull/3703#issuecomment-410578804 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=h1) Report > Merging [#3703](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/dc64649d98f9c6ddb508b49b52a61fc7799e4343?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3703 +/- ## === Coverage 77.56% 77.56% === Files 204 204 Lines 1576615766 === Hits1222912229 Misses 3537 3537 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=footer). Last update [dc64649...1b4961a](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #3703: [AIRFLOW-2857] Fix verify_gpl_dependency for Read the Docs
codecov-io commented on issue #3703: [AIRFLOW-2857] Fix verify_gpl_dependency for Read the Docs URL: https://github.com/apache/incubator-airflow/pull/3703#issuecomment-410578804 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=h1) Report > Merging [#3703](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/dc64649d98f9c6ddb508b49b52a61fc7799e4343?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3703 +/- ## === Coverage 77.56% 77.56% === Files 204 204 Lines 1576615766 === Hits1222912229 Misses 3537 3537 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=footer). Last update [dc64649...1b4961a](https://codecov.io/gh/apache/incubator-airflow/pull/3703?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] holdenk commented on issue #3692: [AIRFLOW-2847] Remove legacy imports support for plugins for 2.0 deprecation.
holdenk commented on issue #3692: [AIRFLOW-2847] Remove legacy imports support for plugins for 2.0 deprecation. URL: https://github.com/apache/incubator-airflow/pull/3692#issuecomment-410577207 Cool, I'm happy to know it's not just me or my env, I'll take a look at that docker JIRA later for tests. Going to rebase this on master since the 2.7 mysql failure in travis seemed unrelated since I didn't touch apt-geting krb This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] holdenk commented on issue #3691: [AIRFLOW-2846] Add missing python test dependency to setup.py
holdenk commented on issue #3691: [AIRFLOW-2846] Add missing python test dependency to setup.py URL: https://github.com/apache/incubator-airflow/pull/3691#issuecomment-410576714 Thank y'all for the review and kind words, I've rebased on master. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tedmiston opened a new pull request #3704: [AIRFLOW-XXX] Add how to run the docs server to docs
tedmiston opened a new pull request #3704: [AIRFLOW-XXX] Add how to run the
docs server to docs
URL: https://github.com/apache/incubator-airflow/pull/3704
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
I noticed that we already have a bash script to serve the docs site for
local testing, however, it's not currently documented. This PR documents it.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
only docs changes
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io commented on issue #3698: [AIRFLOW-2855] Check Cron Expression Validity in DagBag.process_file()
codecov-io commented on issue #3698: [AIRFLOW-2855] Check Cron Expression Validity in DagBag.process_file() URL: https://github.com/apache/incubator-airflow/pull/3698#issuecomment-410575101 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3698?src=pr=h1) Report > Merging [#3698](https://codecov.io/gh/apache/incubator-airflow/pull/3698?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/dc64649d98f9c6ddb508b49b52a61fc7799e4343?src=pr=desc) will **increase** coverage by `0.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3698?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3698 +/- ## == + Coverage 77.56% 77.58% +0.01% == Files 204 204 Lines 1576615780 +14 == + Hits1222912243 +14 Misses 3537 3537 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/3698?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3698/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `88.65% <100%> (+0.06%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3698?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3698?src=pr=footer). Last update [dc64649...68f5f3d](https://codecov.io/gh/apache/incubator-airflow/pull/3698?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tedmiston commented on issue #3703: [AIRFLOW-2857] Fix verify_gpl_dependency for Read the Docs
tedmiston commented on issue #3703: [AIRFLOW-2857] Fix verify_gpl_dependency for Read the Docs URL: https://github.com/apache/incubator-airflow/pull/3703#issuecomment-410575022 /cc @kaxil This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2857) Fix verify_gpl_dependency for Read the Docs
[
https://issues.apache.org/jira/browse/AIRFLOW-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569676#comment-16569676
]
ASF GitHub Bot commented on AIRFLOW-2857:
-
tedmiston opened a new pull request #3703: [AIRFLOW-2857] Fix
verify_gpl_dependency for Read the Docs
URL: https://github.com/apache/incubator-airflow/pull/3703
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2857
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Per the mailing list thread "Readthedocs is Broken" on Aug 5, 2018, the Read
the Docs build process is currently broken per the recent addition of
https://github.com/apache/incubator-airflow/pull/3660.
The [RTD build
environment](http://docs.readthedocs.io/en/latest/builds.html#build-environment)
provides a `READTHEDOCS` env var boolean which it sets to true. I have used
that to toggle on `SLUGIFY_USES_TEXT_UNIDECODE` only when that env var is
supplied to restrict it to that environment.
Related change: The mock module [is used in
conf.py](https://github.com/apache/incubator-airflow/blob/3157287e8c1aba7649cb7de80dd402889725/docs/conf.py#L16)
is supplied to Sphinx to build the docs, but it was missing from the `doc`
install extra in setup.py, so I've added that.
Here's how I tested this:
```
pip install -e .[doc] # (fails)
READTHEDOCS=True pip install -e .[doc] # (succeeds)
cd docs/
./build.sh
./start_doc_server.sh
```
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
A successful build on https://readthedocs.org/projects/airflow/builds/ will
confirm that this is working in the Read the Docs environment.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix verify_gpl_dependency for Read the Docs
> ---
>
> Key: AIRFLOW-2857
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2857
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Taylor Edmiston
>Assignee: Taylor Edmiston
>Priority: Major
>
> Per the mailing list thread "Readthedocs is Broken" on Aug 5, 2018, the Read
> the Docs build process is currently broken per the recent addition of
> [https://github.com/apache/incubator-airflow/pull/3660|https://github.com/apache/incubator-airflow/pull/3660.].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] tedmiston opened a new pull request #3703: [AIRFLOW-2857] Fix verify_gpl_dependency for Read the Docs
tedmiston opened a new pull request #3703: [AIRFLOW-2857] Fix
verify_gpl_dependency for Read the Docs
URL: https://github.com/apache/incubator-airflow/pull/3703
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2857
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Per the mailing list thread "Readthedocs is Broken" on Aug 5, 2018, the Read
the Docs build process is currently broken per the recent addition of
https://github.com/apache/incubator-airflow/pull/3660.
The [RTD build
environment](http://docs.readthedocs.io/en/latest/builds.html#build-environment)
provides a `READTHEDOCS` env var boolean which it sets to true. I have used
that to toggle on `SLUGIFY_USES_TEXT_UNIDECODE` only when that env var is
supplied to restrict it to that environment.
Related change: The mock module [is used in
conf.py](https://github.com/apache/incubator-airflow/blob/3157287e8c1aba7649cb7de80dd402889725/docs/conf.py#L16)
is supplied to Sphinx to build the docs, but it was missing from the `doc`
install extra in setup.py, so I've added that.
Here's how I tested this:
```
pip install -e .[doc] # (fails)
READTHEDOCS=True pip install -e .[doc] # (succeeds)
cd docs/
./build.sh
./start_doc_server.sh
```
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
A successful build on https://readthedocs.org/projects/airflow/builds/ will
confirm that this is working in the Read the Docs environment.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (AIRFLOW-2857) Fix verify_gpl_dependency for Read the Docs
[ https://issues.apache.org/jira/browse/AIRFLOW-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taylor Edmiston updated AIRFLOW-2857: - Description: Per the mailing list thread "Readthedocs is Broken" on Aug 5, 2018, the Read the Docs build process is currently broken per the recent addition of [https://github.com/apache/incubator-airflow/pull/3660|https://github.com/apache/incubator-airflow/pull/3660.]. (was: Per the mailing list thread "Readthedocs is Broken" on Aug 5, 2018, the Read the Docs build process is currently broken per the recent addition of [https://github.com/apache/incubator-airflow/pull/3660|https://github.com/apache/incubator-airflow/pull/3660.]) > Fix verify_gpl_dependency for Read the Docs > --- > > Key: AIRFLOW-2857 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2857 > Project: Apache Airflow > Issue Type: Bug >Reporter: Taylor Edmiston >Assignee: Taylor Edmiston >Priority: Major > > Per the mailing list thread "Readthedocs is Broken" on Aug 5, 2018, the Read > the Docs build process is currently broken per the recent addition of > [https://github.com/apache/incubator-airflow/pull/3660|https://github.com/apache/incubator-airflow/pull/3660.]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2857) Fix verify_gpl_dependency for Read the Docs
Taylor Edmiston created AIRFLOW-2857: Summary: Fix verify_gpl_dependency for Read the Docs Key: AIRFLOW-2857 URL: https://issues.apache.org/jira/browse/AIRFLOW-2857 Project: Apache Airflow Issue Type: Bug Reporter: Taylor Edmiston Assignee: Taylor Edmiston Per the mailing list thread "Readthedocs is Broken" on Aug 5, 2018, the Read the Docs build process is currently broken per the recent addition of [https://github.com/apache/incubator-airflow/pull/3660|https://github.com/apache/incubator-airflow/pull/3660.] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2853) Add maintainers to the README
[ https://issues.apache.org/jira/browse/AIRFLOW-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand resolved AIRFLOW-2853. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request #3699 [https://github.com/apache/incubator-airflow/pull/3699] > Add maintainers to the README > - > > Key: AIRFLOW-2853 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2853 > Project: Apache Airflow > Issue Type: Task >Reporter: Andy Cooper >Assignee: Andy Cooper >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2853) Add maintainers to the README
[ https://issues.apache.org/jira/browse/AIRFLOW-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569626#comment-16569626 ] ASF GitHub Bot commented on AIRFLOW-2853: - r39132 closed pull request #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/README.md b/README.md index bbe17207f7..0f5c71446c 100644 --- a/README.md +++ b/README.md @@ -84,10 +84,6 @@ As the Airflow community grows, we'd like to keep track of who is using the platform. Please send a PR with your company name and @githubhandle if you may. -Committers: - -- Refer to [Committers](https://cwiki.apache.org/confluence/display/AIRFLOW/Committers) - Currently **officially** using Airflow: 1. [6play](https://www.6play.fr) [[@lemourA](https://github.com/lemoura), [@achaussende](https://github.com/achaussende), [@d-nguyen](https://github.com/d-nguyen), [@julien-gm](https://github.com/julien-gm)] @@ -274,6 +270,15 @@ Currently **officially** using Airflow: 1. [Zymergen](https://www.zymergen.com/) 1. [99](https://99taxis.com) [[@fbenevides](https://github.com/fbenevides), [@gustavoamigo](https://github.com/gustavoamigo) & [@mmmaia](https://github.com/mmmaia)] +## Who Maintains Airflow? + +Airflow is the work of the +[community](https://github.com/apache/incubator-airflow/graphs/contributors), +but the [core committers/maintainers](https://people.apache.org/committers-by-project.html#airflow) +are responsible for reviewing and merging PRs as well as steering conversation around new feature requests. +If you would like to become a maintainer, please review the Apache Airflow +[committer requirements](https://cwiki.apache.org/confluence/display/AIRFLOW/Committers). + ## Links - [Documentation](https://airflow.incubator.apache.org/) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add maintainers to the README > - > > Key: AIRFLOW-2853 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2853 > Project: Apache Airflow > Issue Type: Task >Reporter: Andy Cooper >Assignee: Andy Cooper >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] r39132 closed pull request #3699: [AIRFLOW-2853] Add official committers to README
r39132 closed pull request #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/README.md b/README.md index bbe17207f7..0f5c71446c 100644 --- a/README.md +++ b/README.md @@ -84,10 +84,6 @@ As the Airflow community grows, we'd like to keep track of who is using the platform. Please send a PR with your company name and @githubhandle if you may. -Committers: - -- Refer to [Committers](https://cwiki.apache.org/confluence/display/AIRFLOW/Committers) - Currently **officially** using Airflow: 1. [6play](https://www.6play.fr) [[@lemourA](https://github.com/lemoura), [@achaussende](https://github.com/achaussende), [@d-nguyen](https://github.com/d-nguyen), [@julien-gm](https://github.com/julien-gm)] @@ -274,6 +270,15 @@ Currently **officially** using Airflow: 1. [Zymergen](https://www.zymergen.com/) 1. [99](https://99taxis.com) [[@fbenevides](https://github.com/fbenevides), [@gustavoamigo](https://github.com/gustavoamigo) & [@mmmaia](https://github.com/mmmaia)] +## Who Maintains Airflow? + +Airflow is the work of the +[community](https://github.com/apache/incubator-airflow/graphs/contributors), +but the [core committers/maintainers](https://people.apache.org/committers-by-project.html#airflow) +are responsible for reviewing and merging PRs as well as steering conversation around new feature requests. +If you would like to become a maintainer, please review the Apache Airflow +[committer requirements](https://cwiki.apache.org/confluence/display/AIRFLOW/Committers). + ## Links - [Documentation](https://airflow.incubator.apache.org/) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2853) Add maintainers to the README
[ https://issues.apache.org/jira/browse/AIRFLOW-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569625#comment-16569625 ] ASF subversion and git services commented on AIRFLOW-2853: -- Commit dc64649d98f9c6ddb508b49b52a61fc7799e4343 in incubator-airflow's branch refs/heads/master from [~ajc] [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=dc64649 ] [AIRFLOW-2853] Add official committers to README Closes #3699 from andscoop/Add-core-commiters-to- readme > Add maintainers to the README > - > > Key: AIRFLOW-2853 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2853 > Project: Apache Airflow > Issue Type: Task >Reporter: Andy Cooper >Assignee: Andy Cooper >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README
r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410562751 LGTM! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook
codecov-io edited a comment on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook URL: https://github.com/apache/incubator-airflow/pull/3700#issuecomment-410537638 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=h1) Report > Merging [#3700](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/3157287e8c1aba7649cb7de80dd402889725?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3700 +/- ## === Coverage 77.56% 77.56% === Files 204 204 Lines 1576615766 === Hits1222912229 Misses 3537 3537 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=footer). Last update [3157287...809a8a7](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook
codecov-io edited a comment on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook URL: https://github.com/apache/incubator-airflow/pull/3700#issuecomment-410537638 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=h1) Report > Merging [#3700](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/3157287e8c1aba7649cb7de80dd402889725?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3700 +/- ## === Coverage 77.56% 77.56% === Files 204 204 Lines 1576615766 === Hits1222912229 Misses 3537 3537 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=footer). Last update [3157287...809a8a7](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] andscoop edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README
andscoop edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410553138 @r39132 Looks like a couple of these links need [team access to github](https://github.com/orgs/apache/teams/airflow-committers/members) and access to [whimsy server](https://whimsy.apache.org/roster/ppmc/airflow). So we can't put those in the file. The best definitive resource I have seen so far is the one @ashb [linked to in review](https://people.apache.org/committers-by-project.html#airflow). Is this the link we should move forward with? edit - removed list of contributed and updated with list of known working links This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] andscoop commented on issue #3699: [AIRFLOW-2853] Add official committers to README
andscoop commented on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410553138 @r39132 Looks like a couple of these links need [team access to github](https://github.com/orgs/apache/teams/airflow-committers/members) and access to [whimsy server](https://whimsy.apache.org/roster/ppmc/airflow). So we can't put those in the file. The best definitive resource I have seen so far is the one @ashb [linked to in review](https://people.apache.org/committers-by-project.html#airflow). Is this the link we should move forward with? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #3702: [AIRFLOW-81] Add ScheduleBlackoutSensor
codecov-io commented on issue #3702: [AIRFLOW-81] Add ScheduleBlackoutSensor URL: https://github.com/apache/incubator-airflow/pull/3702#issuecomment-410552388 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3702?src=pr=h1) Report > Merging [#3702](https://codecov.io/gh/apache/incubator-airflow/pull/3702?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/3157287e8c1aba7649cb7de80dd402889725?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3702?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3702 +/- ## === Coverage 77.56% 77.56% === Files 204 204 Lines 1576615766 === Hits1222912229 Misses 3537 3537 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3702?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3702?src=pr=footer). Last update [3157287...8d2e1e1](https://codecov.io/gh/apache/incubator-airflow/pull/3702?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] r39132 commented on issue #3702: [AIRFLOW-81] Add ScheduleBlackoutSensor
r39132 commented on issue #3702: [AIRFLOW-81] Add ScheduleBlackoutSensor URL: https://github.com/apache/incubator-airflow/pull/3702#issuecomment-410552307 @criccomini What do you think of this? There is definitely a need for this type of functionality (e.g. scheduled maintenance windows). Is a sensor the way to go? Won't we be blocking celery workers? CC @bolkedebruin @Fokko This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] r39132 commented on issue #3702: [AIRFLOW-81] Add ScheduleBlackoutSensor
r39132 commented on issue #3702: [AIRFLOW-81] Add ScheduleBlackoutSensor URL: https://github.com/apache/incubator-airflow/pull/3702#issuecomment-410552112 @andscoop In your PRs, please replace `AIRFLOW-XXX` with the actual Jira issue this PR addresses, which is `AIRFLOW-81` in this case. This way, we can link a PR to a JIRA. Pls update your other Jiras accordingly. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] r39132 edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README
r39132 edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410550116 @andscoop I granted you access... though I see in your PR, that you do still link to the cwiki. You may not have much to update in the wiki itself. I do like the "Who maintains Airflow" section that you added. I would be open to merging this PR if you didn't have the explicit list of committers listed .. we have enough PRs to merge as it is and there a quite a few steps involved in promoting a contributor to committer status -- I'd be happy not to add yet another step to it. That said, you could add a link to https://github.com/orgs/apache/teams/airflow-committers/members on the read me so people could see the list of committers in a single hop. Would this work for you? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] r39132 edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README
r39132 edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410550116 @andscoop I granted you access... please update the wiki. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README
r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410550116 @andscoop I graned you access... please update the wiki. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-81) Scheduler blackout time period
[
https://issues.apache.org/jira/browse/AIRFLOW-81?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569586#comment-16569586
]
ASF GitHub Bot commented on AIRFLOW-81:
---
andscoop opened a new pull request #3702: [AIRFLOW-81] add
ScheduleBlackoutSensor
URL: https://github.com/apache/incubator-airflow/pull/3702
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
After reviewing some of the older jira issues, I found one that could be
simply solved via a sensorOperator as Chris suggested in the comments. This
operator and test should be good enough to close out an ancient issue that
hasn't gotten a lot of traction
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Adds three test which validate the functionality of sensor properly
returning true, false and error handling.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Scheduler blackout time period
> --
>
> Key: AIRFLOW-81
> URL: https://issues.apache.org/jira/browse/AIRFLOW-81
> Project: Apache Airflow
> Issue Type: Wish
> Components: scheduler
>Reporter: Sean McIntyre
>Assignee: Andy Cooper
>Priority: Minor
> Labels: features
>
> I have the need for a scheduler blackout time period in Airflow.
> My team, which uses Airflow, has been asked to not query one of my company's
> data sources between midnight and 7 AM. When we launch big backfills on this
> data source, it would be nice to have the Scheduler not schedule some
> TaskInstances during the blackout hours.
> We (@r39132 and @ledsusop) brainstormed a few ideas on gitter on how to do
> this...
> (1) Put more state/logic in the TaskInstance and Scheduler like this:
> my_task = PythonOperator(
> task_id='my_task',
> python_callable=my_command_that_access_the_datasource,
> provide_context=True,
> dag=dag,
> blackout=my_blackout_logic_for_the_datasource # <---
> )
> where my_blackout_logic is some function I provide that the scheduler calls
> to determine whether or not it is the blackout period.
> (2) Pause DAGs on nightly basis. This can be done with the `pause_dag` CLI
> command scheduled by cron / Jenkins. However could this be considered a core
> feature to bring into the Airflow UI and scheduling system?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] andscoop opened a new pull request #3702: [AIRFLOW-81] add ScheduleBlackoutSensor
andscoop opened a new pull request #3702: [AIRFLOW-81] add
ScheduleBlackoutSensor
URL: https://github.com/apache/incubator-airflow/pull/3702
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
After reviewing some of the older jira issues, I found one that could be
simply solved via a sensorOperator as Chris suggested in the comments. This
operator and test should be good enough to close out an ancient issue that
hasn't gotten a lot of traction
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Adds three test which validate the functionality of sensor properly
returning true, false and error handling.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] andscoop commented on issue #3699: [AIRFLOW-2853] Add official committers to README
andscoop commented on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410549554 @r39132 please try again - just officially signed up for the cwiki which is apparently different than the jira signup. I understand the concern with maintaining this is 3 different places. Main reason for the PR is IMO that docs which don't live with the code get left behind. I will gladly update the wiki if we choose not to move forward with the PR. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #3701: [AIRFLOW-2856] Pass in SLUGIFY_USES_TEXT_UNIDECODE=yes ENV to docker run
codecov-io commented on issue #3701: [AIRFLOW-2856] Pass in SLUGIFY_USES_TEXT_UNIDECODE=yes ENV to docker run URL: https://github.com/apache/incubator-airflow/pull/3701#issuecomment-410549010 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3701?src=pr=h1) Report > Merging [#3701](https://codecov.io/gh/apache/incubator-airflow/pull/3701?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/3157287e8c1aba7649cb7de80dd402889725?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3701?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3701 +/- ## == - Coverage 77.56% 77.55% -0.01% == Files 204 204 Lines 1576615766 == - Hits1222912228 -1 - Misses 3537 3538 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/3701?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3701/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `88.54% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3701?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3701?src=pr=footer). Last update [3157287...fab2e7e](https://codecov.io/gh/apache/incubator-airflow/pull/3701?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] r39132 commented on issue #3656: [WIP][AIRFLOW-2803] Fix all ESLint issues
r39132 commented on issue #3656: [WIP][AIRFLOW-2803] Fix all ESLint issues URL: https://github.com/apache/incubator-airflow/pull/3656#issuecomment-410548883 Any updates on this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] bolkedebruin commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook
bolkedebruin commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook URL: https://github.com/apache/incubator-airflow/pull/3700#issuecomment-410548047 @kaxil @Fokko what do you guys think? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README
r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410547561 I do like what you have written though and would love you to alter the cwiki page. I was trying to add you an admin to the cwiki, but could not find you for some reason. I tried to add andscoop and Andy Cooper. if you go by some other name, do let me know. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README
r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410547447 Thx @andscoop. I agree we are not there yet. Ok, so I sent a note to the dev list that we have 3 places currently where the committer list is maintained : https://lists.apache.org/thread.html/15cf9c3f0d45a08f988fe9f3ff0fa6092ef8a8655e6087177fbc4a51@%3Cdev.airflow.apache.org%3E Merging this PR would require yet another list to be updated. This is why I originally linked via the cwiki. If you look at the cwiki right now, I did add all 3 places where committer lists are maintained. Does this satisfy your need? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2856) Update Docker Env Setup Docs to Account for verify_gpl_dependency()
[
https://issues.apache.org/jira/browse/AIRFLOW-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569574#comment-16569574
]
ASF GitHub Bot commented on AIRFLOW-2856:
-
andscoop opened a new pull request #3701: [AIRFLOW-2856] Pass in
SLUGIFY_USES_TEXT_UNIDECODE=yes ENV to docker run
URL: https://github.com/apache/incubator-airflow/pull/3701
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Setting up a local docker test environment was breaking due to the
additional of the
[verify_gpl_dependency](https://github.com/apache/incubator-airflow/blob/master/setup.py#L39
) in setup.py. This PR fixes that by passing in the
SLUGIFY_USES_TEXT_UNIDECODE=yes env to `docker run`
It also now uses the -w arg on `docker run` in order to put user directly
into /airflow dir. This removes the need to `cd` into that directory in the
next step.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Who will test the tests?
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Update Docker Env Setup Docs to Account for verify_gpl_dependency()
> ---
>
> Key: AIRFLOW-2856
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2856
> Project: Apache Airflow
> Issue Type: Improvement
>Reporter: Andy Cooper
>Assignee: Andy Cooper
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] andscoop opened a new pull request #3701: [AIRFLOW-2856] Pass in SLUGIFY_USES_TEXT_UNIDECODE=yes ENV to docker run
andscoop opened a new pull request #3701: [AIRFLOW-2856] Pass in
SLUGIFY_USES_TEXT_UNIDECODE=yes ENV to docker run
URL: https://github.com/apache/incubator-airflow/pull/3701
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Setting up a local docker test environment was breaking due to the
additional of the
[verify_gpl_dependency](https://github.com/apache/incubator-airflow/blob/master/setup.py#L39
) in setup.py. This PR fixes that by passing in the
SLUGIFY_USES_TEXT_UNIDECODE=yes env to `docker run`
It also now uses the -w arg on `docker run` in order to put user directly
into /airflow dir. This removes the need to `cd` into that directory in the
next step.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Who will test the tests?
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ashb commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook
ashb commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook URL: https://github.com/apache/incubator-airflow/pull/3700#issuecomment-410545540 @bolkedebruin Updated as you suggested This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] andscoop commented on issue #3699: [AIRFLOW-2853] Add official committers to README
andscoop commented on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410545427 @r39132 I did not originally research thoroughly. It would require us to set owners to various sections of the codebase so that those owners are prompted for automatic review when a PR for that section is submitted. CODEOWNERS may be the direction the project wants to go eventually - but I don't believe it (the project) currently goes into the level of detail of accepting committers for specific sections of the codebase. While some committers may be more inclined to review PRs from a specific section, from the outside looking in, it appears that most committers/maintainers float across various sections of the codebase. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README
r39132 commented on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410545007 @andscoop In your Jira, you referenced creating a CODEOWNERS files. Why is that not the way to proceed? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ashb commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook
ashb commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook URL: https://github.com/apache/incubator-airflow/pull/3700#issuecomment-410544687 Confirmed, no other occurances in hooks, and airflow/contrib/operators/kubernetes_pod_operator.py loads via `airflow.contrib.kubernetes` already. Yes, doing via a.c.k8s makes sense actually. Will change over. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2852) Can't start GUI program in regular run
[ https://issues.apache.org/jira/browse/AIRFLOW-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569564#comment-16569564 ] Jianyang Zhao commented on AIRFLOW-2852: Need to specify Environment=DISPLAY=localhost:0.0 which is not needed for test > Can't start GUI program in regular run > -- > > Key: AIRFLOW-2852 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2852 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Jianyang Zhao >Priority: Major > > I tried to use IBC ([https://github.com/IbcAlpha/IBC)] within a ib_insync > ([https://github.com/erdewit/ib_insync] > https://github.com/erdewit/ib_insync/blob/master/ib_insync/ibcontroller.py) > script. It will start a java program with GUI. I tried both PythonOperator > and BashOperator. However, everything works fine with test, but never shows > the GUI program in schedule run / command line run / command line backfill. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (AIRFLOW-2852) Can't start GUI program in regular run
[ https://issues.apache.org/jira/browse/AIRFLOW-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianyang Zhao closed AIRFLOW-2852. -- Resolution: Not A Problem > Can't start GUI program in regular run > -- > > Key: AIRFLOW-2852 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2852 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Jianyang Zhao >Priority: Major > > I tried to use IBC ([https://github.com/IbcAlpha/IBC)] within a ib_insync > ([https://github.com/erdewit/ib_insync] > https://github.com/erdewit/ib_insync/blob/master/ib_insync/ibcontroller.py) > script. It will start a java program with GUI. I tried both PythonOperator > and BashOperator. However, everything works fine with test, but never shows > the GUI program in schedule run / command line run / command line backfill. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2856) Update Docker Env Setup Docs to Account for verify_gpl_dependency()
Andy Cooper created AIRFLOW-2856: Summary: Update Docker Env Setup Docs to Account for verify_gpl_dependency() Key: AIRFLOW-2856 URL: https://issues.apache.org/jira/browse/AIRFLOW-2856 Project: Apache Airflow Issue Type: Improvement Reporter: Andy Cooper Assignee: Andy Cooper -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2821) Doc "Plugins" can be improved
[ https://issues.apache.org/jira/browse/AIRFLOW-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569539#comment-16569539 ] ASF subversion and git services commented on AIRFLOW-2821: -- Commit 3157287e8c1aba7649cb7de80dd402889725 in incubator-airflow's branch refs/heads/master from Xiaodong [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=3157287 ] [AIRFLOW-2821] Refine Doc "Plugins" (#3664) > Doc "Plugins" can be improved > - > > Key: AIRFLOW-2821 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2821 > Project: Apache Airflow > Issue Type: Improvement > Components: Documentation >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > The doc "Plugins" is not very clear in some sense. > Something that users may need to take note is not mentioned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2821) Doc "Plugins" can be improved
[ https://issues.apache.org/jira/browse/AIRFLOW-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-2821. - Resolution: Fixed Resolved by https://github.com/apache/incubator-airflow/pull/3664 > Doc "Plugins" can be improved > - > > Key: AIRFLOW-2821 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2821 > Project: Apache Airflow > Issue Type: Improvement > Components: Documentation >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > The doc "Plugins" is not very clear in some sense. > Something that users may need to take note is not mentioned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2821) Doc "Plugins" can be improved
[
https://issues.apache.org/jira/browse/AIRFLOW-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569538#comment-16569538
]
ASF GitHub Bot commented on AIRFLOW-2821:
-
kaxil closed pull request #3664: [AIRFLOW-2821] Refine Doc "Plugins"
URL: https://github.com/apache/incubator-airflow/pull/3664
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/plugins.rst b/docs/plugins.rst
index a21f1007ff..3f1f7ee804 100644
--- a/docs/plugins.rst
+++ b/docs/plugins.rst
@@ -78,6 +78,26 @@ looks like:
menu_links = []
+You can derive it by inheritance (please refer to the example below).
+Please note ``name`` inside this class must be specified.
+
+After the plugin is imported into Airflow,
+you can invoke it using statement like
+
+
+.. code:: python
+
+from airflow.{type, like "operators", "sensors"}.{name specificed inside
the plugin class} import *
+
+
+When you write your own plugins, make sure you understand them well.
+There are some essential properties for each type of plugin.
+For example,
+
+* For ``Operator`` plugin, an ``execute`` method is compulsory.
+* For ``Sensor`` plugin, a ``poke`` method returning a Boolean value is
compulsory.
+
+
Example
---
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Doc "Plugins" can be improved
> -
>
> Key: AIRFLOW-2821
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2821
> Project: Apache Airflow
> Issue Type: Improvement
> Components: Documentation
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Minor
>
> The doc "Plugins" is not very clear in some sense.
> Something that users may need to take note is not mentioned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] kaxil closed pull request #3664: [AIRFLOW-2821] Refine Doc "Plugins"
kaxil closed pull request #3664: [AIRFLOW-2821] Refine Doc "Plugins"
URL: https://github.com/apache/incubator-airflow/pull/3664
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/plugins.rst b/docs/plugins.rst
index a21f1007ff..3f1f7ee804 100644
--- a/docs/plugins.rst
+++ b/docs/plugins.rst
@@ -78,6 +78,26 @@ looks like:
menu_links = []
+You can derive it by inheritance (please refer to the example below).
+Please note ``name`` inside this class must be specified.
+
+After the plugin is imported into Airflow,
+you can invoke it using statement like
+
+
+.. code:: python
+
+from airflow.{type, like "operators", "sensors"}.{name specificed inside
the plugin class} import *
+
+
+When you write your own plugins, make sure you understand them well.
+There are some essential properties for each type of plugin.
+For example,
+
+* For ``Operator`` plugin, an ``execute`` method is compulsory.
+* For ``Sensor`` plugin, a ``poke`` method returning a Boolean value is
compulsory.
+
+
Example
---
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] bolkedebruin commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook
bolkedebruin commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook URL: https://github.com/apache/incubator-airflow/pull/3700#issuecomment-410538108 Can you verify other hooks / operators as well? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README
codecov-io edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410536262 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=h1) Report > Merging [#3699](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/87013d7f290bbed0a3028e10b6d5ac51061b7cb3?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3699 +/- ## === Coverage 77.57% 77.57% === Files 205 205 Lines 1577215772 === Hits1223512235 Misses 3537 3537 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=footer). Last update [87013d7...0960662](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2850) Remove deprecated airflow.utils.apply_defaults
[ https://issues.apache.org/jira/browse/AIRFLOW-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569535#comment-16569535 ] ASF GitHub Bot commented on AIRFLOW-2850: - kaxil closed pull request #3695: [AIRFLOW-2850] Remove deprecated airflow.utils.apply_defaults URL: https://github.com/apache/incubator-airflow/pull/3695 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/airflow/contrib/operators/awsbatch_operator.py b/airflow/contrib/operators/awsbatch_operator.py index 353fbbb0a0..50c6c2c319 100644 --- a/airflow/contrib/operators/awsbatch_operator.py +++ b/airflow/contrib/operators/awsbatch_operator.py @@ -24,7 +24,7 @@ from airflow.exceptions import AirflowException from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.contrib.hooks.aws_hook import AwsHook diff --git a/airflow/contrib/operators/ecs_operator.py b/airflow/contrib/operators/ecs_operator.py index 60540f5e28..319441d297 100644 --- a/airflow/contrib/operators/ecs_operator.py +++ b/airflow/contrib/operators/ecs_operator.py @@ -20,7 +20,7 @@ from airflow.exceptions import AirflowException from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.contrib.hooks.aws_hook import AwsHook diff --git a/airflow/contrib/operators/emr_add_steps_operator.py b/airflow/contrib/operators/emr_add_steps_operator.py index 643ffe9c1b..2f2f4afcec 100644 --- a/airflow/contrib/operators/emr_add_steps_operator.py +++ b/airflow/contrib/operators/emr_add_steps_operator.py @@ -17,7 +17,7 @@ # specific language governing permissions and limitations # under the License. from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException from airflow.contrib.hooks.emr_hook import EmrHook diff --git a/airflow/contrib/operators/emr_create_job_flow_operator.py b/airflow/contrib/operators/emr_create_job_flow_operator.py index 89be12f065..1e0dfc386f 100644 --- a/airflow/contrib/operators/emr_create_job_flow_operator.py +++ b/airflow/contrib/operators/emr_create_job_flow_operator.py @@ -18,7 +18,7 @@ # under the License. from airflow.contrib.hooks.emr_hook import EmrHook from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException diff --git a/airflow/contrib/operators/emr_terminate_job_flow_operator.py b/airflow/contrib/operators/emr_terminate_job_flow_operator.py index 50407a14ac..d265733842 100644 --- a/airflow/contrib/operators/emr_terminate_job_flow_operator.py +++ b/airflow/contrib/operators/emr_terminate_job_flow_operator.py @@ -17,7 +17,7 @@ # specific language governing permissions and limitations # under the License. from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException from airflow.contrib.hooks.emr_hook import EmrHook diff --git a/airflow/contrib/sensors/datadog_sensor.py b/airflow/contrib/sensors/datadog_sensor.py index fb455d6cd7..817fee061a 100644 --- a/airflow/contrib/sensors/datadog_sensor.py +++ b/airflow/contrib/sensors/datadog_sensor.py @@ -18,7 +18,7 @@ # under the License. from airflow.sensors.base_sensor_operator import BaseSensorOperator from airflow.contrib.hooks.datadog_hook import DatadogHook -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException from datadog import api diff --git a/airflow/contrib/sensors/emr_base_sensor.py b/airflow/contrib/sensors/emr_base_sensor.py index abeb421b9d..6dcb9fc5f3 100644 --- a/airflow/contrib/sensors/emr_base_sensor.py +++ b/airflow/contrib/sensors/emr_base_sensor.py @@ -17,7 +17,7 @@ # specific language governing permissions and limitations # under the License. from airflow.sensors.base_sensor_operator import BaseSensorOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException diff --git a/airflow/contrib/sensors/emr_job_flow_sensor.py b/airflow/contrib/sensors/emr_job_flow_sensor.py index 31d16a066b..806b63bda3 100644 --- a/airflow/contrib/sensors/emr_job_flow_sensor.py +++ b/airflow/contrib/sensors/emr_job_flow_sensor.py @@ -18,7 +18,7 @@ # under the License. from airflow.contrib.hooks.emr_hook import EmrHook from
[jira] [Commented] (AIRFLOW-2850) Remove deprecated airflow.utils.apply_defaults
[ https://issues.apache.org/jira/browse/AIRFLOW-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569536#comment-16569536 ] ASF subversion and git services commented on AIRFLOW-2850: -- Commit b0e912cea90b63520a8e354e9dcc27471c50a738 in incubator-airflow's branch refs/heads/master from Taylor D. Edmiston [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=b0e912c ] [AIRFLOW-2850] Remove deprecated airflow.utils.apply_defaults (#3695) > Remove deprecated airflow.utils.apply_defaults > -- > > Key: AIRFLOW-2850 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2850 > Project: Apache Airflow > Issue Type: Improvement > Components: utils >Affects Versions: 2.0.0 >Reporter: Taylor Edmiston >Assignee: Taylor Edmiston >Priority: Minor > > This PR removes the wrapper function apply_defaults that's had a deprecation > warning since 2016. As similar "to be deprecated" stuff is removed for 2.0 > in #3692 ([AIRFLOW-2847]), this felt like a good time to take care of related > things. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2850) Remove deprecated airflow.utils.apply_defaults
[ https://issues.apache.org/jira/browse/AIRFLOW-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-2850. - Resolution: Fixed Resolved by https://github.com/apache/incubator-airflow/pull/3695 > Remove deprecated airflow.utils.apply_defaults > -- > > Key: AIRFLOW-2850 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2850 > Project: Apache Airflow > Issue Type: Improvement > Components: utils >Affects Versions: 2.0.0 >Reporter: Taylor Edmiston >Assignee: Taylor Edmiston >Priority: Minor > > This PR removes the wrapper function apply_defaults that's had a deprecation > warning since 2016. As similar "to be deprecated" stuff is removed for 2.0 > in #3692 ([AIRFLOW-2847]), this felt like a good time to take care of related > things. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] kaxil closed pull request #3695: [AIRFLOW-2850] Remove deprecated airflow.utils.apply_defaults
kaxil closed pull request #3695: [AIRFLOW-2850] Remove deprecated airflow.utils.apply_defaults URL: https://github.com/apache/incubator-airflow/pull/3695 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/airflow/contrib/operators/awsbatch_operator.py b/airflow/contrib/operators/awsbatch_operator.py index 353fbbb0a0..50c6c2c319 100644 --- a/airflow/contrib/operators/awsbatch_operator.py +++ b/airflow/contrib/operators/awsbatch_operator.py @@ -24,7 +24,7 @@ from airflow.exceptions import AirflowException from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.contrib.hooks.aws_hook import AwsHook diff --git a/airflow/contrib/operators/ecs_operator.py b/airflow/contrib/operators/ecs_operator.py index 60540f5e28..319441d297 100644 --- a/airflow/contrib/operators/ecs_operator.py +++ b/airflow/contrib/operators/ecs_operator.py @@ -20,7 +20,7 @@ from airflow.exceptions import AirflowException from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.contrib.hooks.aws_hook import AwsHook diff --git a/airflow/contrib/operators/emr_add_steps_operator.py b/airflow/contrib/operators/emr_add_steps_operator.py index 643ffe9c1b..2f2f4afcec 100644 --- a/airflow/contrib/operators/emr_add_steps_operator.py +++ b/airflow/contrib/operators/emr_add_steps_operator.py @@ -17,7 +17,7 @@ # specific language governing permissions and limitations # under the License. from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException from airflow.contrib.hooks.emr_hook import EmrHook diff --git a/airflow/contrib/operators/emr_create_job_flow_operator.py b/airflow/contrib/operators/emr_create_job_flow_operator.py index 89be12f065..1e0dfc386f 100644 --- a/airflow/contrib/operators/emr_create_job_flow_operator.py +++ b/airflow/contrib/operators/emr_create_job_flow_operator.py @@ -18,7 +18,7 @@ # under the License. from airflow.contrib.hooks.emr_hook import EmrHook from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException diff --git a/airflow/contrib/operators/emr_terminate_job_flow_operator.py b/airflow/contrib/operators/emr_terminate_job_flow_operator.py index 50407a14ac..d265733842 100644 --- a/airflow/contrib/operators/emr_terminate_job_flow_operator.py +++ b/airflow/contrib/operators/emr_terminate_job_flow_operator.py @@ -17,7 +17,7 @@ # specific language governing permissions and limitations # under the License. from airflow.models import BaseOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException from airflow.contrib.hooks.emr_hook import EmrHook diff --git a/airflow/contrib/sensors/datadog_sensor.py b/airflow/contrib/sensors/datadog_sensor.py index fb455d6cd7..817fee061a 100644 --- a/airflow/contrib/sensors/datadog_sensor.py +++ b/airflow/contrib/sensors/datadog_sensor.py @@ -18,7 +18,7 @@ # under the License. from airflow.sensors.base_sensor_operator import BaseSensorOperator from airflow.contrib.hooks.datadog_hook import DatadogHook -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException from datadog import api diff --git a/airflow/contrib/sensors/emr_base_sensor.py b/airflow/contrib/sensors/emr_base_sensor.py index abeb421b9d..6dcb9fc5f3 100644 --- a/airflow/contrib/sensors/emr_base_sensor.py +++ b/airflow/contrib/sensors/emr_base_sensor.py @@ -17,7 +17,7 @@ # specific language governing permissions and limitations # under the License. from airflow.sensors.base_sensor_operator import BaseSensorOperator -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults from airflow.exceptions import AirflowException diff --git a/airflow/contrib/sensors/emr_job_flow_sensor.py b/airflow/contrib/sensors/emr_job_flow_sensor.py index 31d16a066b..806b63bda3 100644 --- a/airflow/contrib/sensors/emr_job_flow_sensor.py +++ b/airflow/contrib/sensors/emr_job_flow_sensor.py @@ -18,7 +18,7 @@ # under the License. from airflow.contrib.hooks.emr_hook import EmrHook from airflow.contrib.sensors.emr_base_sensor import EmrBaseSensor -from airflow.utils import apply_defaults +from airflow.utils.decorators import apply_defaults class EmrJobFlowSensor(EmrBaseSensor): diff --git
[GitHub] codecov-io commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook
codecov-io commented on issue #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook URL: https://github.com/apache/incubator-airflow/pull/3700#issuecomment-410537638 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=h1) Report > Merging [#3700](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/87013d7f290bbed0a3028e10b6d5ac51061b7cb3?src=pr=desc) will **decrease** coverage by `0.02%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3700 +/- ## == - Coverage 77.57% 77.54% -0.03% == Files 205 205 Lines 1577215772 == - Hits1223512231 -4 - Misses 3537 3541 +4 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3700/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5) | `82.49% <0%> (-0.27%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3700/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `88.54% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=footer). Last update [87013d7...dbebac2](https://codecov.io/gh/apache/incubator-airflow/pull/3700?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2839) Answer a question in doc: what if I have connections with the same conn_id in both metadata and environment variables?
[ https://issues.apache.org/jira/browse/AIRFLOW-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569532#comment-16569532 ] ASF GitHub Bot commented on AIRFLOW-2839: - kaxil closed pull request #3678: [AIRFLOW-2839] Refine Doc Concepts->Connections URL: https://github.com/apache/incubator-airflow/pull/3678 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/docs/concepts.rst b/docs/concepts.rst index f879ced012..09c1293806 100644 --- a/docs/concepts.rst +++ b/docs/concepts.rst @@ -333,6 +333,17 @@ from ``BaseHook``, Airflow will choose one connection randomly, allowing for some basic load balancing and fault tolerance when used in conjunction with retries. +Airflow also has the ability to reference connections via environment +variables from the operating system. But it only supports URI format. If you +need to specify ``extra`` for your connection, please use web UI. + +If connections with the same ``conn_id`` are defined in both Airflow metadata +database and environment variables, only the one in environment variables +will be referenced by Airflow (for example, given ``conn_id`` ``postgres_master``, +Airflow will search for ``AIRFLOW_CONN_POSTGRES_MASTER`` +in environment variables first and directly reference it if found, +before it starts to search in metadata database). + Many hooks have a default ``conn_id``, where operators using that hook do not need to supply an explicit connection ID. For example, the default ``conn_id`` for the :class:`~airflow.hooks.postgres_hook.PostgresHook` is This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Answer a question in doc: what if I have connections with the same conn_id in > both metadata and environment variables? > -- > > Key: AIRFLOW-2839 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2839 > Project: Apache Airflow > Issue Type: Improvement > Components: configuration, Documentation >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Critical > Fix For: 2.0.0 > > > We can define connections either in metadata database or environment > variables. > > However, how about there are connections with the same conn_id in both > metadata database and environment variables? Will one of them be picked? > Which one? Or a random one will be picked? > > The answer is: Airflow will check environment variables > first.[https://github.com/apache/incubator-airflow/blob/master/airflow/hooks/base_hook.py#L70] > The "random picking" mechanism will only be applicable for multiple > connections with the same conn_id in metadata database (if there is no > connection with the same conn_id defined in environment variables). > > This should be made clear in the documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-2839) Answer a question in doc: what if I have connections with the same conn_id in both metadata and environment variables?
[ https://issues.apache.org/jira/browse/AIRFLOW-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-2839. - Resolution: Fixed Fix Version/s: 2.0.0 Resolved by https://github.com/apache/incubator-airflow/pull/3678 > Answer a question in doc: what if I have connections with the same conn_id in > both metadata and environment variables? > -- > > Key: AIRFLOW-2839 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2839 > Project: Apache Airflow > Issue Type: Improvement > Components: configuration, Documentation >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Critical > Fix For: 2.0.0 > > > We can define connections either in metadata database or environment > variables. > > However, how about there are connections with the same conn_id in both > metadata database and environment variables? Will one of them be picked? > Which one? Or a random one will be picked? > > The answer is: Airflow will check environment variables > first.[https://github.com/apache/incubator-airflow/blob/master/airflow/hooks/base_hook.py#L70] > The "random picking" mechanism will only be applicable for multiple > connections with the same conn_id in metadata database (if there is no > connection with the same conn_id defined in environment variables). > > This should be made clear in the documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2839) Answer a question in doc: what if I have connections with the same conn_id in both metadata and environment variables?
[ https://issues.apache.org/jira/browse/AIRFLOW-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569533#comment-16569533 ] ASF subversion and git services commented on AIRFLOW-2839: -- Commit c0c63ae2a495b8f808c36acc35d752e916393954 in incubator-airflow's branch refs/heads/master from Xiaodong [ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=c0c63ae ] [AIRFLOW-2839] Refine Doc Concepts->Connections (#3678) > Answer a question in doc: what if I have connections with the same conn_id in > both metadata and environment variables? > -- > > Key: AIRFLOW-2839 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2839 > Project: Apache Airflow > Issue Type: Improvement > Components: configuration, Documentation >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Critical > Fix For: 2.0.0 > > > We can define connections either in metadata database or environment > variables. > > However, how about there are connections with the same conn_id in both > metadata database and environment variables? Will one of them be picked? > Which one? Or a random one will be picked? > > The answer is: Airflow will check environment variables > first.[https://github.com/apache/incubator-airflow/blob/master/airflow/hooks/base_hook.py#L70] > The "random picking" mechanism will only be applicable for multiple > connections with the same conn_id in metadata database (if there is no > connection with the same conn_id defined in environment variables). > > This should be made clear in the documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] kaxil closed pull request #3678: [AIRFLOW-2839] Refine Doc Concepts->Connections
kaxil closed pull request #3678: [AIRFLOW-2839] Refine Doc Concepts->Connections URL: https://github.com/apache/incubator-airflow/pull/3678 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/docs/concepts.rst b/docs/concepts.rst index f879ced012..09c1293806 100644 --- a/docs/concepts.rst +++ b/docs/concepts.rst @@ -333,6 +333,17 @@ from ``BaseHook``, Airflow will choose one connection randomly, allowing for some basic load balancing and fault tolerance when used in conjunction with retries. +Airflow also has the ability to reference connections via environment +variables from the operating system. But it only supports URI format. If you +need to specify ``extra`` for your connection, please use web UI. + +If connections with the same ``conn_id`` are defined in both Airflow metadata +database and environment variables, only the one in environment variables +will be referenced by Airflow (for example, given ``conn_id`` ``postgres_master``, +Airflow will search for ``AIRFLOW_CONN_POSTGRES_MASTER`` +in environment variables first and directly reference it if found, +before it starts to search in metadata database). + Many hooks have a default ``conn_id``, where operators using that hook do not need to supply an explicit connection ID. For example, the default ``conn_id`` for the :class:`~airflow.hooks.postgres_hook.PostgresHook` is This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README
codecov-io edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410536262 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=h1) Report > Merging [#3699](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/87013d7f290bbed0a3028e10b6d5ac51061b7cb3?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3699 +/- ## === Coverage 77.57% 77.57% === Files 205 205 Lines 1577215772 === Hits1223512235 Misses 3537 3537 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=footer). Last update [87013d7...0960662](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README
codecov-io edited a comment on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410536262 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=h1) Report > Merging [#3699](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/87013d7f290bbed0a3028e10b6d5ac51061b7cb3?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3699 +/- ## === Coverage 77.57% 77.57% === Files 205 205 Lines 1577215772 === Hits1223512235 Misses 3537 3537 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=footer). Last update [87013d7...0960662](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ashb commented on a change in pull request #3699: [AIRFLOW-2853] Add official committers to README
ashb commented on a change in pull request #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#discussion_r207743727 ## File path: README.md ## @@ -274,6 +270,35 @@ Currently **officially** using Airflow: 1. [Zymergen](https://www.zymergen.com/) 1. [99](https://99taxis.com) [[@fbenevides](https://github.com/fbenevides), [@gustavoamigo](https://github.com/gustavoamigo) & [@mmmaia](https://github.com/mmmaia)] +## Who Maintains Airflow? + +Airflow is the work of the +[community](https://github.com/apache/incubator-airflow/graphs/contributors), +but the following people are responsible for reviewing and merging PRs as well +as steering conversation around new feature requests. If you would like to +become a maintainer, please review the Apache Airflow +[committer requirements](https://cwiki.apache.org/confluence/display/AIRFLOW/Committers). + +Currently **officially** maintaining Airflow (ordered alphabetically): + +1. [Alex Guziel](https://github.com/saguziel) +1. [Alex Van Boxel](https://github.com/alexvanboxel) +1. [Arthur Wiedmer](https://github.com/artwr) +1. [Ash Berlin-Taylor](https://github.com/ash) +1. [Bolke de Bruin](https://github.com/bolkedebruin) +1. [Chris Riccomini](https://github.com/criccomini) +1. Dan Davydov +1. [Fokko Driesprong](https://github.com/fokko) +1. [Jeremiah Lowin](https://github.com/jlowin) +1. [Joy Gao](https://github.com/joygao) +1. [Kaxil Naik](https://github.com/kaxil) +1. [Maxime Beauchemin](https://github.com/mistercrunch) +1. [Patrick Leo Tardif](https://github.com/patrickleotardif) +1. [Siddharth Anand](https://github.com/r39132) +1. Steven Yvinec-Kruyk +1. [Sumit Maheshwari](https://github.com/msumit) Review comment: Oh so he is - it's just sorted by user id :) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #3699: [AIRFLOW-2853] Add official committers to README
codecov-io commented on issue #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#issuecomment-410536262 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=h1) Report > Merging [#3699](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/87013d7f290bbed0a3028e10b6d5ac51061b7cb3?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3699 +/- ## === Coverage 77.57% 77.57% === Files 205 205 Lines 1577215772 === Hits1223512235 Misses 3537 3537 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=footer). Last update [87013d7...0960662](https://codecov.io/gh/apache/incubator-airflow/pull/3699?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] andscoop commented on a change in pull request #3699: [AIRFLOW-2853] Add official committers to README
andscoop commented on a change in pull request #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#discussion_r207743532 ## File path: README.md ## @@ -274,6 +270,35 @@ Currently **officially** using Airflow: 1. [Zymergen](https://www.zymergen.com/) 1. [99](https://99taxis.com) [[@fbenevides](https://github.com/fbenevides), [@gustavoamigo](https://github.com/gustavoamigo) & [@mmmaia](https://github.com/mmmaia)] +## Who Maintains Airflow? + +Airflow is the work of the +[community](https://github.com/apache/incubator-airflow/graphs/contributors), +but the following people are responsible for reviewing and merging PRs as well +as steering conversation around new feature requests. If you would like to +become a maintainer, please review the Apache Airflow +[committer requirements](https://cwiki.apache.org/confluence/display/AIRFLOW/Committers). + +Currently **officially** maintaining Airflow (ordered alphabetically): + +1. [Alex Guziel](https://github.com/saguziel) +1. [Alex Van Boxel](https://github.com/alexvanboxel) +1. [Arthur Wiedmer](https://github.com/artwr) +1. [Ash Berlin-Taylor](https://github.com/ash) +1. [Bolke de Bruin](https://github.com/bolkedebruin) +1. [Chris Riccomini](https://github.com/criccomini) +1. Dan Davydov +1. [Fokko Driesprong](https://github.com/fokko) +1. [Jeremiah Lowin](https://github.com/jlowin) +1. [Joy Gao](https://github.com/joygao) +1. [Kaxil Naik](https://github.com/kaxil) +1. [Maxime Beauchemin](https://github.com/mistercrunch) +1. [Patrick Leo Tardif](https://github.com/patrickleotardif) +1. [Siddharth Anand](https://github.com/r39132) +1. Steven Yvinec-Kruyk +1. [Sumit Maheshwari](https://github.com/msumit) Review comment: @ashb I have updated according to that list, just a note, I left Alex on as he is on the list you linked to. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] andscoop commented on a change in pull request #3699: [AIRFLOW-2853] Add official committers to README
andscoop commented on a change in pull request #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#discussion_r207743223 ## File path: README.md ## @@ -274,6 +270,35 @@ Currently **officially** using Airflow: 1. [Zymergen](https://www.zymergen.com/) 1. [99](https://99taxis.com) [[@fbenevides](https://github.com/fbenevides), [@gustavoamigo](https://github.com/gustavoamigo) & [@mmmaia](https://github.com/mmmaia)] +## Who Maintains Airflow? + +Airflow is the work of the +[community](https://github.com/apache/incubator-airflow/graphs/contributors), +but the following people are responsible for reviewing and merging PRs as well +as steering conversation around new feature requests. If you would like to +become a maintainer, please review the Apache Airflow +[committer requirements](https://cwiki.apache.org/confluence/display/AIRFLOW/Committers). + +Currently **officially** maintaining Airflow (ordered alphabetically): + +1. [Alex Guziel](https://github.com/saguziel) +1. [Alex Van Boxel](https://github.com/alexvanboxel) +1. [Arthur Wiedmer](https://github.com/artwr) +1. [Ash Berlin-Taylor](https://github.com/ash) +1. [Bolke de Bruin](https://github.com/bolkedebruin) +1. [Chris Riccomini](https://github.com/criccomini) +1. Dan Davydov +1. [Fokko Driesprong](https://github.com/fokko) +1. [Jeremiah Lowin](https://github.com/jlowin) +1. [Joy Gao](https://github.com/joygao) +1. [Kaxil Naik](https://github.com/kaxil) +1. [Maxime Beauchemin](https://github.com/mistercrunch) +1. [Patrick Leo Tardif](https://github.com/patrickleotardif) +1. [Siddharth Anand](https://github.com/r39132) +1. Steven Yvinec-Kruyk +1. [Sumit Maheshwari](https://github.com/msumit) Review comment: @ashb Perfect, I was looking for this and was unable to find it. I was going off of https://incubator.apache.org/projects/airflow.html . Will update This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ashb commented on a change in pull request #3699: [AIRFLOW-2853] Add official committers to README
ashb commented on a change in pull request #3699: [AIRFLOW-2853] Add official committers to README URL: https://github.com/apache/incubator-airflow/pull/3699#discussion_r207743076 ## File path: README.md ## @@ -274,6 +270,35 @@ Currently **officially** using Airflow: 1. [Zymergen](https://www.zymergen.com/) 1. [99](https://99taxis.com) [[@fbenevides](https://github.com/fbenevides), [@gustavoamigo](https://github.com/gustavoamigo) & [@mmmaia](https://github.com/mmmaia)] +## Who Maintains Airflow? + +Airflow is the work of the +[community](https://github.com/apache/incubator-airflow/graphs/contributors), +but the following people are responsible for reviewing and merging PRs as well +as steering conversation around new feature requests. If you would like to +become a maintainer, please review the Apache Airflow +[committer requirements](https://cwiki.apache.org/confluence/display/AIRFLOW/Committers). + +Currently **officially** maintaining Airflow (ordered alphabetically): + +1. [Alex Guziel](https://github.com/saguziel) +1. [Alex Van Boxel](https://github.com/alexvanboxel) +1. [Arthur Wiedmer](https://github.com/artwr) +1. [Ash Berlin-Taylor](https://github.com/ash) +1. [Bolke de Bruin](https://github.com/bolkedebruin) +1. [Chris Riccomini](https://github.com/criccomini) +1. Dan Davydov +1. [Fokko Driesprong](https://github.com/fokko) +1. [Jeremiah Lowin](https://github.com/jlowin) +1. [Joy Gao](https://github.com/joygao) +1. [Kaxil Naik](https://github.com/kaxil) +1. [Maxime Beauchemin](https://github.com/mistercrunch) +1. [Patrick Leo Tardif](https://github.com/patrickleotardif) +1. [Siddharth Anand](https://github.com/r39132) +1. Steven Yvinec-Kruyk +1. [Sumit Maheshwari](https://github.com/msumit) Review comment: This list is slightly out of date - you are missing some people, and Alex Guziel isn't a commiter? - the authoritative list is http://people.apache.org/committers-by-project.html#airflow This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2853) Add maintainers to the README
[
https://issues.apache.org/jira/browse/AIRFLOW-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569519#comment-16569519
]
ASF GitHub Bot commented on AIRFLOW-2853:
-
andscoop opened a new pull request #3699: [AIRFLOW-2853] Add official
committers to README
URL: https://github.com/apache/incubator-airflow/pull/3699
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
This PR brings a list of maintainers into the code repo so that they can
easily be found and live with the repo itself.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Docs
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add maintainers to the README
> -
>
> Key: AIRFLOW-2853
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2853
> Project: Apache Airflow
> Issue Type: Task
>Reporter: Andy Cooper
>Assignee: Andy Cooper
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] andscoop opened a new pull request #3699: [AIRFLOW-2853] Add official committers to README
andscoop opened a new pull request #3699: [AIRFLOW-2853] Add official
committers to README
URL: https://github.com/apache/incubator-airflow/pull/3699
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
This PR brings a list of maintainers into the code repo so that they can
easily be found and live with the repo itself.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Docs
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (AIRFLOW-2853) Add maintainers to the README
[ https://issues.apache.org/jira/browse/AIRFLOW-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Cooper updated AIRFLOW-2853: - Summary: Add maintainers to the README (was: Create a CODEOWNERS file for repo) > Add maintainers to the README > - > > Key: AIRFLOW-2853 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2853 > Project: Apache Airflow > Issue Type: Task >Reporter: Andy Cooper >Assignee: Andy Cooper >Priority: Major > > https://blog.github.com/2017-07-06-introducing-code-owners/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2853) Add maintainers to the README
[ https://issues.apache.org/jira/browse/AIRFLOW-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Cooper updated AIRFLOW-2853: - Description: (was: https://blog.github.com/2017-07-06-introducing-code-owners/) > Add maintainers to the README > - > > Key: AIRFLOW-2853 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2853 > Project: Apache Airflow > Issue Type: Task >Reporter: Andy Cooper >Assignee: Andy Cooper >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2855) Need to Check Validity of Cron Expression When Process DAG File/Zip File
[
https://issues.apache.org/jira/browse/AIRFLOW-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569486#comment-16569486
]
ASF GitHub Bot commented on AIRFLOW-2855:
-
XD-DENG opened a new pull request #3698: [AIRFLOW-2855] Check Cron Expression
Validity in DagBag.process_file()
URL: https://github.com/apache/incubator-airflow/pull/3698
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2855
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
`schedule_interval` of DAGs can either be timedelta or a Cron expression.
When it's a Cron expression, there is no mechanism to check its validity at
this moment. If there is anything wrong with the Cron expression itself, it
will cause issues when methods `following_schedule()` and
`previous_schedule()` are invoked (will affect scheduling).
However, exceptions will only be written into logs. From Web UI, it’s hard
for users to identify this issue & the source while no new task can be
initiated (especially for users who’re not very familiar with Cron).
It may be good to show error messages in web UI when a DAG's Cron
expression (as schedule_interval) can not be parsed by `croniter` properly
(this is implemented by adding these exceptions into metadata `import_error`,
whose entries will be shown as error messages in web UI).
Related tests are added as well.
**Screenshot**
https://user-images.githubusercontent.com/11539188/43687000-0ced5a5e-9901-11e8-85ba-b1d8ce0fce9f.png;>
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Need to Check Validity of Cron Expression When Process DAG File/Zip File
>
>
> Key: AIRFLOW-2855
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2855
> Project: Apache Airflow
> Issue Type: Improvement
> Components: DAG
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Critical
>
> *schedule_interval* of DAGs can either be *timedelta* or a *Cron expression*.
> When it's a Cron expression, there is no mechanism to check its validity at
> this moment. If there is anything wrong with the Cron expression itself, it
> will cause issues when methods _*following_schedule(**)*_ and
> _*previous_schedule()*_ are invoked (will affect scheduling). However,
> exceptions will only be written into logs. From Web UI, it’s hard for users
> to identify this issue & the source while no new task can be initiated
> (especially for users who’re not very familiar with Cron).
> It may be good to show error messages in web UI when a DAG's Cron expression
> (as schedule_interval) can not be parsed by *croniter* properly.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] XD-DENG opened a new pull request #3698: [AIRFLOW-2855] Check Cron Expression Validity in DagBag.process_file()
XD-DENG opened a new pull request #3698: [AIRFLOW-2855] Check Cron Expression
Validity in DagBag.process_file()
URL: https://github.com/apache/incubator-airflow/pull/3698
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-2855
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
`schedule_interval` of DAGs can either be timedelta or a Cron expression.
When it's a Cron expression, there is no mechanism to check its validity at
this moment. If there is anything wrong with the Cron expression itself, it
will cause issues when methods `following_schedule()` and
`previous_schedule()` are invoked (will affect scheduling).
However, exceptions will only be written into logs. From Web UI, it’s hard
for users to identify this issue & the source while no new task can be
initiated (especially for users who’re not very familiar with Cron).
It may be good to show error messages in web UI when a DAG's Cron
expression (as schedule_interval) can not be parsed by `croniter` properly
(this is implemented by adding these exceptions into metadata `import_error`,
whose entries will be shown as error messages in web UI).
Related tests are added as well.
**Screenshot**
https://user-images.githubusercontent.com/11539188/43687000-0ced5a5e-9901-11e8-85ba-b1d8ce0fce9f.png;>
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (AIRFLOW-2855) Need to Check Validity of Cron Expression When Process DAG File/Zip File
[ https://issues.apache.org/jira/browse/AIRFLOW-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaodong DENG updated AIRFLOW-2855: --- Description: *schedule_interval* of DAGs can either be *timedelta* or a *Cron expression*. When it's a Cron expression, there is no mechanism to check its validity at this moment. If there is anything wrong with the Cron expression itself, it will cause issues when methods _*following_schedule(**)*_ and _*previous_schedule()*_ are invoked (will affect scheduling). However, exceptions will only be written into logs. From Web UI, it’s hard for users to identify this issue & the source while no new task can be initiated (especially for users who’re not very familiar with Cron). It may be good to show error messages in web UI when a DAG's Cron expression (as schedule_interval) can not be parsed by *croniter* properly. was: *schedule_interval* of DAGs can either be *datetime* or a *Cron expression*. When it's a Cron expression, there is no mechanism to check its validity at this moment. If there is anything wrong with the Cron expression itself, it will cause issues when methods _*following_schedule(**)*_ and _*previous_schedule()*_ are invoked (will affect scheduling). However, exceptions will only be written into logs. From Web UI, it’s hard for users to identify this issue & the source while no new task can be initiated (especially for users who’re not very familiar with Cron). It may be good to show error messages in web UI when a DAG's Cron expression (as schedule_interval) can not be parsed by *croniter* properly. > Need to Check Validity of Cron Expression When Process DAG File/Zip File > > > Key: AIRFLOW-2855 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2855 > Project: Apache Airflow > Issue Type: Improvement > Components: DAG >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Critical > > *schedule_interval* of DAGs can either be *timedelta* or a *Cron expression*. > When it's a Cron expression, there is no mechanism to check its validity at > this moment. If there is anything wrong with the Cron expression itself, it > will cause issues when methods _*following_schedule(**)*_ and > _*previous_schedule()*_ are invoked (will affect scheduling). However, > exceptions will only be written into logs. From Web UI, it’s hard for users > to identify this issue & the source while no new task can be initiated > (especially for users who’re not very familiar with Cron). > It may be good to show error messages in web UI when a DAG's Cron expression > (as schedule_interval) can not be parsed by *croniter* properly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2855) Need to Check Validity of Cron Expression When Process DAG File/Zip File
Xiaodong DENG created AIRFLOW-2855: -- Summary: Need to Check Validity of Cron Expression When Process DAG File/Zip File Key: AIRFLOW-2855 URL: https://issues.apache.org/jira/browse/AIRFLOW-2855 Project: Apache Airflow Issue Type: Improvement Components: DAG Reporter: Xiaodong DENG Assignee: Xiaodong DENG *schedule_interval* of DAGs can either be *datetime* or a *Cron expression*. When it's a Cron expression, there is no mechanism to check its validity at this moment. If there is anything wrong with the Cron expression itself, it will cause issues when methods _*following_schedule(**)*_ and _*previous_schedule()*_ are invoked (will affect scheduling). However, exceptions will only be written into logs. From Web UI, it’s hard for users to identify this issue & the source while no new task can be initiated (especially for users who’re not very familiar with Cron). It may be good to show error messages in web UI when a DAG's Cron expression (as schedule_interval) can not be parsed by *croniter* properly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ashb commented on issue #3684: [AIRFLOW-2840] - add update connections cli option
ashb commented on issue #3684: [AIRFLOW-2840] - add update connections cli option URL: https://github.com/apache/incubator-airflow/pull/3684#issuecomment-410517934 @bolkedeburin no worries on the label, I was just using them as a way of tracking in the PRs page which ones we've looked at - entirely manual for now. I think the default should refuse to update multiple, with an --update-all flag This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] bolkedebruin edited a comment on issue #3684: [AIRFLOW-2840] - add update connections cli option
bolkedebruin edited a comment on issue #3684: [AIRFLOW-2840] - add update connections cli option URL: https://github.com/apache/incubator-airflow/pull/3684#issuecomment-410501309 Please move the core of the code out of the cli (I.e. split between api/Commons and changes to the connection model) and make sure it works through the REST API as well. Direct access to the database should be limited in the cli as we will move away from this in the future. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] bolkedebruin commented on issue #3684: [AIRFLOW-2840] - add update connections cli option
bolkedebruin commented on issue #3684: [AIRFLOW-2840] - add update connections cli option URL: https://github.com/apache/incubator-airflow/pull/3684#issuecomment-410514576 O sorry @ashb I think I accidentally removed the label This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #3697: [AIRFLOW-2854] kubernetes_pod_operator add more configuration items
codecov-io commented on issue #3697: [AIRFLOW-2854] kubernetes_pod_operator add more configuration items URL: https://github.com/apache/incubator-airflow/pull/3697#issuecomment-410507534 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=h1) Report > Merging [#3697](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/87013d7f290bbed0a3028e10b6d5ac51061b7cb3?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3697 +/- ## === Coverage 77.57% 77.57% === Files 205 205 Lines 1577215772 === Hits1223512235 Misses 3537 3537 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=tree) | Coverage Δ | | |---|---|---| | [...irflow/example\_dags/example\_kubernetes\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/3697/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9rdWJlcm5ldGVzX29wZXJhdG9yLnB5) | `75% <ø> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=footer). Last update [87013d7...02da54b](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3697: [AIRFLOW-2854] kubernetes_pod_operator add more configuration items
codecov-io edited a comment on issue #3697: [AIRFLOW-2854] kubernetes_pod_operator add more configuration items URL: https://github.com/apache/incubator-airflow/pull/3697#issuecomment-410507534 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=h1) Report > Merging [#3697](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/87013d7f290bbed0a3028e10b6d5ac51061b7cb3?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#3697 +/- ## === Coverage 77.57% 77.57% === Files 205 205 Lines 1577215772 === Hits1223512235 Misses 3537 3537 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=tree) | Coverage Δ | | |---|---|---| | [...irflow/example\_dags/example\_kubernetes\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/3697/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9rdWJlcm5ldGVzX29wZXJhdG9yLnB5) | `75% <ø> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=footer). Last update [87013d7...02da54b](https://codecov.io/gh/apache/incubator-airflow/pull/3697?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Fokko commented on issue #3691: [AIRFLOW-2846] Add missing python test dependency to setup.py
Fokko commented on issue #3691: [AIRFLOW-2846] Add missing python test dependency to setup.py URL: https://github.com/apache/incubator-airflow/pull/3691#issuecomment-410503637 In case who missed it: https://www.youtube.com/watch?v=5Il3w2P7IcM Awesome work @holdenk for getting people inspired to contribute to open source software! I've checked. The tox version being installed with `apt-get` by Travis is also 3.1.3. Not completely sure why this failed :-) In the end we should rely less on Travis and move to a Dockerized setup to make testing easier. There is a great effort going on in https://github.com/apache/incubator-airflow/pull/3393/ but there are still some lose ends that need to be fixed before we can merge it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] bolkedebruin commented on issue #3684: [AIRFLOW-2840] - add update connections cli option
bolkedebruin commented on issue #3684: [AIRFLOW-2840] - add update connections cli option URL: https://github.com/apache/incubator-airflow/pull/3684#issuecomment-410501309 Please move the core of the code out of the cli (e.g. make it part of the Connection model, see DagRun for an example) and make sure it works through the REST API as well. Direct access to the database should be limited in the cli as we will move away from this in the future. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feng-tao commented on a change in pull request #3650: [AIRFLOW-2703] exceptions from scheduler's heartbeat is handled so that scheduler does not crash
feng-tao commented on a change in pull request #3650: [AIRFLOW-2703] exceptions
from scheduler's heartbeat is handled so that scheduler does not crash
URL: https://github.com/apache/incubator-airflow/pull/3650#discussion_r207726248
##
File path: airflow/jobs.py
##
@@ -160,33 +160,36 @@ def heartbeat(self):
heart rate. If you go over 60 seconds before calling it, it won't
sleep at all.
"""
-with create_session() as session:
-job = session.query(BaseJob).filter_by(id=self.id).one()
-make_transient(job)
-session.commit()
+try:
+with create_session() as session:
+job = session.query(BaseJob).filter_by(id=self.id).one()
+make_transient(job)
+session.commit()
-if job.state == State.SHUTDOWN:
-self.kill()
+if job.state == State.SHUTDOWN:
+self.kill()
-# Figure out how long to sleep for
-sleep_for = 0
-if job.latest_heartbeat:
-sleep_for = max(
-0,
-self.heartrate - (
-timezone.utcnow() - job.latest_heartbeat).total_seconds())
+# Figure out how long to sleep for
+sleep_for = 0
+if job.latest_heartbeat:
+sleep_for = max(
+0,
+self.heartrate - (timezone.utcnow() -
+ job.latest_heartbeat).total_seconds())
-sleep(sleep_for)
+sleep(sleep_for)
-# Update last heartbeat time
-with create_session() as session:
-job = session.query(BaseJob).filter(BaseJob.id == self.id).first()
-job.latest_heartbeat = timezone.utcnow()
-session.merge(job)
-session.commit()
+# Update last heartbeat time
+with create_session() as session:
+job = session.query(BaseJob).filter(BaseJob.id ==
self.id).first()
+job.latest_heartbeat = timezone.utcnow()
+session.merge(job)
+session.commit()
-self.heartbeat_callback(session=session)
-self.log.debug('[heartbeat]')
+self.heartbeat_callback(session=session)
+self.log.debug('[heartbeat]')
+except Exception as e:
Review comment:
exception here is too board. What exception are we expecting? I think we
need to be a little bit more specific.
And scheduler could always get restarted by runit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
