[GitHub] deagon opened a new pull request #3975: [AIRFLOW-XXX] Pin docker requirement version

[GitBox]

deagon opened a new pull request #3975: [AIRFLOW-XXX] Pin docker requirement 
version
URL: https://github.com/apache/incubator-airflow/pull/3975
 
 
   The method "create_container" in APIClient of docker has been incompatible 
from version 3.0.0.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] 7yl4r commented on issue #3754: [Airflow-1737] Handle `execution_date`s with fractional seconds in www/views

[GitBox]

7yl4r commented on issue #3754: [Airflow-1737] Handle `execution_date`s with 
fractional seconds in www/views
URL: 
https://github.com/apache/incubator-airflow/pull/3754#issuecomment-425601366
 
 
   Yes, this is only an issue in the 1.9 branch. Since there isn't a 1.9.1 
planned I will close this and focus my efforts on keeping nodes up-to-date. 
:wink: :+1: 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] 7yl4r closed pull request #3754: [Airflow-1737] Handle `execution_date`s with fractional seconds in www/views

[GitBox]

7yl4r closed pull request #3754: [Airflow-1737] Handle `execution_date`s with 
fractional seconds in www/views
URL: https://github.com/apache/incubator-airflow/pull/3754
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/utils/dates.py b/airflow/utils/dates.py
index 43b87f4b00..0a4b536bed 100644
--- a/airflow/utils/dates.py
+++ b/airflow/utils/dates.py
@@ -225,3 +225,18 @@ def days_ago(n, hour=0, minute=0, second=0, microsecond=0):
 second=second,
 microsecond=microsecond)
 return today - timedelta(days=n)
+
+
+def execution_date_ptime(date_string):
+"""
+Get datetime.datetime object from given date_string assuming format
+string is ISO8601-like (aka looks like an execution_date).
+Like datetime.strptime but for execution_dates.
+Handles execution dates with *and* without fractional seconds.
+"""
+ISO8601_FMT = '%Y-%m-%d %H:%M:%S'
+ISO8601_FMT_FRACTIONAL = ISO8601_FMT + '.%f'
+if len(date_string) > 19:  # eg: '-MM-DD HH:MM:SS.fff'
+return datetime.strptime(date_string, ISO8601_FMT_FRACTIONAL)
+else:  # eg '-MM-DD HH:MM:SS'
+return datetime.strptime(ISO8601_FMT)
diff --git a/airflow/www/views.py b/airflow/www/views.py
index 63af88fd4b..83a653ca11 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -75,7 +75,11 @@
 from airflow.utils.state import State
 from airflow.utils.db import provide_session
 from airflow.utils.helpers import alchemy_to_dict
-from airflow.utils.dates import infer_time_unit, scale_time_units
+from airflow.utils.dates import (
+infer_time_unit,
+scale_time_units,
+execution_date_ptime
+)
 from airflow.www import utils as wwwutils
 from airflow.www.forms import DateTimeForm, DateTimeWithNumRunsForm
 from airflow.www.validators import GreaterEqualThan
@@ -2527,10 +2531,12 @@ def set_task_instance_state(self, ids, target_state, 
session=None):
 count = len(ids)
 for id in ids:
 task_id, dag_id, execution_date = id.split(',')
-execution_date = datetime.strptime(execution_date, '%Y-%m-%d 
%H:%M:%S')
-ti = session.query(TI).filter(TI.task_id == task_id,
-  TI.dag_id == dag_id,
-  TI.execution_date == 
execution_date).one()
+execution_date = execution_date_ptime(execution_date)
+ti = session.query(TI).filter(
+TI.task_id == task_id,
+TI.dag_id == dag_id,
+TI.execution_date == execution_date
+).one()
 ti.state = target_state
 session.commit()
 flash(
@@ -2547,10 +2553,12 @@ def delete_task_instances(self, ids, session=None):
 count = 0
 for id in ids:
 task_id, dag_id, execution_date = id.split(',')
-execution_date = datetime.strptime(execution_date, '%Y-%m-%d 
%H:%M:%S')
-count += session.query(TI).filter(TI.task_id == task_id,
-  TI.dag_id == dag_id,
-  TI.execution_date == 
execution_date).delete()
+execution_date = execution_date_ptime(execution_date)
+count += session.query(TI).filter(
+TI.task_id == task_id,
+TI.dag_id == dag_id,
+TI.execution_date == execution_date
+).delete()
 session.commit()
 flash("{count} task instances were deleted".format(**locals()))
 except Exception as ex:


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] feng-tao commented on issue #3754: [Airflow-1737] Handle `execution_date`s with fractional seconds in www/views

[GitBox]

feng-tao commented on issue #3754: [Airflow-1737] Handle `execution_date`s with 
fractional seconds in www/views
URL: 
https://github.com/apache/incubator-airflow/pull/3754#issuecomment-425593191
 
 
   @ashb, I think this issue is only on 1.9 branch. The master branch has fixed 
this 
issue(https://github.com/apache/incubator-airflow/commit/313f5bac4a3f804094bcd583e0e5fbc3b5f405bb).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashb commented on issue #3754: [Airflow-1737] Handle `execution_date`s with fractional seconds in www/views

[GitBox]

ashb commented on issue #3754: [Airflow-1737] Handle `execution_date`s with 
fractional seconds in www/views
URL: 
https://github.com/apache/incubator-airflow/pull/3754#issuecomment-425588065
 
 
   Do we still need this in 1.10/master or does the linked commit also fix the 
issue reported here?
   
   We're unlikely to create a 1.9.1, so if this is still needed could you 
retarget against master (and I'll cherry-pick it in to the 1.10.1 branch.)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashb opened a new pull request #3974: [AIRFLOW-XXX] Speed up DagBagTest cases

[GitBox]

ashb opened a new pull request #3974: [AIRFLOW-XXX] Speed up DagBagTest cases
URL: https://github.com/apache/incubator-airflow/pull/3974
 
 
   I noticed that many of the tests of DagBags operate on a specific DAG
   only, and don't need to load the example or test dags. By not loading
   the dags we don't need to this shaves about 10-20s of test time.
   
   Nothing huge, but every little bit counts.
   
   **Before**:
   
   + nosetests tests.models.DagBagTest
   
   --
   Ran 12 tests in 30.855s
   
   OK
   [2018-09-28 22:42:17,846] {settings.py:193} DEBUG - Disposing DB 
connection pool (PID 1467)
   
   real0m47.582s
   user0m8.300s
   sys 0m4.700s
   
   **After**:
   
   + nosetests tests.models.DagBagTest
   
   --
   Ran 12 tests in 7.784s
   
   OK
   [2018-09-28 22:44:39,475] {settings.py:193} DEBUG - Disposing DB 
connection pool (PID 1490)
   
   real0m30.399s
   user0m9.960s
   sys 0m3.180s
   
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [ ] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
   
   ### Description
   
   - [ ] Here are some details about my PR, including screenshots of any UI 
changes:
   
   ### Tests
   
   - [ ] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [ ] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [ ] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - When adding new operators/hooks/sensors, the autoclass documentation 
generation needs to be added.
   
   ### Code Quality
   
   - [ ] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashb opened a new pull request #3973: [AIRFLOW-XXX] Don't spam test logs with "bad cron expression" messages

[GitBox]

ashb opened a new pull request #3973: [AIRFLOW-XXX] Don't spam test logs with 
"bad cron expression" messages
URL: https://github.com/apache/incubator-airflow/pull/3973
 
 
   
   
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] https://issues.apache.org/jira/browse/AIRFLOW-XXX
   
   ### Description
   
   We needed these test dags to check the behaviour of invalid cron
   expressions, but by default we were loading them every time we create a
   DagBag (which many, many tests to).
   
   Instead we ignore these known-bad dags by default, and the test checking
   those (tests/models.py:DagBagTest.test_process_file_cron_validity_check)
   is already explicitly processing those DAGs directly, so it remains
   tested.
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason: Just makes test output a tiny bit easier to read
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - When adding new operators/hooks/sensors, the autoclass documentation 
generation needs to be added.
   
   ### Code Quality
   
   - [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] KevinYang21 commented on issue #3873: [Airflow-2760] Decouple DAG parsing loop from scheduler loop

[GitBox]

KevinYang21 commented on issue #3873: [Airflow-2760] Decouple DAG parsing loop 
from scheduler loop
URL: 
https://github.com/apache/incubator-airflow/pull/3873#issuecomment-425585509
 
 
   @ashb Thank you very much for the reviews! Totally understand your concern 
and I can definitely wait or try to work with other committers to have it 
tested to provide more confidence.
   
   Any volunteers :D ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] KevinYang21 commented on a change in pull request #3873: [Airflow-2760] Decouple DAG parsing loop from scheduler loop

[GitBox]

KevinYang21 commented on a change in pull request #3873: [Airflow-2760] 
Decouple DAG parsing loop from scheduler loop
URL: https://github.com/apache/incubator-airflow/pull/3873#discussion_r221396901
 
 

 ##
 File path: airflow/jobs.py
 ##
 @@ -1551,163 +1473,111 @@ def _execute(self):
 (executors.LocalExecutor, executors.SequentialExecutor):
 pickle_dags = True
 
-# Use multiple processes to parse and generate tasks for the
-# DAGs in parallel. By processing them in separate processes,
-# we can get parallelism and isolation from potentially harmful
-# user code.
-self.log.info(
-"Processing files using up to %s processes at a time",
-self.max_threads)
 self.log.info("Running execute loop for %s seconds", self.run_duration)
 self.log.info("Processing each file at most %s times", self.num_runs)
-self.log.info(
-"Process each file at most once every %s seconds",
-self.file_process_interval)
-self.log.info(
-"Checking for new files in %s every %s seconds",
-self.subdir,
-self.dag_dir_list_interval)
 
 # Build up a list of Python files that could contain DAGs
 self.log.info("Searching for files in %s", self.subdir)
 known_file_paths = list_py_file_paths(self.subdir)
 self.log.info("There are %s files in %s", len(known_file_paths), 
self.subdir)
 
-def processor_factory(file_path):
+def processor_factory(file_path, zombies):
 return DagFileProcessor(file_path,
 pickle_dags,
-self.dag_ids)
+self.dag_ids,
+zombies)
+
+# When using sqlite, we do not use async_mode
+# so the scheduler job and DAG parser don't access the DB at the same 
time.
+async_mode = not self.using_sqlite
 
-processor_manager = DagFileProcessorManager(self.subdir,
-known_file_paths,
-self.max_threads,
-self.file_process_interval,
-self.num_runs,
-processor_factory)
+self.processor_agent = DagFileProcessorAgent(self.subdir,
+ known_file_paths,
+ self.num_runs,
+ processor_factory,
+ async_mode)
 
 try:
-self._execute_helper(processor_manager)
+self._execute_helper()
 finally:
+self.processor_agent.end()
 self.log.info("Exited execute loop")
 
-# Kill all child processes on exit since we don't want to leave
-# them as orphaned.
-pids_to_kill = processor_manager.get_all_pids()
-if len(pids_to_kill) > 0:
-# First try SIGTERM
-this_process = psutil.Process(os.getpid())
-# Only check child processes to ensure that we don't have a 
case
-# where we kill the wrong process because a child process died
-# but the PID got reused.
-child_processes = [x for x in 
this_process.children(recursive=True)
-   if x.is_running() and x.pid in pids_to_kill]
-for child in child_processes:
-self.log.info("Terminating child PID: %s", child.pid)
-child.terminate()
-# TODO: Remove magic number
-timeout = 5
-self.log.info(
-"Waiting up to %s seconds for processes to exit...", 
timeout)
-try:
-psutil.wait_procs(
-child_processes, timeout=timeout,
-callback=lambda x: self.log.info('Terminated PID %s', 
x.pid))
-except psutil.TimeoutExpired:
-self.log.debug("Ran out of time while waiting for 
processes to exit")
-
-# Then SIGKILL
-child_processes = [x for x in 
this_process.children(recursive=True)
-   if x.is_running() and x.pid in pids_to_kill]
-if len(child_processes) > 0:
-self.log.info("SIGKILL processes that did not terminate 
gracefully")
-for child in child_processes:
-self.log.info("Killing child PID: %s", child.pid)
-child.kill()
-child.wait()
-
-def _execute_helper(self, processor_manager):
-

[GitHub] KevinYang21 commented on a change in pull request #3873: [Airflow-2760] Decouple DAG parsing loop from scheduler loop

[GitBox]

KevinYang21 commented on a change in pull request #3873: [Airflow-2760] 
Decouple DAG parsing loop from scheduler loop
URL: https://github.com/apache/incubator-airflow/pull/3873#discussion_r221396632
 
 

 ##
 File path: airflow/config_templates/default_airflow.cfg
 ##
 @@ -70,6 +70,7 @@ simple_log_format = %%(asctime)s %%(levelname)s - 
%%(message)s
 # we need to escape the curly braces by adding an additional curly brace
 log_filename_template =  ti.dag_id / ti.task_id / ts 
/ try_number .log
 log_processor_filename_template =  filename .log
+dag_processor_manager_log_location = 
{AIRFLOW_HOME}/logs/dag_processor_manager/dag_processor_manager.log
 
 Review comment:
   lol, I started with `log_dag_processor_manager_location` and then switched 
back because it reads bit weird to me... (`log_filename_template` reads good 
but `log_prodcessor_file_template` also reads weird to me, wondering if 
`log_prodcessor_file_template` was just named because the author saw the 
previous template config starts with log) I would actually even prefer to 
rename `log_processor_filename_template ` to make the names more intuitive to 
understand.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (AIRFLOW-2003) Use flask caching instead of flask cache

[Ash Berlin-Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ash Berlin-Taylor updated AIRFLOW-2003:
---
Fix Version/s: (was: 2.0.0)
   1.10.0

> Use flask caching instead of flask cache 
> -
>
> Key: AIRFLOW-2003
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2003
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Bolke de Bruin
>Priority: Major
> Fix For: 1.10.0
>
>
> Flask cache has been unmaintained for over 3 years. Flask-caching is the 
> community maintained version



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ashb commented on a change in pull request #3873: [Airflow-2760] Decouple DAG parsing loop from scheduler loop

[GitBox]

ashb commented on a change in pull request #3873: [Airflow-2760] Decouple DAG 
parsing loop from scheduler loop
URL: https://github.com/apache/incubator-airflow/pull/3873#discussion_r221389771
 
 

 ##
 File path: airflow/jobs.py
 ##
 @@ -1551,163 +1473,111 @@ def _execute(self):
 (executors.LocalExecutor, executors.SequentialExecutor):
 pickle_dags = True
 
-# Use multiple processes to parse and generate tasks for the
-# DAGs in parallel. By processing them in separate processes,
-# we can get parallelism and isolation from potentially harmful
-# user code.
-self.log.info(
-"Processing files using up to %s processes at a time",
-self.max_threads)
 self.log.info("Running execute loop for %s seconds", self.run_duration)
 self.log.info("Processing each file at most %s times", self.num_runs)
-self.log.info(
-"Process each file at most once every %s seconds",
-self.file_process_interval)
-self.log.info(
-"Checking for new files in %s every %s seconds",
-self.subdir,
-self.dag_dir_list_interval)
 
 # Build up a list of Python files that could contain DAGs
 self.log.info("Searching for files in %s", self.subdir)
 known_file_paths = list_py_file_paths(self.subdir)
 self.log.info("There are %s files in %s", len(known_file_paths), 
self.subdir)
 
-def processor_factory(file_path):
+def processor_factory(file_path, zombies):
 return DagFileProcessor(file_path,
 pickle_dags,
-self.dag_ids)
+self.dag_ids,
+zombies)
+
+# When using sqlite, we do not use async_mode
+# so the scheduler job and DAG parser don't access the DB at the same 
time.
+async_mode = not self.using_sqlite
 
-processor_manager = DagFileProcessorManager(self.subdir,
-known_file_paths,
-self.max_threads,
-self.file_process_interval,
-self.num_runs,
-processor_factory)
+self.processor_agent = DagFileProcessorAgent(self.subdir,
+ known_file_paths,
+ self.num_runs,
+ processor_factory,
+ async_mode)
 
 try:
-self._execute_helper(processor_manager)
+self._execute_helper()
 finally:
+self.processor_agent.end()
 self.log.info("Exited execute loop")
 
-# Kill all child processes on exit since we don't want to leave
-# them as orphaned.
-pids_to_kill = processor_manager.get_all_pids()
-if len(pids_to_kill) > 0:
-# First try SIGTERM
-this_process = psutil.Process(os.getpid())
-# Only check child processes to ensure that we don't have a 
case
-# where we kill the wrong process because a child process died
-# but the PID got reused.
-child_processes = [x for x in 
this_process.children(recursive=True)
-   if x.is_running() and x.pid in pids_to_kill]
-for child in child_processes:
-self.log.info("Terminating child PID: %s", child.pid)
-child.terminate()
-# TODO: Remove magic number
-timeout = 5
-self.log.info(
-"Waiting up to %s seconds for processes to exit...", 
timeout)
-try:
-psutil.wait_procs(
-child_processes, timeout=timeout,
-callback=lambda x: self.log.info('Terminated PID %s', 
x.pid))
-except psutil.TimeoutExpired:
-self.log.debug("Ran out of time while waiting for 
processes to exit")
-
-# Then SIGKILL
-child_processes = [x for x in 
this_process.children(recursive=True)
-   if x.is_running() and x.pid in pids_to_kill]
-if len(child_processes) > 0:
-self.log.info("SIGKILL processes that did not terminate 
gracefully")
-for child in child_processes:
-self.log.info("Killing child PID: %s", child.pid)
-child.kill()
-child.wait()
-
-def _execute_helper(self, processor_manager):
-"""
-  

[GitHub] ashb commented on a change in pull request #3873: [Airflow-2760] Decouple DAG parsing loop from scheduler loop

[GitBox]

ashb commented on a change in pull request #3873: [Airflow-2760] Decouple DAG 
parsing loop from scheduler loop
URL: https://github.com/apache/incubator-airflow/pull/3873#discussion_r221389524
 
 

 ##
 File path: airflow/config_templates/default_airflow.cfg
 ##
 @@ -70,6 +70,7 @@ simple_log_format = %%(asctime)s %%(levelname)s - 
%%(message)s
 # we need to escape the curly braces by adding an additional curly brace
 log_filename_template =  ti.dag_id / ti.task_id / ts 
/ try_number .log
 log_processor_filename_template =  filename .log
+dag_processor_manager_log_location = 
{AIRFLOW_HOME}/logs/dag_processor_manager/dag_processor_manager.log
 
 Review comment:
   For consistency can we call this `log_dag_processor_manager_location` (i.e. 
so that it starts with `log_` like the two before it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (AIRFLOW-2574) initdb fails when mysql password contains percent sign

[Ash Berlin-Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ash Berlin-Taylor updated AIRFLOW-2574:
---
Affects Version/s: 1.9.0
   1.10.0
Fix Version/s: 1.10.1

> initdb fails when mysql password contains percent sign
> --
>
> Key: AIRFLOW-2574
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2574
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: db
>Affects Versions: 1.9.0, 1.10.0
>Reporter: Zihao Zhang
>Priority: Minor
> Fix For: 1.10.1
>
>
> [db.py|https://github.com/apache/incubator-airflow/blob/3358551c8e73d9019900f7a85f18ebfd88591450/airflow/utils/db.py#L345]
>  uses 
> [config.set_main_option|http://alembic.zzzcomputing.com/en/latest/api/config.html#alembic.config.Config.set_main_option]
>  which says "A raw percent sign not part of an interpolation symbol must 
> therefore be escaped"
> When there is a percent sign in database connection string, this will crash 
> due to bad interpolation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-1412) airflow task logs can be arbitrarily large (leading to webserver log page hanging)

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632527#comment-16632527
 ] 

ASF GitHub Bot commented on AIRFLOW-1412:
-

ashb closed pull request #2442: [AIRFLOW-1412] Add config to truncate the task 
log file
URL: https://github.com/apache/incubator-airflow/pull/2442
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/bin/cli.py b/airflow/bin/cli.py
index f568d5dc8e..9447112de0 100755
--- a/airflow/bin/cli.py
+++ b/airflow/bin/cli.py
@@ -457,6 +457,20 @@ def run(args, dag=None):
 logging.root.handlers[0].flush()
 logging.root.handlers = []
 
+# truncate log files
+max_log_file_size = conf.get('core', 'MAX_LOG_FILE_SIZE_BYTES')
+if (
+max_log_file_size and
+os.path.exists(filename) and
+os.path.getsize(filename) > int(max_log_file_size)
+):
+command = 'truncate -s %s %s' % (max_log_file_size, filename)
+try:
+subprocess.check_call(command, shell=True)
+except subprocess.CalledProcessError as e:
+# don't fail the process just because log file truncate fails
+logging.error(e)
+
 # store logs remotely
 remote_base = conf.get('core', 'REMOTE_BASE_LOG_FOLDER')
 
diff --git a/airflow/config_templates/default_airflow.cfg 
b/airflow/config_templates/default_airflow.cfg
index ddd1ba816e..49a5c057e3 100644
--- a/airflow/config_templates/default_airflow.cfg
+++ b/airflow/config_templates/default_airflow.cfg
@@ -35,6 +35,9 @@ dags_folder = {AIRFLOW_HOME}/dags
 # This path must be absolute
 base_log_folder = {AIRFLOW_HOME}/logs
 
+# The max size of each task log file
+max_log_file_size_bytes =
+
 # Airflow can store logs remotely in AWS S3 or Google Cloud Storage. Users
 # must supply a remote location URL (starting with either 's3://...' or
 # 'gs://...') and an Airflow connection id that provides access to the storage


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> airflow task logs can be arbitrarily large (leading to webserver log page 
> hanging)
> --
>
> Key: AIRFLOW-1412
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1412
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: webserver, worker
>Reporter: Angela Zhang
>Assignee: Angela Zhang
>Priority: Minor
> Fix For: 1.10.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (AIRFLOW-1412) airflow task logs can be arbitrarily large (leading to webserver log page hanging)

[Ash Berlin-Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ash Berlin-Taylor resolved AIRFLOW-1412.

   Resolution: Fixed
Fix Version/s: 1.10.0

> airflow task logs can be arbitrarily large (leading to webserver log page 
> hanging)
> --
>
> Key: AIRFLOW-1412
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1412
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: webserver, worker
>Reporter: Angela Zhang
>Assignee: Angela Zhang
>Priority: Minor
> Fix For: 1.10.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ashb commented on issue #2442: [AIRFLOW-1412] Add config to truncate the task log file

[GitBox]

ashb commented on issue #2442: [AIRFLOW-1412] Add config to truncate the task 
log file
URL: 
https://github.com/apache/incubator-airflow/pull/2442#issuecomment-425557317
 
 
   > Yeah I think as long as the webserver can lazily fetch the logs 1 page at 
a time, that would solve the main problem here. Any idea when that would be 
shipped?
   
   In 1.10 it turns out!. Closing this now.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashb closed pull request #2442: [AIRFLOW-1412] Add config to truncate the task log file

[GitBox]

ashb closed pull request #2442: [AIRFLOW-1412] Add config to truncate the task 
log file
URL: https://github.com/apache/incubator-airflow/pull/2442
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/bin/cli.py b/airflow/bin/cli.py
index f568d5dc8e..9447112de0 100755
--- a/airflow/bin/cli.py
+++ b/airflow/bin/cli.py
@@ -457,6 +457,20 @@ def run(args, dag=None):
 logging.root.handlers[0].flush()
 logging.root.handlers = []
 
+# truncate log files
+max_log_file_size = conf.get('core', 'MAX_LOG_FILE_SIZE_BYTES')
+if (
+max_log_file_size and
+os.path.exists(filename) and
+os.path.getsize(filename) > int(max_log_file_size)
+):
+command = 'truncate -s %s %s' % (max_log_file_size, filename)
+try:
+subprocess.check_call(command, shell=True)
+except subprocess.CalledProcessError as e:
+# don't fail the process just because log file truncate fails
+logging.error(e)
+
 # store logs remotely
 remote_base = conf.get('core', 'REMOTE_BASE_LOG_FOLDER')
 
diff --git a/airflow/config_templates/default_airflow.cfg 
b/airflow/config_templates/default_airflow.cfg
index ddd1ba816e..49a5c057e3 100644
--- a/airflow/config_templates/default_airflow.cfg
+++ b/airflow/config_templates/default_airflow.cfg
@@ -35,6 +35,9 @@ dags_folder = {AIRFLOW_HOME}/dags
 # This path must be absolute
 base_log_folder = {AIRFLOW_HOME}/logs
 
+# The max size of each task log file
+max_log_file_size_bytes =
+
 # Airflow can store logs remotely in AWS S3 or Google Cloud Storage. Users
 # must supply a remote location URL (starting with either 's3://...' or
 # 'gs://...') and an Airflow connection id that provides access to the storage


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashb commented on issue #3945: [AIRFLOW-3112] Make SFTP hook to inherit SSH hook

[GitBox]

ashb commented on issue #3945: [AIRFLOW-3112] Make SFTP hook to inherit SSH hook
URL: 
https://github.com/apache/incubator-airflow/pull/3945#issuecomment-425556002
 
 
   At this point I'd probably say lets make the maintain the back-compat for 
2.0 (likely to be the next release, so people have an easy upgrade path) and to 
be removed in 2.1. I would be happy with just saying "removed in a future 
version" and not being specific at this point though.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-3131) Start Date is not honoured in case of task is failed due to Mysql connection failure

[Tanuj Gupta (JIRA)]

Tanuj Gupta created AIRFLOW-3131:


 Summary: Start Date is not honoured in case of task is failed due 
to Mysql connection failure
 Key: AIRFLOW-3131
 URL: https://issues.apache.org/jira/browse/AIRFLOW-3131
 Project: Apache Airflow
  Issue Type: Bug
  Components: cli, db
Affects Versions: 1.9.0
Reporter: Tanuj Gupta


Sometimes, when we trigger some DAG then task's start sate is seen as null in 
database. In this we have observed that "airflow run" command fails due to 
mysql connection failure and in turn task state is set as FAILED but start date 
is not updated in the database. 

 
Traceback (most recent call last): File "/usr/src/venv/bin/airflow", line 27, 
in  args.func(args) File 
"/usr/src/venv/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 387, 
in run run_job.run() File 
"/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 193, in 
run id_ = self.id File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", 
line 242, in __get__ return self.impl.get(instance_state(instance), dict_) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", 
line 594, in get value = state._load_expired(state, passive) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/state.py", line 
608, in _load_expired self.manager.deferred_scalar_loader(self, toload) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/loading.py", 
line 876, in load_scalar_attributes only_load_props=attribute_names) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/loading.py", 
line 188, in load_on_ident identity_token=identity_token File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/loading.py", 
line 250, in load_on_pk_identity return q.one() File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 
2947, in one ret = self.one_or_none() File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 
2917, in one_or_none ret = list(self) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 
2988, in __iter__ return self._execute_and_instances(context) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 
3009, in _execute_and_instances close_with_result=True) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 
3018, in _get_bind_args **kw File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 
3000, in _connection_from_session conn = self.session.connection(**kw) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", 
line 1035, in connection execution_options=execution_options) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", 
line 1040, in _connection_for_bind engine, execution_options) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", 
line 409, in _connection_for_bind conn = bind.contextual_connect() File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", 
line 2123, in contextual_connect self._wrap_pool_connect(self.pool.connect, 
None), File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", 
line 2162, in _wrap_pool_connect e, dialect, self) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", 
line 1476, in _handle_dbapi_exception_noconnection exc_info File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", 
line 265, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, 
cause=cause) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", 
line 2158, in _wrap_pool_connect return fn() File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 403, 
in connect return _ConnectionFairy._checkout(self) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 791, 
in _checkout fairy = _ConnectionRecord.checkout(pool) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 532, 
in checkout rec = pool._do_get() File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 
1287, in _do_get return self._create_connection() File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 350, 
in _create_connection return _ConnectionRecord(self) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 477, 
in __init__ self.__connect(first_connect_check=True) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 674, 
in __connect connection = pool._invoke_creator(self) File 
"/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py",
 line 106, in connect 

[GitHub] codecov-io edited a comment on issue #2442: [AIRFLOW-1412] Add config to truncate the task log file

[GitBox]

codecov-io edited a comment on issue #2442: [AIRFLOW-1412] Add config to 
truncate the task log file
URL: 
https://github.com/apache/incubator-airflow/pull/2442#issuecomment-315239241
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2442?src=pr=h1)
 Report
   > Merging 
[#2442](https://codecov.io/gh/apache/incubator-airflow/pull/2442?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/304c2afd20c0ea3139054a38f9eb38d5fa7d930e?src=pr=desc)
 will **decrease** coverage by `8.24%`.
   > The diff coverage is `28.57%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/2442/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/2442?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#2442  +/-   ##
   ==
   - Coverage   77.49%   69.25%   -8.25% 
   ==
 Files 200  146  -54 
 Lines   1589311251-4642 
   ==
   - Hits12317 7792-4525 
   + Misses   3576 3459 -117
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/2442?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `51.61% <28.57%> (-13.13%)` | :arrow_down: |
   | 
[airflow/operators/email\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZW1haWxfb3BlcmF0b3IucHk=)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[airflow/hooks/pig\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9waWdfaG9vay5weQ==)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[airflow/operators/redshift\_to\_s3\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcmVkc2hpZnRfdG9fczNfb3BlcmF0b3IucHk=)
 | `0% <0%> (-95.56%)` | :arrow_down: |
   | 
[airflow/hooks/jdbc\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9qZGJjX2hvb2sucHk=)
 | `0% <0%> (-94.45%)` | :arrow_down: |
   | 
[airflow/operators/s3\_file\_transform\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvczNfZmlsZV90cmFuc2Zvcm1fb3BlcmF0b3IucHk=)
 | `0% <0%> (-93.88%)` | :arrow_down: |
   | 
[airflow/hooks/druid\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kcnVpZF9ob29rLnB5)
 | `0% <0%> (-87.68%)` | :arrow_down: |
   | 
[airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5)
 | `0% <0%> (-80.62%)` | :arrow_down: |
   | 
[airflow/hooks/S3\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9TM19ob29rLnB5)
 | `22.27% <0%> (-72.05%)` | :arrow_down: |
   | 
[airflow/hooks/mssql\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9tc3NxbF9ob29rLnB5)
 | `6.66% <0%> (-66.67%)` | :arrow_down: |
   | ... and [198 
more](https://codecov.io/gh/apache/incubator-airflow/pull/2442/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2442?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2442?src=pr=footer).
 Last update 
[304c2af...1ba7b13](https://codecov.io/gh/apache/incubator-airflow/pull/2442?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] feng-tao commented on issue #3972: [AIRFLOW-XXX] Add Compass to companies list

[GitBox]

feng-tao commented on issue #3972: [AIRFLOW-XXX] Add Compass to companies list
URL: 
https://github.com/apache/incubator-airflow/pull/3972#issuecomment-425535433
 
 
   lgtm


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] feng-tao closed pull request #3972: [AIRFLOW-XXX] Add Compass to companies list

[GitBox]

feng-tao closed pull request #3972: [AIRFLOW-XXX] Add Compass to companies list
URL: https://github.com/apache/incubator-airflow/pull/3972
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/README.md b/README.md
index ee85e56e8c..7fb856c05e 100644
--- a/README.md
+++ b/README.md
@@ -135,6 +135,7 @@ Currently **officially** using Airflow:
 1. [Clover Health](https://www.cloverhealth.com) 
[[@gwax](https://github.com/gwax) & 
[@vansivallab](https://github.com/vansivallab)]
 1. [Chartboost](https://www.chartboost.com) 
[[@cgelman](https://github.com/cgelman) & [@dclubb](https://github.com/dclubb)]
 1. [Collectivehealth Inc.](https://www.collectivehealth.com) 
[@retornam](https://github.com/retornam)
+1. [Compass](https://www.compass.com) 
[[@wdhorton](https://github.com/wdhorton)]
 1. [ContaAzul](https://www.contaazul.com) 
[[@bern4rdelli](https://github.com/bern4rdelli), 
[@renanleme](https://github.com/renanleme) & 
[@sabino](https://github.com/sabino)]
 1. [Cotap](https://github.com/cotap/) [[@maraca](https://github.com/maraca) & 
[@richardchew](https://github.com/richardchew)]
 1. [Craig@Work](https://www.craigatwork.com)


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3972: [AIRFLOW-XXX] Add Compass to companies list

[GitBox]

codecov-io commented on issue #3972: [AIRFLOW-XXX] Add Compass to companies list
URL: 
https://github.com/apache/incubator-airflow/pull/3972#issuecomment-425529968
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=h1)
 Report
   > Merging 
[#3972](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/023c37ae8dc8cde812be6929957ebf6385b2a7cb?src=pr=desc)
 will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3972/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=tree)
   
   ```diff
   @@   Coverage Diff   @@
   ##   master#3972   +/-   ##
   ===
 Coverage   75.04%   75.04%   
   ===
 Files 199  199   
 Lines   1596515965   
   ===
 Hits1198111981   
 Misses   3984 3984
   ```
   
   
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=footer).
 Last update 
[023c37a...7619927](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3972: [AIRFLOW-XXX] Add Compass to companies list

[GitBox]

codecov-io edited a comment on issue #3972: [AIRFLOW-XXX] Add Compass to 
companies list
URL: 
https://github.com/apache/incubator-airflow/pull/3972#issuecomment-425529968
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=h1)
 Report
   > Merging 
[#3972](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/023c37ae8dc8cde812be6929957ebf6385b2a7cb?src=pr=desc)
 will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3972/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=tree)
   
   ```diff
   @@   Coverage Diff   @@
   ##   master#3972   +/-   ##
   ===
 Coverage   75.04%   75.04%   
   ===
 Files 199  199   
 Lines   1596515965   
   ===
 Hits1198111981   
 Misses   3984 3984
   ```
   
   
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=footer).
 Last update 
[023c37a...7619927](https://codecov.io/gh/apache/incubator-airflow/pull/3972?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #2372: [AIRFLOW-393] Add callback for FTP downloads

[GitBox]

codecov-io edited a comment on issue #2372: [AIRFLOW-393] Add callback for FTP 
downloads
URL: 
https://github.com/apache/incubator-airflow/pull/2372#issuecomment-309161862
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=h1)
 Report
   > Merging 
[#2372](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/023c37ae8dc8cde812be6929957ebf6385b2a7cb?src=pr=desc)
 will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/2372/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=tree)
   
   ```diff
   @@   Coverage Diff   @@
   ##   master#2372   +/-   ##
   ===
 Coverage   75.04%   75.04%   
   ===
 Files 199  199   
 Lines   1596515965   
   ===
 Hits1198111981   
 Misses   3984 3984
   ```
   
   
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=footer).
 Last update 
[023c37a...9325330](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #2372: [AIRFLOW-393] Add callback for FTP downloads

[GitBox]

codecov-io edited a comment on issue #2372: [AIRFLOW-393] Add callback for FTP 
downloads
URL: 
https://github.com/apache/incubator-airflow/pull/2372#issuecomment-309161862
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=h1)
 Report
   > Merging 
[#2372](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/023c37ae8dc8cde812be6929957ebf6385b2a7cb?src=pr=desc)
 will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/2372/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=tree)
   
   ```diff
   @@   Coverage Diff   @@
   ##   master#2372   +/-   ##
   ===
 Coverage   75.04%   75.04%   
   ===
 Files 199  199   
 Lines   1596515965   
   ===
 Hits1198111981   
 Misses   3984 3984
   ```
   
   
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=footer).
 Last update 
[023c37a...9325330](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wdhorton opened a new pull request #3972: [AIRFLOW-XXX] Add Compass to companies list

[GitBox]

wdhorton opened a new pull request #3972: [AIRFLOW-XXX] Add Compass to 
companies list
URL: https://github.com/apache/incubator-airflow/pull/3972
 
 
   We're using Airflow at Compass now!
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] YingboWang commented on issue #3798: [AIRFLOW-2951] Update dag_run table end_date when state change

[GitBox]

YingboWang commented on issue #3798: [AIRFLOW-2951] Update dag_run table 
end_date when state change
URL: 
https://github.com/apache/incubator-airflow/pull/3798#issuecomment-425510483
 
 
   Sure. Let me take a look this weekend.
   
   On Tue, Sep 25, 2018 at 11:44 AM Ash Berlin-Taylor 
   wrote:
   
   > This one would be good to get in to 1.10, but the experimental API looks
   > quite different and is conflicting - @YingboWang
   >  if you have time would you be able to see
   > if it's easy to backport this to v1.10-test branch?
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or mute the thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #2372: [AIRFLOW-393] Add callback for FTP downloads

[GitBox]

codecov-io edited a comment on issue #2372: [AIRFLOW-393] Add callback for FTP 
downloads
URL: 
https://github.com/apache/incubator-airflow/pull/2372#issuecomment-309161862
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=h1)
 Report
   > Merging 
[#2372](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/4c572a4b2dd16725d0223fc6a08cd8550ba9cb96?src=pr=desc)
 will **increase** coverage by `<.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/2372/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#2372  +/-   ##
   ==
   + Coverage   74.95%   74.95%   +<.01% 
   ==
 Files 199  199  
 Lines   1596515965  
   ==
   + Hits1196611967   +1 
   + Misses   3999 3998   -1
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/configuration.py](https://codecov.io/gh/apache/incubator-airflow/pull/2372/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5)
 | `89.25% <0%> (+0.37%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=footer).
 Last update 
[4c572a4...0ebd197](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #2372: [AIRFLOW-393] Add callback for FTP downloads

[GitBox]

codecov-io edited a comment on issue #2372: [AIRFLOW-393] Add callback for FTP 
downloads
URL: 
https://github.com/apache/incubator-airflow/pull/2372#issuecomment-309161862
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=h1)
 Report
   > Merging 
[#2372](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/4c572a4b2dd16725d0223fc6a08cd8550ba9cb96?src=pr=desc)
 will **increase** coverage by `<.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/2372/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#2372  +/-   ##
   ==
   + Coverage   74.95%   74.95%   +<.01% 
   ==
 Files 199  199  
 Lines   1596515965  
   ==
   + Hits1196611967   +1 
   + Misses   3999 3998   -1
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/configuration.py](https://codecov.io/gh/apache/incubator-airflow/pull/2372/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5)
 | `89.25% <0%> (+0.37%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=footer).
 Last update 
[4c572a4...0ebd197](https://codecov.io/gh/apache/incubator-airflow/pull/2372?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3124) Broken webserver debug mode (RBAC)

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632142#comment-16632142
 ] 

ASF GitHub Bot commented on AIRFLOW-3124:
-

feng-tao closed pull request #3958: [AIRFLOW-3124] Fix RBAC webserver debug mode
URL: https://github.com/apache/incubator-airflow/pull/3958
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/bin/cli.py b/airflow/bin/cli.py
index 1c5494ead1..09bd0c1806 100644
--- a/airflow/bin/cli.py
+++ b/airflow/bin/cli.py
@@ -870,8 +870,12 @@ def webserver(args):
 print(
 "Starting the web server on port {0} and host {1}.".format(
 args.port, args.hostname))
-app = create_app_rbac(conf) if settings.RBAC else create_app(conf)
-app.run(debug=True, port=args.port, host=args.hostname,
+if settings.RBAC:
+app, _ = create_app_rbac(conf, testing=conf.get('core', 
'unit_test_mode'))
+else:
+app = create_app(conf, testing=conf.get('core', 'unit_test_mode'))
+app.run(debug=True, use_reloader=False if app.config['TESTING'] else 
True,
+port=args.port, host=args.hostname,
 ssl_context=(ssl_cert, ssl_key) if ssl_cert and ssl_key else 
None)
 else:
 os.environ['SKIP_DAGS_PARSING'] = 'True'
diff --git a/tests/cli/test_cli.py b/tests/cli/test_cli.py
index 93ec0576e6..aeafdd85fe 100644
--- a/tests/cli/test_cli.py
+++ b/tests/cli/test_cli.py
@@ -139,7 +139,21 @@ def test_ready_prefix_on_cmdline_dead_process(self):
 
self.assertEqual(get_num_ready_workers_running(self.gunicorn_master_proc), 0)
 
 def test_cli_webserver_debug(self):
-p = psutil.Popen(["airflow", "webserver", "-d"])
+env = os.environ.copy()
+p = psutil.Popen(["airflow", "webserver", "-d"], env=env)
+sleep(3)  # wait for webserver to start
+return_code = p.poll()
+self.assertEqual(
+None,
+return_code,
+"webserver terminated with return code {} in debug 
mode".format(return_code))
+p.terminate()
+p.wait()
+
+def test_cli_rbac_webserver_debug(self):
+env = os.environ.copy()
+env['AIRFLOW__WEBSERVER__RBAC'] = 'True'
+p = psutil.Popen(["airflow", "webserver", "-d"], env=env)
 sleep(3)  # wait for webserver to start
 return_code = p.poll()
 self.assertEqual(


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Broken webserver debug mode (RBAC)
> --
>
> Key: AIRFLOW-3124
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3124
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: webapp, webserver
>Reporter: Aaron Kosel
>Assignee: Aaron Kosel
>Priority: Minor
>
> {code:java}
> Traceback (most recent call last):
> File "/usr/local/bin/airflow", line 7, in 
> exec(compile(f.read(), __file__, 'exec'))
> File "/airflow/airflow/bin/airflow", line 32, in 
> args.func(args)
> File "/airflow/airflow/utils/cli.py", line 74, in wrapper
> return f(*args, **kwargs)
> File "/airflow/airflow/bin/cli.py", line 875, in webserver
> app.run(debug=True, port=args.port, host=args.hostname,
> AttributeError: 'tuple' object has no attribute 'run'
> {code}
> Nearly the same issue as https://issues.apache.org/jira/browse/AIRFLOW-2204, 
> but only affecting RBAC debug mode. The problem is that `create_app` returns 
> a tuple, but the `cli` script expects to just receive the flask app back 
> without the appbuilder.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3129) Improve test coverage

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632140#comment-16632140
 ] 

ASF GitHub Bot commented on AIRFLOW-3129:
-

feng-tao closed pull request #3970: [AIRFLOW-3129] Backfill mysql hook unit 
tests.
URL: https://github.com/apache/incubator-airflow/pull/3970
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/tests/hooks/test_mysql_hook.py b/tests/hooks/test_mysql_hook.py
index d112f880e9..415d430fc4 100644
--- a/tests/hooks/test_mysql_hook.py
+++ b/tests/hooks/test_mysql_hook.py
@@ -18,12 +18,81 @@
 # under the License.
 #
 
+import json
 import mock
 import unittest
 
+import MySQLdb.cursors
+
+from airflow import models
 from airflow.hooks.mysql_hook import MySqlHook
 
 
+class TestMySqlHookConn(unittest.TestCase):
+
+def setUp(self):
+super(TestMySqlHookConn, self).setUp()
+
+self.connection = models.Connection(
+login='login',
+password='password',
+host='host',
+schema='schema',
+)
+
+self.db_hook = MySqlHook()
+self.db_hook.get_connection = mock.Mock()
+self.db_hook.get_connection.return_value = self.connection
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn(self, mock_connect):
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['user'], 'login')
+self.assertEqual(kwargs['passwd'], 'password')
+self.assertEqual(kwargs['host'], 'host')
+self.assertEqual(kwargs['db'], 'schema')
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn_port(self, mock_connect):
+self.connection.port = 3307
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['port'], 3307)
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn_charset(self, mock_connect):
+self.connection.extra = json.dumps({'charset': 'utf-8'})
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['charset'], 'utf-8')
+self.assertEqual(kwargs['use_unicode'], True)
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn_cursor(self, mock_connect):
+self.connection.extra = json.dumps({'cursor': 'sscursor'})
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['cursorclass'], MySQLdb.cursors.SSCursor)
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn_local_infile(self, mock_connect):
+self.connection.extra = json.dumps({'local_infile': True})
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['local_infile'], 1)
+
+
 class TestMySqlHook(unittest.TestCase):
 
 def setUp(self):
@@ -85,3 +154,20 @@ def test_run_multi_queries(self):
 self.assertEqual(kwargs, {})
 self.cur.execute.assert_called_with(sql[1])
 self.conn.commit.assert_not_called()
+
+def test_bulk_load(self):
+self.db_hook.bulk_load('table', '/tmp/file')
+self.cur.execute.assert_called_once_with("""
+LOAD DATA LOCAL INFILE '/tmp/file'
+INTO TABLE table
+""")
+
+def test_bulk_dump(self):
+self.db_hook.bulk_dump('table', '/tmp/file')
+self.cur.execute.assert_called_once_with("""
+SELECT * INTO OUTFILE '/tmp/file'
+FROM table
+""")
+
+def test_serialize_cell(self):
+self.assertEqual('foo', self.db_hook._serialize_cell('foo', None))


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Improve test coverage
> -
>
> Key: AIRFLOW-3129
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3129
> Project: Apache Airflow
>  Issue Type: Improvement
>  

[GitHub] feng-tao commented on issue #3958: [AIRFLOW-3124] Fix RBAC webserver debug mode

[GitBox]

feng-tao commented on issue #3958: [AIRFLOW-3124] Fix RBAC webserver debug mode
URL: 
https://github.com/apache/incubator-airflow/pull/3958#issuecomment-425503528
 
 
   CI pass


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] feng-tao closed pull request #3958: [AIRFLOW-3124] Fix RBAC webserver debug mode

[GitBox]

feng-tao closed pull request #3958: [AIRFLOW-3124] Fix RBAC webserver debug mode
URL: https://github.com/apache/incubator-airflow/pull/3958
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/bin/cli.py b/airflow/bin/cli.py
index 1c5494ead1..09bd0c1806 100644
--- a/airflow/bin/cli.py
+++ b/airflow/bin/cli.py
@@ -870,8 +870,12 @@ def webserver(args):
 print(
 "Starting the web server on port {0} and host {1}.".format(
 args.port, args.hostname))
-app = create_app_rbac(conf) if settings.RBAC else create_app(conf)
-app.run(debug=True, port=args.port, host=args.hostname,
+if settings.RBAC:
+app, _ = create_app_rbac(conf, testing=conf.get('core', 
'unit_test_mode'))
+else:
+app = create_app(conf, testing=conf.get('core', 'unit_test_mode'))
+app.run(debug=True, use_reloader=False if app.config['TESTING'] else 
True,
+port=args.port, host=args.hostname,
 ssl_context=(ssl_cert, ssl_key) if ssl_cert and ssl_key else 
None)
 else:
 os.environ['SKIP_DAGS_PARSING'] = 'True'
diff --git a/tests/cli/test_cli.py b/tests/cli/test_cli.py
index 93ec0576e6..aeafdd85fe 100644
--- a/tests/cli/test_cli.py
+++ b/tests/cli/test_cli.py
@@ -139,7 +139,21 @@ def test_ready_prefix_on_cmdline_dead_process(self):
 
self.assertEqual(get_num_ready_workers_running(self.gunicorn_master_proc), 0)
 
 def test_cli_webserver_debug(self):
-p = psutil.Popen(["airflow", "webserver", "-d"])
+env = os.environ.copy()
+p = psutil.Popen(["airflow", "webserver", "-d"], env=env)
+sleep(3)  # wait for webserver to start
+return_code = p.poll()
+self.assertEqual(
+None,
+return_code,
+"webserver terminated with return code {} in debug 
mode".format(return_code))
+p.terminate()
+p.wait()
+
+def test_cli_rbac_webserver_debug(self):
+env = os.environ.copy()
+env['AIRFLOW__WEBSERVER__RBAC'] = 'True'
+p = psutil.Popen(["airflow", "webserver", "-d"], env=env)
 sleep(3)  # wait for webserver to start
 return_code = p.poll()
 self.assertEqual(


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] feng-tao commented on issue #3970: [AIRFLOW-3129] Backfill mysql hook unit tests.

[GitBox]

feng-tao commented on issue #3970: [AIRFLOW-3129] Backfill mysql hook unit 
tests.
URL: 
https://github.com/apache/incubator-airflow/pull/3970#issuecomment-425503325
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] feng-tao closed pull request #3970: [AIRFLOW-3129] Backfill mysql hook unit tests.

[GitBox]

feng-tao closed pull request #3970: [AIRFLOW-3129] Backfill mysql hook unit 
tests.
URL: https://github.com/apache/incubator-airflow/pull/3970
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/tests/hooks/test_mysql_hook.py b/tests/hooks/test_mysql_hook.py
index d112f880e9..415d430fc4 100644
--- a/tests/hooks/test_mysql_hook.py
+++ b/tests/hooks/test_mysql_hook.py
@@ -18,12 +18,81 @@
 # under the License.
 #
 
+import json
 import mock
 import unittest
 
+import MySQLdb.cursors
+
+from airflow import models
 from airflow.hooks.mysql_hook import MySqlHook
 
 
+class TestMySqlHookConn(unittest.TestCase):
+
+def setUp(self):
+super(TestMySqlHookConn, self).setUp()
+
+self.connection = models.Connection(
+login='login',
+password='password',
+host='host',
+schema='schema',
+)
+
+self.db_hook = MySqlHook()
+self.db_hook.get_connection = mock.Mock()
+self.db_hook.get_connection.return_value = self.connection
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn(self, mock_connect):
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['user'], 'login')
+self.assertEqual(kwargs['passwd'], 'password')
+self.assertEqual(kwargs['host'], 'host')
+self.assertEqual(kwargs['db'], 'schema')
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn_port(self, mock_connect):
+self.connection.port = 3307
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['port'], 3307)
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn_charset(self, mock_connect):
+self.connection.extra = json.dumps({'charset': 'utf-8'})
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['charset'], 'utf-8')
+self.assertEqual(kwargs['use_unicode'], True)
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn_cursor(self, mock_connect):
+self.connection.extra = json.dumps({'cursor': 'sscursor'})
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['cursorclass'], MySQLdb.cursors.SSCursor)
+
+@mock.patch('airflow.hooks.mysql_hook.MySQLdb.connect')
+def test_get_conn_local_infile(self, mock_connect):
+self.connection.extra = json.dumps({'local_infile': True})
+self.db_hook.get_conn()
+mock_connect.assert_called_once()
+args, kwargs = mock_connect.call_args
+self.assertEqual(args, ())
+self.assertEqual(kwargs['local_infile'], 1)
+
+
 class TestMySqlHook(unittest.TestCase):
 
 def setUp(self):
@@ -85,3 +154,20 @@ def test_run_multi_queries(self):
 self.assertEqual(kwargs, {})
 self.cur.execute.assert_called_with(sql[1])
 self.conn.commit.assert_not_called()
+
+def test_bulk_load(self):
+self.db_hook.bulk_load('table', '/tmp/file')
+self.cur.execute.assert_called_once_with("""
+LOAD DATA LOCAL INFILE '/tmp/file'
+INTO TABLE table
+""")
+
+def test_bulk_dump(self):
+self.db_hook.bulk_dump('table', '/tmp/file')
+self.cur.execute.assert_called_once_with("""
+SELECT * INTO OUTFILE '/tmp/file'
+FROM table
+""")
+
+def test_serialize_cell(self):
+self.assertEqual('foo', self.db_hook._serialize_cell('foo', None))


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3125) Add monitoring on Task Instance creation rate

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632138#comment-16632138
 ] 

ASF GitHub Bot commented on AIRFLOW-3125:
-

feng-tao closed pull request #3966: [AIRFLOW-3125] Monitor Task Instances 
creation rates
URL: https://github.com/apache/incubator-airflow/pull/3966
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/models.py b/airflow/models.py
index e4a50bc476..9d1d64ee73 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -5282,6 +5282,9 @@ def verify_integrity(self, session=None):
 continue
 
 if task.task_id not in task_ids:
+Stats.incr(
+"task_instance_created-{}".format(task.__class__.__name__),
+1, 1)
 ti = TaskInstance(task, self.execution_date)
 session.add(ti)
 


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add monitoring on Task Instance creation rate
> -
>
> Key: AIRFLOW-3125
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3125
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Mingye Xia
>Assignee: Mingye Xia
>Priority: Major
>
> Monitoring on Task Instance creation rate can give us some visibility on how 
> much workload we are putting on Airflow. It can be used for resource 
> allocation in the long run (i.e. to determine when we should scale up 
> workers) and and debugging in scenarios like creation rate for certain types 
> of Task Instances spike.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] feng-tao closed pull request #3966: [AIRFLOW-3125] Monitor Task Instances creation rates

[GitBox]

feng-tao closed pull request #3966: [AIRFLOW-3125] Monitor Task Instances 
creation rates
URL: https://github.com/apache/incubator-airflow/pull/3966
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/models.py b/airflow/models.py
index e4a50bc476..9d1d64ee73 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -5282,6 +5282,9 @@ def verify_integrity(self, session=None):
 continue
 
 if task.task_id not in task_ids:
+Stats.incr(
+"task_instance_created-{}".format(task.__class__.__name__),
+1, 1)
 ti = TaskInstance(task, self.execution_date)
 session.add(ti)
 


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] feng-tao commented on issue #3966: [AIRFLOW-3125] Monitor Task Instances creation rates

[GitBox]

feng-tao commented on issue #3966: [AIRFLOW-3125] Monitor Task Instances 
creation rates
URL: 
https://github.com/apache/incubator-airflow/pull/3966#issuecomment-425502916
 
 
   merging now.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3123) Allow nested use of DAG as a context manager

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632125#comment-16632125
 ] 

ASF GitHub Bot commented on AIRFLOW-3123:
-

aoen closed pull request #3956: [AIRFLOW-3123] Allow nested use of  DAG as a 
context manager
URL: https://github.com/apache/incubator-airflow/pull/3956
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/models.py b/airflow/models.py
index a2de010c51..e4dffd3b47 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -3391,7 +3391,7 @@ def __init__(
 self.on_success_callback = on_success_callback
 self.on_failure_callback = on_failure_callback
 
-self._context_manager_set = False
+self._old_context_manager_dags = []
 
 self._comps = {
 'dag_id',
@@ -3440,16 +3440,13 @@ def __hash__(self):
 
 def __enter__(self):
 global _CONTEXT_MANAGER_DAG
-if not self._context_manager_set:
-self._old_context_manager_dag = _CONTEXT_MANAGER_DAG
-_CONTEXT_MANAGER_DAG = self
-self._context_manager_set = True
+self._old_context_manager_dags.append(_CONTEXT_MANAGER_DAG)
+_CONTEXT_MANAGER_DAG = self
 return self
 
 def __exit__(self, _type, _value, _tb):
 global _CONTEXT_MANAGER_DAG
-_CONTEXT_MANAGER_DAG = self._old_context_manager_dag
-self._context_manager_set = False
+_CONTEXT_MANAGER_DAG = self._old_context_manager_dags.pop()
 
 # /Context Manager --
 
diff --git a/tests/models.py b/tests/models.py
index 999b1be1bb..7e3c2929e0 100644
--- a/tests/models.py
+++ b/tests/models.py
@@ -141,11 +141,13 @@ def test_dag_as_context_manager(self):
 with dag:
 with dag:
 op7 = DummyOperator(task_id='op7')
-op8 = DummyOperator(task_id='op8')
-op8.dag = dag2
+op8 = DummyOperator(task_id='op8')
+op9 = DummyOperator(task_id='op8')
+op9.dag = dag2
 
 self.assertEqual(op7.dag, dag)
-self.assertEqual(op8.dag, dag2)
+self.assertEqual(op8.dag, dag)
+self.assertEqual(op9.dag, dag2)
 
 def test_dag_topological_sort(self):
 dag = DAG(


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow nested use of  DAG as a context manager
> -
>
> Key: AIRFLOW-3123
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3123
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: DAG
>Reporter: Newton Le
>Assignee: Newton Le
>Priority: Major
>
> DAG context manager fails under some cases with nested contexts:
> {code:python}
> with DAG( ... ) as dag:
>   op1 = Operator()
>   with dag:
> op2 = Operator()
>   op3 = Operator
> {code}
> op3 will not continue to be assigned the original DAG after exiting the 
> nested context.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] aoen closed pull request #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

aoen closed pull request #3956: [AIRFLOW-3123] Allow nested use of  DAG as a 
context manager
URL: https://github.com/apache/incubator-airflow/pull/3956
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/models.py b/airflow/models.py
index a2de010c51..e4dffd3b47 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -3391,7 +3391,7 @@ def __init__(
 self.on_success_callback = on_success_callback
 self.on_failure_callback = on_failure_callback
 
-self._context_manager_set = False
+self._old_context_manager_dags = []
 
 self._comps = {
 'dag_id',
@@ -3440,16 +3440,13 @@ def __hash__(self):
 
 def __enter__(self):
 global _CONTEXT_MANAGER_DAG
-if not self._context_manager_set:
-self._old_context_manager_dag = _CONTEXT_MANAGER_DAG
-_CONTEXT_MANAGER_DAG = self
-self._context_manager_set = True
+self._old_context_manager_dags.append(_CONTEXT_MANAGER_DAG)
+_CONTEXT_MANAGER_DAG = self
 return self
 
 def __exit__(self, _type, _value, _tb):
 global _CONTEXT_MANAGER_DAG
-_CONTEXT_MANAGER_DAG = self._old_context_manager_dag
-self._context_manager_set = False
+_CONTEXT_MANAGER_DAG = self._old_context_manager_dags.pop()
 
 # /Context Manager --
 
diff --git a/tests/models.py b/tests/models.py
index 999b1be1bb..7e3c2929e0 100644
--- a/tests/models.py
+++ b/tests/models.py
@@ -141,11 +141,13 @@ def test_dag_as_context_manager(self):
 with dag:
 with dag:
 op7 = DummyOperator(task_id='op7')
-op8 = DummyOperator(task_id='op8')
-op8.dag = dag2
+op8 = DummyOperator(task_id='op8')
+op9 = DummyOperator(task_id='op8')
+op9.dag = dag2
 
 self.assertEqual(op7.dag, dag)
-self.assertEqual(op8.dag, dag2)
+self.assertEqual(op8.dag, dag)
+self.assertEqual(op9.dag, dag2)
 
 def test_dag_topological_sort(self):
 dag = DAG(


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mingyexia commented on issue #3966: [AIRFLOW-3125] Monitor Task Instances creation rates

[GitBox]

mingyexia commented on issue #3966: [AIRFLOW-3125] Monitor Task Instances 
creation rates
URL: 
https://github.com/apache/incubator-airflow/pull/3966#issuecomment-42549
 
 
   @ashb I tried to find other places where TIs are created yet everything I've 
found used this function. But I'm very new to Airflow, might have missed some 
cases. Will keep an eye on it :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kaxil commented on a change in pull request #3971: [AIRFLOW-3130] Add CLI docs for users command

[GitBox]

kaxil commented on a change in pull request #3971: [AIRFLOW-3130] Add CLI docs 
for users command
URL: https://github.com/apache/incubator-airflow/pull/3971#discussion_r221314021
 
 

 ##
 File path: UPDATING.md
 ##
 @@ -31,6 +31,27 @@ some bugs.
 The new `sync_parallelism` config option will control how many processes 
CeleryExecutor will use to
 fetch celery task state in parallel. Default value is max(1, number of cores - 
1)
 
+### CLI Changes
+
+The ability to manipulate users from the command line has been changed. 
'airflow create_user' and 'airflow delete_user' and 'airflow list_users' has 
been grouped to a single command `airflow users` with optional flags 
`--create`, `--list` and `--delete`.
+
+Example Usage:
+
+To create a new user:
+```bash
+airflow users --create --username jondoe --lastname doe --firstname jon 
--email j...@apache.org --role Viewer --password test
+```
+
+To list users:
+```bash
+airflow users --list
+```
+
+To delete a user:
+```bash
+airflow users --username jondoe
 
 Review comment:
   No, it should be neither `airflow users --delete --username jondoe`. Have 
updated it in the master :) Thanks for catching this docs issues.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] akosel commented on a change in pull request #2372: [AIRFLOW-393] Add callback for FTP downloads

[GitBox]

akosel commented on a change in pull request #2372: [AIRFLOW-393] Add callback 
for FTP downloads
URL: https://github.com/apache/incubator-airflow/pull/2372#discussion_r221313853
 
 

 ##
 File path: airflow/contrib/hooks/ftp_hook.py
 ##
 @@ -161,6 +165,9 @@ def retrieve_file(self, remote_full_path, 
local_full_path_or_buffer):
 :param local_full_path_or_buffer: full path to the local file or a
 file-like buffer
 :type local_full_path_or_buffer: str or file-like buffer
+:param callback: callback that can be used for tracking download
+progress, among other things
+:type callback: callable
 
 Review comment:
   Good idea! I also added an example section to the docstring. If that seems 
excessive, I can remove it. Emphasis on the need to handle writing to the file 
inside of the custom callback if it is used.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/4968a52116e4a2a7512410d4b386909af77a5c0c?src=pr=desc)
 will **decrease** coverage by `0.01%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3956  +/-   ##
   ==
   - Coverage   74.97%   74.95%   -0.02% 
   ==
 Files 199  199  
 Lines   1596515962   -3 
   ==
   - Hits1196911964   -5 
   - Misses   3996 3998   +2
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.65% <100%> (+0.02%)` | :arrow_up: |
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.13% <0%> (-0.27%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[4968a52...17a6512](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/4968a52116e4a2a7512410d4b386909af77a5c0c?src=pr=desc)
 will **decrease** coverage by `0.01%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3956  +/-   ##
   ==
   - Coverage   74.97%   74.95%   -0.02% 
   ==
 Files 199  199  
 Lines   1596515962   -3 
   ==
   - Hits1196911964   -5 
   - Misses   3996 3998   +2
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.65% <100%> (+0.02%)` | :arrow_up: |
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.13% <0%> (-0.27%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[4968a52...17a6512](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] XD-DENG commented on a change in pull request #3971: [AIRFLOW-3130] Add CLI docs for users command

[GitBox]

XD-DENG commented on a change in pull request #3971: [AIRFLOW-3130] Add CLI 
docs for users command
URL: https://github.com/apache/incubator-airflow/pull/3971#discussion_r221300151
 
 

 ##
 File path: UPDATING.md
 ##
 @@ -31,6 +31,27 @@ some bugs.
 The new `sync_parallelism` config option will control how many processes 
CeleryExecutor will use to
 fetch celery task state in parallel. Default value is max(1, number of cores - 
1)
 
+### CLI Changes
+
+The ability to manipulate users from the command line has been changed. 
'airflow create_user' and 'airflow delete_user' and 'airflow list_users' has 
been grouped to a single command `airflow users` with optional flags 
`--create`, `--list` and `--delete`.
+
+Example Usage:
+
+To create a new user:
+```bash
+airflow users --create --username jondoe --lastname doe --firstname jon 
--email j...@apache.org --role Viewer --password test
+```
+
+To list users:
+```bash
+airflow users --list
+```
+
+To delete a user:
+```bash
+airflow users --username jondoe
 
 Review comment:
   Should this be `airflow users --delete jondoe`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Closed] (AIRFLOW-3130) Add CLI docs for users command

[Siddharth Anand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siddharth Anand closed AIRFLOW-3130.


> Add CLI docs for users command
> --
>
> Key: AIRFLOW-3130
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3130
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (AIRFLOW-3130) Add CLI docs for users command

[Siddharth Anand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siddharth Anand resolved AIRFLOW-3130.
--
Resolution: Fixed

> Add CLI docs for users command
> --
>
> Key: AIRFLOW-3130
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3130
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] XD-DENG commented on issue #3936: [AIRFLOW-3051] Change CLI to make users ops similar to connections

[GitBox]

XD-DENG commented on issue #3936: [AIRFLOW-3051] Change CLI to make users ops 
similar to connections
URL: 
https://github.com/apache/incubator-airflow/pull/3936#issuecomment-425479803
 
 
   Cool! Thanks @kaxil 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3130) Add CLI docs for users command

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632039#comment-16632039
 ] 

ASF GitHub Bot commented on AIRFLOW-3130:
-

r39132 closed pull request #3971: [AIRFLOW-3130] Add CLI docs for users command
URL: https://github.com/apache/incubator-airflow/pull/3971
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/UPDATING.md b/UPDATING.md
index 93ea069838..110c488197 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -31,6 +31,27 @@ some bugs.
 The new `sync_parallelism` config option will control how many processes 
CeleryExecutor will use to
 fetch celery task state in parallel. Default value is max(1, number of cores - 
1)
 
+### CLI Changes
+
+The ability to manipulate users from the command line has been changed. 
'airflow create_user' and 'airflow delete_user' and 'airflow list_users' has 
been grouped to a single command `airflow users` with optional flags 
`--create`, `--list` and `--delete`.
+
+Example Usage:
+
+To create a new user:
+```bash
+airflow users --create --username jondoe --lastname doe --firstname jon 
--email j...@apache.org --role Viewer --password test
+```
+
+To list users:
+```bash
+airflow users --list
+```
+
+To delete a user:
+```bash
+airflow users --username jondoe
+```
+
 ## Airflow 1.10
 
 Installation and upgrading requires setting `SLUGIFY_USES_TEXT_UNIDECODE=yes` 
in your environment or
diff --git a/docs/security.rst b/docs/security.rst
index 89b5605daa..23f7cc0303 100644
--- a/docs/security.rst
+++ b/docs/security.rst
@@ -19,7 +19,7 @@ Password
 .. note::
 
This is for flask-admin based web UI only. If you are using FAB-based web 
UI with RBAC feature,
-   please use command line interface ``create_user`` to create accounts, or do 
that in the FAB-based UI itself.
+   please use command line interface ``airflow users --create`` to create 
accounts, or do that in the FAB-based UI itself.
 
 One of the simplest mechanisms for authentication is requiring users to 
specify a password before logging in.
 Password authentication requires the used of the ``password`` subpackage in 
your requirements file. Password hashing


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add CLI docs for users command
> --
>
> Key: AIRFLOW-3130
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3130
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] r39132 closed pull request #3971: [AIRFLOW-3130] Add CLI docs for users command

[GitBox]

r39132 closed pull request #3971: [AIRFLOW-3130] Add CLI docs for users command
URL: https://github.com/apache/incubator-airflow/pull/3971
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/UPDATING.md b/UPDATING.md
index 93ea069838..110c488197 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -31,6 +31,27 @@ some bugs.
 The new `sync_parallelism` config option will control how many processes 
CeleryExecutor will use to
 fetch celery task state in parallel. Default value is max(1, number of cores - 
1)
 
+### CLI Changes
+
+The ability to manipulate users from the command line has been changed. 
'airflow create_user' and 'airflow delete_user' and 'airflow list_users' has 
been grouped to a single command `airflow users` with optional flags 
`--create`, `--list` and `--delete`.
+
+Example Usage:
+
+To create a new user:
+```bash
+airflow users --create --username jondoe --lastname doe --firstname jon 
--email j...@apache.org --role Viewer --password test
+```
+
+To list users:
+```bash
+airflow users --list
+```
+
+To delete a user:
+```bash
+airflow users --username jondoe
+```
+
 ## Airflow 1.10
 
 Installation and upgrading requires setting `SLUGIFY_USES_TEXT_UNIDECODE=yes` 
in your environment or
diff --git a/docs/security.rst b/docs/security.rst
index 89b5605daa..23f7cc0303 100644
--- a/docs/security.rst
+++ b/docs/security.rst
@@ -19,7 +19,7 @@ Password
 .. note::
 
This is for flask-admin based web UI only. If you are using FAB-based web 
UI with RBAC feature,
-   please use command line interface ``create_user`` to create accounts, or do 
that in the FAB-based UI itself.
+   please use command line interface ``airflow users --create`` to create 
accounts, or do that in the FAB-based UI itself.
 
 One of the simplest mechanisms for authentication is requiring users to 
specify a password before logging in.
 Password authentication requires the used of the ``password`` subpackage in 
your requirements file. Password hashing


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] r39132 commented on issue #3918: [AIRFLOW-3088] Include slack-compatible emoji image.

[GitBox]

r39132 commented on issue #3918: [AIRFLOW-3088] Include slack-compatible emoji 
image.
URL: 
https://github.com/apache/incubator-airflow/pull/3918#issuecomment-425479412
 
 
   Cool! @sethwoodworth 
   
Can you add `transparent` in the image name just to make it more obvious 
`airflow_64x64_emoji_transparent.png`? We currently have some png that do not 
have a transparent background.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kaxil edited a comment on issue #3936: [AIRFLOW-3051] Change CLI to make users ops similar to connections

[GitBox]

kaxil edited a comment on issue #3936: [AIRFLOW-3051] Change CLI to make users 
ops similar to connections
URL: 
https://github.com/apache/incubator-airflow/pull/3936#issuecomment-425479143
 
 
   @XD-DENG  Thanks. I have now added it in 
https://github.com/apache/incubator-airflow/pull/3971/files
   
   This would be released in Airflow 2.0


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kaxil commented on issue #3936: [AIRFLOW-3051] Change CLI to make users ops similar to connections

[GitBox]

kaxil commented on issue #3936: [AIRFLOW-3051] Change CLI to make users ops 
similar to connections
URL: 
https://github.com/apache/incubator-airflow/pull/3936#issuecomment-425479143
 
 
   @XD-DENG  Thanks. I have now added it in 
https://github.com/apache/incubator-airflow/pull/3971/files


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3130) Add CLI docs for users command

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632034#comment-16632034
 ] 

ASF GitHub Bot commented on AIRFLOW-3130:
-

kaxil opened a new pull request #3971: [AIRFLOW-3130] Add CLI docs for users 
command
URL: https://github.com/apache/incubator-airflow/pull/3971
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI 
changes:
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - When adding new operators/hooks/sensors, the autoclass documentation 
generation needs to be added.
   
   ### Code Quality
   
   - [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add CLI docs for users command
> --
>
> Key: AIRFLOW-3130
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3130
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] kaxil opened a new pull request #3971: [AIRFLOW-3130] Add CLI docs for users command

[GitBox]

kaxil opened a new pull request #3971: [AIRFLOW-3130] Add CLI docs for users 
command
URL: https://github.com/apache/incubator-airflow/pull/3971
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI 
changes:
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - When adding new operators/hooks/sensors, the autoclass documentation 
generation needs to be added.
   
   ### Code Quality
   
   - [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-3130) Add CLI docs for users command

[Kaxil Naik (JIRA)]

Kaxil Naik created AIRFLOW-3130:
---

 Summary: Add CLI docs for users command
 Key: AIRFLOW-3130
 URL: https://issues.apache.org/jira/browse/AIRFLOW-3130
 Project: Apache Airflow
  Issue Type: Improvement
Reporter: Kaxil Naik
Assignee: Kaxil Naik
 Fix For: 2.0.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] XD-DENG edited a comment on issue #3936: [AIRFLOW-3051] Change CLI to make users ops similar to connections

[GitBox]

XD-DENG edited a comment on issue #3936: [AIRFLOW-3051] Change CLI to make 
users ops similar to connections
URL: 
https://github.com/apache/incubator-airflow/pull/3936#issuecomment-425473276
 
 
   Hi @kaxil , when is this planned to be released? Since any documentation in 
which `airflow create_user` command is mentioned should be updated accordingly, 
which is not considered yet (and maybe they should only be updated when this 
new `airflow users` command is released officially?).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] XD-DENG commented on issue #3936: [AIRFLOW-3051] Change CLI to make users ops similar to connections

[GitBox]

XD-DENG commented on issue #3936: [AIRFLOW-3051] Change CLI to make users ops 
similar to connections
URL: 
https://github.com/apache/incubator-airflow/pull/3936#issuecomment-425473276
 
 
   Hi @kaxil , when is this planned to be released? Since any documentation in 
which `airflow create_user` command is mentioned should be updated accordingly, 
which is not considered yet (and they should only be updated when this new 
`airflow users` command is released officially).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3970: [AIRFLOW-3129] Backfill mysql hook unit tests.

[GitBox]

codecov-io edited a comment on issue #3970: [AIRFLOW-3129] Backfill mysql hook 
unit tests.
URL: 
https://github.com/apache/incubator-airflow/pull/3970#issuecomment-425455176
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=h1)
 Report
   > Merging 
[#3970](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/4968a52116e4a2a7512410d4b386909af77a5c0c?src=pr=desc)
 will **increase** coverage by `0.08%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3970/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3970  +/-   ##
   ==
   + Coverage   74.97%   75.05%   +0.08% 
   ==
 Files 199  199  
 Lines   1596515965  
   ==
   + Hits1196911982  +13 
   + Misses   3996 3983  -13
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3970/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.13% <0%> (-0.27%)` | :arrow_down: |
   | 
[airflow/hooks/mysql\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3970/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9teXNxbF9ob29rLnB5)
 | `90% <0%> (+32%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=footer).
 Last update 
[4968a52...31fa1a6](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3970: [AIRFLOW-3129] Backfill mysql hook unit tests.

[GitBox]

codecov-io commented on issue #3970: [AIRFLOW-3129] Backfill mysql hook unit 
tests.
URL: 
https://github.com/apache/incubator-airflow/pull/3970#issuecomment-425455176
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=h1)
 Report
   > Merging 
[#3970](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/4968a52116e4a2a7512410d4b386909af77a5c0c?src=pr=desc)
 will **increase** coverage by `0.08%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3970/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3970  +/-   ##
   ==
   + Coverage   74.97%   75.05%   +0.08% 
   ==
 Files 199  199  
 Lines   1596515965  
   ==
   + Hits1196911982  +13 
   + Misses   3996 3983  -13
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3970/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.13% <0%> (-0.27%)` | :arrow_down: |
   | 
[airflow/hooks/mysql\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3970/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9teXNxbF9ob29rLnB5)
 | `90% <0%> (+32%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=footer).
 Last update 
[4968a52...31fa1a6](https://codecov.io/gh/apache/incubator-airflow/pull/3970?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-3129) Improve test coverage

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631896#comment-16631896
 ] 

ASF GitHub Bot commented on AIRFLOW-3129:
-

jmcarp opened a new pull request #3970: [AIRFLOW-3129] Backfill mysql hook unit 
tests.
URL: https://github.com/apache/incubator-airflow/pull/3970
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI 
changes:
   
   Backfill missing tests from mysql hook.
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - When adding new operators/hooks/sensors, the autoclass documentation 
generation needs to be added.
   
   ### Code Quality
   
   - [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Improve test coverage
> -
>
> Key: AIRFLOW-3129
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3129
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Josh Carp
>Priority: Minor
>
> Overall test coverage is about 75%. It would be great to improve coverage. 
> I'll start by backfilling some missing tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jmcarp opened a new pull request #3970: [AIRFLOW-3129] Backfill mysql hook unit tests.

[GitBox]

jmcarp opened a new pull request #3970: [AIRFLOW-3129] Backfill mysql hook unit 
tests.
URL: https://github.com/apache/incubator-airflow/pull/3970
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI 
changes:
   
   Backfill missing tests from mysql hook.
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - When adding new operators/hooks/sensors, the autoclass documentation 
generation needs to be added.
   
   ### Code Quality
   
   - [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-3129) Improve test coverage

[Josh Carp (JIRA)]

Josh Carp created AIRFLOW-3129:
--

 Summary: Improve test coverage
 Key: AIRFLOW-3129
 URL: https://issues.apache.org/jira/browse/AIRFLOW-3129
 Project: Apache Airflow
  Issue Type: Improvement
Reporter: Josh Carp


Overall test coverage is about 75%. It would be great to improve coverage. I'll 
start by backfilling some missing tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] codecov-io edited a comment on issue #3969: [AIRFLOW-2912] Add Deploy and Delete operators for GCF

[GitBox]

codecov-io edited a comment on issue #3969: [AIRFLOW-2912] Add Deploy and 
Delete operators for GCF
URL: 
https://github.com/apache/incubator-airflow/pull/3969#issuecomment-425430405
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=h1)
 Report
   > Merging 
[#3969](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/dd4e3cfa7737803fb1c185c843151d1b11a27107?src=pr=desc)
 will **increase** coverage by `<.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3969/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3969  +/-   ##
   ==
   + Coverage   74.95%   74.95%   +<.01% 
   ==
 Files 199  199  
 Lines   1596515965  
   ==
   + Hits1196611967   +1 
   + Misses   3999 3998   -1
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3969/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.66% <0%> (+0.04%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=footer).
 Last update 
[dd4e3cf...0dcb3ab](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3969: [AIRFLOW-2912] Add Deploy and Delete operators for GCF

[GitBox]

codecov-io commented on issue #3969: [AIRFLOW-2912] Add Deploy and Delete 
operators for GCF
URL: 
https://github.com/apache/incubator-airflow/pull/3969#issuecomment-425430405
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=h1)
 Report
   > Merging 
[#3969](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/dd4e3cfa7737803fb1c185c843151d1b11a27107?src=pr=desc)
 will **increase** coverage by `<.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3969/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3969  +/-   ##
   ==
   + Coverage   74.95%   74.95%   +<.01% 
   ==
 Files 199  199  
 Lines   1596515965  
   ==
   + Hits1196611967   +1 
   + Misses   3999 3998   -1
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3969/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.66% <0%> (+0.04%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=footer).
 Last update 
[dd4e3cf...0dcb3ab](https://codecov.io/gh/apache/incubator-airflow/pull/3969?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3918: [AIRFLOW-3088] Include slack-compatible emoji image.

[GitBox]

codecov-io edited a comment on issue #3918: [AIRFLOW-3088] Include 
slack-compatible emoji image.
URL: 
https://github.com/apache/incubator-airflow/pull/3918#issuecomment-425424623
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=h1)
 Report
   > Merging 
[#3918](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/dd4e3cfa7737803fb1c185c843151d1b11a27107?src=pr=desc)
 will **increase** coverage by `0.03%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3918/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3918  +/-   ##
   ==
   + Coverage   74.95%   74.98%   +0.03% 
   ==
 Files 199  199  
 Lines   1596515965  
   ==
   + Hits1196611971   +5 
   + Misses   3999 3994   -5
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3918/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.66% <0%> (+0.04%)` | :arrow_up: |
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3918/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.48% <0%> (+0.35%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=footer).
 Last update 
[dd4e3cf...6a07d3f](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3918: [AIRFLOW-3088] Include slack-compatible emoji image.

[GitBox]

codecov-io commented on issue #3918: [AIRFLOW-3088] Include slack-compatible 
emoji image.
URL: 
https://github.com/apache/incubator-airflow/pull/3918#issuecomment-425424623
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=h1)
 Report
   > Merging 
[#3918](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/dd4e3cfa7737803fb1c185c843151d1b11a27107?src=pr=desc)
 will **increase** coverage by `0.03%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3918/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3918  +/-   ##
   ==
   + Coverage   74.95%   74.98%   +0.03% 
   ==
 Files 199  199  
 Lines   1596515965  
   ==
   + Hits1196611971   +5 
   + Misses   3999 3994   -5
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3918/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.66% <0%> (+0.04%)` | :arrow_up: |
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3918/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.48% <0%> (+0.35%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=footer).
 Last update 
[dd4e3cf...6a07d3f](https://codecov.io/gh/apache/incubator-airflow/pull/3918?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (AIRFLOW-3104) .airflowignore feature is not mentioned at all in documentation

[Ash Berlin-Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ash Berlin-Taylor resolved AIRFLOW-3104.

Resolution: Fixed

> .airflowignore feature is not mentioned at all in documentation
> ---
>
> Key: AIRFLOW-3104
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3104
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Minor
> Fix For: 2.0.0
>
>
> This is a nice feature, but not mentioned at all in anywhere of the 
> documentation.
> Relevant information can only be found in source code comment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AIRFLOW-3104) .airflowignore feature is not mentioned at all in documentation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631784#comment-16631784
 ] 

ASF GitHub Bot commented on AIRFLOW-3104:
-

ashb closed pull request #3939: [AIRFLOW-3104] Add .airflowignore info into doc
URL: https://github.com/apache/incubator-airflow/pull/3939
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/models.py b/airflow/models.py
index 6154ca64f4..6ea4638a51 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -522,10 +522,12 @@ def collect_dags(
 Given a file path or a folder, this method looks for python modules,
 imports them and adds them to the dagbag collection.
 
-Note that if a .airflowignore file is found while processing,
-the directory, it will behaves much like a .gitignore does,
+Note that if a ``.airflowignore`` file is found while processing
+the directory, it will behave much like a ``.gitignore``,
 ignoring files that match any of the regex patterns specified
-in the file. **Note**: The patterns in .airflowignore are treated as
+in the file.
+
+**Note**: The patterns in .airflowignore are treated as
 un-anchored regexes, not shell-like glob patterns.
 """
 start_dttm = timezone.utcnow()
diff --git a/docs/concepts.rst b/docs/concepts.rst
index 7316477225..a30a7026bc 100644
--- a/docs/concepts.rst
+++ b/docs/concepts.rst
@@ -863,3 +863,32 @@ do the same, but then it is more to use a virtualenv and 
pip.
to be available on the system if a module needs those. In other words only
pure python modules can be packaged.
 
+
+.airflowignore
+''
+
+A ``.airflowignore`` file specifies the directories or files in ``DAG_FOLDER``
+that Airflow should intentionally ignore. Each line in ``.airflowignore``
+specifies a regular expression pattern, and directories or files whose names
+(not DAG id) match any of the patterns would be ignored (under the hood,
+``re.findall()`` is used to match the pattern). Overall it works like a
+``.gitignore`` file.
+
+``.airflowignore`` file should be put in your ``DAG_FOLDER``.
+For example, you can prepare a ``.airflowignore`` file with contents
+
+.. code::
+
+project_a
+tenant_[\d]
+
+
+Then files like "project_a_dag_1.py", "TESTING_project_a.py", "tenant_1.py",
+"project_a/dag_1.py", and "tenant_1/dag_1.py" in your ``DAG_FOLDER`` would be 
ignored
+(If a directory's name matches any of the patterns, this directory and all its 
subfolders
+would not be scanned by Airflow at all. This improves efficiency of DAG 
finding).
+
+The scope of a ``.airflowignore`` file is the directory it is in plus all its 
subfolders.
+You can also prepare ``.airflowignore`` file for a subfolder in ``DAG_FOLDER`` 
and it
+would only be applicable for that subfolder.
+


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> .airflowignore feature is not mentioned at all in documentation
> ---
>
> Key: AIRFLOW-3104
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3104
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Minor
> Fix For: 2.0.0
>
>
> This is a nice feature, but not mentioned at all in anywhere of the 
> documentation.
> Relevant information can only be found in source code comment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ashb closed pull request #3939: [AIRFLOW-3104] Add .airflowignore info into doc

[GitBox]

ashb closed pull request #3939: [AIRFLOW-3104] Add .airflowignore info into doc
URL: https://github.com/apache/incubator-airflow/pull/3939
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/models.py b/airflow/models.py
index 6154ca64f4..6ea4638a51 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -522,10 +522,12 @@ def collect_dags(
 Given a file path or a folder, this method looks for python modules,
 imports them and adds them to the dagbag collection.
 
-Note that if a .airflowignore file is found while processing,
-the directory, it will behaves much like a .gitignore does,
+Note that if a ``.airflowignore`` file is found while processing
+the directory, it will behave much like a ``.gitignore``,
 ignoring files that match any of the regex patterns specified
-in the file. **Note**: The patterns in .airflowignore are treated as
+in the file.
+
+**Note**: The patterns in .airflowignore are treated as
 un-anchored regexes, not shell-like glob patterns.
 """
 start_dttm = timezone.utcnow()
diff --git a/docs/concepts.rst b/docs/concepts.rst
index 7316477225..a30a7026bc 100644
--- a/docs/concepts.rst
+++ b/docs/concepts.rst
@@ -863,3 +863,32 @@ do the same, but then it is more to use a virtualenv and 
pip.
to be available on the system if a module needs those. In other words only
pure python modules can be packaged.
 
+
+.airflowignore
+''
+
+A ``.airflowignore`` file specifies the directories or files in ``DAG_FOLDER``
+that Airflow should intentionally ignore. Each line in ``.airflowignore``
+specifies a regular expression pattern, and directories or files whose names
+(not DAG id) match any of the patterns would be ignored (under the hood,
+``re.findall()`` is used to match the pattern). Overall it works like a
+``.gitignore`` file.
+
+``.airflowignore`` file should be put in your ``DAG_FOLDER``.
+For example, you can prepare a ``.airflowignore`` file with contents
+
+.. code::
+
+project_a
+tenant_[\d]
+
+
+Then files like "project_a_dag_1.py", "TESTING_project_a.py", "tenant_1.py",
+"project_a/dag_1.py", and "tenant_1/dag_1.py" in your ``DAG_FOLDER`` would be 
ignored
+(If a directory's name matches any of the patterns, this directory and all its 
subfolders
+would not be scanned by Airflow at all. This improves efficiency of DAG 
finding).
+
+The scope of a ``.airflowignore`` file is the directory it is in plus all its 
subfolders.
+You can also prepare ``.airflowignore`` file for a subfolder in ``DAG_FOLDER`` 
and it
+would only be applicable for that subfolder.
+


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-2912) Add operators for Google Cloud Functions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631776#comment-16631776
 ] 

ASF GitHub Bot commented on AIRFLOW-2912:
-

potiuk opened a new pull request #3969: [AIRFLOW-2912] Add Deploy and Delete 
operators for GCF
URL: https://github.com/apache/incubator-airflow/pull/3969
 
 
   ### Jira
   
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. 
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI 
changes:
   
   Both Deploy and Delete operators interact with Google
   Cloud Functions to manage functions. Both are idempotent
   and make use of GcfHook - hook that encapsulates
   communication with GCP over GCP API.
   
   Invoke operator is also in the works, but it waits for alpha
   IAM functionality that will soon be available and helps
   with protecting invoke operation.
   
   Part of this commit message is also Body Field Validator
   that is meant to be reusable (but for now is an integral
   part of GCF operators). It will be separated out into
   utils in one of the subsequent Pull Requests where it
   will be reused in other operators (GCE ones)
   
   ### Tests
   
   - [X] My PR adds the following unit tests 
   * GcfFunctionDeployTest class
   * GcfFunctionDeleteTest class
   
   63 tests in total (includes parameterised test) 
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - When adding new operators/hooks/sensors, the autoclass documentation 
generation needs to be added.
   
   ### Code Quality
   
   - [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add operators for Google Cloud Functions
> 
>
> Key: AIRFLOW-2912
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2912
> Project: Apache Airflow
>  Issue Type: New Feature
>Reporter: Dariusz Aniszewski
>Assignee: Jarek Potiuk
>Priority: Major
>
> It would be nice to be able to create, delete and call Cloud Functions



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] potiuk opened a new pull request #3969: [AIRFLOW-2912] Add Deploy and Delete operators for GCF

[GitBox]

potiuk opened a new pull request #3969: [AIRFLOW-2912] Add Deploy and Delete 
operators for GCF
URL: https://github.com/apache/incubator-airflow/pull/3969
 
 
   ### Jira
   
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. 
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI 
changes:
   
   Both Deploy and Delete operators interact with Google
   Cloud Functions to manage functions. Both are idempotent
   and make use of GcfHook - hook that encapsulates
   communication with GCP over GCP API.
   
   Invoke operator is also in the works, but it waits for alpha
   IAM functionality that will soon be available and helps
   with protecting invoke operation.
   
   Part of this commit message is also Body Field Validator
   that is meant to be reusable (but for now is an integral
   part of GCF operators). It will be separated out into
   utils in one of the subsequent Pull Requests where it
   will be reused in other operators (GCE ones)
   
   ### Tests
   
   - [X] My PR adds the following unit tests 
   * GcfFunctionDeployTest class
   * GcfFunctionDeleteTest class
   
   63 tests in total (includes parameterised test) 
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - When adding new operators/hooks/sensors, the autoclass documentation 
generation needs to be added.
   
   ### Code Quality
   
   - [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ashb commented on a change in pull request #2372: [AIRFLOW-393] Add callback for FTP downloads

[GitBox]

ashb commented on a change in pull request #2372: [AIRFLOW-393] Add callback 
for FTP downloads
URL: https://github.com/apache/incubator-airflow/pull/2372#discussion_r221235162
 
 

 ##
 File path: airflow/contrib/hooks/ftp_hook.py
 ##
 @@ -161,6 +165,9 @@ def retrieve_file(self, remote_full_path, 
local_full_path_or_buffer):
 :param local_full_path_or_buffer: full path to the local file or a
 file-like buffer
 :type local_full_path_or_buffer: str or file-like buffer
+:param callback: callback that can be used for tracking download
+progress, among other things
+:type callback: callable
 
 Review comment:
   Can we have some docs on what the callback is called with (what args), when 
it is called, what doesn't happen normally if you use a callback (i.e. I think 
it won't write any files.)
   
   Also possibly as implemeneted now it would be better to make callback at 
local_full_path_or_bucket to be mutually exclusive - if a callback is passed 
the local_* argument is ignored.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sethwoodworth commented on issue #3918: [AIRFLOW-3088] Include slack-compatible emoji image.

[GitBox]

sethwoodworth commented on issue #3918: [AIRFLOW-3088] Include slack-compatible 
emoji image.
URL: 
https://github.com/apache/incubator-airflow/pull/3918#issuecomment-425417323
 
 
   @r39132 Sorry for the delay, I wasn't getting notifications for a little 
while there.
   
   + Image moved to /docs/images/logo subfolder
   + Preview image added to PR description
   + Rebased, squashed, & pushed


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3958: [AIRFLOW-3124] Fix RBAC webserver debug mode

[GitBox]

codecov-io commented on issue #3958: [AIRFLOW-3124] Fix RBAC webserver debug 
mode
URL: 
https://github.com/apache/incubator-airflow/pull/3958#issuecomment-425411512
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3958?src=pr=h1)
 Report
   > Merging 
[#3958](https://codecov.io/gh/apache/incubator-airflow/pull/3958?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/f4f8027cbf61ce2ed6a9989facf6c99dffb12f66?src=pr=desc)
 will **increase** coverage by `0.02%`.
   > The diff coverage is `0%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3958/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3958?src=pr=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master   #3958  +/-   ##
   =
   + Coverage   74.88%   74.9%   +0.02% 
   =
 Files 200 200  
 Lines   15974   15976   +2 
   =
   + Hits11962   11967   +5 
   + Misses   40124009   -3
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3958?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/3958/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `64.43% <0%> (-0.16%)` | :arrow_down: |
   | 
[airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/3958/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==)
 | `72.04% <0%> (+0.07%)` | :arrow_up: |
   | 
[airflow/configuration.py](https://codecov.io/gh/apache/incubator-airflow/pull/3958/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5)
 | `89.25% <0%> (+0.74%)` | :arrow_up: |
   | 
[airflow/www\_rbac/app.py](https://codecov.io/gh/apache/incubator-airflow/pull/3958/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcHAucHk=)
 | `97.77% <0%> (+2.22%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3958?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3958?src=pr=footer).
 Last update 
[f4f8027...90686be](https://codecov.io/gh/apache/incubator-airflow/pull/3958?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3952: [AIRFLOW-XXX] Update GCS logging docs for latest code

[GitBox]

codecov-io commented on issue #3952: [AIRFLOW-XXX] Update GCS logging docs for 
latest code
URL: 
https://github.com/apache/incubator-airflow/pull/3952#issuecomment-425411097
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3952?src=pr=h1)
 Report
   > Merging 
[#3952](https://codecov.io/gh/apache/incubator-airflow/pull/3952?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/17a9a0fbe9a543b21496db1b6997e8164767b380?src=pr=desc)
 will **increase** coverage by `59.77%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3952/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3952?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3952   +/-   ##
   ===
   + Coverage   15.14%   74.91%   +59.77% 
   ===
 Files 200  200   
 Lines   1597415974   
   ===
   + Hits 241911967 +9548 
   + Misses  13555 4007 -9548
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3952?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | 
[airflow/operators/dummy\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZHVtbXlfb3BlcmF0b3IucHk=)
 | `100% <0%> (+22.22%)` | :arrow_up: |
   | ... and [150 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3952/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3952?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3952?src=pr=footer).
 Last update 
[17a9a0f...51b7ad7](https://codecov.io/gh/apache/incubator-airflow/pull/3952?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Closed] (AIRFLOW-2675) Run commands error after installed

[Guoqiang Ding (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guoqiang Ding closed AIRFLOW-2675.
--
Resolution: Not A Problem

> Run commands error after installed
> --
>
> Key: AIRFLOW-2675
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2675
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: dependencies
>Reporter: Guoqiang Ding
>Assignee: Guoqiang Ding
>Priority: Major
>
> After I installed airflow in my test environment, it raises error on the 
> dependencies.
>  
> {code:java}
> //
> $ airflow
> Traceback (most recent call last):
> File "/opt/incubator-airflow/env/bin/airflow", line 6, in 
> exec(compile(open(__file__).read(), __file__, 'exec'))
> File "/opt/incubator-airflow/airflow/bin/airflow", line 21, in 
> from airflow import configuration
> File "/opt/incubator-airflow/airflow/__init__.py", line 37, in 
> from airflow.models import DAG
> File "/opt/incubator-airflow/airflow/models.py", line 31, in 
> import cryptography
> ImportError: No module named cryptography
> {code}
>  
>  
> And after I run "pip install cryptography" mannually, it raises error on the 
> k8s dependencies when I run "airflow init db".
>  
> {code:java}
> //
> $ airflow initdb
> [2018-06-26 10:52:08,259] {__init__.py:51} INFO - Using executor 
> SequentialExecutor
> DB: sqlite:home/openstack/airflow/airflow.db
> [2018-06-26 10:52:08,394] {db.py:338} INFO - Creating tables
> INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
> INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
> INFO  [alembic.runtime.migration] Running upgrade  -> e3a246e0dc1, current 
> schema
> INFO  [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 
> 1507a7289a2f, create is_encrypted
> /opt/incubator-airflow/env/local/lib/python2.7/site-packages/alembic/util/messaging.py:69:
>  UserWarning: Skipping unsupported ALTER for creation of implicit constraint
>   warnings.warn(msg)
> INFO  [alembic.runtime.migration] Running upgrade 1507a7289a2f -> 
> 13eb55f81627, maintain history for compatibility with earlier migrations
> INFO  [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 
> 338e90f54d61, More logging into task_isntance
> INFO  [alembic.runtime.migration] Running upgrade 338e90f54d61 -> 
> 52d714495f0, job_id indices
> INFO  [alembic.runtime.migration] Running upgrade 52d714495f0 -> 
> 502898887f84, Adding extra to Log
> INFO  [alembic.runtime.migration] Running upgrade 502898887f84 -> 
> 1b38cef5b76e, add dagrun
> INFO  [alembic.runtime.migration] Running upgrade 1b38cef5b76e -> 
> 2e541a1dcfed, task_duration
> INFO  [alembic.runtime.migration] Running upgrade 2e541a1dcfed -> 
> 40e67319e3a9, dagrun_config
> INFO  [alembic.runtime.migration] Running upgrade 40e67319e3a9 -> 
> 561833c1c74b, add password column to user
> INFO  [alembic.runtime.migration] Running upgrade 561833c1c74b -> 4446e08588, 
> dagrun start end
> INFO  [alembic.runtime.migration] Running upgrade 4446e08588 -> bbc73705a13e, 
> Add notification_sent column to sla_miss
> INFO  [alembic.runtime.migration] Running upgrade bbc73705a13e -> 
> bba5a7cfc896, Add a column to track the encryption state of the 'Extra' field 
> in connection
> INFO  [alembic.runtime.migration] Running upgrade bba5a7cfc896 -> 
> 1968acfc09e3, add is_encrypted column to variable table
> INFO  [alembic.runtime.migration] Running upgrade 1968acfc09e3 -> 
> 2e82aab8ef20, rename user table
> INFO  [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> 
> 211e584da130, add TI state index
> INFO  [alembic.runtime.migration] Running upgrade 211e584da130 -> 
> 64de9cddf6c9, add task fails journal table
> INFO  [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> 
> f2ca10b85618, add dag_stats table
> INFO  [alembic.runtime.migration] Running upgrade f2ca10b85618 -> 
> 4addfa1236f1, Add fractional seconds to mysql tables
> INFO  [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> 
> 8504051e801b, xcom dag task indices
> INFO  [alembic.runtime.migration] Running upgrade 8504051e801b -> 
> 5e7d17757c7a, add pid field to TaskInstance
> INFO  [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> 
> 127d2bf2dfa7, Add dag_id/state index on dag_run table
> INFO  [alembic.runtime.migration] Running upgrade 127d2bf2dfa7 -> 
> cc1e65623dc7, add max tries column to task instance
> WARNI [airflow.utils.log.logging_mixin.LoggingMixin] Could not import 
> KubernetesPodOperator: No module named kubernetes
> WARNI [airflow.utils.log.logging_mixin.LoggingMixin] Install kubernetes 
> dependencies with:     pip install airflow['kubernetes']
> INFO  [alembic.runtime.migration] Running upgrade cc1e65623dc7 -> 
> bdaa763e6c56, Make xcom value column a large binary
> INFO  [alembic.runtime.migration] Running upgrade 

[jira] [Commented] (AIRFLOW-2675) Run commands error after installed

[Guoqiang Ding (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631738#comment-16631738
 ] 

Guoqiang Ding commented on AIRFLOW-2675:


Thanks a lot. I'll close it.

> Run commands error after installed
> --
>
> Key: AIRFLOW-2675
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2675
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: dependencies
>Reporter: Guoqiang Ding
>Assignee: Guoqiang Ding
>Priority: Major
>
> After I installed airflow in my test environment, it raises error on the 
> dependencies.
>  
> {code:java}
> //
> $ airflow
> Traceback (most recent call last):
> File "/opt/incubator-airflow/env/bin/airflow", line 6, in 
> exec(compile(open(__file__).read(), __file__, 'exec'))
> File "/opt/incubator-airflow/airflow/bin/airflow", line 21, in 
> from airflow import configuration
> File "/opt/incubator-airflow/airflow/__init__.py", line 37, in 
> from airflow.models import DAG
> File "/opt/incubator-airflow/airflow/models.py", line 31, in 
> import cryptography
> ImportError: No module named cryptography
> {code}
>  
>  
> And after I run "pip install cryptography" mannually, it raises error on the 
> k8s dependencies when I run "airflow init db".
>  
> {code:java}
> //
> $ airflow initdb
> [2018-06-26 10:52:08,259] {__init__.py:51} INFO - Using executor 
> SequentialExecutor
> DB: sqlite:home/openstack/airflow/airflow.db
> [2018-06-26 10:52:08,394] {db.py:338} INFO - Creating tables
> INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
> INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
> INFO  [alembic.runtime.migration] Running upgrade  -> e3a246e0dc1, current 
> schema
> INFO  [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 
> 1507a7289a2f, create is_encrypted
> /opt/incubator-airflow/env/local/lib/python2.7/site-packages/alembic/util/messaging.py:69:
>  UserWarning: Skipping unsupported ALTER for creation of implicit constraint
>   warnings.warn(msg)
> INFO  [alembic.runtime.migration] Running upgrade 1507a7289a2f -> 
> 13eb55f81627, maintain history for compatibility with earlier migrations
> INFO  [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 
> 338e90f54d61, More logging into task_isntance
> INFO  [alembic.runtime.migration] Running upgrade 338e90f54d61 -> 
> 52d714495f0, job_id indices
> INFO  [alembic.runtime.migration] Running upgrade 52d714495f0 -> 
> 502898887f84, Adding extra to Log
> INFO  [alembic.runtime.migration] Running upgrade 502898887f84 -> 
> 1b38cef5b76e, add dagrun
> INFO  [alembic.runtime.migration] Running upgrade 1b38cef5b76e -> 
> 2e541a1dcfed, task_duration
> INFO  [alembic.runtime.migration] Running upgrade 2e541a1dcfed -> 
> 40e67319e3a9, dagrun_config
> INFO  [alembic.runtime.migration] Running upgrade 40e67319e3a9 -> 
> 561833c1c74b, add password column to user
> INFO  [alembic.runtime.migration] Running upgrade 561833c1c74b -> 4446e08588, 
> dagrun start end
> INFO  [alembic.runtime.migration] Running upgrade 4446e08588 -> bbc73705a13e, 
> Add notification_sent column to sla_miss
> INFO  [alembic.runtime.migration] Running upgrade bbc73705a13e -> 
> bba5a7cfc896, Add a column to track the encryption state of the 'Extra' field 
> in connection
> INFO  [alembic.runtime.migration] Running upgrade bba5a7cfc896 -> 
> 1968acfc09e3, add is_encrypted column to variable table
> INFO  [alembic.runtime.migration] Running upgrade 1968acfc09e3 -> 
> 2e82aab8ef20, rename user table
> INFO  [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> 
> 211e584da130, add TI state index
> INFO  [alembic.runtime.migration] Running upgrade 211e584da130 -> 
> 64de9cddf6c9, add task fails journal table
> INFO  [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> 
> f2ca10b85618, add dag_stats table
> INFO  [alembic.runtime.migration] Running upgrade f2ca10b85618 -> 
> 4addfa1236f1, Add fractional seconds to mysql tables
> INFO  [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> 
> 8504051e801b, xcom dag task indices
> INFO  [alembic.runtime.migration] Running upgrade 8504051e801b -> 
> 5e7d17757c7a, add pid field to TaskInstance
> INFO  [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> 
> 127d2bf2dfa7, Add dag_id/state index on dag_run table
> INFO  [alembic.runtime.migration] Running upgrade 127d2bf2dfa7 -> 
> cc1e65623dc7, add max tries column to task instance
> WARNI [airflow.utils.log.logging_mixin.LoggingMixin] Could not import 
> KubernetesPodOperator: No module named kubernetes
> WARNI [airflow.utils.log.logging_mixin.LoggingMixin] Install kubernetes 
> dependencies with:     pip install airflow['kubernetes']
> INFO  [alembic.runtime.migration] Running upgrade cc1e65623dc7 -> 
> bdaa763e6c56, Make xcom value column a large binary
> INFO  

[GitHub] codecov-io commented on issue #3955: [AIRFLOW-3121] Define closed property on StreamLogWriter

[GitBox]

codecov-io commented on issue #3955: [AIRFLOW-3121] Define closed property on 
StreamLogWriter
URL: 
https://github.com/apache/incubator-airflow/pull/3955#issuecomment-425409276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3955?src=pr=h1)
 Report
   > Merging 
[#3955](https://codecov.io/gh/apache/incubator-airflow/pull/3955?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.78%`.
   > The diff coverage is `50%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3955/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3955?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3955   +/-   ##
   ===
   + Coverage   15.14%   74.93%   +59.78% 
   ===
 Files 200  200   
 Lines   1597415976+2 
   ===
   + Hits 241911971 +9552 
   + Misses  13555 4005 -9550
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3955?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/utils/log/logging\_mixin.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9sb2cvbG9nZ2luZ19taXhpbi5weQ==)
 | `96.29% <50%> (+43.13%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3955/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3955?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3955?src=pr=footer).
 Last update 
[7cd9a26...1f9f3e1](https://codecov.io/gh/apache/incubator-airflow/pull/3955?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3960: [AIRFLOW-2966] Catch ApiException in the Kubernetes Executor

[GitBox]

codecov-io commented on issue #3960: [AIRFLOW-2966] Catch ApiException in the 
Kubernetes Executor
URL: 
https://github.com/apache/incubator-airflow/pull/3960#issuecomment-425409131
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3960?src=pr=h1)
 Report
   > Merging 
[#3960](https://codecov.io/gh/apache/incubator-airflow/pull/3960?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/481daeec0b05e2212629a80f7264113860a485c8?src=pr=desc)
 will **increase** coverage by `0.32%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3960/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3960?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3960  +/-   ##
   ==
   + Coverage   74.69%   75.02%   +0.32% 
   ==
 Files 200  200  
 Lines   1597415974  
   ==
   + Hits1193211984  +52 
   + Misses   4042 3990  -52
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3960?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/3960/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=)
 | `68.85% <0%> (+0.06%)` | :arrow_up: |
   | 
[airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/3960/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==)
 | `72.04% <0%> (+0.14%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/3960/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `64.59% <0%> (+0.23%)` | :arrow_up: |
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3960/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.39% <0%> (+0.26%)` | :arrow_up: |
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3960/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `89.24% <0%> (+0.57%)` | :arrow_up: |
   | 
[airflow/www\_rbac/app.py](https://codecov.io/gh/apache/incubator-airflow/pull/3960/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcHAucHk=)
 | `97.77% <0%> (+1.11%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/base\_task\_runner.py](https://codecov.io/gh/apache/incubator-airflow/pull/3960/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL2Jhc2VfdGFza19ydW5uZXIucHk=)
 | `79.66% <0%> (+1.69%)` | :arrow_up: |
   | 
[airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3960/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5)
 | `79.67% <0%> (+3.25%)` | :arrow_up: |
   | 
[airflow/hooks/hdfs\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3960/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oZGZzX2hvb2sucHk=)
 | `92.5% <0%> (+60%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3960?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3960?src=pr=footer).
 Last update 
[481daee...ad02b62](https://codecov.io/gh/apache/incubator-airflow/pull/3960?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3950: [AIRFLOW-3119] Enable loglevel on celery worker and inherit from airflow.cfg

[GitBox]

codecov-io edited a comment on issue #3950: [AIRFLOW-3119] Enable loglevel on 
celery worker and inherit from airflow.cfg
URL: 
https://github.com/apache/incubator-airflow/pull/3950#issuecomment-425408487
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=h1)
 Report
   > Merging 
[#3950](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/17a9a0fbe9a543b21496db1b6997e8164767b380?src=pr=desc)
 will **increase** coverage by `59.79%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3950/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3950   +/-   ##
   ===
   + Coverage   15.14%   74.93%   +59.79% 
   ===
 Files 200  200   
 Lines   1597415974   
   ===
   + Hits 241911970 +9551 
   + Misses  13555 4004 -9551
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `64.59% <ø> (+50.71%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=footer).
 Last update 
[17a9a0f...81dbc0f](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3950: [AIRFLOW-3119] Enable loglevel on celery worker and inherit from airflow.cfg

[GitBox]

codecov-io commented on issue #3950: [AIRFLOW-3119] Enable loglevel on celery 
worker and inherit from airflow.cfg
URL: 
https://github.com/apache/incubator-airflow/pull/3950#issuecomment-425408487
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=h1)
 Report
   > Merging 
[#3950](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/17a9a0fbe9a543b21496db1b6997e8164767b380?src=pr=desc)
 will **increase** coverage by `59.79%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3950/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3950   +/-   ##
   ===
   + Coverage   15.14%   74.93%   +59.79% 
   ===
 Files 200  200   
 Lines   1597415974   
   ===
   + Hits 241911970 +9551 
   + Misses  13555 4004 -9551
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `64.59% <ø> (+50.71%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3950/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=footer).
 Last update 
[17a9a0f...81dbc0f](https://codecov.io/gh/apache/incubator-airflow/pull/3950?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3945: [AIRFLOW-3112] Make SFTP hook to inherit SSH hook

[GitBox]

codecov-io commented on issue #3945: [AIRFLOW-3112] Make SFTP hook to inherit 
SSH hook
URL: 
https://github.com/apache/incubator-airflow/pull/3945#issuecomment-425407404
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3945?src=pr=h1)
 Report
   > Merging 
[#3945](https://codecov.io/gh/apache/incubator-airflow/pull/3945?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/720ea1b11c7640d98d9139d52076f9b2beae7dd1?src=pr=desc)
 will **increase** coverage by `1.77%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3945/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3945?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3945  +/-   ##
   ==
   + Coverage74.9%   76.67%   +1.77% 
   ==
 Files 200  200  
 Lines   1597017118+1148 
   ==
   + Hits1196213126+1164 
   + Misses   4008 3992  -16
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3945?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3945/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.48% <0%> (+0.35%)` | :arrow_up: |
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3945/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `92.59% <0%> (+3.99%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3945?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3945?src=pr=footer).
 Last update 
[720ea1b...9702b6a](https://codecov.io/gh/apache/incubator-airflow/pull/3945?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #2334: [AIRFLOW-1252] API - Fix when conf is in JSON body

[GitBox]

codecov-io edited a comment on issue #2334: [AIRFLOW-1252] API - Fix when conf 
is in JSON body
URL: 
https://github.com/apache/incubator-airflow/pull/2334#issuecomment-337199496
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2334?src=pr=h1)
 Report
   > Merging 
[#2334](https://codecov.io/gh/apache/incubator-airflow/pull/2334?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/010924a06f173f0ec32fcdf2b9025539c89931cb?src=pr=desc)
 will **increase** coverage by `0.23%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/2334/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/2334?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#2334  +/-   ##
   ==
   + Coverage   74.69%   74.93%   +0.23% 
   ==
 Files 200  200  
 Lines   1597415975   +1 
   ==
   + Hits1193211971  +39 
   + Misses   4042 4004  -38
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/2334?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/www/api/experimental/endpoints.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYXBpL2V4cGVyaW1lbnRhbC9lbmRwb2ludHMucHk=)
 | `90.9% <100%> (+0.76%)` | :arrow_up: |
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.62% <0%> (-0.05%)` | :arrow_down: |
   | 
[airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=)
 | `68.85% <0%> (+0.06%)` | :arrow_up: |
   | 
[airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==)
 | `72.04% <0%> (+0.14%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `64.59% <0%> (+0.23%)` | :arrow_up: |
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.39% <0%> (+0.26%)` | :arrow_up: |
   | 
[airflow/www\_rbac/app.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcHAucHk=)
 | `97.77% <0%> (+1.11%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/base\_task\_runner.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL2Jhc2VfdGFza19ydW5uZXIucHk=)
 | `79.66% <0%> (+1.69%)` | :arrow_up: |
   | 
[airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5)
 | `79.67% <0%> (+3.25%)` | :arrow_up: |
   | 
[airflow/hooks/hdfs\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/2334/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oZGZzX2hvb2sucHk=)
 | `92.5% <0%> (+60%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2334?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2334?src=pr=footer).
 Last update 
[010924a...a18f970](https://codecov.io/gh/apache/incubator-airflow/pull/2334?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.79%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3956   +/-   ##
   ===
   + Coverage   15.14%   74.93%   +59.79% 
   ===
 Files 200  200   
 Lines   1597415971-3 
   ===
   + Hits 241911968 +9549 
   + Misses  13555 4003 -9552
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.65% <100%> (+60.88%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[7cd9a26...9ec6683](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.79%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3956   +/-   ##
   ===
   + Coverage   15.14%   74.93%   +59.79% 
   ===
 Files 200  200   
 Lines   1597415971-3 
   ===
   + Hits 241911968 +9549 
   + Misses  13555 4003 -9552
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.65% <100%> (+60.88%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[7cd9a26...9ec6683](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.78%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3956   +/-   ##
   ===
   + Coverage   15.14%   74.92%   +59.78% 
   ===
 Files 200  200   
 Lines   1597415971-3 
   ===
   + Hits 241911967 +9548 
   + Misses  13555 4004 -9551
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.61% <100%> (+60.84%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[7cd9a26...9ec6683](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.78%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3956   +/-   ##
   ===
   + Coverage   15.14%   74.92%   +59.78% 
   ===
 Files 200  200   
 Lines   1597415971-3 
   ===
   + Hits 241911967 +9548 
   + Misses  13555 4004 -9551
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.61% <100%> (+60.84%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[7cd9a26...9ec6683](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-2952) Dockerized CI pipeline has silently broken integration testing for KubernetesExecutor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631690#comment-16631690
 ] 

ASF GitHub Bot commented on AIRFLOW-2952:
-

kaxil closed pull request #3957: [AIRFLOW-2952] Fix Kubernetes CI
URL: https://github.com/apache/incubator-airflow/pull/3957
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/example_dags_kubernetes/__init__.py 
b/airflow/contrib/example_dags/__init__.py
similarity index 100%
rename from airflow/example_dags_kubernetes/__init__.py
rename to airflow/contrib/example_dags/__init__.py
diff --git a/airflow/example_dags_kubernetes/example_kubernetes_annotation.py 
b/airflow/example_dags_kubernetes/example_kubernetes_annotation.py
deleted file mode 100644
index 058baf6990..00
--- a/airflow/example_dags_kubernetes/example_kubernetes_annotation.py
+++ /dev/null
@@ -1,47 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-#
-#   http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-from __future__ import print_function
-import airflow
-from airflow.operators.python_operator import PythonOperator
-from airflow.models import DAG
-
-args = {
-'owner': 'airflow',
-'start_date': airflow.utils.dates.days_ago(2)
-}
-
-dag = DAG(
-dag_id='example_kubernetes_annotation', default_args=args,
-schedule_interval=None
-)
-
-
-def print_stuff():
-print("annotated!")
-
-
-# You can use annotations on your kubernetes pods!
-start_task = PythonOperator(
-task_id="start_task", python_callable=print_stuff, dag=dag,
-executor_config={
-"KubernetesExecutor": {
-"annotations": {"test": "annotation"}
-}
-}
-)
diff --git a/scripts/ci/kubernetes/docker/airflow-test-env-init.sh 
b/scripts/ci/kubernetes/docker/airflow-test-env-init.sh
index 33593fd1ca..05f8e20aea 100755
--- a/scripts/ci/kubernetes/docker/airflow-test-env-init.sh
+++ b/scripts/ci/kubernetes/docker/airflow-test-env-init.sh
@@ -18,8 +18,9 @@
 #  under the License.
 
 cd /usr/local/lib/python2.7/dist-packages/airflow && \
-cp -R example_dags_kubernetes/* /root/airflow/dags/ && \
+cp -R example_dags/* /root/airflow/dags/ && \
+cp -R contrib/example_dags/example_kubernetes_*.py /root/airflow/dags/ && \
 airflow initdb && \
 alembic upgrade heads && \
-(airflow create_user -u airflow -l airflow -f jon -e airf...@apache.org -r 
Admin -p airflow || true) && \
+(airflow users --create --username airflow --lastname airflow --firstname jon 
--email airf...@apache.org --role Admin --password airflow || true) && \
 echo "retrieved from mount" > /root/test_volume/test.txt


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Dockerized CI pipeline has silently broken integration testing for 
> KubernetesExecutor
> -
>
> Key: AIRFLOW-2952
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2952
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Daniel Imberman
>Assignee: Daniel Imberman
>Priority: Blocker
> Fix For: 2.0.0
>
>
> [~gcuriel] [~bolke] [~Fokko]
> Looking at all recent builds the new CI pipeline is silently reverting the 
> kubernetes tests to the normal airflow tests.
> Before
> https://travis-ci.org/apache/incubator-airflow/jobs/418914949#L1007
> After:
> [https://travis-ci.org/apache/incubator-airflow/jobs/419062412#L4970]
> This means that kubernetes builds will pass without actually testing on a 
> kubernetes cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.78%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3956   +/-   ##
   ===
   + Coverage   15.14%   74.92%   +59.78% 
   ===
 Files 200  200   
 Lines   1597415971-3 
   ===
   + Hits 241911967 +9548 
   + Misses  13555 4004 -9551
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.61% <100%> (+60.84%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[7cd9a26...9ec6683](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kaxil closed pull request #3957: [AIRFLOW-2952] Fix Kubernetes CI

[GitBox]

kaxil closed pull request #3957: [AIRFLOW-2952] Fix Kubernetes CI
URL: https://github.com/apache/incubator-airflow/pull/3957
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/example_dags_kubernetes/__init__.py 
b/airflow/contrib/example_dags/__init__.py
similarity index 100%
rename from airflow/example_dags_kubernetes/__init__.py
rename to airflow/contrib/example_dags/__init__.py
diff --git a/airflow/example_dags_kubernetes/example_kubernetes_annotation.py 
b/airflow/example_dags_kubernetes/example_kubernetes_annotation.py
deleted file mode 100644
index 058baf6990..00
--- a/airflow/example_dags_kubernetes/example_kubernetes_annotation.py
+++ /dev/null
@@ -1,47 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-#
-#   http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-from __future__ import print_function
-import airflow
-from airflow.operators.python_operator import PythonOperator
-from airflow.models import DAG
-
-args = {
-'owner': 'airflow',
-'start_date': airflow.utils.dates.days_ago(2)
-}
-
-dag = DAG(
-dag_id='example_kubernetes_annotation', default_args=args,
-schedule_interval=None
-)
-
-
-def print_stuff():
-print("annotated!")
-
-
-# You can use annotations on your kubernetes pods!
-start_task = PythonOperator(
-task_id="start_task", python_callable=print_stuff, dag=dag,
-executor_config={
-"KubernetesExecutor": {
-"annotations": {"test": "annotation"}
-}
-}
-)
diff --git a/scripts/ci/kubernetes/docker/airflow-test-env-init.sh 
b/scripts/ci/kubernetes/docker/airflow-test-env-init.sh
index 33593fd1ca..05f8e20aea 100755
--- a/scripts/ci/kubernetes/docker/airflow-test-env-init.sh
+++ b/scripts/ci/kubernetes/docker/airflow-test-env-init.sh
@@ -18,8 +18,9 @@
 #  under the License.
 
 cd /usr/local/lib/python2.7/dist-packages/airflow && \
-cp -R example_dags_kubernetes/* /root/airflow/dags/ && \
+cp -R example_dags/* /root/airflow/dags/ && \
+cp -R contrib/example_dags/example_kubernetes_*.py /root/airflow/dags/ && \
 airflow initdb && \
 alembic upgrade heads && \
-(airflow create_user -u airflow -l airflow -f jon -e airf...@apache.org -r 
Admin -p airflow || true) && \
+(airflow users --create --username airflow --lastname airflow --firstname jon 
--email airf...@apache.org --role Admin --password airflow || true) && \
 echo "retrieved from mount" > /root/test_volume/test.txt


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.76%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master   #3956   +/-   ##
   ==
   + Coverage   15.14%   74.9%   +59.76% 
   ==
 Files 200 200   
 Lines   15974   15971-3 
   ==
   + Hits 2419   11963 +9544 
   + Misses  135554008 -9547
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.61% <100%> (+60.84%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[7cd9a26...9ec6683](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.78%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#3956   +/-   ##
   ===
   + Coverage   15.14%   74.92%   +59.78% 
   ===
 Files 200  200   
 Lines   1597415971-3 
   ===
   + Hits 241911967 +9548 
   + Misses  13555 4004 -9551
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.61% <100%> (+60.84%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[7cd9a26...9ec6683](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io commented on issue #3956: [AIRFLOW-3123] Allow nested use of  DAG as 
a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.76%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master   #3956   +/-   ##
   ==
   + Coverage   15.14%   74.9%   +59.76% 
   ==
 Files 200 200   
 Lines   15974   15971-3 
   ==
   + Hits 2419   11963 +9544 
   + Misses  135554008 -9547
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.61% <100%> (+60.84%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[7cd9a26...9ec6683](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of DAG as a context manager

[GitBox]

codecov-io edited a comment on issue #3956: [AIRFLOW-3123] Allow nested use of  
DAG as a context manager
URL: 
https://github.com/apache/incubator-airflow/pull/3956#issuecomment-425398276
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=h1)
 Report
   > Merging 
[#3956](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/7cd9a26418ce9cb120f1cacd9fdcfe43fe5c0254?src=pr=desc)
 will **increase** coverage by `59.76%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3956/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master   #3956   +/-   ##
   ==
   + Coverage   15.14%   74.9%   +59.76% 
   ==
 Files 200 200   
 Lines   15974   15971-3 
   ==
   + Hits 2419   11963 +9544 
   + Misses  135554008 -9547
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.61% <100%> (+60.84%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+2.85%)` | :arrow_up: |
   | 
[airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=)
 | `86.95% <0%> (+4.34%)` | :arrow_up: |
   | 
[airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=)
 | `55.76% <0%> (+5.76%)` | :arrow_up: |
   | 
[airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5)
 | `91.66% <0%> (+14.58%)` | :arrow_up: |
   | 
[airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==)
 | `81.15% <0%> (+15.21%)` | :arrow_up: |
   | 
[airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==)
 | `15.47% <0%> (+15.47%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL19faW5pdF9fLnB5)
 | `63.63% <0%> (+18.18%)` | :arrow_up: |
   | 
[airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==)
 | `33.6% <0%> (+18.4%)` | :arrow_up: |
   | 
[airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==)
 | `74.28% <0%> (+19.99%)` | :arrow_up: |
   | ... and [151 
more](https://codecov.io/gh/apache/incubator-airflow/pull/3956/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=footer).
 Last update 
[7cd9a26...9ec6683](https://codecov.io/gh/apache/incubator-airflow/pull/3956?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io commented on issue #3957: [AIRFLOW-2952] Fix Kubernetes CI

[GitBox]

codecov-io commented on issue #3957: [AIRFLOW-2952] Fix Kubernetes CI
URL: 
https://github.com/apache/incubator-airflow/pull/3957#issuecomment-425398017
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3957?src=pr=h1)
 Report
   > Merging 
[#3957](https://codecov.io/gh/apache/incubator-airflow/pull/3957?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/9238bce3174c1c7dc660aa61275d54632f507719?src=pr=desc)
 will **increase** coverage by `0.24%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3957/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3957?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3957  +/-   ##
   ==
   + Coverage74.7%   74.95%   +0.24% 
   ==
 Files 200  199   -1 
 Lines   1597415965   -9 
   ==
   + Hits1193411967  +33 
   + Misses   4040 3998  -42
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3957?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3957/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.13% <0%> (-0.27%)` | :arrow_down: |
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3957/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.66% <0%> (+0.04%)` | :arrow_up: |
   | 
[airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/3957/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=)
 | `68.85% <0%> (+0.06%)` | :arrow_up: |
   | 
[airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/3957/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==)
 | `72.04% <0%> (+0.14%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/3957/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `64.59% <0%> (+0.23%)` | :arrow_up: |
   | 
[airflow/www\_rbac/app.py](https://codecov.io/gh/apache/incubator-airflow/pull/3957/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcHAucHk=)
 | `97.77% <0%> (+1.11%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/base\_task\_runner.py](https://codecov.io/gh/apache/incubator-airflow/pull/3957/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL2Jhc2VfdGFza19ydW5uZXIucHk=)
 | `79.66% <0%> (+1.69%)` | :arrow_up: |
   | 
[airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3957/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5)
 | `79.67% <0%> (+3.25%)` | :arrow_up: |
   | 
[airflow/hooks/hdfs\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3957/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oZGZzX2hvb2sucHk=)
 | `92.5% <0%> (+60%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3957?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3957?src=pr=footer).
 Last update 
[9238bce...fc1826c](https://codecov.io/gh/apache/incubator-airflow/pull/3957?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] codecov-io edited a comment on issue #3966: [AIRFLOW-3125] Monitor Task Instances creation rates

[GitBox]

codecov-io edited a comment on issue #3966: [AIRFLOW-3125] Monitor Task 
Instances creation rates
URL: 
https://github.com/apache/incubator-airflow/pull/3966#issuecomment-425395576
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3966?src=pr=h1)
 Report
   > Merging 
[#3966](https://codecov.io/gh/apache/incubator-airflow/pull/3966?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-airflow/commit/010924a06f173f0ec32fcdf2b9025539c89931cb?src=pr=desc)
 will **increase** coverage by `0.23%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-airflow/pull/3966/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/3966?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#3966  +/-   ##
   ==
   + Coverage   74.69%   74.93%   +0.23% 
   ==
 Files 200  200  
 Lines   1597415975   +1 
   ==
   + Hits1193211971  +39 
   + Misses   4042 4004  -38
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-airflow/pull/3966?src=pr=tree) 
| Coverage Δ | |
   |---|---|---|
   | 
[airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3966/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=)
 | `88.66% <100%> (ø)` | :arrow_up: |
   | 
[airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/3966/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=)
 | `68.85% <0%> (+0.06%)` | :arrow_up: |
   | 
[airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/3966/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==)
 | `72.04% <0%> (+0.14%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/3966/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `64.59% <0%> (+0.23%)` | :arrow_up: |
   | 
[airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/3966/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5)
 | `82.39% <0%> (+0.26%)` | :arrow_up: |
   | 
[airflow/www\_rbac/app.py](https://codecov.io/gh/apache/incubator-airflow/pull/3966/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcHAucHk=)
 | `97.77% <0%> (+1.11%)` | :arrow_up: |
   | 
[airflow/task/task\_runner/base\_task\_runner.py](https://codecov.io/gh/apache/incubator-airflow/pull/3966/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL2Jhc2VfdGFza19ydW5uZXIucHk=)
 | `79.66% <0%> (+1.69%)` | :arrow_up: |
   | 
[airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3966/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5)
 | `79.67% <0%> (+3.25%)` | :arrow_up: |
   | 
[airflow/hooks/hdfs\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3966/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oZGZzX2hvb2sucHk=)
 | `92.5% <0%> (+60%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3966?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3966?src=pr=footer).
 Last update 
[010924a...8c4f4b9](https://codecov.io/gh/apache/incubator-airflow/pull/3966?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services