[GitHub] [airflow] stale[bot] removed a comment on issue #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files

[GitBox]

stale[bot] removed a comment on issue #6788: WIP: [AIRFLOW-5944] Rendering 
templated_fields without accessing DAG files
URL: https://github.com/apache/airflow/pull/6788#issuecomment-578458377
 
 
   This issue has been automatically marked as stale because it has not had 
recent activity. It will be closed if no further activity occurs. Thank you for 
your contributions.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid exceptions when printing task instance

[GitBox]

mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid 
exceptions when printing task instance
URL: https://github.com/apache/airflow/pull/7257#discussion_r370971910
 
 

 ##
 File path: airflow/models/taskinstance.py
 ##
 @@ -1137,18 +1143,30 @@ def handle_failure(self, error, test_mode=None, 
context=None, session=None):
 'dag_id=%s, task_id=%s, execution_date=%s, 
start_date=%s, end_date=%s',
 self.dag_id,
 self.task_id,
-self.execution_date.strftime('%Y%m%dT%H%M%S'),
-self.start_date.strftime('%Y%m%dT%H%M%S'),
-self.end_date.strftime('%Y%m%dT%H%M%S'))
+self.execution_date.strftime('%Y%m%dT%H%M%S') if 
hasattr(
+self,
+'execution_date') and self.execution_date else '',
+self.start_date.strftime('%Y%m%dT%H%M%S') if hasattr(
+self,
+'start_date') and self.start_date else '',
+self.end_date.strftime('%Y%m%dT%H%M%S') if hasattr(
 
 Review comment:
   Can you indicate which test has this problem?  It seems to me that the test 
in this place may fail only in one situation - when execution_date == None, 
because the existence of the attribute is guaranteed by metaclase - SQlAlchemy.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-4360) Add metrics for measuring Airflow Webserver performance using Statsd

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-4360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023697#comment-17023697
 ] 

ASF GitHub Bot commented on AIRFLOW-4360:
-

stale[bot] commented on pull request #5139: [AIRFLOW-4360] Add performance 
metrics for Webserver UI.
URL: https://github.com/apache/airflow/pull/5139
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add metrics for measuring Airflow Webserver performance using Statsd
> 
>
> Key: AIRFLOW-4360
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4360
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: webserver
>Reporter: Kunal Shah
>Assignee: Kunal Shah
>Priority: Minor
>
> There are few metrics that are valuable to track for Webserver performance:
> 1) How long does tree / graph view take to load DAGs? How many failures?
> 2) How long does it take to fetch logs from the AJAX endpoint? Does it ever 
> fail? How often do DAG creators view their task logs?
> Such metrics can be extended further to other actions like manual running, 
> deleting, clearing of tasks and dagruns.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] stale[bot] closed pull request #5139: [AIRFLOW-4360] Add performance metrics for Webserver UI.

[GitBox]

stale[bot] closed pull request #5139: [AIRFLOW-4360] Add performance metrics 
for Webserver UI.
URL: https://github.com/apache/airflow/pull/5139
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] stale[bot] commented on issue #6606: [AIRFLOW-6014] - handle pods which are preempted and deleted by kuber…

[GitBox]

stale[bot] commented on issue #6606: [AIRFLOW-6014] - handle pods which are 
preempted and deleted by kuber…
URL: https://github.com/apache/airflow/pull/6606#issuecomment-578458380
 
 
   This issue has been automatically marked as stale because it has not had 
recent activity. It will be closed if no further activity occurs. Thank you for 
your contributions.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] stale[bot] commented on issue #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files

[GitBox]

stale[bot] commented on issue #6788: WIP: [AIRFLOW-5944] Rendering 
templated_fields without accessing DAG files
URL: https://github.com/apache/airflow/pull/6788#issuecomment-578458377
 
 
   This issue has been automatically marked as stale because it has not had 
recent activity. It will be closed if no further activity occurs. Thank you for 
your contributions.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] stale[bot] commented on issue #6767: [AIRFLOW-6208] Implement fileno in StreamLogWriter

[GitBox]

stale[bot] commented on issue #6767: [AIRFLOW-6208] Implement fileno in 
StreamLogWriter
URL: https://github.com/apache/airflow/pull/6767#issuecomment-578458378
 
 
   This issue has been automatically marked as stale because it has not had 
recent activity. It will be closed if no further activity occurs. Thank you for 
your contributions.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6641) Kubernetest tests happen to be flaky

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023684#comment-17023684
 ] 

ASF subversion and git services commented on AIRFLOW-6641:
--

Commit 4e7bb5f2553d4dee68b2bdcbebd944462acee957 in airflow's branch 
refs/heads/master from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=4e7bb5f ]

[AIRFLOW-6641] Better diagnostics for kubernetes flaky tests (#7261)

Kubernetes tests are sometimes flaky - they do not reach desired state and we
do not know why. We need to make it less flaky and have enough diagnostics to
understand what's going on.

> Kubernetest tests happen to be flaky
> 
>
> Key: AIRFLOW-6641
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6641
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> Kubernetes tests are sometimes flaky - they do not reach desired state and we 
> do not know why. We need to make it less flaky and have enough diagnostics to 
> understand what's going on.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6641) Kubernetest tests happen to be flaky

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023685#comment-17023685
 ] 

ASF subversion and git services commented on AIRFLOW-6641:
--

Commit 276410eec209f292b2613d9cd304ae782ca76638 in airflow's branch 
refs/heads/v1-10-test from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=276410e ]

[AIRFLOW-6641] Better diagnostics for kubernetes flaky tests (#7261)

Kubernetes tests are sometimes flaky - they do not reach desired state and we
do not know why. We need to make it less flaky and have enough diagnostics to
understand what's going on.

(cherry picked from commit 4e7bb5f2553d4dee68b2bdcbebd944462acee957)


> Kubernetest tests happen to be flaky
> 
>
> Key: AIRFLOW-6641
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6641
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> Kubernetes tests are sometimes flaky - they do not reach desired state and we 
> do not know why. We need to make it less flaky and have enough diagnostics to 
> understand what's going on.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk merged pull request #7261: [AIRFLOW-6641] Better diagnostics for kubernetes flaky tests

[GitBox]

potiuk merged pull request #7261: [AIRFLOW-6641] Better diagnostics for 
kubernetes flaky tests
URL: https://github.com/apache/airflow/pull/7261
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6641) Kubernetest tests happen to be flaky

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023683#comment-17023683
 ] 

ASF GitHub Bot commented on AIRFLOW-6641:
-

potiuk commented on pull request #7261: [AIRFLOW-6641] Better diagnostics for 
kubernetes flaky tests
URL: https://github.com/apache/airflow/pull/7261
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Kubernetest tests happen to be flaky
> 
>
> Key: AIRFLOW-6641
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6641
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> Kubernetes tests are sometimes flaky - they do not reach desired state and we 
> do not know why. We need to make it less flaky and have enough diagnostics to 
> understand what's going on.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk commented on a change in pull request #7261: [AIRFLOW-6641] Better diagnostics for kubernetes flaky tests

[GitBox]

potiuk commented on a change in pull request #7261: [AIRFLOW-6641] Better 
diagnostics for kubernetes flaky tests
URL: https://github.com/apache/airflow/pull/7261#discussion_r370962245
 
 

 ##
 File path: tests/runtime/kubernetes/test_kubernetes_executor.py
 ##
 @@ -92,18 +96,39 @@ def monitor_task(self, host, execution_date, dag_id, 
task_id, expected_final_sta
 tries += 1
 except requests.exceptions.ConnectionError as e:
 check_call(["echo", "api call failed. trying again. error 
{}".format(e)])
-
+if state != expected_final_state:
+print("The expected state is wrong {} != {} 
(expected)!".format(state, expected_final_state))
+self.dump_kubernetes_logs()
 self.assertEqual(state, expected_final_state)
 
-# Maybe check if we can retrieve the logs, but then we need to extend 
the API
+def dump_kubernetes_logs(self):
+if os.environ.get('ENABLE_KIND_CLUSTER') == 'true':
+self.dump_kind_logs()
+
+def dump_kind_logs(self):
+tempdir_path = mkdtemp()
 
 Review comment:
   I want to cherry-pick it to 1.10  - with python 2.7 support, I considered 
TemporaryDirectory - but it won't work there I am afraid .


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6643) Kerberos tests are still flaky

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023680#comment-17023680
 ] 

ASF subversion and git services commented on AIRFLOW-6643:
--

Commit f5d2c437f5d59571788600cbb617720d59777f66 in airflow's branch 
refs/heads/v1-10-test from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=f5d2c43 ]

[AIRFLOW-6643] Fix flakiness of kerberos tests

Kerberos test were still flaky because checking for kerberos token happens
before active waiting for kerberos to be ready.


> Kerberos tests are still flaky
> --
>
> Key: AIRFLOW-6643
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6643
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> Kerberos test are still flaky because checking for kerberos token happens 
> before active waiting for kerberos to be ready.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6642) Flaky test_heartbeat_failed_fast test

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023681#comment-17023681
 ] 

ASF subversion and git services commented on AIRFLOW-6642:
--

Commit 16f9f2174cc819b5477588551c47f3a91dcd24ed in airflow's branch 
refs/heads/v1-10-test from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=16f9f21 ]

[AIRFLOW-6642] Make local task job test less flaky (#7262)

The heartbeat failed fast test is flaky - sometimes the time needed to run the
test is longer than expected.

(cherry picked from commit 8936180afb9395cbf4af2ae885136caee8b2e3b7)


> Flaky test_heartbeat_failed_fast test
> -
>
> Key: AIRFLOW-6642
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6642
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> The heartbeat failed fast test is flaky - sometimes the time needed to run 
> the test is longer than expected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6642) Flaky test_heartbeat_failed_fast test

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023679#comment-17023679
 ] 

ASF subversion and git services commented on AIRFLOW-6642:
--

Commit 8936180afb9395cbf4af2ae885136caee8b2e3b7 in airflow's branch 
refs/heads/master from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=8936180 ]

[AIRFLOW-6642] Make local task job test less flaky (#7262)

The heartbeat failed fast test is flaky - sometimes the time needed to run the
test is longer than expected.

> Flaky test_heartbeat_failed_fast test
> -
>
> Key: AIRFLOW-6642
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6642
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> The heartbeat failed fast test is flaky - sometimes the time needed to run 
> the test is longer than expected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6642) Flaky test_heartbeat_failed_fast test

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023678#comment-17023678
 ] 

ASF GitHub Bot commented on AIRFLOW-6642:
-

potiuk commented on pull request #7262: [AIRFLOW-6642] Make local task job test 
less flaky
URL: https://github.com/apache/airflow/pull/7262
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Flaky test_heartbeat_failed_fast test
> -
>
> Key: AIRFLOW-6642
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6642
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> The heartbeat failed fast test is flaky - sometimes the time needed to run 
> the test is longer than expected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk merged pull request #7262: [AIRFLOW-6642] Make local task job test less flaky

[GitBox]

potiuk merged pull request #7262: [AIRFLOW-6642] Make local task job test less 
flaky
URL: https://github.com/apache/airflow/pull/7262
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7263: [AIRFLOW-XXXX] Improve docstring of AwsHook

[GitBox]

codecov-io edited a comment on issue #7263: [AIRFLOW-] Improve docstring of 
AwsHook
URL: https://github.com/apache/airflow/pull/7263#issuecomment-578452382
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=h1) 
Report
   > Merging 
[#7263](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/6584e0d887a6fbc98ab21ba062a03e30cac2923f?src=pr=desc)
 will **increase** coverage by `52.09%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7263/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7263   +/-   ##
   ===
   + Coverage   32.96%   85.05%   +52.09% 
   ===
 Files 791  791   
 Lines   4013740137   
   ===
   + Hits1323134140+20909 
   + Misses  26906 5997-20909
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/contrib/hooks/aws\_hook.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2F3c19ob29rLnB5)
 | `77.31% <ø> (+63.86%)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-43.8%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `87.57% <0%> (+0.62%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <0%> (+2.1%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+3.57%)` | :arrow_up: |
   | ... and [646 
more](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=footer). 
Last update 
[6584e0d...d2c299f](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7263: [AIRFLOW-XXXX] Improve docstring of AwsHook

[GitBox]

codecov-io commented on issue #7263: [AIRFLOW-] Improve docstring of AwsHook
URL: https://github.com/apache/airflow/pull/7263#issuecomment-578452382
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=h1) 
Report
   > Merging 
[#7263](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/6584e0d887a6fbc98ab21ba062a03e30cac2923f?src=pr=desc)
 will **increase** coverage by `52.09%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7263/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7263   +/-   ##
   ===
   + Coverage   32.96%   85.05%   +52.09% 
   ===
 Files 791  791   
 Lines   4013740137   
   ===
   + Hits1323134140+20909 
   + Misses  26906 5997-20909
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/contrib/hooks/aws\_hook.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2F3c19ob29rLnB5)
 | `77.31% <ø> (+63.86%)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-43.8%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `87.57% <0%> (+0.62%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <0%> (+2.1%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+3.57%)` | :arrow_up: |
   | ... and [646 
more](https://codecov.io/gh/apache/airflow/pull/7263/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=footer). 
Last update 
[6584e0d...d2c299f](https://codecov.io/gh/apache/airflow/pull/7263?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7262: [AIRFLOW-6642] Make local task job test less flaky

[GitBox]

codecov-io edited a comment on issue #7262: [AIRFLOW-6642] Make local task job 
test less flaky
URL: https://github.com/apache/airflow/pull/7262#issuecomment-578452207
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=h1) 
Report
   > Merging 
[#7262](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/6584e0d887a6fbc98ab21ba062a03e30cac2923f?src=pr=desc)
 will **increase** coverage by `52.09%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7262/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7262   +/-   ##
   ===
   + Coverage   32.96%   85.05%   +52.09% 
   ===
 Files 791  791   
 Lines   4013740137   
   ===
   + Hits1323134140+20909 
   + Misses  26906 5997-20909
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-43.8%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `87.57% <0%> (+0.62%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <0%> (+2.1%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+3.57%)` | :arrow_up: |
   | 
[airflow/www/forms.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvZm9ybXMucHk=)
 | `100% <0%> (+4.44%)` | :arrow_up: |
   | ... and [645 
more](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=footer). 
Last update 
[6584e0d...006716f](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7262: [AIRFLOW-6642] Make local task job test less flaky

[GitBox]

codecov-io commented on issue #7262: [AIRFLOW-6642] Make local task job test 
less flaky
URL: https://github.com/apache/airflow/pull/7262#issuecomment-578452207
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=h1) 
Report
   > Merging 
[#7262](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/6584e0d887a6fbc98ab21ba062a03e30cac2923f?src=pr=desc)
 will **increase** coverage by `52.09%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7262/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7262   +/-   ##
   ===
   + Coverage   32.96%   85.05%   +52.09% 
   ===
 Files 791  791   
 Lines   4013740137   
   ===
   + Hits1323134140+20909 
   + Misses  26906 5997-20909
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-43.8%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `87.57% <0%> (+0.62%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <0%> (+2.1%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+3.57%)` | :arrow_up: |
   | 
[airflow/www/forms.py](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvZm9ybXMucHk=)
 | `100% <0%> (+4.44%)` | :arrow_up: |
   | ... and [645 
more](https://codecov.io/gh/apache/airflow/pull/7262/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=footer). 
Last update 
[6584e0d...006716f](https://codecov.io/gh/apache/airflow/pull/7262?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7261: [AIRFLOW-6641] Better diagnostics for kubernetes flaky tests

[GitBox]

mik-laj commented on a change in pull request #7261: [AIRFLOW-6641] Better 
diagnostics for kubernetes flaky tests
URL: https://github.com/apache/airflow/pull/7261#discussion_r370961585
 
 

 ##
 File path: tests/runtime/kubernetes/test_kubernetes_executor.py
 ##
 @@ -92,18 +96,39 @@ def monitor_task(self, host, execution_date, dag_id, 
task_id, expected_final_sta
 tries += 1
 except requests.exceptions.ConnectionError as e:
 check_call(["echo", "api call failed. trying again. error 
{}".format(e)])
-
+if state != expected_final_state:
+print("The expected state is wrong {} != {} 
(expected)!".format(state, expected_final_state))
+self.dump_kubernetes_logs()
 self.assertEqual(state, expected_final_state)
 
-# Maybe check if we can retrieve the logs, but then we need to extend 
the API
+def dump_kubernetes_logs(self):
+if os.environ.get('ENABLE_KIND_CLUSTER') == 'true':
+self.dump_kind_logs()
+
+def dump_kind_logs(self):
+tempdir_path = mkdtemp()
 
 Review comment:
   I think it's better to use `tempfile.TemporaryDirectory`. Using mkdtemp is 
an old approach and is often dangerous because the file is not deleted 
automatically. This is a small nit in tests, but there was one issues that 
caught my eye.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7261: [AIRFLOW-6641] Better diagnostics for kubernetes flaky tests

[GitBox]

codecov-io commented on issue #7261: [AIRFLOW-6641] Better diagnostics for 
kubernetes flaky tests
URL: https://github.com/apache/airflow/pull/7261#issuecomment-578451391
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=h1) 
Report
   > Merging 
[#7261](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/6584e0d887a6fbc98ab21ba062a03e30cac2923f?src=pr=desc)
 will **increase** coverage by `52.09%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7261/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7261   +/-   ##
   ===
   + Coverage   32.96%   85.05%   +52.09% 
   ===
 Files 791  791   
 Lines   4013740137   
   ===
   + Hits1323134139+20908 
   + Misses  26906 5998-20908
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-43.8%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `87.57% <0%> (+0.62%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <0%> (+2.1%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+3.57%)` | :arrow_up: |
   | 
[airflow/www/forms.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvZm9ybXMucHk=)
 | `100% <0%> (+4.44%)` | :arrow_up: |
   | ... and [645 
more](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=footer). 
Last update 
[6584e0d...0c384c2](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7261: [AIRFLOW-6641] Better diagnostics for kubernetes flaky tests

[GitBox]

codecov-io edited a comment on issue #7261: [AIRFLOW-6641] Better diagnostics 
for kubernetes flaky tests
URL: https://github.com/apache/airflow/pull/7261#issuecomment-578451391
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=h1) 
Report
   > Merging 
[#7261](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/6584e0d887a6fbc98ab21ba062a03e30cac2923f?src=pr=desc)
 will **increase** coverage by `52.09%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7261/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7261   +/-   ##
   ===
   + Coverage   32.96%   85.05%   +52.09% 
   ===
 Files 791  791   
 Lines   4013740137   
   ===
   + Hits1323134139+20908 
   + Misses  26906 5998-20908
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-43.8%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `87.57% <0%> (+0.62%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <0%> (+2.1%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+3.57%)` | :arrow_up: |
   | 
[airflow/www/forms.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvZm9ybXMucHk=)
 | `100% <0%> (+4.44%)` | :arrow_up: |
   | ... and [645 
more](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=footer). 
Last update 
[6584e0d...0c384c2](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7261: [AIRFLOW-6641] Better diagnostics for kubernetes flaky tests

[GitBox]

codecov-io edited a comment on issue #7261: [AIRFLOW-6641] Better diagnostics 
for kubernetes flaky tests
URL: https://github.com/apache/airflow/pull/7261#issuecomment-578451391
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=h1) 
Report
   > Merging 
[#7261](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/6584e0d887a6fbc98ab21ba062a03e30cac2923f?src=pr=desc)
 will **increase** coverage by `52.09%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7261/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7261   +/-   ##
   ===
   + Coverage   32.96%   85.05%   +52.09% 
   ===
 Files 791  791   
 Lines   4013740137   
   ===
   + Hits1323134139+20908 
   + Misses  26906 5998-20908
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-43.8%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   | 
[airflow/bin/airflow](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vYWlyZmxvdw==)
 | `84.61% <0%> (ø)` | :arrow_up: |
   | 
[airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=)
 | `87.57% <0%> (+0.62%)` | :arrow_up: |
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <0%> (+2.1%)` | :arrow_up: |
   | 
[airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5)
 | `100% <0%> (+3.57%)` | :arrow_up: |
   | 
[airflow/www/forms.py](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvZm9ybXMucHk=)
 | `100% <0%> (+4.44%)` | :arrow_up: |
   | ... and [645 
more](https://codecov.io/gh/apache/airflow/pull/7261/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=footer). 
Last update 
[6584e0d...0c384c2](https://codecov.io/gh/apache/airflow/pull/7261?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6643) Kerberos tests are still flaky

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023675#comment-17023675
 ] 

ASF GitHub Bot commented on AIRFLOW-6643:
-

potiuk commented on pull request #7264: [AIRFLOW-6643] Fix flakiness of 
kerberos tests
URL: https://github.com/apache/airflow/pull/7264
 
 
   Kerberos test were still flaky because checking for kerberos token happens
   before active waiting for kerberos to be ready.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Kerberos tests are still flaky
> --
>
> Key: AIRFLOW-6643
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6643
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> Kerberos test are still flaky because checking for kerberos token happens 
> before active waiting for kerberos to be ready.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk opened a new pull request #7264: [AIRFLOW-6643] Fix flakiness of kerberos tests

[GitBox]

potiuk opened a new pull request #7264: [AIRFLOW-6643] Fix flakiness of 
kerberos tests
URL: https://github.com/apache/airflow/pull/7264
 
 
   Kerberos test were still flaky because checking for kerberos token happens
   before active waiting for kerberos to be ready.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-6643) Kerberos tests are still flaky

[Jarek Potiuk (Jira)]

Jarek Potiuk created AIRFLOW-6643:
-

 Summary: Kerberos tests are still flaky
 Key: AIRFLOW-6643
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6643
 Project: Apache Airflow
  Issue Type: Improvement
  Components: ci, tests
Affects Versions: 1.10.7, 2.0.0
Reporter: Jarek Potiuk


Kerberos test are still flaky because checking for kerberos token happens 
before active waiting for kerberos to be ready.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mustafagok opened a new pull request #7263: [AIRFLOW-XXXX] Improve docstring of AwsHook

[GitBox]

mustafagok opened a new pull request #7263: [AIRFLOW-] Improve docstring of 
AwsHook
URL: https://github.com/apache/airflow/pull/7263
 
 
   Add documentation for verify parameter.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6642) Flaky test_heartbeat_failed_fast test

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023662#comment-17023662
 ] 

ASF GitHub Bot commented on AIRFLOW-6642:
-

potiuk commented on pull request #7262: [AIRFLOW-6642] Make local task job test 
less flaky
URL: https://github.com/apache/airflow/pull/7262
 
 
   The heartbeat failed fast test is flaky - sometimes the time needed to run 
the
   test is longer than expected.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Flaky test_heartbeat_failed_fast test
> -
>
> Key: AIRFLOW-6642
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6642
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> The heartbeat failed fast test is flaky - sometimes the time needed to run 
> the test is longer than expected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk opened a new pull request #7262: [AIRFLOW-6642] Make local task job test less flaky

[GitBox]

potiuk opened a new pull request #7262: [AIRFLOW-6642] Make local task job test 
less flaky
URL: https://github.com/apache/airflow/pull/7262
 
 
   The heartbeat failed fast test is flaky - sometimes the time needed to run 
the
   test is longer than expected.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-6642) Flaky test_heartbeat_failed_fast test

[Jarek Potiuk (Jira)]

Jarek Potiuk created AIRFLOW-6642:
-

 Summary: Flaky test_heartbeat_failed_fast test
 Key: AIRFLOW-6642
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6642
 Project: Apache Airflow
  Issue Type: Improvement
  Components: ci, tests
Affects Versions: 1.10.7, 2.0.0
Reporter: Jarek Potiuk


The heartbeat failed fast test is flaky - sometimes the time needed to run the 
test is longer than expected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6641) Kubernetest tests happen to be flaky

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023658#comment-17023658
 ] 

ASF GitHub Bot commented on AIRFLOW-6641:
-

potiuk commented on pull request #7261: [AIRFLOW-6641] Better diagnostics for 
kubernetes flaky tests
URL: https://github.com/apache/airflow/pull/7261
 
 
   Kubernetes tests are sometimes flaky - they do not reach desired state and we
   do not know why. We need to make it less flaky and have enough diagnostics to
   understand what's going on.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Kubernetest tests happen to be flaky
> 
>
> Key: AIRFLOW-6641
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6641
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> Kubernetes tests are sometimes flaky - they do not reach desired state and we 
> do not know why. We need to make it less flaky and have enough diagnostics to 
> understand what's going on.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk opened a new pull request #7261: [AIRFLOW-6641] Better diagnostics for kubernetes flaky tests

[GitBox]

potiuk opened a new pull request #7261: [AIRFLOW-6641] Better diagnostics for 
kubernetes flaky tests
URL: https://github.com/apache/airflow/pull/7261
 
 
   Kubernetes tests are sometimes flaky - they do not reach desired state and we
   do not know why. We need to make it less flaky and have enough diagnostics to
   understand what's going on.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-6641) Kubernetest tests happen to be flaky

[Jarek Potiuk (Jira)]

Jarek Potiuk created AIRFLOW-6641:
-

 Summary: Kubernetest tests happen to be flaky
 Key: AIRFLOW-6641
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6641
 Project: Apache Airflow
  Issue Type: Improvement
  Components: ci
Affects Versions: 1.10.7, 2.0.0
Reporter: Jarek Potiuk


Kubernetes tests are sometimes flaky - they do not reach desired state and we 
do not know why. We need to make it less flaky and have enough diagnostics to 
understand what's going on.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk commented on a change in pull request #7257: [AIRFLOW-6636] Avoid exceptions when printing task instance

[GitBox]

potiuk commented on a change in pull request #7257: [AIRFLOW-6636] Avoid 
exceptions when printing task instance
URL: https://github.com/apache/airflow/pull/7257#discussion_r370957648
 
 

 ##
 File path: airflow/models/taskinstance.py
 ##
 @@ -1137,18 +1143,30 @@ def handle_failure(self, error, test_mode=None, 
context=None, session=None):
 'dag_id=%s, task_id=%s, execution_date=%s, 
start_date=%s, end_date=%s',
 self.dag_id,
 self.task_id,
-self.execution_date.strftime('%Y%m%dT%H%M%S'),
-self.start_date.strftime('%Y%m%dT%H%M%S'),
-self.end_date.strftime('%Y%m%dT%H%M%S'))
+self.execution_date.strftime('%Y%m%dT%H%M%S') if 
hasattr(
+self,
+'execution_date') and self.execution_date else '',
+self.start_date.strftime('%Y%m%dT%H%M%S') if hasattr(
+self,
+'start_date') and self.start_date else '',
+self.end_date.strftime('%Y%m%dT%H%M%S') if hasattr(
 
 Review comment:
   I agree in principle. It should be as you described. But currently we face 
different reality. Solving it in the way you describe is much bigger task.
   
   I just want to solve current tests not to show failures where there are no 
failures at all. Failing exception in "log" is not a good "test" so it does not 
hide the problem. Logs should never "detect" a problem - they should print the 
current state of the object and not randomly fail.
   
   Right now in logs you see unrelated failures, you have inconsistency in the 
way how those attributes are treated - and sometimes you think those are the 
problems with tests - where they aren't. 
   
   So right now I am not trying to solve this "bigger" problem only the small 
one. But I created 
   the issue and marked you there 
https://issues.apache.org/jira/browse/AIRFLOW-6640 to solve the bigger issue. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6640) Improve the taskinstance to make sure that atributes are not dynamic

[Jarek Potiuk (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023651#comment-17023651
 ] 

Jarek Potiuk commented on AIRFLOW-6640:
---

[~kamil.bregula] - I think you might be interested in solving it at some point 
in time (possibly before release 2.0).

> Improve the taskinstance to make sure that atributes are not dynamic
> 
>
> Key: AIRFLOW-6640
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6640
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> Task instance date-related attributes are dynamically defined - they should 
> be defined once in init() rather than dynamically added.
> See [https://github.com/apache/airflow/pull/7257#discussion_r370930282] for 
> more details.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (AIRFLOW-6640) Improve the taskinstance to make sure that atributes are not dynamic

[Jarek Potiuk (Jira)]

Jarek Potiuk created AIRFLOW-6640:
-

 Summary: Improve the taskinstance to make sure that atributes are 
not dynamic
 Key: AIRFLOW-6640
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6640
 Project: Apache Airflow
  Issue Type: Improvement
  Components: core
Affects Versions: 1.10.7, 2.0.0
Reporter: Jarek Potiuk


Task instance date-related attributes are dynamically defined - they should be 
defined once in init() rather than dynamically added.

See [https://github.com/apache/airflow/pull/7257#discussion_r370930282] for 
more details.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor

[GitBox]

mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add 
spark-on-k8s operator/hook/sensor
URL: https://github.com/apache/airflow/pull/7163#discussion_r370954667
 
 

 ##
 File path: docs/operators-and-hooks-ref.rst
 ##
 @@ -1049,8 +1049,11 @@ These integrations allow you to perform various 
operations using various softwar
* - `Kubernetes `__
  - :doc:`How to use `
  -
+ - :mod:`airflow.contrib.hooks.kubernetes_hook`
 
 Review comment:
   This is a difficult question, because it requires many components to make it 
work. After a long thought, I think it can stay here, but I will ask other 
people. It can be considered as part of the Kubernetes extension, so the 
operator and hook can be in one place.
   CC: @kaxil @nuclearpinguin @potiuk 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor

[GitBox]

mik-laj commented on issue #7163: [AIRFLOW-6542] add spark-on-k8s 
operator/hook/sensor
URL: https://github.com/apache/airflow/pull/7163#issuecomment-578440378
 
 
   > the spark-on-k8s-operator is being developed by GCP but it's not GCP 
service(can run on all Kubernetes flavors, not just GKE) so I don't know if it 
should follow GCP guidelines, or if it should and can be included in GCP system 
tests
   
   The requirements for GCP are very restrictive and these are the best we 
have. Other providers and developers can also learn from them and after 
changes, introduce to their operators. We also want to have tests for other 
services, but there were no people willing to develop system tests. Here, I 
think it is worth developing them, but this is not necessary if you do not have 
time.  However, we currently have a cluster in the CI environment, so writing 
tests should not be problematic. 
https://github.com/apache/airflow/blob/master/TESTING.rst#running-tests-with-kubernetes


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor

[GitBox]

mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add 
spark-on-k8s operator/hook/sensor
URL: https://github.com/apache/airflow/pull/7163#discussion_r370953103
 
 

 ##
 File path: airflow/contrib/operators/spark_kubernetes_operator.py
 ##
 @@ -0,0 +1,75 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Optional
+
+from kubernetes import client
+
+from airflow.contrib.hooks.kubernetes_hook import Kuberneteshook
+from airflow.exceptions import AirflowException
+from airflow.models import BaseOperator
+from airflow.utils.decorators import apply_defaults
+
+
+class SparkKubernetesOperator(BaseOperator):
+"""
+creates sparkapplication object in kubernetes cluster
+
+:param sparkapplication_object: kubernetes custom_resource_definition of 
sparkApplication
+:param namespace: kubernetes namespace to put sparkApplication
+:param kube_config: kubernetes kube_config path
+:param in_cluster: if airflow runs inside kubernetes pod take 
configuration from inside the cluster.
 
 Review comment:
   I think `kube_config` and `in_cluster` should be stored in connection and 
not be passed by the user. These values configured by the administrator should 
be constant. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor

[GitBox]

mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add 
spark-on-k8s operator/hook/sensor
URL: https://github.com/apache/airflow/pull/7163#discussion_r370952981
 
 

 ##
 File path: airflow/contrib/operators/spark_kubernetes_operator.py
 ##
 @@ -0,0 +1,75 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Optional
+
+from kubernetes import client
+
+from airflow.contrib.hooks.kubernetes_hook import Kuberneteshook
+from airflow.exceptions import AirflowException
+from airflow.models import BaseOperator
+from airflow.utils.decorators import apply_defaults
+
+
+class SparkKubernetesOperator(BaseOperator):
+"""
+creates sparkapplication object in kubernetes cluster
 
 Review comment:
   ```suggestion
   Creates Spark Application Object in Kubernetes cluster:
   ```
   Can you also add link to: 
https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/docs/api-docs.md#sparkapplication
 ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor

[GitBox]

mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add 
spark-on-k8s operator/hook/sensor
URL: https://github.com/apache/airflow/pull/7163#discussion_r370952030
 
 

 ##
 File path: airflow/contrib/hooks/kubernetes_hook.py
 ##
 @@ -0,0 +1,50 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Optional
+
+from kubernetes import client, config
+
+from airflow.hooks.base_hook import BaseHook
+
+
+class Kuberneteshook(BaseHook):
+"""
+creates kubernetes API connection.
+:param kube_config: kubeconfig - kubernetes connection string
+:param in_cluster: if airflow runs inside kubernetes pod take 
configuration from inside the cluster.
+"""
+
+def __init__(
+self,
+kube_config: Optional[str] = None,
+in_cluster: bool = False
+):
+self.kube_config = kube_config
+self.in_cluster = in_cluster
+
+def get_conn(self):
 
 Review comment:
   Can you add typehint here also? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor

[GitBox]

mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add 
spark-on-k8s operator/hook/sensor
URL: https://github.com/apache/airflow/pull/7163#discussion_r370951969
 
 

 ##
 File path: airflow/contrib/operators/spark_kubernetes_operator.py
 ##
 @@ -0,0 +1,77 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import Optional
+
+from kubernetes import client
+
+from airflow.contrib.hooks.kubernetes_hook import Kuberneteshook
+from airflow.exceptions import AirflowException
+from airflow.models import BaseOperator
+from airflow.utils.decorators import apply_defaults
+
+
+class SparkKubernetesOperator(BaseOperator):
+"""
+creates sparkapplication object in kubernetes cluster
+
+:param sparkapplication_object: kubernetes custom_resource_definition of 
sparkApplication
+:param namespace: kubernetes namespace to put sparkApplication
+:param kube_config: kubernetes kube_config path
+:param in_cluster: if airflow runs inside kubernetes pod take 
configuration from inside the cluster.
+"""
+
+template_fields = ['sparkapplication_object', 'namespace', 'kube_config']
+template_ext = ()
+ui_color = '#f4a460'
+
+@apply_defaults
+def __init__(self,
+ sparkapplication_object: dict,
+ namespace: str = 'default',
+ kube_config: Optional[str] = None,
+ in_cluster: bool = False,
+ *args, **kwargs) -> None:
+super().__init__(*args, **kwargs)
+self.sparkapplication_object = sparkapplication_object
+self.namespace = namespace
+self.kube_config = kube_config
+self.in_cluster = in_cluster
+if kwargs.get('xcom_push') is not None:
+raise AirflowException("'xcom_push' was deprecated, use 
'BaseOperator.do_xcom_push' instead")
+
+def execute(self, context):
+self.log.info("creating sparkApplication")
+hook = Kuberneteshook(
+kube_config=self.kube_config,
+in_cluster=self.in_cluster
+)
+api_client = hook.get_conn()
+api = client.CustomObjectsApi(api_client)
 
 Review comment:
   Hook should allow reuse of code. This hook ensures the best integration of 
Airflow with an external library.  It is perfectly fine if you need to create 
more methods, but you will not need to create all the methods that are 
available in the client, because not all methods make sense in the case of 
Airflow e.g. no one will perform cluster administrative operations using 
Airflow. If you create a method, you will be able to provide additional unique 
improvements for Airflow, e.g. wait for the correct completion of the 
operation, or provide a default value for some parameters (e.g. project ID or 
namespace). A possible improvement that I can now imagine is storing the 
namespace value in the connection configuration.  A similar improvement exists 
in the integration of Google Cloud Platform - 
https://github.com/apache/airflow/blob/43dcc13/airflow/gcp/hooks/base.py#L332-L358


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor

[GitBox]

mik-laj commented on a change in pull request #7163: [AIRFLOW-6542] add 
spark-on-k8s operator/hook/sensor
URL: https://github.com/apache/airflow/pull/7163#discussion_r370950333
 
 

 ##
 File path: airflow/contrib/example_dags/example_spark_kubernetes_operator.py
 ##
 @@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import timedelta
+
+import yaml
+
+# [START import_module]
+# The DAG object; we'll need this to instantiate a DAG
+from airflow import DAG
+# Operators; we need this to operate!
+from airflow.contrib.operators.spark_kubernetes_operator import 
SparkKubernetesOperator
+from airflow.contrib.sensors.spark_kubernetes_sensor import 
SparkKubernetesSensor
+from airflow.utils.dates import days_ago
+
+# [END import_module]
+
+# [START default_args]
+# These args will get passed on to each operator
+# You can override them on a per-task basis during operator initialization
+default_args = {
+'owner': 'airflow',
+'depends_on_past': False,
+'start_date': days_ago(1),
+'email': ['airf...@example.com'],
+'email_on_failure': False,
+'email_on_retry': False,
+'max_active_runs': 1
+}
+# [END default_args]
+
+# [START instantiate_dag]
+spark_application_yaml = """
+#
+# Copyright 2017 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: "sparkoperator.k8s.io/v1beta2"
+kind: SparkApplication
+metadata:
+  name: spark-pi
+  namespace: default
+spec:
+  type: Scala
+  mode: cluster
+  image: "gcr.io/spark-operator/spark:v2.4.4"
+  imagePullPolicy: Always
+  mainClass: org.apache.spark.examples.SparkPi
+  mainApplicationFile: 
"local:///opt/spark/examples/jars/spark-examples_2.11-2.4.4.jar"
+  sparkVersion: "2.4.4"
+  restartPolicy:
+type: Never
+  volumes:
+- name: "test-volume"
+  hostPath:
+path: "/tmp"
+type: Directory
+  driver:
+cores: 1
+coreLimit: "1200m"
+memory: "512m"
+labels:
+  version: 2.4.4
+serviceAccount: default
+volumeMounts:
+  - name: "test-volume"
+mountPath: "/tmp"
+  executor:
+cores: 1
+instances: 1
+memory: "512m"
+labels:
+  version: 2.4.4
+volumeMounts:
+  - name: "test-volume"
+mountPath: "/tmp"
+"""
 
 Review comment:
   Mixing code from different technologies is very problematic.I remember there 
is a word that describes this problem, but I don't remember the exact name. A 
similar problem occurs with SQL and other languages. SQL can be used in other 
code, but when it is very complex people started using ORM to connect the two 
worlds - SQL and Objects. 
   
   You can create a file in the directory next to the file. For example. If 
your Python file is in the ``airflow/contrib/example_dags`` directory, you can 
keep additional files in the ``airflow/contrib/example_dags`` directory


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] stale[bot] commented on issue #6790: [AIRFLOW-6210] Reuse SQLAlchemy session in ti_deps

[GitBox]

stale[bot] commented on issue #6790: [AIRFLOW-6210] Reuse SQLAlchemy session in 
ti_deps
URL: https://github.com/apache/airflow/pull/6790#issuecomment-578433274
 
 
   This issue has been automatically marked as stale because it has not had 
recent activity. It will be closed if no further activity occurs. Thank you for 
your contributions.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (AIRFLOW-6639) Remove duplicate Output format choices from CLI docs

[Kaxil Naik (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kaxil Naik resolved AIRFLOW-6639.
-
Resolution: Fixed

> Remove duplicate Output format choices from CLI docs
> 
>
> Key: AIRFLOW-6639
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6639
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
> Fix For: 2.0.0
>
>
> {noformat}
> Output table format. The specified value is passed to the tabulate module 
> (https://pypi.org/project/tabulate/). Valid values are: 
> (fancy_grid|github|grid|html|jira|latex|latex_booktabs|latex_raw|mediawiki|moinmoin|orgtbl|pipe|plain|presto|psql|rst|simple|textile|tsv|youtrack)
> {noformat}
> The above is repeated twice. 
> https://airflow.readthedocs.io/en/latest/cli-ref.html#tasks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6639) Remove duplicate Output format choices from CLI docs

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023624#comment-17023624
 ] 

ASF GitHub Bot commented on AIRFLOW-6639:
-

kaxil commented on pull request #7259: [AIRFLOW-6639] Remove duplicate Output 
format choices from CLI docs
URL: https://github.com/apache/airflow/pull/7259
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Remove duplicate Output format choices from CLI docs
> 
>
> Key: AIRFLOW-6639
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6639
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
> Fix For: 2.0.0
>
>
> {noformat}
> Output table format. The specified value is passed to the tabulate module 
> (https://pypi.org/project/tabulate/). Valid values are: 
> (fancy_grid|github|grid|html|jira|latex|latex_booktabs|latex_raw|mediawiki|moinmoin|orgtbl|pipe|plain|presto|psql|rst|simple|textile|tsv|youtrack)
> {noformat}
> The above is repeated twice. 
> https://airflow.readthedocs.io/en/latest/cli-ref.html#tasks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6639) Remove duplicate Output format choices from CLI docs

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023625#comment-17023625
 ] 

ASF subversion and git services commented on AIRFLOW-6639:
--

Commit 6584e0d887a6fbc98ab21ba062a03e30cac2923f in airflow's branch 
refs/heads/master from Kaxil Naik
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=6584e0d ]

[AIRFLOW-6639] Remove duplicate Output format choices from CLI docs (#7259)



> Remove duplicate Output format choices from CLI docs
> 
>
> Key: AIRFLOW-6639
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6639
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
> Fix For: 2.0.0
>
>
> {noformat}
> Output table format. The specified value is passed to the tabulate module 
> (https://pypi.org/project/tabulate/). Valid values are: 
> (fancy_grid|github|grid|html|jira|latex|latex_booktabs|latex_raw|mediawiki|moinmoin|orgtbl|pipe|plain|presto|psql|rst|simple|textile|tsv|youtrack)
> {noformat}
> The above is repeated twice. 
> https://airflow.readthedocs.io/en/latest/cli-ref.html#tasks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] kaxil merged pull request #7259: [AIRFLOW-6639] Remove duplicate Output format choices from CLI docs

[GitBox]

kaxil merged pull request #7259: [AIRFLOW-6639] Remove duplicate Output format 
choices from CLI docs
URL: https://github.com/apache/airflow/pull/7259
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (AIRFLOW-6639) Remove duplicate Output format choices from CLI docs

[Kaxil Naik (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kaxil Naik updated AIRFLOW-6639:

Fix Version/s: 2.0.0

> Remove duplicate Output format choices from CLI docs
> 
>
> Key: AIRFLOW-6639
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6639
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
> Fix For: 2.0.0
>
>
> {noformat}
> Output table format. The specified value is passed to the tabulate module 
> (https://pypi.org/project/tabulate/). Valid values are: 
> (fancy_grid|github|grid|html|jira|latex|latex_booktabs|latex_raw|mediawiki|moinmoin|orgtbl|pipe|plain|presto|psql|rst|simple|textile|tsv|youtrack)
> {noformat}
> The above is repeated twice. 
> https://airflow.readthedocs.io/en/latest/cli-ref.html#tasks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] kaxil commented on a change in pull request #7119: [AIRFLOW-5840] Add operator extra link to external task sensor

[GitBox]

kaxil commented on a change in pull request #7119: [AIRFLOW-5840] Add operator 
extra link to external task sensor
URL: https://github.com/apache/airflow/pull/7119#discussion_r370940067
 
 

 ##
 File path: airflow/sensors/external_task_sensor.py
 ##
 @@ -16,22 +16,71 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
-
 import datetime
 import os
-from typing import Optional, Union
+from typing import FrozenSet, Optional, Union
 
 from sqlalchemy import func
 
+from airflow.configuration import conf
 from airflow.exceptions import AirflowException
-from airflow.models import DagBag, DagModel, DagRun, TaskInstance
+from airflow.models import BaseOperatorLink, DagBag, DagModel, DagRun, 
TaskInstance
 from airflow.operators.dummy_operator import DummyOperator
 from airflow.sensors.base_sensor_operator import BaseSensorOperator
 from airflow.utils.decorators import apply_defaults
 from airflow.utils.session import provide_session
 from airflow.utils.state import State
 
 
+def get_possible_target_execution_dates(execution_date, execution_delta, 
execution_date_fn):
+"""
+Gets the execution date(s) of an external DAG for which an
+ExternalTaskSensor should succeed on. Default is the execution
+date itself, but it may be modified if a non-null execution delta
+or execution date function is passed in.
+
+:param execution_date: The execution date of the sensor
+:type execution_date: datetime.datetime
+:param execution_delta: Time difference between the sensor
+execution date and the target DAG run execution date. Positive
+delta looks back in time.
+:type execution_delta: Optional[datetime.timedelta]
+:param execution_date_fn: Function to compute the execution date(s)
+of the target DAG run to look at given the sensor's execution
+date.
+:type execution_date_fn: Optional[Callable]
+:return: Execution date(s) to wait for
+:rtype: List[datetime.datetime]
+"""
+if execution_delta:
+dttm = execution_date - execution_delta
+elif execution_date_fn:
+dttm = execution_date_fn(execution_date)
+else:
+dttm = execution_date
+
+return dttm if isinstance(dttm, list) else [dttm]
+
+
+class ExternalTaskLink(BaseOperatorLink):
+name = 'External DAG'
+
+def get_link(self, operator, dttm):
+possible_execution_dates = get_possible_target_execution_dates(
+execution_date=dttm,
+execution_delta=getattr(operator, 'execution_delta', None),
+execution_date_fn=None,
 
 Review comment:
   Any updates on the xcom bit @rconroy293 ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable

[GitBox]

kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make 
send_task_to_executor timeout configurable
URL: https://github.com/apache/airflow/pull/7143#discussion_r370939852
 
 

 ##
 File path: airflow/executors/celery_executor.py
 ##
 @@ -41,6 +41,8 @@
 
 CELERY_SEND_ERR_MSG_HEADER = 'Error sending Celery task'
 
+OPERATION_TIMEOUT = conf.getint('celery', 'operation_timeout')
 
 Review comment:
   Can we add a test for this? Let's say a user change this setting to 
`conf.get('celery', 'operation_timeout')` that test should fail.
   
   Also let's add a fallback value.
   
   ```
   OPERATION_TIMEOUT = conf.getint('celery', 'operation_timeout', fallback=2)
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #7256: [AIRFLOW-6635] Speed up static checks

[GitBox]

potiuk commented on issue #7256: [AIRFLOW-6635] Speed up static checks
URL: https://github.com/apache/airflow/pull/7256#issuecomment-578415871
 
 
   We are now to 9 mins elapsed time for static checks total if both jobs run 
in parallell


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable

[GitBox]

kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make 
send_task_to_executor timeout configurable
URL: https://github.com/apache/airflow/pull/7143#discussion_r370939414
 
 

 ##
 File path: airflow/config_templates/config.yml
 ##
 @@ -1159,6 +1159,14 @@
   type: string
   example: ~
   default: "prefork"
+- name: operation_timeout
+  description: |
+The number of seconds to wait before timing out send task or fetch 
task state operations
 
 Review comment:
   ```suggestion
   The number of seconds to wait before timing out ``send_task`` 
(``fetch_celery_task_state``) or ``fetch_task`` (``send_task_to_executor``) 
state operations
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (AIRFLOW-6638) test_serialized_dags test is flaky when removing files.

[Jarek Potiuk (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jarek Potiuk resolved AIRFLOW-6638.
---
Fix Version/s: 1.10.8
   Resolution: Fixed

> test_serialized_dags test is flaky when removing files.
> ---
>
> Key: AIRFLOW-6638
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6638
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci, serialization, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
> Fix For: 1.10.8
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6638) test_serialized_dags test is flaky when removing files.

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023594#comment-17023594
 ] 

ASF subversion and git services commented on AIRFLOW-6638:
--

Commit 9346bf0513d9970e43470d8d74cf0b4cad37f68e in airflow's branch 
refs/heads/master from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=9346bf0 ]

[AIRFLOW-6638] Remove flakiness test from test_serialized_db remove (#7258)

The test was failing randomly because it returned the DAGs in random order and
sometimes it failed because test_serialized_dag was removing only one file
from the list of files but the file could be repeated in
case a dag file had more than one DAG defined.

> test_serialized_dags test is flaky when removing files.
> ---
>
> Key: AIRFLOW-6638
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6638
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci, serialization, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6638) test_serialized_dags test is flaky when removing files.

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023593#comment-17023593
 ] 

ASF GitHub Bot commented on AIRFLOW-6638:
-

potiuk commented on pull request #7258: [AIRFLOW-6638] Remove flakiness test 
from test_serialized_db remove
URL: https://github.com/apache/airflow/pull/7258
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> test_serialized_dags test is flaky when removing files.
> ---
>
> Key: AIRFLOW-6638
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6638
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci, serialization, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk merged pull request #7258: [AIRFLOW-6638] Remove flakiness test from test_serialized_db remove

[GitBox]

potiuk merged pull request #7258: [AIRFLOW-6638] Remove flakiness test from 
test_serialized_db remove
URL: https://github.com/apache/airflow/pull/7258
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7260: [AIRFLOW-XXXX] Fix typo in error when getting dataflow jobs

[GitBox]

codecov-io commented on issue #7260: [AIRFLOW-] Fix typo in error when 
getting dataflow jobs
URL: https://github.com/apache/airflow/pull/7260#issuecomment-578412985
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=h1) 
Report
   > Merging 
[#7260](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/8325dd5507c95eca15a030b2aa5f3e6391734eb9?src=pr=desc)
 will **decrease** coverage by `0.27%`.
   > The diff coverage is `0%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7260/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7260  +/-   ##
   ==
   - Coverage   85.33%   85.05%   -0.28% 
   ==
 Files 791  791  
 Lines   4013740137  
   ==
   - Hits3425134140 -111 
   - Misses   5886 5997 +111
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=)
 | `88.37% <0%> (ø)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   | 
[airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==)
 | `91.88% <0%> (+0.28%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=footer). 
Last update 
[8325dd5...1fb23e3](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7260: [AIRFLOW-XXXX] Fix typo in error when getting dataflow jobs

[GitBox]

codecov-io edited a comment on issue #7260: [AIRFLOW-] Fix typo in error 
when getting dataflow jobs
URL: https://github.com/apache/airflow/pull/7260#issuecomment-578412985
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=h1) 
Report
   > Merging 
[#7260](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/8325dd5507c95eca15a030b2aa5f3e6391734eb9?src=pr=desc)
 will **decrease** coverage by `0.27%`.
   > The diff coverage is `0%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7260/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7260  +/-   ##
   ==
   - Coverage   85.33%   85.05%   -0.28% 
   ==
 Files 791  791  
 Lines   4013740137  
   ==
   - Hits3425134140 -111 
   - Misses   5886 5997 +111
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=)
 | `88.37% <0%> (ø)` | :arrow_up: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   | 
[airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7260/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==)
 | `91.88% <0%> (+0.28%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=footer). 
Last update 
[8325dd5...1fb23e3](https://codecov.io/gh/apache/airflow/pull/7260?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7259: [AIRFLOW-6639] Remove duplicate Output format choices from CLI docs

[GitBox]

codecov-io edited a comment on issue #7259: [AIRFLOW-6639] Remove duplicate 
Output format choices from CLI docs
URL: https://github.com/apache/airflow/pull/7259#issuecomment-578412347
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=h1) 
Report
   > Merging 
[#7259](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/8858a9107eac97cbc85978e909e6d6b85a2c211f?src=pr=desc)
 will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7259/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=tree)
   
   ```diff
   @@   Coverage Diff   @@
   ##   master#7259   +/-   ##
   ===
 Coverage   85.33%   85.33%   
   ===
 Files 791  791   
 Lines   4013740137   
   ===
 Hits3425234252   
 Misses   5885 5885
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7259/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <ø> (ø)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=footer). 
Last update 
[8858a91...a1c52c2](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7259: [AIRFLOW-6639] Remove duplicate Output format choices from CLI docs

[GitBox]

codecov-io commented on issue #7259: [AIRFLOW-6639] Remove duplicate Output 
format choices from CLI docs
URL: https://github.com/apache/airflow/pull/7259#issuecomment-578412347
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=h1) 
Report
   > Merging 
[#7259](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/8858a9107eac97cbc85978e909e6d6b85a2c211f?src=pr=desc)
 will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7259/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=tree)
   
   ```diff
   @@   Coverage Diff   @@
   ##   master#7259   +/-   ##
   ===
 Coverage   85.33%   85.33%   
   ===
 Files 791  791   
 Lines   4013740137   
   ===
 Hits3425234252   
 Misses   5885 5885
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7259/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5)
 | `94.73% <ø> (ø)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=footer). 
Last update 
[8858a91...a1c52c2](https://codecov.io/gh/apache/airflow/pull/7259?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7258: [AIRFLOW-6638] Remove flakiness test from test_serialized_db remove

[GitBox]

codecov-io commented on issue #7258: [AIRFLOW-6638] Remove flakiness test from 
test_serialized_db remove
URL: https://github.com/apache/airflow/pull/7258#issuecomment-578411708
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7258?src=pr=h1) 
Report
   > Merging 
[#7258](https://codecov.io/gh/apache/airflow/pull/7258?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/ce3635d64f2eba9616709fb9d998676aa34b0875?src=pr=desc)
 will **decrease** coverage by `0.27%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7258/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7258?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7258  +/-   ##
   ==
   - Coverage   85.33%   85.05%   -0.28% 
   ==
 Files 791  791  
 Lines   4013740137  
   ==
   - Hits3425234140 -112 
   - Misses   5885 5997 +112
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7258?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7258/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7258/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7258/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7258/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7258/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7258?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7258?src=pr=footer). 
Last update 
[ce3635d...05e2234](https://codecov.io/gh/apache/airflow/pull/7258?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #7260: [AIRFLOW-XXXX] Fix typo in error when getting dataflow jobs

[GitBox]

mik-laj commented on issue #7260: [AIRFLOW-] Fix typo in error when getting 
dataflow jobs
URL: https://github.com/apache/airflow/pull/7260#issuecomment-578410808
 
 
   You can also create a second commit using Github' Web UI. Then the warning 
will disappear.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj edited a comment on issue #7260: [AIRFLOW-XXXX] Fix typo in error when getting dataflow jobs

[GitBox]

mik-laj edited a comment on issue #7260: [AIRFLOW-] Fix typo in error when 
getting dataflow jobs
URL: https://github.com/apache/airflow/pull/7260#issuecomment-578410808
 
 
   You can also create a second commit using Github' Web UI. Then the warning 
will disappear. You should add "[JIRA-]" prefix to commit message. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] JonnyDaenen edited a comment on issue #7260: [AIRFLOW-XXXX] Fix typo in error when getting dataflow jobs

[GitBox]

JonnyDaenen edited a comment on issue #7260: [AIRFLOW-] Fix typo in error 
when getting dataflow jobs
URL: https://github.com/apache/airflow/pull/7260#issuecomment-578410485
 
 
   Is there a way to easily change the commit message without creating a new 
fork/commit?
   Also, how should I improve my commit message?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] JonnyDaenen commented on issue #7260: [AIRFLOW-XXXX] Fix typo in error when getting dataflow jobs

[GitBox]

JonnyDaenen commented on issue #7260: [AIRFLOW-] Fix typo in error when 
getting dataflow jobs
URL: https://github.com/apache/airflow/pull/7260#issuecomment-578410485
 
 
   Is there a way to easily change the commit title without creating a new 
fork/commit?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] JonnyDaenen opened a new pull request #7260: Fix typo in error for when getting dataflow jobs

[GitBox]

JonnyDaenen opened a new pull request #7260: Fix typo in error for when getting 
dataflow jobs
URL: https://github.com/apache/airflow/pull/7260
 
 
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [ ] Description above provides context of the change
   - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [ ] Unit tests coverage for changes (not needed for documentation changes)
   - [ ] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [ ] Relevant documentation is updated including usage instructions.
   - [ ] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7260: Fix typo in error for when getting dataflow jobs

[GitBox]

boring-cyborg[bot] commented on issue #7260: Fix typo in error for when getting 
dataflow jobs
URL: https://github.com/apache/airflow/pull/7260#issuecomment-578410035
 
 
   Congratulations on your first Pull Request and welcome to the Apache Airflow 
community! If you have any issues or are unsure about any anything please check 
our Contribution Guide 
(https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (flake8, pylint and type 
annotations). Our [pre-commits]( 
https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks)
 will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in 
`docs/` directory). Adding a new operator? Check this short 
[guide](https://github.com/apache/airflow/blob/master/docs/howto/custom-operator.rst)
 Consider adding an example DAG that shows how users should use it.
   - Consider using [Breeze 
environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for 
testing locally, it’s a heavy docker but it ships with a working Airflow and a 
lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get 
the final approval from Committers.
   Apache Airflow is a community-driven project and together we are making it 
better .
   In case of doubts contact the developers at:
   Mailing List: d...@airflow.apache.org
   Slack: https://apache-airflow-slack.herokuapp.com/
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] zhongjiajie commented on issue #6900: [AIRFLOW-6346] Enhance dag default_view and orientation

[GitBox]

zhongjiajie commented on issue #6900: [AIRFLOW-6346] Enhance dag default_view 
and orientation
URL: https://github.com/apache/airflow/pull/6900#issuecomment-578409576
 
 
   According to 
https://github.com/apache/airflow/pull/6900#discussion_r363106891 no one know 
the reason,  @potiuk @kaxil I think is ok to change property name from 
`_default_view` to `default_view`, and I test it locally and it work, So pls 
task a look a this PR


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6639) Remove duplicate Output format choices from CLI docs

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023579#comment-17023579
 ] 

ASF GitHub Bot commented on AIRFLOW-6639:
-

kaxil commented on pull request #7259: [AIRFLOW-6639] Remove duplicate Output 
format choices from CLI docs
URL: https://github.com/apache/airflow/pull/7259
 
 
   ```
   Output table format. The specified value is passed to the tabulate module 
(https://pypi.org/project/tabulate/). Valid values are: 
(fancy_grid|github|grid|html|jira|latex|latex_booktabs|latex_raw|mediawiki|moinmoin|orgtbl|pipe|plain|presto|psql|rst|simple|textile|tsv|youtrack)
   ```
   
   The above is repeated twice. 
https://airflow.readthedocs.io/en/latest/cli-ref.html#tasks
   
   
![image](https://user-images.githubusercontent.com/8811558/73122413-be6c9a00-3faa-11ea-993c-a2b11f947068.png)
   
   
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Remove duplicate Output format choices from CLI docs
> 
>
> Key: AIRFLOW-6639
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6639
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
>
> {noformat}
> Output table format. The specified value is passed to the tabulate module 
> (https://pypi.org/project/tabulate/). Valid values are: 
> (fancy_grid|github|grid|html|jira|latex|latex_booktabs|latex_raw|mediawiki|moinmoin|orgtbl|pipe|plain|presto|psql|rst|simple|textile|tsv|youtrack)
> {noformat}
> The above is repeated twice. 
> https://airflow.readthedocs.io/en/latest/cli-ref.html#tasks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] kaxil opened a new pull request #7259: [AIRFLOW-6639] Remove duplicate Output format choices from CLI docs

[GitBox]

kaxil opened a new pull request #7259: [AIRFLOW-6639] Remove duplicate Output 
format choices from CLI docs
URL: https://github.com/apache/airflow/pull/7259
 
 
   ```
   Output table format. The specified value is passed to the tabulate module 
(https://pypi.org/project/tabulate/). Valid values are: 
(fancy_grid|github|grid|html|jira|latex|latex_booktabs|latex_raw|mediawiki|moinmoin|orgtbl|pipe|plain|presto|psql|rst|simple|textile|tsv|youtrack)
   ```
   
   The above is repeated twice. 
https://airflow.readthedocs.io/en/latest/cli-ref.html#tasks
   
   
![image](https://user-images.githubusercontent.com/8811558/73122413-be6c9a00-3faa-11ea-993c-a2b11f947068.png)
   
   
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] zhongjiajie commented on a change in pull request #6900: [AIRFLOW-6346] Enhance dag default_view and orientation

[GitBox]

zhongjiajie commented on a change in pull request #6900: [AIRFLOW-6346] Enhance 
dag default_view and orientation
URL: https://github.com/apache/airflow/pull/6900#discussion_r37093
 
 

 ##
 File path: airflow/models/dag.py
 ##
 @@ -296,8 +298,16 @@ def __init__(
 self.max_active_runs = max_active_runs
 self.dagrun_timeout = dagrun_timeout
 self.sla_miss_callback = sla_miss_callback
-self._default_view = default_view
 
 Review comment:
   So I think is ok to change to `default_view `


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6639) Remove duplicate Output format choices from CLI docs

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023578#comment-17023578
 ] 

ASF subversion and git services commented on AIRFLOW-6639:
--

Commit a1c52c20f885b5db60147880baafaa97ce0c68b5 in airflow's branch 
refs/heads/AIRFLOW-6639 from Kaxil Naik
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=a1c52c2 ]

[AIRFLOW-6639] Remove duplicate Output format choices from CLI docs


> Remove duplicate Output format choices from CLI docs
> 
>
> Key: AIRFLOW-6639
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6639
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Kaxil Naik
>Assignee: Kaxil Naik
>Priority: Minor
>
> {noformat}
> Output table format. The specified value is passed to the tabulate module 
> (https://pypi.org/project/tabulate/). Valid values are: 
> (fancy_grid|github|grid|html|jira|latex|latex_booktabs|latex_raw|mediawiki|moinmoin|orgtbl|pipe|plain|presto|psql|rst|simple|textile|tsv|youtrack)
> {noformat}
> The above is repeated twice. 
> https://airflow.readthedocs.io/en/latest/cli-ref.html#tasks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] mik-laj commented on issue #7248: [AIRFLOW-4204] Update super() calls

[GitBox]

mik-laj commented on issue #7248: [AIRFLOW-4204] Update super() calls
URL: https://github.com/apache/airflow/pull/7248#issuecomment-578409238
 
 
   I want to do this when I finish AIP-21 to avoid conflicts. The pyupgrade 
also deletes file encoding information - `# -*- coding: utf-8 -*-`, so it's a 
lot of conflicts.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-6639) Remove duplicate Output format choices from CLI docs

[Kaxil Naik (Jira)]

Kaxil Naik created AIRFLOW-6639:
---

 Summary: Remove duplicate Output format choices from CLI docs
 Key: AIRFLOW-6639
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6639
 Project: Apache Airflow
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.0.0
Reporter: Kaxil Naik
Assignee: Kaxil Naik



{noformat}
Output table format. The specified value is passed to the tabulate module 
(https://pypi.org/project/tabulate/). Valid values are: 
(fancy_grid|github|grid|html|jira|latex|latex_booktabs|latex_raw|mediawiki|moinmoin|orgtbl|pipe|plain|presto|psql|rst|simple|textile|tsv|youtrack)
{noformat}


The above is repeated twice. 
https://airflow.readthedocs.io/en/latest/cli-ref.html#tasks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk merged pull request #7256: [AIRFLOW-6635] Speed up static checks

[GitBox]

potiuk merged pull request #7256: [AIRFLOW-6635] Speed up static checks
URL: https://github.com/apache/airflow/pull/7256
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (AIRFLOW-6635) Speed up static checks

[Jarek Potiuk (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jarek Potiuk resolved AIRFLOW-6635.
---
Fix Version/s: 2.0.0
   Resolution: Fixed

> Speed up static checks
> --
>
> Key: AIRFLOW-6635
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6635
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
> Fix For: 2.0.0
>
>
> Pylint checks take most of the time of static checks. We have currently two 
> checks for pylint:
>  * pylint checks for main code
>  * pylint checks for tests
> The main code checks run for about 3 minutes, where pylint checks for tests 
> runs for 6 minutes. - which adds up to 9 minutes of the static check. We can 
> speed it up by splitting static checks to "everything + pylint" and 
> "pylint_tests only". 
> Also we can likely speed up the tests pylint check by enabling native 
> parallel pylint. We cannot do it for main tests because we use custom plugins 
> there, but those are not used in tests for now  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6635) Speed up static checks

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023576#comment-17023576
 ] 

ASF GitHub Bot commented on AIRFLOW-6635:
-

potiuk commented on pull request #7256: [AIRFLOW-6635] Speed up static checks
URL: https://github.com/apache/airflow/pull/7256
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Speed up static checks
> --
>
> Key: AIRFLOW-6635
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6635
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
>
> Pylint checks take most of the time of static checks. We have currently two 
> checks for pylint:
>  * pylint checks for main code
>  * pylint checks for tests
> The main code checks run for about 3 minutes, where pylint checks for tests 
> runs for 6 minutes. - which adds up to 9 minutes of the static check. We can 
> speed it up by splitting static checks to "everything + pylint" and 
> "pylint_tests only". 
> Also we can likely speed up the tests pylint check by enabling native 
> parallel pylint. We cannot do it for main tests because we use custom plugins 
> there, but those are not used in tests for now  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6635) Speed up static checks

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023577#comment-17023577
 ] 

ASF subversion and git services commented on AIRFLOW-6635:
--

Commit 8325dd5507c95eca15a030b2aa5f3e6391734eb9 in airflow's branch 
refs/heads/master from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=8325dd5 ]

[AIRFLOW-6635] Speed up static checks (#7256)



> Speed up static checks
> --
>
> Key: AIRFLOW-6635
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6635
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
> Fix For: 2.0.0
>
>
> Pylint checks take most of the time of static checks. We have currently two 
> checks for pylint:
>  * pylint checks for main code
>  * pylint checks for tests
> The main code checks run for about 3 minutes, where pylint checks for tests 
> runs for 6 minutes. - which adds up to 9 minutes of the static check. We can 
> speed it up by splitting static checks to "everything + pylint" and 
> "pylint_tests only". 
> Also we can likely speed up the tests pylint check by enabling native 
> parallel pylint. We cannot do it for main tests because we use custom plugins 
> there, but those are not used in tests for now  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk commented on issue #7248: [AIRFLOW-4204] Update super() calls

[GitBox]

potiuk commented on issue #7248: [AIRFLOW-4204] Update super() calls
URL: https://github.com/apache/airflow/pull/7248#issuecomment-578408944
 
 
   @mik-laj. Good idea! PR is welcome for that :) 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6638) test_serialized_dags test is flaky when removing files.

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023574#comment-17023574
 ] 

ASF subversion and git services commented on AIRFLOW-6638:
--

Commit 7bac8fd740da67e854fc377a4692e77120da8ddd in airflow's branch 
refs/heads/v1-10-test from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=7bac8fd ]

fixup! [AIRFLOW-6638] Remove flakiness test from test_serialized_db remove


> test_serialized_dags test is flaky when removing files.
> ---
>
> Key: AIRFLOW-6638
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6638
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci, serialization, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6638) test_serialized_dags test is flaky when removing files.

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023573#comment-17023573
 ] 

ASF subversion and git services commented on AIRFLOW-6638:
--

Commit 81b6ecc6e929a485566224be3fc1726e05d3bbba in airflow's branch 
refs/heads/v1-10-test from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=81b6ecc ]

[AIRFLOW-6638] Remove flakiness test from test_serialized_db remove

The test was failing randomly because it returned the DAGs in random
order and sometimes it failed because:

a) example_external_test_marker.py was repeated and had the same
   id as example_external_test_marker_dag.py

b) test_serialized_dag was removing only one file from the list
   of files but the file could be repeated in case a dag file
   had more than one DAG defined.


> test_serialized_dags test is flaky when removing files.
> ---
>
> Key: AIRFLOW-6638
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6638
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci, serialization, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6638) test_serialized_dags test is flaky when removing files.

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023572#comment-17023572
 ] 

ASF GitHub Bot commented on AIRFLOW-6638:
-

potiuk commented on pull request #7258: [AIRFLOW-6638] Remove flakiness test 
from test_serialized_db remove
URL: https://github.com/apache/airflow/pull/7258
 
 
   The test was failing randomly because it returned the DAGs in random order 
and
   sometimes it failed because test_serialized_dag was removing only one file
   from the list of files but the file could be repeated in
   case a dag file had more than one DAG defined.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> test_serialized_dags test is flaky when removing files.
> ---
>
> Key: AIRFLOW-6638
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6638
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci, serialization, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk opened a new pull request #7258: [AIRFLOW-6638] Remove flakiness test from test_serialized_db remove

[GitBox]

potiuk opened a new pull request #7258: [AIRFLOW-6638] Remove flakiness test 
from test_serialized_db remove
URL: https://github.com/apache/airflow/pull/7258
 
 
   The test was failing randomly because it returned the DAGs in random order 
and
   sometimes it failed because test_serialized_dag was removing only one file
   from the list of files but the file could be repeated in
   case a dag file had more than one DAG defined.
   
   ---
   Issue link: WILL BE INSERTED BY 
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = 
JIRA ID*
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow 
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   * For document-only changes commit message can start with 
`[AIRFLOW-]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in 
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request 
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
 for more information.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-6636) Sometimes start/execution/end dates are not set for tasks (mostly in tests)

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023568#comment-17023568
 ] 

ASF subversion and git services commented on AIRFLOW-6636:
--

Commit a8b08e6ea25fe4986dc3b8ff3251751f3d7e96d4 in airflow's branch 
refs/heads/v1-10-test from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=a8b08e6 ]

[AIRFLOW-6636] Avoid exceptions when printing task instance

Sometimes (mainly in tests) tasks do not have start/execution/end date set and
printing task instance information throws an exception which cover the real
problems and makes diagnostics difficult.


> Sometimes start/execution/end dates are not set for tasks (mostly in tests)
> ---
>
> Key: AIRFLOW-6636
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6636
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> Sometimes (mainly in tests) tasks do not have start/execution/end date set 
> and printing task instance information throws an exception which cover the 
> real problems and makes diagnostics difficult.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6637) Wrong "airflow task test" command used in tests in 1.10.7

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023569#comment-17023569
 ] 

ASF subversion and git services commented on AIRFLOW-6637:
--

Commit ebfc7122bd094e4876900a1a09aa558776884649 in airflow's branch 
refs/heads/v1-10-test from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=ebfc712 ]

[AIRFLOW-6637] Fix airflow test command in 1.10.x

In test_standard_task_runner.py 'airflow task test" command was used but in
1.10 it should be 'airflow test' instead. The tests usually passed because they
were not testing the actual execution and killed the tasks quickly but
sometimes (when the execution of airflow task test command was fast enough) the
tests failed.


> Wrong "airflow task test" command used in tests in 1.10.7 
> --
>
> Key: AIRFLOW-6637
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6637
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci, tests
>Affects Versions: 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>
> In test_standard_task_runner.py 'airflow task test" command was used but in 
> 1.10 it should be 'airflow test' instead. The tests usually passed because 
> they were not testing the actual execution and killed the tasks quickly but 
> sometimes (when the execution of airflow task test command was fast enough) 
> the tests failed. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-6638) test_serialized_dags test is flaky when removing files.

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023570#comment-17023570
 ] 

ASF subversion and git services commented on AIRFLOW-6638:
--

Commit 2437ec99fcc331dfd339f0bfab1e7bb9e413f9a6 in airflow's branch 
refs/heads/v1-10-test from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=2437ec9 ]

[AIRFLOW-6638] Remove flakiness test from test_serialized_db remove

The test was failing randomly because it returned the DAGs in random
order and sometimes it failed because:

a) example_external_test_marker.py was repeated and had the same
   id as example_external_test_marker_dag.py

b) test_serialized_dag was removing only one file from the list
   of files but the file could be repeated in case a dag file
   had more than one DAG defined.


> test_serialized_dags test is flaky when removing files.
> ---
>
> Key: AIRFLOW-6638
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6638
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci, serialization, tests
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (AIRFLOW-6638) test_serialized_dags test is flaky when removing files.

[Jarek Potiuk (Jira)]

Jarek Potiuk created AIRFLOW-6638:
-

 Summary: test_serialized_dags test is flaky when removing files.
 Key: AIRFLOW-6638
 URL: https://issues.apache.org/jira/browse/AIRFLOW-6638
 Project: Apache Airflow
  Issue Type: Bug
  Components: ci, serialization, tests
Affects Versions: 1.10.7, 2.0.0
Reporter: Jarek Potiuk






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] codecov-io edited a comment on issue #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable

[GitBox]

codecov-io edited a comment on issue #7143: [AIRFLOW-6527] Make 
send_task_to_executor timeout configurable
URL: https://github.com/apache/airflow/pull/7143#issuecomment-573518018
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7143?src=pr=h1) 
Report
   > Merging 
[#7143](https://codecov.io/gh/apache/airflow/pull/7143?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/8858a9107eac97cbc85978e909e6d6b85a2c211f?src=pr=desc)
 will **decrease** coverage by `52.78%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7143/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7143?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ##   master#7143   +/-   ##
   ===
   - Coverage   85.33%   32.54%   -52.79% 
   ===
 Files 791  790-1 
 Lines   4013740125   -12 
   ===
   - Hits3425213060-21192 
   - Misses   588527065+21180
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7143?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5)
 | `85.13% <100%> (-3.31%)` | :arrow_down: |
   | 
[...low/contrib/operators/wasb\_delete\_blob\_operator.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy93YXNiX2RlbGV0ZV9ibG9iX29wZXJhdG9yLnB5)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[...flow/contrib/example\_dags/example\_qubole\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9zZW5zb3IucHk=)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[...ample\_dags/example\_emr\_job\_flow\_automatic\_steps.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX2Vtcl9qb2JfZmxvd19hdXRvbWF0aWNfc3RlcHMucHk=)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[...contrib/example\_dags/example\_papermill\_operator.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3BhcGVybWlsbF9vcGVyYXRvci5weQ==)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[airflow/hooks/mssql\_hook.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9tc3NxbF9ob29rLnB5)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[airflow/hooks/webhdfs\_hook.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy93ZWJoZGZzX2hvb2sucHk=)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[airflow/contrib/sensors/emr\_base\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvZW1yX2Jhc2Vfc2Vuc29yLnB5)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[airflow/operators/python\_operator.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcHl0aG9uX29wZXJhdG9yLnB5)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[airflow/contrib/operators/gcs\_list\_operator.py](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3NfbGlzdF9vcGVyYXRvci5weQ==)
 | `0% <0%> (-100%)` | :arrow_down: |
   | ... and [648 
more](https://codecov.io/gh/apache/airflow/pull/7143/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7143?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7143?src=pr=footer). 
Last update 
[8858a91...69a9a9a](https://codecov.io/gh/apache/airflow/pull/7143?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] yuqian90 commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable

[GitBox]

yuqian90 commented on a change in pull request #7143: [AIRFLOW-6527] Make 
send_task_to_executor timeout configurable
URL: https://github.com/apache/airflow/pull/7143#discussion_r370933947
 
 

 ##
 File path: airflow/config_templates/default_airflow.cfg
 ##
 @@ -537,6 +537,9 @@ ssl_cacert =
 # https://docs.celeryproject.org/en/latest/userguide/concurrency/eventlet.html
 pool = prefork
 
+# The number of seconds to wait before timing out apply_async
+connection_timeout = 2
 
 Review comment:
   @kaxil @ashb  I renamed it to `operation_timeout` because it's the timeout 
for the `send_task_to_executor()` and `fetch_celery_task_state()` operations. 
Please let me know if you have better suggestions.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #7248: [AIRFLOW-4204] Update super() calls

[GitBox]

mik-laj commented on issue #7248: [AIRFLOW-4204] Update super() calls
URL: https://github.com/apache/airflow/pull/7248#issuecomment-578406696
 
 
   How will we prevent regression so that a similar situation does not happen 
in the future? I think [pyupgrade](https://github.com/asottile/pyupgrade) can 
solve this and similar problems


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7257: [AIRFLOW-6636] Avoid exceptions when printing task instance

[GitBox]

codecov-io commented on issue #7257: [AIRFLOW-6636] Avoid exceptions when 
printing task instance
URL: https://github.com/apache/airflow/pull/7257#issuecomment-578402860
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=h1) 
Report
   > Merging 
[#7257](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/ce3635d64f2eba9616709fb9d998676aa34b0875?src=pr=desc)
 will **decrease** coverage by `0.28%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7257/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7257  +/-   ##
   ==
   - Coverage   85.33%   85.05%   -0.29% 
   ==
 Files 791  791  
 Lines   4013740137  
   ==
   - Hits3425234137 -115 
   - Misses   5885 6000 +115
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5)
 | `94.53% <ø> (-0.44%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=footer). 
Last update 
[ce3635d...9de1854](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7257: [AIRFLOW-6636] Avoid exceptions when printing task instance

[GitBox]

codecov-io edited a comment on issue #7257: [AIRFLOW-6636] Avoid exceptions 
when printing task instance
URL: https://github.com/apache/airflow/pull/7257#issuecomment-578402860
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=h1) 
Report
   > Merging 
[#7257](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/ce3635d64f2eba9616709fb9d998676aa34b0875?src=pr=desc)
 will **decrease** coverage by `0.28%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/7257/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=tree)
   
   ```diff
   @@Coverage Diff @@
   ##   master#7257  +/-   ##
   ==
   - Coverage   85.33%   85.05%   -0.29% 
   ==
 Files 791  791  
 Lines   4013740137  
   ==
   - Hits3425234137 -115 
   - Misses   5885 6000 +115
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5)
 | `94.53% <ø> (-0.44%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5)
 | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | 
[...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7257/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==)
 | `76.47% <0%> (-21.18%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=footer). 
Last update 
[ce3635d...9de1854](https://codecov.io/gh/apache/airflow/pull/7257?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid exceptions when printing task instance

[GitBox]

mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid 
exceptions when printing task instance
URL: https://github.com/apache/airflow/pull/7257#discussion_r370930282
 
 

 ##
 File path: airflow/models/taskinstance.py
 ##
 @@ -1137,18 +1143,30 @@ def handle_failure(self, error, test_mode=None, 
context=None, session=None):
 'dag_id=%s, task_id=%s, execution_date=%s, 
start_date=%s, end_date=%s',
 self.dag_id,
 self.task_id,
-self.execution_date.strftime('%Y%m%dT%H%M%S'),
-self.start_date.strftime('%Y%m%dT%H%M%S'),
-self.end_date.strftime('%Y%m%dT%H%M%S'))
+self.execution_date.strftime('%Y%m%dT%H%M%S') if 
hasattr(
+self,
+'execution_date') and self.execution_date else '',
+self.start_date.strftime('%Y%m%dT%H%M%S') if hasattr(
+self,
+'start_date') and self.start_date else '',
+self.end_date.strftime('%Y%m%dT%H%M%S') if hasattr(
 
 Review comment:
   I think this attribute should always be created. If it is not needed, it 
should have an empty value. It is better to solve the problem of missing 
attributes than to hide the problem. All attributes, that the class has, should 
be created in the __init__ method.
   ```python
   self.start_date.strftime('%Y%m%dT%H%M%S') if self.start_date else None,
   ```
   The use of getattr is a trick that allows you to write dynamic code, but we 
have a known class here that has known attributes.  We don't need dynamic 
objects here, but stable and solid types.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid exceptions when printing task instance

[GitBox]

mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid 
exceptions when printing task instance
URL: https://github.com/apache/airflow/pull/7257#discussion_r370930282
 
 

 ##
 File path: airflow/models/taskinstance.py
 ##
 @@ -1137,18 +1143,30 @@ def handle_failure(self, error, test_mode=None, 
context=None, session=None):
 'dag_id=%s, task_id=%s, execution_date=%s, 
start_date=%s, end_date=%s',
 self.dag_id,
 self.task_id,
-self.execution_date.strftime('%Y%m%dT%H%M%S'),
-self.start_date.strftime('%Y%m%dT%H%M%S'),
-self.end_date.strftime('%Y%m%dT%H%M%S'))
+self.execution_date.strftime('%Y%m%dT%H%M%S') if 
hasattr(
+self,
+'execution_date') and self.execution_date else '',
+self.start_date.strftime('%Y%m%dT%H%M%S') if hasattr(
+self,
+'start_date') and self.start_date else '',
+self.end_date.strftime('%Y%m%dT%H%M%S') if hasattr(
 
 Review comment:
   I think this attribute should always be created. If it is not needed, it 
should have an empty value. It is better to solve the problem of missing 
attributes than to hide the problem. All attributes, that the class has, should 
be created in the __init__ method.
   ```
   self.start_date.strftime('%Y%m%dT%H%M%S') if self.start_date else None,
   ```
   The use of getattr is a trick that allows you to write dynamic code, but we 
have a known class here that has known attributes.  We don't need dynamic 
objects here, but stable and solid types.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid exceptions when printing task instance

[GitBox]

mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid 
exceptions when printing task instance
URL: https://github.com/apache/airflow/pull/7257#discussion_r370930282
 
 

 ##
 File path: airflow/models/taskinstance.py
 ##
 @@ -1137,18 +1143,30 @@ def handle_failure(self, error, test_mode=None, 
context=None, session=None):
 'dag_id=%s, task_id=%s, execution_date=%s, 
start_date=%s, end_date=%s',
 self.dag_id,
 self.task_id,
-self.execution_date.strftime('%Y%m%dT%H%M%S'),
-self.start_date.strftime('%Y%m%dT%H%M%S'),
-self.end_date.strftime('%Y%m%dT%H%M%S'))
+self.execution_date.strftime('%Y%m%dT%H%M%S') if 
hasattr(
+self,
+'execution_date') and self.execution_date else '',
+self.start_date.strftime('%Y%m%dT%H%M%S') if hasattr(
+self,
+'start_date') and self.start_date else '',
+self.end_date.strftime('%Y%m%dT%H%M%S') if hasattr(
 
 Review comment:
   I think this attribute should always be created. If it is not needed, it 
should have an empty value. It is better to solve the problem of missing 
attributes than to hide the problem. All attributes, that the class has, should 
be created in the __init__ method.
   ```
   self.start_date.strftime('%Y%m%dT%H%M%S') if self.start_date else None,
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid exceptions when printing task instance

[GitBox]

mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid 
exceptions when printing task instance
URL: https://github.com/apache/airflow/pull/7257#discussion_r370930282
 
 

 ##
 File path: airflow/models/taskinstance.py
 ##
 @@ -1137,18 +1143,30 @@ def handle_failure(self, error, test_mode=None, 
context=None, session=None):
 'dag_id=%s, task_id=%s, execution_date=%s, 
start_date=%s, end_date=%s',
 self.dag_id,
 self.task_id,
-self.execution_date.strftime('%Y%m%dT%H%M%S'),
-self.start_date.strftime('%Y%m%dT%H%M%S'),
-self.end_date.strftime('%Y%m%dT%H%M%S'))
+self.execution_date.strftime('%Y%m%dT%H%M%S') if 
hasattr(
+self,
+'execution_date') and self.execution_date else '',
+self.start_date.strftime('%Y%m%dT%H%M%S') if hasattr(
+self,
+'start_date') and self.start_date else '',
+self.end_date.strftime('%Y%m%dT%H%M%S') if hasattr(
 
 Review comment:
   I think this attribute should always be created. If it is not needed, it 
should have an empty value. It is better to solve the problem of missing 
attributes than to hide the problem. All attributes, that the class has, should 
be created in the __init__ method.
   ```
   self.start_date.strftime('%Y%m%dT%H%M%S') if 
self.start_date else None,
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid exceptions when printing task instance

[GitBox]

mik-laj commented on a change in pull request #7257: [AIRFLOW-6636] Avoid 
exceptions when printing task instance
URL: https://github.com/apache/airflow/pull/7257#discussion_r370930282
 
 

 ##
 File path: airflow/models/taskinstance.py
 ##
 @@ -1137,18 +1143,30 @@ def handle_failure(self, error, test_mode=None, 
context=None, session=None):
 'dag_id=%s, task_id=%s, execution_date=%s, 
start_date=%s, end_date=%s',
 self.dag_id,
 self.task_id,
-self.execution_date.strftime('%Y%m%dT%H%M%S'),
-self.start_date.strftime('%Y%m%dT%H%M%S'),
-self.end_date.strftime('%Y%m%dT%H%M%S'))
+self.execution_date.strftime('%Y%m%dT%H%M%S') if 
hasattr(
+self,
+'execution_date') and self.execution_date else '',
+self.start_date.strftime('%Y%m%dT%H%M%S') if hasattr(
+self,
+'start_date') and self.start_date else '',
+self.end_date.strftime('%Y%m%dT%H%M%S') if hasattr(
 
 Review comment:
   I think this attribute should always be created. If it is not needed, it 
should have an empty value. It is better to solve the problem of missing 
attributes than to hide the problem. All attributes, that the class has, should 
be created in the __init__ method.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services