[GitHub] [airflow] mingrammer commented on issue #6474: [AIRFLOW-5787] Moving AWS SQS to /providers/aws.

[GitBox]

mingrammer commented on issue #6474: [AIRFLOW-5787] Moving AWS SQS to 
/providers/aws.
URL: https://github.com/apache/airflow/pull/6474#issuecomment-550954144
 
 
   This should be moved into `amazon/aws` based on 
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Votingresults(seevotingbelow)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mingrammer edited a comment on issue #6474: [AIRFLOW-5787] Moving AWS SQS to /providers/aws.

[GitBox]

mingrammer edited a comment on issue #6474: [AIRFLOW-5787] Moving AWS SQS to 
/providers/aws.
URL: https://github.com/apache/airflow/pull/6474#issuecomment-550954144
 
 
   This should be moved into `amazon/aws` I think, based on 
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Votingresults(seevotingbelow)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mingrammer commented on issue #6502: [AIRFLOW-5786] Migrate AWS SNS to /providers/aws

[GitBox]

mingrammer commented on issue #6502: [AIRFLOW-5786] Migrate AWS SNS to 
/providers/aws
URL: https://github.com/apache/airflow/pull/6502#issuecomment-550951012
 
 
   Updated AIP-21: 
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Votingresults(seevotingbelow)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Update S3Hook import paths [AIP-21]

[GitBox]

mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Update S3Hook import 
paths [AIP-21]
URL: https://github.com/apache/airflow/pull/6465#issuecomment-550946555
 
 
   By the final voting result, It seems to have to move `S3Hook` into 
`/providers/amazon/aws`.
   
   Should I update it with an another PR?
   
   > 
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Votingresults(seevotingbelow)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Update S3Hook import paths [AIP-21]

[GitBox]

mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Update S3Hook import 
paths [AIP-21]
URL: https://github.com/apache/airflow/pull/6465#issuecomment-550946555
 
 
   By the final voting result, It seems to have to move `S3Hook` into 
`/providers/amazon/aws`.
   
   Should I update it with an another PR?
   
   > By the final voting result, It seems to have to move `S3Hook` into 
`/providers/amazon/aws`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mingrammer commented on issue #6465: [AIRFLOW-5803] Update S3Hook import paths [AIP-21]

[GitBox]

mingrammer commented on issue #6465: [AIRFLOW-5803] Update S3Hook import paths 
[AIP-21]
URL: https://github.com/apache/airflow/pull/6465#issuecomment-550946555
 
 
   By the final voting result, It seems to have to move `S3Hook` into 
`/providers/amazon/aws`.
   
   Should I update it with an another PR?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] XD-DENG commented on issue #6510: AIRFLOW-5835-unicode-encing-error-in-init

[GitBox]

XD-DENG commented on issue #6510: AIRFLOW-5835-unicode-encing-error-in-init
URL: https://github.com/apache/airflow/pull/6510#issuecomment-550863598
 
 
   Hi @jojo19893 , please ensure that your PR subject and your commit message 
follows the convention listed in the PR template.
   
   The specific format helps GitHub link with the specific JIRA ticket, and 
ease generating CHANGELOG.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] zhongjiajie commented on issue #6487: [AIRFLOW-XXX] Alphabetical table and remove duplicate

[GitBox]

zhongjiajie commented on issue #6487: [AIRFLOW-XXX] Alphabetical table and 
remove duplicate
URL: https://github.com/apache/airflow/pull/6487#issuecomment-550838772
 
 
   @mik-laj I change Google service part and ignore the prefix "Cloud", PTAL


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-5863) Airflow dag with past end_date doesn't trigger schedule

[Alex Anto Navis Lawrence (Jira)]

Alex Anto Navis Lawrence created AIRFLOW-5863:
-

 Summary: Airflow dag with past end_date doesn't trigger schedule
 Key: AIRFLOW-5863
 URL: https://issues.apache.org/jira/browse/AIRFLOW-5863
 Project: Apache Airflow
  Issue Type: Bug
  Components: scheduler
Affects Versions: 1.10.4
Reporter: Alex Anto Navis Lawrence


Hi all,

I have a airflow dag with the following configuration:

{{}}
{code:java}
{
 'start_date': datetime(2017, 1, 1, 0, 0, 0)
 'end_date': datetime(2019, 6, 1, 0, 0, 0),
  schedule_interval='@yearly',
  catchup=True
}

{code}
{{}}

For the above, 2 schedules are created and run.

{{}}
{code:java}
=> 2017-01-01, 2018-01-01.{code}
It doesn't create for 2019-01-01, since I understand the scheduler schedules 
only after the schedule interval is completed. In this case 2019 is still not 
over, so it is not scheduled.

Since I have given the end_date which is in the past, shouldn't the scheduler 
understand the end_date is over and trigger that.

I know there is a solution where by adjusting the start_date to one year back 
and adjusting the date ranges(execution_date + 1 year) within the job can solve 
this issue. (e.g. schedule it from start_date: 2016-01-01). So we will have 3 
schedules (2016, 2017, 2018) to get all the windows.

Is there anyone way to handle this or am I missing something ?. Any pointers 
would be really helpful.

{{}}

{{}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (AIRFLOW-5863) Airflow dag with past end_date doesn't trigger schedule

[Alex Anto Navis Lawrence (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Anto Navis Lawrence updated AIRFLOW-5863:
--
Description: 
Hi all,

I have a airflow dag with the following configuration:
{code:java}
{
 'start_date': datetime(2017, 1, 1, 0, 0, 0)
 'end_date': datetime(2019, 6, 1, 0, 0, 0),
  schedule_interval='@yearly',
  catchup=True
}

{code}
For the above, 2 schedules are created and run.
{code:java}
=> 2017-01-01, 2018-01-01.{code}
It doesn't create for 2019-01-01, since I understand the scheduler schedules 
only after the schedule interval is completed. In this case 2019 is still not 
over, so it is not scheduled.

Since I have given the end_date which is in the past, shouldn't the scheduler 
understand the end_date is over and trigger that.

I know there is a solution where by adjusting the start_date to one year back 
and adjusting the date ranges(execution_date + 1 year) within the job can solve 
this issue. (e.g. schedule it from start_date: 2016-01-01). So we will have 3 
schedules (2016, 2017, 2018) to get all the windows.

Is there anyone way to handle this or am I missing something ?. Any pointers 
would be really helpful.

 

  was:
Hi all,

I have a airflow dag with the following configuration:

{{}}
{code:java}
{
 'start_date': datetime(2017, 1, 1, 0, 0, 0)
 'end_date': datetime(2019, 6, 1, 0, 0, 0),
  schedule_interval='@yearly',
  catchup=True
}

{code}
{{}}

For the above, 2 schedules are created and run.

{{}}
{code:java}
=> 2017-01-01, 2018-01-01.{code}
It doesn't create for 2019-01-01, since I understand the scheduler schedules 
only after the schedule interval is completed. In this case 2019 is still not 
over, so it is not scheduled.

Since I have given the end_date which is in the past, shouldn't the scheduler 
understand the end_date is over and trigger that.

I know there is a solution where by adjusting the start_date to one year back 
and adjusting the date ranges(execution_date + 1 year) within the job can solve 
this issue. (e.g. schedule it from start_date: 2016-01-01). So we will have 3 
schedules (2016, 2017, 2018) to get all the windows.

Is there anyone way to handle this or am I missing something ?. Any pointers 
would be really helpful.

{{}}

{{}}


> Airflow dag with past end_date doesn't trigger schedule
> ---
>
> Key: AIRFLOW-5863
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5863
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 1.10.4
>Reporter: Alex Anto Navis Lawrence
>Priority: Minor
>  Labels: beginner
>
> Hi all,
> I have a airflow dag with the following configuration:
> {code:java}
> {
>  'start_date': datetime(2017, 1, 1, 0, 0, 0)
>  'end_date': datetime(2019, 6, 1, 0, 0, 0),
>   schedule_interval='@yearly',
>   catchup=True
> }
> {code}
> For the above, 2 schedules are created and run.
> {code:java}
> => 2017-01-01, 2018-01-01.{code}
> It doesn't create for 2019-01-01, since I understand the scheduler schedules 
> only after the schedule interval is completed. In this case 2019 is still not 
> over, so it is not scheduled.
> Since I have given the end_date which is in the past, shouldn't the scheduler 
> understand the end_date is over and trigger that.
> I know there is a solution where by adjusting the start_date to one year back 
> and adjusting the date ranges(execution_date + 1 year) within the job can 
> solve this issue. (e.g. schedule it from start_date: 2016-01-01). So we will 
> have 3 schedules (2016, 2017, 2018) to get all the windows.
> Is there anyone way to handle this or am I missing something ?. Any pointers 
> would be really helpful.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] douglasmakey commented on issue #6514: AIRFLOW-5590: Add dagrun_id to the response of experimental API on trigger.

[GitBox]

douglasmakey commented on issue #6514: AIRFLOW-5590: Add dagrun_id to the 
response of experimental API on trigger.
URL: https://github.com/apache/airflow/pull/6514#issuecomment-550747018
 
 
   I am not sure if I need to change the`json_client` to return the message and 
dagrun_id. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #6513: [AIRFLOW-5862] Fix failing build on kubernetes

[GitBox]

potiuk commented on issue #6513: [AIRFLOW-5862] Fix failing build on kubernetes
URL: https://github.com/apache/airflow/pull/6513#issuecomment-550727287
 
 
   I made the tests a bit more "smart"  - they will also run kubernetes/unit 
tests also whenever any of the scripts/ci files change. That should prevent 
this happening in the future.
   
   Please take a look asap - for now we have master failing.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk edited a comment on issue #6513: [AIRFLOW-5862] Fix failing build on kubernetes

[GitBox]

potiuk edited a comment on issue #6513: [AIRFLOW-5862] Fix failing build on 
kubernetes
URL: https://github.com/apache/airflow/pull/6513#issuecomment-550508032
 
 
   It worked in the PR but did not work after merging to master because of the 
optimisations of Travis build scripts - > common dir was missing in the new 
Docker and it was needed in the new Kubernetes scripts but the Kubernetes 
scripts have not changed so we have a weakness in the optimisation scripts.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #6501: Prod image

[GitBox]

potiuk commented on issue #6501: Prod image
URL: https://github.com/apache/airflow/pull/6501#issuecomment-550661751
 
 
   The current "Prod image support" is 
https://github.com/apache/airflow/pull/6501/commits/700582e8b189652125181a147a2d1372e43e3714
 (but it will change after rebases).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #6501: Prod image

[GitBox]

potiuk commented on issue #6501: Prod image
URL: https://github.com/apache/airflow/pull/6501#issuecomment-550658642
 
 
   It is based on some previous (not yet merged PRs) so  the prod image is just 
the last commit I have not updated the descrription yet as it is a Draft PR 
still. Just take a look at the last commit.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ryw commented on issue #6501: Prod image

[GitBox]

ryw commented on issue #6501: Prod image
URL: https://github.com/apache/airflow/pull/6501#issuecomment-550565552
 
 
   72 changed files hard to see the work on prod docker - is there some issue w 
the PR?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] villasv commented on issue #5883: [AIRFLOW-5194] Add error handler to action log

[GitBox]

villasv commented on issue #5883: [AIRFLOW-5194] Add error handler to action log
URL: https://github.com/apache/airflow/pull/5883#issuecomment-550559808
 
 
   Heyyy just wanted do bump this PR :-)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-5167) Dependancy conflict with grpc-google-iam-v1

[Eric G (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968795#comment-16968795
 ] 

Eric G commented on AIRFLOW-5167:
-

+1 I have this issue as well

> Dependancy conflict with grpc-google-iam-v1
> ---
>
> Key: AIRFLOW-5167
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5167
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: gcp
>Affects Versions: 1.10.4
>Reporter: Sujay Mansingh
>Priority: Trivial
>
> If we use a tool that does script requirements checking (e.g. 
> {{pip-compile}}) then an erroe is thrown when handling the dependencies for 
> {{apache-airflow[gcp_api]}}
> {code:java}
> $ pip-compile --version
> pip-compile, version 3.7.0
> $ cat requirements.in
> apache-airflow[gcp_api]
> $ pip-compile -o requirements.txt requirements.in
> Could not find a version that matches 
> grpc-google-iam-v1<0.12dev,<0.13dev,>=0.11.4,>=0.12.3
> Tried: 0.9.0, 0.10.0, 0.10.1, 0.11.1, 0.11.3, 0.11.4, 0.12.0, 0.12.1, 0.12.2, 
> 0.12.3
> There are incompatible versions in the resolved dependencies.
> {code}
> Looks like a set of inconsistent dependencies in the google libs :/
> {code:java}
> $ pip-compile -v -o requirements.txt requirements.in | grep -i grpc-google-iam
>   google-cloud-container==0.3.0 requires 
> google-api-core[grpc]<2.0.0dev,>=1.14.0, grpc-google-iam-v1<0.13dev,>=0.12.3
>   google-cloud-bigtable==0.33.0 requires 
> google-api-core[grpc]<2.0.0dev,>=1.6.0, google-cloud-core<2.0dev,>=1.0.0, 
> grpc-google-iam-v1<0.12dev,>=0.11.4
>   google-cloud-spanner==1.9.0 requires 
> google-api-core[grpc,grpcgcp]<2.0.0dev,>=1.4.1, 
> google-cloud-core<2.0dev,>=1.0.0, grpc-google-iam-v1<0.12dev,>=0.11.4
>   adding ['grpc-google-iam-v1', '<0.12dev,<0.13dev,>=0.11.4,>=0.12.3', '[]']
>   grpc-google-iam-v1<0.12dev,<0.13dev,>=0.11.4,>=0.12.3
> Could not find a version that matches 
> grpc-google-iam-v1<0.12dev,<0.13dev,>=0.11.4,>=0.12.3
> Tried: 0.9.0, 0.10.0, 0.10.1, 0.11.1, 0.11.3, 0.11.4, 0.12.0, 0.12.1, 0.12.2, 
> 0.12.3
> There are incompatible versions in the resolved dependencies.
> {code}
> So it looks like {{google-cloud-bigtable==0.33.0}} and 
> {{google-cloud-spanner==1.9.0}} require a version that is {{>=0.11.4}} and 
> {{<0.12dev}}
> {{google-cloud-container==0.3.0}} upsets everything though by specifying 
> version {{>=0.12.3}} and {{<0.13dev}}
> For now, we can work around issue by not checking any conflicts when 
> installing {{apache_airflow[gcp]}}.
> That isn't ideal
>  - All it means is that the version of {{grpc-google-iam-v1}} that is 
> specified last will be installed
>  - I'm assuming that there are no major api changes in {{grpc-google-iam}} 
> between {{<0.12dev}} and {{<0.13dev}}. If there are then I imagine tests 
> would fail, but it still feels a little wrong to allow use of a lib that 
> doesn't match the version required
>  - Relaxing the check just means that we could have a conflict elsewhere that 
> isn't reported
> I understand this is not an issue with airflow, but more to do with the 
> google python libraries.
>  However it does affect us when we try to use airflow and be strict with 
> python requirements.
> Any ideas on what to do?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE]

[GitBox]

potiuk commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO 
NOT MERGE]
URL: https://github.com/apache/airflow/pull/5659#issuecomment-550528138
 
 
   Great!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #6496: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

[GitBox]

potiuk commented on issue #6496: [AIRFLOW-5704] Improve Kind Kubernetes scripts 
for local testing
URL: https://github.com/apache/airflow/pull/6496#issuecomment-550524575
 
 
   @feluelle  - wait for #6500 :)... That's even better.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (AIRFLOW-5860) Add the field `dagrun_id` to the response of experimental API on trigger.

[Douglas Mendez (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-5860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Douglas Mendez updated AIRFLOW-5860:

External issue URL: https://github.com/apache/airflow/pull/6514

> Add the field `dagrun_id` to the response of experimental API on trigger.
> -
>
> Key: AIRFLOW-5860
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5860
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: api, cli
>Affects Versions: 1.10.6
>Reporter: Douglas Mendez
>Priority: Major
>
> We are using experimental REST API for automating Airflow, triggering Dags 
> from one of our microservices.
> If the API could return the `dagrun_id` it would be great for us, we could 
> save it in our DB with data related, with this we could find the log for 
> specific DAgRun quickly.
>  
> *Response example*
>  
> {code:java}
> // code placeholder
> {
>'execution_date': execution_date, 
>'message': message, 
>'dagrun_id': 'dagrun_id
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk commented on issue #6513: [AIRFLOW-5862] Fix failing build on kubernetes

[GitBox]

potiuk commented on issue #6513: [AIRFLOW-5862] Fix failing build on kubernetes
URL: https://github.com/apache/airflow/pull/6513#issuecomment-550508032
 
 
   It worked in the PR but did not work after merging to master because of an 
unexpected dependency between images and scripts. The PR build was using the 
previous image with old app_deploy scripts and the new image used the new image 
with the new scripts -  with missing common directory in .dockerignore :). 
Weird bug but this is what you get to when you get docker-in-docker-in-docker.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-5590) Add 'run_id' to trigger DAG run API response as a field

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968724#comment-16968724
 ] 

ASF GitHub Bot commented on AIRFLOW-5590:
-

douglasmakey commented on pull request #6514: AIRFLOW-5590: Add dagrun_id to 
the response of experimental API on trigger.
URL: https://github.com/apache/airflow/pull/6514
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [ ] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
 - In case you are proposing a fundamental code change, you need to create 
an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
 - In case you are adding a dependency, check if the license complies with 
the [ASF 3rd Party License 
Policy](https://www.apache.org/legal/resolved.html#category-x).
   
   ### Description
   
   - [ ] Here are some details about my PR, including screenshots of any UI 
changes:
   
   ### Tests
   
   - [ ] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [ ] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [ ] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add 'run_id' to trigger DAG run API response as a field
> ---
>
> Key: AIRFLOW-5590
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5590
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: api
>Affects Versions: 1.10.5
>Reporter: Anton Kumpan
>Assignee: Anton Kumpan
>Priority: Major
>
> Idea is the same as in AIRFLOW-4482
> Currently API response looks like this:
> {code:json}
> {
> 'execution_date': '2019-05-08T07:03:09+00:00', 
> 'message': 'Created  manual__2019-05-08T07:03:09+00:00, externally triggered: True>'
> }
> {code}
>  
> It would be nice to add run_id as a separate field, so that response will 
> look like:
> {code:json}
> {
> 'run_id': 'manual__2019-05-08T07:03:09+00:00',
> 'execution_date': '2019-05-08T07:03:09+00:00', 
> 'message': 'Created  triggered: True>'
> }
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] douglasmakey opened a new pull request #6514: AIRFLOW-5590: Add dagrun_id to the response of experimental API on trigger.

[GitBox]

douglasmakey opened a new pull request #6514: AIRFLOW-5590: Add dagrun_id to 
the response of experimental API on trigger.
URL: https://github.com/apache/airflow/pull/6514
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [ ] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
 - In case you are proposing a fundamental code change, you need to create 
an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
 - In case you are adding a dependency, check if the license complies with 
the [ASF 3rd Party License 
Policy](https://www.apache.org/legal/resolved.html#category-x).
   
   ### Description
   
   - [ ] Here are some details about my PR, including screenshots of any UI 
changes:
   
   ### Tests
   
   - [ ] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [ ] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [ ] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-5862) Build fails on new kubernetes approach

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968717#comment-16968717
 ] 

ASF GitHub Bot commented on AIRFLOW-5862:
-

potiuk commented on pull request #6513: [AIRFLOW-5862] Fix failing build on 
kubernetes
URL: https://github.com/apache/airflow/pull/6513
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-5862
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI 
changes:
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Build fails on new kubernetes approach
> --
>
> Key: AIRFLOW-5862
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5862
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk opened a new pull request #6513: [AIRFLOW-5862] Fix failing build on kubernetes

[GitBox]

potiuk opened a new pull request #6513: [AIRFLOW-5862] Fix failing build on 
kubernetes
URL: https://github.com/apache/airflow/pull/6513
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-5862
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI 
changes:
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (AIRFLOW-5862) Build fails on new kubernetes approach

[Jarek Potiuk (Jira)]

Jarek Potiuk created AIRFLOW-5862:
-

 Summary: Build fails on new kubernetes approach
 Key: AIRFLOW-5862
 URL: https://issues.apache.org/jira/browse/AIRFLOW-5862
 Project: Apache Airflow
  Issue Type: Bug
  Components: ci
Affects Versions: 2.0.0
Reporter: Jarek Potiuk






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (AIRFLOW-5861) DatabricksRunNowOperator enforcing argument that should be able to be empty

[John Dyer (Jira)]

John Dyer created AIRFLOW-5861:
--

 Summary: DatabricksRunNowOperator enforcing argument that should 
be able to be empty
 Key: AIRFLOW-5861
 URL: https://issues.apache.org/jira/browse/AIRFLOW-5861
 Project: Apache Airflow
  Issue Type: Bug
  Components: operators
Affects Versions: 1.10.6
Reporter: John Dyer


It looks like /airflow/utils/decorators.py: apply_defaults makes sure that 
arguments without a default value are enforced.  In the 
DatabricksRunNowOperator, the documentation specifies:
{noformat}
There are two ways to instantiate this operator.

In the first way, you can take the JSON payload that you typically use
to call the ``api/2.0/jobs/run-now`` endpoint and pass it directly
to our ``DatabricksRunNowOperator`` through the ``json`` parameter.
For example ::

json = {
  "job_id": 42,
  "notebook_params": {
"dry-run": "true",
"oldest-time-to-consider": "1457570074236"
  }
}

notebook_run = DatabricksRunNowOperator(task_id='notebook_run', 
json=json){noformat}
But, when we run it that way, we get:
{noformat}
Broken DAG: [/home/airflow-user/my-awesome-dag.py] Argument ['job_id'] is 
required{noformat}
So, my thought to fix this is to provide a default for job_id as None, here:
[https://github.com/apache/airflow/blob/v1-10-stable/airflow/contrib/operators/databricks_operator.py#L433]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-5811) Add metric for externally killed task count

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968699#comment-16968699
 ] 

ASF GitHub Bot commented on AIRFLOW-5811:
-

feng-tao commented on pull request #6466: [AIRFLOW-5811] add metric for 
externally killed task count
URL: https://github.com/apache/airflow/pull/6466
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add metric for externally killed task count
> ---
>
> Key: AIRFLOW-5811
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5811
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: scheduler
>Affects Versions: 2.0.0
>Reporter: QP Hou
>Assignee: QP Hou
>Priority: Trivial
>
> Number of externally killed task would be very useful for monitoring init 
> container error when using k8s executor.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-5811) Add metric for externally killed task count

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968700#comment-16968700
 ] 

ASF subversion and git services commented on AIRFLOW-5811:
--

Commit 6bcbd487925c2cb3bd10f1a379a0d8d23133c420 in airflow's branch 
refs/heads/master from Qingping Hou
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=6bcbd48 ]

[AIRFLOW-5811] add metric for externally killed task count (#6466)



> Add metric for externally killed task count
> ---
>
> Key: AIRFLOW-5811
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5811
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: scheduler
>Affects Versions: 2.0.0
>Reporter: QP Hou
>Assignee: QP Hou
>Priority: Trivial
>
> Number of externally killed task would be very useful for monitoring init 
> container error when using k8s executor.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (AIRFLOW-5811) Add metric for externally killed task count

[Tao Feng (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tao Feng resolved AIRFLOW-5811.
---
Fix Version/s: 1.10.7
   Resolution: Fixed

> Add metric for externally killed task count
> ---
>
> Key: AIRFLOW-5811
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5811
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: scheduler
>Affects Versions: 2.0.0
>Reporter: QP Hou
>Assignee: QP Hou
>Priority: Trivial
> Fix For: 1.10.7
>
>
> Number of externally killed task would be very useful for monitoring init 
> container error when using k8s executor.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] feng-tao merged pull request #6466: [AIRFLOW-5811] add metric for externally killed task count

[GitBox]

feng-tao merged pull request #6466: [AIRFLOW-5811] add metric for externally 
killed task count
URL: https://github.com/apache/airflow/pull/6466
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-5848) [AIRFLOW-5668] Rename PostgresToGoogleCloudStorageOperator to PostgresToGcsOperator

[Sayed Mohammad Hossein Torabi (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968679#comment-16968679
 ] 

Sayed Mohammad Hossein Torabi commented on AIRFLOW-5848:


This issue is already done!

> [AIRFLOW-5668] Rename PostgresToGoogleCloudStorageOperator to 
> PostgresToGcsOperator
> ---
>
> Key: AIRFLOW-5848
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5848
> Project: Apache Airflow
>  Issue Type: Sub-task
>  Components: gcp
>Affects Versions: 1.10.6
>Reporter: Michał Słowikowski
>Priority: Minor
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (AIRFLOW-5860) Add the field `dagrun_id` to the response of experimental API on trigger.

[Douglas Mendez (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-5860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Douglas Mendez updated AIRFLOW-5860:

Description: 
We are using experimental REST API for automating Airflow, triggering Dags from 
one of our microservices.

If the API could return the `dagrun_id` it would be great for us, we could save 
it in our DB with data related, with this we could find the log for specific 
DAgRun quickly.

 

*Response example*

 
{code:java}
// code placeholder
{
   'execution_date': execution_date, 
   'message': message, 
   'dagrun_id': 'dagrun_id
}
{code}

  was:
We are using experimental REST API for automating Airflow, triggering Dags from 
one of our microservices.

If the API could return the `dagrun_id` it would be great for us, we could save 
it in our DB with data related, with this we could find the log for specific 
DAgRun quickly.

 

*Response example*
{'execution_date': execution_date, 'message': {color:#009100}message,
{color}'dagrun_id': 'dagrun_id
}


> Add the field `dagrun_id` to the response of experimental API on trigger.
> -
>
> Key: AIRFLOW-5860
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5860
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: api, cli
>Affects Versions: 1.10.6
>Reporter: Douglas Mendez
>Priority: Major
>
> We are using experimental REST API for automating Airflow, triggering Dags 
> from one of our microservices.
> If the API could return the `dagrun_id` it would be great for us, we could 
> save it in our DB with data related, with this we could find the log for 
> specific DAgRun quickly.
>  
> *Response example*
>  
> {code:java}
> // code placeholder
> {
>'execution_date': execution_date, 
>'message': message, 
>'dagrun_id': 'dagrun_id
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (AIRFLOW-5860) Add the field `dagrun_id` to the response of experimental API on trigger.

[Douglas Mendez (Jira)]

Douglas Mendez created AIRFLOW-5860:
---

 Summary: Add the field `dagrun_id` to the response of experimental 
API on trigger.
 Key: AIRFLOW-5860
 URL: https://issues.apache.org/jira/browse/AIRFLOW-5860
 Project: Apache Airflow
  Issue Type: Improvement
  Components: api, cli
Affects Versions: 1.10.6
Reporter: Douglas Mendez


We are using experimental REST API for automating Airflow, triggering Dags from 
one of our microservices.

If the API could return the `dagrun_id` it would be great for us, we could save 
it in our DB with data related, with this we could find the log for specific 
DAgRun quickly.

 

*Response example*
{'execution_date': execution_date, 'message': {color:#009100}message,
{color}'dagrun_id': 'dagrun_id
}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-3189) DbAPI get_uri returns invalid uri if schema is None

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968617#comment-16968617
 ] 

ASF GitHub Bot commented on AIRFLOW-3189:
-

eladkal commented on pull request #5633: [AIRFLOW-3189] fix DbAPI get_uri 
returns invalid uri if schema is None
URL: https://github.com/apache/airflow/pull/5633
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> DbAPI get_uri returns invalid uri if schema is None
> ---
>
> Key: AIRFLOW-3189
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3189
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: hooks
>Affects Versions: 1.10.0
>Reporter: Thomas Haederle
>Priority: Minor
>
> the current implementation of get_uri attaches the schema name to the URI 
> even if no schema was specified.
> This leads to errors in downstream functions such as returning an invalid 
> sqlalchemy engine.
> we should add a simple check, such when the schema is None, it should not be 
> appended to the URI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] eladkal commented on issue #5633: [AIRFLOW-3189] fix DbAPI get_uri returns invalid uri if schema is None

[GitBox]

eladkal commented on issue #5633: [AIRFLOW-3189] fix DbAPI get_uri returns 
invalid uri if schema is None
URL: https://github.com/apache/airflow/pull/5633#issuecomment-550455735
 
 
   @potiuk don't have time to work on that PR anymore. be my guest to continue 
if you'd like


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] eladkal closed pull request #5633: [AIRFLOW-3189] fix DbAPI get_uri returns invalid uri if schema is None

[GitBox]

eladkal closed pull request #5633: [AIRFLOW-3189] fix DbAPI get_uri returns 
invalid uri if schema is None
URL: https://github.com/apache/airflow/pull/5633
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-4879) Add poll_interval and schema to PrestoHook

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968606#comment-16968606
 ] 

ASF GitHub Bot commented on AIRFLOW-4879:
-

eladkal commented on pull request #5515:  [AIRFLOW-4879] add poll_interval and 
schema to PrestoHook
URL: https://github.com/apache/airflow/pull/5515
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add poll_interval and schema to PrestoHook
> --
>
> Key: AIRFLOW-4879
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4879
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: hooks
>Affects Versions: 1.10.3
>Reporter: Elad
>Assignee: Elad
>Priority: Minor
>
> PR will follow



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] eladkal closed pull request #5515: [AIRFLOW-4879] add poll_interval and schema to PrestoHook

[GitBox]

eladkal closed pull request #5515:  [AIRFLOW-4879] add poll_interval and schema 
to PrestoHook
URL: https://github.com/apache/airflow/pull/5515
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] eladkal commented on issue #5515: [AIRFLOW-4879] add poll_interval and schema to PrestoHook

[GitBox]

eladkal commented on issue #5515:  [AIRFLOW-4879] add poll_interval and schema 
to PrestoHook
URL: https://github.com/apache/airflow/pull/5515#issuecomment-550449197
 
 
   Issues with rebasing.
   Will open new PR


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] efedotova commented on a change in pull request #6450: [AIRFLOW-XXX] Local development environments - polish documentation

[GitBox]

efedotova commented on a change in pull request #6450: [AIRFLOW-XXX] Local 
development environments - polish documentation
URL: https://github.com/apache/airflow/pull/6450#discussion_r343262059
 
 

 ##
 File path: LOCAL_VIRTUALENV.rst
 ##
 @@ -18,115 +18,156 @@
 
 .. contents:: :local:
 
+About Local Virtual Environment (virtualenv)
+
 
-Local virtualenv environment
-
+Use the local virtualenv development option in the combination with the 
_`Breeze `_
+development environment. This option helps you benefit from the infrastructure 
provided
+by your IDE and work in the enviroment where all necessary dependencies and 
tests are 
+available and set up within Docker images.
 
-Installation
-
+But you can also use the local virtualenv as a standalone development option of
+you develop Airflow functionality that does not incur large external 
dependencies and 
+CI test coverage.
 
-Install Python (3.5 or 3.6), MySQL, and libxml by using system-level
-package managers like yum, apt-get for Linux, or Homebrew for Mac OS at
-first. Refer to the `Dockerfile `__ for a comprehensive list
+These are examples of the development options available with the local 
virtualenv in your IDE:
+
+* local debugging;
+* Airflow source view;
+* autocompletion;
+* documentation support;
+* unit tests.
+
+This document describes minimum requirements and insructions for using a 
standalone version of the local virtualenv.
+
+Prerequisites
+=
+
+Required Software Packages
+--
+
+Use system-level package managers like yum, apt-get for Linux, or 
+Homebrew for macOS to install required software packages:
+
+* Python (3.5 or 3.6)
+* MySQL
+* libxml
+
+Refer to the `Dockerfile `__ for a comprehensive list
 of required packages.
 
-In order to use your IDE you need you can use the virtual environment.
-Ideally you should setup virtualenv for all python versions that Airflow
-supports (3.5, 3.6). An easy way to create the virtualenv is to use
-`virtualenvwrapper `__
-- it allows you to easily switch between virtualenvs using ``workon``
-command and mange your virtual environments more easily. Typically
-creating the environment can be done by:
+Extra Packages
+--
 
-.. code:: bash
+You can also install extra packages (like ``[gcp]``, etc) via
+``pip install -e [EXTRA1,EXTRA2 ...]``. However, some of them may  
+have additional install and setup requirements for your local system.
 
-mkvirtualenv  --python=python
+For example, if you have a trouble installing the mysql client on macOS and get
+an error as follows:
 
-Then you need to install python PIP requirements. Typically it can be
-done with: ``pip install -e ".[devel]"``.
+.. code:: text
 
-After creating the virtualenv, run this command to create the Airflow
-sqlite database:
+ld: library not found for -lssl
+
+you should set LIBRARY\_PATH before running ``pip install``:
 
 .. code:: bash
 
-airflow db init
+export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/opt/openssl/lib/
 
+You are STRONGLY encouraged to also install and use `pre-commit hooks 
`_ 
+for your local virtualenv development environment. Pre-commit hooks can speed 
up your 
+development cycle a lot.
 
-Creating virtualenv can be automated with `Breeze environment 
`_
+The full list of extras is available in ``_.
 
-Once initialization is done, you should select the virtualenv you
-initialized as the project's default virtualenv in your IDE.
+Creating a Local virtualenv
+===
 
-After setting it up - you can use the usual "Run Test" option of the IDE
-and have the autocomplete and documentation support from IDE as well as
-you can debug and view the sources of Airflow - which is very helpful
-during development.
+To use your IDE for Airflow development and testing, you need to configure a 
virtual 
+environment. Ideally you should set up virtualenv for all Python versions that 
Airflow
+supports (3.5, 3.6). 
 
-Installing other extras

+Consider using the
+`virtualenvwrapper `__
+that allows you to create virtual environments and easily switch between them 
with the 
+``workon`` command. 
 
-You can also other extras (like ``[mysql]``, ``[gcp]`` etc. via
-``pip install -e [EXTRA1,EXTRA2 ...]``. However some of the extras have 
additional
-system requirements and you might need to install additional packages on your
-local machine.
+1. Create an environment as follows:
 
-For example if you have trouble installing mysql client on MacOS and you have
-an error similar to
+   ``mkvirtualenv  --python=python``
 
-.. code:: text
+2. Install Python PIP requirements:
 
-ld: library not found for -lssl
+   ``pip install -e ".[devel]"``
 
-you should set LIBRARY\_PATH before running ``pip install``:
+3. Create the 

[jira] [Commented] (AIRFLOW-5824) Add a new hook and operator for AWS DataSync

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968560#comment-16968560
 ] 

ASF GitHub Bot commented on AIRFLOW-5824:
-

baolsen commented on pull request #6512: [AIRFLOW-5824] Added AWS DataSync Hook 
and Operator
URL: https://github.com/apache/airflow/pull/6512
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 
   ### Description
   - [x] Here are some details about my PR:
   
   Added AWS DataSync Hook and Operator, along with appropriate tests and 
documentation.
   
   ### Tests
   - [x] My PR adds the following unit tests:
   
   Tests added for both new components (DataSync hook + operator).
   
   The tests require the moto Python library, specifically moto_datasync. 
   I only recently added moto_datasync to moto, so the Airflow tests for AWS 
DataSync will be skipped if moto_datasync is not present. 
   However all tests pass on my local copy of moto which includes moto_datasync
   
   
   ### Commits
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add a new hook and operator for AWS DataSync
> 
>
> Key: AIRFLOW-5824
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5824
> Project: Apache Airflow
>  Issue Type: New Feature
>  Components: aws
>Affects Versions: 1.10.5
>Reporter: Bjorn Olsen
>Assignee: Bjorn Olsen
>Priority: Minor
>
> I'm working on some code to add a hook and operator for AWS DataSync task 
> executions.
> Just creating a Jira ticket for visibility



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] baolsen opened a new pull request #6512: [AIRFLOW-5824] Added AWS DataSync Hook and Operator

[GitBox]

baolsen opened a new pull request #6512: [AIRFLOW-5824] Added AWS DataSync Hook 
and Operator
URL: https://github.com/apache/airflow/pull/6512
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   - [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 
   ### Description
   - [x] Here are some details about my PR:
   
   Added AWS DataSync Hook and Operator, along with appropriate tests and 
documentation.
   
   ### Tests
   - [x] My PR adds the following unit tests:
   
   Tests added for both new components (DataSync hook + operator).
   
   The tests require the moto Python library, specifically moto_datasync. 
   I only recently added moto_datasync to moto, so the Airflow tests for AWS 
DataSync will be skipped if moto_datasync is not present. 
   However all tests pass on my local copy of moto which includes moto_datasync
   
   
   ### Commits
   - [x] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   - [x] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (AIRFLOW-5859) Tasks locking and heartbeat warnings

[Jacob Ward (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Ward updated AIRFLOW-5859:

Description: 
Having two potentially related issues.

Issue 1:

Some of my tasks (has only been PythonOperators so far) are starting and then 
doing nothing. I had a task that usually executes in 10-30 minutes running for 
over 24hrs without any error messages in the logs (other than the heartbeat 
warnings shown below) and without failing.

So far this has only happened to tasks inside sub-dags, not sure if that's to 
do with it?

The logs for the sub-dag show: 


{code:}
[2019-11-05 16:41:34,364] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:34,364] {backfill_job.py:363} INFO - [backfill progress] | finished run 0 
of 1 | tasks waiting: 7 | succeeded: 5 | running: 1 | failed: 0 | skipped: 1 | 
deadlocked: 0 | not ready: 7
[2019-11-05 16:41:34,831] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:34,830] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 
s) < heartrate(5.0 s), sleeping for 4.986693 s
[2019-11-05 16:41:39,376] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:39,376] {backfill_job.py:363} INFO - [backfill progress] | finished run 0 
of 1 | tasks waiting: 7 | succeeded: 5 | running: 1 | failed: 0 | skipped: 1 | 
deadlocked: 0 | not ready: 7
[2019-11-05 16:41:39,859] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:39,859] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 
s) < heartrate(5.0 s), sleeping for 4.986141 s
{code}
repeated during that 24hr period.


Issue 2:

In all of the logs this warning message is printed every 5 seconds:


{code:}
[2019-11-06 14:25:29,466] {logging_mixin.py:112} INFO - [2019-11-06 
14:25:29,465] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 
s) < heartrate(5.0 s), sleeping for 4.987354 s {code}


  was:
Having two potentially related issues.

Issue 1:

Some of my tasks (has only been PythonOperators so far) are starting and then 
doing nothing. I had a task that usually executes in 10-30 minutes running for 
over 24hrs without any error messages in the logs (other than the heartbeat 
warnings shown below) and without failing.

So far this has only happened to tasks inside sub-dags, not sure if that's to 
do with it?

The logs for the sub-dag show: 


{code:java}
[2019-11-05 16:41:34,364] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:34,364] {backfill_job.py:363} INFO - [backfill progress] | finished run 0 
of 1 | tasks waiting: 7 | succeeded: 5 | running: 1 | failed: 0 | skipped: 1 | 
deadlocked: 0 | not ready: 7
[2019-11-05 16:41:34,831] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:34,830] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 
s) < heartrate(5.0 s), sleeping for 4.986693 s
[2019-11-05 16:41:39,376] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:39,376] {backfill_job.py:363} INFO - [backfill progress] | finished run 0 
of 1 | tasks waiting: 7 | succeeded: 5 | running: 1 | failed: 0 | skipped: 1 | 
deadlocked: 0 | not ready: 7
[2019-11-05 16:41:39,859] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:39,859] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 
s) < heartrate(5.0 s), sleeping for 4.986141 s
{code}
repeated during that 24hr period.


Issue 2:

In all of the logs this warning message is printed every 5 seconds:

{{}}
{code:java}
[2019-11-06 14:25:29,466] {logging_mixin.py:112} INFO - [2019-11-06 
14:25:29,465] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 
s) < heartrate(5.0 s), sleeping for 4.987354 s {code}
{{}}


> Tasks locking and heartbeat warnings
> 
>
> Key: AIRFLOW-5859
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5859
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: DagRun
>Affects Versions: 1.10.6
> Environment: Airflow using LocalExecutor and Postgres
>Reporter: Jacob Ward
>Priority: Major
>
> Having two potentially related issues.
> Issue 1:
> Some of my tasks (has only been PythonOperators so far) are starting and then 
> doing nothing. I had a task that usually executes in 10-30 minutes running 
> for over 24hrs without any error messages in the logs (other than the 
> heartbeat warnings shown below) and without failing.
> So far this has only happened to tasks inside sub-dags, not sure if that's to 
> do with it?
> The logs for the sub-dag show: 
> {code:}
> [2019-11-05 16:41:34,364] {logging_mixin.py:112} INFO - [2019-11-05 
> 16:41:34,364] {backfill_job.py:363} INFO - [backfill progress] | finished run 
> 0 of 1 | tasks waiting: 7 | succeeded: 5 | running: 1 | failed: 0 | skipped: 
> 1 | deadlocked: 0 | not ready: 7
> [2019-11-05 16:41:34,831] {logging_mixin.py:112} INFO - [2019-11-05 
> 16:41:34,830] {local_task_job.py:124} WARNING - Time since last 
> 

[jira] [Created] (AIRFLOW-5859) Tasks locking and heartbeat warnings

[Jacob Ward (Jira)]

Jacob Ward created AIRFLOW-5859:
---

 Summary: Tasks locking and heartbeat warnings
 Key: AIRFLOW-5859
 URL: https://issues.apache.org/jira/browse/AIRFLOW-5859
 Project: Apache Airflow
  Issue Type: Bug
  Components: DagRun
Affects Versions: 1.10.6
 Environment: Airflow using LocalExecutor and Postgres
Reporter: Jacob Ward


Having two potentially related issues.

Issue 1:

Some of my tasks (has only been PythonOperators so far) are starting and then 
doing nothing. I had a task that usually executes in 10-30 minutes running for 
over 24hrs without any error messages in the logs (other than the heartbeat 
warnings shown below) and without failing.

So far this has only happened to tasks inside sub-dags, not sure if that's to 
do with it?

The logs for the sub-dag show: 


{code:java}
[2019-11-05 16:41:34,364] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:34,364] {backfill_job.py:363} INFO - [backfill progress] | finished run 0 
of 1 | tasks waiting: 7 | succeeded: 5 | running: 1 | failed: 0 | skipped: 1 | 
deadlocked: 0 | not ready: 7
[2019-11-05 16:41:34,831] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:34,830] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 
s) < heartrate(5.0 s), sleeping for 4.986693 s
[2019-11-05 16:41:39,376] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:39,376] {backfill_job.py:363} INFO - [backfill progress] | finished run 0 
of 1 | tasks waiting: 7 | succeeded: 5 | running: 1 | failed: 0 | skipped: 1 | 
deadlocked: 0 | not ready: 7
[2019-11-05 16:41:39,859] {logging_mixin.py:112} INFO - [2019-11-05 
16:41:39,859] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 
s) < heartrate(5.0 s), sleeping for 4.986141 s
{code}
repeated during that 24hr period.


Issue 2:

In all of the logs this warning message is printed every 5 seconds:

{{}}
{code:java}
[2019-11-06 14:25:29,466] {logging_mixin.py:112} INFO - [2019-11-06 
14:25:29,465] {local_task_job.py:124} WARNING - Time since last heartbeat(0.01 
s) < heartrate(5.0 s), sleeping for 4.987354 s {code}
{{}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] TobKed commented on a change in pull request #6299: [AIRFLOW-5631] Change way of running GCP system tests

[GitBox]

TobKed commented on a change in pull request #6299: [AIRFLOW-5631] Change way 
of running GCP system tests
URL: https://github.com/apache/airflow/pull/6299#discussion_r343211153
 
 

 ##
 File path: tests/gcp/operators/test_bigquery_system.py
 ##
 @@ -17,40 +17,32 @@
 # specific language governing permissions and limitations
 # under the License.
 """System tests for Google Cloud Build operators"""
-import unittest
 
 from tests.gcp.operators.test_bigquery_system_helper import 
GCPBigQueryTestHelper
-from tests.gcp.utils.base_gcp_system_test_case import SKIP_TEST_WARNING, 
TestDagGcpSystem
 from tests.gcp.utils.gcp_authenticator import GCP_BIGQUERY_KEY
+from tests.test_utils.gcp_system_helpers import GCP_DAG_FOLDER, 
provide_gcp_context, skip_gcp_system
+from tests.test_utils.system_tests_class import SystemTest
 
 
-@unittest.skipIf(TestDagGcpSystem.skip_check(GCP_BIGQUERY_KEY), 
SKIP_TEST_WARNING)
-class BigQueryExampleDagsSystemTest(TestDagGcpSystem):
+@skip_gcp_system(GCP_BIGQUERY_KEY, require_local_executor=True)
 
 Review comment:
   Sorry for late response. I will try to fix it but I have limited time due to 
my holidays :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (AIRFLOW-5704) Docker scripts for kind kubernetes tests can be improved

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968487#comment-16968487
 ] 

ASF subversion and git services commented on AIRFLOW-5704:
--

Commit 8e789a33a3c95fc91aa9be71d2d7329132dc25f9 in airflow's branch 
refs/heads/master from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=8e789a3 ]

[AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing (#6496)

* [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

* Fixed problem that Kubernetes tests were testing latest master
  rather than what came from the local sources.
* Moved Kubernetes scripts to 'in_container' dir where they belong now
* Kubernetes tests are now better suited for running locally
* Kubernetes cluster is not deleted until environment is stopped
* Kubernetes image is built outside of the container and passed as .tar
* Kubectl version name is corrected in the Dockerfile
* Kubernetes Version can be used to select Kubernetes versio
* Running kubernetes scripts is now easy in Breeze
* Instructions on how to run Kubernetes tests are updated
* Better flags in Breeze are used to run Kubernetes environment/tests
* The old "bare" environment is replaced by --no-deps switch


> Docker scripts for kind kubernetes tests can be improved
> 
>
> Key: AIRFLOW-5704
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5704
> Project: Apache Airflow
>  Issue Type: Sub-task
>  Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Assignee: Jarek Potiuk
>Priority: Major
>
> The docker CI image for kind tests can be improved
>  
>  * Kubernetes Version and all the installation of docker + kubectl + kind can 
> be added back
>  * Running kubernetes scripts should be possible from within breeze without 
> special "kubernetes" environment
>  * --env breeze switch should be removed
>  * "bare" environment should be replaced by --no-deps switch
>  * ENV variable should disappear
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-5704) Docker scripts for kind kubernetes tests can be improved

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968485#comment-16968485
 ] 

ASF GitHub Bot commented on AIRFLOW-5704:
-

potiuk commented on pull request #6496: [AIRFLOW-5704] Improve Kind Kubernetes 
scripts for local testing
URL: https://github.com/apache/airflow/pull/6496
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Docker scripts for kind kubernetes tests can be improved
> 
>
> Key: AIRFLOW-5704
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5704
> Project: Apache Airflow
>  Issue Type: Sub-task
>  Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Assignee: Jarek Potiuk
>Priority: Major
>
> The docker CI image for kind tests can be improved
>  
>  * Kubernetes Version and all the installation of docker + kubectl + kind can 
> be added back
>  * Running kubernetes scripts should be possible from within breeze without 
> special "kubernetes" environment
>  * --env breeze switch should be removed
>  * "bare" environment should be replaced by --no-deps switch
>  * ENV variable should disappear
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-5704) Docker scripts for kind kubernetes tests can be improved

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968486#comment-16968486
 ] 

ASF subversion and git services commented on AIRFLOW-5704:
--

Commit 8e789a33a3c95fc91aa9be71d2d7329132dc25f9 in airflow's branch 
refs/heads/master from Jarek Potiuk
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=8e789a3 ]

[AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing (#6496)

* [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

* Fixed problem that Kubernetes tests were testing latest master
  rather than what came from the local sources.
* Moved Kubernetes scripts to 'in_container' dir where they belong now
* Kubernetes tests are now better suited for running locally
* Kubernetes cluster is not deleted until environment is stopped
* Kubernetes image is built outside of the container and passed as .tar
* Kubectl version name is corrected in the Dockerfile
* Kubernetes Version can be used to select Kubernetes versio
* Running kubernetes scripts is now easy in Breeze
* Instructions on how to run Kubernetes tests are updated
* Better flags in Breeze are used to run Kubernetes environment/tests
* The old "bare" environment is replaced by --no-deps switch


> Docker scripts for kind kubernetes tests can be improved
> 
>
> Key: AIRFLOW-5704
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5704
> Project: Apache Airflow
>  Issue Type: Sub-task
>  Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Assignee: Jarek Potiuk
>Priority: Major
>
> The docker CI image for kind tests can be improved
>  
>  * Kubernetes Version and all the installation of docker + kubectl + kind can 
> be added back
>  * Running kubernetes scripts should be possible from within breeze without 
> special "kubernetes" environment
>  * --env breeze switch should be removed
>  * "bare" environment should be replaced by --no-deps switch
>  * ENV variable should disappear
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk merged pull request #6496: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

[GitBox]

potiuk merged pull request #6496: [AIRFLOW-5704] Improve Kind Kubernetes 
scripts for local testing
URL: https://github.com/apache/airflow/pull/6496
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] TobKed commented on issue #6511: [AIRFLOW-XXX] Improve the PubSub documentation

[GitBox]

TobKed commented on issue #6511: [AIRFLOW-XXX] Improve the PubSub documentation
URL: https://github.com/apache/airflow/pull/6511#issuecomment-550386853
 
 
   this PR refers to review made on 
https://github.com/PolideaInternal/airflow/pull/401
   CC @mschickensoup


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] TobKed edited a comment on issue #6511: [AIRFLOW-XXX] Improve the PubSub documentation

[GitBox]

TobKed edited a comment on issue #6511: [AIRFLOW-XXX] Improve the PubSub 
documentation
URL: https://github.com/apache/airflow/pull/6511#issuecomment-550386853
 
 
   this PR refers to review made on 
https://github.com/PolideaInternal/airflow/pull/401
   CC @mschickensoup @mik-laj 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #6496: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

[GitBox]

potiuk commented on a change in pull request #6496: [AIRFLOW-5704] Improve Kind 
Kubernetes scripts for local testing
URL: https://github.com/apache/airflow/pull/6496#discussion_r343192412
 
 

 ##
 File path: BREEZE.rst
 ##
 @@ -79,11 +79,11 @@ images maintained on the Docker Hub in the 
``apache/airflow`` repository.
 
 There are three images that we are currently managing:
 
-* **CI image*** that is used for testing od both Unit tests and static check 
tests.
-  It contains a lot test-related packages (size of ~1GB).
-  Its tag follows the pattern of ``-python-ci``
+* **CI image*** that is used for testing. It contains a lot more test-related 
installed software
+  (size of ~1GB). Its tag follows the pattern of 
``-python-ci``
   (for example, ``apache/airflow:master-python3.6-ci``). The image is built 
using the
   ``_ Dockerfile.
+* **Production-ready image** that is used TODO
 
 Review comment:
   Ah yeah. That was the intention to add it later. Must have slipped my 
"commit split" mastery (i got quite good at it recently).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] TobKed opened a new pull request #6511: [AIRFLOW-XXX] Improve the PubSub documentation

[GitBox]

TobKed opened a new pull request #6511: [AIRFLOW-XXX] Improve the PubSub 
documentation
URL: https://github.com/apache/airflow/pull/6511
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [ ] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
 - https://issues.apache.org/jira/browse/AIRFLOW-XXX
 - In case you are fixing a typo in the documentation you can prepend your 
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
 - In case you are proposing a fundamental code change, you need to create 
an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
 - In case you are adding a dependency, check if the license complies with 
the [ASF 3rd Party License 
Policy](https://www.apache.org/legal/resolved.html#category-x).
   
   ### Description
   
   - [ ] Here are some details about my PR, including screenshots of any UI 
changes:
   
   ### Tests
   
   - [ ] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   ### Commits
   
   - [ ] My commits all reference Jira issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
 1. Subject is separated from body by a blank line
 1. Subject is limited to 50 characters (not including Jira issue reference)
 1. Subject does not end with a period
 1. Subject uses the imperative mood ("add", not "adding")
 1. Body wraps at 72 characters
 1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [ ] In case of new functionality, my PR adds documentation that describes 
how to use it.
 - All the public functions and the classes in the PR contain docstrings 
that explain what it does
 - If you implement backwards incompatible changes, please leave a note in 
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so 
we can assign it to a appropriate release
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] TobKed commented on issue #6393: [AIRFLOW-5718] Add SFTPToGoogleCloudStorageOperator

[GitBox]

TobKed commented on issue #6393: [AIRFLOW-5718] Add 
SFTPToGoogleCloudStorageOperator
URL: https://github.com/apache/airflow/pull/6393#issuecomment-550382708
 
 
   @mik-laj done! Travis is happy now.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] yegeniy commented on issue #6281: Run batches of (self-terminating) EMR JobFlows [AIRFLOW-5763]

[GitBox]

yegeniy commented on issue #6281: Run batches of (self-terminating) EMR 
JobFlows [AIRFLOW-5763]
URL: https://github.com/apache/airflow/pull/6281#issuecomment-550356275
 
 
   Absolutely. I do think Ash’s two suggestions in the comment in that JIRA
   are the way to go if you want steps submitted only after the cluster is in
   a READY state.
   
(Not related to the suggestion Personally I like submitting my EMR steps
   when I request the cluster to get the Airflow scheduler out of the way as
   soon as possible - slightly reduces chance of orphaning the cluster)
   
   On Tue, Nov 5, 2019 at 8:18 AM OmerJog  wrote:
   
   > @yegeniy  maybe some of the code here can be
   > modified to fix https://issues.apache.org/jira/browse/AIRFLOW-5328
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or unsubscribe
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] OmerJog commented on issue #6088: [AIRFLOW-5349] Add schedulername option for KubernetesPodOperator

[GitBox]

OmerJog commented on issue #6088: [AIRFLOW-5349] Add schedulername option for 
KubernetesPodOperator
URL: https://github.com/apache/airflow/pull/6088#issuecomment-550352217
 
 
   @antonymayi  I don't think you meant to change 1043 files for this PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow-site] kgabryje opened a new pull request #135: CHanged placeholder text, added search by continent

[GitBox]

kgabryje opened a new pull request #135: CHanged placeholder text, added search 
by continent
URL: https://github.com/apache/airflow-site/pull/135
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (AIRFLOW-5858) Airflow celery worker missing heartbeat

[Nikhil SInghal (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil SInghal updated AIRFLOW-5858:

Description: 
Our Airflow setup uses Celery Executors and Redis as a broker. We are facing a 
issue of missing heartbeat from celery workers. Once this heartbeat is missed 
the worker stops taking any new task. It also appears as offline in the Celery 
flower UI. Manual restart of workers fixes this problem.

Wanted to know if this is a known issue or being faced by other users in the 
community

These are the logs from failure. 



[2019-11-06 02:30:36,368: INFO/MainProcess] missed heartbeat from 
celery@dp-airflow-worker-6cb4b596f8-nzdrt

worker: Warm shutdown (MainProcess)

-- celery@dp-airflow-worker-6cb4b596f8-4qkfg v4.1.1 (latentcall)
  -
--- * *** * -- Linux-4.9.0-9-amd64-x86_64-with-debian-10.1 2019-11-05 15:24:34
-- * -  ---
- ** -- [config]
- ** -- .> app: airflow.executors.celery_executor:0x7f2a01250cf8
- ** -- .> transport: 
redis://:**@redis-11313.internal.c3160.ap-southeast-1-mz.ec2.cloud.rlrcp.com:11313//
- ** -- .> results: 
postgresql://airflow:**@airflowdbprod.ckvce9fjaook.ap-southeast-1.rds.amazonaws.com:5432/airflowdb
- *** --- * --- .> concurrency: 64 (prefork)
-- ***  .> task events: OFF (enable -E to monitor tasks in this worker)
--- * -
 -- [queues]
 .> default exchange=default(direct) key=default


[tasks]
 . airflow.executors.celery_executor.execute_command

  was:
Our Airflow setup uses Celery Executors and Redis as a broker. We are facing a 
issue of missing heartbeat from celery workers. Once this heartbeat is missed 
the worker stops taking any new task. It also appears as offline in the Celery 
flower UI. Manual restart of workers fixes this problem.

Wanted to know if this is a known issue or being faced by other users in the 
community

These are the logs from failure. 
[2019-11-06 02:30:36,368: INFO/MainProcess] missed heartbeat from 
celery@dp-airflow-worker-6cb4b596f8-nzdrt

worker: Warm shutdown (MainProcess)

 -- celery@dp-airflow-worker-6cb4b596f8-4qkfg v4.1.1 (latentcall)
  -
--- * ***  * -- Linux-4.9.0-9-amd64-x86_64-with-debian-10.1 2019-11-05 15:24:34
-- * -  ---
- ** -- [config]
- ** -- .> app:         airflow.executors.celery_executor:0x7f2a01250cf8
- ** -- .> transport:   
redis://:**@redis-11313.internal.c3160.ap-southeast-1-mz.ec2.cloud.rlrcp.com:11313//
- ** -- .> results: 
postgresql://airflow:**@airflowdbprod.ckvce9fjaook.ap-southeast-1.rds.amazonaws.com:5432/airflowdb
    
- *** --- * --- .> concurrency: 64 (prefork)
-- ***  .> task events: OFF (enable -E to monitor tasks in this worker)
--- * -
 -- [queues]
                .> default          exchange=default(direct) key=default


[tasks]
  . airflow.executors.celery_executor.execute_command


> Airflow celery worker missing heartbeat
> ---
>
> Key: AIRFLOW-5858
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5858
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: celery
>Affects Versions: 1.10.2
>Reporter: Nikhil SInghal
>Priority: Major
>
> Our Airflow setup uses Celery Executors and Redis as a broker. We are facing 
> a issue of missing heartbeat from celery workers. Once this heartbeat is 
> missed the worker stops taking any new task. It also appears as offline in 
> the Celery flower UI. Manual restart of workers fixes this problem.
> Wanted to know if this is a known issue or being faced by other users in the 
> community
> These are the logs from failure. 
> [2019-11-06 02:30:36,368: INFO/MainProcess] missed heartbeat from 
> celery@dp-airflow-worker-6cb4b596f8-nzdrt
> worker: Warm shutdown (MainProcess)
> -- celery@dp-airflow-worker-6cb4b596f8-4qkfg v4.1.1 (latentcall)
>   -
> --- * *** * -- Linux-4.9.0-9-amd64-x86_64-with-debian-10.1 2019-11-05 15:24:34
> -- * -  ---
> - ** -- [config]
> - ** -- .> app: airflow.executors.celery_executor:0x7f2a01250cf8
> - ** -- .> transport: 
> redis://:**@redis-11313.internal.c3160.ap-southeast-1-mz.ec2.cloud.rlrcp.com:11313//
> - ** -- .> results: 
> postgresql://airflow:**@airflowdbprod.ckvce9fjaook.ap-southeast-1.rds.amazonaws.com:5432/airflowdb
> - *** --- * --- .> concurrency: 64 (prefork)
> -- ***  .> task events: OFF (enable -E to monitor tasks in this 
> worker)
> --- * -
>  -- [queues]
>  .> default exchange=default(direct) key=default
> [tasks]
>  . airflow.executors.celery_executor.execute_command



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (AIRFLOW-5858) Airflow celery worker missing heartbeat

[Nikhil SInghal (Jira)]

Nikhil SInghal created AIRFLOW-5858:
---

 Summary: Airflow celery worker missing heartbeat
 Key: AIRFLOW-5858
 URL: https://issues.apache.org/jira/browse/AIRFLOW-5858
 Project: Apache Airflow
  Issue Type: Bug
  Components: celery
Affects Versions: 1.10.2
Reporter: Nikhil SInghal


Our Airflow setup uses Celery Executors and Redis as a broker. We are facing a 
issue of missing heartbeat from celery workers. Once this heartbeat is missed 
the worker stops taking any new task. It also appears as offline in the Celery 
flower UI. Manual restart of workers fixes this problem.

Wanted to know if this is a known issue or being faced by other users in the 
community

These are the logs from failure. 
[2019-11-06 02:30:36,368: INFO/MainProcess] missed heartbeat from 
celery@dp-airflow-worker-6cb4b596f8-nzdrt

worker: Warm shutdown (MainProcess)

 -- celery@dp-airflow-worker-6cb4b596f8-4qkfg v4.1.1 (latentcall)
  -
--- * ***  * -- Linux-4.9.0-9-amd64-x86_64-with-debian-10.1 2019-11-05 15:24:34
-- * -  ---
- ** -- [config]
- ** -- .> app:         airflow.executors.celery_executor:0x7f2a01250cf8
- ** -- .> transport:   
redis://:**@redis-11313.internal.c3160.ap-southeast-1-mz.ec2.cloud.rlrcp.com:11313//
- ** -- .> results: 
postgresql://airflow:**@airflowdbprod.ckvce9fjaook.ap-southeast-1.rds.amazonaws.com:5432/airflowdb
    
- *** --- * --- .> concurrency: 64 (prefork)
-- ***  .> task events: OFF (enable -E to monitor tasks in this worker)
--- * -
 -- [queues]
                .> default          exchange=default(direct) key=default


[tasks]
  . airflow.executors.celery_executor.execute_command



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-330) Decorated PythonOperator python_callable functions don't show the original function in task code view

[jack (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968374#comment-16968374
 ] 

jack commented on AIRFLOW-330:
--

if it's only Python 2 issue than probably this isn't needed anymore 

> Decorated PythonOperator python_callable functions don't show the original 
> function in task code view
> -
>
> Key: AIRFLOW-330
> URL: https://issues.apache.org/jira/browse/AIRFLOW-330
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: webserver
>Reporter: Jon McKenzie
>Priority: Minor
>
> In Python 3.4 or below, if you try to decorate the {{python_callable}} to a 
> {{PythonOperator}} in the following manner (i.e. like the manual application 
> of a standard Python decorator using {{functools.wraps}}):
> {noformat}
> task.python_callable = wrap(task.python_callable)
> {noformat}
> ...the code view of that task in the web UI shows the code for the {{wrap}} 
> function rather than the initial {{python_callable}}. 
> The fix is to run something like this (where {{inspect.unwrap}} is available 
> in Python 3.4+):
> {noformat}
> inspect.getsource(inspect.unwrap(func))
> {noformat}
> ...rather than:
> {noformat}
> inspect.getsource(func)
> {noformat}
> I'm not sure if this is something worth fixing or not, since I believe Python 
> 3.5+ implements the above fix (although I believe it would still be an issue 
> in Python 2.x).
> Just for some background, I'm writing a higher level API around Airflow that 
> takes tasks as arguments and connects their inputs via {{XCom}} (among other 
> things). The callables I want my API users to write aren't going to need 
> access to any of the task context (only so that they don't need to know 
> Airflow internals), hence the need to decorate them appropriately.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (AIRFLOW-5857) Add a get_job_state in the Databricks hook

[Francesco Mucio (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Francesco Mucio updated AIRFLOW-5857:
-
Description: 
In order to create a sensor able to monitor the status of a Spark job using the 
Databricks API, we need to add a method in the [Databricks 
hook|https://github.com/apache/airflow/blob/master/airflow/contrib/hooks/databricks_hook.py]
 to return the state.

Also the RunState class needs to be modifies with a new property is_running.

Finally a sensor will be added to use this new method in the hook.

 

  was:
In order to create a sensor able to monitor the status of a Spark job using the 
Databricks API, we need to add a method in the [Databricks 
hook|https://github.com/apache/airflow/blob/master/airflow/contrib/hooks/databricks_hook.py]
 to return the state.

Also the RunState class needs to be modifies with a new property is_running.


> Add a get_job_state in the Databricks hook 
> ---
>
> Key: AIRFLOW-5857
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5857
> Project: Apache Airflow
>  Issue Type: New Feature
>  Components: contrib, hooks
>Affects Versions: 1.10.6
>Reporter: Francesco Mucio
>Assignee: Francesco Mucio
>Priority: Minor
>
> In order to create a sensor able to monitor the status of a Spark job using 
> the Databricks API, we need to add a method in the [Databricks 
> hook|https://github.com/apache/airflow/blob/master/airflow/contrib/hooks/databricks_hook.py]
>  to return the state.
> Also the RunState class needs to be modifies with a new property is_running.
> Finally a sensor will be added to use this new method in the hook.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] ashb commented on a change in pull request #6223: [AIRFLOW-5578] Adding the ability to define custom IndexView

[GitBox]

ashb commented on a change in pull request #6223: [AIRFLOW-5578] Adding the 
ability to define custom IndexView
URL: https://github.com/apache/airflow/pull/6223#discussion_r343099973
 
 

 ##
 File path: airflow/www/app.py
 ##
 @@ -99,10 +99,19 @@ def create_app(config=None, session=None, testing=False, 
app_name="Airflow"):
 """Your CUSTOM_SECURITY_MANAGER must now extend 
AirflowSecurityManager,
  not FAB's security manager.""")
 
+from airflow.www.blueprints import AirflowIndexView
+from flask_appbuilder import IndexView
+index_view_class = app.config.get('INDEX_VIEW_CLASS') or \
 
 Review comment:
   ```suggestion
   index_view_class = app.config.get('INDEX_VIEW_CLASS', 
AirflowIndexView)
   ```
   
   Will work I think..


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ashb commented on a change in pull request #6223: [AIRFLOW-5578] Adding the ability to define custom IndexView

[GitBox]

ashb commented on a change in pull request #6223: [AIRFLOW-5578] Adding the 
ability to define custom IndexView
URL: https://github.com/apache/airflow/pull/6223#discussion_r343100499
 
 

 ##
 File path: airflow/www/blueprints.py
 ##
 @@ -18,10 +18,19 @@
 # under the License.
 #
 from flask import Blueprint, redirect, url_for
+from flask_appbuilder import IndexView, expose
 
 routes = Blueprint('routes', __name__)
 
+# Place any Flask Blueprint routes (non Flask-Appbuilder ones) here
 
-@routes.route('/')
-def index():
-return redirect(url_for('Airflow.index'))
+# We can't put this into airflow.www.views because then we would have to 
import the views module
+# inside airflow.www.app prior to the appbuilder object being set up.
+# This would break other code in views not using the cached_appbuilder() 
function.
+
+
+class AirflowIndexView(IndexView):
 
 Review comment:
   I think this is better suited inside airflow/www/app.py


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] OmerJog commented on issue #6350: [AIRFLOW-5681] Allow specification of a tag or hash for the git_sync init container

[GitBox]

OmerJog commented on issue #6350: [AIRFLOW-5681] Allow specification of a tag 
or hash for the git_sync init container
URL: https://github.com/apache/airflow/pull/6350#issuecomment-550312550
 
 
   @george-miller 
   The test you added failed:
   
   ```
   
   55) FAIL: test_make_pod_git_sync_rev 
(tests.kubernetes.test_worker_configuration.TestKubernetesWorkerConfiguration)
   --
  Traceback (most recent call last):
   tests/kubernetes/test_worker_configuration.py line 444 in 
test_make_pod_git_sync_rev
 'The git_sync_rev env did not get into the init container')
  AssertionError: {'name': 'GIT_SYNC_REV',
   'value': ,
   'value_from': None} not found in [{'name': 'GIT_SYNC_REPO',
   'value': ,
   'value_from': None}, {'name': 'GIT_SYNC_BRANCH',
   'value': ,
   'value_from': None}, {'name': 'GIT_SYNC_ROOT',
   'value': ,
   'value_from': None}, {'name': 'GIT_SYNC_DEST', 'value': 'repo', 
'value_from': None}, {'name': 'GIT_SYNC_REV', 'value': 'sampletag', 
'value_from': None}, {'name': 'GIT_SYNC_DEPTH', 'value': '1', 'value_from': 
None}, {'name': 'GIT_SYNC_ONE_TIME', 'value': 'true', 'value_from': None}, 
{'name': 'GIT_SYNC_USERNAME',
   'value': ,
   'value_from': None}, {'name': 'GIT_SYNC_PASSWORD',
   'value': ,
   'value_from': None}, {'name': 'GIT_SYNC_USERNAME',
   'value': None,
   'value_from': {'config_map_key_ref': None,
  'field_ref': None,
  'resource_field_ref': None,
  'secret_key_ref': {'key': 'GIT_SYNC_USERNAME',
 'name': ,
 'optional': None}}}, {'name': 
'GIT_SYNC_PASSWORD',
   'value': None,
   'value_from': {'config_map_key_ref': None,
  'field_ref': None,
  'resource_field_ref': None,
  'secret_key_ref': {'key': 'GIT_SYNC_PASSWORD',
 'name': ,
 'optional': None}}}, {'name': 
'GIT_SSH_KEY_FILE', 'value': '/etc/git-secret/ssh', 'value_from': None}, 
{'name': 'GIT_SYNC_SSH', 'value': 'true', 'value_from': None}, {'name': 
'GIT_KNOWN_HOSTS', 'value': 'true', 'value_from': None}, {'name': 
'GIT_SSH_KNOWN_HOSTS_FILE',
   'value': '/etc/git-secret/known_hosts',
   'value_from': None}] : The git_sync_rev env did not get into the init 
container
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] OmerJog edited a comment on issue #6379: [AIRFLOW-5708] Optionally check task pools when parsing dags.

[GitBox]

OmerJog edited a comment on issue #6379: [AIRFLOW-5708] Optionally check task 
pools when parsing dags.
URL: https://github.com/apache/airflow/pull/6379#issuecomment-550311561
 
 
   Related  https://issues.apache.org/jira/browse/AIRFLOW-4927


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] OmerJog commented on issue #6379: [AIRFLOW-5708] Optionally check task pools when parsing dags.

[GitBox]

OmerJog commented on issue #6379: [AIRFLOW-5708] Optionally check task pools 
when parsing dags.
URL: https://github.com/apache/airflow/pull/6379#issuecomment-550311561
 
 
   Related / duplicate https://issues.apache.org/jira/browse/AIRFLOW-4927


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #6393: [AIRFLOW-5718] Add SFTPToGoogleCloudStorageOperator

[GitBox]

mik-laj commented on issue #6393: [AIRFLOW-5718] Add 
SFTPToGoogleCloudStorageOperator
URL: https://github.com/apache/airflow/pull/6393#issuecomment-550309339
 
 
   @TobKed Travis is sad. Can you fix it?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[airflow-site] branch aip-11 updated: Fix the navbar overlap on the home page (#131)

[kamilbregula]

This is an automated email from the ASF dual-hosted git repository.

kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git


The following commit(s) were added to refs/heads/aip-11 by this push:
 new 4c6fbb2  Fix the navbar overlap on the home page (#131)
4c6fbb2 is described below

commit 4c6fbb245720400843d2256a66f360f42698180c
Author: Kamil Breguła 
AuthorDate: Wed Nov 6 14:23:10 2019 +0100

Fix the navbar overlap on the home page (#131)
---
 landing-pages/site/assets/scss/_home-page.scss  | 8 ++--
 landing-pages/site/layouts/_default/baseof.html | 4 ++--
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/landing-pages/site/assets/scss/_home-page.scss 
b/landing-pages/site/assets/scss/_home-page.scss
index ebd0dec..bda7e24 100644
--- a/landing-pages/site/assets/scss/_home-page.scss
+++ b/landing-pages/site/assets/scss/_home-page.scss
@@ -19,8 +19,12 @@
 @import "media";
 
 .home-page-layout {
-  &.container {
-padding-top: 80px;
+  &.base-layout {
+padding-top: 70px;
+
+@media (max-width: $mobile) {
+  padding-top: 16px;
+}
   }
 }
 .principles-header {
diff --git a/landing-pages/site/layouts/_default/baseof.html 
b/landing-pages/site/layouts/_default/baseof.html
index ae82540..1dcb2e8 100644
--- a/landing-pages/site/layouts/_default/baseof.html
+++ b/landing-pages/site/layouts/_default/baseof.html
@@ -41,8 +41,8 @@
 
 
 
-
-
+
+
 {{ block "main" . }}{{ end }}
 
 

[GitHub] [airflow-site] mik-laj merged pull request #131: Fix the navbar overlap on the home page

[GitBox]

mik-laj merged pull request #131: Fix the navbar overlap on the home page
URL: https://github.com/apache/airflow-site/pull/131
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (AIRFLOW-5846) Implementing Sqlite hook tests

[Jarek Potiuk (Jira)]

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jarek Potiuk resolved AIRFLOW-5846.
---
Fix Version/s: 2.0.0
   Resolution: Fixed

> Implementing Sqlite hook tests
> --
>
> Key: AIRFLOW-5846
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5846
> Project: Apache Airflow
>  Issue Type: Test
>  Components: tests
>Affects Versions: 1.10.6
>Reporter: Sayed Mohammad Hossein Torabi
>Assignee: Sayed Mohammad Hossein Torabi
>Priority: Minor
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-5846) Implementing Sqlite hook tests

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968359#comment-16968359
 ] 

ASF subversion and git services commented on AIRFLOW-5846:
--

Commit b1b9a3a251f9b5a490bed451bf1c158764319ba0 in airflow's branch 
refs/heads/master from Hossein Torabi
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=b1b9a3a ]

[AIRFLOW-5846] Implementing Sqlite hook tests (#6509)



> Implementing Sqlite hook tests
> --
>
> Key: AIRFLOW-5846
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5846
> Project: Apache Airflow
>  Issue Type: Test
>  Components: tests
>Affects Versions: 1.10.6
>Reporter: Sayed Mohammad Hossein Torabi
>Assignee: Sayed Mohammad Hossein Torabi
>Priority: Minor
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AIRFLOW-5846) Implementing Sqlite hook tests

[ASF GitHub Bot (Jira)]

[ 
https://issues.apache.org/jira/browse/AIRFLOW-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968357#comment-16968357
 ] 

ASF GitHub Bot commented on AIRFLOW-5846:
-

potiuk commented on pull request #6509: [AIRFLOW-5846] Implementing Sqlite hook 
tests
URL: https://github.com/apache/airflow/pull/6509
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Implementing Sqlite hook tests
> --
>
> Key: AIRFLOW-5846
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5846
> Project: Apache Airflow
>  Issue Type: Test
>  Components: tests
>Affects Versions: 1.10.6
>Reporter: Sayed Mohammad Hossein Torabi
>Assignee: Sayed Mohammad Hossein Torabi
>Priority: Minor
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [airflow] potiuk merged pull request #6509: [AIRFLOW-5846] Implementing Sqlite hook tests

[GitBox]

potiuk merged pull request #6509: [AIRFLOW-5846] Implementing Sqlite hook tests
URL: https://github.com/apache/airflow/pull/6509
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #6496: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

[GitBox]

codecov-io edited a comment on issue #6496: [AIRFLOW-5704] Improve Kind 
Kubernetes scripts for local testing
URL: https://github.com/apache/airflow/pull/6496#issuecomment-549367986
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=h1) 
Report
   > Merging 
[#6496](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/d4ff52974373a6b4bf423917b8880c2492c51c0c?src=pr=desc)
 will **decrease** coverage by `0.3%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/6496/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master   #6496  +/-   ##
   =
   - Coverage  84%   83.7%   -0.31% 
   =
 Files 635 635  
 Lines   36715   36715  
   =
   - Hits30844   30733 -111 
   - Misses   58715982 +111
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5)
 | `33.33% <0%> (-41.67%)` | :arrow_down: |
   | 
[...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==)
 | `70.14% <0%> (-28.36%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=footer). 
Last update 
[d4ff529...9d921a1](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #6496: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

[GitBox]

codecov-io edited a comment on issue #6496: [AIRFLOW-5704] Improve Kind 
Kubernetes scripts for local testing
URL: https://github.com/apache/airflow/pull/6496#issuecomment-549367986
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=h1) 
Report
   > Merging 
[#6496](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/d4ff52974373a6b4bf423917b8880c2492c51c0c?src=pr=desc)
 will **decrease** coverage by `0.3%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/6496/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master   #6496  +/-   ##
   =
   - Coverage  84%   83.7%   -0.31% 
   =
 Files 635 635  
 Lines   36715   36715  
   =
   - Hits30844   30733 -111 
   - Misses   58715982 +111
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5)
 | `33.33% <0%> (-41.67%)` | :arrow_down: |
   | 
[...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==)
 | `70.14% <0%> (-28.36%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=footer). 
Last update 
[d4ff529...9d921a1](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #6496: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

[GitBox]

codecov-io edited a comment on issue #6496: [AIRFLOW-5704] Improve Kind 
Kubernetes scripts for local testing
URL: https://github.com/apache/airflow/pull/6496#issuecomment-549367986
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=h1) 
Report
   > Merging 
[#6496](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/d4ff52974373a6b4bf423917b8880c2492c51c0c?src=pr=desc)
 will **decrease** coverage by `0.3%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/6496/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master   #6496  +/-   ##
   =
   - Coverage  84%   83.7%   -0.31% 
   =
 Files 635 635  
 Lines   36715   36715  
   =
   - Hits30844   30733 -111 
   - Misses   58715982 +111
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5)
 | `33.33% <0%> (-41.67%)` | :arrow_down: |
   | 
[...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6496/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==)
 | `70.14% <0%> (-28.36%)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=footer). 
Last update 
[d4ff529...9d921a1](https://codecov.io/gh/apache/airflow/pull/6496?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feluelle commented on issue #6489: [AIRFLOW-3959] [AIRFLOW-4026] add UI filter with DAGs tags?

[GitBox]

feluelle commented on issue #6489: [AIRFLOW-3959] [AIRFLOW-4026] add UI filter 
with DAGs tags?
URL: https://github.com/apache/airflow/pull/6489#issuecomment-550297295
 
 
   Note that we also need to update the commit message to include both tickets 
so that both Jira tickets get linked to this PR. But it can also be done by the 
committer who merges it if the PR is already ready to be merged. But in your 
case I see you need to do some fixes first ;)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feluelle edited a comment on issue #6489: [AIRFLOW-3959] [AIRFLOW-4026] add UI filter with DAGs tags?

[GitBox]

feluelle edited a comment on issue #6489: [AIRFLOW-3959] [AIRFLOW-4026] add UI 
filter with DAGs tags?
URL: https://github.com/apache/airflow/pull/6489#issuecomment-550294632
 
 
   > How do I link this PR to two issues? [AIRFLOW-3959] [AIRFLOW-4026] add UI 
filter with DAGs tags?
   
   Since you also added consistency for tags (in the db) I would mention that 
or remove the UI part of it. Like: `[AIRFLOW-3959][AIRFLOW-4026] Add filter by 
DAG tags`
   
   > I can add display the tags next to the dag_id, but what will happen if a 
dag has a lot of labels?
   I guess it's an edge case that will not really happen, I will add it.
   
   It could happen and I think it should be handled. I suggest to just wrap the 
line?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feluelle commented on issue #6489: [AIRFLOW-3959] [AIRFLOW-4026] add UI filter with DAGs tags?

[GitBox]

feluelle commented on issue #6489: [AIRFLOW-3959] [AIRFLOW-4026] add UI filter 
with DAGs tags?
URL: https://github.com/apache/airflow/pull/6489#issuecomment-550294632
 
 
   > How do I link this PR to two issues? [AIRFLOW-3959] [AIRFLOW-4026] add UI 
filter with DAGs tags?
   
   Since you also added consistency for tags (in the db) I would mention that 
or remove the UI part of it. Like: `[AIRFLOW-3959] [AIRFLOW-4026] Add filter by 
DAG tags`
   
   > I can add display the tags next to the dag_id, but what will happen if a 
dag has a lot of labels?
   I guess it's an edge case that will not really happen, I will add it.
   
   It could happen and I think it should be handled. I suggest to just wrap the 
line?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feluelle commented on a change in pull request #6473: [AIRFLOW-5819] Update AWSBatchOperator default value

[GitBox]

feluelle commented on a change in pull request #6473: [AIRFLOW-5819] Update 
AWSBatchOperator default value
URL: https://github.com/apache/airflow/pull/6473#discussion_r343070664
 
 

 ##
 File path: airflow/contrib/operators/awsbatch_operator.py
 ##
 @@ -93,7 +93,7 @@ def __init__(self, job_name, job_definition, job_queue, 
overrides, array_propert
 self.job_definition = job_definition
 self.job_queue = job_queue
 self.overrides = overrides
-self.array_properties = array_properties
+self.array_properties = array_properties if array_properties is not 
None else {}
 
 Review comment:
   ```python
   >>> a = None or {}
   >>> a
   {}
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feluelle commented on a change in pull request #6473: [AIRFLOW-5819] Update AWSBatchOperator default value

[GitBox]

feluelle commented on a change in pull request #6473: [AIRFLOW-5819] Update 
AWSBatchOperator default value
URL: https://github.com/apache/airflow/pull/6473#discussion_r343069637
 
 

 ##
 File path: airflow/contrib/operators/awsbatch_operator.py
 ##
 @@ -93,7 +93,7 @@ def __init__(self, job_name, job_definition, job_queue, 
overrides, array_propert
 self.job_definition = job_definition
 self.job_queue = job_queue
 self.overrides = overrides
-self.array_properties = array_properties
+self.array_properties = array_properties if array_properties is not 
None else {}
 
 Review comment:
   ```suggestion
   self.array_properties = array_properties or {}
   ```
   Why not just like that?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #6509: [AIRFLOW-5846] Implementing Sqlite hook tests

[GitBox]

codecov-io commented on issue #6509: [AIRFLOW-5846] Implementing Sqlite hook 
tests
URL: https://github.com/apache/airflow/pull/6509#issuecomment-550284728
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=h1) 
Report
   > Merging 
[#6509](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/d4ff52974373a6b4bf423917b8880c2492c51c0c?src=pr=desc)
 will **decrease** coverage by `0.39%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/6509/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master#6509 +/-   ##
   =
   - Coverage  84%   83.61%   -0.4% 
   =
 Files 635  635 
 Lines   3671536715 
   =
   - Hits3084430700-144 
   - Misses   5871 6015+144
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5)
 | `33.33% <0%> (-41.67%)` | :arrow_down: |
   | 
[...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==)
 | `70.14% <0%> (-28.36%)` | :arrow_down: |
   | 
[airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5)
 | `91.52% <0%> (-1.7%)` | :arrow_down: |
   | 
[airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5)
 | `89.83% <0%> (-1.7%)` | :arrow_down: |
   | 
[airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==)
 | `89.9% <0%> (-1.53%)` | :arrow_down: |
   | 
[airflow/hooks/postgres\_hook.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9wb3N0Z3Jlc19ob29rLnB5)
 | `92.85% <0%> (-1.43%)` | :arrow_down: |
   | ... and [1 
more](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=footer). 
Last update 
[d4ff529...5ef6764](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #6509: [AIRFLOW-5846] Implementing Sqlite hook tests

[GitBox]

codecov-io edited a comment on issue #6509: [AIRFLOW-5846] Implementing Sqlite 
hook tests
URL: https://github.com/apache/airflow/pull/6509#issuecomment-550284728
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=h1) 
Report
   > Merging 
[#6509](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=desc) into 
[master](https://codecov.io/gh/apache/airflow/commit/d4ff52974373a6b4bf423917b8880c2492c51c0c?src=pr=desc)
 will **decrease** coverage by `0.39%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/airflow/pull/6509/graphs/tree.svg?width=650=WdLKlKHOAU=150=pr)](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=tree)
   
   ```diff
   @@Coverage Diff@@
   ##   master#6509 +/-   ##
   =
   - Coverage  84%   83.61%   -0.4% 
   =
 Files 635  635 
 Lines   3671536715 
   =
   - Hits3084430700-144 
   - Misses   5871 6015+144
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=tree) | 
Coverage Δ | |
   |---|---|---|
   | 
[airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=)
 | `0% <0%> (-100%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==)
 | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | 
[airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==)
 | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | 
[airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==)
 | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | 
[airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5)
 | `33.33% <0%> (-41.67%)` | :arrow_down: |
   | 
[...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==)
 | `70.14% <0%> (-28.36%)` | :arrow_down: |
   | 
[airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5)
 | `91.52% <0%> (-1.7%)` | :arrow_down: |
   | 
[airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5)
 | `89.83% <0%> (-1.7%)` | :arrow_down: |
   | 
[airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==)
 | `89.9% <0%> (-1.53%)` | :arrow_down: |
   | 
[airflow/hooks/postgres\_hook.py](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9wb3N0Z3Jlc19ob29rLnB5)
 | `92.85% <0%> (-1.43%)` | :arrow_down: |
   | ... and [1 
more](https://codecov.io/gh/apache/airflow/pull/6509/diff?src=pr=tree-more) 
| |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=footer). 
Last update 
[d4ff529...5ef6764](https://codecov.io/gh/apache/airflow/pull/6509?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feluelle commented on a change in pull request #6496: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

[GitBox]

feluelle commented on a change in pull request #6496: [AIRFLOW-5704] Improve 
Kind Kubernetes scripts for local testing
URL: https://github.com/apache/airflow/pull/6496#discussion_r343047685
 
 

 ##
 File path: BREEZE.rst
 ##
 @@ -79,11 +79,11 @@ images maintained on the Docker Hub in the 
``apache/airflow`` repository.
 
 There are three images that we are currently managing:
 
-* **CI image*** that is used for testing od both Unit tests and static check 
tests.
-  It contains a lot test-related packages (size of ~1GB).
-  Its tag follows the pattern of ``-python-ci``
+* **CI image*** that is used for testing. It contains a lot more test-related 
installed software
+  (size of ~1GB). Its tag follows the pattern of 
``-python-ci``
   (for example, ``apache/airflow:master-python3.6-ci``). The image is built 
using the
   ``_ Dockerfile.
+* **Production-ready image** that is used TODO
 
 Review comment:
   ```suggestion
   * **Production-ready image** is in WIP state. See 
https://github.com/apache/airflow/pull/6501 for more information.
   ```
   Or remove it completely and add a full description to #6501 instead?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow-site] mik-laj merged pull request #120: Add rating component with Google Analytics integration

[GitBox]

mik-laj merged pull request #120: Add rating component with Google Analytics 
integration
URL: https://github.com/apache/airflow-site/pull/120
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[airflow-site] branch aip-11 updated (82148f0 -> 7421384)

[kamilbregula]

This is an automated email from the ASF dual-hosted git repository.

kamilbregula pushed a change to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git.


from 82148f0  Add new sample blog post (#127)
 add 7421384  Add rating component with Google Analytics integration (#120)

No new revisions were added by this update.

Summary of changes:
 landing-pages/site/assets/icons/star-icon.svg  |  5 +++
 .../site/assets/scss/{_pager.scss => _rating.scss} | 32 +--
 landing-pages/site/assets/scss/main-custom.scss|  1 +
 .../layouts/partials/{quote.html => rating.html}   | 11 +++---
 landing-pages/src/index.js |  1 +
 landing-pages/src/{index.js => js/rating.js}   | 46 ++
 6 files changed, 69 insertions(+), 27 deletions(-)
 create mode 100644 landing-pages/site/assets/icons/star-icon.svg
 copy landing-pages/site/assets/scss/{_pager.scss => _rating.scss} (70%)
 copy landing-pages/site/layouts/partials/{quote.html => rating.html} (74%)
 copy landing-pages/src/{index.js => js/rating.js} (50%)

[GitHub] [airflow-site] mik-laj merged pull request #127: Add new sample blog post

[GitBox]

mik-laj merged pull request #127: Add new sample blog post
URL: https://github.com/apache/airflow-site/pull/127
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow-site] mik-laj merged pull request #128: Fix content of navbar link item

[GitBox]

mik-laj merged pull request #128: Fix content of navbar link item
URL: https://github.com/apache/airflow-site/pull/128
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow-site] mik-laj closed pull request #105: Minor fixes

[GitBox]

mik-laj closed pull request #105: Minor fixes
URL: https://github.com/apache/airflow-site/pull/105
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow-site] mik-laj merged pull request #130: Ignore cases in integration search

[GitBox]

mik-laj merged pull request #130: Ignore cases in integration search
URL: https://github.com/apache/airflow-site/pull/130
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[airflow-site] branch aip-11 updated: Fix content of navbar link item (#128)

[kamilbregula]

This is an automated email from the ASF dual-hosted git repository.

kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git


The following commit(s) were added to refs/heads/aip-11 by this push:
 new 54d0e7e  Fix content of navbar link item (#128)
54d0e7e is described below

commit 54d0e7e0f64876808fdaef8ba14b2a8c03969aab
Author: Kamil Breguła 
AuthorDate: Wed Nov 6 12:51:24 2019 +0100

Fix content of navbar link item (#128)
---
 landing-pages/site/assets/scss/_navbar.scss | 1 +
 1 file changed, 1 insertion(+)

diff --git a/landing-pages/site/assets/scss/_navbar.scss 
b/landing-pages/site/assets/scss/_navbar.scss
index f4f1659..a928143 100644
--- a/landing-pages/site/assets/scss/_navbar.scss
+++ b/landing-pages/site/assets/scss/_navbar.scss
@@ -49,6 +49,7 @@
 @extend .bodytext__medium--greyish-brown;
 margin-right: 30px;
 position: relative;
+width: fit-content;
 
 &::before, &::after {
   content: "";

[airflow-site] branch aip-11 updated: Make arrow in accordion smaller on mobile devicese (#133)

[kamilbregula]

This is an automated email from the ASF dual-hosted git repository.

kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git


The following commit(s) were added to refs/heads/aip-11 by this push:
 new 32aff96  Make arrow in accordion smaller on mobile devicese (#133)
32aff96 is described below

commit 32aff9650f0830280da35cdfaee0bc5afafe48f5
Author: Kamil Breguła 
AuthorDate: Wed Nov 6 12:49:29 2019 +0100

Make arrow in accordion smaller on mobile devicese (#133)
---
 landing-pages/site/assets/scss/_accordion.scss | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/landing-pages/site/assets/scss/_accordion.scss 
b/landing-pages/site/assets/scss/_accordion.scss
index 6bdcce6..330ec7b 100644
--- a/landing-pages/site/assets/scss/_accordion.scss
+++ b/landing-pages/site/assets/scss/_accordion.scss
@@ -97,6 +97,12 @@ details.accordion {
   }
 }
 
+.accordion__arrow {
+  width: 28px;
+  height: 28px;
+  top: 5px;
+}
+
 .accordion__content {
   &.indented {
 margin-left: 80px;

[GitHub] [airflow-site] mik-laj merged pull request #126: Small typo face-to face => face-to-face

[GitBox]

mik-laj merged pull request #126: Small typo face-to face => face-to-face
URL: https://github.com/apache/airflow-site/pull/126
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[airflow-site] branch aip-11 updated: Add new sample blog post (#127)

[kamilbregula]

This is an automated email from the ASF dual-hosted git repository.

kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git


The following commit(s) were added to refs/heads/aip-11 by this push:
 new 82148f0  Add new sample blog post (#127)
82148f0 is described below

commit 82148f01f2b3cb86a928f2469e8c84d92e2c3dd2
Author: Kamil Breguła 
AuthorDate: Wed Nov 6 12:52:13 2019 +0100

Add new sample blog post (#127)
---
 ...Its-a-breeze-to-develop-apache-airflow.md => Grumpy-cat.md} | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git 
a/landing-pages/site/content/en/blog/Its-a-breeze-to-develop-apache-airflow.md 
b/landing-pages/site/content/en/blog/Grumpy-cat.md
similarity index 97%
rename from 
landing-pages/site/content/en/blog/Its-a-breeze-to-develop-apache-airflow.md
rename to landing-pages/site/content/en/blog/Grumpy-cat.md
index cf7c326..233741a 100644
--- 
a/landing-pages/site/content/en/blog/Its-a-breeze-to-develop-apache-airflow.md
+++ b/landing-pages/site/content/en/blog/Grumpy-cat.md
@@ -1,6 +1,6 @@
 ---
-title: "It's a breeze to develop Apache Airflow"
-linkTitle: "It's a breeze to develop Apache Airflow"
+title: "Grumpy cat"
+linkTitle: "Grumpy cat"
 author: "Jarek Potiuk"
 description: "Working on an Open Source project such as Apache Airflow is very 
demanding but also equally rewarding when you realize how many businesses use 
it every day."
 tags: ["Community"]
@@ -30,9 +30,15 @@ How do you define productivity though? And how can you 
measure it? I have a poss
 
 I could think of a number of projects where I followed that philosophy of work 
and where it boosted the productivity of my team. Today, however, I’d like to 
focus on Apache Airflow.
 
+{{< youtube 8IHLQxOghUQ >}}
+
 # The Apache Airflow project’s setup
 Initially, we started contributing to this fantastic open-source project with 
a team of three which then grew to five. When we kicked it off a year ago, I 
realized pretty soon where the biggest bottlenecks and areas for improvement in 
terms of productivity were. Even with the help of our client, who provided us 
with a “homegrown” development environment it took us literally days to set it 
up and learn some basics.
 
+{{< gist spf13 7896402 >}}
+
 Apache Airflow is a thoroughly tested project—it has almost 4,000 tests with 
around 80% coverage and varying complexity (from simple unit tests to 
end-to-end system tests). Airflow follows a modern software project philosophy: 
every single Pull Request can only be merged if all the tests pass. But that 
creates another problem—having to run all the tests for all the backends (there 
are 3 of them) and different python versions. At the time of starting the 
project we still supported 2.7, 3. [...]
 
 This is hardly acceptable to any developer. Usually you can work on several 
issues in parallel. Such waiting is not the worst thing that can happen, but 
there are always costs of context-switching, distractions, getting out of the 
flow and the good old “I already forgot what I was doing” phase by the time the 
build is completed. It’s even worse when you want to make a small Pull Request. 
For example, when you find a bug and have just one line of fix for it. You 
submit the Pull Request an [...]
+
+{{< tweet 1189242550403653632 >}}

[GitHub] [airflow-site] mik-laj merged pull request #134: Improve position of change suggestion button

[GitBox]

mik-laj merged pull request #134: Improve position of change suggestion button
URL: https://github.com/apache/airflow-site/pull/134
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[airflow-site] branch aip-11 updated: Improve position of change suggestion button (#134)

[kamilbregula]

This is an automated email from the ASF dual-hosted git repository.

kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git


The following commit(s) were added to refs/heads/aip-11 by this push:
 new 8ebce27  Improve position of change suggestion button (#134)
8ebce27 is described below

commit 8ebce270127299478551446deda89fb43d5cdac1
Author: Kamil Breguła 
AuthorDate: Wed Nov 6 12:50:26 2019 +0100

Improve position of change suggestion button (#134)
---
 landing-pages/site/assets/scss/_base-layout.scss|  4 +++-
 landing-pages/site/layouts/_default/baseof.html |  8 +---
 landing-pages/site/layouts/blog/baseof.html |  8 +---
 landing-pages/site/layouts/case-studies/baseof.html |  8 +---
 landing-pages/site/layouts/community/baseof.html|  8 +---
 landing-pages/site/layouts/install/baseof.html  |  8 +---
 landing-pages/site/layouts/meetups/baseof.html  |  8 +---
 landing-pages/site/layouts/taxonomy/baseof.html | 10 ++
 8 files changed, 39 insertions(+), 23 deletions(-)

diff --git a/landing-pages/site/assets/scss/_base-layout.scss 
b/landing-pages/site/assets/scss/_base-layout.scss
index be4dfef..c98271f 100644
--- a/landing-pages/site/assets/scss/_base-layout.scss
+++ b/landing-pages/site/assets/scss/_base-layout.scss
@@ -25,7 +25,9 @@
   &--button {
 display: flex;
 position: sticky;
-right: 60px;
+width: fit-content;
+margin-left: auto;
+margin-right: 40px;
 bottom: 40px;
 justify-content: flex-end;
 z-index: 1;
diff --git a/landing-pages/site/layouts/_default/baseof.html 
b/landing-pages/site/layouts/_default/baseof.html
index ba4792f..ae82540 100644
--- a/landing-pages/site/layouts/_default/baseof.html
+++ b/landing-pages/site/layouts/_default/baseof.html
@@ -41,14 +41,16 @@
 
 
 
-
-{{ block "main" . }}{{ end }}
+
+
+{{ block "main" . }}{{ end }}
+
 
 https://github.com;>
 {{ partial "buttons/button-with-icon" (dict "text" "Suggest a 
change on this page" "class" "button-fixed") }}
 
 
-
+
 
 {{ partialCached "footer.html" . }}
 {{ partialCached "scripts.html" . }}
diff --git a/landing-pages/site/layouts/blog/baseof.html 
b/landing-pages/site/layouts/blog/baseof.html
index f73b028..0fa1bf0 100644
--- a/landing-pages/site/layouts/blog/baseof.html
+++ b/landing-pages/site/layouts/blog/baseof.html
@@ -27,14 +27,16 @@
 {{ partial "navbar.html" . }}
 
 
-
-{{ block "main" . }}{{ end }}
+
+
+{{ block "main" . }}{{ end }}
+
 
 https://github.com;>
 {{ partial "buttons/button-with-icon" (dict "text" "Suggest a 
change on this page") }}
 
 
-
+
 
 {{ partialCached "footer.html" . }}
 {{ partialCached "scripts.html" . }}
diff --git a/landing-pages/site/layouts/case-studies/baseof.html 
b/landing-pages/site/layouts/case-studies/baseof.html
index 78ffb77..8add970 100644
--- a/landing-pages/site/layouts/case-studies/baseof.html
+++ b/landing-pages/site/layouts/case-studies/baseof.html
@@ -27,14 +27,16 @@
 {{ partial "navbar.html" . }}
 
 
-
-{{ block "main" . }}{{ end }}
+
+
+{{ block "main" . }}{{ end }}
+
 
 https://github.com;>
 {{ partial "buttons/button-with-icon" (dict "text" "Suggest a 
change on this page" "class" "button-fixed") }}
 
 
-
+
 
 {{ partialCached "footer.html" . }}
 {{ partialCached "scripts.html" . }}
diff --git a/landing-pages/site/layouts/community/baseof.html 
b/landing-pages/site/layouts/community/baseof.html
index 78ffb77..8add970 100644
--- a/landing-pages/site/layouts/community/baseof.html
+++ b/landing-pages/site/layouts/community/baseof.html
@@ -27,14 +27,16 @@
 {{ partial "navbar.html" . }}
 
 
-
-{{ block "main" . }}{{ end }}
+
+
+{{ block "main" . }}{{ end }}
+
 
 https://github.com;>
 {{ partial "buttons/button-with-icon" (dict "text" "Suggest a 
change on this page" "class" "button-fixed") }}
 
 
-
+
 
 {{ partialCached "footer.html" . }}
 {{ partialCached "scripts.html" . }}
diff --git a/landing-pages/site/layouts/install/baseof.html 
b/landing-pages/site/layouts/install/baseof.html
index 78ffb77..8add970 100644
--- a/landing-pages/site/layouts/install/baseof.html
+++ b/landing-pages/site/layouts/install/baseof.html
@@ -27,14 +27,16 @@
 {{ partial "navbar.html" . }}
 
 
-
-{{ block "main" . }}{{ end }}
+
+
+{{ block "main" . }}{{ end }}
+
 
 https://github.com;>
 {{ partial "buttons/button-with-icon" (dict "text" "Suggest a 
change on this page" "class" "button-fixed") }}
 

[GitHub] [airflow-site] mik-laj commented on issue #105: Minor fixes

[GitBox]

mik-laj commented on issue #105: Minor fixes
URL: https://github.com/apache/airflow-site/pull/105#issuecomment-550275646
 
 
   Supersed by:
   https://github.com/apache/airflow-site/pull/126
   https://github.com/apache/airflow-site/pull/127
   https://github.com/apache/airflow-site/pull/128
   https://github.com/apache/airflow-site/pull/129
   https://github.com/apache/airflow-site/pull/130
   https://github.com/apache/airflow-site/pull/131
   https://github.com/apache/airflow-site/pull/132
   https://github.com/apache/airflow-site/pull/133
   https://github.com/apache/airflow-site/pull/134


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[airflow-site] branch aip-11 updated: Improve the gap between integrators (#132)

[kamilbregula]

This is an automated email from the ASF dual-hosted git repository.

kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git


The following commit(s) were added to refs/heads/aip-11 by this push:
 new 23e7973  Improve the gap between integrators (#132)
23e7973 is described below

commit 23e7973083b895920fd88e7389118573f8ace9c7
Author: Kamil Breguła 
AuthorDate: Wed Nov 6 12:49:00 2019 +0100

Improve the gap between integrators (#132)
---
 landing-pages/site/assets/scss/_home-page.scss | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/landing-pages/site/assets/scss/_home-page.scss 
b/landing-pages/site/assets/scss/_home-page.scss
index b8ac901..ebd0dec 100644
--- a/landing-pages/site/assets/scss/_home-page.scss
+++ b/landing-pages/site/assets/scss/_home-page.scss
@@ -36,6 +36,10 @@
   }
 }
 
-#integration .search-form {
-  margin-bottom: 24px;
+#integrations .list-items {
+  margin-top: 40px;
+
+  @media (max-width: $mobile) {
+margin-top: 20px;
+  }
 }