[GitHub] [airflow] feluelle commented on a change in pull request #8895: Add Delete/Create S3 bucket operators

2020-05-21 Thread GitBox
feluelle commented on a change in pull request #8895: URL: https://github.com/apache/airflow/pull/8895#discussion_r428473778 ## File path: airflow/providers/amazon/aws/operators/s3_bucket.py ## @@ -0,0 +1,119 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [airflow] Khrol commented on pull request #7622: [AIRFLOW-6983] Use SingletonThreadPool for database communication

2020-05-21 Thread GitBox
Khrol commented on pull request #7622: URL: https://github.com/apache/airflow/pull/7622#issuecomment-631932084 @ashb the same tests are green on SQLite but failing on MySQL and PG. This might be related to the warning above. Any ideas on how to troubleshoot? I can't reproduce the problem

[GitHub] [airflow] eladkal commented on a change in pull request #8942: #8525 Add SQL Branch Operator

2020-05-21 Thread GitBox
eladkal commented on a change in pull request #8942: URL: https://github.com/apache/airflow/pull/8942#discussion_r428493813 ## File path: airflow/operators/sql_branch_operator.py ## @@ -0,0 +1,165 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] ashb commented on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
ashb commented on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-631960980 @mik-laj That's why I asked about CI, not system tests. A large part of the slow down for our tests was pulling images, so if we can reduce the size of that image we can speed

[GitHub] [airflow] ashb commented on a change in pull request #8943: Python base images are stored in cache

2020-05-21 Thread GitBox
ashb commented on a change in pull request #8943: URL: https://github.com/apache/airflow/pull/8943#discussion_r428522890 ## File path: scripts/ci/_utils.sh ## @@ -227,6 +227,9 @@ function initialize_common_environment { # Artifact name suffix for SVN packaging

[GitHub] [airflow] ashb commented on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
ashb commented on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-631970936 Might be worth investigating, at least once we've gotten rid of all the items on this list which are far bigger. https://github.com/wagoodman/dive is a useful too for exploring

[GitHub] [airflow] potiuk opened a new pull request #8943: Python base images are stored in cache

2020-05-21 Thread GitBox
potiuk opened a new pull request #8943: URL: https://github.com/apache/airflow/pull/8943 All PRs will used cached "latest good" version of the python base images from our GitHub registry. The python versions in the Github Registry will only get updated after a master build (which

[airflow] branch v1-10-test updated: Hive/Hadoop minicluster needs JDK8 and JAVA_HOME to work (#8938)

2020-05-21 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 4b40eaf Hive/Hadoop minicluster needs

[GitHub] [airflow] feluelle commented on a change in pull request #8895: Add Delete/Create S3 bucket operators

2020-05-21 Thread GitBox
feluelle commented on a change in pull request #8895: URL: https://github.com/apache/airflow/pull/8895#discussion_r428468425 ## File path: airflow/providers/amazon/aws/example_dags/example_s3_bucket.py ## @@ -0,0 +1,70 @@ +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] randr97 commented on a change in pull request #8895: Add Delete/Create S3 bucket operators

2020-05-21 Thread GitBox
randr97 commented on a change in pull request #8895: URL: https://github.com/apache/airflow/pull/8895#discussion_r428471046 ## File path: airflow/providers/amazon/aws/example_dags/example_s3_bucket.py ## @@ -0,0 +1,70 @@ +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] ashb commented on a change in pull request #8944: Update example webserver_config.py to show correct CSRF config

2020-05-21 Thread GitBox
ashb commented on a change in pull request #8944: URL: https://github.com/apache/airflow/pull/8944#discussion_r428525338 ## File path: airflow/config_templates/default_webserver_config.py ## @@ -34,7 +34,7 @@ SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN') #

[GitHub] [airflow] mik-laj commented on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
mik-laj commented on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-631970084 @ashb On CI, we don't use it yet. This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (AIRFLOW-6983) Disabled connection pool in CLI might be harmful

2020-05-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112867#comment-17112867 ] ASF GitHub Bot commented on AIRFLOW-6983: - Khrol commented on pull request #7622: URL:

[GitHub] [airflow] sylr commented on issue #8907: Airflow web UI is slow

2020-05-21 Thread GitBox
sylr commented on issue #8907: URL: https://github.com/apache/airflow/issues/8907#issuecomment-631951361 I don't see anything suspicious is the logs. But know that I think of it, 5s latency looks very much like a DNS timeout. Any chance each request might trigger a DNS resolve

[GitHub] [airflow] mik-laj commented on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
mik-laj commented on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-631959453 @ashb we use `gcloud` and `aws-cli` in system tests. We don't always have an operator. gcloud is even used by the Airflow operators.

[GitHub] [airflow] ashb opened a new pull request #8944: Update example webserver_config.py to show correct CSRF config

2020-05-21 Thread GitBox
ashb opened a new pull request #8944: URL: https://github.com/apache/airflow/pull/8944 CSRF_ENABLED does nothing. Thankfully, due to sensible defaults in flask-wtf, CSRF is on by default, but we should set this correctly. Fixes #8915 --- Make sure to mark the

[GitHub] [airflow] ashb commented on issue #8915: CSRF configuration is missing the WTF_ prefix

2020-05-21 Thread GitBox
ashb commented on issue #8915: URL: https://github.com/apache/airflow/issues/8915#issuecomment-631965434 @zachliu Ahha gotcha, yes. PR to fix that for new installs (we can't do anything about existing/already generated webserver_config.py, but at least it would stop some people tearing

[GitHub] [airflow] feluelle commented on a change in pull request #8895: Add Delete/Create S3 bucket operators

2020-05-21 Thread GitBox
feluelle commented on a change in pull request #8895: URL: https://github.com/apache/airflow/pull/8895#discussion_r428473778 ## File path: airflow/providers/amazon/aws/operators/s3_bucket.py ## @@ -0,0 +1,119 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [airflow] milton0825 closed pull request #8889: Support custom Variable implementation

2020-05-21 Thread GitBox
milton0825 closed pull request #8889: URL: https://github.com/apache/airflow/pull/8889 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] ashb commented on a change in pull request #8940: Remove side-effect of session in FAB

2020-05-21 Thread GitBox
ashb commented on a change in pull request #8940: URL: https://github.com/apache/airflow/pull/8940#discussion_r428511975 ## File path: airflow/www/app.py ## @@ -95,9 +96,18 @@ def create_app(config=None, session=None, testing=False, app_name="Airflow"):

[GitHub] [airflow] ashb commented on a change in pull request #8940: Remove side-effect of session in FAB

2020-05-21 Thread GitBox
ashb commented on a change in pull request #8940: URL: https://github.com/apache/airflow/pull/8940#discussion_r428511353 ## File path: airflow/www/app.py ## @@ -95,9 +96,18 @@ def create_app(config=None, session=None, testing=False, app_name="Airflow"):

[GitHub] [airflow] ashb commented on pull request #7622: [AIRFLOW-6983] Use SingletonThreadPool for database communication

2020-05-21 Thread GitBox
ashb commented on pull request #7622: URL: https://github.com/apache/airflow/pull/7622#issuecomment-631955423 H. Let me take a look This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (AIRFLOW-6983) Disabled connection pool in CLI might be harmful

2020-05-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112916#comment-17112916 ] ASF GitHub Bot commented on AIRFLOW-6983: - ashb commented on pull request #7622: URL:

[GitHub] [airflow] ashb commented on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
ashb commented on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-631957581 /cc @jhtimmins This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] ashb commented on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
ashb commented on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-631957488 @potiuk Do our _CI_ tests need `gcloud` or `aws` CLI tools? Between them the add something like 500Mb to the image. (It would be _really_ nice if docker let you easily compose

[GitHub] [airflow] ashb commented on a change in pull request #8943: Python base images are stored in cache

2020-05-21 Thread GitBox
ashb commented on a change in pull request #8943: URL: https://github.com/apache/airflow/pull/8943#discussion_r428521369 ## File path: scripts/ci/_utils.sh ## @@ -227,6 +227,9 @@ function initialize_common_environment { # Artifact name suffix for SVN packaging

[GitHub] [airflow] bensta commented on issue #8467: ImportError: cannot import name 'bytes_type' from 'oauthlib.common'

2020-05-21 Thread GitBox
bensta commented on issue #8467: URL: https://github.com/apache/airflow/issues/8467#issuecomment-631968018 I face the same issue, forcing oauthlib 2.1.0 does not solve the issue for me. I am testing on Minikube, official apache/airflow-python3.6 image, trying to run with flask RBAC,

[GitHub] [airflow] ashb commented on pull request #8256: updated _write_args on PythonVirtualenvOperator

2020-05-21 Thread GitBox
ashb commented on pull request #8256: URL: https://github.com/apache/airflow/pull/8256#issuecomment-631983650 This appears to have not run the main unitests either. Please rebase @maganaluis This is an automated message

[GitHub] [airflow] ashb commented on a change in pull request #8256: updated _write_args on PythonVirtualenvOperator

2020-05-21 Thread GitBox
ashb commented on a change in pull request #8256: URL: https://github.com/apache/airflow/pull/8256#discussion_r428543926 ## File path: tests/operators/test_python_operator.py ## @@ -340,6 +341,80 @@ def test_echo_env_variables(self): t.run(start_date=DEFAULT_DATE,

[GitHub] [airflow] ashb opened a new pull request #8945: Remove singularity from CI images

2020-05-21 Thread GitBox
ashb opened a new pull request #8945: URL: https://github.com/apache/airflow/pull/8945 The singularity operator tests _have always_ used mocking, so we were adding 700MB to our docker image for nothing. Fixes #8774 --- Make sure to mark the boxes below before

[GitHub] [airflow] sylr commented on issue #8907: Airflow web UI is slow

2020-05-21 Thread GitBox
sylr commented on issue #8907: URL: https://github.com/apache/airflow/issues/8907#issuecomment-631986877 After tcpdump'ing the calico interface of the pod I see DNS storms happening regularly: ``` 09:28:59.015823 IP 10.100.200.247.48834 > 10.200.0.10.53: 5541+ A?

[GitHub] [airflow] ashb merged pull request #8910: Accessibility - Add TaskInstance State to tiTooltip in RBAC UI

2020-05-21 Thread GitBox
ashb merged pull request #8910: URL: https://github.com/apache/airflow/pull/8910 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated (f17b4bb -> a9dfd7d)

2020-05-21 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f17b4bb Fix DagRun Prefix for Performance script (#8934) add a9dfd7d Remove side-effect of session in

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8910: Accessibility - Add TaskInstance State to tiTooltip in RBAC UI

2020-05-21 Thread GitBox
boring-cyborg[bot] commented on pull request #8910: URL: https://github.com/apache/airflow/pull/8910#issuecomment-631995867 Awesome work, congrats on your first merged pull request! This is an automated message from the

[GitHub] [airflow] zacharya19 commented on pull request #8897: Filter dags by clicking on tag

2020-05-21 Thread GitBox
zacharya19 commented on pull request #8897: URL: https://github.com/apache/airflow/pull/8897#issuecomment-632002345 @Sangarshanan It's not an issue with your code (obviously :) ), look like master is broken. This is an

[GitHub] [airflow] sylr commented on issue #8907: Airflow web UI is slow

2020-05-21 Thread GitBox
sylr commented on issue #8907: URL: https://github.com/apache/airflow/issues/8907#issuecomment-632007715 Turning off the pgbouncer makes the endpoint `/static/bootstrap-toggle.min.js` unresponsive. This is an automated

[GitHub] [airflow] potiuk commented on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
potiuk commented on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-632008036 @ashb @mik-laj -> issues added for gcloud and awscli. Feel free to implement if you have time This is an automated

[GitHub] [airflow] potiuk commented on pull request #8943: Python base images are stored in cache

2020-05-21 Thread GitBox
potiuk commented on pull request #8943: URL: https://github.com/apache/airflow/pull/8943#issuecomment-632011170 Seems we are back to the usual speed :) And caching the python image works as well: ``` 2020-05-21T10:02:14.4034205Z Pulling the image

[GitHub] [airflow] kaxil opened a new pull request #8948: Fix docstring in DagFileProcessor._schedule_task_instances

2020-05-21 Thread GitBox
kaxil opened a new pull request #8948: URL: https://github.com/apache/airflow/pull/8948 Fix docstring in DagFileProcessor._schedule_task_instances --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x]

[GitHub] [airflow] ashb closed issue #8774: Move singularity out of main CI tests and into separate docker image

2020-05-21 Thread GitBox
ashb closed issue #8774: URL: https://github.com/apache/airflow/issues/8774 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] ashb merged pull request #8945: Remove singularity from CI images

2020-05-21 Thread GitBox
ashb merged pull request #8945: URL: https://github.com/apache/airflow/pull/8945 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] kaxil merged pull request #8948: Fix docstring in DagFileProcessor._schedule_task_instances

2020-05-21 Thread GitBox
kaxil merged pull request #8948: URL: https://github.com/apache/airflow/pull/8948 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk merged pull request #8943: Python base images are stored in cache

2020-05-21 Thread GitBox
potiuk merged pull request #8943: URL: https://github.com/apache/airflow/pull/8943 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] ashb commented on pull request #8256: updated _write_args on PythonVirtualenvOperator

2020-05-21 Thread GitBox
ashb commented on pull request #8256: URL: https://github.com/apache/airflow/pull/8256#issuecomment-631981510 @maganaluis This only applies to 1.10.x correct, and isn't a problem with the code in master? Is it just because in master we don't pass _all_ arguments, only the ones that the

[GitHub] [airflow] turbaszek commented on issue #8703: Support for set in XCom serialization

2020-05-21 Thread GitBox
turbaszek commented on issue #8703: URL: https://github.com/apache/airflow/issues/8703#issuecomment-631986100 @kaxil @ashb @mik-laj @potiuk @zhongjiajie what's your opinion? Should we silently make `set` JSON-serializable or not?

[airflow] branch master updated (8476c1e -> f17b4bb)

2020-05-21 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 8476c1e Hive/Hadoop minicluster needs JDK8 and JAVA_HOME to work (#8938) add f17b4bb Fix DagRun Prefix

[GitHub] [airflow] kaxil merged pull request #8934: Fix DagRun Prefix for Performance script

2020-05-21 Thread GitBox
kaxil merged pull request #8934: URL: https://github.com/apache/airflow/pull/8934 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] mik-laj merged pull request #8940: Remove side-effect of session in FAB

2020-05-21 Thread GitBox
mik-laj merged pull request #8940: URL: https://github.com/apache/airflow/pull/8940 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk edited a comment on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
potiuk edited a comment on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-632003742 We can get those also used from docker images. We have now the docker socket mapped inside the docker container for breeze so we can do `docker pull

[GitHub] [airflow] kaxil opened a new pull request #8949: Add note in Updating.md about the removel of DagRun.ID_PREFIX

2020-05-21 Thread GitBox
kaxil opened a new pull request #8949: URL: https://github.com/apache/airflow/pull/8949 It was removed in https://github.com/apache/airflow/pull/7583 and https://github.com/apache/airflow/pull/7484 --- Make sure to mark the boxes below before creating PR: [x] - [x]

[GitHub] [airflow] ashb closed issue #8915: CSRF configuration is missing the WTF_ prefix

2020-05-21 Thread GitBox
ashb closed issue #8915: URL: https://github.com/apache/airflow/issues/8915 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[airflow] branch master updated (f3f74c7 -> 8d3acd7)

2020-05-21 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f3f74c7 Add TaskInstance state to TI Tooltip to be colour-blind friendlier (#8910) add 8d3acd7 Fix

[airflow] branch master updated (47413d9 -> 16206cd)

2020-05-21 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 47413d9 Remove singularity from CI images (#8945) add 16206cd Update example webserver_config.py to show

[GitHub] [airflow] ashb merged pull request #8944: Update example webserver_config.py to show correct CSRF config

2020-05-21 Thread GitBox
ashb merged pull request #8944: URL: https://github.com/apache/airflow/pull/8944 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated (8d3acd7 -> 47413d9)

2020-05-21 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 8d3acd7 Fix docstring in DagFileProcessor._schedule_task_instances (#8948) add 47413d9 Remove singularity from

[GitHub] [airflow] kaxil merged pull request #8949: Add note in Updating.md about the removel of DagRun.ID_PREFIX

2020-05-21 Thread GitBox
kaxil merged pull request #8949: URL: https://github.com/apache/airflow/pull/8949 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated (16206cd -> 97b6cc7)

2020-05-21 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 16206cd Update example webserver_config.py to show correct CSRF config (#8944) add 97b6cc7 Add note in

[GitHub] [airflow] ashb commented on a change in pull request #8256: updated _write_args on PythonVirtualenvOperator

2020-05-21 Thread GitBox
ashb commented on a change in pull request #8256: URL: https://github.com/apache/airflow/pull/8256#discussion_r428541035 ## File path: airflow/operators/python_operator.py ## @@ -330,13 +330,28 @@ def _write_string_args(self, filename): def _write_args(self,

[GitHub] [airflow] ashb commented on pull request #8928: Bump httplib2 from 0.17.3 to 0.18.0 in /requirements

2020-05-21 Thread GitBox
ashb commented on pull request #8928: URL: https://github.com/apache/airflow/pull/8928#issuecomment-631990670 @potiuk Does changing these files like this change which version is used in our test? If so I should add it to the "important files" list. If not then dependabot is

[GitHub] [airflow] potiuk commented on a change in pull request #8943: Python base images are stored in cache

2020-05-21 Thread GitBox
potiuk commented on a change in pull request #8943: URL: https://github.com/apache/airflow/pull/8943#discussion_r428558788 ## File path: scripts/ci/_utils.sh ## @@ -227,6 +227,9 @@ function initialize_common_environment { # Artifact name suffix for SVN packaging

[GitHub] [airflow] potiuk opened a new issue #8947: Move aws-cli out of the CI image

2020-05-21 Thread GitBox
potiuk opened a new issue #8947: URL: https://github.com/apache/airflow/issues/8947 **Description** The awsli binary can be used from the docker image https://hub.docker.com/r/amazon/aws-cli and script written to hide pulling and mounting authorisation volumes to it. **Use

[airflow] branch master updated (97b6cc7 -> 41481bb)

2020-05-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 97b6cc7 Add note in Updating.md about the removel of DagRun.ID_PREFIX (#8949) add 41481bb Python base

[GitHub] [airflow] ashb commented on a change in pull request #8256: updated _write_args on PythonVirtualenvOperator

2020-05-21 Thread GitBox
ashb commented on a change in pull request #8256: URL: https://github.com/apache/airflow/pull/8256#discussion_r428542842 ## File path: tests/operators/test_python_operator.py ## @@ -340,6 +341,80 @@ def test_echo_env_variables(self): t.run(start_date=DEFAULT_DATE,

[GitHub] [airflow] ashb commented on pull request #8910: Accessibility - Add TaskInstance State to tiTooltip in RBAC UI

2020-05-21 Thread GitBox
ashb commented on pull request #8910: URL: https://github.com/apache/airflow/pull/8910#issuecomment-631996140 Thanks @harrisjoseph (:gb: :pencil2: FTW.) This is an automated message from the Apache Git Service. To respond

[airflow] branch master updated (a9dfd7d -> f3f74c7)

2020-05-21 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from a9dfd7d Remove side-effect of session in FAB (#8940) add f3f74c7 Add TaskInstance state to TI Tooltip to be

[GitHub] [airflow] potiuk commented on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
potiuk commented on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-632003742 We can get those also used from docker images. We have now the docker socket mapped inside the docker image so we can do `docker pull google/cloud-sdk:latest` - we can even

[GitHub] [airflow] potiuk commented on issue #8785: Move out all unnecessary binaries installed in CI image

2020-05-21 Thread GitBox
potiuk commented on issue #8785: URL: https://github.com/apache/airflow/issues/8785#issuecomment-632004132 That's basicallly what I planned to do with all the binaries -> but it requires a bit of work. If anyone can pick this up - I am more than happy to review :)

[GitHub] [airflow] harrisjoseph commented on pull request #8910: Accessibility - Add TaskInstance State to tiTooltip in RBAC UI

2020-05-21 Thread GitBox
harrisjoseph commented on pull request #8910: URL: https://github.com/apache/airflow/pull/8910#issuecomment-632003904 Thank you. It's an honour and a privilege.  This is an automated message from the Apache Git Service. To

[GitHub] [airflow] potiuk opened a new issue #8946: Move Gcloud out of the CI image

2020-05-21 Thread GitBox
potiuk opened a new issue #8946: URL: https://github.com/apache/airflow/issues/8946 **Description** The gcloud binary can be used from the docker image https://hub.docker.com/r/google/cloud-sdk/ and script written to hide pulling and mounting authorisation volumes to it.

[GitHub] [airflow] casassg commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-21 Thread GitBox
casassg commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r428968991 ## File path: airflow/models/xcom_arg.py ## @@ -83,7 +83,7 @@ def __getitem__(self, item): """ Implements xcomresult['some_result_key']

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8963: SparkSubmitOperator could not get Exit Code after log stream interrupted by k8s old resource version exception

2020-05-21 Thread GitBox
boring-cyborg[bot] commented on issue #8963: URL: https://github.com/apache/airflow/issues/8963#issuecomment-632429783 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] ywan2017 opened a new issue #8963: SparkSubmitOperator could not get Exit Code after log stream interrupted by k8s old resource version exception

2020-05-21 Thread GitBox
ywan2017 opened a new issue #8963: URL: https://github.com/apache/airflow/issues/8963 # SparkSubmitOperator could not get Exit Code after log stream interrupted by k8s old resource version exception ## description I use airflow to schedule spark jobs on k8s using

[airflow] branch v1-10-test updated: Cache 1 10 ci images (#8955) (#8959)

2020-05-21 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 179fbb6 Cache 1 10 ci images (#8955)

[GitHub] [airflow] ashb commented on pull request #8904: Move Kubernetes modules out of contrib to match 2.0 layout

2020-05-21 Thread GitBox
ashb commented on pull request #8904: URL: https://github.com/apache/airflow/pull/8904#issuecomment-632373196 @dimberman Needs rebasing now please. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] ashb merged pull request #8959: Cache 1 10 ci images (#8955)

2020-05-21 Thread GitBox
ashb merged pull request #8959: URL: https://github.com/apache/airflow/pull/8959 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk commented on pull request #8957: Pin google-cloud-datacatalog to <0.8

2020-05-21 Thread GitBox
potiuk commented on pull request #8957: URL: https://github.com/apache/airflow/pull/8957#issuecomment-632332841 Needs py3.7 update as well. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] potiuk commented on pull request #8958: Add caching

2020-05-21 Thread GitBox
potiuk commented on pull request #8958: URL: https://github.com/apache/airflow/pull/8958#issuecomment-632333728 I am quite sure it's not :) This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8961: Fix the dag_id fetching for /taskinstance/list UI page

2020-05-21 Thread GitBox
boring-cyborg[bot] commented on pull request #8961: URL: https://github.com/apache/airflow/pull/8961#issuecomment-632362157 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] pyuelee opened a new pull request #8961: Fix the dag_id fetching for /taskinstance/list UI page

2020-05-21 Thread GitBox
pyuelee opened a new pull request #8961: URL: https://github.com/apache/airflow/pull/8961 --- Make sure to mark the boxes below before creating PR: [x] - [ ] Description above provides context of the change - [ ] Unit tests coverage for changes (not needed for documentation

[GitHub] [airflow] aoen commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-21 Thread GitBox
aoen commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r428954360 ## File path: airflow/operators/python.py ## @@ -145,6 +147,141 @@ def execute_callable(self): return self.python_callable(*self.op_args,

[GitHub] [airflow] casassg commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-21 Thread GitBox
casassg commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r428972097 ## File path: airflow/operators/python.py ## @@ -145,6 +147,141 @@ def execute_callable(self): return self.python_callable(*self.op_args,

[GitHub] [airflow] yuqian90 commented on pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-21 Thread GitBox
yuqian90 commented on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-632423049 Thanks. I commented about some small fixes to the doc inline. Please investigate the `continuous-integration/travis-ci/pr` failure and I'll accept once it passes.

[GitHub] [airflow] yuqian90 commented on a change in pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-21 Thread GitBox
yuqian90 commented on a change in pull request #8867: URL: https://github.com/apache/airflow/pull/8867#discussion_r428992336 ## File path: UPDATING.md ## @@ -62,6 +62,17 @@ https://developers.google.com/style/inclusive-documentation --> +### BaseSensorOperator to make

[airflow] tag nightly-master updated (2121f49 -> 94a7673)

2020-05-21 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from 2121f49 (commit) to 94a7673 (commit) from

[GitHub] [airflow] ashb commented on pull request #8958: Add caching

2020-05-21 Thread GitBox
ashb commented on pull request #8958: URL: https://github.com/apache/airflow/pull/8958#issuecomment-632332155 Not sure what's in this... But not sure it's what you wanted to do :) This is an automated message from the Apache

[GitHub] [airflow] kaxil merged pull request #8957: Pin google-cloud-datacatalog to <0.8

2020-05-21 Thread GitBox
kaxil merged pull request #8957: URL: https://github.com/apache/airflow/pull/8957 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] casassg opened a new pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-21 Thread GitBox
casassg opened a new pull request #8962: URL: https://github.com/apache/airflow/pull/8962 Airflow [AIP-31](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-31:+Airflow+functional+DAG+definition) task decorator implementation. This decorator should facilitate wrapping a function

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-21 Thread GitBox
boring-cyborg[bot] commented on pull request #8962: URL: https://github.com/apache/airflow/pull/8962#issuecomment-632362835 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] yuqian90 commented on pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-21 Thread GitBox
yuqian90 commented on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-632418891 Can you look into "continuous-integration/travis-ci/pr — The Travis CI build failed" ? The build should not be failing. Try rebase onto the latest origin master and push

[GitHub] [airflow] casassg commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-21 Thread GitBox
casassg commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r428975003 ## File path: airflow/operators/python.py ## @@ -145,6 +147,141 @@ def execute_callable(self): return self.python_callable(*self.op_args,

[GitHub] [airflow] casassg commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-21 Thread GitBox
casassg commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r428991468 ## File path: airflow/operators/python.py ## @@ -145,6 +147,141 @@ def execute_callable(self): return self.python_callable(*self.op_args,

[GitHub] [airflow] MNic commented on issue #8494: TriggerDagOperator sometimes causes database to trigger UniqueViolation constraint

2020-05-21 Thread GitBox
MNic commented on issue #8494: URL: https://github.com/apache/airflow/issues/8494#issuecomment-632441166 We're having the same issue when trying to trigger the same DAG twice in a second. We're using the CLI to trigger but it doesn't expose the `replace_microseconds` param. Would

[GitHub] [airflow] snazzyfox commented on issue #8933: pyhive is installed without Hive dependencies

2020-05-21 Thread GitBox
snazzyfox commented on issue #8933: URL: https://github.com/apache/airflow/issues/8933#issuecomment-632345025 Yep, but `pyhive` is still needed for Hive. But since airflow is installing it without hive support, it's not actually usable unless packages for hive support are installed

[GitHub] [airflow] mik-laj opened a new pull request #8960: Remove side-effect in test_get_dag_code

2020-05-21 Thread GitBox
mik-laj opened a new pull request #8960: URL: https://github.com/apache/airflow/pull/8960 To trigger a side effect, run the following command ```bash pytest tests/www/api/experimental/test_endpoints.py::TestApiExperimental_1_True::test_get_dag_code --with-db-init ``` ---

[GitHub] [airflow] kweitzner edited a comment on issue #8923: Google AdManager

2020-05-21 Thread GitBox
kweitzner edited a comment on issue #8923: URL: https://github.com/apache/airflow/issues/8923#issuecomment-632429124 Very interested in this, I built my own based off Concorde airflow plug-in, but I have low experience with airflow right now.

[GitHub] [airflow] kweitzner commented on issue #8923: Google AdManager

2020-05-21 Thread GitBox
kweitzner commented on issue #8923: URL: https://github.com/apache/airflow/issues/8923#issuecomment-632429124 Very interested in this, I built my own based off Concorde airflow plug-in, but I have low experience with airflow right.

[GitHub] [airflow] potiuk commented on pull request #8928: Bump httplib2 from 0.17.3 to 0.18.0 in /requirements

2020-05-21 Thread GitBox
potiuk commented on pull request #8928: URL: https://github.com/apache/airflow/pull/8928#issuecomment-632336230 I am afraid it only works with setup.py where we do not have open-ended conditions. We have quite a bit too complex setup trying to be both library and app,

[airflow] branch master updated (90a07d8 -> dd72040)

2020-05-21 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 90a07d8 Cache 1 10 ci images (#8955) add dd72040 Pin Version of Azure Cosmos to <4 (#8956) No new

[GitHub] [airflow] kaxil merged pull request #8956: Pin Version of Azure Cosmos to <4

2020-05-21 Thread GitBox
kaxil merged pull request #8956: URL: https://github.com/apache/airflow/pull/8956 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated (dd72040 -> 94a7673)

2020-05-21 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from dd72040 Pin Version of Azure Cosmos to <4 (#8956) add 94a7673 Pin google-cloud-datacatalog to <0.8

  1   2   >