[GitHub] [airflow] ecerulm opened a new issue #14597: Provide jinja template syntax to access connections

2021-03-03 Thread GitBox
ecerulm opened a new issue #14597: URL: https://github.com/apache/airflow/issues/14597 **Description** Expose the connection into the jinja template context via `conn..{host,port,login,password,extra_config,etc}` Today is possible to conveniently access [airflow's variables](

[GitHub] [airflow] mik-laj commented on pull request #14595: Fix functionality of "Demo Mode", refactor/simplify naming

2021-03-03 Thread GitBox
mik-laj commented on pull request #14595: URL: https://github.com/apache/airflow/pull/14595#issuecomment-790387821 > I'm interested in the genesis of this feature, but wasn't able to track down a PR in my initial searching. See: https://github.com/apache/airflow/commit/afe1b1323

[GitHub] [airflow] sfc-gh-kbregula removed a comment on pull request #14595: Fix functionality of "Demo Mode", refactor/simplify naming

2021-03-03 Thread GitBox
sfc-gh-kbregula removed a comment on pull request #14595: URL: https://github.com/apache/airflow/pull/14595#issuecomment-790385923 > I'm interested in the genesis of this feature, but wasn't able to track down a PR in my initial searching. See: https://github.com/apache/airflow/c

[GitHub] [airflow] sfc-gh-kbregula edited a comment on pull request #14595: Fix functionality of "Demo Mode", refactor/simplify naming

2021-03-03 Thread GitBox
sfc-gh-kbregula edited a comment on pull request #14595: URL: https://github.com/apache/airflow/pull/14595#issuecomment-790385923 > I'm interested in the genesis of this feature, but wasn't able to track down a PR in my initial searching. See: https://github.com/apache/airflow/co

[GitHub] [airflow] boring-cyborg[bot] commented on issue #14596: Max Active Run Feature is not Running

2021-03-03 Thread GitBox
boring-cyborg[bot] commented on issue #14596: URL: https://github.com/apache/airflow/issues/14596#issuecomment-790386452 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automate

[GitHub] [airflow] hinduja1234toko opened a new issue #14596: Max Active Run Feature is not Running

2021-03-03 Thread GitBox
hinduja1234toko opened a new issue #14596: URL: https://github.com/apache/airflow/issues/14596 We have made make **max_active_run =3** in our dag and in config file we have made **max_active_run_per_dag=10** But our code is running only on one run. Here is the configuration for the da

[GitHub] [airflow] sfc-gh-kbregula edited a comment on pull request #14595: Fix functionality of "Demo Mode", refactor/simplify naming

2021-03-03 Thread GitBox
sfc-gh-kbregula edited a comment on pull request #14595: URL: https://github.com/apache/airflow/pull/14595#issuecomment-790385923 > I'm interested in the genesis of this feature, but wasn't able to track down a PR in my initial searching. See: https://github.com/apache/airflow/co

[GitHub] [airflow] sfc-gh-kbregula commented on pull request #14595: Fix functionality of "Demo Mode", refactor/simplify naming

2021-03-03 Thread GitBox
sfc-gh-kbregula commented on pull request #14595: URL: https://github.com/apache/airflow/pull/14595#issuecomment-790385923 > I'm interested in the genesis of this feature, but wasn't able to track down a PR in my initial searching. See: https://github.com/apache/airflow/commit/afe1b1

[GitHub] [airflow] mik-laj commented on issue #14591: trigger specific task in an existing DAG run

2021-03-03 Thread GitBox
mik-laj commented on issue #14591: URL: https://github.com/apache/airflow/issues/14591#issuecomment-790372874 no. This feature is not working. you must use kubectl exec to invoke a remote command. This is an automated mess

[GitHub] [airflow] mik-laj commented on issue #14489: Add a retry with wait interval for SSH operator

2021-03-03 Thread GitBox
mik-laj commented on issue #14489: URL: https://github.com/apache/airflow/issues/14489#issuecomment-790370996 We have 2 hooks that allows us to connect to SSH server. Second hook have retry mechanism. See: https://github.com/apache/airflow/blob/3f7ebfdfe2a1fa90b0854028a5db057adacd46c1/airf

[GitHub] [airflow] auvipy commented on issue #8117: HATEOS for API

2021-03-03 Thread GitBox
auvipy commented on issue #8117: URL: https://github.com/apache/airflow/issues/8117#issuecomment-790280475 https://flask-rest-jsonapi.readthedocs.io/en/latest/ could helpto add json-api 1.0/1.1 based HATEOS REST API This is

[GitHub] [airflow] auvipy commented on issue #13591: Graceful termination does not work with apache chart

2021-03-03 Thread GitBox
auvipy commented on issue #13591: URL: https://github.com/apache/airflow/issues/13591#issuecomment-790278819 this sounds interesting https://github.com/celery/billiard/issues/273#issuecomment-734461528 This is an automated m

[GitHub] [airflow] ryanahamilton opened a new pull request #14595: Fix functionality of "Demo Mode", refactor/simplify naming

2021-03-03 Thread GitBox
ryanahamilton opened a new pull request #14595: URL: https://github.com/apache/airflow/pull/14595 "Demo Mode" wasn't functioning properly (obfuscating sensitive info). This update fixes the implementation so the Tree, Graph, and Code view blur contents as expected. - Refactored the

[GitHub] [airflow] marcosmarxm opened a new pull request #14594: Fix some small typos also TESTING.rst

2021-03-03 Thread GitBox
marcosmarxm opened a new pull request #14594: URL: https://github.com/apache/airflow/pull/14594 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for mo

[GitHub] [airflow] ianstanton commented on pull request #12441: Fix file permission issue when running git-sync in the KubernetesExecutor mode

2021-03-03 Thread GitBox
ianstanton commented on pull request #12441: URL: https://github.com/apache/airflow/pull/12441#issuecomment-790222727 @potiuk Can you give this another look? Do you think this is good to merge? This is an automated message fr

[GitHub] [airflow] ianstanton commented on a change in pull request #13209: Add support for worker persistence with KEDA v2.0.0 in helm chart

2021-03-03 Thread GitBox
ianstanton commented on a change in pull request #13209: URL: https://github.com/apache/airflow/pull/13209#discussion_r586936226 ## File path: chart/README.md ## @@ -323,22 +323,30 @@ helm install --name my-release \ KEDA stands for Kubernetes Event Driven Autoscaling. [KED

[GitHub] [airflow] github-actions[bot] commented on pull request #14144: Deactivate trigger, refresh, and delete controls on dag detail view.

2021-03-03 Thread GitBox
github-actions[bot] commented on pull request #14144: URL: https://github.com/apache/airflow/pull/14144#issuecomment-790206810 [The Workflow run](https://github.com/apache/airflow/actions/runs/619439234) is cancelling this PR. Building image for the PR has been cancelled -

[GitHub] [airflow] github-actions[bot] commented on pull request #14144: Deactivate trigger, refresh, and delete controls on dag detail view.

2021-03-03 Thread GitBox
github-actions[bot] commented on pull request #14144: URL: https://github.com/apache/airflow/pull/14144#issuecomment-790206532 [The Workflow run](https://github.com/apache/airflow/actions/runs/619473161) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build

[GitHub] [airflow] boring-cyborg[bot] commented on issue #14593: airflow2.0 - Run the task with the specified user will report an error

2021-03-03 Thread GitBox
boring-cyborg[bot] commented on issue #14593: URL: https://github.com/apache/airflow/issues/14593#issuecomment-790205612 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automate

[GitHub] [airflow] zqd759592942 opened a new issue #14593: airflow2.0 - Run the task with the specified user will report an error

2021-03-03 Thread GitBox
zqd759592942 opened a new issue #14593: URL: https://github.com/apache/airflow/issues/14593 Use the specified user to run the task, and an error will be reported when the scheduler log path('/bigdata/airflow/logs/scheduler/2021-03-01') is not created the next day. -

[GitHub] [airflow] github-actions[bot] commented on pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
github-actions[bot] commented on pull request #14492: URL: https://github.com/apache/airflow/pull/14492#issuecomment-790205365 [The Workflow run](https://github.com/apache/airflow/actions/runs/619447959) is cancelling this PR. Building images for the PR has failed. Follow the the workflow

[GitHub] [airflow] marcosmarxm commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
marcosmarxm commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586934907 ## File path: airflow/providers/airbyte/operators/airbyte.py ## @@ -0,0 +1,77 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

[GitHub] [airflow] Gaurang033 edited a comment on issue #14489: Add a retry with wait interval for SSH operator

2021-03-03 Thread GitBox
Gaurang033 edited a comment on issue #14489: URL: https://github.com/apache/airflow/issues/14489#issuecomment-790204665 @mik-laj can you elaborate on the generic hook thing? I was thinking to modify the existing hook and add an option for this. --

[GitHub] [airflow] Gaurang033 commented on issue #14489: Add a retry with wait interval for SSH operator

2021-03-03 Thread GitBox
Gaurang033 commented on issue #14489: URL: https://github.com/apache/airflow/issues/14489#issuecomment-790204665 @Gaurang033 can you elaborate on the generic hook thing? I was thinking to modify the existing hook and add an option for this. --

[GitHub] [airflow] marcosmarxm commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
marcosmarxm commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586933619 ## File path: airflow/providers/airbyte/operators/airbyte.py ## @@ -0,0 +1,83 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

[GitHub] [airflow] marcosmarxm commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
marcosmarxm commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586933619 ## File path: airflow/providers/airbyte/operators/airbyte.py ## @@ -0,0 +1,83 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

[GitHub] [airflow] marcosmarxm commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
marcosmarxm commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586932770 ## File path: docs/apache-airflow-providers-airbyte/operators/airbyte.rst ## @@ -0,0 +1,39 @@ + .. Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] marcosmarxm edited a comment on pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
marcosmarxm edited a comment on pull request #14492: URL: https://github.com/apache/airflow/pull/14492#issuecomment-790202277 > Also I think it's important to add example dag & documentation about the provider itself. Maybe some information about setting up the connection like: > https:

[GitHub] [airflow] marcosmarxm commented on pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
marcosmarxm commented on pull request #14492: URL: https://github.com/apache/airflow/pull/14492#issuecomment-790202277 > Also I think it's important to add example dag & documentation about the provider itself. Maybe some information about setting up the connection like: > https://airfl

[airflow] branch master updated (8ef862e -> 3f7ebfd)

2021-03-03 Thread ryanahamilton
This is an automated email from the ASF dual-hosted git repository. ryanahamilton pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 8ef862e Doc: Fix typo in Jira Providers link (#14585) add 3f7ebfd fix ui bugs in tree view (#14566) N

[GitHub] [airflow] ryanahamilton closed issue #14473: DagRun duration not visible in tree view tooltip if not currently running

2021-03-03 Thread GitBox
ryanahamilton closed issue #14473: URL: https://github.com/apache/airflow/issues/14473 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] ryanahamilton merged pull request #14566: Fix ui bugs in tree view

2021-03-03 Thread GitBox
ryanahamilton merged pull request #14566: URL: https://github.com/apache/airflow/pull/14566 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [airflow] ryanahamilton closed issue #14486: tree view task instances have too much left padding in webserver UI

2021-03-03 Thread GitBox
ryanahamilton closed issue #14486: URL: https://github.com/apache/airflow/issues/14486 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] github-actions[bot] commented on pull request #14566: Fix ui bugs in tree view

2021-03-03 Thread GitBox
github-actions[bot] commented on pull request #14566: URL: https://github.com/apache/airflow/pull/14566#issuecomment-790165245 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14500: Clear tasks by task ids in REST API

2021-03-03 Thread GitBox
ephraimbuddy commented on a change in pull request #14500: URL: https://github.com/apache/airflow/pull/14500#discussion_r586901576 ## File path: airflow/api_connexion/openapi/v1.yaml ## @@ -2218,6 +2218,13 @@ components: type: boolean default: true +

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14588: Fill DagBag from dag_folder setting for upgrade rules

2021-03-03 Thread GitBox
ephraimbuddy commented on a change in pull request #14588: URL: https://github.com/apache/airflow/pull/14588#discussion_r586892236 ## File path: airflow/upgrade/rules/custom_operator_metaclass_rule.py ## @@ -17,6 +17,7 @@ from __future__ import absolute_import +from airflo

[GitHub] [airflow] john-jac opened a new issue #14592: Unreachable Secrets Backend Causes Web Server Crash

2021-03-03 Thread GitBox
john-jac opened a new issue #14592: URL: https://github.com/apache/airflow/issues/14592 **Apache Airflow version**: 1.10.12 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): n/a **Environment**: - **Cloud provider or hardware con

[GitHub] [airflow] ToxaZ commented on issue #9373: path to file as sql parameter in PostgresOperator not working anymore

2021-03-03 Thread GitBox
ToxaZ commented on issue #9373: URL: https://github.com/apache/airflow/issues/9373#issuecomment-790112337 @eladkal which version you have tested? I'm still getting an error on 1.10.10, 1.10.12, 1.10.14 This is an automated m

[GitHub] [airflow] ConstantinoSchillebeeckx commented on issue #14591: trigger specific task in an existing DAG run

2021-03-03 Thread GitBox
ConstantinoSchillebeeckx commented on issue #14591: URL: https://github.com/apache/airflow/issues/14591#issuecomment-790088177 We would be triggering tasks on a deployed QA environment; is it possible to use the Airflow CLI to interact with a remote deployment? I see the [Docs hints at th

[GitHub] [airflow] mik-laj edited a comment on issue #14591: trigger specific task in an existing DAG run

2021-03-03 Thread GitBox
mik-laj edited a comment on issue #14591: URL: https://github.com/apache/airflow/issues/14591#issuecomment-790082269 For development, you can consider to use docker-compose and airflow tasks test command. This is an automat

[GitHub] [airflow] mik-laj commented on issue #14591: trigger specific task in an existing DAG run

2021-03-03 Thread GitBox
mik-laj commented on issue #14591: URL: https://github.com/apache/airflow/issues/14591#issuecomment-790082269 For development, you can consider to use docker-compose and airflow tasks tests command. This is an automated mes

[GitHub] [airflow] ConstantinoSchillebeeckx opened a new issue #14591: trigger specific task in an existing DAG run

2021-03-03 Thread GitBox
ConstantinoSchillebeeckx opened a new issue #14591: URL: https://github.com/apache/airflow/issues/14591 **Description** I may be missing it, but I couldn't find a way of trigger an individual task to run within an existing DAG run through the REST API. **Use ca

[GitHub] [airflow] potiuk commented on pull request #14531: Running tests in parallel for self-hosted runners

2021-03-03 Thread GitBox
potiuk commented on pull request #14531: URL: https://github.com/apache/airflow/pull/14531#issuecomment-790050472 > Not once in that comment you linked to did you say that you tested it on AWS. The whole comment was after the PR failed. I is exclusively about the tests that are fail

[GitHub] [airflow] bparhy opened a new issue #14590: Tasks getting Queued but not running

2021-03-03 Thread GitBox
bparhy opened a new issue #14590: URL: https://github.com/apache/airflow/issues/14590 **Apache Airflow version**: 1.10.14 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`):v1.20.2 **Environment**: airflow on k8s - **Cloud p

[GitHub] [airflow] boring-cyborg[bot] commented on issue #14590: Tasks getting Queued but not running

2021-03-03 Thread GitBox
boring-cyborg[bot] commented on issue #14590: URL: https://github.com/apache/airflow/issues/14590#issuecomment-790025052 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automate

[GitHub] [airflow] jmcarp opened a new pull request #14589: Drop third-party lazy proxy.

2021-03-03 Thread GitBox
jmcarp opened a new pull request #14589: URL: https://github.com/apache/airflow/pull/14589 We already get a local object proxy from werkzeug, and it's trivial to add memoization with the standard library, so we can drop the third-party dependency without losing functionality. This patc

[GitHub] [airflow] github-actions[bot] commented on pull request #14581: Ensure last_scheduler_run is updated in DB

2021-03-03 Thread GitBox
github-actions[bot] commented on pull request #14581: URL: https://github.com/apache/airflow/pull/14581#issuecomment-790017739 [The Workflow run](https://github.com/apache/airflow/actions/runs/618706229) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build

[GitHub] [airflow] github-actions[bot] commented on pull request #14581: Ensure last_scheduler_run is updated in DB

2021-03-03 Thread GitBox
github-actions[bot] commented on pull request #14581: URL: https://github.com/apache/airflow/pull/14581#issuecomment-790017258 [The Workflow run](https://github.com/apache/airflow/actions/runs/618692618) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #14588: Rules

2021-03-03 Thread GitBox
boring-cyborg[bot] commented on pull request #14588: URL: https://github.com/apache/airflow/pull/14588#issuecomment-790008267 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Con

[GitHub] [airflow] kenfinster opened a new pull request #14588: Rules

2021-03-03 Thread GitBox
kenfinster opened a new pull request #14588: URL: https://github.com/apache/airflow/pull/14588 For some reason: BaseOperatorMetaclassRule NoAdditionalArgsInOperatorsRule Don't consider the dag folder set by the user. This causes issues when the user needs to set the dag folder with

[GitHub] [airflow] jbampton opened a new pull request #14587: Add pre-commit check to sort alpha and uniquify hooks in STATIC_CODE_…

2021-03-03 Thread GitBox
jbampton opened a new pull request #14587: URL: https://github.com/apache/airflow/pull/14587 …CHECKS.rst --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)**

[GitHub] [airflow] ianlofs edited a comment on issue #14364: Missing schedule_delay metric

2021-03-03 Thread GitBox
ianlofs edited a comment on issue #14364: URL: https://github.com/apache/airflow/issues/14364#issuecomment-789984552 Found this because I was doing a deep dive through this recently because I am currently trying to figure out the best way to monitor for DAGs that never start. This is where

[GitHub] [airflow] ianlofs edited a comment on issue #14364: Missing schedule_delay metric

2021-03-03 Thread GitBox
ianlofs edited a comment on issue #14364: URL: https://github.com/apache/airflow/issues/14364#issuecomment-789984552 Found this because I was doing a deep dive through this recently because I am currently trying to figure out the best way to monitor for DAGs that never start. This is where

[GitHub] [airflow] ianlofs commented on issue #14364: Missing schedule_delay metric

2021-03-03 Thread GitBox
ianlofs commented on issue #14364: URL: https://github.com/apache/airflow/issues/14364#issuecomment-789984552 Found this because I was doing a deep five through this recently because I am currently trying to figure out the best way to monitor for DAGs that never start. This is where this i

[GitHub] [airflow] arbass22 commented on a change in pull request #12158: StatsD metrics have tags + generate reference docs

2021-03-03 Thread GitBox
arbass22 commented on a change in pull request #12158: URL: https://github.com/apache/airflow/pull/12158#discussion_r586697071 ## File path: airflow/models/taskinstance.py ## @@ -1255,11 +1258,13 @@ def signal_handler(signum, frame): # pylint: disable=unused-argument

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14501: Fix set task instance form test

2021-03-03 Thread GitBox
ephraimbuddy commented on a change in pull request #14501: URL: https://github.com/apache/airflow/pull/14501#discussion_r586684031 ## File path: tests/api_connexion/schemas/test_task_instance_schema.py ## @@ -199,7 +199,6 @@ def test_success(self): @parameterized.expand(

[GitHub] [airflow] TheRum commented on pull request #14418: Fix #14417 : Failing to submit Druid ingestion task

2021-03-03 Thread GitBox
TheRum commented on pull request #14418: URL: https://github.com/apache/airflow/pull/14418#issuecomment-789947742 @xinbinhuang Yes, I want to add the tests but not sure how to start. I have no idea. Please guide me. This is

[GitHub] [airflow] github-actions[bot] commented on pull request #14579: Restore correct terminal with to interactive breeze usage

2021-03-03 Thread GitBox
github-actions[bot] commented on pull request #14579: URL: https://github.com/apache/airflow/pull/14579#issuecomment-789947105 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] xinbinhuang commented on pull request #14418: Fix #14417 : Failing to submit Druid ingestion task

2021-03-03 Thread GitBox
xinbinhuang commented on pull request #14418: URL: https://github.com/apache/airflow/pull/14418#issuecomment-789941418 Hi @TheRum, just want to check in to see how's the progress. Are you available for adding the tests? Let me know if I can help with anything -

[GitHub] [airflow] boring-cyborg[bot] commented on issue #14586: AttributeError: 'DatabaseBackend' object has no attribute 'task_cls'

2021-03-03 Thread GitBox
boring-cyborg[bot] commented on issue #14586: URL: https://github.com/apache/airflow/issues/14586#issuecomment-789933104 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automate

[GitHub] [airflow] nicolasdonoso opened a new issue #14586: AttributeError: 'DatabaseBackend' object has no attribute 'task_cls'

2021-03-03 Thread GitBox
nicolasdonoso opened a new issue #14586: URL: https://github.com/apache/airflow/issues/14586 **Apache Airflow version**: 2.0.1 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): 1.18.15 **Environment**: - **Cloud provider o

[GitHub] [airflow] xinbinhuang commented on issue #14573: DockerOperator: OCI Runtime create failed: exec: "PYTHONPATH=.": executable file not found in $PATH: unknown

2021-03-03 Thread GitBox
xinbinhuang commented on issue #14573: URL: https://github.com/apache/airflow/issues/14573#issuecomment-789908559 Does below work? The `command` should be the same as what you pass in `... -v /xyz:/xyz -it ` using normal docker command ```python t = DockerOperator(

[GitHub] [airflow] antontimenko edited a comment on issue #14205: Scheduler "deadlocks" itself when max_active_runs_per_dag is reached by up_for_retry tasks

2021-03-03 Thread GitBox
antontimenko edited a comment on issue #14205: URL: https://github.com/apache/airflow/issues/14205#issuecomment-789903499 @zachliu this is an important thing for me, because in production setup we use Airflow sometimes to run lot's of "jobs" (DAG runs) with different configs. For example w

[GitHub] [airflow] antontimenko commented on issue #14205: Scheduler "deadlocks" itself when max_active_runs_per_dag is reached by up_for_retry tasks

2021-03-03 Thread GitBox
antontimenko commented on issue #14205: URL: https://github.com/apache/airflow/issues/14205#issuecomment-789903499 @zachliu this is an important thing for me, because in production setup we use Airflow sometimes to run lot's of "jobs" (DAG runs) with different configs. For example we somet

[GitHub] [airflow] xinbinhuang edited a comment on issue #14580: Request for sub process

2021-03-03 Thread GitBox
xinbinhuang edited a comment on issue #14580: URL: https://github.com/apache/airflow/issues/14580#issuecomment-789898758 You can use the `wait_for_completion` parameter for the `TriggerDagRunOperator` ```python test_trigger_dagrun = TriggerDagRunOperator( dag=dag,

[GitHub] [airflow] xinbinhuang commented on issue #14580: Request for sub process

2021-03-03 Thread GitBox
xinbinhuang commented on issue #14580: URL: https://github.com/apache/airflow/issues/14580#issuecomment-789898758 You can use the `wait_for_completion` parameter for the `TriggerDagRunOperator` (by default, it's `False`) ```python test_trigger_dagrun = TriggerDagRunOperator(

[GitHub] [airflow] dstandish commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
dstandish commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586538602 ## File path: airflow/providers/airbyte/operators/airbyte.py ## @@ -0,0 +1,83 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# o

[GitHub] [airflow] xinbinhuang edited a comment on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)

2021-03-03 Thread GitBox
xinbinhuang edited a comment on issue #14561: URL: https://github.com/apache/airflow/issues/14561#issuecomment-789324241 Can you share a bit more details on how you start the webserver and docker? It seems to be missing the `snowflake` dependencies(i.e. `snowflake-connector-python` package

[GitHub] [airflow] xinbinhuang edited a comment on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)

2021-03-03 Thread GitBox
xinbinhuang edited a comment on issue #14561: URL: https://github.com/apache/airflow/issues/14561#issuecomment-789895391 I tried to reproduce on my end but I didn't get any errors. ```bash # outside docker container docker run -it --rm -v "${PWD}"/dags:/opt/airflow/dags apache/a

[GitHub] [airflow] github-actions[bot] commented on pull request #14521: Add Asana Provider

2021-03-03 Thread GitBox
github-actions[bot] commented on pull request #14521: URL: https://github.com/apache/airflow/pull/14521#issuecomment-789895668 [The Workflow run](https://github.com/apache/airflow/actions/runs/617730632) is cancelling this PR. Building images for the PR has failed. Follow the the workflow

[GitHub] [airflow] xinbinhuang commented on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)

2021-03-03 Thread GitBox
xinbinhuang commented on issue #14561: URL: https://github.com/apache/airflow/issues/14561#issuecomment-789895391 I tried to reproduce on my end but I didn't get any errors. ```bash docker run -it --rm -v "${PWD}"/dags:/opt/airflow/dags apache/airflow:2.0.1 bash pip install apa

[airflow] branch master updated (3ba5f50 -> 8ef862e)

2021-03-03 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 3ba5f50 Fix grammar/typo in concepts.rst (#14582) add 8ef862e Doc: Fix typo in Jira Providers link (#14585

[GitHub] [airflow] kaxil merged pull request #14585: Fix typo on Jira link

2021-03-03 Thread GitBox
kaxil merged pull request #14585: URL: https://github.com/apache/airflow/pull/14585 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #14585: Fix typo on Jira link

2021-03-03 Thread GitBox
boring-cyborg[bot] commented on pull request #14585: URL: https://github.com/apache/airflow/pull/14585#issuecomment-789890333 Awesome work, congrats on your first merged pull request! This is an automated message from the

[GitHub] [airflow] dstandish commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
dstandish commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586538602 ## File path: airflow/providers/airbyte/operators/airbyte.py ## @@ -0,0 +1,83 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# o

[GitHub] [airflow] bbovenzi commented on pull request #14566: Fix ui bugs in tree view

2021-03-03 Thread GitBox
bbovenzi commented on pull request #14566: URL: https://github.com/apache/airflow/pull/14566#issuecomment-789873595 Before: https://user-images.githubusercontent.com/4600967/109841387-9bc99d80-7c0e-11eb-9030-60667099a5c0.png";> After: https://user-images.githubusercontent.co

[airflow] branch v2-0-test updated (6941434 -> 178dde3)

2021-03-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git. discard 6941434 By default PIP will install all packages in .local folder (#14125) new 178dde3 By default PIP wil

[airflow] 01/01: By default PIP will install all packages in .local folder (#14125)

2021-03-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 178dde3aa920365b6a482fc5261f365aaec3f9a5 Author: Jarek Potiuk AuthorDate: Sun Feb 21 21:09:13 2021 +0100 By def

[GitHub] [airflow] potiuk commented on a change in pull request #14531: Running tests in parallel for self-hosted runners

2021-03-03 Thread GitBox
potiuk commented on a change in pull request #14531: URL: https://github.com/apache/airflow/pull/14531#discussion_r586588251 ## File path: scripts/ci/testing/ci_run_airflow_testing.sh ## @@ -170,5 +161,282 @@ do echo "**

[GitHub] [airflow] potiuk commented on a change in pull request #14531: Running tests in parallel for self-hosted runners

2021-03-03 Thread GitBox
potiuk commented on a change in pull request #14531: URL: https://github.com/apache/airflow/pull/14531#discussion_r586587023 ## File path: .github/workflows/ci.yml ## @@ -35,6 +35,8 @@ env: DB_RESET: "true" VERBOSE: "true" DOCKER_CACHE: "pulled" + # Do not show all ou

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #14585: Fix typo on Jira link

2021-03-03 Thread GitBox
boring-cyborg[bot] commented on pull request #14585: URL: https://github.com/apache/airflow/pull/14585#issuecomment-789868854 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Con

[GitHub] [airflow] andrewelkins opened a new pull request #14585: Fix typo on Jira link

2021-03-03 Thread GitBox
andrewelkins opened a new pull request #14585: URL: https://github.com/apache/airflow/pull/14585 Typo change only. No additional tests needed. This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [airflow] potiuk commented on a change in pull request #14531: Running tests in parallel for self-hosted runners

2021-03-03 Thread GitBox
potiuk commented on a change in pull request #14531: URL: https://github.com/apache/airflow/pull/14531#discussion_r586582496 ## File path: scripts/ci/testing/ci_run_airflow_testing.sh ## @@ -170,5 +161,282 @@ do echo "**

[GitHub] [airflow] boring-cyborg[bot] commented on issue #14584: Externally triggered DAG is marked as 'success'  when 'start_date' is set to timezone.utcnow() and passed in a DAG level params.

2021-03-03 Thread GitBox
boring-cyborg[bot] commented on issue #14584: URL: https://github.com/apache/airflow/issues/14584#issuecomment-789850777 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automate

[GitHub] [airflow] PrarthiJain opened a new issue #14584: Externally triggered DAG is marked as 'success'  when 'start_date' is set to timezone.utcnow() and passed in a DAG level params.

2021-03-03 Thread GitBox
PrarthiJain opened a new issue #14584: URL: https://github.com/apache/airflow/issues/14584 **Apache Airflow version**: **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): **Environment**: - **Cloud provider or hardware config

[GitHub] [airflow] zachliu commented on issue #14205: Scheduler "deadlocks" itself when max_active_runs_per_dag is reached by up_for_retry tasks

2021-03-03 Thread GitBox
zachliu commented on issue #14205: URL: https://github.com/apache/airflow/issues/14205#issuecomment-789850566 @antontimenko ah i see, i have `max_active_runs_per_dag = 16`. and i rarely reach `16` task runs per dag. but when https://github.com/apache/airflow/issues/9975 occurs, the `up_for

[GitHub] [airflow] dstandish commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
dstandish commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586565706 ## File path: airflow/providers/airbyte/operators/airbyte.py ## @@ -0,0 +1,83 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# o

[GitHub] [airflow] marcosmarxm commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
marcosmarxm commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586564036 ## File path: airflow/providers/airbyte/operators/airbyte.py ## @@ -0,0 +1,83 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

[GitHub] [airflow] ashb commented on pull request #14579: Restore correct terminal with to interactive breeze usage

2021-03-03 Thread GitBox
ashb commented on pull request #14579: URL: https://github.com/apache/airflow/pull/14579#issuecomment-789840792 Just updating brew on my mac laptop to test out a fix. This is an automated message from the Apache Git Service.

[GitHub] [airflow] boring-cyborg[bot] commented on issue #14583: Document Airflow Metrics Optimal Values

2021-03-03 Thread GitBox
boring-cyborg[bot] commented on issue #14583: URL: https://github.com/apache/airflow/issues/14583#issuecomment-789828611 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automate

[GitHub] [airflow] john-jac opened a new issue #14583: Document Airflow Metrics Optimal Values

2021-03-03 Thread GitBox
john-jac opened a new issue #14583: URL: https://github.com/apache/airflow/issues/14583 **Description** Users of Airflow appreciate the [metrics](https://airflow.apache.org/docs/apache-airflow/stable/logging-monitoring/metrics.html) that Airflow emits, however are often confused as

[GitHub] [airflow] dstandish commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
dstandish commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586538602 ## File path: airflow/providers/airbyte/operators/airbyte.py ## @@ -0,0 +1,83 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# o

[GitHub] [airflow] marcosmarxm commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
marcosmarxm commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586535877 ## File path: airflow/providers/airbyte/hooks/airbyte.py ## @@ -0,0 +1,99 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] ashb commented on a change in pull request #14531: Running tests in parallel for self-hosted runners

2021-03-03 Thread GitBox
ashb commented on a change in pull request #14531: URL: https://github.com/apache/airflow/pull/14531#discussion_r586410651 ## File path: scripts/ci/testing/ci_run_airflow_testing.sh ## @@ -170,5 +161,282 @@ do echo "

[GitHub] [airflow] dstandish commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
dstandish commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586523021 ## File path: airflow/providers/airbyte/sensors/airbyte.py ## @@ -0,0 +1,74 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] potiuk commented on pull request #14579: Restore correct terminal with to interactive breeze usage

2021-03-03 Thread GitBox
potiuk commented on pull request #14579: URL: https://github.com/apache/airflow/pull/14579#issuecomment-789811694 Yep. It's tricky with 'Posix-ish' level of support on MacOS This is an automated message from the Apache Git Se

[GitHub] [airflow] ashb edited a comment on pull request #14577: Use libyaml C library when available.

2021-03-03 Thread GitBox
ashb edited a comment on pull request #14577: URL: https://github.com/apache/airflow/pull/14577#issuecomment-789798312 Oh, To make new providers work with 2.0.0 I'm going to have to do ``` try: import airflow.utils.yaml as yaml except ImportError: import yaml ``

[GitHub] [airflow] ashb commented on pull request #14577: Use libyaml C library when available.

2021-03-03 Thread GitBox
ashb commented on pull request #14577: URL: https://github.com/apache/airflow/pull/14577#issuecomment-789798312 Oh, To make new providers work with 2.0.0 I'm going to have to do ``` try: import airflow.utils.yaml as yaml except ImportError: import yaml ```

[GitHub] [airflow] marcosmarxm commented on a change in pull request #14492: Adds new community provider Airbyte

2021-03-03 Thread GitBox
marcosmarxm commented on a change in pull request #14492: URL: https://github.com/apache/airflow/pull/14492#discussion_r586510009 ## File path: airflow/providers/airbyte/operators/airbyte.py ## @@ -0,0 +1,83 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

[airflow] branch master updated (b994585 -> 3ba5f50)

2021-03-03 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from b994585 Replace deprecated postgresql chart location with bitnami/postgresql (#13928) add 3ba5f50 Fix gra

  1   2   3   >