[GitHub] [airflow] rchangj commented on issue #17558: New job fail immediately after start, no logs

2021-08-12 Thread GitBox
rchangj commented on issue #17558: URL: https://github.com/apache/airflow/issues/17558#issuecomment-898195873 Hi, I updated to the latest airflow 2.1.2. I got this error: Python version: 3.7.10 Airflow version: 2.1.2 Node: ip-10-77-52-33.vpc.internal

[GitHub] [airflow] mik-laj commented on pull request #17591: Use gunicorn to serve logs generated by worker

2021-08-12 Thread GitBox
mik-laj commented on pull request #17591: URL: https://github.com/apache/airflow/pull/17591#issuecomment-898119170 @potiuk @kaxil Updated PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] rchangj commented on issue #17558: New job fail immediately after start, no logs

2021-08-12 Thread GitBox
rchangj commented on issue #17558: URL: https://github.com/apache/airflow/issues/17558#issuecomment-898201909 Providers info: apache-airflow-providers-amazon==2.1.0 apache-airflow-providers-apache-hdfs==2.0.0 apache-airflow-providers-ftp==2.0.0

[airflow] branch constraints-2-1 updated: Updating constraints. Build id:1112181300

2021-08-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-1 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-1 by this push: new d2c6feb Updating

[GitHub] [airflow] rchangj removed a comment on issue #17558: New job fail immediately after start, no logs

2021-08-12 Thread GitBox
rchangj removed a comment on issue #17558: URL: https://github.com/apache/airflow/issues/17558#issuecomment-898195873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] ferruzzi commented on a change in pull request #16571: Implemented Basic EKS Integration

2021-08-12 Thread GitBox
ferruzzi commented on a change in pull request #16571: URL: https://github.com/apache/airflow/pull/16571#discussion_r688203241 ## File path: airflow/providers/amazon/aws/hooks/eks.py ## @@ -0,0 +1,420 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] roestzwiee edited a comment on issue #15340: helm install airflow in namespace get error: File "", line 32, in TimeoutError: There are still unapplied migrations af

2021-08-12 Thread GitBox
roestzwiee edited a comment on issue #15340: URL: https://github.com/apache/airflow/issues/15340#issuecomment-897729319 tldr; @autarchprinceps I'm having the same trouble here. When deploying the chart to our local machines all runs fine. But deploying the chart to our cluster

[GitHub] [airflow] duc-luong-tnsl commented on issue #17568: Airflow all scheduler crashed and cannot restore because of incorrect Group name

2021-08-12 Thread GitBox
duc-luong-tnsl commented on issue #17568: URL: https://github.com/apache/airflow/issues/17568#issuecomment-897642286 Thank you @potiuk , I would like to contribute to Airflow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on pull request #17578: Improve validation of Group id

2021-08-12 Thread GitBox
potiuk commented on pull request #17578: URL: https://github.com/apache/airflow/pull/17578#issuecomment-897668062 Should be good to go - I quarantined the "mark_success_no_kill" now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] ephraimbuddy opened a new pull request #17581: Remove the use of multiprocessing in TestLocalTaskJob and Improve Tests

2021-08-12 Thread GitBox
ephraimbuddy opened a new pull request #17581: URL: https://github.com/apache/airflow/pull/17581 This PR removes the use of multiprocessing in TestLocalTaskJob and improves the test to be more reliable --- **^ Add meaningful description above** Read the **[Pull

[GitHub] [airflow] ephraimbuddy commented on pull request #17580: Quarantine test_mark_success_no_kill test

2021-08-12 Thread GitBox
ephraimbuddy commented on pull request #17580: URL: https://github.com/apache/airflow/pull/17580#issuecomment-897708820 Hi @potiuk, I have been working on these tests and I think we should try my solution before we quarantine樂 -- This is an automated message from the Apache Git Service.

[GitHub] [airflow] saimon46 opened a new issue #17582: DAG trigger with config params visualisation with proper deserialisation

2021-08-12 Thread GitBox
saimon46 opened a new issue #17582: URL: https://github.com/apache/airflow/issues/17582 **Description** Add a proper `params` configuration in the trigger page of a DAG showing the dict without the serialisation format `__var` and `__type` but with a flat, normal dict.

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17582: DAG trigger with config params visualisation with proper deserialisation

2021-08-12 Thread GitBox
boring-cyborg[bot] commented on issue #17582: URL: https://github.com/apache/airflow/issues/17582#issuecomment-897722963 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] eladkal commented on pull request #17576: Add pre/post execution hooks

2021-08-12 Thread GitBox
eladkal commented on pull request #17576: URL: https://github.com/apache/airflow/pull/17576#issuecomment-897731012 The discussion in the issue is pretty extensive. I want to ask/offer another point of view on this functionality. We have `VerticaToMySqlOperator` - This operator has pre &

[GitHub] [airflow] potiuk closed issue #17571: Collapsing doc section is not working

2021-08-12 Thread GitBox
potiuk closed issue #17571: URL: https://github.com/apache/airflow/issues/17571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #17571: Collapsing doc section is not working

2021-08-12 Thread GitBox
potiuk commented on issue #17571: URL: https://github.com/apache/airflow/issues/17571#issuecomment-897746081 Well. It is as intended. Click on other section to expand it or to navigate out and it will be gone. The "exxpansion" indicates really the document section you are in so even if

[GitHub] [airflow] uranusjr commented on pull request #16352: AIP-39: ``DagRun.data_interval_start`` and ``data_interval_end``

2021-08-12 Thread GitBox
uranusjr commented on pull request #16352: URL: https://github.com/apache/airflow/pull/16352#issuecomment-897654921 Yeah they are failing on `main` as well, I think we recently changed something that caused a SQL incompatibility in the Python 3.6 + MySQL combination. I plan to take a look

[GitHub] [airflow] kaxil merged pull request #17567: Path correction in docs for airflow core

2021-08-12 Thread GitBox
kaxil merged pull request #17567: URL: https://github.com/apache/airflow/pull/17567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #17567: Path correction in docs for airflow core

2021-08-12 Thread GitBox
boring-cyborg[bot] commented on pull request #17567: URL: https://github.com/apache/airflow/pull/17567#issuecomment-897656715 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log

[airflow] branch main updated: AIP-39: DagRun.data_interval_start|end (#16352)

2021-08-12 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4556828 AIP-39: DagRun.data_interval_start|end

[GitHub] [airflow] uranusjr merged pull request #16352: AIP-39: ``DagRun.data_interval_start`` and ``data_interval_end``

2021-08-12 Thread GitBox
uranusjr merged pull request #16352: URL: https://github.com/apache/airflow/pull/16352 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] uranusjr closed issue #16372: Update template context to deprecate `execution_date` and add new "interval" properties

2021-08-12 Thread GitBox
uranusjr closed issue #16372: URL: https://github.com/apache/airflow/issues/16372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] uranusjr closed issue #16304: Add data_interval_start and data_interval_end columns to DagRun table, and set them when creating DagRuns.

2021-08-12 Thread GitBox
uranusjr closed issue #16304: URL: https://github.com/apache/airflow/issues/16304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ashb commented on pull request #16352: AIP-39: ``DagRun.data_interval_start`` and ``data_interval_end``

2021-08-12 Thread GitBox
ashb commented on pull request #16352: URL: https://github.com/apache/airflow/pull/16352#issuecomment-897678067  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] luoyuliuyin commented on issue #17535: the filter condition of taskinstance does not work

2021-08-12 Thread GitBox
luoyuliuyin commented on issue #17535: URL: https://github.com/apache/airflow/issues/17535#issuecomment-897730415 > > > Okay, I have drafted a PR. > > Your changes are looking good. Care to open a full PR to review? Okay, the PR address is:

[GitHub] [airflow] potiuk commented on issue #17546: ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: cannot allocate memory in static TLS block

2021-08-12 Thread GitBox
potiuk commented on issue #17546: URL: https://github.com/apache/airflow/issues/17546#issuecomment-897731840 Hmm. Then I guess the bug is in one of the libraries added: `gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4

[GitHub] [airflow] ashb opened a new pull request #17577: Have the dag_maker fixture (optionally) give SerializedDAGs

2021-08-12 Thread GitBox
ashb opened a new pull request #17577: URL: https://github.com/apache/airflow/pull/17577 All but one test in test_scheduler_job.py wants to operate on serialized dags, so it makes sense to have this be done in the dag_maker for us, to make each test "smaller". ---

[GitHub] [airflow] github-actions[bot] commented on pull request #17577: Have the dag_maker fixture (optionally) give SerializedDAGs

2021-08-12 Thread GitBox
github-actions[bot] commented on pull request #17577: URL: https://github.com/apache/airflow/pull/17577#issuecomment-897623173 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] kaxil commented on a change in pull request #17577: Have the dag_maker fixture (optionally) give SerializedDAGs

2021-08-12 Thread GitBox
kaxil commented on a change in pull request #17577: URL: https://github.com/apache/airflow/pull/17577#discussion_r687693111 ## File path: tests/conftest.py ## @@ -451,34 +451,60 @@ def dag_maker(request): The dag_maker.create_dagrun takes the same arguments as

[GitHub] [airflow] amit-talreja commented on pull request #17567: Path correction in docs for airflow core

2021-08-12 Thread GitBox
amit-talreja commented on pull request #17567: URL: https://github.com/apache/airflow/pull/17567#issuecomment-897623958 Not to mention @kaxil . It was way small but I am looking forward to contribute as much I can. Thanks. -- This is an automated message from the Apache Git

[GitHub] [airflow] potiuk commented on issue #17568: Airflow all scheduler crashed and cannot restore because of incorrect Group name

2021-08-12 Thread GitBox
potiuk commented on issue #17568: URL: https://github.com/apache/airflow/issues/17568#issuecomment-897637467 Yep. I just added a better diagnostics in this case in #17578. This should perform the validation at an earlier stage - when the task group is parsed, rather than when the task is

[GitHub] [airflow] uranusjr edited a comment on pull request #16352: AIP-39: ``DagRun.data_interval_start`` and ``data_interval_end``

2021-08-12 Thread GitBox
uranusjr edited a comment on pull request #16352: URL: https://github.com/apache/airflow/pull/16352#issuecomment-897654921 Yeah they are failing on `main` as well, I think we recently changed something that caused a SQL incompatibility in the Python 3.8 + MySQL combination. I plan to take

[GitHub] [airflow] kaxil commented on pull request #16352: AIP-39: ``DagRun.data_interval_start`` and ``data_interval_end``

2021-08-12 Thread GitBox
kaxil commented on pull request #16352: URL: https://github.com/apache/airflow/pull/16352#issuecomment-897664981 We can probably merge this PR in that case -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] potiuk opened a new issue #17579: Quarantine test_mark_success_no_kill

2021-08-12 Thread GitBox
potiuk opened a new issue #17579: URL: https://github.com/apache/airflow/issues/17579 Happens to be flaky: https://github.com/apache/airflow/pull/17578/checks?check_run_id=3312137736#step:6:4809 ``` __ TestLocalTaskJob.test_mark_success_no_kill

[GitHub] [airflow] potiuk opened a new pull request #17580: Quarantine test_mark_success_no_kill test

2021-08-12 Thread GitBox
potiuk opened a new pull request #17580: URL: https://github.com/apache/airflow/pull/17580 This test is flaky. Logging it in: #17579 --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] ephraimbuddy commented on pull request #17581: Remove the use of multiprocessing in TestLocalTaskJob and Improve Tests

2021-08-12 Thread GitBox
ephraimbuddy commented on pull request #17581: URL: https://github.com/apache/airflow/pull/17581#issuecomment-897738711 Hmm, got a failure locally now after several runs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] potiuk commented on pull request #17545: Airflow exceptions pass-thru during "on_execute_callback"

2021-08-12 Thread GitBox
potiuk commented on pull request #17545: URL: https://github.com/apache/airflow/pull/17545#issuecomment-897519700 So we went very far from throwing exception in a method :). I think we are talking about two different things and we should split this diacusion off to a short term generic

[GitHub] [airflow] timothyclarke opened a new issue #17574: [BUG] Helm chart does not correctly support sslmode for PostgreSQL connection

2021-08-12 Thread GitBox
timothyclarke opened a new issue #17574: URL: https://github.com/apache/airflow/issues/17574 **Apache Airflow version**: 2.12 **Apache Airflow Provider versions** : not applicable - none / standard / unknown (delete as applicable). I am installing the helm chart from this repo

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17574: [BUG] Helm chart does not correctly support sslmode for PostgreSQL connection

2021-08-12 Thread GitBox
boring-cyborg[bot] commented on issue #17574: URL: https://github.com/apache/airflow/issues/17574#issuecomment-897557382 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] github-actions[bot] commented on pull request #17567: Path correction in docs for airflow core

2021-08-12 Thread GitBox
github-actions[bot] commented on pull request #17567: URL: https://github.com/apache/airflow/pull/17567#issuecomment-897601790 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide

[GitHub] [airflow] duc-luong-tnsl commented on issue #17568: Airflow all scheduler crashed and cannot restore because of incorrect Group name

2021-08-12 Thread GitBox
duc-luong-tnsl commented on issue #17568: URL: https://github.com/apache/airflow/issues/17568#issuecomment-897621824 I know how to fix it. But this exception makes the scheduler crashed -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] timothyclarke commented on issue #16013: CLI 'kubernetes cleanup-pods' fails on invalid label key

2021-08-12 Thread GitBox
timothyclarke commented on issue #16013: URL: https://github.com/apache/airflow/issues/16013#issuecomment-897626892 @XD-DENG When is `2.1.3` going to be released ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] potiuk opened a new pull request #17578: Improve validation of Group id

2021-08-12 Thread GitBox
potiuk opened a new pull request #17578: URL: https://github.com/apache/airflow/pull/17578 When Group id of task group is used to prefix task id, it should follow the same limitation that task_id has, plus it should not have '.'. The '.' is used to separate groups in task id so it

[airflow] branch main updated (944dc32 -> 951006e)

2021-08-12 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 944dc32 Fix Invalid log order in ElasticsearchTaskHandler (#17551) add 951006e Path correction in docs for

[GitHub] [airflow-site] ryanahamilton closed pull request #464: Remove temporary banner promoting Airflow Summit

2021-08-12 Thread GitBox
ryanahamilton closed pull request #464: URL: https://github.com/apache/airflow-site/pull/464 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow-site] ryanahamilton opened a new pull request #464: Remove temporary banner promoting Airflow Summit

2021-08-12 Thread GitBox
ryanahamilton opened a new pull request #464: URL: https://github.com/apache/airflow-site/pull/464 This reverts commit 3cc2089e59754b4c51a6a5b3dd93b2286ae85dfb introduced in #403. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] ashb commented on a change in pull request #17581: Remove the use of multiprocessing in TestLocalTaskJob and Improve Tests

2021-08-12 Thread GitBox
ashb commented on a change in pull request #17581: URL: https://github.com/apache/airflow/pull/17581#discussion_r687800528 ## File path: tests/jobs/test_local_task_job.py ## @@ -273,42 +274,41 @@ def test_heartbeat_failed_fast(self): delta = (time2 -

[GitHub] [airflow] roestzwiee commented on issue #15340: helm install airflow in namespace get error: File "", line 32, in TimeoutError: There are still unapplied migrations after 60

2021-08-12 Thread GitBox
roestzwiee commented on issue #15340: URL: https://github.com/apache/airflow/issues/15340#issuecomment-897729319 @autarchprinceps I'm having the same trouble here. When deploying the chart to our local machines all runs fine. But deploying the chart to our cluster using helm does not

[GitHub] [airflow-ci-infra] ashb commented on a change in pull request #37: Remove unnecessary packer role setting

2021-08-12 Thread GitBox
ashb commented on a change in pull request #37: URL: https://github.com/apache/airflow-ci-infra/pull/37#discussion_r687837943 ## File path: github-runner-ami/packer/ubuntu2004.pkr.hcl ## @@ -35,10 +35,10 @@ variable "session_manager_instance_profile_name" { } source

[GitHub] [airflow] potiuk edited a comment on issue #17546: ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: cannot allocate memory in static TLS block

2021-08-12 Thread GitBox
potiuk edited a comment on issue #17546: URL: https://github.com/apache/airflow/issues/17546#issuecomment-896803730 Looking at the workaround, I don't think there is a `8.0.19` version of mysql client for `buster` (debian10) debian. The last one I could find was `8.0.20`

[GitHub] [airflow] potiuk commented on issue #17546: ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: cannot allocate memory in static TLS block

2021-08-12 Thread GitBox
potiuk commented on issue #17546: URL: https://github.com/apache/airflow/issues/17546#issuecomment-897735738 BTW. Thanks for the comment - corrected the "/usr" prefix in case someone has similar problem :) -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] rubenjv7 opened a new issue #17570: error to open

2021-08-12 Thread GitBox
rubenjv7 opened a new issue #17570: URL: https://github.com/apache/airflow/issues/17570 **Apache Airflow version**: **Apache Airflow Provider versions** (please include all providers that are relevant to your bug): **Kubernetes version (if you are

[GitHub] [airflow] JavierLopezT opened a new issue #17571: Collapsing doc section is not working

2021-08-12 Thread GitBox
JavierLopezT opened a new issue #17571: URL: https://github.com/apache/airflow/issues/17571 In the content menu, you can expand a section, but if you click again to collapse, it does nothing ![Uploading Captura de pantalla 2021-08-12 a las 11.12.10.png…]() -- This is an automated

[GitHub] [airflow] JavierLopezT opened a new issue #17572: Doc web page differente from code

2021-08-12 Thread GitBox
JavierLopezT opened a new issue #17572: URL: https://github.com/apache/airflow/issues/17572 In the web https://airflow.apache.org/docs/apache-airflow/stable/howto/customize-dag-ui-page-instance-name.html you can see ![Captura de pantalla 2021-08-12 a las 11 34

[GitHub] [airflow] potiuk commented on issue #17568: Airflow all scheduler crashed and cannot restore because of incorrect Group name

2021-08-12 Thread GitBox
potiuk commented on issue #17568: URL: https://github.com/apache/airflow/issues/17568#issuecomment-897594350 You cannot use 'space' in task_id. Your task id is 'Create grafana alert.alert_ESCALATION_AGING_FORWARD'. remove spaces and replace them with _ and it should start working fine.

[GitHub] [airflow] potiuk closed issue #17568: Airflow all scheduler crashed and cannot restore because of incorrect Group name

2021-08-12 Thread GitBox
potiuk closed issue #17568: URL: https://github.com/apache/airflow/issues/17568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk edited a comment on issue #17568: Airflow all scheduler crashed and cannot restore because of incorrect Group name

2021-08-12 Thread GitBox
potiuk edited a comment on issue #17568: URL: https://github.com/apache/airflow/issues/17568#issuecomment-897594350 You cannot use 'space' in task_id. Your task id is 'Create grafana alert.alert_ESCALATION_AGING_FORWARD'. remove spaces and replace them with _ and it should start working

[GitHub] [airflow] potiuk commented on issue #17566: Airflow with latest slim-buster(3.9) along with latest stable debian release.

2021-08-12 Thread GitBox
potiuk commented on issue #17566: URL: https://github.com/apache/airflow/issues/17566#issuecomment-897460105 @pukaleashish -. I assume you are talking about Airflow images. Your (quite rude and not something that is expected in our community to be honest) request is invalid on many

[GitHub] [airflow] potiuk closed issue #17566: Airflow with latest slim-buster(3.9) along with latest stable debian release.

2021-08-12 Thread GitBox
potiuk closed issue #17566: URL: https://github.com/apache/airflow/issues/17566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] dinigo commented on issue #13750: Support Standard SQL in BigQuery Sensor

2021-08-12 Thread GitBox
dinigo commented on issue #13750: URL: https://github.com/apache/airflow/issues/13750#issuecomment-897528058 WIP, have something, just struggling with the tests. I'll open it so we can review "collectively" -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] uranusjr commented on pull request #17544: Rescure if a DagRun's DAG was removed from db

2021-08-12 Thread GitBox
uranusjr commented on pull request #17544: URL: https://github.com/apache/airflow/pull/17544#issuecomment-897538890 Writing a proper test turns out to be a long journey figuring out a right combination to trigger this bug. So to trigger this bug, a DAG should _not_ be in the DagBag, but

[GitHub] [airflow] madhusudhan0542 opened a new issue #17573: importing Operators directly from DAGS folder in airflow

2021-08-12 Thread GitBox
madhusudhan0542 opened a new issue #17573: URL: https://github.com/apache/airflow/issues/17573 ### Discussed in https://github.com/apache/airflow/discussions/17569 Originally posted by **madhusudhan0542** August 12, 2021 My intent is to create a Operator folder in dags

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17573: importing Operators directly from DAGS folder in airflow

2021-08-12 Thread GitBox
boring-cyborg[bot] commented on issue #17573: URL: https://github.com/apache/airflow/issues/17573#issuecomment-897549133 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk edited a comment on issue #17568: Airflow all scheduler crashed and cannot restore because of incorrect Group name

2021-08-12 Thread GitBox
potiuk edited a comment on issue #17568: URL: https://github.com/apache/airflow/issues/17568#issuecomment-897594350 You cannot use 'space' in task_id. Your task id is 'Create grafana alert.alert_ESCALATION_AGING_FORWARD'. remove spaces and replace them with _ and it should start working

[GitHub] [airflow] bbovenzi commented on issue #17535: the filter condition of taskinstance does not work

2021-08-12 Thread GitBox
bbovenzi commented on issue #17535: URL: https://github.com/apache/airflow/issues/17535#issuecomment-897600625 > > Okay, I have drafted a PR. Your changes are looking good. Care to open a full PR to review? -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] kaxil commented on pull request #17545: Airflow exceptions pass-thru during "on_execute_callback"

2021-08-12 Thread GitBox
kaxil commented on pull request #17545: URL: https://github.com/apache/airflow/pull/17545#issuecomment-897600884 > can be known at the time of the DAG run whether or not to mark a task as skipped. That is not true in all cases though, DagRun creation does not mean your Task will

[GitHub] [airflow] kaxil commented on pull request #16352: AIP-39: ``DagRun.data_interval_start`` and ``data_interval_end``

2021-08-12 Thread GitBox
kaxil commented on pull request #16352: URL: https://github.com/apache/airflow/pull/16352#issuecomment-897602865 Failing on all Python 3.8 cases though? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] malthe opened a new pull request #17576: Add pre/post execution hooks

2021-08-12 Thread GitBox
malthe opened a new pull request #17576: URL: https://github.com/apache/airflow/pull/17576 This adds optional operator parameters `pre_execute` and `post_execute`, both of which run synchronously immediately before and after task execution. For example, this can be used to skip

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #16866: Remove default_args pattern + added get_current_context() use for Core Airflow example DAGs

2021-08-12 Thread GitBox
ephraimbuddy commented on a change in pull request #16866: URL: https://github.com/apache/airflow/pull/16866#discussion_r687488308 ## File path: airflow/example_dags/example_xcom.py ## @@ -78,6 +104,14 @@ def puller(**kwargs): pull = PythonOperator(

[GitHub] [airflow] malthe commented on pull request #17545: Airflow exceptions pass-thru during "on_execute_callback"

2021-08-12 Thread GitBox
malthe commented on pull request #17545: URL: https://github.com/apache/airflow/pull/17545#issuecomment-897477934 @potiuk it could be declarative as well; assuming that AIP-39 does not exist, it could simply be a cron expression. If it matches the execution date (or if the property is

[GitHub] [airflow] uranusjr commented on issue #17566: Airflow with latest slim-buster(3.9) along with latest stable debian release.

2021-08-12 Thread GitBox
uranusjr commented on issue #17566: URL: https://github.com/apache/airflow/issues/17566#issuecomment-897523906 Airflow’s images are based on CPython’s builds (https://hub.docker.com/_/python); `apache/airflow:2.1.2-python3.6`, for example, is built against `python:3.6-slim-buster`. So if

[GitHub] [airflow] malthe commented on pull request #17545: Airflow exceptions pass-thru during "on_execute_callback"

2021-08-12 Thread GitBox
malthe commented on pull request #17545: URL: https://github.com/apache/airflow/pull/17545#issuecomment-897564521 @potiuk with AIP-39, won't the isolation change since the concrete timetable implementation is now defined in DAG code? I'm not seeing anything about decoupling the timetable

[GitHub] [airflow] kaxil opened a new pull request #17575: Make `pandas` an optional core dependency

2021-08-12 Thread GitBox
kaxil opened a new pull request #17575: URL: https://github.com/apache/airflow/pull/17575 We only use `pandas` in `DbApiHook.get_pandas_df`. Not all users use it, plus while `pandas` now supports many pre-compiled packages it still can take forever where it needs to be compiled.

[GitHub] [airflow] malthe commented on pull request #17545: Airflow exceptions pass-thru during "on_execute_callback"

2021-08-12 Thread GitBox
malthe commented on pull request #17545: URL: https://github.com/apache/airflow/pull/17545#issuecomment-897609829 I'll go ahead and close this pull request. The discussion can continue in separate tracks – one for pre/post execute hooks and another for declarative task scheduling

[GitHub] [airflow] malthe closed pull request #17545: Airflow exceptions pass-thru during "on_execute_callback"

2021-08-12 Thread GitBox
malthe closed pull request #17545: URL: https://github.com/apache/airflow/pull/17545 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] yevsh commented on issue #17395: DAG is executed twice on submit

2021-08-12 Thread GitBox
yevsh commented on issue #17395: URL: https://github.com/apache/airflow/issues/17395#issuecomment-897611069 @ephraimbuddy thank you! waiting with anticipation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] JavierLopezT commented on pull request #17379: Add Mongo projections to hook and transfer

2021-08-12 Thread GitBox
JavierLopezT commented on pull request #17379: URL: https://github.com/apache/airflow/pull/17379#issuecomment-897471707 > The mongo tests are failing All green now :) I started using Breeze yesterday (finally) so that I don't have these test failures in github -- This is an

[GitHub] [airflow] pukaleashish commented on issue #17566: Airflow with latest slim-buster(3.9) along with latest stable debian release.

2021-08-12 Thread GitBox
pukaleashish commented on issue #17566: URL: https://github.com/apache/airflow/issues/17566#issuecomment-897502028 Thanks for sharing info. I didnt mean to be rude here. I was just asking the details when the slim buster would be updated to latest version. Anyways i will go

[GitHub] [airflow] kaxil commented on pull request #17567: Path correction in docs for airflow core

2021-08-12 Thread GitBox
kaxil commented on pull request #17567: URL: https://github.com/apache/airflow/pull/17567#issuecomment-897602069 Thanks @amit-talreja for catching this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] kaxil edited a comment on pull request #16352: AIP-39: ``DagRun.data_interval_start`` and ``data_interval_end``

2021-08-12 Thread GitBox
kaxil edited a comment on pull request #16352: URL: https://github.com/apache/airflow/pull/16352#issuecomment-897602865 Failing on all Python 3.8 cases though, still unrelated to the change though -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] uranusjr closed issue #17572: Doc web page differente from code

2021-08-12 Thread GitBox
uranusjr closed issue #17572: URL: https://github.com/apache/airflow/issues/17572 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] uranusjr commented on issue #17572: Doc web page differente from code

2021-08-12 Thread GitBox
uranusjr commented on issue #17572: URL: https://github.com/apache/airflow/issues/17572#issuecomment-897500718 Because the fix fa811057a6ae0fc6c5e4bff1e18971c262a42a4c happened after 2.1.2 was released, and we don’t go back and fix the 2.1.2 documentation. Please wait for 2.1.3 and all

[GitHub] [airflow] potiuk edited a comment on issue #17574: [BUG] Helm chart does not correctly support sslmode for PostgreSQL connection

2021-08-12 Thread GitBox
potiuk edited a comment on issue #17574: URL: https://github.com/apache/airflow/issues/17574#issuecomment-897759148 > I am using an AWS RDS instance and so the CA certs SHOULD be in the OS by default. If they are not then they are available in the AWS RDS PKI truststore. This is

[GitHub] [airflow] potiuk edited a comment on issue #17574: [BUG] Helm chart does not correctly support sslmode for PostgreSQL connection

2021-08-12 Thread GitBox
potiuk edited a comment on issue #17574: URL: https://github.com/apache/airflow/issues/17574#issuecomment-897759148 > I am using an AWS RDS instance and so the CA certs SHOULD be in the OS by default. If they are not then they are available in the AWS RDS PKI truststore. This is

[GitHub] [airflow] potiuk edited a comment on issue #17574: [BUG] Helm chart does not correctly support sslmode for PostgreSQL connection

2021-08-12 Thread GitBox
potiuk edited a comment on issue #17574: URL: https://github.com/apache/airflow/issues/17574#issuecomment-897759148 > I am using an AWS RDS instance and so the CA certs SHOULD be in the OS by default. If they are not then they are available in the AWS RDS PKI truststore. This is

[GitHub] [airflow] barun-mazumdar edited a comment on issue #17219: sqlite3.OperationalError: no such column: variable.description

2021-08-12 Thread GitBox
barun-mazumdar edited a comment on issue #17219: URL: https://github.com/apache/airflow/issues/17219#issuecomment-897769421 It happening also on docker container from image: airflow 2.1.2, which I try to run with default sqlite db of airflow. I just run the airflow db init, airflow

[GitHub] [airflow] potiuk commented on pull request #17575: Make `pandas` an optional core dependency

2021-08-12 Thread GitBox
potiuk commented on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-897788822 And it should work for old airflow versions as well. AFAIK when you specifiy extra that is missing, it will be silently ignored, so it will work for pre 2.2 Airflow versions

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17584: Replace constraints file requirement with `pyproject.toml` version rules

2021-08-12 Thread GitBox
boring-cyborg[bot] commented on issue #17584: URL: https://github.com/apache/airflow/issues/17584#issuecomment-897820987 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] aaronsteers opened a new issue #17584: Replace constraints file requirement with `pyproject.toml` version rules

2021-08-12 Thread GitBox
aaronsteers opened a new issue #17584: URL: https://github.com/apache/airflow/issues/17584 **Description** It would be wonderful to not have to use a custom constraints file during pip installs. My understanding is that these rules can be defined within `pyproject.toml` and

[GitHub] [airflow] kurtqq commented on pull request #16796: AwsGlueJobOperator: add run_job_kwargs to Glue job run

2021-08-12 Thread GitBox
kurtqq commented on pull request #16796: URL: https://github.com/apache/airflow/pull/16796#issuecomment-897852073 i guess pr is waiting for unit tests? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] uranusjr edited a comment on issue #17584: Replace constraints file requirement with `pyproject.toml` version rules

2021-08-12 Thread GitBox
uranusjr edited a comment on issue #17584: URL: https://github.com/apache/airflow/issues/17584#issuecomment-897866259 Could you provide some references for this? Putting package metadata in `pyproject.toml` ([PEP 621](https://www.python.org/dev/peps/pep-0621/)) is not intended to replace

[GitHub] [airflow] uranusjr commented on issue #17584: Replace constraints file requirement with `pyproject.toml` version rules

2021-08-12 Thread GitBox
uranusjr commented on issue #17584: URL: https://github.com/apache/airflow/issues/17584#issuecomment-897866259 Could you provide some references for this? Putting package metadata in `pyproject.toml` ([PEP 621](https://www.python.org/dev/peps/pep-0621/)) is not intended to replace

[GitHub] [airflow] potiuk closed issue #17574: [BUG] Helm chart does not correctly support sslmode for PostgreSQL connection

2021-08-12 Thread GitBox
potiuk closed issue #17574: URL: https://github.com/apache/airflow/issues/17574 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #17574: [BUG] Helm chart does not correctly support sslmode for PostgreSQL connection

2021-08-12 Thread GitBox
potiuk commented on issue #17574: URL: https://github.com/apache/airflow/issues/17574#issuecomment-897759148 This is postgres' requirement, not airflow's (And for a very good reason). Postgres will never read from the certificate chain installed in the system for a very good reason. What

[GitHub] [airflow] github-actions[bot] commented on pull request #17580: Quarantine test_mark_success_no_kill test

2021-08-12 Thread GitBox
github-actions[bot] commented on pull request #17580: URL: https://github.com/apache/airflow/pull/17580#issuecomment-89965 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] ephraimbuddy commented on pull request #17580: Quarantine test_mark_success_no_kill test

2021-08-12 Thread GitBox
ephraimbuddy commented on pull request #17580: URL: https://github.com/apache/airflow/pull/17580#issuecomment-89421 Let's get it merged while I work on the other PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] potiuk closed issue #17570: error to open

2021-08-12 Thread GitBox
potiuk closed issue #17570: URL: https://github.com/apache/airflow/issues/17570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy merged pull request #17580: Quarantine test_mark_success_no_kill test

2021-08-12 Thread GitBox
ephraimbuddy merged pull request #17580: URL: https://github.com/apache/airflow/pull/17580 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk edited a comment on pull request #17575: Make `pandas` an optional core dependency

2021-08-12 Thread GitBox
potiuk edited a comment on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-897788822 And the Dockerfile will continue to work for old airflow versions as well. AFAIK when you specifiy extra that is missing, it will be silently ignored, so it will work

[GitHub] [airflow] potiuk commented on pull request #17545: Airflow exceptions pass-thru during "on_execute_callback"

2021-08-12 Thread GitBox
potiuk commented on pull request #17545: URL: https://github.com/apache/airflow/pull/17545#issuecomment-897793611 > @potiuk with AIP-39, won't the isolation change since the concrete timetable implementation is now defined in DAG code? I'm not seeing anything about decoupling the

  1   2   3   >