[GitHub] [airflow] roshankp opened a new issue #18365: trigger dag not showing start time and end time in the tree view

2021-09-19 Thread GitBox
roshankp opened a new issue #18365: URL: https://github.com/apache/airflow/issues/18365 ### Apache Airflow version 2.1.3 ### Operating System macosx ### Versions of Apache Airflow Providers apache-airflow==1!2.1.3+astro.1

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18365: trigger dag not showing start time and end time in the tree view

2021-09-19 Thread GitBox
boring-cyborg[bot] commented on issue #18365: URL: https://github.com/apache/airflow/issues/18365#issuecomment-922652694 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] eladkal closed issue #13733: Airflow task timeout not working in AWSBatchOperator

2021-09-19 Thread GitBox
eladkal closed issue #13733: URL: https://github.com/apache/airflow/issues/13733 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eladkal commented on issue #13733: Airflow task timeout not working in AWSBatchOperator

2021-09-19 Thread GitBox
eladkal commented on issue #13733: URL: https://github.com/apache/airflow/issues/13733#issuecomment-922647554 fixed by https://github.com/apache/airflow/pull/17081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] eladkal commented on issue #13454: ProjecId and Region are not templated fields in DataprocDeleteClusterOperator

2021-09-19 Thread GitBox
eladkal commented on issue #13454: URL: https://github.com/apache/airflow/issues/13454#issuecomment-922647381 fixed by https://github.com/apache/airflow/pull/14086 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] eladkal closed issue #13454: ProjecId and Region are not templated fields in DataprocDeleteClusterOperator

2021-09-19 Thread GitBox
eladkal closed issue #13454: URL: https://github.com/apache/airflow/issues/13454 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eladkal closed issue #9299: Unable to store xcom because of MySQL BLOB type limitation

2021-09-19 Thread GitBox
eladkal closed issue #9299: URL: https://github.com/apache/airflow/issues/9299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eladkal commented on issue #9299: Unable to store xcom because of MySQL BLOB type limitation

2021-09-19 Thread GitBox
eladkal commented on issue #9299: URL: https://github.com/apache/airflow/issues/9299#issuecomment-922647064 I'm closing this as won't fix. You can set alternative xcom backend if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] prmkumar64 opened a new issue #18364: sqlite3.OperationalError: no such column: dag.last_parsed_time

2021-09-19 Thread GitBox
prmkumar64 opened a new issue #18364: URL: https://github.com/apache/airflow/issues/18364 ### Apache Airflow version 2.1.0 ### Operating System Linux ### Versions of Apache Airflow Providers (sandbox) airflow@airflowvm:/home$ pip freeze |grep

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18364: sqlite3.OperationalError: no such column: dag.last_parsed_time

2021-09-19 Thread GitBox
boring-cyborg[bot] commented on issue #18364: URL: https://github.com/apache/airflow/issues/18364#issuecomment-922643063 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[airflow] branch main updated: Add support for templated fields in PapermillOperator (#18357)

2021-09-19 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f382a79 Add support for templated fields in

[GitHub] [airflow] eladkal merged pull request #18357: Add support for templated fields in PapermillOperator

2021-09-19 Thread GitBox
eladkal merged pull request #18357: URL: https://github.com/apache/airflow/pull/18357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eladkal closed issue #12081: PapermillOperator does not take user defined macros

2021-09-19 Thread GitBox
eladkal closed issue #12081: URL: https://github.com/apache/airflow/issues/12081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] zhongjiajie commented on pull request #8231: Dag bulk_sync_to_db dag_tag only remove not exists

2021-09-19 Thread GitBox
zhongjiajie commented on pull request #8231: URL: https://github.com/apache/airflow/pull/8231#issuecomment-922635254 It doesn't matter, nice to see you again. @potiuk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] edwardwang888 commented on a change in pull request #18356: Explain scheduler fine-tuning better

2021-09-19 Thread GitBox
edwardwang888 commented on a change in pull request #18356: URL: https://github.com/apache/airflow/pull/18356#discussion_r711870134 ## File path: docs/apache-airflow/concepts/scheduler.rst ## @@ -138,12 +141,101 @@ The following databases are fully supported and provide an

[GitHub] [airflow] edwardwang888 commented on a change in pull request #18356: Explain scheduler fine-tuning better

2021-09-19 Thread GitBox
edwardwang888 commented on a change in pull request #18356: URL: https://github.com/apache/airflow/pull/18356#discussion_r711869019 ## File path: docs/apache-airflow/concepts/scheduler.rst ## @@ -138,12 +141,101 @@ The following databases are fully supported and provide an

[GitHub] [airflow] edwardwang888 commented on a change in pull request #18356: Explain scheduler fine-tuning better

2021-09-19 Thread GitBox
edwardwang888 commented on a change in pull request #18356: URL: https://github.com/apache/airflow/pull/18356#discussion_r711869019 ## File path: docs/apache-airflow/concepts/scheduler.rst ## @@ -138,12 +141,101 @@ The following databases are fully supported and provide an

[GitHub] [airflow] datsabk commented on issue #18239: Airflow Celery Worker logs inaccessible

2021-09-19 Thread GitBox
datsabk commented on issue #18239: URL: https://github.com/apache/airflow/issues/18239#issuecomment-922619410 Hello @dimon222 - The workers are sitting behind a Kubernetes service. Hence, the logs need to be accessible via the service name. To support this, I have used a different

[GitHub] [airflow] MatrixManAtYrService opened a new issue #18363: http patched a dag_run to state=failed while it was running, but the change didn't persist

2021-09-19 Thread GitBox
MatrixManAtYrService opened a new issue #18363: URL: https://github.com/apache/airflow/issues/18363 ### Apache Airflow version 2.2.0b1 (beta snapshot) ### Operating System debian buster ### Versions of Apache Airflow Providers n/a ### Deployment

[GitHub] [airflow] morhook commented on issue #18041: Tasks intermittently gets terminated with SIGTERM on kubernetes executor

2021-09-19 Thread GitBox
morhook commented on issue #18041: URL: https://github.com/apache/airflow/issues/18041#issuecomment-922604323 Same thing for me. I suspect my database might be getting slower also. El El dom, sep. 19, 2021 a las 12:17, Felipe Angelim < ***@***.***> escribió: > @laserpedro

[GitHub] [airflow] pyerbiz commented on pull request #17355: [11912] [skip ci] Add how-to Guide for Azure operators

2021-09-19 Thread GitBox
pyerbiz commented on pull request #17355: URL: https://github.com/apache/airflow/pull/17355#issuecomment-922596307 @kaxil Not right now. I'll try to this weekend and seek help if I need it. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] SamWheating commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
SamWheating commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711842380 ## File path: tests/core/test_sqlalchemy_config.py ## @@ -77,7 +77,7 @@ def test_sql_alchemy_connect_args( with conf_vars(config):

[GitHub] [airflow] tkang007 edited a comment on issue #13322: Backfill crashes with "KeyError: TaskInstanceKey" when task has retries

2021-09-19 Thread GitBox
tkang007 edited a comment on issue #13322: URL: https://github.com/apache/airflow/issues/13322#issuecomment-922529114 This issue still happened with apache-airflow==2.1.3 and 2.1.4 version. In my case, `airflow dags test` was failed after DB reinitialized without `airflow dags

[GitHub] [airflow] SamWheating commented on a change in pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-19 Thread GitBox
SamWheating commented on a change in pull request #18197: URL: https://github.com/apache/airflow/pull/18197#discussion_r711840591 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -663,6 +765,16 @@ def execute(self, context) -> dict:

[GitHub] [airflow] SamWheating commented on a change in pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-19 Thread GitBox
SamWheating commented on a change in pull request #18197: URL: https://github.com/apache/airflow/pull/18197#discussion_r711839969 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -527,7 +629,7 @@ def __init__( cluster_name: str,

[GitHub] [airflow] SamWheating commented on a change in pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-19 Thread GitBox
SamWheating commented on a change in pull request #18197: URL: https://github.com/apache/airflow/pull/18197#discussion_r711838902 ## File path: tests/providers/google/cloud/operators/test_dataproc_system.py ## @@ -54,12 +75,14 @@ def setUp(self):

[GitHub] [airflow] github-actions[bot] commented on pull request #18357: Add support for templated fields in PapermillOperator

2021-09-19 Thread GitBox
github-actions[bot] commented on pull request #18357: URL: https://github.com/apache/airflow/pull/18357#issuecomment-922588996 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] SamWheating commented on a change in pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-19 Thread GitBox
SamWheating commented on a change in pull request #18197: URL: https://github.com/apache/airflow/pull/18197#discussion_r711838154 ## File path: tests/always/test_project_structure.py ## @@ -189,6 +189,7 @@ class TestGoogleProviderProjectStructure(unittest.TestCase):

[GitHub] [airflow] SamWheating commented on a change in pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-19 Thread GitBox
SamWheating commented on a change in pull request #18197: URL: https://github.com/apache/airflow/pull/18197#discussion_r711837965 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -663,6 +765,16 @@ def execute(self, context) -> dict:

[GitHub] [airflow] SamWheating commented on a change in pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-19 Thread GitBox
SamWheating commented on a change in pull request #18197: URL: https://github.com/apache/airflow/pull/18197#discussion_r711837965 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -663,6 +765,16 @@ def execute(self, context) -> dict:

[airflow] branch constraints-main updated: Updating constraints. Build id:1251858004

2021-09-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new f416a4c8 Updating

[GitHub] [airflow] SamWheating commented on a change in pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-19 Thread GitBox
SamWheating commented on a change in pull request #18197: URL: https://github.com/apache/airflow/pull/18197#discussion_r711836314 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -48,6 +54,101 @@ ) +class

[GitHub] [airflow] SamWheating commented on a change in pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-19 Thread GitBox
SamWheating commented on a change in pull request #18197: URL: https://github.com/apache/airflow/pull/18197#discussion_r711835811 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -48,6 +54,101 @@ ) +class

[GitHub] [airflow] taylorfinnell commented on issue #18011: Task stuck in upstream_failed

2021-09-19 Thread GitBox
taylorfinnell commented on issue #18011: URL: https://github.com/apache/airflow/issues/18011#issuecomment-922582638 Thanks! It seems to me that setting is specific to the k8s executor - but we are using the CeleryExecutor -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] ephraimbuddy edited a comment on issue #18011: Task stuck in upstream_failed

2021-09-19 Thread GitBox
ephraimbuddy edited a comment on issue #18011: URL: https://github.com/apache/airflow/issues/18011#issuecomment-922573041 @taylorfinnell , I will suggest you to increase the value of this configuration worker_pods_pending_timeout, not sure if it’ll resolve it but it’s also connected with

[GitHub] [airflow] ephraimbuddy edited a comment on issue #18011: Task stuck in upstream_failed

2021-09-19 Thread GitBox
ephraimbuddy edited a comment on issue #18011: URL: https://github.com/apache/airflow/issues/18011#issuecomment-922545376 > Trying to get to a reproducible step here... > > Is there an existing "unit" test (or could you help me write a unit test) for: > > 1. A -> B dag >

[GitHub] [airflow] ariesunique commented on issue #11919: Add how-to Guide for Slack operators

2021-09-19 Thread GitBox
ariesunique commented on issue #11919: URL: https://github.com/apache/airflow/issues/11919#issuecomment-922574958 No. Someone else can take it. On Sun, Sep 19, 2021, 8:44 PM Kaxil Naik ***@***.***> wrote: > @ariesunique Are you working on this?

[GitHub] [airflow] ephraimbuddy edited a comment on issue #18011: Task stuck in upstream_failed

2021-09-19 Thread GitBox
ephraimbuddy edited a comment on issue #18011: URL: https://github.com/apache/airflow/issues/18011#issuecomment-922545376 > Trying to get to a reproducible step here... > > Is there an existing "unit" test (or could you help me write a unit test) for: > > 1. A -> B dag >

[GitHub] [airflow] ephraimbuddy commented on issue #18011: Task stuck in upstream_failed

2021-09-19 Thread GitBox
ephraimbuddy commented on issue #18011: URL: https://github.com/apache/airflow/issues/18011#issuecomment-922573041 @taylorfinnell , I will suggest you to increase the value of this configuration worker_pods_pending_timeout_check_interval, not sure if it’ll resolve it but it’s also

[GitHub] [airflow] kaxil commented on pull request #17355: [11912] [skip ci] Add how-to Guide for Azure operators

2021-09-19 Thread GitBox
kaxil commented on pull request #17355: URL: https://github.com/apache/airflow/pull/17355#issuecomment-922569632 @pyerbiz Do you need any help with this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] kaxil commented on issue #11919: Add how-to Guide for Slack operators

2021-09-19 Thread GitBox
kaxil commented on issue #11919: URL: https://github.com/apache/airflow/issues/11919#issuecomment-922569314 @ariesunique Are you working on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] kaxil closed issue #16302: DB migration: TaskInstance tables from `execution_date` to `run_id`

2021-09-19 Thread GitBox
kaxil closed issue #16302: URL: https://github.com/apache/airflow/issues/16302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil closed issue #15785: Chart: Add Integration test to check changing Executor

2021-09-19 Thread GitBox
kaxil closed issue #15785: URL: https://github.com/apache/airflow/issues/15785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] github-actions[bot] commented on pull request #18306: Chart: warn when webserver secret key isn't set

2021-09-19 Thread GitBox
github-actions[bot] commented on pull request #18306: URL: https://github.com/apache/airflow/pull/18306#issuecomment-922562536 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] github-actions[bot] commented on issue #8731: Logging module (custom) no longer working

2021-09-19 Thread GitBox
github-actions[bot] commented on issue #8731: URL: https://github.com/apache/airflow/issues/8731#issuecomment-922560455 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] github-actions[bot] closed issue #8731: Logging module (custom) no longer working

2021-09-19 Thread GitBox
github-actions[bot] closed issue #8731: URL: https://github.com/apache/airflow/issues/8731 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] dnz-siliang edited a comment on issue #18009: Task groups are not stable in the UI

2021-09-19 Thread GitBox
dnz-siliang edited a comment on issue #18009: URL: https://github.com/apache/airflow/issues/18009#issuecomment-922556939 Meeting the same issue from airflow 2.1.3 (Airflow on EKS with CeleryKubernetesExecutor). Tasks are keep changing location on tree view. However we didn't meet

[GitHub] [airflow] dnz-siliang commented on issue #18009: Task groups are not stable in the UI

2021-09-19 Thread GitBox
dnz-siliang commented on issue #18009: URL: https://github.com/apache/airflow/issues/18009#issuecomment-922556939 Meeting the same issue from airflow 2.1.3. Tasks are keep changing location on tree view. However we didn't meet this issue on 2.1.2. Any ideas? -- This is an

[GitHub] [airflow] edwardwang888 commented on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
edwardwang888 commented on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922555221 > Depends what you mean by "core". There is no mentioning of "core" in the extras doc: Ah okay, that is a good point. > It's clear that `devel` is for all

[GitHub] [airflow] potiuk commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711817305 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -38,3 +40,14 @@ services: timeout: 10s retries: 10 restart: always +

[GitHub] [airflow] potiuk commented on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
potiuk commented on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922550696 > Does test_json.py count as core testing? Depends what you mean by "core". There is no mentioning of "core" in the extras doc: From

[GitHub] [airflow] kaxil commented on a change in pull request #18360: Fixed broken URL

2021-09-19 Thread GitBox
kaxil commented on a change in pull request #18360: URL: https://github.com/apache/airflow/pull/18360#discussion_r711812696 ## File path: docs/apache-airflow/security/api.rst ## @@ -84,7 +84,7 @@ principal exists in the keytab file. Basic authentication

[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
edwardwang888 edited a comment on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922545748 > We have PLENTY of tests that will fail when some extra is not installed. That's why we have https://github.com/apache/airflow/blob/main/BREEZE.rst with all

[GitHub] [airflow] ephraimbuddy closed issue #16412: API for marking the running DAG state to success or failure

2021-09-19 Thread GitBox
ephraimbuddy closed issue #16412: URL: https://github.com/apache/airflow/issues/16412 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy commented on issue #16412: API for marking the running DAG state to success or failure

2021-09-19 Thread GitBox
ephraimbuddy commented on issue #16412: URL: https://github.com/apache/airflow/issues/16412#issuecomment-922546569 Closed by #17839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
edwardwang888 edited a comment on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922545748 > We have PLENTY of tests that will fail when some extra is not installed. That's why we have https://github.com/apache/airflow/blob/main/BREEZE.rst with all

[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
edwardwang888 edited a comment on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922545748 > We have PLENTY of tests that will fail when some extra is not installed. That's why we have https://github.com/apache/airflow/blob/main/BREEZE.rst with all

[GitHub] [airflow] ephraimbuddy commented on issue #18011: Task stuck in upstream_failed

2021-09-19 Thread GitBox
ephraimbuddy commented on issue #18011: URL: https://github.com/apache/airflow/issues/18011#issuecomment-922546341 > Hi @ephraimbuddy - I work with @WattsInABox. We don't see `FATAL: sorry, too many clients already.` but we do see: > > ``` > Traceback (most recent call last):

[GitHub] [airflow] edwardwang888 commented on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
edwardwang888 commented on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922545748 > We have PLENTY of tests that will fail when some extra is not installed. That's why we have https://github.com/apache/airflow/blob/main/BREEZE.rst with all deps

[GitHub] [airflow] ephraimbuddy commented on issue #18011: Task stuck in upstream_failed

2021-09-19 Thread GitBox
ephraimbuddy commented on issue #18011: URL: https://github.com/apache/airflow/issues/18011#issuecomment-922545376 > Trying to get to a reproducible step here... > > Is there an existing "unit" test (or could you help me write a unit test) for: > > 1. A -> B dag > 2. A set

[airflow] branch constraints-main updated: Updating constraints. Build id:1251488358

2021-09-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 9301a9c Updating

[GitHub] [airflow] mik-laj commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
mik-laj commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711810390 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -38,3 +40,14 @@ services: timeout: 10s retries: 10 restart: always +

[GitHub] [airflow] mik-laj commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
mik-laj commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711810390 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -38,3 +40,14 @@ services: timeout: 10s retries: 10 restart: always +

[GitHub] [airflow] potiuk commented on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
potiuk commented on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922542839 > I guess the concern there is about the local development environment? Of course, if you run a test locally and numpy is missing, you can just install it. We have

[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
edwardwang888 edited a comment on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922540200 > > We install that in Dockerfile so we have it while running tests > > Agree. The "test" environment has all the deps in - including pandas and numpy. The

[GitHub] [airflow] edwardwang888 commented on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
edwardwang888 commented on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922540200 > > We install that in Dockerfile so we have it while running tests > > Agree. The "test" environment has all the deps in - including pandas and numpy. The "test"

[GitHub] [airflow] potiuk commented on a change in pull request #18360: Fixed broken URL

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18360: URL: https://github.com/apache/airflow/pull/18360#discussion_r711805182 ## File path: docs/apache-airflow/security/api.rst ## @@ -84,7 +84,7 @@ principal exists in the keytab file. Basic authentication

[GitHub] [airflow] potiuk commented on a change in pull request #18360: Fixed broken URL

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18360: URL: https://github.com/apache/airflow/pull/18360#discussion_r711805182 ## File path: docs/apache-airflow/security/api.rst ## @@ -84,7 +84,7 @@ principal exists in the keytab file. Basic authentication

[GitHub] [airflow] potiuk commented on a change in pull request #18360: Fixed broken URL

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18360: URL: https://github.com/apache/airflow/pull/18360#discussion_r711805140 ## File path: docs/apache-airflow/security/api.rst ## @@ -84,7 +84,7 @@ principal exists in the keytab file. Basic authentication

[GitHub] [airflow] potiuk commented on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
potiuk commented on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922538744 > We install that in Dockerfile so we have it while running tests Agree. The "test" environment has all the deps in - including pandas and numpy. The "test" dependency

[GitHub] [airflow] mik-laj commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
mik-laj commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711804728 ## File path: docs/apache-airflow/howto/set-up-database.rst ## @@ -226,6 +227,27 @@ For more information regarding setup of the PostgresSQL

[GitHub] [airflow] potiuk commented on issue #13322: Backfill crashes with "KeyError: TaskInstanceKey" when task has retries

2021-09-19 Thread GitBox
potiuk commented on issue #13322: URL: https://github.com/apache/airflow/issues/13322#issuecomment-922538291 > This issue still happened with apache-airflow==2.1.3 and 2.1.4 version. > > In my case, `airflow dags test` was failed after DB reinitialized without `airflow dags

[GitHub] [airflow] potiuk commented on issue #14489: Add a retry with wait interval for SSH operator

2021-09-19 Thread GitBox
potiuk commented on issue #14489: URL: https://github.com/apache/airflow/issues/14489#issuecomment-922538167 assigned it to you @nshetty15 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711802946 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -20,12 +20,12 @@ services: airflow: environment: - BACKEND=mssql - -

[GitHub] [airflow] mik-laj commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
mik-laj commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711802455 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -20,12 +20,12 @@ services: airflow: environment: - BACKEND=mssql -

[GitHub] [airflow] potiuk commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711802537 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -20,12 +20,12 @@ services: airflow: environment: - BACKEND=mssql - -

[GitHub] [airflow] mik-laj commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
mik-laj commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711802455 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -20,12 +20,12 @@ services: airflow: environment: - BACKEND=mssql -

[GitHub] [airflow] potiuk commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711801973 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -20,12 +20,12 @@ services: airflow: environment: - BACKEND=mssql - -

[airflow] branch main updated (14b3e66 -> 1008d8b)

2021-09-19 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 14b3e66 Improve coverage for airflow.cli package (#18220) add 1008d8b Set default branch in codecov.yml

[GitHub] [airflow] kaxil merged pull request #18361: Set default branch in codecov.yml

2021-09-19 Thread GitBox
kaxil merged pull request #18361: URL: https://github.com/apache/airflow/pull/18361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711801218 ## File path: docs/apache-airflow/howto/set-up-database.rst ## @@ -226,6 +226,27 @@ For more information regarding setup of the PostgresSQL connection,

[GitHub] [airflow] potiuk commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711800824 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -20,12 +20,12 @@ services: airflow: environment: - BACKEND=mssql - -

[GitHub] [airflow] potiuk commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
potiuk commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711800510 ## File path: docs/apache-airflow/howto/set-up-database.rst ## @@ -226,6 +226,27 @@ For more information regarding setup of the PostgresSQL connection,

[GitHub] [airflow] mik-laj commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
mik-laj commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711800393 ## File path: scripts/ci/docker-compose/backend-mssql.yml ## @@ -20,12 +20,12 @@ services: airflow: environment: - BACKEND=mssql -

[GitHub] [airflow] mik-laj commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
mik-laj commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711800173 ## File path: docs/apache-airflow/howto/set-up-database.rst ## @@ -226,6 +226,27 @@ For more information regarding setup of the PostgresSQL

[GitHub] [airflow] mik-laj commented on a change in pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
mik-laj commented on a change in pull request #18362: URL: https://github.com/apache/airflow/pull/18362#discussion_r711800094 ## File path: docs/apache-airflow/howto/set-up-database.rst ## @@ -226,6 +226,27 @@ For more information regarding setup of the PostgresSQL

[GitHub] [airflow] potiuk commented on pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
potiuk commented on pull request #18362: URL: https://github.com/apache/airflow/pull/18362#issuecomment-922533669 cc: @aneesh-joseph -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk opened a new pull request #18362: Fix random deadlocks in MSSQL database

2021-09-19 Thread GitBox
potiuk opened a new pull request #18362: URL: https://github.com/apache/airflow/pull/18362 Default isolation level for the MSSQL database is READ_COMMITTED but its implementation in MSSQL might cause random deadlocks for select queries - because by default MSSQL uses shared locks

[GitHub] [airflow] mik-laj opened a new pull request #18361: Set default branch in codecov.yml

2021-09-19 Thread GitBox
mik-laj opened a new pull request #18361: URL: https://github.com/apache/airflow/pull/18361 The codecov uses master as default branch, See: https://docs.codecov.com/docs/codecov-yaml#master-copy Hopefully this will solve the codecov stats missing problem. --- **^

[airflow] branch main updated (11e3453 -> 14b3e66)

2021-09-19 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 11e3453 Creating ADF pipeline run operator, sensor + ADF custom conn fields (#17885) add 14b3e66

[GitHub] [airflow] mik-laj merged pull request #18220: Improve coverage for airflow.cli package

2021-09-19 Thread GitBox
mik-laj merged pull request #18220: URL: https://github.com/apache/airflow/pull/18220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil commented on a change in pull request #18360: Fixed broken URL

2021-09-19 Thread GitBox
kaxil commented on a change in pull request #18360: URL: https://github.com/apache/airflow/pull/18360#discussion_r711796564 ## File path: docs/apache-airflow/security/api.rst ## @@ -84,7 +84,7 @@ principal exists in the keytab file. Basic authentication

[GitHub] [airflow] kaxil commented on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
kaxil commented on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922529365 > @kaxil Should this import also be put inside a try block, if `numpy` is not a dependency required in the `devel` extra? > >

[GitHub] [airflow] tkang007 commented on issue #13322: Backfill crashes with "KeyError: TaskInstanceKey" when task has retries

2021-09-19 Thread GitBox
tkang007 commented on issue #13322: URL: https://github.com/apache/airflow/issues/13322#issuecomment-922529114 This issue still happened with apache-airflow==2.1.3 and 2.1.4 version. In my case, `airflow dags test` was failed after DB reinitialized without `airflow dags backfill'

[GitHub] [airflow] nshetty15 commented on issue #14489: Add a retry with wait interval for SSH operator

2021-09-19 Thread GitBox
nshetty15 commented on issue #14489: URL: https://github.com/apache/airflow/issues/14489#issuecomment-922528336 @eladkal Can I take this up? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] Akhi1 opened a new pull request #18360: Fixed broken URL

2021-09-19 Thread GitBox
Akhi1 opened a new pull request #18360: URL: https://github.com/apache/airflow/pull/18360 URL for "Basic username password authentication" in this readme was broken --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency

2021-09-19 Thread GitBox
edwardwang888 edited a comment on pull request #17575: URL: https://github.com/apache/airflow/pull/17575#issuecomment-922516186 @kaxil Should this import also be put inside a try block, if `numpy` is not a dependency required in the `devel` extra?

[GitHub] [airflow] alex-astronomer commented on pull request #17839: Add DAG run abort endpoint

2021-09-19 Thread GitBox
alex-astronomer commented on pull request #17839: URL: https://github.com/apache/airflow/pull/17839#issuecomment-922517598 Could use more documentation about how to use this endpoint. Not sure if that exists anywhere, I couldn't find in the `v1.yaml` file. Had to dig into source

[GitHub] [airflow] alex-astronomer edited a comment on pull request #17839: Add DAG run abort endpoint

2021-09-19 Thread GitBox
alex-astronomer edited a comment on pull request #17839: URL: https://github.com/apache/airflow/pull/17839#issuecomment-922517386 Also closes: #16412 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] alex-astronomer commented on pull request #17839: Add DAG run abort endpoint

2021-09-19 Thread GitBox
alex-astronomer commented on pull request #17839: URL: https://github.com/apache/airflow/pull/17839#issuecomment-922517386 Also closes: #17839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

  1   2   >