[GitHub] [airflow] sowhatim commented on issue #18989: CeleryKubernetesExecutor does not shutdown gracefully in 2.2.0

2022-01-14 Thread GitBox
sowhatim commented on issue #18989: URL: https://github.com/apache/airflow/issues/18989#issuecomment-1012974092 Facing the similar issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] uranusjr commented on issue #19902: DuplicateTaskIdFound when reusing tasks with task_group decorator

2022-01-14 Thread GitBox
uranusjr commented on issue #19902: URL: https://github.com/apache/airflow/issues/19902#issuecomment-1012992340 Oh the cause is simpler than I though; fixing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] uranusjr opened a new pull request #20870: Fix task ID deduplication in @task_group

2022-01-14 Thread GitBox
uranusjr opened a new pull request #20870: URL: https://github.com/apache/airflow/pull/20870 When appending `__{n}` for task ID deduplication, we are currently using `dag.task_ids`, which prepends the surrounding task group ID. This means that we must also use the group-prefixed *current*

[GitHub] [airflow] github-actions[bot] commented on pull request #20871: Fix failing main

2022-01-14 Thread GitBox
github-actions[bot] commented on pull request #20871: URL: https://github.com/apache/airflow/pull/20871#issuecomment-1013030185 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] kaxil commented on a change in pull request #20871: Fix failing main

2022-01-14 Thread GitBox
kaxil commented on a change in pull request #20871: URL: https://github.com/apache/airflow/pull/20871#discussion_r784776186 ## File path: airflow/api_connexion/endpoints/task_instance_endpoint.py ## @@ -326,7 +326,13 @@ def post_set_task_instances_state(*, dag_id: str,

[GitHub] [airflow] ephraimbuddy opened a new pull request #20871: Fix failing main

2022-01-14 Thread GitBox
ephraimbuddy opened a new pull request #20871: URL: https://github.com/apache/airflow/pull/20871 When I merged #18724 the jobs ran successfully but it's now failing in main. This PR fixes it --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] Bowrna commented on pull request #20848: Static check in Breeze2

2022-01-14 Thread GitBox
Bowrna commented on pull request #20848: URL: https://github.com/apache/airflow/pull/20848#issuecomment-1013026737 @potiuk I resolved the above issue and it works :) I could generate the python file during pre-commit process. I have added the python file generation part in the

[GitHub] [airflow] Bowrna edited a comment on pull request #20848: Static check in Breeze2

2022-01-14 Thread GitBox
Bowrna edited a comment on pull request #20848: URL: https://github.com/apache/airflow/pull/20848#issuecomment-1013026737 @potiuk I resolved the above issue and it works :) I could generate the python file during pre-commit process. I have added the python file generation part in the

[GitHub] [airflow] kaxil commented on a change in pull request #20743: Serialize mapped tasks and task groups

2022-01-14 Thread GitBox
kaxil commented on a change in pull request #20743: URL: https://github.com/apache/airflow/pull/20743#discussion_r784788806 ## File path: airflow/models/baseoperator.py ## @@ -1741,6 +1788,28 @@ def leaves(self) -> List["MappedOperator"]: def has_dag(self):

[GitHub] [airflow] eladkal commented on issue #20780: [Provider] Huawei Cloud: Object Storage Service

2022-01-14 Thread GitBox
eladkal commented on issue #20780: URL: https://github.com/apache/airflow/issues/20780#issuecomment-1012989236 @CelastrinaLadon assigned to you. Feel free to open PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] eladkal commented on issue #20396: Add separate name attribut for dag so that dag can be easily renamed

2022-01-14 Thread GitBox
eladkal commented on issue #20396: URL: https://github.com/apache/airflow/issues/20396#issuecomment-1012995249 Since this is AIP-36 related. i suggest to take it to the AIP comment page / relevant thread in the mailing list... We usualy dont discuss AIPs in github issues -- This is an

[GitHub] [airflow] uranusjr commented on a change in pull request #20870: Fix task ID deduplication in @task_group

2022-01-14 Thread GitBox
uranusjr commented on a change in pull request #20870: URL: https://github.com/apache/airflow/pull/20870#discussion_r784746579 ## File path: airflow/decorators/base.py ## @@ -86,15 +89,18 @@ def get_unique_task_id( if tg_task_id not in dag.task_ids: return

[GitHub] [airflow] kaxil commented on a change in pull request #20743: Serialize mapped tasks and task groups

2022-01-14 Thread GitBox
kaxil commented on a change in pull request #20743: URL: https://github.com/apache/airflow/pull/20743#discussion_r784758697 ## File path: airflow/models/baseoperator.py ## @@ -1672,20 +1677,52 @@ def _validate_kwarg_names_for_mapping(cls: Type[BaseOperator], func_name: str, v

[GitHub] [airflow] kaxil opened a new pull request #20872: Unpin ``cattrs``

2022-01-14 Thread GitBox
kaxil opened a new pull request #20872: URL: https://github.com/apache/airflow/pull/20872 This was pinned because of issue mentioned in https://github.com/apache/airflow/issues/16172 . However this was fixed in 1.8.0 of cattrs by https://github.com/python-attrs/cattrs/issues/151

[GitHub] [airflow] kaxil commented on a change in pull request #20871: Fix failing main

2022-01-14 Thread GitBox
kaxil commented on a change in pull request #20871: URL: https://github.com/apache/airflow/pull/20871#discussion_r784778808 ## File path: airflow/api_connexion/endpoints/task_instance_endpoint.py ## @@ -326,7 +326,13 @@ def post_set_task_instances_state(*, dag_id: str,

[GitHub] [airflow] github-actions[bot] commented on pull request #20872: Unpin ``cattrs``

2022-01-14 Thread GitBox
github-actions[bot] commented on pull request #20872: URL: https://github.com/apache/airflow/pull/20872#issuecomment-1013050672 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] ephraimbuddy closed issue #18672: Add `run_id` to this `/dags/{dag_id}/updateTaskInstancesState` API endpoint

2022-01-14 Thread GitBox
ephraimbuddy closed issue #18672: URL: https://github.com/apache/airflow/issues/18672 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy merged pull request #18724: Use `DagRun.run_id` instead of `execution_date` when updating state of TIs(UI & REST API)

2022-01-14 Thread GitBox
ephraimbuddy merged pull request #18724: URL: https://github.com/apache/airflow/pull/18724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (2b4bf7f -> 14ee831)

2022-01-14 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 2b4bf7f Use `DagRun.run_id` instead of `execution_date` when updating state of TIs(UI & REST API) (#18724)

[GitHub] [airflow] val2k edited a comment on issue #10790: Copy of [AIRFLOW-5071] JIRA: Thousands of Executor reports task instance X finished (success) although the task says its queued. Was the task

2022-01-14 Thread GitBox
val2k edited a comment on issue #10790: URL: https://github.com/apache/airflow/issues/10790#issuecomment-1012934214 We face the same issue with tasks that stay indefinitely in a queued status, except that we don't see tasks as `up_for_retry`. It happens randomly within our DAGs. The task

[GitHub] [airflow] val2k commented on issue #10790: Copy of [AIRFLOW-5071] JIRA: Thousands of Executor reports task instance X finished (success) although the task says its queued. Was the task killed

2022-01-14 Thread GitBox
val2k commented on issue #10790: URL: https://github.com/apache/airflow/issues/10790#issuecomment-1012934214 We face the same issue with tasks that stay indefinitely in a queued status, except that we don't see tasks as `up_for_retry`. It happens randomly within our DAGs. The task will

[GitHub] [airflow] ephraimbuddy merged pull request #19769: Handle stuck queued tasks in Celery for db backend

2022-01-14 Thread GitBox
ephraimbuddy merged pull request #19769: URL: https://github.com/apache/airflow/pull/19769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy closed issue #19699: task_instances stuck in "queued" and are missing corresponding celery_taskmeta entries

2022-01-14 Thread GitBox
ephraimbuddy closed issue #19699: URL: https://github.com/apache/airflow/issues/19699 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (AIRFLOW-5071) Thousand os Executor reports task instance X finished (success) although the task says its queued. Was the task killed externally?

2022-01-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17476036#comment-17476036 ] ASF GitHub Bot commented on AIRFLOW-5071: - val2k edited a comment on issue #10790: URL:

[jira] [Commented] (AIRFLOW-5071) Thousand os Executor reports task instance X finished (success) although the task says its queued. Was the task killed externally?

2022-01-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17476035#comment-17476035 ] ASF GitHub Bot commented on AIRFLOW-5071: - val2k commented on issue #10790: URL:

[GitHub] [airflow] mgorsk1 commented on issue #20849: Provide 'vanilla' Airflow Docker image

2022-01-14 Thread GitBox
mgorsk1 commented on issue #20849: URL: https://github.com/apache/airflow/issues/20849#issuecomment-1012899889 Thanks @potiuk! Let me know if I can be of any help with this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[airflow] branch main updated (a20d577 -> 2b4bf7f)

2022-01-14 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from a20d577 Fixing flaky task that fails in first check ocasionally (#19132) add 2b4bf7f Use

[GitHub] [airflow] kaxil opened a new issue #20880: Bump `jsonschema` version for Airflow 2.3

2022-01-14 Thread GitBox
kaxil opened a new issue #20880: URL: https://github.com/apache/airflow/issues/20880 ### Body We are using jsonschema 3.2.0 which is a 2 years old version. Based on https://github.com/Julian/jsonschema/blob/main/CHANGELOG.rst there might be the following change that might

[GitHub] [airflow] ashb commented on a change in pull request #20878: Unpin ``argcomplete`` and ``colorlog``

2022-01-14 Thread GitBox
ashb commented on a change in pull request #20878: URL: https://github.com/apache/airflow/pull/20878#discussion_r784971014 ## File path: setup.cfg ## @@ -81,14 +81,14 @@ setup_requires =

[GitHub] [airflow] kaxil closed issue #19641: EKSCreateNodegroupOperator - missing kwargs

2022-01-14 Thread GitBox
kaxil closed issue #19641: URL: https://github.com/apache/airflow/issues/19641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil merged pull request #20819: Adds support for optional kwargs in the EKS Operators

2022-01-14 Thread GitBox
kaxil merged pull request #20819: URL: https://github.com/apache/airflow/pull/20819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #20798: Add "Greater/Smaller than or Equal" to filters in the browse views

2022-01-14 Thread GitBox
boring-cyborg[bot] commented on pull request #20798: URL: https://github.com/apache/airflow/pull/20798#issuecomment-1013328482 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] bbovenzi merged pull request #20798: Add "Greater/Smaller than or Equal" to filters in the browse views

2022-01-14 Thread GitBox
bbovenzi merged pull request #20798: URL: https://github.com/apache/airflow/pull/20798 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (03b08aa -> e5c2384)

2022-01-14 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 03b08aa Fix Vault Hook default connection name (#20792) add e5c2384 Add "Greater/Smaller than or Equal" to

[GitHub] [airflow] kaxil merged pull request #20879: Remove redundant ``dataclass`` dependency

2022-01-14 Thread GitBox
kaxil merged pull request #20879: URL: https://github.com/apache/airflow/pull/20879 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil merged pull request #20878: Unpin ``argcomplete`` and ``colorlog``

2022-01-14 Thread GitBox
kaxil merged pull request #20878: URL: https://github.com/apache/airflow/pull/20878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] jedcunningham commented on a change in pull request #20391: Fix task instance concurrency limit in a DAG affecting other DAGs

2022-01-14 Thread GitBox
jedcunningham commented on a change in pull request #20391: URL: https://github.com/apache/airflow/pull/20391#discussion_r785071599 ## File path: airflow/jobs/scheduler_job.py ## @@ -289,8 +290,21 @@ def _executable_task_instances_to_queued(self, max_tis: int, session:

[GitHub] [airflow] SasanAhmadi commented on issue #16919: error when using mysql_to_s3 (TypeError: cannot safely cast non-equivalent object to int64)

2022-01-14 Thread GitBox
SasanAhmadi commented on issue #16919: URL: https://github.com/apache/airflow/issues/16919#issuecomment-1013424780 Hi Elad, just got a confirmation from my manager at Skillz to wrap this up in the next week. got a dedicated time from my job to finish this contribution. I am sorry it is

[GitHub] [airflow] khalidmammadov opened a new pull request #20881: Fix a test case inside tests/models that leaves a trace in the DB

2022-01-14 Thread GitBox
khalidmammadov opened a new pull request #20881: URL: https://github.com/apache/airflow/pull/20881 This fixes `test_sync_perm_for_dag` test case inside `test_dagbag.py` that leaves a record for a test DAG permission in the DB. It causes failure on another test case

[GitHub] [airflow] o-nikolas commented on a change in pull request #19022: CloudwatchTaskHandler params to create log group/stream

2022-01-14 Thread GitBox
o-nikolas commented on a change in pull request #19022: URL: https://github.com/apache/airflow/pull/19022#discussion_r785081536 ## File path: airflow/config_templates/airflow_local_settings.py ## @@ -194,13 +193,17 @@

[airflow] branch constraints-main updated: Updating constraints. Build id:1698955978

2022-01-14 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 7039cb8 Updating

[GitHub] [airflow] saisujithkp removed a comment on pull request #6799: [AIRFLOW-XXX] KubernetesPodOperator: fix _extract_xcom for larger json files

2022-01-14 Thread GitBox
saisujithkp removed a comment on pull request #6799: URL: https://github.com/apache/airflow/pull/6799#issuecomment-1013489871 We are facing a similar issue with Airflow Version 2.1.2 The program has a very large JSON object that should be inserted as an XCOM. But `cat

[GitHub] [airflow] kaxil edited a comment on issue #20876: Airflow database upgrade fails with "psycopg2.errors.NotNullViolation: column "map_index" of relation "task_instance" contains null value"s

2022-01-14 Thread GitBox
kaxil edited a comment on issue #20876: URL: https://github.com/apache/airflow/issues/20876#issuecomment-1013263547 Map Index hasn't been introduced in Airflow 2.2.3 and nor intended to, it is currently available in the main branch and will be released in Airflow 2.3. So it is very likely

[GitHub] [airflow] kaxil edited a comment on issue #20876: Airflow database upgrade fails with "psycopg2.errors.NotNullViolation: column "map_index" of relation "task_instance" contains null value"s

2022-01-14 Thread GitBox
kaxil edited a comment on issue #20876: URL: https://github.com/apache/airflow/issues/20876#issuecomment-1013263547 Map Index hasn't been introduced in Airflow 2.2.3 and nor intended to, it is currently available in the main branch and will be released in Airflow 2.3. So it is very likely

[GitHub] [airflow] kaxil commented on pull request #20879: Remove redundant ``dataclass`` dependency

2022-01-14 Thread GitBox
kaxil commented on pull request #20879: URL: https://github.com/apache/airflow/pull/20879#issuecomment-1013278085 > Duplicate of #20549 I changed this PR to include `dataclass` only which isn't covered in that PR and we don't have any code conditionals for it. -- This is an

[GitHub] [airflow] kaxil commented on a change in pull request #20878: Unpin ``argcomplete`` and ``colorlog``

2022-01-14 Thread GitBox
kaxil commented on a change in pull request #20878: URL: https://github.com/apache/airflow/pull/20878#discussion_r784984948 ## File path: setup.cfg ## @@ -81,14 +81,14 @@ setup_requires =

[GitHub] [airflow] o-nikolas commented on a change in pull request #20858: Move some base_aws logging from info to debug level

2022-01-14 Thread GitBox
o-nikolas commented on a change in pull request #20858: URL: https://github.com/apache/airflow/pull/20858#discussion_r785076651 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -199,8 +199,6 @@ def _read_credentials_from_connection(self) ->

[GitHub] [airflow] mik-laj commented on issue #20867: Ingress for Statsd

2022-01-14 Thread GitBox
mik-laj commented on issue #20867: URL: https://github.com/apache/airflow/issues/20867#issuecomment-1013459400 We already have defined annotations on service, so preometheus should detect stated service:

[GitHub] [airflow] kaxil commented on a change in pull request #20794: Helm Chart - support for priorityClassName

2022-01-14 Thread GitBox
kaxil commented on a change in pull request #20794: URL: https://github.com/apache/airflow/pull/20794#discussion_r785013493 ## File path: chart/values.yaml ## @@ -514,6 +514,7 @@ workers: # Select certain nodes for airflow worker pods. nodeSelector: {} +

[airflow] branch main updated (9157734 -> bbe1eff)

2022-01-14 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 9157734 Unpin ``argcomplete`` and ``colorlog`` (#20878) add bbe1eff Remove redundant ``dataclass``

[GitHub] [airflow] jedcunningham commented on a change in pull request #20794: Helm Chart - support for priorityClassName

2022-01-14 Thread GitBox
jedcunningham commented on a change in pull request #20794: URL: https://github.com/apache/airflow/pull/20794#discussion_r785063195 ## File path: chart/templates/flower/flower-deployment.yaml ## @@ -67,6 +67,7 @@ spec: tolerations: {{ toYaml $tolerations | indent 8 }}

[GitHub] [airflow] mik-laj commented on issue #20867: Ingress for Statsd

2022-01-14 Thread GitBox
mik-laj commented on issue #20867: URL: https://github.com/apache/airflow/issues/20867#issuecomment-1013460161 Here is section in Prometheus docs: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#service -- This is an automated message from the Apache Git

[GitHub] [airflow] ssatia commented on issue #20615: Status of testing Providers that were prepared on December 31, 2021

2022-01-14 Thread GitBox
ssatia commented on issue #20615: URL: https://github.com/apache/airflow/issues/20615#issuecomment-1013467246 #20264 works as expected. We had the unexpected occur again and it was handled correctly this time around  -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] kaxil edited a comment on issue #20876: Airflow database upgrade fails with "psycopg2.errors.NotNullViolation: column "map_index" of relation "task_instance" contains null value"s

2022-01-14 Thread GitBox
kaxil edited a comment on issue #20876: URL: https://github.com/apache/airflow/issues/20876#issuecomment-1013263547 Map Index hasn't been introduced in Airflow 2.2.3 and nor intended to, it is currently available in the main branch and will be released in Airflow 2.3. So it is very likely

[GitHub] [airflow] kaxil commented on a change in pull request #20878: Unpin ``argcomplete`` and ``colorlog``

2022-01-14 Thread GitBox
kaxil commented on a change in pull request #20878: URL: https://github.com/apache/airflow/pull/20878#discussion_r784984948 ## File path: setup.cfg ## @@ -81,14 +81,14 @@ setup_requires =

[airflow] branch main updated (8881458 -> 307d356)

2022-01-14 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 8881458 Unpin ``cattrs`` (#20872) add 307d356 AWS: Adds support for optional kwargs in the EKS Operators

[airflow] branch main updated (e5c2384 -> 9157734)

2022-01-14 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from e5c2384 Add "Greater/Smaller than or Equal" to filters in the browse views (#20602) (#20798) add 9157734

[GitHub] [airflow] saisujithkp commented on pull request #6799: [AIRFLOW-XXX] KubernetesPodOperator: fix _extract_xcom for larger json files

2022-01-14 Thread GitBox
saisujithkp commented on pull request #6799: URL: https://github.com/apache/airflow/pull/6799#issuecomment-1013489871 We are facing a similar issue with Airflow Version 2.1.2 The program has a very large JSON object that should be inserted as an XCOM. But `cat

[GitHub] [airflow] kaxil commented on a change in pull request #20878: Unpin ``argcomplete`` and ``colorlog``

2022-01-14 Thread GitBox
kaxil commented on a change in pull request #20878: URL: https://github.com/apache/airflow/pull/20878#discussion_r784984948 ## File path: setup.cfg ## @@ -81,14 +81,14 @@ setup_requires =

[GitHub] [airflow] jedcunningham commented on a change in pull request #20802: Only validate Params when DAG is triggered

2022-01-14 Thread GitBox
jedcunningham commented on a change in pull request #20802: URL: https://github.com/apache/airflow/pull/20802#discussion_r784989929 ## File path: airflow/models/dagbag.py ## @@ -398,6 +400,9 @@ def _process_modules(self, filepath, mods, file_last_changed_on_disk):

[GitHub] [airflow] jedcunningham merged pull request #20787: Chart: fix extra containers docs and remove krb excess if

2022-01-14 Thread GitBox
jedcunningham merged pull request #20787: URL: https://github.com/apache/airflow/pull/20787 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (bbe1eff -> 5ca5693)

2022-01-14 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from bbe1eff Remove redundant ``dataclass`` dependency (#20879) add 5ca5693 Chart: fix extra containers docs

[GitHub] [airflow] github-actions[bot] commented on pull request #20787: Chart: fix extra containers docs and remove krb excess if

2022-01-14 Thread GitBox
github-actions[bot] commented on pull request #20787: URL: https://github.com/apache/airflow/pull/20787#issuecomment-1013486363 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] ashb commented on a change in pull request #20878: Unpin ``argcomplete`` and ``colorlog``

2022-01-14 Thread GitBox
ashb commented on a change in pull request #20878: URL: https://github.com/apache/airflow/pull/20878#discussion_r784971210 ## File path: setup.cfg ## @@ -81,14 +81,14 @@ setup_requires =

[GitHub] [airflow] kaxil commented on issue #20862: Unexpected keyword argument 'resolver' during `airflow db init`

2022-01-14 Thread GitBox
kaxil commented on issue #20862: URL: https://github.com/apache/airflow/issues/20862#issuecomment-1013269706 > We should do that in our setup.py as well. Or, since the error came from the _vendor_ connexion, it may be possible to fix this in the Airflow code base. What’s our policy about

[GitHub] [airflow] github-actions[bot] commented on pull request #20879: Remove Python <3.7 only dependencies

2022-01-14 Thread GitBox
github-actions[bot] commented on pull request #20879: URL: https://github.com/apache/airflow/pull/20879#issuecomment-1013276042 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] fjmacagno commented on issue #20862: Unexpected keyword argument 'resolver' during `airflow db init`

2022-01-14 Thread GitBox
fjmacagno commented on issue #20862: URL: https://github.com/apache/airflow/issues/20862#issuecomment-1013320235 Adding a constraints file fixed the issue, but it sounds like this may still be something that needs to be handled? I will leave it open, feel free to close if it isn't. --

[GitHub] [airflow] bbovenzi closed issue #20602: Add greater/less than or equal to filters in the browse views

2022-01-14 Thread GitBox
bbovenzi closed issue #20602: URL: https://github.com/apache/airflow/issues/20602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #20882: [WIP] Add Looker PDT operators

2022-01-14 Thread GitBox
boring-cyborg[bot] commented on pull request #20882: URL: https://github.com/apache/airflow/pull/20882#issuecomment-1013350915 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] alekseiloginov opened a new pull request #20882: [WIP] Add Looker PDT operators

2022-01-14 Thread GitBox
alekseiloginov opened a new pull request #20882: URL: https://github.com/apache/airflow/pull/20882 WIP: Add Looker PDTs operator This PR adds new Airflow operators that make use of the upcoming Looker PDT Start/Stop API that will be released soon. These operators allow to manage

[GitHub] [airflow] mike-mcdonald opened a new pull request #20883: fix: Update custom connection field processing

2022-01-14 Thread GitBox
mike-mcdonald opened a new pull request #20883: URL: https://github.com/apache/airflow/pull/20883 Fixes issue where custom connection fields are not updated because `extra` field is in form and has previous values, overriding custom field values. Adds portion of connection form tests to

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #20883: fix: Update custom connection field processing

2022-01-14 Thread GitBox
boring-cyborg[bot] commented on pull request #20883: URL: https://github.com/apache/airflow/pull/20883#issuecomment-1013373302 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] ferruzzi commented on a change in pull request #20814: fix: cloudwatch logs fetch logic

2022-01-14 Thread GitBox
ferruzzi commented on a change in pull request #20814: URL: https://github.com/apache/airflow/pull/20814#discussion_r785094194 ## File path: airflow/providers/amazon/aws/hooks/logs.py ## @@ -99,7 +97,7 @@ def get_log_events( yield from events -if

[GitHub] [airflow] SamWheating commented on issue #19289: Block migrations on MySQL when using bad collations/encoding

2022-01-14 Thread GitBox
SamWheating commented on issue #19289: URL: https://github.com/apache/airflow/issues/19289#issuecomment-1013392308 We just got caught by exactly this issue when upgrading from 2.1.2 to 2.2.2. Can I take this issue on? I would love to have this check in place for my future self 

[GitHub] [airflow] dacort commented on issue #20215: EMR serverless, new operator

2022-01-14 Thread GitBox
dacort commented on issue #20215: URL: https://github.com/apache/airflow/issues/20215#issuecomment-1013392808 @yeshbash @ferruzzi If you haven't already, please sign up for the preview here: https://pages.awscloud.com/EMR-Serverless-Preview.html We've started to let folks in and you

[GitHub] [airflow] kaxil closed issue #20876: Airflow database upgrade fails with "psycopg2.errors.NotNullViolation: column "map_index" of relation "task_instance" contains null value"s

2022-01-14 Thread GitBox
kaxil closed issue #20876: URL: https://github.com/apache/airflow/issues/20876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil commented on issue #20876: Airflow database upgrade fails with "psycopg2.errors.NotNullViolation: column "map_index" of relation "task_instance" contains null value"s

2022-01-14 Thread GitBox
kaxil commented on issue #20876: URL: https://github.com/apache/airflow/issues/20876#issuecomment-1013263547 Map Index hasn't been introduced in Airflow 2.2.3 and nor intended to, it is currently available in the main branch and will be released in Airflow 2.3. So it is very likely you

[GitHub] [airflow] kaxil commented on issue #20877: Allow for Markup in UI page title

2022-01-14 Thread GitBox
kaxil commented on issue #20877: URL: https://github.com/apache/airflow/issues/20877#issuecomment-1013262404 Assigned the issue to you :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] github-actions[bot] commented on pull request #20802: Only validate Params when DAG is triggered

2022-01-14 Thread GitBox
github-actions[bot] commented on pull request #20802: URL: https://github.com/apache/airflow/pull/20802#issuecomment-1013285414 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #20792: Fix Vault default conn name

2022-01-14 Thread GitBox
boring-cyborg[bot] commented on pull request #20792: URL: https://github.com/apache/airflow/pull/20792#issuecomment-1013304635 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] kaxil merged pull request #20792: Fix Vault default conn name

2022-01-14 Thread GitBox
kaxil merged pull request #20792: URL: https://github.com/apache/airflow/pull/20792 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] github-actions[bot] commented on pull request #20878: Unpin ``argcomplete`` and ``colorlog``

2022-01-14 Thread GitBox
github-actions[bot] commented on pull request #20878: URL: https://github.com/apache/airflow/pull/20878#issuecomment-1013314696 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] jedcunningham commented on a change in pull request #20881: Fix a test case inside tests/models that leaves a trace in the DB

2022-01-14 Thread GitBox
jedcunningham commented on a change in pull request #20881: URL: https://github.com/apache/airflow/pull/20881#discussion_r785161712 ## File path: tests/models/test_dagbag.py ## @@ -837,6 +837,7 @@ def _sync_perms(): mock_sync_perm_for_dag.assert_called_once_with(

[GitHub] [airflow] SamWheating commented on issue #18939: Upgrading from version 2.1.4 to 2.2.0 fails during mysql db upgrade with error Can't DROP 'dag_id'

2022-01-14 Thread GitBox
SamWheating commented on issue #18939: URL: https://github.com/apache/airflow/issues/18939#issuecomment-1013463456 If anyone else is having this issue - we got into this state despite using the correct collation and encoding, here's what we found. In our case, the

[airflow] branch main updated: Fix Vault Hook default connection name (#20792)

2022-01-14 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 03b08aa Fix Vault Hook default connection name

[GitHub] [airflow] saisujithkp opened a new issue #20884: JSONDecodeError caused by incomplete JSON created by cat /airflow/xcom/return.json when using KubernetePodOperator

2022-01-14 Thread GitBox
saisujithkp opened a new issue #20884: URL: https://github.com/apache/airflow/issues/20884 ### Apache Airflow Provider(s) cncf-kubernetes ### Versions of Apache Airflow Providers _No response_ ### Apache Airflow version 2.1.2 ### Operating System

[GitHub] [airflow] jedcunningham edited a comment on issue #20873: Distinguish links in DAG documentation from code

2022-01-14 Thread GitBox
jedcunningham edited a comment on issue #20873: URL: https://github.com/apache/airflow/issues/20873#issuecomment-1013500431 What is your `doc_md`? I can only reproduce this with: ``` doc_md="Test `code block` and a [`link`](https://airflow.apache.org).", But

[GitHub] [airflow] SamWheating commented on a change in pull request #20733: Add Audit Log View to Dag View

2022-01-14 Thread GitBox
SamWheating commented on a change in pull request #20733: URL: https://github.com/apache/airflow/pull/20733#discussion_r785235303 ## File path: airflow/www/views.py ## @@ -3127,6 +3127,50 @@ def robots(self): """ return

[GitHub] [airflow] jedcunningham commented on issue #20873: Distinguish links in DAG documentation from code

2022-01-14 Thread GitBox
jedcunningham commented on issue #20873: URL: https://github.com/apache/airflow/issues/20873#issuecomment-1013500431 What is your `doc_md`? I can only reproduce this with: ``` doc_md="Test `code block` and a [`link`](https://airflow.apache.org).", But if I

[airflow] branch constraints-main updated: Updating constraints. Build id:1699673559

2022-01-14 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 426780f Updating

[GitHub] [airflow] cploonker commented on pull request #18675: New Tree View

2022-01-14 Thread GitBox
cploonker commented on pull request #18675: URL: https://github.com/apache/airflow/pull/18675#issuecomment-1013594704 Is it possible to include dependency lines into this view using something like https://observablehq.com/@erikbrinkman/d3-dag-topological

[GitHub] [airflow] SamWheating commented on a change in pull request #20733: Add Audit Log View to Dag View

2022-01-14 Thread GitBox
SamWheating commented on a change in pull request #20733: URL: https://github.com/apache/airflow/pull/20733#discussion_r785235303 ## File path: airflow/www/views.py ## @@ -3127,6 +3127,50 @@ def robots(self): """ return

[GitHub] [airflow] subkanthi commented on a change in pull request #20860: Branch python operator decorator

2022-01-14 Thread GitBox
subkanthi commented on a change in pull request #20860: URL: https://github.com/apache/airflow/pull/20860#discussion_r785250871 ## File path: airflow/decorators/branch_python.py ## @@ -0,0 +1,100 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] subkanthi commented on a change in pull request #20860: Branch python operator decorator

2022-01-14 Thread GitBox
subkanthi commented on a change in pull request #20860: URL: https://github.com/apache/airflow/pull/20860#discussion_r785250956 ## File path: tests/decorators/test_branch_python.py ## @@ -0,0 +1,116 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] edithturn commented on pull request #20701: [WIP] Check free space python

2022-01-14 Thread GitBox
edithturn commented on pull request #20701: URL: https://github.com/apache/airflow/pull/20701#issuecomment-1013526470 @potiuk they are working: airflow-freespace in **test** and **build images** workflows are working on my repo (main branch). These workflows failed in another step.

[GitHub] [airflow] subkanthi commented on a change in pull request #20860: Branch python operator decorator

2022-01-14 Thread GitBox
subkanthi commented on a change in pull request #20860: URL: https://github.com/apache/airflow/pull/20860#discussion_r785252124 ## File path: tests/decorators/test_branch_python.py ## @@ -0,0 +1,116 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] ephraimbuddy commented on issue #14672: Tasks intermittently get terminated with SIGTERM on K8s executor

2022-01-14 Thread GitBox
ephraimbuddy commented on issue #14672: URL: https://github.com/apache/airflow/issues/14672#issuecomment-1013622147 @stroykova can you create a separate issue for the behaviour you experienced. It looks different from other issues here because you're using multiprocessing in your code

[GitHub] [airflow] SamWheating commented on a change in pull request #20733: Add Audit Log View to Dag View

2022-01-14 Thread GitBox
SamWheating commented on a change in pull request #20733: URL: https://github.com/apache/airflow/pull/20733#discussion_r785236325 ## File path: airflow/config_templates/default_airflow.cfg ## @@ -658,6 +658,16 @@ auto_refresh_interval = 3 # Boolean for displaying warning for

[GitHub] [airflow] sandeepsaimone edited a comment on issue #14672: Tasks intermittently get terminated with SIGTERM on K8s executor

2022-01-14 Thread GitBox
sandeepsaimone edited a comment on issue #14672: URL: https://github.com/apache/airflow/issues/14672#issuecomment-1013568265 Hi Guys, I am running Mwaa version 2.0.2 and the dag run more than 5hrs, but in the end the task are marked has failed and there is no error in the logs.

[GitHub] [airflow] sandeepsaimone commented on issue #14672: Tasks intermittently get terminated with SIGTERM on K8s executor

2022-01-14 Thread GitBox
sandeepsaimone commented on issue #14672: URL: https://github.com/apache/airflow/issues/14672#issuecomment-1013568265 Hi Guys, I am running Mwaa version 2.0.2 and the dag run more than 5hrs, but in the end the task are marked has failed and there is no error in the logs. Below are

  1   2   >