[GitHub] [airflow] edwardwang888 commented on issue #16877: Cleared task instances in manual runs should have borders

2021-09-07 Thread GitBox
edwardwang888 commented on issue #16877: URL: https://github.com/apache/airflow/issues/16877#issuecomment-914925020 @bbovenzi The PR should be ready now. Let me know if you have any comments. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] github-actions[bot] commented on pull request #18076: Adding CI check to verify __init__.py file exists in providers' example_dags directories

2021-09-07 Thread GitBox
github-actions[bot] commented on pull request #18076: URL: https://github.com/apache/airflow/pull/18076#issuecomment-914912656 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18077: Unable to determine underlying reason for email alert send failure

2021-09-07 Thread GitBox
boring-cyborg[bot] commented on issue #18077: URL: https://github.com/apache/airflow/issues/18077#issuecomment-914890851 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] kolban-google opened a new issue #18077: Unable to determine underlying reason for email alert send failure

2021-09-07 Thread GitBox
kolban-google opened a new issue #18077: URL: https://github.com/apache/airflow/issues/18077 ### Apache Airflow version 2.1.2 ### Operating System Debian ### Versions of Apache Airflow Providers Google Composer ### Deployment Composer

[GitHub] [airflow] uranusjr commented on issue #18050: Duplicating the same connection twice gives "Integrity error, probably unique constraint"

2021-09-07 Thread GitBox
uranusjr commented on issue #18050: URL: https://github.com/apache/airflow/issues/18050#issuecomment-914872094 Please do! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] subkanthi commented on issue #18050: Duplicating the same connection twice gives "Integrity error, probably unique constraint"

2021-09-07 Thread GitBox
subkanthi commented on issue #18050: URL: https://github.com/apache/airflow/issues/18050#issuecomment-914859881 @potiuk can I take this please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] josh-fell commented on pull request #17421: Add ShortCircuitOperator configurability for respecting downstream trigger rules

2021-09-07 Thread GitBox
josh-fell commented on pull request #17421: URL: https://github.com/apache/airflow/pull/17421#issuecomment-914802601 Rebased and resolved conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] josh-fell opened a new pull request #18076: Adding CI check to verify __init__.py file exists in providers' example_dags directories

2021-09-07 Thread GitBox
josh-fell opened a new pull request #18076: URL: https://github.com/apache/airflow/pull/18076 Two of the last three provider releases have had missing `__init__.py` files in the `example_dags` directory making these DAGs not part of an explicit namespace package. Based on [a previous PR

[GitHub] [airflow] uranusjr commented on issue #18075: Airflow installation fails when with latest `setuptools`

2021-09-07 Thread GitBox
uranusjr commented on issue #18075: URL: https://github.com/apache/airflow/issues/18075#issuecomment-914789476 Looks like this should be submitted to https://github.com/mitsuhiko/flask-openid/issues/59 instead. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] edgarrmondragon opened a new issue #18075: Airflow installation fails when with latest `setuptools`

2021-09-07 Thread GitBox
edgarrmondragon opened a new issue #18075: URL: https://github.com/apache/airflow/issues/18075 ### Apache Airflow version 2.1.2 ### Operating System Ubuntu 20.04 ### Versions of Apache Airflow Providers NA ### Deployment Other Docker-based

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18075: Airflow installation fails when with latest `setuptools`

2021-09-07 Thread GitBox
boring-cyborg[bot] commented on issue #18075: URL: https://github.com/apache/airflow/issues/18075#issuecomment-914780147 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] wolfier commented on pull request #18073: Fix BigQuery data extraction in BigQueryToMySqlOperator

2021-09-07 Thread GitBox
wolfier commented on pull request #18073: URL: https://github.com/apache/airflow/pull/18073#issuecomment-914748758 My bad! The tests I ran skipped the flake8 test for some reason :( I made sure to run the test for the test file explicitly and it did pass 爛 -- This is an automated

[GitHub] [airflow] mik-laj commented on pull request #18073: Fix BigQuery data extraction in BigQueryToMySqlOperator

2021-09-07 Thread GitBox
mik-laj commented on pull request #18073: URL: https://github.com/apache/airflow/pull/18073#issuecomment-914730384 @wolfier Static checks are sad  ``` tests/providers/google/cloud/transfers/test_bigquery_to_mysql.py:43:17: E126 continuation line over-indented for hanging indent

[GitHub] [airflow] github-actions[bot] commented on issue #13925: Duplicate entry with MySQL backend (AriFlow 2.0.0)

2021-09-07 Thread GitBox
github-actions[bot] commented on issue #13925: URL: https://github.com/apache/airflow/issues/13925#issuecomment-914716838 This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further

[GitHub] [airflow] github-actions[bot] closed pull request #16821: Add more filter options to list_keys of S3Hook

2021-09-07 Thread GitBox
github-actions[bot] closed pull request #16821: URL: https://github.com/apache/airflow/pull/16821 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] github-actions[bot] closed pull request #16647: Move FABs base Security Manager into Airflow.

2021-09-07 Thread GitBox
github-actions[bot] closed pull request #16647: URL: https://github.com/apache/airflow/pull/16647 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] wolfier commented on pull request #18073: Fix BigQuery data extraction in BigQueryToMySqlOperator

2021-09-07 Thread GitBox
wolfier commented on pull request #18073: URL: https://github.com/apache/airflow/pull/18073#issuecomment-914709800 Good call! I have updated the test and made sure it works locally -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] collinmcnulty commented on pull request #18071: Change from dynamic date to fixed date in examples

2021-09-07 Thread GitBox
collinmcnulty commented on pull request #18071: URL: https://github.com/apache/airflow/pull/18071#issuecomment-914688800 I'm fairly sure the remaining test failures are not due to the code changes in this PR. -- This is an automated message from the Apache Git Service. To respond to the

[airflow] branch main updated: Change id collation for MySQL to case-sensitive (#18072)

2021-09-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new b658a42 Change id collation for MySQL to

[GitHub] [airflow] potiuk merged pull request #18072: Change id collation for MySQL to case-sensitive

2021-09-07 Thread GitBox
potiuk merged pull request #18072: URL: https://github.com/apache/airflow/pull/18072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk closed issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
potiuk closed issue #17897: URL: https://github.com/apache/airflow/issues/17897 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] github-actions[bot] commented on pull request #18072: Change id collation for MySQL to case-sensitive

2021-09-07 Thread GitBox
github-actions[bot] commented on pull request #18072: URL: https://github.com/apache/airflow/pull/18072#issuecomment-914683841 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] mik-laj commented on pull request #18073: Fix BigQuery data extraction in BigQueryToMySqlOperator

2021-09-07 Thread GitBox
mik-laj commented on pull request #18073: URL: https://github.com/apache/airflow/pull/18073#issuecomment-914680463 Can you also update tests? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[airflow] branch v2-2-test updated (fe34582 -> 79d8557)

2021-09-07 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch v2-2-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from fe34582 New google operator: SQLToGoogleSheetsOperator (#17887) add 0791719 Add support for kinit

[airflow] branch constraints-2-2 updated (6610e2d -> 17cff54)

2021-09-07 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch constraints-2-2 in repository https://gitbox.apache.org/repos/asf/airflow.git. from 6610e2d Updating constraints. Build id:1161365036 add 4ed9e3f Updating constraints. Build

[GitHub] [airflow] dimon222 edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914671906 @potiuk Well, when I briefly read collation docs I was kind of similar feeling that they went nuts on this. I will avoid going much offtop, but in my opinion there's

[GitHub] [airflow] dimon222 edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914671906 @potiuk Well, when I briefly read collation docs I was kind of similar feeling that they went nuts on this. I will avoid going much offtop, but in my opinion there's

[GitHub] [airflow] dimon222 commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914671906 @potiuk Well, when I briefly read collation docs I was kind of similar feeling that they went nuts on this. I will avoid going much offtop, but in my opinion there's a

[GitHub] [airflow] potiuk commented on pull request #18072: Change id collation for MySQL to case-sensitive

2021-09-07 Thread GitBox
potiuk commented on pull request #18072: URL: https://github.com/apache/airflow/pull/18072#issuecomment-914668029 BTW. It's generally green (just mssql failure). Would love to merge it and forget about it. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on pull request #18072: Change id collation for MySQL to case-sensitive

2021-09-07 Thread GitBox
potiuk commented on pull request #18072: URL: https://github.com/apache/airflow/pull/18072#issuecomment-914666848 This is the last nail. Please. Let the MySQL encoding madness end ... For anyone interested in my rant about the MySQL encoding, here:

[GitHub] [airflow] potiuk commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
potiuk commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914664135 > PS: solution works for both mariadb and mysql Great you checked - I also already saw that Our CI tests pass with utf8mb3_bin. Great that you found it before we made it a

[GitHub] [airflow] binhnefits opened a new issue #18074: AzureContainerInstancesOperator is not working due to argument error

2021-09-07 Thread GitBox
binhnefits opened a new issue #18074: URL: https://github.com/apache/airflow/issues/18074 ### Apache Airflow Provider(s) microsoft-azure ### Versions of Apache Airflow Providers 3.1.0 ### Apache Airflow version 2.1.3 (latest released) ### Operating

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18074: AzureContainerInstancesOperator is not working due to argument error

2021-09-07 Thread GitBox
boring-cyborg[bot] commented on issue #18074: URL: https://github.com/apache/airflow/issues/18074#issuecomment-914660496 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] tnyz commented on pull request #17998: Fix error when create external table using table resource

2021-09-07 Thread GitBox
tnyz commented on pull request #17998: URL: https://github.com/apache/airflow/pull/17998#issuecomment-914653474 I have not closely examine other operators but they might suffer from the same issue too -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] dimon222 commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914653325 @potiuk I would highly recommend to also add test case for such scenario... Just incase. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] tnyz commented on pull request #17998: Fix error when create external table using table resource

2021-09-07 Thread GitBox
tnyz commented on pull request #17998: URL: https://github.com/apache/airflow/pull/17998#issuecomment-914653135 @potiuk I agree, please review the updated pr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] wolfier opened a new pull request #18073: Fix BigQuery data extraction in BigQueryToMySqlOperator

2021-09-07 Thread GitBox
wolfier opened a new pull request #18073: URL: https://github.com/apache/airflow/pull/18073 closes: #17198 Uses the list_rows function from BigQueryHook to query data. If there no more rows, job is finished. -- This is an automated message from the Apache Git Service.

[GitHub] [airflow] potiuk commented on pull request #18068: Switch dumb-init to propagate signals to child process only

2021-09-07 Thread GitBox
potiuk commented on pull request #18068: URL: https://github.com/apache/airflow/pull/18068#issuecomment-914646284 Might be we will fix it only for 2.2. That will be far safer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] dimon222 edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914644779 Since table for me was created long back in Airflow 1 times, the option for collation in airflow.cfg wasn't much useful, as something have to convert all the existing

[GitHub] [airflow] potiuk commented on pull request #18068: Switch dumb-init to propagate signals to child process only

2021-09-07 Thread GitBox
potiuk commented on pull request #18068: URL: https://github.com/apache/airflow/pull/18068#issuecomment-914646015 Yeah. As expected - need to fix the tests first :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] dimon222 edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914644779 Since table for me was created long back in Airflow 1 times, the option for collation in airflow.cfg wasn't much useful, as something have to convert all the existing

[GitHub] [airflow] dimon222 edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914644779 Since table for me was created back then, the option for collation in airflow.cfg wasn't much useful, as something have to convert all the indexes. I have done more

[GitHub] [airflow] dimon222 commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914644779 Since table for me was created back then, the option for collation in airflow.cfg wasn't much useful, as something have to convert all the indexes. I have done more tests

[GitHub] [airflow] potiuk commented on pull request #17998: Fix error when create external table using table resource

2021-09-07 Thread GitBox
potiuk commented on pull request #17998: URL: https://github.com/apache/airflow/pull/17998#issuecomment-914642118 Ah I see now. I have not looked that far.I am not sure though if that is the **right** fix, I think - looking at the code deeper, is that this is really a mixture of old and

[GitHub] [airflow] potiuk commented on pull request #18067: Update non-working example in documentation

2021-09-07 Thread GitBox
potiuk commented on pull request #18067: URL: https://github.com/apache/airflow/pull/18067#issuecomment-914625069 I think it will fail because of tabs rather than spaces :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] github-actions[bot] commented on pull request #18067: Update non-working example in documentation

2021-09-07 Thread GitBox
github-actions[bot] commented on pull request #18067: URL: https://github.com/apache/airflow/pull/18067#issuecomment-914625011 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide

[GitHub] [airflow] potiuk commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
potiuk commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914623151 Added #18072 - I will let it run all tests (part of our CI is to create MySQL DB with the collation) and see if it works. -- This is an automated message from the Apache Git

[GitHub] [airflow] potiuk opened a new pull request #18072: Change id collation for MySQL to case-sensitive

2021-09-07 Thread GitBox
potiuk opened a new pull request #18072: URL: https://github.com/apache/airflow/pull/18072 For quite some time we recommended MySQL collation to be utf8mb3_general_ci in order to avoid too-large-index size. Turns out that this collation is case-insensitive (that's where ci

[GitHub] [airflow] potiuk edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
potiuk edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914621919 Thanks! Also for being persistent. Glad that (I think) we know the problem now. I am adding a PR to fix our docs with it! -- This is an automated message from the Apache

[GitHub] [airflow] potiuk commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
potiuk commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914621919 Thanks! For being persistent. Glad that (I think) we know the problem now. I am adding a PR to fix our docs with it! -- This is an automated message from the Apache Git Service.

[GitHub] [airflow] collinmcnulty opened a new pull request #18071: Change from dynamic date to fixed date in examples

2021-09-07 Thread GitBox
collinmcnulty opened a new pull request #18071: URL: https://github.com/apache/airflow/pull/18071 The example dags all use a dynamic start date, but this is not good practice. In my experience, many people new to Airflow continue to use dynamic start dates in their real dags and it causes

[GitHub] [airflow] jedcunningham commented on pull request #17445: Add info log how to fix: More than one pod running with labels

2021-09-07 Thread GitBox
jedcunningham commented on pull request #17445: URL: https://github.com/apache/airflow/pull/17445#issuecomment-914616883 #18070 will fix the underlying issue allowing `is_delete_operator_pod=False` and `reattach_on_restart=True` to function together properly during retries. -- This

[GitHub] [airflow] potiuk edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
potiuk edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914609904 All right . It seems that MySQL is worse than I thought as they have no good utf8 case sensitive collation (!)

[GitHub] [airflow] dimon222 edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914610246 > You likely have this set as `sql_engine_collation_for_ids=utf8mb3_general_ci` parameter in airflow.cfg Worse, it's commented away in my config :D Perhaps was

[GitHub] [airflow] jedcunningham opened a new pull request #18070: Fix KubernetesPodOperator reattach when not deleting pods

2021-09-07 Thread GitBox
jedcunningham opened a new pull request #18070: URL: https://github.com/apache/airflow/pull/18070 This change: - ignores any pods already marked as complete instead of trying to reattach to them. - properly marks all 'finished' unsuccessful pods that won't be deleted.

[GitHub] [airflow] dimon222 commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914610246 > You likely have this set as `sql_engine_collation_for_ids=utf8mb3_general_ci` parameter in airflow.cfg Worse, it's commented away in my config :D Perhaps was missed

[GitHub] [airflow] potiuk commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
potiuk commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914609904 All right . It seems that MySQL is worse than I thought as they have no good utf8 case sensitive collation (!)

[GitHub] [airflow] potiuk commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
potiuk commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914607025 https://airflow.apache.org/docs/apache-airflow/stable/howto/set-up-database.html?highlight=collation you can find it here -- This is an automated message from the Apache Git

[GitHub] [airflow] potiuk commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
potiuk commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914606789 Ah. That's actually great finding. I had no idea ci stands for case-insensitive !. We do have utf8mb3_general_ci mentioned in the docs as workaround to set for

[GitHub] [airflow] rachelwigell opened a new issue #18069: Move auto-refresh interval to config variable

2021-09-07 Thread GitBox
rachelwigell opened a new issue #18069: URL: https://github.com/apache/airflow/issues/18069 ### Description Auto-refresh is an awesome new feature! Right now it occurs every [3 seconds](https://github.com/astronomer/airflow/blob/main/airflow/www/static/js/tree.js#L463), and this

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18069: Move auto-refresh interval to config variable

2021-09-07 Thread GitBox
boring-cyborg[bot] commented on issue #18069: URL: https://github.com/apache/airflow/issues/18069#issuecomment-914602138 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] tnyz commented on pull request #17998: Fix error when create external table using table resource

2021-09-07 Thread GitBox
tnyz commented on pull request #17998: URL: https://github.com/apache/airflow/pull/17998#issuecomment-914596659 @potiuk to be more specific, this is the block when executing the command from table resource, using None will result in `TableReference.from_string(None)` right? ```

[GitHub] [airflow] frankbreetz closed pull request #18067: Update non-working example in documentation

2021-09-07 Thread GitBox
frankbreetz closed pull request #18067: URL: https://github.com/apache/airflow/pull/18067 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #15604: BigQueryCreateExternalTableOperator table_resource parameter is impossible to use

2021-09-07 Thread GitBox
potiuk commented on issue #15604: URL: https://github.com/apache/airflow/issues/15604#issuecomment-914590987 Fixed in here: https://github.com/apache/airflow/pull/17900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] potiuk edited a comment on issue #15604: BigQueryCreateExternalTableOperator table_resource parameter is impossible to use

2021-09-07 Thread GitBox
potiuk edited a comment on issue #15604: URL: https://github.com/apache/airflow/issues/15604#issuecomment-914589782 This is already fixed in 5.1.0 version of Airflow Google provider. You can upgrade the provider to that version and use it. Also in case you use an earlier version of the

[GitHub] [airflow] dimon222 edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914589729 Upon further research I have found term `collate` that seems to be set to case insensitive `utf8_general_ci` by default for connection, database and server. I'm currently

[GitHub] [airflow] potiuk closed issue #15604: BigQueryCreateExternalTableOperator table_resource parameter is impossible to use

2021-09-07 Thread GitBox
potiuk closed issue #15604: URL: https://github.com/apache/airflow/issues/15604 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #15604: BigQueryCreateExternalTableOperator table_resource parameter is impossible to use

2021-09-07 Thread GitBox
potiuk commented on issue #15604: URL: https://github.com/apache/airflow/issues/15604#issuecomment-914589782 This is already fixed in 5.1.0 version of Airflow provider. You can upgrade the provider to that version and use it. Also in case you use an earlier version of the provider, you

[GitHub] [airflow] dimon222 commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914589729 Upon further research I have term `collate` that seems to be set to case insensitive `utf8_general_ci` by default for connection, database and server. I'm currently checking why

[GitHub] [airflow] potiuk commented on pull request #18067: Update non-working example in documentation

2021-09-07 Thread GitBox
potiuk commented on pull request #18067: URL: https://github.com/apache/airflow/pull/18067#issuecomment-914587217 > @potiuk You are right, that is better, I tested that and it works as expected. shall you amend the changes? -- This is an automated message from the Apache Git

[GitHub] [airflow] potiuk commented on pull request #17998: Fix error when create external table using table resource

2021-09-07 Thread GitBox
potiuk commented on pull request #17998: URL: https://github.com/apache/airflow/pull/17998#issuecomment-914586819 There were some `default` values in the past google provider that was fixed in 5.1.0. The default values were set for few parameters (where they should not be if

[GitHub] [airflow] frankbreetz commented on pull request #18067: Update non-working example in documentation

2021-09-07 Thread GitBox
frankbreetz commented on pull request #18067: URL: https://github.com/apache/airflow/pull/18067#issuecomment-914582882 @potiuk You are right, that is better, I tested that and it works as expected. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] dimon222 edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914582333 > Just one more question - can you see the same if you just "change" the tag (i.e. no case-sensitive-only change) "test" -> "test1" for example? Since we cannot reproduce

[GitHub] [airflow] dimon222 edited a comment on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 edited a comment on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914582333 > Just one more question - can you see the same if you just "change" the tag (i.e. no case-sensitive-only change) "test" -> "test1" for example? Since we cannot reproduce

[GitHub] [airflow] dimon222 commented on issue #17897: Dag tags are not refreshing if case sensitivity changed

2021-09-07 Thread GitBox
dimon222 commented on issue #17897: URL: https://github.com/apache/airflow/issues/17897#issuecomment-914582333 > Just one more question - can you see the same if you just "change" the tag (i.e. no case-sensitive-only change) "test" -> "test1" for example? Since we cannot reproduce it, I

[GitHub] [airflow] potiuk commented on issue #17962: Warn if robots.txt is accessed

2021-09-07 Thread GitBox
potiuk commented on issue #17962: URL: https://github.com/apache/airflow/issues/17962#issuecomment-914580057 Feel free! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk commented on pull request #18067: Update non-working example in documentation

2021-09-07 Thread GitBox
potiuk commented on pull request #18067: URL: https://github.com/apache/airflow/pull/18067#issuecomment-914578495 I think better fix will be to add `*` in the example. I believe `chain(*[DummyOperator(task_id='op' + i) for i in range(1, 6)])` should work -- This is an

[GitHub] [airflow] potiuk commented on pull request #18068: Switch dumb-init to propagate signals to child process only

2021-09-07 Thread GitBox
potiuk commented on pull request #18068: URL: https://github.com/apache/airflow/pull/18068#issuecomment-914558494 > Not yet. I was waiting for the results of CI tests - they include upgrading the helm chart in various scenarios I believe, but yeah. I am going to start it locally to test

[GitHub] [airflow] potiuk commented on pull request #18068: Switch dumb-init to propagate signals to child process only

2021-09-07 Thread GitBox
potiuk commented on pull request #18068: URL: https://github.com/apache/airflow/pull/18068#issuecomment-914557739 > @potiuk, have you verified that other components (and all executors) exit on their own without getting SIGKILL'd? Not yet. I was waiting for the results of CI tests -

[GitHub] [airflow] jedcunningham commented on a change in pull request #18068: Switch dumb-init to propagate signals to child process only

2021-09-07 Thread GitBox
jedcunningham commented on a change in pull request #18068: URL: https://github.com/apache/airflow/pull/18068#discussion_r703753568 ## File path: docs/docker-stack/entrypoint.rst ## @@ -161,6 +161,40 @@ If there are any other arguments - they are simply passed to the

[GitHub] [airflow] potiuk opened a new pull request #18068: Switch dumb-init to propagate signals to child process only

2021-09-07 Thread GitBox
potiuk opened a new pull request #18068: URL: https://github.com/apache/airflow/pull/18068 Default behaviour of dumb-init is to propagate signals to the process group rather than to the single child it uses. This is protective behaviour, in case a user runs 'bash -c' command

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #18067: Update non-working example in documentation

2021-09-07 Thread GitBox
boring-cyborg[bot] commented on pull request #18067: URL: https://github.com/apache/airflow/pull/18067#issuecomment-914531126 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] frankbreetz opened a new pull request #18067: Update non-working example in documentation

2021-09-07 Thread GitBox
frankbreetz opened a new pull request #18067: URL: https://github.com/apache/airflow/pull/18067 Chain function cannot accept a list, but the example provided supplies with a list. Supplied example did not create chain dependencies as expected Replaced non-working example with working

[GitHub] [airflow] jedcunningham commented on a change in pull request #17236: [Airflow 16364] Add conn_timeout and cmd_timeout params to SSHOperator; add conn_timeout param to SSHHook

2021-09-07 Thread GitBox
jedcunningham commented on a change in pull request #17236: URL: https://github.com/apache/airflow/pull/17236#discussion_r703718249 ## File path: tests/providers/ssh/operators/test_ssh.py ## @@ -138,7 +160,9 @@ def test_no_output_command(self): def

[GitHub] [airflow] ordonezf edited a comment on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
ordonezf edited a comment on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914500677 It seems to work  ```bash k logs -f airflow-worker-57c7dcc9f9-fhqgz * Serving Flask app "airflow.utils.serve_logs" (lazy loading) * Environment:

[GitHub] [airflow] potiuk commented on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
potiuk commented on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914502808 I will update it and I think it might make sense to even backport it to 2.1.4. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] fatmumuhomer commented on a change in pull request #17236: [Airflow 16364] Add conn_timeout and cmd_timeout params to SSHOperator; add conn_timeout param to SSHHook

2021-09-07 Thread GitBox
fatmumuhomer commented on a change in pull request #17236: URL: https://github.com/apache/airflow/pull/17236#discussion_r703713799 ## File path: tests/providers/ssh/operators/test_ssh.py ## @@ -138,7 +160,9 @@ def test_no_output_command(self): def test_arg_checking(self):

[GitHub] [airflow] jedcunningham commented on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
jedcunningham commented on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914500874 This is why everything goes through bash (bug in older version images): https://github.com/apache/airflow/pull/13526 -- This is an automated message from the Apache Git

[GitHub] [airflow] ordonezf commented on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
ordonezf commented on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914500677 It seems to work  ```bash k logs -f airflow-worker-57c7dcc9f9-fhqgz * Serving Flask app "airflow.utils.serve_logs" (lazy loading) * Environment: production

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #18065: Limit the number of queued dagruns created by the Scheduler

2021-09-07 Thread GitBox
ephraimbuddy commented on a change in pull request #18065: URL: https://github.com/apache/airflow/pull/18065#discussion_r703709458 ## File path: airflow/jobs/scheduler_job.py ## @@ -823,14 +823,30 @@ def _create_dag_runs(self, dag_models: Iterable[DagModel], session: Session)

[GitHub] [airflow] potiuk edited a comment on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
potiuk edited a comment on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914496541 BTW. why do we have [bash -c exec ''] instead of just `airflow celery` (It would not work the same way and in both cases we need --`--single-child` but would not need to

[GitHub] [airflow] potiuk edited a comment on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
potiuk edited a comment on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914496541 BTW. why do we have [bash -c exec ''] instead of just `airflow celery` (It would not work the same way and in both cases we need --`--single-child` but would not need to

[GitHub] [airflow] potiuk commented on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
potiuk commented on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914496541 BTW. why do we have [bash -c exec ''] instead of just `airflow celery` (It would not work the same way and in both cases we need --`--single-child` but would not need to start bash

[GitHub] [airflow] jedcunningham commented on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
jedcunningham commented on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914494723 Works for me. At one point KubernetesExecutor relied on it to exit cleanly, but that is now fixed and I'm not aware of any remaining need. We'd have to test to make sure.

[GitHub] [airflow] potiuk commented on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
potiuk commented on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914490873 Just wonder if we had a reason of leaving the default behaviour of dumb-init, but I see no reason if we confirm that this works WDYT @jedcunningham ? -- This is an automated

[GitHub] [airflow] SamWheating commented on issue #18023: Many queued Dagruns created for DagRuns with a start_date in the past

2021-09-07 Thread GitBox
SamWheating commented on issue #18023: URL: https://github.com/apache/airflow/issues/18023#issuecomment-914490044 After further investigation, I think that most of the performance issues we saw were due to the issue now resolved by https://github.com/apache/airflow/pull/17945. If

[GitHub] [airflow] laserpedro commented on issue #17507: Task processes killed with WARNING - Recorded pid does not match the current pid

2021-09-07 Thread GitBox
laserpedro commented on issue #17507: URL: https://github.com/apache/airflow/issues/17507#issuecomment-914489342 Is there any workaround since my dags are no longer working properly ? tks -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] potiuk commented on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
potiuk commented on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914488115 @ordonezf - can you set `DUMB_INIT_SETSID=0` as environment variable and see if that works ? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
potiuk commented on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914487236 just read the same docs :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on issue #18066: Chart version 1.1.0 does not gracefully shutdown workers

2021-09-07 Thread GitBox
potiuk commented on issue #18066: URL: https://github.com/apache/airflow/issues/18066#issuecomment-914487122 Yeah DUMB_INIT_SETSID=0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

  1   2   3   >