[airflow] branch main updated: Drop Python 3.6 compatibility objects/modules (#24048)

2022-05-31 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1dccaad46b Drop Python 3.6 compatibility

[GitHub] [airflow] uranusjr merged pull request #24048: Drop Python 3.6 compatibility objects/modules

2022-05-31 Thread GitBox
uranusjr merged PR #24048: URL: https://github.com/apache/airflow/pull/24048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] uranusjr commented on a diff in pull request #23971: Unify DbApiHook.run() method with the methods which override it

2022-05-31 Thread GitBox
uranusjr commented on code in PR #23971: URL: https://github.com/apache/airflow/pull/23971#discussion_r886324507 ## airflow/hooks/dbapi.py: ## @@ -188,25 +206,21 @@ def run(self, sql, autocommit=False, parameters=None, handler=None): self.set_autocommit(conn,

[GitHub] [airflow] uranusjr commented on a diff in pull request #23971: Unify DbApiHook.run() method with the methods which override it

2022-05-31 Thread GitBox
uranusjr commented on code in PR #23971: URL: https://github.com/apache/airflow/pull/23971#discussion_r886324138 ## airflow/hooks/dbapi.py: ## @@ -188,25 +206,21 @@ def run(self, sql, autocommit=False, parameters=None, handler=None): self.set_autocommit(conn,

[GitHub] [airflow] nirutgupta commented on issue #20779: Logs of tasks when running is not available with kubernetesexecutor on webserver UI

2022-05-31 Thread GitBox
nirutgupta commented on issue #20779: URL: https://github.com/apache/airflow/issues/20779#issuecomment-1143110095 I created a new directory custom_log made it available under that. ``` RUN mkdir ${AIRFLOW_USER_HOME}/custom_log COPY config/custom_log/k8s_task_handler.py

[airflow] branch main updated: Fix StatD timing metric units (#21106)

2022-05-31 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1507ca48d7 Fix StatD timing metric units

[GitHub] [airflow] uranusjr closed issue #20804: Some timing metrics are in seconds but reported as milliseconds

2022-05-31 Thread GitBox
uranusjr closed issue #20804: Some timing metrics are in seconds but reported as milliseconds URL: https://github.com/apache/airflow/issues/20804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #21106: Fix StatD timing metric units

2022-05-31 Thread GitBox
boring-cyborg[bot] commented on PR #21106: URL: https://github.com/apache/airflow/pull/21106#issuecomment-1143109857 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] uranusjr merged pull request #21106: Fix StatD timing metric units

2022-05-31 Thread GitBox
uranusjr merged PR #21106: URL: https://github.com/apache/airflow/pull/21106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] github-actions[bot] commented on pull request #24050: Removing magic numbers from api_connecxion

2022-05-31 Thread GitBox
github-actions[bot] commented on PR #24050: URL: https://github.com/apache/airflow/pull/24050#issuecomment-1143085147 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the

[GitHub] [airflow] bharanidharan14 closed pull request #23717: Implement Azure Service Bus Queue Operators

2022-05-31 Thread GitBox
bharanidharan14 closed pull request #23717: Implement Azure Service Bus Queue Operators URL: https://github.com/apache/airflow/pull/23717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] bharanidharan14 closed pull request #24042: Azure service bus subscription

2022-05-31 Thread GitBox
bharanidharan14 closed pull request #24042: Azure service bus subscription URL: https://github.com/apache/airflow/pull/24042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] bharanidharan14 opened a new pull request, #24063: Implemented Azure service bus subscription Operators

2022-05-31 Thread GitBox
bharanidharan14 opened a new pull request, #24063: URL: https://github.com/apache/airflow/pull/24063 - Added ASBCreateSubscriptionOperator - Added ASBUpdateSubscriptionOperator - Added ASBReceiveSubscriptionMessageOperator - Added ASBDeleteSubscriptionOperator - example DAG -

[GitHub] [airflow] uranusjr commented on issue #24060: `airflow db check-migrations -t 0` fails to check migrations in airflow 2.3

2022-05-31 Thread GitBox
uranusjr commented on issue #24060: URL: https://github.com/apache/airflow/issues/24060#issuecomment-1143072783 cc @ephraimbuddy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] uranusjr commented on pull request #23972: Adding connections for sql operator unit tests so they pass

2022-05-31 Thread GitBox
uranusjr commented on PR #23972: URL: https://github.com/apache/airflow/pull/23972#issuecomment-1143072530 Please also run pre-commit locally to fix the errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] uranusjr commented on a diff in pull request #24046: Added impersonation_chain for dataflow operators

2022-05-31 Thread GitBox
uranusjr commented on code in PR #24046: URL: https://github.com/apache/airflow/pull/24046#discussion_r886283747 ## airflow/providers/google/cloud/hooks/dataflow.py: ## @@ -972,16 +972,31 @@ def start_sql_job( :param on_new_job_callback: Callback called when the job is

[GitHub] [airflow] ferruzzi commented on pull request #24058: Convert Athena Sample DAG to System Test

2022-05-31 Thread GitBox
ferruzzi commented on PR #24058: URL: https://github.com/apache/airflow/pull/24058#issuecomment-1143061508 `Tests / Helm Chart Executor Upgrade (pull_request)` is failing, but I don't _think_ that's me. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] ferruzzi closed pull request #24058: Convert Athena Sample DAG to System Test

2022-05-31 Thread GitBox
ferruzzi closed pull request #24058: Convert Athena Sample DAG to System Test URL: https://github.com/apache/airflow/pull/24058 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] thinhnd2104 commented on a diff in pull request #23881: Generic S3 to SQL

2022-05-31 Thread GitBox
thinhnd2104 commented on code in PR #23881: URL: https://github.com/apache/airflow/pull/23881#discussion_r886250728 ## airflow/operators/s3_to_sql.py: ## @@ -0,0 +1,117 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

[GitHub] [airflow] epapineau commented on pull request #22938: Enable dbt Cloud provider to interact with single tenant instances

2022-05-31 Thread GitBox
epapineau commented on PR #22938: URL: https://github.com/apache/airflow/pull/22938#issuecomment-1142979885 @josh-fell I was able to resolve the unit test failures locally by updating the value of `dbt_cloud_conn_id` to `dbt_cloud` (the name of my connection locally). Should I push those

[GitHub] [airflow] epapineau commented on a diff in pull request #22938: Enable dbt Cloud provider to interact with single tenant instances

2022-05-31 Thread GitBox
epapineau commented on code in PR #22938: URL: https://github.com/apache/airflow/pull/22938#discussion_r886245432 ## tests/providers/dbt/cloud/hooks/test_dbt_cloud.py: ## @@ -119,16 +122,37 @@ def setup_class(self): password=TOKEN, ) +#

[GitHub] [airflow] github-actions[bot] closed pull request #21797: Fixed dag level access permissions

2022-05-31 Thread GitBox
github-actions[bot] closed pull request #21797: Fixed dag level access permissions URL: https://github.com/apache/airflow/pull/21797 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[airflow] branch main updated: Fix grid view for mapped tasks (#24059)

2022-05-31 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6e96f04eb5 Fix grid view for mapped tasks

[GitHub] [airflow] jedcunningham merged pull request #24059: Fix grid view for mapped tasks

2022-05-31 Thread GitBox
jedcunningham merged PR #24059: URL: https://github.com/apache/airflow/pull/24059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch v2-3-test updated: Re-add --force-build flag (#24061)

2022-05-31 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-3-test by this push: new e4a31e6e2f Re-add --force-build flag

[airflow] branch dependabot/npm_and_yarn/airflow/ui/eventsource-1.1.1 created (now eba3645ccf)

2022-05-31 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/airflow/ui/eventsource-1.1.1 in repository https://gitbox.apache.org/repos/asf/airflow.git at eba3645ccf Bump eventsource from 1.0.7 to 1.1.1 in /airflow/ui

[GitHub] [airflow] dependabot[bot] opened a new pull request, #24062: Bump eventsource from 1.0.7 to 1.1.1 in /airflow/ui

2022-05-31 Thread GitBox
dependabot[bot] opened a new pull request, #24062: URL: https://github.com/apache/airflow/pull/24062 Bumps [eventsource](https://github.com/EventSource/eventsource) from 1.0.7 to 1.1.1. Changelog Sourced from

[airflow] branch main updated: Re-add --force-build flag (#24061)

2022-05-31 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5005ff2735 Re-add --force-build flag (#24061)

[GitHub] [airflow] potiuk merged pull request #24061: Re-add --force-build flag

2022-05-31 Thread GitBox
potiuk merged PR #24061: URL: https://github.com/apache/airflow/pull/24061 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] github-actions[bot] commented on pull request #24061: Re-add --force-build flag

2022-05-31 Thread GitBox
github-actions[bot] commented on PR #24061: URL: https://github.com/apache/airflow/pull/24061#issuecomment-1142767710 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the

[GitHub] [airflow] potiuk opened a new pull request, #24061: Re-add --force-build flag

2022-05-31 Thread GitBox
potiuk opened a new pull request, #24061: URL: https://github.com/apache/airflow/pull/24061 After #24052 we also need to add --force-build flag as for Python 3.7 rebuilding CI cache would have been silently ignored as no image building would be needed --- **^ Add

[GitHub] [airflow] github-actions[bot] commented on pull request #24046: Added impersonation_chain for dataflow operators

2022-05-31 Thread GitBox
github-actions[bot] commented on PR #24046: URL: https://github.com/apache/airflow/pull/24046#issuecomment-1142754708 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the

[GitHub] [airflow] github-actions[bot] commented on pull request #24059: Fix grid view for mapped tasks

2022-05-31 Thread GitBox
github-actions[bot] commented on PR #24059: URL: https://github.com/apache/airflow/pull/24059#issuecomment-1142752014 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the

[GitHub] [airflow] ferruzzi commented on issue #24030: amazon-provider: Make components more consistent

2022-05-31 Thread GitBox
ferruzzi commented on issue #24030: URL: https://github.com/apache/airflow/issues/24030#issuecomment-1142742387 This is a great breakdown. We've been putting a lot of effort into standardizing things in the AWS package and this will definitely help. This may be out of scope, or

[GitHub] [airflow] thesuperzapper opened a new issue, #24060: `airflow db check-migrations -t 0` fails to check migrations in airflow 2.3

2022-05-31 Thread GitBox
thesuperzapper opened a new issue, #24060: URL: https://github.com/apache/airflow/issues/24060 ### Apache Airflow version 2.3.1 (latest released) ### What happened As of Airflow 2.3.0 the `airflow db check-migrations -t 0` command will ALWAYS think there are unapplied

[GitHub] [airflow] jedcunningham opened a new pull request, #24059: Fix grid view for mapped tasks

2022-05-31 Thread GitBox
jedcunningham opened a new pull request, #24059: URL: https://github.com/apache/airflow/pull/24059 #23951 accidentally broke the grid view for mapped tasks. This fixes it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[airflow] branch constraints-2-3 updated: Updating constraints. Build id:2418080392

2022-05-31 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-3 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-3 by this push: new 30a23b8db5 Updating

[airflow] branch constraints-main updated: Updating constraints. Build id:2417008593

2022-05-31 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 18b84989ef Updating

[GitHub] [airflow] ferruzzi commented on pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
ferruzzi commented on PR #24057: URL: https://github.com/apache/airflow/pull/24057#issuecomment-1142702009 That's cool, thanks for contributing it back to the community! Thanks for the changes, I know a lot of them were little nitpicks. I'll need to finish looking at the other

[GitHub] [airflow] uranusjr commented on pull request #21560: Add template_fields and constructor parameters to displaying

2022-05-31 Thread GitBox
uranusjr commented on PR #21560: URL: https://github.com/apache/airflow/pull/21560#issuecomment-1142700940 We probably should hide this output behind a flag since the argument names can intefere with how the script can be used with `grep`. -- This is an automated message from the Apache

[GitHub] [airflow] uranusjr commented on a diff in pull request #21560: Add template_fields and constructor parameters to displaying

2022-05-31 Thread GitBox
uranusjr commented on code in PR #21560: URL: https://github.com/apache/airflow/pull/21560#discussion_r886178881 ## scripts/tools/list-integrations.py: ## @@ -63,7 +63,12 @@ def _find_clazzes(directory, base_class): ] for found_clazz in integration_clazzes:

[GitHub] [airflow] jonathansp commented on pull request #23641: fix: DatabricksSubmitRunOperator and DatabricksRunNowOperator cannot define .json as template_ext (#23622)

2022-05-31 Thread GitBox
jonathansp commented on PR #23641: URL: https://github.com/apache/airflow/pull/23641#issuecomment-1142698255 it looks good now @potiuk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] uranusjr commented on a diff in pull request #24039: Fix GCSToGCSOperator cannot copy a single file/folder without copying other files/folders with that prefix

2022-05-31 Thread GitBox
uranusjr commented on code in PR #24039: URL: https://github.com/apache/airflow/pull/24039#discussion_r886177489 ## airflow/providers/google/cloud/transfers/gcs_to_gcs.py: ## @@ -341,6 +345,8 @@ def _copy_source_without_wildcard(self, hook, prefix): raise

[airflow] 01/01: Shaves-off about 3 minutes from usage of ARM instances on CI (#24052)

2022-05-31 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 9eaa746e0ebf416749025a42519e3525ae27084d Author: Jarek Potiuk AuthorDate: Tue May 31 17:37:50 2022 +0200

[airflow] branch v2-3-test updated (89f5592f3d -> 9eaa746e0e)

2022-05-31 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git discard 89f5592f3d Shaves-off about 3 minutes from usage of ARM instances on CI (#24052) new 9eaa746e0e Shaves-off

[GitHub] [airflow] ferruzzi commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
ferruzzi commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886174822 ## airflow/providers/amazon/aws/example_dags/example_appflow.py: ## @@ -0,0 +1,101 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] igorborgest commented on pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
igorborgest commented on PR #24057: URL: https://github.com/apache/airflow/pull/24057#issuecomment-1142694358 > Thanks for tackling this, it's a lot of work! > > I have not used AppFlow, I assume you've tested these against the actual service and they work as you'd expect? I left

[GitHub] [airflow] ferruzzi commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
ferruzzi commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886174209 ## airflow/providers/amazon/aws/operators/appflow.py: ## @@ -0,0 +1,562 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] igorborgest commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
igorborgest commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886172884 ## airflow/providers/amazon/aws/operators/appflow.py: ## @@ -0,0 +1,562 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] igorborgest commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
igorborgest commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886172708 ## airflow/providers/amazon/aws/example_dags/example_appflow.py: ## @@ -0,0 +1,101 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] igorborgest commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
igorborgest commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886172553 ## airflow/providers/amazon/aws/operators/appflow.py: ## @@ -0,0 +1,562 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] ferruzzi commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
ferruzzi commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886168527 ## airflow/providers/amazon/aws/operators/appflow.py: ## @@ -0,0 +1,562 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] ferruzzi commented on pull request #24058: Convert Athena Sample DAG to System Test

2022-05-31 Thread GitBox
ferruzzi commented on PR #24058: URL: https://github.com/apache/airflow/pull/24058#issuecomment-1142682524 @mnojek @potiuk @bhirsz - Other system test migrations will be relying on these helper methods so, if at all possible, we're hoping to get this approved and merged ASAP with the goal

[GitHub] [airflow] ferruzzi opened a new pull request, #24058: Convert Athena Sample DAG to System Test

2022-05-31 Thread GitBox
ferruzzi opened a new pull request, #24058: URL: https://github.com/apache/airflow/pull/24058 This is the first PR in our new project to convert all AWS Sample DAGs into AIP-47-compliant System Tests. It includes the required changes to the Athena Sample DAG and docs, a few helper

[GitHub] [airflow] ferruzzi commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
ferruzzi commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886138033 ## airflow/providers/amazon/aws/hooks/appflow.py: ## @@ -0,0 +1,51 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

[GitHub] [airflow] uranusjr commented on a diff in pull request #24044: Have consistent types between the ORM and the migration files

2022-05-31 Thread GitBox
uranusjr commented on code in PR #24044: URL: https://github.com/apache/airflow/pull/24044#discussion_r886155452 ## airflow/models/serialized_dag.py: ## @@ -67,11 +67,11 @@ class SerializedDagModel(Base): dag_id = Column(String(ID_LEN), primary_key=True) fileloc =

[GitHub] [airflow] uranusjr commented on a diff in pull request #24044: Have consistent types between the ORM and the migration files

2022-05-31 Thread GitBox
uranusjr commented on code in PR #24044: URL: https://github.com/apache/airflow/pull/24044#discussion_r886154632 ## airflow/migrations/env.py: ## @@ -51,7 +54,20 @@ def include_object(_, name, type_, *args): # my_important_option = config.get_main_option("my_important_option")

[GitHub] [airflow] igorborgest commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
igorborgest commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886153220 ## airflow/providers/amazon/aws/example_dags/example_appflow.py: ## @@ -0,0 +1,101 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] igorborgest commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
igorborgest commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886151171 ## airflow/providers/amazon/aws/operators/appflow.py: ## @@ -0,0 +1,562 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] igorborgest commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
igorborgest commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886150988 ## airflow/providers/amazon/aws/operators/appflow.py: ## @@ -0,0 +1,562 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] potiuk closed issue #23917: Wrong dependecy version of requests for Databricks provider

2022-05-31 Thread GitBox
potiuk closed issue #23917: Wrong dependecy version of requests for Databricks provider URL: https://github.com/apache/airflow/issues/23917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on issue #23917: Wrong dependecy version of requests for Databricks provider

2022-05-31 Thread GitBox
potiuk commented on issue #23917: URL: https://github.com/apache/airflow/issues/23917#issuecomment-1142659862 Closed by #23927 (@koconder please let us know if some more change is needed). -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] vincbeck commented on a diff in pull request #23881: Generic S3 to SQL

2022-05-31 Thread GitBox
vincbeck commented on code in PR #23881: URL: https://github.com/apache/airflow/pull/23881#discussion_r886134984 ## airflow/operators/s3_to_sql.py: ## @@ -0,0 +1,117 @@ +# Review Comment: In my opinion this file should be in the Amazon provider package. It should be in

[GitHub] [airflow] igorborgest commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
igorborgest commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886135299 ## airflow/providers/amazon/aws/example_dags/example_appflow.py: ## @@ -0,0 +1,101 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] vincbeck commented on a diff in pull request #24057: Amazon appflow

2022-05-31 Thread GitBox
vincbeck commented on code in PR #24057: URL: https://github.com/apache/airflow/pull/24057#discussion_r886123715 ## airflow/providers/amazon/aws/example_dags/example_appflow.py: ## @@ -0,0 +1,101 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[airflow] branch v2-3-test updated: Shaves-off about 3 minutes from usage of ARM instances on CI (#24052)

2022-05-31 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-3-test by this push: new 89f5592f3d Shaves-off about 3 minutes

[GitHub] [airflow] igorborgest opened a new pull request, #24057: Amazon appflow

2022-05-31 Thread GitBox
igorborgest opened a new pull request, #24057: URL: https://github.com/apache/airflow/pull/24057 Hi! It's a proposal to add [Amazon AppFlow](https://aws.amazon.com/appflow) support for the `apache-airflow-providers-amazon` package. This is an initial support tested only

[GitHub] [airflow] potiuk commented on pull request #21560: Add template_fields and constructor parameters to displaying

2022-05-31 Thread GitBox
potiuk commented on PR #21560: URL: https://github.com/apache/airflow/pull/21560#issuecomment-1142607375 I guess you are going to continue @chenglongyan ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] chenglongyan opened a new pull request, #21560: Add template_fields and constructor parameters to displaying

2022-05-31 Thread GitBox
chenglongyan opened a new pull request, #21560: URL: https://github.com/apache/airflow/pull/21560 This pr add template_fields and constructor parameters to output of [list-integrations.py](https://github.com/apache/airflow/blob/main/scripts/tools/list-integrations.py) ### Output

[GitHub] [airflow] mariasumedre-3pg commented on a diff in pull request #24054: Add param docs to KubernetesHook and KubernetesPodOperator (#23955)

2022-05-31 Thread GitBox
mariasumedre-3pg commented on code in PR #24054: URL: https://github.com/apache/airflow/pull/24054#discussion_r886080369 ## airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py: ## @@ -142,6 +150,12 @@ class KubernetesPodOperator(BaseOperator): :param

[GitHub] [airflow] potiuk commented on issue #23901: execution_timeout doesn't work with PythonVirtualenvOperator

2022-05-31 Thread GitBox
potiuk commented on issue #23901: URL: https://github.com/apache/airflow/issues/23901#issuecomment-1142555234 I think the best place to describe it is "timeout" parameter docsctring in BaseOperator. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] potiuk commented on issue #23901: execution_timeout doesn't work with PythonVirtualenvOperator

2022-05-31 Thread GitBox
potiuk commented on issue #23901: URL: https://github.com/apache/airflow/issues/23901#issuecomment-1142550746 There is a big difference. All those processes are under control of Airflow unlike the process you spawn in your code. Yeah it would be eat to update the docs.could you

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #24044: Have consistent types between the ORM and the migration files

2022-05-31 Thread GitBox
ephraimbuddy commented on code in PR #24044: URL: https://github.com/apache/airflow/pull/24044#discussion_r886038814 ## airflow/migrations/versions/0111_2_3_3_compare_types_between_orm_and_db.py: ## @@ -0,0 +1,79 @@ +# +# Licensed to the Apache Software Foundation (ASF) under

[GitHub] [airflow] KarthikRajashekaran commented on issue #20779: Logs of tasks when running is not available with kubernetesexecutor on webserver UI

2022-05-31 Thread GitBox
KarthikRajashekaran commented on issue #20779: URL: https://github.com/apache/airflow/issues/20779#issuecomment-1142524690 @nirutgupta Where should the file k8s task handler.py be located in the airflow path? -- This is an automated message from the Apache Git Service. To respond to the

[airflow] branch main updated: Use KubernetesHook to create api client in KubernetesPodOperator (#20578)

2022-05-31 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 60eb9e106f Use KubernetesHook to create api

[GitHub] [airflow] dstandish merged pull request #20578: Use KubernetesHook to create api client in KubernetesPodOperator

2022-05-31 Thread GitBox
dstandish merged PR #20578: URL: https://github.com/apache/airflow/pull/20578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] cccs-seb commented on issue #23901: execution_timeout doesn't work with PythonVirtualenvOperator

2022-05-31 Thread GitBox
cccs-seb commented on issue #23901: URL: https://github.com/apache/airflow/issues/23901#issuecomment-1142515694 @potiuk BashOperator and PythonVirtualenvOperator essentially run code very similar to each other (both run a subprocess to run the task code). PythonVirtualenvOperator uses the

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #24044: Have consistent types between the ORM and the migration files

2022-05-31 Thread GitBox
ephraimbuddy commented on code in PR #24044: URL: https://github.com/apache/airflow/pull/24044#discussion_r886005258 ## airflow/models/dagpickle.py: ## @@ -39,7 +39,7 @@ class DagPickle(Base): id = Column(Integer, primary_key=True) pickle =

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #24044: Have consistent types between the ORM and the migration files

2022-05-31 Thread GitBox
ephraimbuddy commented on code in PR #24044: URL: https://github.com/apache/airflow/pull/24044#discussion_r886001355 ## airflow/models/serialized_dag.py: ## @@ -67,11 +67,11 @@ class SerializedDagModel(Base): dag_id = Column(String(ID_LEN), primary_key=True) fileloc =

[GitHub] [airflow] yfyg commented on issue #23935: No option to set blocksize when retrieving a file in ftphook

2022-05-31 Thread GitBox
yfyg commented on issue #23935: URL: https://github.com/apache/airflow/issues/23935#issuecomment-1142418746 Hi Sorry, did not have the time to handle it yet. If someone wants to pick it up, feel free. If not I will probably be able to check it in the next 2 weeks. Thanks

[GitHub] [airflow] rbiegacz commented on a diff in pull request #24054: Add param docs to KubernetesHook and KubernetesPodOperator (#23955)

2022-05-31 Thread GitBox
rbiegacz commented on code in PR #24054: URL: https://github.com/apache/airflow/pull/24054#discussion_r885906030 ## airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py: ## @@ -142,6 +150,12 @@ class KubernetesPodOperator(BaseOperator): :param priority_class_name:

[GitHub] [airflow] potiuk commented on issue #23935: No option to set blocksize when retrieving a file in ftphook

2022-05-31 Thread GitBox
potiuk commented on issue #23935: URL: https://github.com/apache/airflow/issues/23935#issuecomment-1142380554 Feel free -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] kevgeo commented on issue #23935: No option to set blocksize when retrieving a file in ftphook

2022-05-31 Thread GitBox
kevgeo commented on issue #23935: URL: https://github.com/apache/airflow/issues/23935#issuecomment-1142371133 Hi @yfyg, are you working on this issue? If not, @eladkal, @josh-fell, I would love to take up this issue. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] mariasumedre-3pg opened a new pull request, #24054: Add param docs to KubernetesHook and KubernetesPodOperator (#23955)

2022-05-31 Thread GitBox
mariasumedre-3pg opened a new pull request, #24054: URL: https://github.com/apache/airflow/pull/24054 Closes #23955 Added param docs for KubernetesHook and KubernetesPodOperator: - KubernetesHook: `in_cluster`, `config_file`, `cluster_context`, `client_configuration`

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #24054: Add param docs to KubernetesHook and KubernetesPodOperator (#23955)

2022-05-31 Thread GitBox
boring-cyborg[bot] commented on PR #24054: URL: https://github.com/apache/airflow/pull/24054#issuecomment-1142363309 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] ashb commented on a diff in pull request #24044: Have consistent types between the ORM and the migration files

2022-05-31 Thread GitBox
ashb commented on code in PR #24044: URL: https://github.com/apache/airflow/pull/24044#discussion_r885849674 ## airflow/migrations/versions/0111_2_3_3_compare_types_between_orm_and_db.py: ## @@ -0,0 +1,79 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] chenglongyan opened a new pull request, #24053: Migrate Zendesk example DAGs to new design AIP-47

2022-05-31 Thread GitBox
chenglongyan opened a new pull request, #24053: URL: https://github.com/apache/airflow/pull/24053 closes: #22471 --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] jerome-aosis commented on issue #23895: Airflow DAGs not refreshed with pullPolicy set to Always with the same container tag

2022-05-31 Thread GitBox
jerome-aosis commented on issue #23895: URL: https://github.com/apache/airflow/issues/23895#issuecomment-1142337976 Thanks for your answer. I use the same dag tag for learning purpose and it's not for production environment. I agree with you, pullPolicy is not working like that

[airflow] branch constraints-2-3 updated: Updating constraints. Build id:2415810643

2022-05-31 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-3 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-3 by this push: new 85accd368b Updating

[GitHub] [airflow] schattian commented on pull request #23658: Fix k8s pod.execute randomly stuck indefinitely by logs consumption (#23497)

2022-05-31 Thread GitBox
schattian commented on PR #23658: URL: https://github.com/apache/airflow/pull/23658#issuecomment-1142318827 > (And Sorry for long delays in review @schattian - we had Airflow Summit last week which kept many of the reviewers busy). didnt know that. no worries! and thanks for the rev

[airflow] branch main updated (8804b1a941 -> d19cb86660)

2022-05-31 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 8804b1a941 Shaves-off about 3 minutes from usage of ARM instances on CI (#24052) add d19cb86660 SSL Bucket,

[GitHub] [airflow] potiuk merged pull request #23891: SSL Bucket, Light Logic Refactor and Docstring Update for Alibaba Provider

2022-05-31 Thread GitBox
potiuk merged PR #23891: URL: https://github.com/apache/airflow/pull/23891 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (2247b6d43c -> 8804b1a941)

2022-05-31 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 2247b6d43c Upgrade to pip 22.1.2 (#24043) add 8804b1a941 Shaves-off about 3 minutes from usage of ARM instances

[GitHub] [airflow] potiuk merged pull request #24052: Shaves-off about 3 minutes from usage of ARM instances on CI

2022-05-31 Thread GitBox
potiuk merged PR #24052: URL: https://github.com/apache/airflow/pull/24052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk closed issue #23901: execution_timeout doesn't work with PythonVirtualenvOperator

2022-05-31 Thread GitBox
potiuk closed issue #23901: execution_timeout doesn't work with PythonVirtualenvOperator URL: https://github.com/apache/airflow/issues/23901 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on issue #23901: execution_timeout doesn't work with PythonVirtualenvOperator

2022-05-31 Thread GitBox
potiuk commented on issue #23901: URL: https://github.com/apache/airflow/issues/23901#issuecomment-1142296307 This is correct. Airflow does not - on its own propagate SIGALRM (this is how timeout is implemented) to a subprocess. And your subprocess would have to handle the SIGALRM in this

[GitHub] [airflow] potiuk commented on issue #23895: Airflow DAGs not refreshed with pullPolicy set to Always with the same container tag

2022-05-31 Thread GitBox
potiuk commented on issue #23895: URL: https://github.com/apache/airflow/issues/23895#issuecomment-1142261423 This is not how Kubernetes/Helm chart works, from the point of view of the definition of the pods have not changed, so it will not update them - hence they will notbe restarted.

[GitHub] [airflow] potiuk closed issue #23895: Airflow DAGs not refreshed with pullPolicy set to Always with the same container tag

2022-05-31 Thread GitBox
potiuk closed issue #23895: Airflow DAGs not refreshed with pullPolicy set to Always with the same container tag URL: https://github.com/apache/airflow/issues/23895 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] potiuk commented on issue #23893: Kubernetes pods are not staying alive when delete_worker_pods set to False

2022-05-31 Thread GitBox
potiuk commented on issue #23893: URL: https://github.com/apache/airflow/issues/23893#issuecomment-1142249552 We have revamped kubernetes with new version of kubernetes library in 2.3.* - @dstandish - do you think that one is likely to be solved by it ? -- This is an automated message

[GitHub] [airflow] kardoo commented on issue #23826: BigQueryInsertJobOperator is broken on any type of job except `query`

2022-05-31 Thread GitBox
kardoo commented on issue #23826: URL: https://github.com/apache/airflow/issues/23826#issuecomment-1142248822 My apologies if I was misunderstood. If I have time in the next couple of weeks, I will let you know. -- This is an automated message from the Apache Git Service. To respond to

  1   2   3   >