[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #28619: Fix code readability, add docstrings to json_client

[GitBox]

ephraimbuddy commented on code in PR #28619:
URL: https://github.com/apache/airflow/pull/28619#discussion_r1058372127


##
airflow/api/client/json_client.py:
##
@@ -24,9 +24,25 @@
 
 
 class Client(api_client.Client):
-"""Json API client implementation."""
+"""Json API client implementation.
 
-def _request(self, url, method="GET", json=None):
+This client is used to interact with a Json API server and perform various 
actions
+such as triggering DAG runs,deleting DAGs, interacting with pools, and 
getting lineage information.
+
+:param _api_base_url: The base URL for the Json API server.
+:param _session: A session object to use for making HTTP requests.
+"""
+
+def _request(self, url: str, json=None, method: str = "GET"):
+"""Make a request to the Json API server.
+
+:param url: The URL to send the request to.
+:param method: The HTTP method to use (e.g. "GET", "POST", "DELETE").
+:param json: A dictionary containing JSON data to send in the request 
body.
+:return: A dictionary containing the JSON response from the server.
+:rtype: dict

Review Comment:
   You can use typing instead of `:return:` and `:rtype:`, I think there's an 
`APIResponse` return type at `airflow/api_connexion/types`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] rkarish opened a new pull request, #28625: Add note for pymssql requirement

[GitBox]

rkarish opened a new pull request, #28625:
URL: https://github.com/apache/airflow/pull/28625

   A user reported an issue on Slack with `MsSqlHook` resulting in a 
segmentation fault. This issue was resolved by following the instructions on 
the `pymssql` PyPI. I added this note to the documentation for the mssql 
provider package.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil merged pull request #28594: Fix Connection.get_extra type

[GitBox]

kaxil merged PR #28594:
URL: https://github.com/apache/airflow/pull/28594


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28594: Fix Connection.get_extra type

[GitBox]

boring-cyborg[bot] commented on PR #28594:
URL: https://github.com/apache/airflow/pull/28594#issuecomment-1366743808

   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (4f006e8c71 -> 5dcbbd6f17)

[kaxilnaik]

This is an automated email from the ASF dual-hosted git repository.

kaxilnaik pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 4f006e8c71 Add .bash and other extensions to SSHOperator template_ext 
(#28617)
 add 5dcbbd6f17 Fix ``Connection.get_extra`` type (#28594)

No new revisions were added by this update.

Summary of changes:
 airflow/models/connection.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[airflow] branch main updated (76186bb588 -> 4f006e8c71)

[potiuk]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 76186bb588 Fix code docstrings (#28622)
 add 4f006e8c71 Add .bash and other extensions to SSHOperator template_ext 
(#28617)

No new revisions were added by this update.

Summary of changes:
 airflow/providers/ssh/operators/ssh.py | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

[GitHub] [airflow] potiuk merged pull request #28617: Add .bash extension to SSHOperator

[GitBox]

potiuk merged PR #28617:
URL: https://github.com/apache/airflow/pull/28617


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on a diff in pull request #28625: Add note for pymssql requirement

[GitBox]

Taragolis commented on code in PR #28625:
URL: https://github.com/apache/airflow/pull/28625#discussion_r1058446218


##
docs/apache-airflow-providers-microsoft-mssql/index.rst:
##
@@ -91,6 +91,8 @@ PIP package  Version required
 ``pymssql``  ``>=2.1.5; platform_machine != 
"aarch64"``
 ===  
==
 
+Note: For ``pymssql`` installation some Linux distributions ``pip`` version is 
too old to support all the flavors of ``manylinux`` wheels, so upgrading 
``pip`` is necessary. An example of such distributions would be Ubuntu 18.04 or 
Python3.6 module in RHEL8 and CentOS8.

Review Comment:
   Hm... Min Python version for stable Airflow is 3.7



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28625: Add note for pymssql requirement

[GitBox]

potiuk commented on code in PR #28625:
URL: https://github.com/apache/airflow/pull/28625#discussion_r1058456144


##
docs/apache-airflow-providers-microsoft-mssql/index.rst:
##
@@ -91,6 +91,8 @@ PIP package  Version required
 ``pymssql``  ``>=2.1.5; platform_machine != 
"aarch64"``
 ===  
==
 
+Note: For ``pymssql`` installation some Linux distributions ``pip`` version is 
too old to support all the flavors of ``manylinux`` wheels, so upgrading 
``pip`` is necessary. An example of such distributions would be Ubuntu 18.04 or 
Python3.6 module in RHEL8 and CentOS8.

Review Comment:
   Yes. Shorten it  - and possibly link to the documentation with "more 
details".



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28627: Change Architecture class into Enum

[GitBox]

potiuk commented on PR #28627:
URL: https://github.com/apache/airflow/pull/28627#issuecomment-1366802671

   Does not seem to have any bad side effect (let's see what tests say too)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk opened a new pull request, #28627: Change Architecture class into Enum

[GitBox]

potiuk opened a new pull request, #28627:
URL: https://github.com/apache/airflow/pull/28627

   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28569: Add general-purpose "notifier" concept to DAGs

[GitBox]

potiuk commented on code in PR #28569:
URL: https://github.com/apache/airflow/pull/28569#discussion_r1058359807


##
airflow/providers/slack/provider.yaml:
##
@@ -39,7 +39,7 @@ versions:
   - 1.0.0
 
 dependencies:
-  - apache-airflow>=2.3.0
+  - apache-airflow>=2.6.0.dev0

Review Comment:
   > Hi @potiuk, @Taragolis what is the right way to mark this PR for 2.6? 
Can't find any clue from the error messages
   
   We do not need that . @Taragolis  is right.
   
   > I think this lines in 
airflow/providers/slack/notifications/slack_notifier.py would be enough, 
   
   Correct. It should nicely work this way. However we might want to to add 
special handling in `verify-provider-packages` command:
   
   ```
   breeze release-management verify-provider-packages --use-airflow-version 
2.3.0 --use-packages-from-dist --airflow-constraints-reference constraints-2.3.0
   ```
   
   This one is failing now because AirflowOptionalProviderFeatureException is 
raised:
   
   https://github.com/apache/airflow/actions/runs/3793310830/jobs/6450758471:
   
   ```
   Traceback (most recent call last):
 File 
"/usr/local/lib/python3.7/site-packages/airflow/providers/slack/notifications/slack_notifier.py",
 line 26, in 
   from airflow.notifications.basenotifier import BaseNotifier
   ModuleNotFoundError: No module named 'airflow.notifications'
   
   During handling of the above exception, another exception occurred:
   
   Traceback (most recent call last):
 File "/opt/airflow/scripts/in_container/verify_providers.py", line 355, in 
import_all_classes
   _module = importlib.import_module(modinfo.name)
 File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in 
import_module
   return _bootstrap._gcd_import(name, package, level)
 File "", line 1006, in _gcd_import
 File "", line 983, in _find_and_load
 File "", line 967, in _find_and_load_unlocked
 File "", line 677, in _load_unlocked
 File "", line 728, in exec_module
 File "", line 219, in 
_call_with_frames_removed
 File 
"/usr/local/lib/python3.7/site-packages/airflow/providers/slack/notifications/slack_notifier.py",
 line 28, in 
   raise AirflowOptionalProviderFeatureException(e)
   airflow.exceptions.AirflowOptionalProviderFeatureException: No module named 
'airflow.notifications'
   ```
   
   We need to catch it and ignore. 
   
   We do not have it implemented yet because:
   
   a) it was not possible before because we had Airflow 2.2+
   b) it was not needed because all the 
"AirflowOptionalProviderFeatureException" were thrown when some libraries were 
missing only (and we had them all installed in our CI image)
   c) this is the first time we are implementing an optional feature that 
depends on Airflow version :)
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #28569: Add general-purpose "notifier" concept to DAGs

[GitBox]

ephraimbuddy commented on code in PR #28569:
URL: https://github.com/apache/airflow/pull/28569#discussion_r1058365263


##
airflow/providers/slack/provider.yaml:
##
@@ -39,7 +39,7 @@ versions:
   - 1.0.0
 
 dependencies:
-  - apache-airflow>=2.3.0
+  - apache-airflow>=2.6.0.dev0

Review Comment:
   Nice. I will catch and ignore it then.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vchiapaikeo commented on pull request #28624: Fix UI caret direction

[GitBox]

vchiapaikeo commented on PR #28624:
URL: https://github.com/apache/airflow/pull/28624#issuecomment-1366756074

   Yup - this latest commit should hopefully fix. Changed the ids as well to 
better reflect the action that is taking place.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28020: Feature/#28015 default args

[GitBox]

potiuk commented on PR #28020:
URL: https://github.com/apache/airflow/pull/28020#issuecomment-1366782782

   Could you please update the description and title of the PR (and commit) if 
you want to get it reviewed? As it is now, I had no idea what should be 
reviewed because neither title nor description says anything useful, so I have 
completely no idea what your intention was.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #28603: SFTPOperator does not finish downloading

[GitBox]

potiuk commented on issue #28603:
URL: https://github.com/apache/airflow/issues/28603#issuecomment-1366733676

One of the reasons for paramico change was that pysftp did not allow for 
the kind of configuration that direct paramiko would. You should likely look at 
`keepalive_interval` and possibly other parameters there. 
   
   The problem that you might experience is because some networking equipment 
between you and the server could have been restarted (usual thing in the cloud) 
and if you do not enable keep-alives, you might as a client not even be aware 
that the other end has stopped responding.
   
   The nature of TCP connection is such that there is no "physical" connection 
set up - those are just networking packets sent over the network to certain 
ports. So if you have no "keepalive" enabled, the client might simply not 
realize the other end is disconnected because if the other end did not have 
time to signal that it has died or the CLOSE message might not have been 
delivered. 
   
   Only [TCP 
keepalives](https://tldp.org/HOWTO/TCP-Keepalive-HOWTO/overview.html) can give 
you the certainty  that you detect this situation. And they are disabled by 
default - you need to enable them - in case of SSHook (and SFTPHook derives 
from it in the new implementation) via the `keepalive_interval` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #28603: SFTPOperator does not finish downloading

[GitBox]

potiuk commented on issue #28603:
URL: https://github.com/apache/airflow/issues/28603#issuecomment-1366737096

   I am closing it provisionally - because I believe keepalive is the solution. 
I believe you might also not need to upgrade the provider - because keepalive 
seems to be also available in previous version of the hook (the SFTP provider 
before also used SSHook as a base and I believe you have 3.0.0 version of ssh 
provider: 
https://airflow.apache.org/docs/apache-airflow-providers-ssh/3.0.0/_api/airflow/providers/ssh/hooks/ssh/index.html
  So likely you can test keepalive without migration (though of course it is 
advised).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk closed issue #28603: SFTPOperator does not finish downloading

[GitBox]

potiuk closed issue #28603: SFTPOperator does not finish downloading
URL: https://github.com/apache/airflow/issues/28603


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (48a051acc4 -> d4dbb0077a)

[potiuk]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 48a051acc4 Minor update to Scheduler documentation (#28620)
 add d4dbb0077a Guard not-yet-expanded ti in trigger rule dep (#28592)

No new revisions were added by this update.

Summary of changes:
 airflow/ti_deps/deps/trigger_rule_dep.py|  8 +++-
 tests/ti_deps/deps/test_trigger_rule_dep.py | 30 +
 2 files changed, 37 insertions(+), 1 deletion(-)

[GitHub] [airflow] potiuk commented on pull request #28592: Guard not-yet-expanded ti in trigger rule dep

[GitBox]

potiuk commented on PR #28592:
URL: https://github.com/apache/airflow/pull/28592#issuecomment-1366746841

   Nice catch!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28624: Fix UI caret direction

[GitBox]

potiuk commented on PR #28624:
URL: https://github.com/apache/airflow/pull/28624#issuecomment-1366754570

   Failing tests :(


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28619: Fix code readability, add docstrings to json_client

[GitBox]

potiuk commented on code in PR #28619:
URL: https://github.com/apache/airflow/pull/28619#discussion_r1058455261


##
airflow/api/client/json_client.py:
##
@@ -24,9 +24,25 @@
 
 
 class Client(api_client.Client):
-"""Json API client implementation."""
+"""Json API client implementation.
 
-def _request(self, url, method="GET", json=None):
+This client is used to interact with a Json API server and perform various 
actions
+such as triggering DAG runs,deleting DAGs, interacting with pools, and 
getting lineage information.
+
+:param _api_base_url: The base URL for the Json API server.
+:param _session: A session object to use for making HTTP requests.
+"""
+
+def _request(self, url: str, json=None, method: str = "GET"):
+"""Make a request to the Json API server.
+
+:param url: The URL to send the request to.
+:param method: The HTTP method to use (e.g. "GET", "POST", "DELETE").
+:param json: A dictionary containing JSON data to send in the request 
body.
+:return: A dictionary containing the JSON response from the server.
+:rtype: dict

Review Comment:
   And I think the error message you pointed, indicate that you did it somehow 
wrongly.
   
   ```
   -> Tuple[str, int, str]
   ```
   
   Is likely what you should have added to the method to have equivalent of 
:rtype



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on issue #28603: SFTPOperator does not finish downloading

[GitBox]

Taragolis commented on issue #28603:
URL: https://github.com/apache/airflow/issues/28603#issuecomment-1366540261

   @ReneTC Could you provide a bit more information about which version of 
providers are you use?
   
   Did you tried latest version of SFTP provider? Because I see in [provider 
changelog](https://airflow.apache.org/docs/apache-airflow-providers-sftp/stable/index.html#changelog)
 that we in 4.0.0 switch from `pysftp` to `paramico` and without actual version 
it is quite difficult to determine is it problem in old version of package or  
this error still persist in new one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk merged pull request #28622: Fix code docstrings

[GitBox]

potiuk merged PR #28622:
URL: https://github.com/apache/airflow/pull/28622


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28622: Fix code docstrings

[GitBox]

boring-cyborg[bot] commented on PR #28622:
URL: https://github.com/apache/airflow/pull/28622#issuecomment-1366599533

   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (2169b39d7b -> 76186bb588)

[potiuk]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 2169b39d7b Migrate Google Cloud hooks test to `pytest` (#28301)
 add 76186bb588 Fix code docstrings (#28622)

No new revisions were added by this update.

Summary of changes:
 airflow/models/baseoperator.py | 16 
 airflow/models/dag.py  | 14 +++---
 airflow/models/dagbag.py   |  4 ++--
 airflow/models/dagrun.py   | 12 ++--
 4 files changed, 23 insertions(+), 23 deletions(-)

[GitHub] [airflow] aydinemre commented on issue #24669: setting `resources` in `KubernetesPodOperator` breaks when used with `Operator.partial`

[GitBox]

aydinemre commented on issue #24669:
URL: https://github.com/apache/airflow/issues/24669#issuecomment-1366600132

   I am trying to use the KubernetesPodOperator in Apache Airflow to specify 
resource requirements for my pods, but I am unable to pass the resources 
parameter to the operator. I have tried several different parameter names (e.g. 
resources, container_resources, k8s_resources) and have also tried passing the 
value as a dictionary and as a k8s.V1ResourceRequirements object, but none of 
these approaches have worked.
   
   I am using Airflow version 2.3.3
   
   I would appreciate any guidance or assistance on how to properly specify 
resource requirements for pods using the KubernetesPodOperator.
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil commented on a diff in pull request #28439: CustomTTYColoredFormatter should inherit TimezoneAware formatter

[GitBox]

kaxil commented on code in PR #28439:
URL: https://github.com/apache/airflow/pull/28439#discussion_r1058395532


##
airflow/utils/log/colored_log.py:
##
@@ -38,7 +40,7 @@
 BOLD_OFF = esc("22")
 
 
-class CustomTTYColoredFormatter(TTYColoredFormatter):
+class CustomTTYColoredFormatter(TTYColoredFormatter, TimezoneAware):

Review Comment:
   Want to add any tests for it :) ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil commented on a diff in pull request #28625: Add note for pymssql requirement

[GitBox]

kaxil commented on code in PR #28625:
URL: https://github.com/apache/airflow/pull/28625#discussion_r1058434982


##
docs/apache-airflow-providers-microsoft-mssql/index.rst:
##
@@ -91,6 +91,8 @@ PIP package  Version required
 ``pymssql``  ``>=2.1.5; platform_machine != 
"aarch64"``
 ===  
==
 
+Note: For ``pymssql`` installation some Linux distributions ``pip`` version is 
too old to support all the flavors of ``manylinux`` wheels, so upgrading 
``pip`` is necessary. An example of such distributions would be Ubuntu 18.04 or 
Python3.6 module in RHEL8 and CentOS8.

Review Comment:
   You should use the sphinx note directive
   
   
https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-note



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk merged pull request #28592: Guard not-yet-expanded ti in trigger rule dep

[GitBox]

potiuk merged PR #28592:
URL: https://github.com/apache/airflow/pull/28592


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #22132: Show average dag run duration

[GitBox]

potiuk commented on issue #22132:
URL: https://github.com/apache/airflow/issues/22132#issuecomment-1366758456

   > Is anyone aware that it's being developed? or is likely to be picked up 
for development?
   
   Marked it as good first issue. That's what I can do as maintainer.  
   
   Just to set expectations right:
   
   If you want to take a shot at it and try it yourself. Also if you have an 
interest in this feature (if you do not have skills) what works well is finding 
someone who would contribute it (for example at your company, customer who has 
the right skills and can contribute it). That's the best way to give back by 
the companies using the freely available software - by contributing back.
   
   This is usually how things work in Open-Source projects. Things get 
implemented, when someone implements it and the best way to get something 
implemented is to either implement it or find someone who will implement it for 
you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28337: Fix conflicting Oracle and SSH providers tests

[GitBox]

potiuk commented on PR #28337:
URL: https://github.com/apache/airflow/pull/28337#issuecomment-1366781416

   > I thought this is possible to achieve by autouse fixture in providers 
conftests.py and module/package scope level
   
   Yes. That's exactly what I thought. Would you like to give it a shot 
@Taragolis ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on pull request #28337: Fix conflicting Oracle and SSH providers tests

[GitBox]

Taragolis commented on PR #28337:
URL: https://github.com/apache/airflow/pull/28337#issuecomment-1366787812

   I tried to do it locally. It required some additional logic in fixture 
because for some unknown reason it not working well on "package" scope.
   
   Need to find this branch, complete, and make a PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis opened a new pull request, #28631: Clear DB between each separate providers tests

[GitBox]

Taragolis opened a new pull request, #28631:
URL: https://github.com/apache/airflow/pull/28631

   Follow-up: 
https://github.com/apache/airflow/pull/28337#issuecomment-1348771336
   
   Some hacks which uses in PR
   1. `module` scope with parsing "provider name" instead of `package` scope. 
pytest decide that package is `airflow` and run only once
   2. Use separate clear tests helpers instead of `airflow.utils.db.resetdb()`. 
locally cause some errors with duplicate constraint.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on issue #28499: "Dependency already registered for DAG" warnings with tasks with multiple outputs

[GitBox]

ephraimbuddy commented on issue #28499:
URL: https://github.com/apache/airflow/issues/28499#issuecomment-1366545805

   I think we should do away with this warning


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #28481: dagwarning, add error handling to purge_inactive_dag_warnings

[GitBox]

ephraimbuddy commented on code in PR #28481:
URL: https://github.com/apache/airflow/pull/28481#discussion_r1058266047


##
airflow/models/dagwarning.py:
##
@@ -72,6 +73,11 @@ def purge_inactive_dag_warnings(cls, session: Session = 
NEW_SESSION) -> None:
 
 :return: None
 """
+cls._purge_inactive_dag_warnings_with_retry(session)
+
+@classmethod
+@retry_db_transaction
+def _purge_inactive_dag_warnings_with_retry(cls, session: Session) -> None:

Review Comment:
   LGTM, let's add a little test



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28617: Add .bash extension to SSHOperator

[GitBox]

potiuk commented on code in PR #28617:
URL: https://github.com/apache/airflow/pull/28617#discussion_r1058300269


##
airflow/providers/ssh/operators/ssh.py:
##
@@ -63,7 +63,10 @@ class SSHOperator(BaseOperator):
 """
 
 template_fields: Sequence[str] = ("command", "environment", "remote_host")
-template_ext: Sequence[str] = (".sh",)
+template_ext: Sequence[str] = (
+".sh",
+".bash",

Review Comment:
   ```suggestion
   ".bash",
   ".csh",
   ".zsh",
   ".dash",
   ".ksh",
   ```
   
   Why not?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #28256: Include full path to Python files under zip path while clearing import errors.

[GitBox]

ephraimbuddy commented on code in PR #28256:
URL: https://github.com/apache/airflow/pull/28256#discussion_r1058346122


##
airflow/dag_processing/manager.py:
##
@@ -777,8 +777,9 @@ def clear_nonexistent_import_errors(self, session):
 :param session: session for ORM operations
 """
 query = session.query(errors.ImportError)
-if self._file_paths:
-query = 
query.filter(~errors.ImportError.filename.in_(self._file_paths))
+files = list_py_file_paths(self._dag_directory, 
include_examples=False, include_zip_paths=True)

Review Comment:
   > Agree. I think having filesystem_path in import errors is a good idea - 
and likely it's an easy one that can be automatically set on migration, so 
should be rather easy to do.
   
   I like the idea too



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vchiapaikeo commented on issue #27166: Carets in Grid view are the wrong way around

[GitBox]

vchiapaikeo commented on issue #27166:
URL: https://github.com/apache/airflow/issues/27166#issuecomment-1366700457

   Wanted to get started with some light UI work here so gave this simple issue 
a shot. Hope that's okay @shashwata27 - I see your PR was a bit stale.
   
   PR: https://github.com/apache/airflow/pull/28624
   
   cc @eladkal 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #28603: SFTPOperator does not finish downloading

[GitBox]

potiuk commented on issue #28603:
URL: https://github.com/apache/airflow/issues/28603#issuecomment-1366737663

   Of course, we can re-open it if you test both keepalive and migration to 
newer version and the issue is still there (though I think some more debugging 
info would be needed for that).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (5dcbbd6f17 -> 4f7ac623c8)

[potiuk]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 5dcbbd6f17 Fix ``Connection.get_extra`` type (#28594)
 add 4f7ac623c8 Fix typo (#28623)

No new revisions were added by this update.

Summary of changes:
 airflow/utils/db.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28620: Minor update to Scheduler documentation

[GitBox]

boring-cyborg[bot] commented on PR #28620:
URL: https://github.com/apache/airflow/pull/28620#issuecomment-1366745647

   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk merged pull request #28623: Fix typo

[GitBox]

potiuk merged PR #28623:
URL: https://github.com/apache/airflow/pull/28623


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (4f7ac623c8 -> 48a051acc4)

[kaxilnaik]

This is an automated email from the ASF dual-hosted git repository.

kaxilnaik pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 4f7ac623c8 Fix typo (#28623)
 add 48a051acc4 Minor update to Scheduler documentation (#28620)

No new revisions were added by this update.

Summary of changes:
 docs/apache-airflow/administration-and-deployment/scheduler.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

[GitHub] [airflow] kaxil merged pull request #28620: Minor update to Scheduler documentation

[GitBox]

kaxil merged PR #28620:
URL: https://github.com/apache/airflow/pull/28620


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] stamixthereal commented on a diff in pull request #28619: Fix code readability, add docstrings to json_client

[GitBox]

stamixthereal commented on code in PR #28619:
URL: https://github.com/apache/airflow/pull/28619#discussion_r1058438631


##
airflow/api/client/json_client.py:
##
@@ -24,9 +24,25 @@
 
 
 class Client(api_client.Client):
-"""Json API client implementation."""
+"""Json API client implementation.
 
-def _request(self, url, method="GET", json=None):
+This client is used to interact with a Json API server and perform various 
actions
+such as triggering DAG runs,deleting DAGs, interacting with pools, and 
getting lineage information.
+
+:param _api_base_url: The base URL for the Json API server.
+:param _session: A session object to use for making HTTP requests.
+"""
+
+def _request(self, url: str, json=None, method: str = "GET"):
+"""Make a request to the Json API server.
+
+:param url: The URL to send the request to.
+:param method: The HTTP method to use (e.g. "GET", "POST", "DELETE").
+:param json: A dictionary containing JSON data to send in the request 
body.
+:return: A dictionary containing the JSON response from the server.
+:rtype: dict

Review Comment:
   Absolutely agree with you, @ephraimbuddy, but in that case I receive 
following message from pre-commit run-mypy
   hook. Should we leave this method this without typehint?

   ![Screenshot from 2022-12-28 
18-49-43](https://user-images.githubusercontent.com/94890848/209837921-b2e560c0-8dc3-4fdc-bfa2-792551e199bc.png)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28569: Add general-purpose "notifier" concept to DAGs

[GitBox]

potiuk commented on code in PR #28569:
URL: https://github.com/apache/airflow/pull/28569#discussion_r1058441026


##
airflow/providers/slack/notifications/slack_notifier.py:
##
@@ -0,0 +1,87 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+import json
+
+from airflow.compat.functools import cached_property
+from airflow.exceptions import AirflowOptionalProviderFeatureException
+
+try:
+from airflow.notifications.basenotifier import BaseNotifier
+except ImportError as e:
+raise AirflowOptionalProviderFeatureException(e)

Review Comment:
   I think you should add more description here in the message. I.e. why this 
feature is not working and how to fix it  (migrate to Airflow 2.6+).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28619: Fix code readability, add docstrings to json_client

[GitBox]

potiuk commented on code in PR #28619:
URL: https://github.com/apache/airflow/pull/28619#discussion_r1058449396


##
airflow/api/client/json_client.py:
##
@@ -24,9 +24,25 @@
 
 
 class Client(api_client.Client):
-"""Json API client implementation."""
+"""Json API client implementation.
 
-def _request(self, url, method="GET", json=None):
+This client is used to interact with a Json API server and perform various 
actions
+such as triggering DAG runs,deleting DAGs, interacting with pools, and 
getting lineage information.
+
+:param _api_base_url: The base URL for the Json API server.
+:param _session: A session object to use for making HTTP requests.
+"""
+
+def _request(self, url: str, json=None, method: str = "GET"):
+"""Make a request to the Json API server.
+
+:param url: The URL to send the request to.
+:param method: The HTTP method to use (e.g. "GET", "POST", "DELETE").
+:param json: A dictionary containing JSON data to send in the request 
body.
+:return: A dictionary containing the JSON response from the server.
+:rtype: dict

Review Comment:
   Usually typing errors mean that there is something wrong and needs to be 
fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] MaicoTimmerman opened a new pull request, #28626: Introduce logging integration for Apache WebHDFS

[GitBox]

MaicoTimmerman opened a new pull request, #28626:
URL: https://github.com/apache/airflow/pull/28626

   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28626: Introduce logging integration for Apache WebHDFS

[GitBox]

boring-cyborg[bot] commented on PR #28626:
URL: https://github.com/apache/airflow/pull/28626#issuecomment-1366767160

   Congratulations on your first Pull Request and welcome to the Apache Airflow 
community! If you have any issues or are unsure about any anything please check 
our Contribution Guide 
(https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (flake8, mypy and type 
annotations). Our [pre-commits]( 
https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks)
 will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in 
`docs/` directory). Adding a new operator? Check this short 
[guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst)
 Consider adding an example DAG that shows how users should use it.
   - Consider using [Breeze 
environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for 
testing locally, it's a heavy docker but it ships with a working Airflow and a 
lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get 
the final approval from Committers.
   - Please follow [ASF Code of 
Conduct](https://www.apache.org/foundation/policies/conduct) for all 
communication including (but not limited to) comments on Pull Requests, Mailing 
list and Slack.
   - Be sure to read the [Airflow Coding style]( 
https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
   Apache Airflow is a community-driven project and together we are making it 
better .
   In case of doubts contact the developers at:
   Mailing List: d...@airflow.apache.org
   Slack: https://s.apache.org/airflow-slack
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on a diff in pull request #28625: Add note for pymssql requirement

[GitBox]

Taragolis commented on code in PR #28625:
URL: https://github.com/apache/airflow/pull/28625#discussion_r1058459895


##
docs/apache-airflow-providers-microsoft-mssql/index.rst:
##
@@ -91,6 +91,8 @@ PIP package  Version required
 ``pymssql``  ``>=2.1.5; platform_machine != 
"aarch64"``
 ===  
==
 
+Note: For ``pymssql`` installation some Linux distributions ``pip`` version is 
too old to support all the flavors of ``manylinux`` wheels, so upgrading 
``pip`` is necessary. An example of such distributions would be Ubuntu 18.04 or 
Python3.6 module in RHEL8 and CentOS8.

Review Comment:
   I just think that it could be general advice not to use system `pip` as well 
as outdated `pip` in 
https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html
   
   We do not have any official restrictions for `pip` version however in 
official Airflow Docker images uses latest version of pip (at the moment when 
new Airflow version released).
   
   But I'm sure that users might have a low of issues if the tried to install 
airflow by use default system pip:
   Ubuntu 18.04: python3-pip 9.0.1
   Ubuntu 20.04: python3-pip 20.0.2
   Debian 10: python3-pip 18.1
   Debian 11: python3-pip 20.3.4
   Centos 7: python3-pip 9.0.3
   Centos 8: python3-pip 9.0.3
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28097: logging poke info when external dag is not none and task_id and task_ids are none

[GitBox]

potiuk commented on PR #28097:
URL: https://github.com/apache/airflow/pull/28097#issuecomment-1366779235

   Tests are failing. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch constraints-main updated: Updating constraints. Build id:

[github-bot]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch constraints-main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/constraints-main by this push:
 new ae05c23304 Updating constraints. Build id:
ae05c23304 is described below

commit ae05c23304f90d199652af084710cd75d90e6585
Author: Automated GitHub Actions commit 
AuthorDate: Wed Dec 28 17:50:07 2022 +

Updating constraints. Build id:

This update in constraints is automatically committed by the CI 
'constraints-push' step based on
HEAD of '' in ''
with commit sha .

All tests passed in this build so we determined we can push the updated 
constraints.

See 
https://github.com/apache/airflow/blob/main/README.md#installing-from-pypi for 
details.
---
 constraints-3.10.txt  | 8 
 constraints-3.7.txt   | 8 
 constraints-3.8.txt   | 8 
 constraints-3.9.txt   | 8 
 constraints-no-providers-3.10.txt | 6 +++---
 constraints-no-providers-3.7.txt  | 6 +++---
 constraints-no-providers-3.8.txt  | 6 +++---
 constraints-no-providers-3.9.txt  | 6 +++---
 constraints-source-providers-3.10.txt | 4 ++--
 constraints-source-providers-3.7.txt  | 4 ++--
 constraints-source-providers-3.8.txt  | 4 ++--
 constraints-source-providers-3.9.txt  | 4 ++--
 12 files changed, 36 insertions(+), 36 deletions(-)

diff --git a/constraints-3.10.txt b/constraints-3.10.txt
index dae4deb2c3..dfe02ecc3f 100644
--- a/constraints-3.10.txt
+++ b/constraints-3.10.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-12-27T21:30:51Z
+# This constraints file was automatically generated on 2022-12-28T17:49:32Z
 # via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
 # This variant of constraints install uses the HEAD of the branch version for 
'apache-airflow' but installs
 # the providers from PIP-released packages at the moment of the constraint 
generation.
@@ -237,7 +237,7 @@ facebook-business==15.0.0
 fastavro==1.7.0
 fasteners==0.18
 fastjsonschema==2.16.2
-filelock==3.8.2
+filelock==3.9.0
 fissix==21.11.13
 flake8-colors==0.1.9
 flake8==6.0.0
@@ -423,7 +423,7 @@ pinotdb==0.4.12
 pipdeptree==2.3.3
 pipx==1.1.0
 pkginfo==1.9.2
-platformdirs==2.6.0
+platformdirs==2.6.1
 pluggy==1.0.0
 ply==3.11
 plyvel==1.5.0
@@ -593,7 +593,7 @@ types-protobuf==4.21.0.2
 types-python-dateutil==2.8.19.5
 types-python-slugify==7.0.0.1
 types-pytz==2022.7.0.0
-types-redis==4.3.21.6
+types-redis==4.3.21.7
 types-requests==2.28.11.7
 types-setuptools==65.6.0.2
 types-tabulate==0.9.0.0
diff --git a/constraints-3.7.txt b/constraints-3.7.txt
index c89ab2db88..a0d3f9db8f 100644
--- a/constraints-3.7.txt
+++ b/constraints-3.7.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-12-27T21:31:27Z
+# This constraints file was automatically generated on 2022-12-28T17:50:04Z
 # via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
 # This variant of constraints install uses the HEAD of the branch version for 
'apache-airflow' but installs
 # the providers from PIP-released packages at the moment of the constraint 
generation.
@@ -237,7 +237,7 @@ facebook-business==15.0.0
 fastavro==1.7.0
 fasteners==0.18
 fastjsonschema==2.16.2
-filelock==3.8.2
+filelock==3.9.0
 fissix==21.11.13
 flake8-colors==0.1.9
 flake8==3.9.2
@@ -425,7 +425,7 @@ pipdeptree==2.3.3
 pipx==1.1.0
 pkginfo==1.9.2
 pkgutil_resolve_name==1.3.10
-platformdirs==2.6.0
+platformdirs==2.6.1
 pluggy==1.0.0
 ply==3.11
 plyvel==1.5.0
@@ -594,7 +594,7 @@ types-protobuf==4.21.0.2
 types-python-dateutil==2.8.19.5
 types-python-slugify==7.0.0.1
 types-pytz==2022.7.0.0
-types-redis==4.3.21.6
+types-redis==4.3.21.7
 types-requests==2.28.11.7
 types-setuptools==65.6.0.2
 types-tabulate==0.9.0.0
diff --git a/constraints-3.8.txt b/constraints-3.8.txt
index c76abc7bb8..30688906b2 100644
--- a/constraints-3.8.txt
+++ b/constraints-3.8.txt
@@ -1,5 +1,5 @@
 #
-# This constraints file was automatically generated on 2022-12-27T21:31:17Z
+# This constraints file was automatically generated on 2022-12-28T17:49:55Z
 # via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
 # This variant of constraints install uses the HEAD of the branch version for 
'apache-airflow' but installs
 # the providers from PIP-released packages at the moment of the constraint 
generation.
@@ -238,7 +238,7 @@ facebook-business==15.0.0
 fastavro==1.7.0
 fasteners==0.18
 fastjsonschema==2.16.2
-filelock==3.8.2
+filelock==3.9.0
 fissix==21.11.13
 flake8-colors==0.1.9
 flake8==6.0.0
@@ -426,7 +426,7 @@ pipdeptree==2.3.3
 pipx==1.1.0
 pkginfo==1.9.2
 pkgutil_resolve_name==1.3.10
-platformdirs==2.6.0
+platformdirs==2.6.1
 pluggy==1.0.0
 ply==3.11
 plyvel==1.5.0
@@ -596,7 +596,7 @@ types-protobuf==4.21.0.2
 

[GitHub] [airflow] Taragolis commented on pull request #28590: collect sawarning as error in pytest

[GitBox]

Taragolis commented on PR #28590:
URL: https://github.com/apache/airflow/pull/28590#issuecomment-1366560903

   @BobDu I think this changes conflicts with 
[pytest-capture-warnings](https://github.com/athinkingape/pytest-capture-warnings)
 plugin. I check locally and with our current settings and plugins nothing 
happen, unfortunetly
   
   Without disable `pytest-capture-warnings`
   ---
   
   ```console
   
   root@a33668319fad:/opt/airflow# pytest 
tests/jobs/test_scheduler_job.py::TestSchedulerJob
   
    test 
session starts 

   platform linux -- Python 3.7.16, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- 
/usr/local/bin/python
   cachedir: .pytest_cache
   rootdir: /opt/airflow, configfile: pytest.ini
   plugins: cov-4.0.0, asyncio-0.20.3, rerunfailures-9.1.1, instafail-0.4.2, 
anyio-3.6.2, timeouts-1.2.1, xdist-3.1.0, requests-mock-1.10.0, 
capture-warnings-0.0.4, httpx-0.21.2, time-machine-2.8.2
   asyncio: mode=strict
   setup timeout: 0.0s, execution timeout: 0.0s, teardown timeout: 0.0s
   collected 142 items
   
   ...
   
   All Warning errors can be found in the warnings.txt file.
   
   ...
   
   == 139 passed, 3 
skipped in 17.45s ==
   ```
   
   Disable `pytest-capture-warnings`
   ---
   
   ```console
   
   root@a33668319fad:/opt/airflow# pytest -p no:capture_warnings 
tests/jobs/test_scheduler_job.py::TestSchedulerJob
   
    test 
session starts 

   platform linux -- Python 3.7.16, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- 
/usr/local/bin/python
   cachedir: .pytest_cache
   rootdir: /opt/airflow, configfile: pytest.ini
   plugins: cov-4.0.0, asyncio-0.20.3, rerunfailures-9.1.1, instafail-0.4.2, 
anyio-3.6.2, timeouts-1.2.1, xdist-3.1.0, requests-mock-1.10.0, httpx-0.21.2, 
time-machine-2.8.2
   asyncio: mode=strict
   setup timeout: 0.0s, execution timeout: 0.0s, teardown timeout: 0.0s
   collected 142 items 
   
   ...
   
   == short 
test summary info 
==
   ...
   FAILED 
tests/jobs/test_scheduler_job.py::TestSchedulerJob::test_find_zombies_nothing - 
sqlalchemy.exc.SAWarning: SELECT statement has a cartesian product ...
   FAILED tests/jobs/test_scheduler_job.py::TestSchedulerJob::test_find_zombies 
- sqlalchemy.exc.SAWarning: SELECT statement has a cartesian product between ...
   FAILED 
tests/jobs/test_scheduler_job.py::TestSchedulerJob::test_find_zombies_handle_failure_callbacks_are_correctly_passed_to_dag_processor
 - sqlalchemy.e...
   == 3 failed, 136 passed, 3 
skipped, 26 warnings in 18.43s 
===
   
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on pull request #28353: Fix the sql syntax in merge_data

[GitBox]

Taragolis commented on PR #28353:
URL: https://github.com/apache/airflow/pull/28353#issuecomment-1366568491

   @avicol doc spellcheck  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] jbampton opened a new pull request, #28623: Fix typo

[GitBox]

jbampton opened a new pull request, #28623:
URL: https://github.com/apache/airflow/pull/28623

   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] raj-wadhwa commented on issue #22132: Show average dag run duration

[GitBox]

raj-wadhwa commented on issue #22132:
URL: https://github.com/apache/airflow/issues/22132#issuecomment-1366736464

   Is there any progress in this area? I was looking for this info to be 
returned in the API for DAG Run info: 
   `ETA` or `Estimated_Time_To_Completion`
   
   Something like this.
   Is anyone aware that it's being developed? or is likely to be picked up for 
development?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #24669: setting `resources` in `KubernetesPodOperator` breaks when used with `Operator.partial`

[GitBox]

potiuk commented on issue #24669:
URL: https://github.com/apache/airflow/issues/24669#issuecomment-1366743181

   I think you should then open a new discussion and explain what you tried and 
show how it did not work. Commenting on closed issue especially one that 
relates to different problem - does not do any good. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28541: Speed up provider validation pre-commit

[GitBox]

potiuk commented on PR #28541:
URL: https://github.com/apache/airflow/pull/28541#issuecomment-1366770329

   Anyone needs some speed-ups :) ? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk opened a new pull request, #28630: Improve "other" test category selection

[GitBox]

potiuk opened a new pull request, #28630:
URL: https://github.com/apache/airflow/pull/28630

   The "Other" test category automatically selects all tests that are not 
included in any of the regular categories. That is to make sure that we do not 
forget to add any directory that has been added. However this led to a long 
directory selection for "Other" category including system tests that have been 
automatically added there. However those tests are always skipped in regular 
tests and collecting those tests during "Other" execution is not needed and 
slows it down.
   
   Similarly System tests changes were treated as "Other change" for incoming 
PRs. This means that any change to system tests would trigger "all tests" 
rather than selective subset of those - the same as any core change.
   
   However System tests are also part of the documentation, so any change in 
system tests should trigger docs
   builds.
   
   This change improves it in a few ways:
   
   * Other tests now do not include "System Tests" - they are treated the same 
way as other test categories (but not included in test category selection for 
now - until we get a good way of breeze-integration for System Tests
   
   * They are also excluded from treating them as "other" change when 
considering which tests to run. Changes to system tests will not trigger "all" 
tests, just those that accompanying changes would trigger.
   
   * The changes to system tests only, however, triggers docs build because 
those are triggered by any source change.a
   
   * The __pycache__ directories are removed from the list of "Other" packages 
to run.
   
   * In order to make sure system tests are pytest-collectable, we perform 
pytest collection for all tests right after downloading the CI images and 
verifying them. This makes sure that the tests are collectible before we even 
attempt to run them - this way we avoid unnecessary machine spin-up and breze 
installation for the multiple jobs that run the tests. This slows down feedback 
time a litle, but should increase overall robustness of the test suite.
   
   * an old, unused nosetest collection script doing the same in the past has 
been removed (it was discovered during implementation)
   
   Noticed in: #28319
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on pull request #28630: Improve "other" test category selection

[GitBox]

potiuk commented on PR #28630:
URL: https://github.com/apache/airflow/pull/28630#issuecomment-1366832706

   The list of "other" folders now is far shorter:
   
   
![image](https://user-images.githubusercontent.com/595491/209854635-3efcf2f6-2bee-4028-82b1-8ecfa0256015.png)
   
   
   Comparing to 
   
   https://github.com/apache/airflow/pull/28319#issuecomment-1349010577


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on a diff in pull request #28569: Add general-purpose "notifier" concept to DAGs

[GitBox]

Taragolis commented on code in PR #28569:
URL: https://github.com/apache/airflow/pull/28569#discussion_r1058237022


##
airflow/providers/slack/provider.yaml:
##
@@ -39,7 +39,7 @@ versions:
   - 1.0.0
 
 dependencies:
-  - apache-airflow>=2.3.0
+  - apache-airflow>=2.6.0.dev0

Review Comment:
   I think this lines in 
`airflow/providers/slack/notifications/slack_notifier.py` would be enough, so 
we keep all other feature compatible with 2.3, and notifiers will work only 
with 2.6 and user get an error if try to use it in old version
   
   ```python
   
   from airflow.exceptions import AirflowOptionalProviderFeatureException
   
   try:
   from airflow.notifications.basenotifier import BaseNotifier
   except ImportError as e:
   raise AirflowOptionalProviderFeatureException(e)
   ```
   
   @potiuk correct me if I wrong. Because 
`AirflowOptionalProviderFeatureException` it it quite new feature - only work 
with Airflow 2.3+ , and we just drop support 2.2 for providers



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ReneTC commented on issue #28603: SFTPOperator does not finish downloading

[GitBox]

ReneTC commented on issue #28603:
URL: https://github.com/apache/airflow/issues/28603#issuecomment-1366555038

   I see. I am using
   
   Composer version:
   2.0.28
   
   Airflow version:
   2.3.3
   
   My providers settings are set to:
   ` apache-airflow-providers-sftp  = "~=2.5.0"`
   
   I will convince admin to update and report back. 
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on pull request #28502: Migrate DagFileProcessor.manage_slas to Internal API

[GitBox]

Taragolis commented on PR #28502:
URL: https://github.com/apache/airflow/pull/28502#issuecomment-1366570974

   @vincbeck need resolve conflicts


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] vchiapaikeo opened a new pull request, #28624: Fix UI caret direction

[GitBox]

vchiapaikeo opened a new pull request, #28624:
URL: https://github.com/apache/airflow/pull/28624

   
   
   closes: [#ISSUE](https://github.com/apache/airflow/issues/27166)
   
   Saw that https://github.com/apache/airflow/pull/27195/files might be stale 
and it was simple enough so I went ahead and created a PR to fix. I also added 
a fix for the dag import errors as well. The result looks like this:
   
   Collapsed task group:
   https://user-images.githubusercontent.com/9200263/209828350-9d6b4b39-3490-4638-a53a-c8650efd98cd.png;>
   
   Uncollapsed task group:
   https://user-images.githubusercontent.com/9200263/209828384-fe73ad4e-5ca7-4020-8b8a-02e3cfb90015.png;>
   
   
   Collapsed dag error:
   https://user-images.githubusercontent.com/9200263/209828434-cd7bb775-18da-4e2e-80ce-ab06eb775ffd.png;>
   
   https://user-images.githubusercontent.com/9200263/209828455-cb74cb94-e52d-4769-9732-cdcb221fe12e.png;>
   
   Uncollapsed dag error:
   https://user-images.githubusercontent.com/9200263/209828490-9b2b1f1a-0dd5-40c8-9b4b-d653ceccfcd2.png;>
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #25032: Dynamic Task Mapping - Depth-first execution

[GitBox]

potiuk commented on issue #25032:
URL: https://github.com/apache/airflow/issues/25032#issuecomment-1366749247

   @wangyaopw As @uranusjr mentioned - it's better not to attempt it yet. There 
are bugs like just fixed https://github.com/apache/airflow/pull/28592 (which is 
likely the fix to your problem) which are preventing from using this feature 
**just yet**.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil merged pull request #28609: Fix Docs Incorrect Example in How To Create Custom Decorator

[GitBox]

kaxil merged PR #28609:
URL: https://github.com/apache/airflow/pull/28609


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28609: Fix Docs Incorrect Example in How To Create Custom Decorator

[GitBox]

boring-cyborg[bot] commented on PR #28609:
URL: https://github.com/apache/airflow/pull/28609#issuecomment-1366755472

   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (d4dbb0077a -> 761aa59f92)

[kaxilnaik]

This is an automated email from the ASF dual-hosted git repository.

kaxilnaik pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from d4dbb0077a Guard not-yet-expanded ti in trigger rule dep (#28592)
 add 761aa59f92 Fix Incorrect Example (#28609)

No new revisions were added by this update.

Summary of changes:
 docs/apache-airflow/howto/create-custom-decorator.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[GitHub] [airflow] rkarish commented on a diff in pull request #28625: Add note for pymssql requirement

[GitBox]

rkarish commented on code in PR #28625:
URL: https://github.com/apache/airflow/pull/28625#discussion_r1058450927


##
docs/apache-airflow-providers-microsoft-mssql/index.rst:
##
@@ -91,6 +91,8 @@ PIP package  Version required
 ``pymssql``  ``>=2.1.5; platform_machine != 
"aarch64"``
 ===  
==
 
+Note: For ``pymssql`` installation some Linux distributions ``pip`` version is 
too old to support all the flavors of ``manylinux`` wheels, so upgrading 
``pip`` is necessary. An example of such distributions would be Ubuntu 18.04 or 
Python3.6 module in RHEL8 and CentOS8.

Review Comment:
   I changed this to use the sphinx note directive. Would it make sense to 
remove the example section of the note because of the min stable python version 
for airflow? I just took it from the documentation for the package on PyPI - 
https://pypi.org/project/pymssql/ 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] rkarish commented on a diff in pull request #28625: Add note for pymssql requirement

[GitBox]

rkarish commented on code in PR #28625:
URL: https://github.com/apache/airflow/pull/28625#discussion_r1058460835


##
docs/apache-airflow-providers-microsoft-mssql/index.rst:
##
@@ -91,6 +91,8 @@ PIP package  Version required
 ``pymssql``  ``>=2.1.5; platform_machine != 
"aarch64"``
 ===  
==
 
+Note: For ``pymssql`` installation some Linux distributions ``pip`` version is 
too old to support all the flavors of ``manylinux`` wheels, so upgrading 
``pip`` is necessary. An example of such distributions would be Ubuntu 18.04 or 
Python3.6 module in RHEL8 and CentOS8.

Review Comment:
   I removed the confusing example and added a link to the package on PyPI for 
more details.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] kaxil commented on a diff in pull request #28625: Add note for pymssql requirement

[GitBox]

kaxil commented on code in PR #28625:
URL: https://github.com/apache/airflow/pull/28625#discussion_r1058492281


##
docs/apache-airflow-providers-microsoft-mssql/index.rst:
##
@@ -91,6 +91,8 @@ PIP package  Version required
 ``pymssql``  ``>=2.1.5; platform_machine != 
"aarch64"``
 ===  
==
 
+.. note:: For ``pymssql`` installation some Linux distributions ``pip`` 
version is too old to support all the flavors of ``manylinux`` wheels, so 
upgrading ``pip`` is necessary. More details can be found on `PyPI 
`_.

Review Comment:
   ```suggestion
   .. note:: ``pip`` version on some Linux distributions ``pip`` might be 
outdated and not support all the flavors of ``manylinux`` wheels, so upgrading 
``pip`` is necessary. More details can be found on `PyPI 
`_.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #28625: Add note for pymssql requirement

[GitBox]

ephraimbuddy commented on code in PR #28625:
URL: https://github.com/apache/airflow/pull/28625#discussion_r1058500304


##
docs/apache-airflow-providers-microsoft-mssql/index.rst:
##
@@ -91,6 +91,8 @@ PIP package  Version required
 ``pymssql``  ``>=2.1.5; platform_machine != 
"aarch64"``
 ===  
==
 
+.. note:: ``pip`` version on some Linux distributions ``pip`` might be 
outdated and not support all the flavors of ``manylinux`` wheels, so upgrading 
``pip`` is necessary. More details can be found on `PyPI 
`_.

Review Comment:
   ```suggestion
   .. note:: ``pip`` version on some Linux distributions might be outdated and 
not support all the flavors of ``manylinux`` wheels, so upgrading ``pip`` is 
necessary. More details can be found on `PyPI 
`_.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (467a5e3ab2 -> 0fae6a0c9e)

[taragolis]

This is an automated email from the ASF dual-hosted git repository.

taragolis pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 467a5e3ab2 Fix calendar view for CronTriggerTimeTable dags (#28411)
 add 0fae6a0c9e Add table_resource to template fields for 
BigQueryCreateEmptyTableOperator (#28235)

No new revisions were added by this update.

Summary of changes:
 airflow/providers/google/cloud/operators/bigquery.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28235: Add table_resource to template fields for BigQueryCreateEmptyTableOperator

[GitBox]

boring-cyborg[bot] commented on PR #28235:
URL: https://github.com/apache/airflow/pull/28235#issuecomment-1366455750

   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis merged pull request #28235: Add table_resource to template fields for BigQueryCreateEmptyTableOperator

[GitBox]

Taragolis merged PR #28235:
URL: https://github.com/apache/airflow/pull/28235


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] stamixthereal opened a new pull request, #28619: Fix code readability, add docstrings to json_client

[GitBox]

stamixthereal opened a new pull request, #28619:
URL: https://github.com/apache/airflow/pull/28619

   Code readability improvements in Client class and methods in json_client


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bdsoha closed pull request #28621: Extracted workspace cleanup to a CI action

[GitBox]

bdsoha closed pull request #28621: Extracted workspace cleanup to a CI action
URL: https://github.com/apache/airflow/pull/28621


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #28280: Inconsistent handling of newlines in tasks

[GitBox]

potiuk commented on issue #28280:
URL: https://github.com/apache/airflow/issues/28280#issuecomment-1366520133

   > Unfortunately we would technically break backward compatibility if we do 
this, but I don’t think there’s a good way around it. Maybe alternatively we 
should introduce a config to normalise when a variable is _read_ (and still 
store them as-is without normalisation) so people can enable it if needed. 
(It’s probably not very relevant for most use cases.)
   
   Yep. The 'read' time normalisation is essentially what 'text=True` does in 
Python stdlib's read. I guess indeed following this approach and adding 
'get(var, text: bool) for variable would be the best way to implement it. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28620: Minor update to Scheduler documentation

[GitBox]

boring-cyborg[bot] commented on PR #28620:
URL: https://github.com/apache/airflow/pull/28620#issuecomment-1366479223

   Congratulations on your first Pull Request and welcome to the Apache Airflow 
community! If you have any issues or are unsure about any anything please check 
our Contribution Guide 
(https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (flake8, mypy and type 
annotations). Our [pre-commits]( 
https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks)
 will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in 
`docs/` directory). Adding a new operator? Check this short 
[guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst)
 Consider adding an example DAG that shows how users should use it.
   - Consider using [Breeze 
environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for 
testing locally, it's a heavy docker but it ships with a working Airflow and a 
lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get 
the final approval from Committers.
   - Please follow [ASF Code of 
Conduct](https://www.apache.org/foundation/policies/conduct) for all 
communication including (but not limited to) comments on Pull Requests, Mailing 
list and Slack.
   - Be sure to read the [Airflow Coding style]( 
https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
   Apache Airflow is a community-driven project and together we are making it 
better .
   In case of doubts contact the developers at:
   Mailing List: d...@airflow.apache.org
   Slack: https://s.apache.org/airflow-slack
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] maxnathaniel opened a new pull request, #28620: Minor update to Scheduler documentation

[GitBox]

maxnathaniel opened a new pull request, #28620:
URL: https://github.com/apache/airflow/pull/28620

   Made some minor changes to the Scheduler documentation. One is a typo, and 
others are changes to make the sentences less awkward.
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bdsoha opened a new pull request, #28621: Extracted workspace cleanup to a CI action

[GitBox]

bdsoha opened a new pull request, #28621:
URL: https://github.com/apache/airflow/pull/28621

   Extracted the workspace cleanup step to a dedicated CI action.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] stamixthereal opened a new pull request, #28622: Fix code docstrings

[GitBox]

stamixthereal opened a new pull request, #28622:
URL: https://github.com/apache/airflow/pull/28622

   Fix docstrings grammar


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] bkossakowska opened a new pull request, #28618: Add deferrable mode to DataprocInstantiateWorkflowTemplateOperator

[GitBox]

bkossakowska opened a new pull request, #28618:
URL: https://github.com/apache/airflow/pull/28618

   
   
   Add deferrable mode to DataprocInstantiateWorkflowTemplateOperator
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] stamixthereal closed pull request #28607: Minor typos and doc-strings added

[GitBox]

stamixthereal closed pull request #28607: Minor typos and doc-strings added
URL: https://github.com/apache/airflow/pull/28607


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #28569: Add general-purpose "notifier" concept to DAGs

[GitBox]

ephraimbuddy commented on code in PR #28569:
URL: https://github.com/apache/airflow/pull/28569#discussion_r1058190702


##
airflow/providers/slack/provider.yaml:
##
@@ -39,7 +39,7 @@ versions:
   - 1.0.0
 
 dependencies:
-  - apache-airflow>=2.3.0
+  - apache-airflow>=2.6.0.dev0

Review Comment:
   Hi @potiuk, @Taragolis what is the right way to mark this PR for 2.6? Can't 
find any clue from the error messages



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (0fae6a0c9e -> 0e349d80bb)

[taragolis]

This is an automated email from the ASF dual-hosted git repository.

taragolis pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 0fae6a0c9e Add table_resource to template fields for 
BigQueryCreateEmptyTableOperator (#28235)
 add 0e349d80bb Add FTPSFileTransmitOperator (#28318)

No new revisions were added by this update.

Summary of changes:
 airflow/providers/ftp/operators/ftp.py |  19 ++-
 .../operators/index.rst|  32 -
 tests/providers/ftp/operators/test_ftp.py  | 150 +
 tests/system/providers/ftp/example_ftp.py  |  35 -
 4 files changed, 202 insertions(+), 34 deletions(-)

[GitHub] [airflow] Taragolis merged pull request #28318: Add FTPSFileTransmitOperator

[GitBox]

Taragolis merged PR #28318:
URL: https://github.com/apache/airflow/pull/28318


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish commented on pull request #28616: Reduce logging level of `gunicorn.access` to warning

[GitBox]

dstandish commented on PR #28616:
URL: https://github.com/apache/airflow/pull/28616#issuecomment-1366477472

   Ah interesting ... thanks for that.  Now the behavior makes a lot more 
sense.   Setting to empty string works for me.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] dstandish closed pull request #28616: Reduce logging level of `gunicorn.access` to warning

[GitBox]

dstandish closed pull request #28616: Reduce logging level of `gunicorn.access` 
to warning
URL: https://github.com/apache/airflow/pull/28616


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[airflow] branch main updated (0e349d80bb -> 2169b39d7b)

[taragolis]

This is an automated email from the ASF dual-hosted git repository.

taragolis pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


from 0e349d80bb Add FTPSFileTransmitOperator (#28318)
 add 2169b39d7b Migrate Google Cloud hooks test to `pytest` (#28301)

No new revisions were added by this update.

Summary of changes:
 tests/providers/google/cloud/hooks/test_automl.py  |   5 +-
 .../providers/google/cloud/hooks/test_bigquery.py  | 108 
 .../google/cloud/hooks/test_bigquery_dts.py|   5 +-
 .../providers/google/cloud/hooks/test_bigtable.py  |  29 +--
 .../google/cloud/hooks/test_cloud_build.py |   5 +-
 .../google/cloud/hooks/test_cloud_composer.py  |   7 +-
 .../google/cloud/hooks/test_cloud_memorystore.py   |  14 +-
 .../providers/google/cloud/hooks/test_cloud_sql.py |  62 +++--
 .../hooks/test_cloud_storage_transfer_service.py   |  39 ++-
 tests/providers/google/cloud/hooks/test_compute.py |  13 +-
 .../google/cloud/hooks/test_compute_ssh.py |   3 +-
 .../google/cloud/hooks/test_datacatalog.py |  18 +-
 .../providers/google/cloud/hooks/test_dataflow.py  | 278 +++--
 .../providers/google/cloud/hooks/test_dataform.py  |   5 +-
 .../providers/google/cloud/hooks/test_dataplex.py  |   6 +-
 .../providers/google/cloud/hooks/test_dataprep.py  |   8 +-
 .../providers/google/cloud/hooks/test_dataproc.py  |  29 ++-
 .../google/cloud/hooks/test_dataproc_metastore.py  |  10 +-
 .../providers/google/cloud/hooks/test_datastore.py |   5 +-
 tests/providers/google/cloud/hooks/test_dlp.py |   5 +-
 .../providers/google/cloud/hooks/test_functions.py |   9 +-
 tests/providers/google/cloud/hooks/test_gcs.py | 109 
 tests/providers/google/cloud/hooks/test_gdm.py |   5 +-
 tests/providers/google/cloud/hooks/test_kms.py |   5 +-
 .../google/cloud/hooks/test_kubernetes_engine.py   |  21 +-
 .../google/cloud/hooks/test_life_sciences.py   |  13 +-
 tests/providers/google/cloud/hooks/test_looker.py  |   5 +-
 .../providers/google/cloud/hooks/test_mlengine.py  |  11 +-
 .../google/cloud/hooks/test_natural_language.py|   5 +-
 .../providers/google/cloud/hooks/test_os_login.py  |  18 +-
 tests/providers/google/cloud/hooks/test_pubsub.py  |  67 +++--
 .../google/cloud/hooks/test_secret_manager.py  |   3 +-
 tests/providers/google/cloud/hooks/test_spanner.py |   9 +-
 .../google/cloud/hooks/test_speech_to_text.py  |   5 +-
 .../google/cloud/hooks/test_stackdriver.py |   3 +-
 tests/providers/google/cloud/hooks/test_tasks.py   |   5 +-
 .../google/cloud/hooks/test_text_to_speech.py  |   5 +-
 .../providers/google/cloud/hooks/test_translate.py |   5 +-
 .../google/cloud/hooks/test_video_intelligence.py  |   5 +-
 tests/providers/google/cloud/hooks/test_vision.py  |  35 ++-
 .../google/cloud/hooks/vertex_ai/test_auto_ml.py   |  10 +-
 .../hooks/vertex_ai/test_batch_prediction_job.py   |  10 +-
 .../cloud/hooks/vertex_ai/test_custom_job.py   |  10 +-
 .../google/cloud/hooks/vertex_ai/test_dataset.py   |  10 +-
 .../cloud/hooks/vertex_ai/test_endpoint_service.py |  10 +-
 .../vertex_ai/test_hyperparameter_tuning_job.py|  10 +-
 .../cloud/hooks/vertex_ai/test_model_service.py|  10 +-
 47 files changed, 527 insertions(+), 540 deletions(-)

[GitHub] [airflow] Taragolis merged pull request #28301: Migrate Google Cloud hooks test to `pytest`

[GitBox]

Taragolis merged PR #28301:
URL: https://github.com/apache/airflow/pull/28301


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] uranusjr commented on pull request #28616: Reduce logging level of `gunicorn.access` to warning

[GitBox]

uranusjr commented on PR #28616:
URL: https://github.com/apache/airflow/pull/28616#issuecomment-1366454296

   I took a look at the surrounding code and it seems that access logs are 
currently explicitly enabled by whoever implemented it in the first place. 
Gunicorn does not emit access logs unless you specify `--access-logfile`, which 
Airflow does here:
   
   
https://github.com/apache/airflow/blob/467a5e3ab287013db2a5381ef4a642e912f8b45b/airflow/cli/commands/webserver_command.py#L405-L406
   
   and there’s no way to turn if off since `args.access_logfile` has a 
hard-coded default of `-` (send to stdout).
   
   If we want to make the stdout log less verbose, perhaps we should simply 
change the default of `access_logfile` to an empty string? Or maybe we should 
respect whoever implemented this and keep the current logging…?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #27952: can not use output of task decorator as input for external_task_ids of ExternalTaskSensor

[GitBox]

potiuk commented on issue #27952:
URL: https://github.com/apache/airflow/issues/27952#issuecomment-1366522100

   Just wondering if we can somehow add a precommit to prevent such cases (and 
scan all our code to search for similar cases) 樂 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28631: Clear DB between each separate providers tests

[GitBox]

potiuk commented on code in PR #28631:
URL: https://github.com/apache/airflow/pull/28631#discussion_r1058531880


##
tests/providers/conftest.py:
##
@@ -0,0 +1,75 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+import pytest
+
+from tests.test_utils import db
+
+# Providers with subpackages
+INNER_PROVIDERS = {

Review Comment:
   There is also one side-effect that we should take into account here.  This 
is fine when we run the tests in CI, but the way it is implemented now, it will 
also run the auto-fixture even if you run a single unit test. This will 
dramatically slow-down any iterations you do I am afraid (and it does not 
really matter if we do it only for Providers or for whole airflow).
   
   I think this can be solved by only doing this when we run for:
   
   1) the first time at all (so some kind of check if db is initialized) 
   2) or when we run in CI (`CI` env variable set to "true").
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on a diff in pull request #28631: Clear DB between each separate providers tests

[GitBox]

Taragolis commented on code in PR #28631:
URL: https://github.com/apache/airflow/pull/28631#discussion_r1058532316


##
tests/providers/conftest.py:
##
@@ -0,0 +1,75 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+import pytest
+
+from tests.test_utils import db
+
+# Providers with subpackages
+INNER_PROVIDERS = {

Review Comment:
   I think we could . Just need to scan directory until found one of the 
folder: `hooks`, `operators`, `sensors`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] Taragolis commented on a diff in pull request #28631: Clear DB between each separate providers tests

[GitBox]

Taragolis commented on code in PR #28631:
URL: https://github.com/apache/airflow/pull/28631#discussion_r1058532316


##
tests/providers/conftest.py:
##
@@ -0,0 +1,75 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+import pytest
+
+from tests.test_utils import db
+
+# Providers with subpackages
+INNER_PROVIDERS = {

Review Comment:
   I think we could just need to scan directory until found one of the folder: 
`hooks`, `operators`, `sensors`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk opened a new pull request, #28633: Variables set in variables.env are automatically exported

[GitBox]

potiuk opened a new pull request, #28633:
URL: https://github.com/apache/airflow/pull/28633

   The variables set in variables.env were not automatically exported.
   
   
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
 for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party 
License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a 
newsfragment file, named `{pr_number}.significant.rst` or 
`{issue_number}.significant.rst`, in 
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28631: Clear DB between each separate providers tests

[GitBox]

potiuk commented on code in PR #28631:
URL: https://github.com/apache/airflow/pull/28631#discussion_r1058576111


##
tests/providers/conftest.py:
##
@@ -0,0 +1,75 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+import pytest
+
+from tests.test_utils import db
+
+# Providers with subpackages
+INNER_PROVIDERS = {
+"alibaba",
+"amazon",
+"apache",
+"atlassian",
+"common",
+"dbt",
+"facebook",
+"google",
+"microsoft",
+}
+PROVIDERS_PACKAGES = set()
+
+
+def get_test_provider_name(m):
+"""Extract provider name from module full qualname."""
+_, _, name = m.__name__.partition("providers.")
+for inner_provider in INNER_PROVIDERS:
+if name.startswith(inner_provider):
+return ".".join(name.split(".", 2)[:2])
+return name.split(".", 1)[0]
+
+
+@pytest.fixture(scope="module", autouse=True)
+def _clear_db_between_providers_tests(request):
+"""Clear DB between each separate provider package test runs."""
+provider_name = get_test_provider_name(request.module)
+if provider_name and provider_name not in PROVIDERS_PACKAGES:
+PROVIDERS_PACKAGES.add(provider_name)
+db.clear_db_runs()

Review Comment:
   > Also tried.
   
   For some reason it works when we run --with-db-init. I think this is a 
**real** bug that we need to trace if it does not work here.  I recall similar 
errors raised by our users.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airflow] potiuk commented on a diff in pull request #28631: Clear DB between each separate providers tests

[GitBox]

potiuk commented on code in PR #28631:
URL: https://github.com/apache/airflow/pull/28631#discussion_r1058576453


##
tests/providers/conftest.py:
##
@@ -0,0 +1,75 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+import pytest
+
+from tests.test_utils import db
+
+# Providers with subpackages
+INNER_PROVIDERS = {
+"alibaba",
+"amazon",
+"apache",
+"atlassian",
+"common",
+"dbt",
+"facebook",
+"google",
+"microsoft",
+}
+PROVIDERS_PACKAGES = set()
+
+
+def get_test_provider_name(m):
+"""Extract provider name from module full qualname."""
+_, _, name = m.__name__.partition("providers.")
+for inner_provider in INNER_PROVIDERS:
+if name.startswith(inner_provider):
+return ".".join(name.split(".", 2)[:2])
+return name.split(".", 1)[0]
+
+
+@pytest.fixture(scope="module", autouse=True)
+def _clear_db_between_providers_tests(request):
+"""Clear DB between each separate provider package test runs."""
+provider_name = get_test_provider_name(request.module)
+if provider_name and provider_name not in PROVIDERS_PACKAGES:
+PROVIDERS_PACKAGES.add(provider_name)
+db.clear_db_runs()

Review Comment:
   (BTW. Now you know why I have not tried it **just yet**, I kinda 
subconsciously expected it won't be super easy). 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org