[GitHub] [airflow] uranusjr commented on a diff in pull request #33570: Fix waiting setup tasks when they are not a direct upstream

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33570: URL: https://github.com/apache/airflow/pull/33570#discussion_r1309928233 ## airflow/models/dag.py: ## @@ -717,6 +718,11 @@ def validate_setup_teardown(self): :meta private: """ for task in self.tasks: +

[GitHub] [airflow] bfeif opened a new issue, #33911: Add get_polars_df() method to BigQueryHook.

2023-08-30 Thread via GitHub
bfeif opened a new issue, #33911: URL: https://github.com/apache/airflow/issues/33911 ### Description This is mostly self-explanatory. The `BigQueryHook` already has a `get_pandas_df()`; this issue aims to add an analogous `get_polars_df()` method that would return a polars

[GitHub] [airflow] boring-cyborg[bot] commented on issue #33911: Add get_polars_df() method to BigQueryHook.

2023-08-30 Thread via GitHub
boring-cyborg[bot] commented on issue #33911: URL: https://github.com/apache/airflow/issues/33911#issuecomment-1698795744 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[GitHub] [airflow] Lee-W commented on pull request #33825: Add DefaultAzureCredential support to AzureContainerRegistryHook

2023-08-30 Thread via GitHub
Lee-W commented on PR #33825: URL: https://github.com/apache/airflow/pull/33825#issuecomment-1698795945 > Right - this is a "Real" problem now you see. Got it. Just get it fixed. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow-site] potiuk commented on pull request #857: Adding information about Open Data Discovery (ODD) integration

2023-08-30 Thread via GitHub
potiuk commented on PR #857: URL: https://github.com/apache/airflow-site/pull/857#issuecomment-1699083119 I think it's best to link to one of your GitHub repos and have a readme there explaining how to integrate airflow. -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] derbeneviv commented on issue #33894: AWS connection doesn't convert verify option to bool

2023-08-30 Thread via GitHub
derbeneviv commented on issue #33894: URL: https://github.com/apache/airflow/issues/33894#issuecomment-1699092379 [Taragolis](https://github.com/Taragolis) well, i didn't, and it seems to work ``` {"endpoint_url": "https://mu_url;, "verify": false} ``` however, that's

[GitHub] [airflow] wolfdn commented on pull request #33914: Cleanup Docker operator logging

2023-08-30 Thread via GitHub
wolfdn commented on PR #33914: URL: https://github.com/apache/airflow/pull/33914#issuecomment-1699123215 > Does this PR aim to solve #33692 ? I think https://github.com/apache/airflow/issues/33692 is unrelated to the issue that I'm trying to solve here. This PR is about the

[GitHub] [airflow] eladkal closed issue #9005: Hive Operator mapred_job_name_template missing from config template

2023-08-30 Thread via GitHub
eladkal closed issue #9005: Hive Operator mapred_job_name_template missing from config template URL: https://github.com/apache/airflow/issues/9005 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] eladkal commented on issue #9005: Hive Operator mapred_job_name_template missing from config template

2023-08-30 Thread via GitHub
eladkal commented on issue #9005: URL: https://github.com/apache/airflow/issues/9005#issuecomment-1698715407 Closing as stale issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] potiuk commented on a diff in pull request #33637: Use a trimmed version of README.md for PyPI

2023-08-30 Thread via GitHub
potiuk commented on code in PR #33637: URL: https://github.com/apache/airflow/pull/33637#discussion_r1309882203 ## setup.cfg: ## @@ -22,7 +22,7 @@ author = Apache Software Foundation author_email = d...@airflow.apache.org url = https://airflow.apache.org/ version = attr:

[GitHub] [airflow] uranusjr commented on a diff in pull request #33570: Fix waiting setup tasks when they are not a direct upstream

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33570: URL: https://github.com/apache/airflow/pull/33570#discussion_r1309949531 ## airflow/ti_deps/deps/trigger_rule_dep.py: ## @@ -121,6 +132,13 @@ def _evaluate_trigger_rule( from airflow.models.operator import needs_expansion

[GitHub] [airflow] Taragolis commented on issue #33688: Tasks taking too long time after 2.7.0 Airflow update

2023-08-30 Thread via GitHub
Taragolis commented on issue #33688: URL: https://github.com/apache/airflow/issues/33688#issuecomment-1698823134 > here is the performance activity for the last 5 hours. The peak at 3 AM we see represents also a peak in activity in terms of DAGs running. Anything particular you want to

[GitHub] [airflow] uranusjr commented on pull request #33746: Refactor: remove unused state - SHUTDOWN

2023-08-30 Thread via GitHub
uranusjr commented on PR #33746: URL: https://github.com/apache/airflow/pull/33746#issuecomment-1698991410 Is `TaskInstanceState.SHUTDOWN` documented, or just `State.SHUTDOWN`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] avkirilishin opened a new pull request, #33915: Take into account removed upstream tasks for non-mapped tasks.

2023-08-30 Thread via GitHub
avkirilishin opened a new pull request, #33915: URL: https://github.com/apache/airflow/pull/33915 Fix the issue #33164 @ephraimbuddy Do you remember, why `if ti.map_index > -1:` was added? --- **^ Add meaningful description above** Read the

[GitHub] [airflow] pankajastro commented on issue #33875: Status of testing Providers that were prepared on August 29, 2023

2023-08-30 Thread via GitHub
pankajastro commented on issue #33875: URL: https://github.com/apache/airflow/issues/33875#issuecomment-1699134019 https://github.com/apache/airflow/pull/33627, https://github.com/apache/airflow/pull/33598 and https://github.com/apache/airflow/pull/33696 looks good -- This is an

[GitHub] [airflow] potiuk commented on a diff in pull request #33637: Use a trimmed version of README.md for PyPI

2023-08-30 Thread via GitHub
potiuk commented on code in PR #33637: URL: https://github.com/apache/airflow/pull/33637#discussion_r1309882203 ## setup.cfg: ## @@ -22,7 +22,7 @@ author = Apache Software Foundation author_email = d...@airflow.apache.org url = https://airflow.apache.org/ version = attr:

[GitHub] [airflow] khmelevskiy opened a new issue, #33912: Bug count dags

2023-08-30 Thread via GitHub
khmelevskiy opened a new issue, #33912: URL: https://github.com/apache/airflow/issues/33912 ### Apache Airflow version 2.7.0 ### What happened If someone rename dag, than count became +1. At the moment I have two times less dags, than I see in web ui. Past

[GitHub] [airflow-site] RamanDamayeu commented on pull request #857: Adding information about Open Data Discovery (ODD) integration

2023-08-30 Thread via GitHub
RamanDamayeu commented on PR #857: URL: https://github.com/apache/airflow-site/pull/857#issuecomment-1698886905 Thanks, very valid point! Of course, I left a link to the implementation of ODD integration with Airflow in a comment, but I agree that it will not be very easy for users

[GitHub] [airflow] wolfdn opened a new pull request, #33914: Cleanup Docker operator logging

2023-08-30 Thread via GitHub
wolfdn opened a new pull request, #33914: URL: https://github.com/apache/airflow/pull/33914 We are using the DockerOperator and noticed that the logs are a bit confusing when there is an error in a Docker container (return code != 0) because Airflow shows the log output of the Docker

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #33914: Cleanup Docker operator logging

2023-08-30 Thread via GitHub
boring-cyborg[bot] commented on PR #33914: URL: https://github.com/apache/airflow/pull/33914#issuecomment-1698955427 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] ashb commented on pull request #33746: Refactor: remove unused state - SHUTDOWN

2023-08-30 Thread via GitHub
ashb commented on PR #33746: URL: https://github.com/apache/airflow/pull/33746#issuecomment-1698956131 We need to add `TaskInstanceState.SHUTDOWN` back even if nothing uses it -- this counts as a breaking change since it has been documented and exists in the code. By removing this

[GitHub] [airflow] uranusjr commented on a diff in pull request #33570: Fix waiting setup tasks when they are not a direct upstream

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33570: URL: https://github.com/apache/airflow/pull/33570#discussion_r1309953671 ## airflow/ti_deps/deps/trigger_rule_dep.py: ## @@ -234,59 +296,72 @@ def _iter_upstream_conditions() -> Iterator[ColumnOperators]:

[GitHub] [airflow] uranusjr commented on a diff in pull request #33864: Replace single quotes by double quotes in tests

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33864: URL: https://github.com/apache/airflow/pull/33864#discussion_r1309970337 ## tests/providers/google/cloud/hooks/test_mlengine.py: ## @@ -1009,11 +1009,11 @@ def test_delete_version(self, mock_get_conn, mock_project_id):

[GitHub] [airflow] uranusjr commented on a diff in pull request #33720: Optimise and migrate to SA2-compatible syntax for TaskReschedule

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33720: URL: https://github.com/apache/airflow/pull/33720#discussion_r1310023089 ## airflow/models/taskreschedule.py: ## @@ -142,9 +154,93 @@ def find_for_task_instance( :param try_number: Look for TaskReschedule of the given

[GitHub] [airflow] eladkal merged pull request #33712: Increase `waiter_max_attempts` default value in `EcsRunTaskOperator`

2023-08-30 Thread via GitHub
eladkal merged PR #33712: URL: https://github.com/apache/airflow/pull/33712 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eladkal closed issue #33711: EcsRunTaskOperator waiter default waiter_max_attempts too low - all Airflow tasks detach from ECS tasks at 10 minutes

2023-08-30 Thread via GitHub
eladkal closed issue #33711: EcsRunTaskOperator waiter default waiter_max_attempts too low - all Airflow tasks detach from ECS tasks at 10 minutes URL: https://github.com/apache/airflow/issues/33711 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #33712: Increase `waiter_max_attempts` default value in `EcsRunTaskOperator`

2023-08-30 Thread via GitHub
boring-cyborg[bot] commented on PR #33712: URL: https://github.com/apache/airflow/pull/33712#issuecomment-1698929785 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional

[GitHub] [airflow] eladkal opened a new issue, #33916: Migrate Celery/Dask/Kubernetes Executor docs to providers

2023-08-30 Thread via GitHub
eladkal opened a new issue, #33916: URL: https://github.com/apache/airflow/issues/33916 ### Body Now that we have Executors in providers we should migrate https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/celery.html

[GitHub] [airflow] mzecov opened a new issue, #33918: prev_start_date_success and manual trigger

2023-08-30 Thread via GitHub
mzecov opened a new issue, #33918: URL: https://github.com/apache/airflow/issues/33918 ### Apache Airflow version 2.7.0 ### What happened Variable `{{ prev_start_date_success }}` returns unexpected results when manually triggering DAG. With following DAG

[GitHub] [airflow] boring-cyborg[bot] commented on issue #33918: prev_start_date_success and manual trigger

2023-08-30 Thread via GitHub
boring-cyborg[bot] commented on issue #33918: URL: https://github.com/apache/airflow/issues/33918#issuecomment-1699117236 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[GitHub] [airflow] uranusjr commented on pull request #33900: Update synapse.py to fix timeout bug

2023-08-30 Thread via GitHub
uranusjr commented on PR #33900: URL: https://github.com/apache/airflow/pull/33900#issuecomment-1698728456 Can a test be added for this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] pankajkoti commented on a diff in pull request #33637: Use a trimmed version of README.md for PyPI

2023-08-30 Thread via GitHub
pankajkoti commented on code in PR #33637: URL: https://github.com/apache/airflow/pull/33637#discussion_r1309958647 ## setup.cfg: ## @@ -22,7 +22,7 @@ author = Apache Software Foundation author_email = d...@airflow.apache.org url = https://airflow.apache.org/ version = attr:

[GitHub] [airflow] uranusjr commented on a diff in pull request #33720: Optimise and migrate to SA2-compatible syntax for TaskReschedule

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33720: URL: https://github.com/apache/airflow/pull/33720#discussion_r1309968634 ## airflow/models/taskreschedule.py: ## @@ -142,9 +154,93 @@ def find_for_task_instance( :param try_number: Look for TaskReschedule of the given

[GitHub] [airflow] Taragolis commented on a diff in pull request #33720: Optimise and migrate to SA2-compatible syntax for TaskReschedule

2023-08-30 Thread via GitHub
Taragolis commented on code in PR #33720: URL: https://github.com/apache/airflow/pull/33720#discussion_r1310007883 ## airflow/models/taskreschedule.py: ## @@ -142,9 +154,93 @@ def find_for_task_instance( :param try_number: Look for TaskReschedule of the given

[GitHub] [airflow] alephmelo commented on issue #31871: pause/unpause dag logs wrong event status

2023-08-30 Thread via GitHub
alephmelo commented on issue #31871: URL: https://github.com/apache/airflow/issues/31871#issuecomment-1698983506 @eladkal Would you be able to review the PR I opened a few days ago?  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk opened a new pull request, #33920: Clarify that DAG authors can also run code in DAG File Processor

2023-08-30 Thread via GitHub
potiuk opened a new pull request, #33920: URL: https://github.com/apache/airflow/pull/33920 Small addition to our security model - it was not entirely clear that DAG authors can also execute code in DAG File Processor and that DAG File Processor can be run in standalone mode effectively

[GitHub] [airflow] pankajastro merged pull request #33822: feat(providers/microsoft): add DefaultAzureCredential support to AzureContainerVolumeHook

2023-08-30 Thread via GitHub
pankajastro merged PR #33822: URL: https://github.com/apache/airflow/pull/33822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] pankajastro merged pull request #33825: Add DefaultAzureCredential support to AzureContainerRegistryHook

2023-08-30 Thread via GitHub
pankajastro merged PR #33825: URL: https://github.com/apache/airflow/pull/33825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy commented on pull request #33746: Refactor: remove unused state - SHUTDOWN

2023-08-30 Thread via GitHub
ephraimbuddy commented on PR #33746: URL: https://github.com/apache/airflow/pull/33746#issuecomment-1698897191 Looks like we should add a significant news fragment for this @Bisk1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] uranusjr commented on pull request #33915: Take into account removed upstream tasks for non-mapped tasks.

2023-08-30 Thread via GitHub
uranusjr commented on PR #33915: URL: https://github.com/apache/airflow/pull/33915#issuecomment-1699121844 The `map_index > -1` condition ensures this only handles mapped task, since e91637f8894cac19c6b467b6669cbcc13184be70 intends to only fix a deadlock that affects mapped tasks. So the

[GitHub] [airflow] pankajkoti commented on pull request #33637: Use a trimmed version of README.md for PyPI

2023-08-30 Thread via GitHub
pankajkoti commented on PR #33637: URL: https://github.com/apache/airflow/pull/33637#issuecomment-1699142821 Static check is failing with the below error: ``` Check if licenses are OK for ApacheFailed - hook id:

[GitHub] [airflow] Lee-W commented on pull request #33822: feat(providers/microsoft): add DefaultAzureCredential support to AzureContainerVolumeHook

2023-08-30 Thread via GitHub
Lee-W commented on PR #33822: URL: https://github.com/apache/airflow/pull/33822#issuecomment-1698797957 Thanks! I couldn't reproduce it locally yesterday and thus feel a bit confused. I'll take a deeper look next time. -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] mobuchowski commented on issue #33909: Split out providers into "standalone" python packages

2023-08-30 Thread via GitHub
mobuchowski commented on issue #33909: URL: https://github.com/apache/airflow/issues/33909#issuecomment-1698815790 I like providers being standalone packages by default. In addition to the benefits described here, it would allow installing providers straight from github via pip. I

[GitHub] [airflow] Taragolis commented on a diff in pull request #33720: Optimise and migrate to SA2-compatible syntax for TaskReschedule

2023-08-30 Thread via GitHub
Taragolis commented on code in PR #33720: URL: https://github.com/apache/airflow/pull/33720#discussion_r1310041422 ## airflow/models/taskreschedule.py: ## @@ -142,9 +154,93 @@ def find_for_task_instance( :param try_number: Look for TaskReschedule of the given

[GitHub] [airflow] eladkal commented on issue #33911: Add get_polars_df() method to BigQueryHook.

2023-08-30 Thread via GitHub
eladkal commented on issue #33911: URL: https://github.com/apache/airflow/issues/33911#issuecomment-1698925741 It needs to be added to `DbApiHook` Adding function similar to:

[GitHub] [airflow] uranusjr commented on a diff in pull request #33637: Use a trimmed version of README.md for PyPI

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33637: URL: https://github.com/apache/airflow/pull/33637#discussion_r1309878809 ## setup.cfg: ## @@ -22,7 +22,7 @@ author = Apache Software Foundation author_email = d...@airflow.apache.org url = https://airflow.apache.org/ version = attr:

[GitHub] [airflow] uranusjr commented on a diff in pull request #33898: Refactor regex in providers

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33898: URL: https://github.com/apache/airflow/pull/33898#discussion_r1310006395 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -461,31 +461,31 @@ def _process_spark_submit_log(self, itr: Iterator[Any]) -> None: #

[GitHub] [airflow] uranusjr commented on a diff in pull request #33898: Refactor regex in providers

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33898: URL: https://github.com/apache/airflow/pull/33898#discussion_r1310006395 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -461,31 +461,31 @@ def _process_spark_submit_log(self, itr: Iterator[Any]) -> None: #

[GitHub] [airflow] uranusjr commented on a diff in pull request #33858: replace unnecessary dict comprehension by dict() in core

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33858: URL: https://github.com/apache/airflow/pull/33858#discussion_r1309971825 ## airflow/www/views.py: ## @@ -3796,11 +3796,11 @@ def historical_metrics_data(self): data = { "dag_run_types": {

[GitHub] [airflow] uranusjr commented on a diff in pull request #33898: Refactor regex in providers

2023-08-30 Thread via GitHub
uranusjr commented on code in PR #33898: URL: https://github.com/apache/airflow/pull/33898#discussion_r1310004191 ## airflow/providers/amazon/aws/utils/__init__.py: ## @@ -66,8 +66,8 @@ def datetime_to_epoch_us(date_time: datetime) -> int: def get_airflow_version() ->

[GitHub] [airflow] potiuk commented on pull request #33746: Refactor: remove unused state - SHUTDOWN

2023-08-30 Thread via GitHub
potiuk commented on PR #33746: URL: https://github.com/apache/airflow/pull/33746#issuecomment-1698910499 > Looks like we should add a significant news fragment for this @Bisk1 @ephraimbuddy -> I am not so sure. Look at the discussion . It's really a cleanup. SHUTDOWN had not been

[GitHub] [airflow] fdemiane commented on issue #33875: Status of testing Providers that were prepared on August 29, 2023

2023-08-30 Thread via GitHub
fdemiane commented on issue #33875: URL: https://github.com/apache/airflow/issues/33875#issuecomment-1699068069 Verified #33500 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] Taragolis closed issue #33894: AWS connection doesn't convert verify option to bool

2023-08-30 Thread via GitHub
Taragolis closed issue #33894: AWS connection doesn't convert verify option to bool URL: https://github.com/apache/airflow/issues/33894 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] barrysteyn commented on issue #34093: The allowed_deserialization_classes regex is broken

2023-09-05 Thread via GitHub
barrysteyn commented on issue #34093: URL: https://github.com/apache/airflow/issues/34093#issuecomment-1706906717 @potiuk Thanks for the reply. I have just one final question (mainly because I am new to the AirFlow community): How does it work from here? Would you like me to submit a PR

[GitHub] [airflow] jedcunningham merged pull request #34106: Use `cached_property` for hook in SparkKubernetesSensor

2023-09-05 Thread via GitHub
jedcunningham merged PR #34106: URL: https://github.com/apache/airflow/pull/34106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk merged pull request #34101: docs(providers/microsoft): improve documentation for AzureFileShareHook DefaultAzureCredential support

2023-09-05 Thread via GitHub
potiuk merged PR #34101: URL: https://github.com/apache/airflow/pull/34101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] vincbeck commented on a diff in pull request #34051: AIP-56: Change FAB username identifier

2023-09-05 Thread via GitHub
vincbeck commented on code in PR #34051: URL: https://github.com/apache/airflow/pull/34051#discussion_r1316043015 ## airflow/models/log.py: ## @@ -37,6 +37,7 @@ class Log(Base): event = Column(String(30)) execution_date = Column(UtcDateTime) owner =

[GitHub] [airflow] adam133 commented on a diff in pull request #32903: allow DatabricksSubmitRunOperator to accept a pipeline name for a pipeline_task

2023-09-05 Thread via GitHub
adam133 commented on code in PR #32903: URL: https://github.com/apache/airflow/pull/32903#discussion_r1315921657 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -215,6 +216,67 @@ def find_job_id_by_name(self, job_name: str) -> int | None: else:

[GitHub] [airflow] potiuk merged pull request #33309: Limit connection resources used in mapped tasks

2023-09-05 Thread via GitHub
potiuk merged PR #33309: URL: https://github.com/apache/airflow/pull/33309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk merged pull request #34102: docs(providers/microsoft): improve documentation for AzureCosmosDBHook DefaultAzureCredential support

2023-09-05 Thread via GitHub
potiuk merged PR #34102: URL: https://github.com/apache/airflow/pull/34102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk merged pull request #34097: docs(providers/microsoft): improve documentation for AzureBaseHook DefaultAzureCredential support

2023-09-05 Thread via GitHub
potiuk merged PR #34097: URL: https://github.com/apache/airflow/pull/34097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy merged pull request #34027: Fix scheduler logic to plan new dag runs by ignoring manual runs

2023-09-05 Thread via GitHub
ephraimbuddy merged PR #34027: URL: https://github.com/apache/airflow/pull/34027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy closed issue #33949: Manual DAG triggers with Logical Date in the Past trigger a second run when schedule is timedelta

2023-09-05 Thread via GitHub
ephraimbuddy closed issue #33949: Manual DAG triggers with Logical Date in the Past trigger a second run when schedule is timedelta URL: https://github.com/apache/airflow/issues/33949 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] josh-fell commented on a diff in pull request #34106: Use `cached_property` for hook in SparkKubernetesSensor

2023-09-05 Thread via GitHub
josh-fell commented on code in PR #34106: URL: https://github.com/apache/airflow/pull/34106#discussion_r1315995816 ## tests/providers/cncf/kubernetes/sensors/test_spark_kubernetes.py: ## @@ -552,13 +552,6 @@ class TestSparkKubernetesSensor: def setup_method(self):

[GitHub] [airflow] potiuk merged pull request #34104: docs(providers/microsoft): improve documentation for AzureContainerVolumeHook DefaultAzureCredential support

2023-09-05 Thread via GitHub
potiuk merged PR #34104: URL: https://github.com/apache/airflow/pull/34104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] barrysteyn commented on issue #34093: The allowed_deserialization_classes regex is broken

2023-09-05 Thread via GitHub
barrysteyn commented on issue #34093: URL: https://github.com/apache/airflow/issues/34093#issuecomment-1706773954 Thanks @potiuk. The ultimate aim of `allowed_deserialization_classes` is provide protection against unexpected class deserialization. Using a regex is goes against that - as

[GitHub] [airflow] potiuk merged pull request #34103: docs(providers/microsoft): improve documentation for WasbHook DefaultAzureCredential support

2023-09-05 Thread via GitHub
potiuk merged PR #34103: URL: https://github.com/apache/airflow/pull/34103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #33959: openlineage, dbt: add OpenLineage support for DBT Cloud

2023-09-05 Thread via GitHub
potiuk commented on PR #33959: URL: https://github.com/apache/airflow/pull/33959#issuecomment-1706754244 rebased. seems GA are back -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] vincbeck commented on a diff in pull request #33901: Fix inheritance chain in security manager

2023-09-05 Thread via GitHub
vincbeck commented on code in PR #33901: URL: https://github.com/apache/airflow/pull/33901#discussion_r1316009816 ## airflow/auth/managers/base_auth_manager.py: ## @@ -24,9 +24,11 @@ from airflow.utils.log.logging_mixin import LoggingMixin if TYPE_CHECKING: +from flask

[GitHub] [airflow] potiuk merged pull request #34096: docs(providers/microsoft): improve documentation for Azure Service Bus hooks DefaultAzureCredential support

2023-09-05 Thread via GitHub
potiuk merged PR #34096: URL: https://github.com/apache/airflow/pull/34096 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #34093: The allowed_deserialization_classes regex is broken

2023-09-05 Thread via GitHub
potiuk commented on issue #34093: URL: https://github.com/apache/airflow/issues/34093#issuecomment-1706899136 I am also fine for that. Regexp fallback was only there as a compatibility measure and I am perfectly fine to do "glob-only" - that would by my way of implemeting it if I were to

[GitHub] [airflow] potiuk merged pull request #34095: docs(providers/microsoft): improve documentation for AzureDataExplorerHook DefaultAzureCredential support

2023-09-05 Thread via GitHub
potiuk merged PR #34095: URL: https://github.com/apache/airflow/pull/34095 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] argibbs commented on issue #33410: SlaMiss Records Never Created for Packaged DAGs

2023-09-05 Thread via GitHub
argibbs commented on issue #33410: URL: https://github.com/apache/airflow/issues/33410#issuecomment-1706903000 Hello, sorry, not deliberately ignoring people, just not checking my mail as often as I should. (Insert real-life-getting-in-the-way comment here). Haven't read the

[GitHub] [airflow] Taragolis commented on issue #34105: Downstream node is executing even though upstream node is in "Yet to run" status

2023-09-05 Thread via GitHub
Taragolis commented on issue #34105: URL: https://github.com/apache/airflow/issues/34105#issuecomment-1706721682 Airflow 2.2.2 released almost two years ago, so that it quite a big chance that it already resolved in the current Airflow:

[GitHub] [airflow] potiuk merged pull request #34091: Add DOAP file containing machine-readable information for Airflow

2023-09-05 Thread via GitHub
potiuk merged PR #34091: URL: https://github.com/apache/airflow/pull/34091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] Lee-W commented on a diff in pull request #34104: docs(providers/microsoft): improve documentation for AzureContainerVolumeHook DefaultAzureCredential support

2023-09-05 Thread via GitHub
Lee-W commented on code in PR #34104: URL: https://github.com/apache/airflow/pull/34104#discussion_r1315763591 ## docs/apache-airflow-providers-microsoft-azure/connections/azure_container_volume.rst: ## @@ -61,9 +63,9 @@ Extra (optional) Specify the extra parameters (as

[GitHub] [airflow] mhenc commented on a diff in pull request #34026: AIP-44 Migrate Job to Internal API

2023-09-05 Thread via GitHub
mhenc commented on code in PR #34026: URL: https://github.com/apache/airflow/pull/34026#discussion_r1316039440 ## airflow/jobs/base_job_runner.py: ## @@ -27,7 +27,7 @@ from airflow.jobs.job import Job from airflow.serialization.pydantic.job import JobPydantic -J =

[GitHub] [airflow] klexadoc commented on issue #34065: Status of testing of Apache Airflow 2.7.1rc2

2023-09-05 Thread via GitHub
klexadoc commented on issue #34065: URL: https://github.com/apache/airflow/issues/34065#issuecomment-1706464703 Confirmed that https://github.com/apache/airflow/issues/33478 is fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk merged pull request #34094: docs(providers/microsoft): improve documentation for AzureDataLakeStorageV2Hook DefaultAzureCredential support

2023-09-05 Thread via GitHub
potiuk merged PR #34094: URL: https://github.com/apache/airflow/pull/34094 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk merged pull request #33898: Refactor regex in providers

2023-09-05 Thread via GitHub
potiuk merged PR #33898: URL: https://github.com/apache/airflow/pull/33898 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] Lee-W opened a new pull request, #34104: docs(providers/microsoft): improve documentation for AzureContainerVolumeHook DefaultAzureCredential support

2023-09-05 Thread via GitHub
Lee-W opened a new pull request, #34104: URL: https://github.com/apache/airflow/pull/34104 Update missing content in https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/azure_container_volume.html --- **^

[GitHub] [airflow] ephraimbuddy commented on issue #34023: Trigger Rule ONE_FAILED does not work in task group with mapped tasks

2023-09-05 Thread via GitHub
ephraimbuddy commented on issue #34023: URL: https://github.com/apache/airflow/issues/34023#issuecomment-1706432286 @hussein-awala thanks for debugging, I was busy with releases the past 2 days, getting to looking at it now -- This is an automated message from the Apache Git Service.

[GitHub] [airflow] vidyapanchalZS opened a new issue, #34105: Downstream node is executing even though upstream node is in "Yet to run" status

2023-09-05 Thread via GitHub
vidyapanchalZS opened a new issue, #34105: URL: https://github.com/apache/airflow/issues/34105 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Hello team, We are facing one issue on Airflow version 2.2.2 where

[GitHub] [airflow] josh-fell opened a new pull request, #34106: Use `cached_property` for hook in SparkKubernetesSensor

2023-09-05 Thread via GitHub
josh-fell opened a new pull request, #34106: URL: https://github.com/apache/airflow/pull/34106 Best practice to is make constructors of operators/sensors as simple as possible. Currently this sensor directly builds a KubernetesHook object in its construction. Yes, there are db or external

[GitHub] [airflow] potiuk closed issue #33178: Flaky `test_xcom_map_error_fails_task` test

2023-09-05 Thread via GitHub
potiuk closed issue #33178: Flaky `test_xcom_map_error_fails_task` test URL: https://github.com/apache/airflow/issues/33178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk commented on pull request #33309: Limit connection resources used in mapped tasks

2023-09-05 Thread via GitHub
potiuk commented on PR #33309: URL: https://github.com/apache/airflow/pull/33309#issuecomment-1706741295 Thanks @uranusjr ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] potiuk commented on pull request #34031: Limit celery by excluding 5.3.2 and 5.3.3

2023-09-02 Thread via GitHub
potiuk commented on PR #34031: URL: https://github.com/apache/airflow/pull/34031#issuecomment-1703851208 I think you really can't event have another 5.3.2 release. PyPi will not let you upload another 5.3.2 even if you delete the previous one -- This is an automated message from the

[GitHub] [airflow] Bisk1 opened a new pull request, #34037: Fix issue with unit tests in subprocesses on Mac

2023-09-02 Thread via GitHub
Bisk1 opened a new pull request, #34037: URL: https://github.com/apache/airflow/pull/34037 When unit tests are run on local virtual env on macOS, a lot of them fail because mocks do not work as expected when they are used in subprocesses. See

[GitHub] [airflow] Nusnus commented on pull request #34031: Limit celery by excluding 5.3.2 and 5.3.3

2023-09-02 Thread via GitHub
Nusnus commented on PR #34031: URL: https://github.com/apache/airflow/pull/34031#issuecomment-1703860020 @hussein-awala @potiuk With every failure, we learn new lessons. As the most recent Owner at Celery, I am still discovering what's under the hood and I've been

[GitHub] [airflow] amoghrajesh commented on a diff in pull request #34004: Introducing short_version flag for building docs

2023-09-02 Thread via GitHub
amoghrajesh commented on code in PR #34004: URL: https://github.com/apache/airflow/pull/34004#discussion_r1313873764 ## dev/breeze/src/airflow_breeze/params/doc_build_params.py: ## @@ -42,6 +55,9 @@ def args_doc_builder(self) -> list[str]:

[GitHub] [airflow] hussein-awala commented on a diff in pull request #33841: Refactor unneeded 'continue' jumps in tests

2023-09-02 Thread via GitHub
hussein-awala commented on code in PR #33841: URL: https://github.com/apache/airflow/pull/33841#discussion_r1314021479 ## tests/serialization/test_serialized_objects.py: ## @@ -39,10 +39,9 @@ def test_recursive_serialize_calls_must_forward_kwargs(): class_def = None

[GitHub] [airflow] pankajkoti commented on issue #34022: Unable to read or write logs on Azure Blob after 2.7.0 update

2023-09-02 Thread via GitHub
pankajkoti commented on issue #34022: URL: https://github.com/apache/airflow/issues/34022#issuecomment-1704004495 you can keep the constraints as is and just install the newer azure provider at the end of your image build process -- This is an automated message from the Apache Git

[GitHub] [airflow] Taragolis opened a new pull request, #34046: Make AWS SecretsManager tests compatible with `moto>=4.2.1`

2023-09-03 Thread via GitHub
Taragolis opened a new pull request, #34046: URL: https://github.com/apache/airflow/pull/34046 I'm not sure what is happen yet but upgrade to latests moto==4.2.2 was break our tests in main, this change make our tests work either with latest `moto` and previous versions

[GitHub] [airflow] NJ-Tokki opened a new issue, #34048: Use Kubernetes-recommended labels

2023-09-03 Thread via GitHub
NJ-Tokki opened a new issue, #34048: URL: https://github.com/apache/airflow/issues/34048 ### Description Feature Kubernetes offers a [list of recommended labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels) for grouping resource

[GitHub] [airflow] Bisk1 opened a new pull request, #34049: Rename variables for dag runs

2023-09-03 Thread via GitHub
Bisk1 opened a new pull request, #34049: URL: https://github.com/apache/airflow/pull/34049 Follow-up to discussion in https://github.com/apache/airflow/pull/34027#pullrequestreview-1608444868 Some variables used when invoking calculate_dagrun_date_fields were renamed to make it

[GitHub] [airflow] Bisk1 commented on pull request #34027: Fix scheduler logic to plan new dag runs by ignoring manual runs

2023-09-03 Thread via GitHub
Bisk1 commented on PR #34027: URL: https://github.com/apache/airflow/pull/34027#issuecomment-1704264908 > Looks good but the renaming should be in a separate PR if it's actually needed @ephraimbuddy Ok I removed it and moved to separate PR

[GitHub] [airflow] hussein-awala commented on pull request #33903: Ensure that tasks wait for running indirect setup

2023-09-03 Thread via GitHub
hussein-awala commented on PR #33903: URL: https://github.com/apache/airflow/pull/33903#issuecomment-1704326323 I understand that `+602 −236 ` is scary for a patch version, but most of the changed code is documentation and tests, and for the changes in `trigger_rule_dep`, @dstandish has

[GitHub] [airflow] ephraimbuddy merged pull request #33903: Ensure that tasks wait for running indirect setup

2023-09-03 Thread via GitHub
ephraimbuddy merged PR #33903: URL: https://github.com/apache/airflow/pull/33903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eumiro opened a new pull request, #34052: Refactor: Simplify loop in aws/triggers/batch.py

2023-09-03 Thread via GitHub
eumiro opened a new pull request, #34052: URL: https://github.com/apache/airflow/pull/34052 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [airflow] potiuk merged pull request #33839: Refactor unneeded 'continue' jumps in dag processing

2023-09-03 Thread via GitHub
potiuk merged PR #33839: URL: https://github.com/apache/airflow/pull/33839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

  1   2   3   4   5   6   7   8   9   10   >