Re: [PR] Implement the breeze tag_providers command [airflow]

2024-03-24 Thread via GitHub
amoghrajesh commented on code in PR #38447: URL: https://github.com/apache/airflow/pull/38447#discussion_r1537083067 ## dev/breeze/doc/09_release_management_tasks.rst: ## @@ -189,6 +189,26 @@ These are all of the available flags for the ``release-prod-images`` command:

Re: [PR] changing dag_processing.processes from UpDownCounter to guage [airflow]

2024-03-24 Thread via GitHub
Bowrna commented on PR #38400: URL: https://github.com/apache/airflow/pull/38400#issuecomment-2017270573 when setting the `dag_processing.processes` as gauge instead of the counter, we may need to initialize the value. Gauge works in the following way: gaugor:333|g the above

Re: [PR] Add legend for duration markline. [airflow]

2024-03-24 Thread via GitHub
dirrao commented on code in PR #38434: URL: https://github.com/apache/airflow/pull/38434#discussion_r1537079098 ## airflow/www/static/js/dag/details/dag/RunDurationChart.tsx: ## @@ -164,7 +164,43 @@ const RunDurationChart = ({ showLandingTimes }: Props) => { `; } +

[PR] Implement the breeze tag_providers command [airflow]

2024-03-24 Thread via GitHub
poorvirohidekar opened a new pull request, #38447: URL: https://github.com/apache/airflow/pull/38447 This PR adds support to move the tag_providers.py script to breeze under release-management and addresses the review comments from the below mentioned PR. Reference:

Re: [PR] Add legend for duration markline. [airflow]

2024-03-24 Thread via GitHub
tirkarthi commented on PR #38434: URL: https://github.com/apache/airflow/pull/38434#issuecomment-2017101047 @simond Thanks, I will update the legend. Please let me know if median total duration is more useful than median run duration. -- This is an automated message from the Apache Git

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536985669 ## .github/workflows/ci-image-build.yml: ## @@ -195,19 +194,19 @@ ${{ inputs.do-build == 'true' && inputs.image-tag || '' }}" run: > breeze

(airflow) branch fix-image-cache updated (579cd39821 -> e4648ad9a1)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git omit 579cd39821 Fix image cache optimizations - speeding up the build add 3832ead221 Fix failing

Re: [PR] Allow users to write dag_id and task_id in their national characters, added display name for dag / task (v2) [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38446: URL: https://github.com/apache/airflow/pull/38446#issuecomment-2017069297 The failing MINSQL ALchemy test here is fixed in #38445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] allows users to write dag_id and task_id in their national characters, added display name for dag / task [airflow]

2024-03-24 Thread via GitHub
jscheffl commented on PR #32520: URL: https://github.com/apache/airflow/pull/32520#issuecomment-2017051703 As linked in #38446 - I tried to spend time on the weekend to continue working on this PR... let's see if we can get this reviewed and completed... -- This is an automated message

Re: [PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38444: URL: https://github.com/apache/airflow/pull/38444#issuecomment-2017045033 Closed in favour of canary test in #38445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] Fix 22073 [airflow]

2024-03-24 Thread via GitHub
jscheffl opened a new pull request, #38446: URL: https://github.com/apache/airflow/pull/38446 This PR tries to continue the work from @xgao1023 in PR #35320. It merges with current main. On top of this I fixed the bugs in pytest, some glitches in UI. I'd LOVE to get this into 2.9.0

[PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38445: URL: https://github.com/apache/airflow/pull/38445 Ths MIN SQLACLHEMY test was based on retrieving sqlalchemy min version from pyproject.toml but since we moved it to hatch_build.py we should read it from there.

(airflow) branch fix-failing-sqlalchemy-test created (now 3832ead221)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-failing-sqlalchemy-test in repository https://gitbox.apache.org/repos/asf/airflow.git at 3832ead221 Fix failing MinSQLAlchemy test after moving sqlalchemy spec This branch includes

Re: [PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk closed pull request #38444: Fix failing MinSQLAlchemy test after moving sqlalchemy spec URL: https://github.com/apache/airflow/pull/38444 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(airflow) 01/01: Fix failing MinSQLAlchemy test after moving sqlalchemy spec

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch fix-failing-sqlalchemy-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 3832ead22158ed963417367082868f5563f8f734 Author: Jarek Potiuk AuthorDate: Mon Mar 25 02:11:39

Re: [PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38444: URL: https://github.com/apache/airflow/pull/38444#discussion_r1536962905 ## .github/workflows/run-unit-tests.yml: ## @@ -126,7 +126,7 @@ jobs: BACKEND_VERSION: "${{ matrix.backend-version }}" DEBUG_RESOURCES: "${{

[PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38444: URL: https://github.com/apache/airflow/pull/38444 Ths MIN SQLACLHEMY test was based on retrieving sqlalchemy min version from pyproject.toml but since we moved it to hatch_build.py we should read it from there.

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536951037 ## .github/workflows/finalize-tests.yml: ## @@ -120,31 +120,48 @@ jobs: run: git push - # Push BuildX cache to GitHub Registry in Apache

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536950612 ## .github/workflows/additional-ci-image-checks.yml: ## @@ -89,42 +89,40 @@ jobs: # delay cache refresh. It does not attempt to upgrade to newer dependencies. #

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536950198 ## .github/workflows/additional-ci-image-checks.yml: ## @@ -89,42 +89,40 @@ jobs: # delay cache refresh. It does not attempt to upgrade to newer dependencies. #

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536950005 ## .github/workflows/additional-ci-image-checks.yml: ## @@ -89,42 +89,40 @@ jobs: # delay cache refresh. It does not attempt to upgrade to newer dependencies. #

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536949904 ## dev/breeze/src/airflow_breeze/utils/docker_command_utils.py: ## @@ -381,12 +381,15 @@ def prepare_base_build_command(image_params: CommonBuildParams) ->

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536949488 ## dev/breeze/src/airflow_breeze/utils/docker_command_utils.py: ## @@ -658,6 +661,7 @@ def autodetect_docker_context(): def get_and_use_docker_context(context: str):

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536949451 ## dev/breeze/src/airflow_breeze/utils/image.py: ## @@ -197,18 +197,8 @@ def tag_image_as_latest(image_params: CommonBuildParams, output: Output | None)

(airflow) branch fix-image-cache updated (64494f3c09 -> 579cd39821)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 64494f3c09 Fix image cache optimizations - speeding up the build add 579cd39821 Fix image cache

Re: [PR] Helm: Add namespace to all namespace-scoped resources [airflow]

2024-03-24 Thread via GitHub
github-actions[bot] commented on PR #33177: URL: https://github.com/apache/airflow/pull/33177#issuecomment-2017004416 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

(airflow) branch fix-image-cache updated (81467fa5e8 -> 64494f3c09)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git omit 81467fa5e8 Fix image cache optimizations - speeding up the build add 64494f3c09 Fix image cache

(airflow) branch fix-image-cache updated (35763472bf -> 81467fa5e8)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 35763472bf Fix image cache optimizations - speeding up the build add 81467fa5e8 Fix image cache

(airflow) branch fix-image-cache updated (3eb5212b71 -> 35763472bf)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 3eb5212b71 Fix image cache optimizations - speeding up the build add 35763472bf Fix image cache

(airflow) branch fix-image-cache updated (44a256052d -> 3eb5212b71)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 44a256052d Fix image cache optimizations - speeding up the build add 20cb9f1770 Upgrade to newer build

(airflow) branch main updated: Upgrade to newer build dependencies including the right Python version (#38443)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 20cb9f1770 Upgrade to newer build dependencies

Re: [PR] Upgrade to newer build dependencies including the right Python version [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38443: URL: https://github.com/apache/airflow/pull/38443 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add legend for duration markline. [airflow]

2024-03-24 Thread via GitHub
simond commented on PR #38434: URL: https://github.com/apache/airflow/pull/38434#issuecomment-2016947663 Just looking at your screenshots - you may want to change the wording to 'median total', 'median run' etc rather than 'mean' - median and mean are different things. Looks like you're

[PR] Upgrade to newer build dependencies including the right Python version [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38443: URL: https://github.com/apache/airflow/pull/38443 This change upgrades build dependencies, including the fact that for earlier Python versions, the set of dependencies might be different than than for later ones. This PR assumes that default python

Re: [PR] Test cache refreshing in CI [JUST TESTING] [airflow]

2024-03-24 Thread via GitHub
potiuk closed pull request #38427: Test cache refreshing in CI [JUST TESTING] URL: https://github.com/apache/airflow/pull/38427 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

(airflow) branch fix-image-cache updated (fe69e7423a -> 44a256052d)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard fe69e7423a Fix image cache optimizations - speeding up the build add 25e5d54192 Print selective-check

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38437: URL: https://github.com/apache/airflow/pull/38437 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch fix-image-cache updated (89a3c70b96 -> fe69e7423a)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 89a3c70b96 Fix image cache optimizations - speeding up the build add fe69e7423a Fix image cache

Re: [I] Wrong DAG duration time [airflow]

2024-03-24 Thread via GitHub
arley-wilches commented on issue #30877: URL: https://github.com/apache/airflow/issues/30877#issuecomment-2016938365 Hi there. I found a similar issue, the duration time does not take into account mapped tasks and I have two tasks one of them took about 3 hours and the other one

[PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38442: URL: https://github.com/apache/airflow/pull/38442 The recent refactors in workflows broke the way how cache had been used in the CI builds. This PR brings back the optimizations by using the cache and rebuilding it.

(airflow) branch fix-image-cache created (now 89a3c70b96)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git at 89a3c70b96 Fix image cache optimizations - speeding up the build This branch includes the following new

(airflow) 01/01: Fix image cache optimizations - speeding up the build

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git commit 89a3c70b9654c7f8812e87d49bc4dfec6979486d Author: Jarek Potiuk AuthorDate: Sun Mar 24 21:21:03 2024 +0100

(airflow) branch test-cache-refreshing updated (90d68148c3 -> 375f9426a3)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch test-cache-refreshing in repository https://gitbox.apache.org/repos/asf/airflow.git discard 90d68148c3 Test cache refreshing in CI add 375f9426a3 Test cache refreshing in CI This update

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016913907 Ah. Now with traceback, it's clear. The old breeze expects "dependencies" property in `project` ... That's fine. This will be fixed once the PR is merged. -- This is an automated

Re: [PR] Apply task instance mutation hook consistently [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38440: URL: https://github.com/apache/airflow/pull/38440#issuecomment-2016912706 NIT: Maybe a test case @jscheffl ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38437: URL: https://github.com/apache/airflow/pull/38437#discussion_r1536890083 ## INSTALL: ## @@ -128,19 +141,19 @@ You can see the list of available envs with: This is what it shows currently:

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (02a6ee9938 -> 83135236cd)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git omit 02a6ee9938 Turn optional-dependencies in pyproject.toml into dynamic

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (2b518b4e25 -> 02a6ee9938)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git omit 2b518b4e25 Turn optional-dependencies in pyproject.toml into dynamic

(airflow) branch main updated: Print selective-check traceback on stdout rather than stderr (#38441)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 25e5d54192 Print selective-check traceback on

Re: [PR] Print selective-check traceback on stdout rather than stderr [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38441: URL: https://github.com/apache/airflow/pull/38441#issuecomment-2016911399 > An alternative would be use use 2>&1 in the shell call to redirect all stderr to stdout? > In total meaning: breeze ci selective-check 2>&1 | tee ${GITHUB_OUTPUT} ? We

Re: [PR] Print selective-check traceback on stdout rather than stderr [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38441: URL: https://github.com/apache/airflow/pull/38441 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
jscheffl commented on code in PR #38437: URL: https://github.com/apache/airflow/pull/38437#discussion_r1536887077 ## INSTALL: ## @@ -128,19 +141,19 @@ You can see the list of available envs with: This is what it shows currently:

(airflow) branch test-cache-refreshing updated (106bdf172b -> 90d68148c3)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch test-cache-refreshing in repository https://gitbox.apache.org/repos/asf/airflow.git discard 106bdf172b Test cache refreshing in CI add 078e7ed135 Mysql change xcom value col type for MySQL

Re: [PR] Print selective-check traceback on stdout rather than stderr [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38441: URL: https://github.com/apache/airflow/pull/38441#issuecomment-2016903858 Should help to diagnose issues like https://github.com/apache/airflow/actions/runs/8410545233/job/23029279435 -- This is an automated message from the Apache Git Service. To respond

[PR] Print selective-check traceback on stdout rather than stderr [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38441: URL: https://github.com/apache/airflow/pull/38441 We are using stdout of selective-check to print diagnostic information and stderr is redirected to produce outputs for GIHUB_OUTPUTS special variable in GitHub Actions. However this means that when

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016894701 Updated documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016894632 > If I run into this we should handle this explicitly. Is it possible to fail install if pip version is too old? No. But it ONLY affects you if you want to install airflow from

(airflow) branch constraints-main updated: Updating constraints. Github run id:8410961843

2024-03-24 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 368b7511da Updating

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (902d144946 -> 2b518b4e25)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard 902d144946 Turn optional-dependencies in pyproject.toml into dynamic

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
jscheffl commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016892056 > Actually looks like `pip 22.0` is the LAST one that does not support this dynamic retrieval of dependencies. So you picked really edge'y case @jscheffl. I will add this to the docs.

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016891091 Actually looks like `pip 22.0` is the LAST one that does not support this dynamic retrieval of dependencies. So you picked really edge'y case @jscheffl. I will add this to the docs.

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (e1139449bd -> 902d144946)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard e1139449bd Turn optional-dependencies in pyproject.toml into dynamic

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (8343dabf24 -> e1139449bd)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard 8343dabf24 Turn optional-dependencies in pyproject.toml into dynamic

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016887762 General advice from `pip` team is to always upgrade to newest version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016887543 Yes. You must upgrade your `pip`. Version 22 is from 2022 and a lot of the functionality and PEPs needed have been approved and implemented later than that. -- This is an automated

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
jscheffl commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016885982 I tried to "naively" install as described in my devenv. 1. Build local CI Image: works 2. Install via pip in local env: FAILS (see 1) 1) Details of pip install in

[PR] Apply task instance mutation hook consistently [airflow]

2024-03-24 Thread via GitHub
jscheffl opened a new pull request, #38440: URL: https://github.com/apache/airflow/pull/38440 We are using the [Cluster Policies](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/cluster-policies.html#task-instance-mutation) and in the the feature of the

Re: [PR] Add executor field to the DB and parameter to the operators [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38054: URL: https://github.com/apache/airflow/pull/38054#issuecomment-2016878718 Yep. We cannot merge it without resolving conflict :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

(airflow) branch main updated (77d2fc7d75 -> 694826d1bd)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 77d2fc7d75 Check task attribute before use in sentry.add_tagging() (#37143) add 694826d1bd refactor: Refactored

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #37937: URL: https://github.com/apache/airflow/pull/37937 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Check task attribute before use in sentry.add_tagging() (#37143)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 77d2fc7d75 Check task attribute before use in

Re: [PR] Check task attribute before use in sentry.add_tagging() [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #37143: URL: https://github.com/apache/airflow/pull/37143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated (d820f13e68 -> 4a4eee113f)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d820f13e68 Use `relative-imports (TID252)` instead of `pygrep` rule (#38428) add 4a4eee113f Delete deprecated

Re: [PR] Delete deprecated AutoML operators (inc. 'google-cloud-automl' dep.) and deprecate hook and links [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38418: URL: https://github.com/apache/airflow/pull/38418 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Improve logging readability with DockerOperator [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38008: URL: https://github.com/apache/airflow/pull/38008#discussion_r1536864652 ## airflow/providers/docker/operators/docker.py: ## @@ -423,13 +447,19 @@ def _run_image_with_mounts(self, target_mounts, add_tmp_variable: bool) -> list[ )

Re: [PR] Adding Task ids in DAG details API endpoint #37564 [airflow]

2024-03-24 Thread via GitHub
msoni1369 commented on PR #37866: URL: https://github.com/apache/airflow/pull/37866#issuecomment-2016869645 > If we call the field `tasks`, users will expect more information, not just `task_ids`. You may change the field to task_ids. In the future, we can deprecate that and have `tasks`

Re: [PR] Support adding custom TI Deps to help DagRun make more flexible TI scheduling decisions [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #37778: URL: https://github.com/apache/airflow/pull/37778#issuecomment-2016865953 Agree with TP that if we were to merge this one, we will need quite a bit more: * documentation with some examples * general use cases where it would be available * some

(airflow) branch main updated (81790aaaa0 -> d820f13e68)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 817900 Fix deprecated `DockerOperator` operator arguments in `MappedOperator` (#38379) add d820f13e68 Use

Re: [PR] Use `relative-imports (TID252)` instead of `pygrep` rule [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38428: URL: https://github.com/apache/airflow/pull/38428 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Fix deprecated `DockerOperator` operator arguments in `MappedOperator` (#38379)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 817900 Fix deprecated `DockerOperator`

Re: [PR] Fix deprecated `DockerOperator` operator arguments in `MappedOperator` [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38379: URL: https://github.com/apache/airflow/pull/38379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated (0cc410b5b8 -> d3e9229105)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 0cc410b5b8 Updat build dependencies after hatchling update (#38439) add d3e9229105 Bugfix/update screenshots

Re: [PR] Bugfix/update screenshots [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38438: URL: https://github.com/apache/airflow/pull/38438 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38437: URL: https://github.com/apache/airflow/pull/38437#discussion_r1536854627 ## airflow_pre_installed_providers.txt: ## @@ -1,7 +1,7 @@ # List of all the providers that are pre-installed when you run `pip install apache-airflow` without

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016858378 cc: @uranusjr -> I'd appreciate your comments here, I believe now we should be **really** compliant with the PEP expectations. I compared the generated METADATA in the .whl files

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (db109c8372 -> 8343dabf24)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard db109c8372 Turn optional-dependencies in pyproject.toml into dynamic

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016854440 After some recent (and looking and old) discussions, it turns out that we need one more thing to be fully PEP-517 compliant. While hatchling allows us (for now) to dynamically modiy

Re: [PR] changing dag_processing.processes from UpDownCounter to guage [airflow]

2024-03-24 Thread via GitHub
Bowrna closed pull request #38400: changing dag_processing.processes from UpDownCounter to guage URL: https://github.com/apache/airflow/pull/38400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] heartbeat recovery message [airflow]

2024-03-24 Thread via GitHub
Bowrna commented on code in PR #34457: URL: https://github.com/apache/airflow/pull/34457#discussion_r1536847700 ## airflow/jobs/job.py: ## @@ -131,22 +146,20 @@ def executors(self): def heartrate(self) -> float: return Job._heartrate(self.job_type) -def

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (ad76bf579b -> db109c8372)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard ad76bf579b Turn optional-dependencies in pyproject.toml into dynamic

(airflow) branch main updated: Updat build dependencies after hatchling update (#38439)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0cc410b5b8 Updat build dependencies after

Re: [PR] Update build dependencies after hatchling update [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38439: URL: https://github.com/apache/airflow/pull/38439 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (e2a3b9cfe9 -> ad76bf579b)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard e2a3b9cfe9 Turn optional-dependencies in pyproject.toml into dynamic

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (4601d60e83 -> e2a3b9cfe9)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git omit 4601d60e83 Turn optional-dependencies in pyproject.toml into dynamic

[PR] Updat build dependencies after hatchling update [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38439: URL: https://github.com/apache/airflow/pull/38439 --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Bugfix/update screenshots [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38438: URL: https://github.com/apache/airflow/pull/38438#issuecomment-2016834179 NICE -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (1e079111ae -> 4601d60e83)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard 1e079111ae Turn optional-dependencies in pyproject.toml into dynamic

[PR] Bugfix/update screenshots [airflow]

2024-03-24 Thread via GitHub
jscheffl opened a new pull request, #38438: URL: https://github.com/apache/airflow/pull/38438 Following my PR #38435 I needed to realize that a lot of other images and screen captures are totally outdated to current UI status. This PR updates a couple/most of them. I was

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (0ba4f701c1 -> 1e079111ae)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard 0ba4f701c1 Turn optional-dependencies in pyproject.toml into dynamic

[PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38437: URL: https://github.com/apache/airflow/pull/38437 While currently hatchling and pip nicely supports dynamic replacement of the dependencies even if they are statically defined, this is not proper according to EP 621. When property of the project is

  1   2   >