Re: [PR] Merge kubernetes pod objects inplace [airflow]

2024-03-24 Thread via GitHub
dirrao commented on PR #38259: URL: https://github.com/apache/airflow/pull/38259#issuecomment-2017330510 > what do you mean by breaking IMHO it's just broken, not breaking - because it's impossible to foresee how one pod executing will impact another one, and since you have no influence on

Re: [PR] Prevent large objects from being stored in the RTIF [airflow]

2024-03-24 Thread via GitHub
uranusjr commented on code in PR #38094: URL: https://github.com/apache/airflow/pull/38094#discussion_r1537110231 ## airflow/serialization/helpers.py: ## @@ -38,7 +40,21 @@ def is_jsonable(x): else: return True +max_length = conf.getint("core", "max_t

Re: [PR] changing dag_processing.processes from UpDownCounter to guage [airflow]

2024-03-24 Thread via GitHub
Bowrna commented on code in PR #38400: URL: https://github.com/apache/airflow/pull/38400#discussion_r1537109916 ## docs/apache-airflow/administration-and-deployment/logging-monitoring/metrics.rst: ## @@ -219,6 +219,7 @@ Name Descri

Re: [PR] Prevent large objects from being stored in the RTIF [airflow]

2024-03-24 Thread via GitHub
ephraimbuddy commented on code in PR #38094: URL: https://github.com/apache/airflow/pull/38094#discussion_r1537106372 ## tests/models/test_renderedtifields.py: ## @@ -148,6 +170,49 @@ def test_get_templated_fields(self, templated_field, expected_rendered_field, da # Fe

Re: [PR] Prevent large objects from being stored in the RTIF [airflow]

2024-03-24 Thread via GitHub
ephraimbuddy commented on code in PR #38094: URL: https://github.com/apache/airflow/pull/38094#discussion_r1537105429 ## tests/models/test_renderedtifields.py: ## @@ -62,6 +64,18 @@ def __ne__(self, other): return not self.__eq__(other) +class LargeStrObject: +d

Re: [PR] Introducing --latest for generate-issue-content-core command [airflow]

2024-03-24 Thread via GitHub
amoghrajesh commented on PR #38250: URL: https://github.com/apache/airflow/pull/38250#issuecomment-2017305898 @potiuk I just did that, lets see the run -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Introducing --latest for generate-issue-content-core command [airflow]

2024-03-24 Thread via GitHub
amoghrajesh closed pull request #38250: Introducing --latest for generate-issue-content-core command URL: https://github.com/apache/airflow/pull/38250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Use Task Output as Extra for Dataset Trigger and DAG Run Conf [airflow]

2024-03-24 Thread via GitHub
dirrao commented on code in PR #38432: URL: https://github.com/apache/airflow/pull/38432#discussion_r1537095769 ## airflow/models/taskinstance.py: ## @@ -2553,21 +2554,30 @@ def _run_raw_task( return None -def _register_dataset_changes(self, *, session: Sess

Re: [PR] Implement the breeze tag_providers command [airflow]

2024-03-24 Thread via GitHub
amoghrajesh commented on code in PR #38447: URL: https://github.com/apache/airflow/pull/38447#discussion_r1537083067 ## dev/breeze/doc/09_release_management_tasks.rst: ## @@ -189,6 +189,26 @@ These are all of the available flags for the ``release-prod-images`` command: :widt

Re: [PR] changing dag_processing.processes from UpDownCounter to guage [airflow]

2024-03-24 Thread via GitHub
Bowrna commented on PR #38400: URL: https://github.com/apache/airflow/pull/38400#issuecomment-2017270573 when setting the `dag_processing.processes` as gauge instead of the counter, we may need to initialize the value. Gauge works in the following way: gaugor:333|g the above co

Re: [PR] Add legend for duration markline. [airflow]

2024-03-24 Thread via GitHub
dirrao commented on code in PR #38434: URL: https://github.com/apache/airflow/pull/38434#discussion_r1537079098 ## airflow/www/static/js/dag/details/dag/RunDurationChart.tsx: ## @@ -164,7 +164,43 @@ const RunDurationChart = ({ showLandingTimes }: Props) => { `; } + fu

[PR] Implement the breeze tag_providers command [airflow]

2024-03-24 Thread via GitHub
poorvirohidekar opened a new pull request, #38447: URL: https://github.com/apache/airflow/pull/38447 This PR adds support to move the tag_providers.py script to breeze under release-management and addresses the review comments from the below mentioned PR. Reference: http

Re: [PR] Add legend for duration markline. [airflow]

2024-03-24 Thread via GitHub
tirkarthi commented on PR #38434: URL: https://github.com/apache/airflow/pull/38434#issuecomment-2017101047 @simond Thanks, I will update the legend. Please let me know if median total duration is more useful than median run duration. -- This is an automated message from the Apache Git Se

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536985669 ## .github/workflows/ci-image-build.yml: ## @@ -195,19 +194,19 @@ ${{ inputs.do-build == 'true' && inputs.image-tag || '' }}" run: > breeze ci-ima

(airflow) branch fix-image-cache updated (579cd39821 -> e4648ad9a1)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git omit 579cd39821 Fix image cache optimizations - speeding up the build add 3832ead221 Fix failing MinSQLAlche

Re: [PR] Allow users to write dag_id and task_id in their national characters, added display name for dag / task (v2) [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38446: URL: https://github.com/apache/airflow/pull/38446#issuecomment-2017069297 The failing MINSQL ALchemy test here is fixed in #38445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] allows users to write dag_id and task_id in their national characters, added display name for dag / task [airflow]

2024-03-24 Thread via GitHub
jscheffl commented on PR #32520: URL: https://github.com/apache/airflow/pull/32520#issuecomment-2017051703 As linked in #38446 - I tried to spend time on the weekend to continue working on this PR... let's see if we can get this reviewed and completed... -- This is an automated message f

Re: [PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38444: URL: https://github.com/apache/airflow/pull/38444#issuecomment-2017045033 Closed in favour of canary test in #38445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] Fix 22073 [airflow]

2024-03-24 Thread via GitHub
jscheffl opened a new pull request, #38446: URL: https://github.com/apache/airflow/pull/38446 This PR tries to continue the work from @xgao1023 in PR #35320. It merges with current main. On top of this I fixed the bugs in pytest, some glitches in UI. I'd LOVE to get this into 2.9.0 a

[PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38445: URL: https://github.com/apache/airflow/pull/38445 Ths MIN SQLACLHEMY test was based on retrieving sqlalchemy min version from pyproject.toml but since we moved it to hatch_build.py we should read it from there. ---

(airflow) branch fix-failing-sqlalchemy-test created (now 3832ead221)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-failing-sqlalchemy-test in repository https://gitbox.apache.org/repos/asf/airflow.git at 3832ead221 Fix failing MinSQLAlchemy test after moving sqlalchemy spec This branch includes the

Re: [PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk closed pull request #38444: Fix failing MinSQLAlchemy test after moving sqlalchemy spec URL: https://github.com/apache/airflow/pull/38444 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

(airflow) 01/01: Fix failing MinSQLAlchemy test after moving sqlalchemy spec

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch fix-failing-sqlalchemy-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 3832ead22158ed963417367082868f5563f8f734 Author: Jarek Potiuk AuthorDate: Mon Mar 25 02:11:39 2024

Re: [PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38444: URL: https://github.com/apache/airflow/pull/38444#discussion_r1536962905 ## .github/workflows/run-unit-tests.yml: ## @@ -126,7 +126,7 @@ jobs: BACKEND_VERSION: "${{ matrix.backend-version }}" DEBUG_RESOURCES: "${{ inputs.debug

[PR] Fix failing MinSQLAlchemy test after moving sqlalchemy spec [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38444: URL: https://github.com/apache/airflow/pull/38444 Ths MIN SQLACLHEMY test was based on retrieving sqlalchemy min version from pyproject.toml but since we moved it to hatch_build.py we should read it from there. ---

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536951037 ## .github/workflows/finalize-tests.yml: ## @@ -120,31 +120,48 @@ jobs: run: git push - # Push BuildX cache to GitHub Registry in Apache reposit

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536950612 ## .github/workflows/additional-ci-image-checks.yml: ## @@ -89,42 +89,40 @@ jobs: # delay cache refresh. It does not attempt to upgrade to newer dependencies. #

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536950198 ## .github/workflows/additional-ci-image-checks.yml: ## @@ -89,42 +89,40 @@ jobs: # delay cache refresh. It does not attempt to upgrade to newer dependencies. #

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536950005 ## .github/workflows/additional-ci-image-checks.yml: ## @@ -89,42 +89,40 @@ jobs: # delay cache refresh. It does not attempt to upgrade to newer dependencies. #

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536949904 ## dev/breeze/src/airflow_breeze/utils/docker_command_utils.py: ## @@ -381,12 +381,15 @@ def prepare_base_build_command(image_params: CommonBuildParams) -> list[str]:

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536949488 ## dev/breeze/src/airflow_breeze/utils/docker_command_utils.py: ## @@ -658,6 +661,7 @@ def autodetect_docker_context(): def get_and_use_docker_context(context: str):

Re: [PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38442: URL: https://github.com/apache/airflow/pull/38442#discussion_r1536949451 ## dev/breeze/src/airflow_breeze/utils/image.py: ## @@ -197,18 +197,8 @@ def tag_image_as_latest(image_params: CommonBuildParams, output: Output | None) chec

(airflow) branch fix-image-cache updated (64494f3c09 -> 579cd39821)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 64494f3c09 Fix image cache optimizations - speeding up the build add 579cd39821 Fix image cache optimiz

Re: [PR] Helm: Add namespace to all namespace-scoped resources [airflow]

2024-03-24 Thread via GitHub
github-actions[bot] commented on PR #33177: URL: https://github.com/apache/airflow/pull/33177#issuecomment-2017004416 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

(airflow) branch fix-image-cache updated (81467fa5e8 -> 64494f3c09)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git omit 81467fa5e8 Fix image cache optimizations - speeding up the build add 64494f3c09 Fix image cache optimiz

(airflow) branch fix-image-cache updated (35763472bf -> 81467fa5e8)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 35763472bf Fix image cache optimizations - speeding up the build add 81467fa5e8 Fix image cache optimiz

(airflow) branch fix-image-cache updated (3eb5212b71 -> 35763472bf)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 3eb5212b71 Fix image cache optimizations - speeding up the build add 35763472bf Fix image cache optimiz

(airflow) branch fix-image-cache updated (44a256052d -> 3eb5212b71)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 44a256052d Fix image cache optimizations - speeding up the build add 20cb9f1770 Upgrade to newer build

(airflow) branch main updated: Upgrade to newer build dependencies including the right Python version (#38443)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 20cb9f1770 Upgrade to newer build dependencies incl

Re: [PR] Upgrade to newer build dependencies including the right Python version [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38443: URL: https://github.com/apache/airflow/pull/38443 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Add legend for duration markline. [airflow]

2024-03-24 Thread via GitHub
simond commented on PR #38434: URL: https://github.com/apache/airflow/pull/38434#issuecomment-2016947663 Just looking at your screenshots - you may want to change the wording to 'median total', 'median run' etc rather than 'mean' - median and mean are different things. Looks like you're usi

[PR] Upgrade to newer build dependencies including the right Python version [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38443: URL: https://github.com/apache/airflow/pull/38443 This change upgrades build dependencies, including the fact that for earlier Python versions, the set of dependencies might be different than than for later ones. This PR assumes that default python

Re: [PR] Test cache refreshing in CI [JUST TESTING] [airflow]

2024-03-24 Thread via GitHub
potiuk closed pull request #38427: Test cache refreshing in CI [JUST TESTING] URL: https://github.com/apache/airflow/pull/38427 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

(airflow) branch fix-image-cache updated (fe69e7423a -> 44a256052d)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard fe69e7423a Fix image cache optimizations - speeding up the build add 25e5d54192 Print selective-check t

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38437: URL: https://github.com/apache/airflow/pull/38437 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch fix-image-cache updated (89a3c70b96 -> fe69e7423a)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git discard 89a3c70b96 Fix image cache optimizations - speeding up the build add fe69e7423a Fix image cache optimiz

Re: [I] Wrong DAG duration time [airflow]

2024-03-24 Thread via GitHub
arley-wilches commented on issue #30877: URL: https://github.com/apache/airflow/issues/30877#issuecomment-2016938365 Hi there. I found a similar issue, the duration time does not take into account mapped tasks and I have two tasks one of them took about 3 hours and the other one to

[PR] Fix image cache optimizations - speeding up the build [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38442: URL: https://github.com/apache/airflow/pull/38442 The recent refactors in workflows broke the way how cache had been used in the CI builds. This PR brings back the optimizations by using the cache and rebuilding it.

(airflow) branch fix-image-cache created (now 89a3c70b96)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git at 89a3c70b96 Fix image cache optimizations - speeding up the build This branch includes the following new com

(airflow) 01/01: Fix image cache optimizations - speeding up the build

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch fix-image-cache in repository https://gitbox.apache.org/repos/asf/airflow.git commit 89a3c70b9654c7f8812e87d49bc4dfec6979486d Author: Jarek Potiuk AuthorDate: Sun Mar 24 21:21:03 2024 +0100

(airflow) branch test-cache-refreshing updated (90d68148c3 -> 375f9426a3)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch test-cache-refreshing in repository https://gitbox.apache.org/repos/asf/airflow.git discard 90d68148c3 Test cache refreshing in CI add 375f9426a3 Test cache refreshing in CI This update ad

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016913907 Ah. Now with traceback, it's clear. The old breeze expects "dependencies" property in `project` ... That's fine. This will be fixed once the PR is merged. -- This is an automated messa

Re: [PR] Apply task instance mutation hook consistently [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38440: URL: https://github.com/apache/airflow/pull/38440#issuecomment-2016912706 NIT: Maybe a test case @jscheffl ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38437: URL: https://github.com/apache/airflow/pull/38437#discussion_r1536890083 ## INSTALL: ## @@ -128,19 +141,19 @@ You can see the list of available envs with: This is what it shows currently: -┏━┳━┳━━┳━━

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (02a6ee9938 -> 83135236cd)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git omit 02a6ee9938 Turn optional-dependencies in pyproject.toml into dynamic propert

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (2b518b4e25 -> 02a6ee9938)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git omit 2b518b4e25 Turn optional-dependencies in pyproject.toml into dynamic propert

(airflow) branch main updated: Print selective-check traceback on stdout rather than stderr (#38441)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 25e5d54192 Print selective-check traceback on stdou

Re: [PR] Print selective-check traceback on stdout rather than stderr [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38441: URL: https://github.com/apache/airflow/pull/38441#issuecomment-2016911399 > An alternative would be use use 2>&1 in the shell call to redirect all stderr to stdout? > In total meaning: breeze ci selective-check 2>&1 | tee ${GITHUB_OUTPUT} ? We already

Re: [PR] Print selective-check traceback on stdout rather than stderr [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38441: URL: https://github.com/apache/airflow/pull/38441 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
jscheffl commented on code in PR #38437: URL: https://github.com/apache/airflow/pull/38437#discussion_r1536887077 ## INSTALL: ## @@ -128,19 +141,19 @@ You can see the list of available envs with: This is what it shows currently: -┏━┳━┳━━┳

(airflow) branch test-cache-refreshing updated (106bdf172b -> 90d68148c3)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch test-cache-refreshing in repository https://gitbox.apache.org/repos/asf/airflow.git discard 106bdf172b Test cache refreshing in CI add 078e7ed135 Mysql change xcom value col type for MySQL

Re: [PR] Print selective-check traceback on stdout rather than stderr [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38441: URL: https://github.com/apache/airflow/pull/38441#issuecomment-2016903858 Should help to diagnose issues like https://github.com/apache/airflow/actions/runs/8410545233/job/23029279435 -- This is an automated message from the Apache Git Service. To respond t

[PR] Print selective-check traceback on stdout rather than stderr [airflow]

2024-03-24 Thread via GitHub
potiuk opened a new pull request, #38441: URL: https://github.com/apache/airflow/pull/38441 We are using stdout of selective-check to print diagnostic information and stderr is redirected to produce outputs for GIHUB_OUTPUTS special variable in GitHub Actions. However this means that when t

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016894701 Updated documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016894632 > If I run into this we should handle this explicitly. Is it possible to fail install if pip version is too old? No. But it ONLY affects you if you want to install airflow from sou

(airflow) branch constraints-main updated: Updating constraints. Github run id:8410961843

2024-03-24 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 368b7511da Updating con

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (902d144946 -> 2b518b4e25)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard 902d144946 Turn optional-dependencies in pyproject.toml into dynamic propert

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
jscheffl commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016892056 > Actually looks like `pip 22.0` is the LAST one that does not support this dynamic retrieval of dependencies. So you picked really edge'y case @jscheffl. I will add this to the docs.

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016891091 Actually looks like `pip 22.0` is the LAST one that does not support this dynamic retrieval of dependencies. So you picked really edge'y case @jscheffl. I will add this to the docs. --

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (e1139449bd -> 902d144946)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard e1139449bd Turn optional-dependencies in pyproject.toml into dynamic propert

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (8343dabf24 -> e1139449bd)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard 8343dabf24 Turn optional-dependencies in pyproject.toml into dynamic propert

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016887762 General advice from `pip` team is to always upgrade to newest version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016887543 Yes. You must upgrade your `pip`. Version 22 is from 2022 and a lot of the functionality and PEPs needed have been approved and implemented later than that. -- This is an automated me

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
jscheffl commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016885982 I tried to "naively" install as described in my devenv. 1. Build local CI Image: works 2. Install via pip in local env: FAILS (see 1) 1) Details of pip install in Xubunt

[PR] Apply task instance mutation hook consistently [airflow]

2024-03-24 Thread via GitHub
jscheffl opened a new pull request, #38440: URL: https://github.com/apache/airflow/pull/38440 We are using the [Cluster Policies](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/cluster-policies.html#task-instance-mutation) and in the the feature of the "

Re: [PR] Add executor field to the DB and parameter to the operators [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38054: URL: https://github.com/apache/airflow/pull/38054#issuecomment-2016878718 Yep. We cannot merge it without resolving conflict :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

(airflow) branch main updated (77d2fc7d75 -> 694826d1bd)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 77d2fc7d75 Check task attribute before use in sentry.add_tagging() (#37143) add 694826d1bd refactor: Refactored _

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #37937: URL: https://github.com/apache/airflow/pull/37937 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Check task attribute before use in sentry.add_tagging() (#37143)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 77d2fc7d75 Check task attribute before use in sentr

Re: [PR] Check task attribute before use in sentry.add_tagging() [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #37143: URL: https://github.com/apache/airflow/pull/37143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (d820f13e68 -> 4a4eee113f)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d820f13e68 Use `relative-imports (TID252)` instead of `pygrep` rule (#38428) add 4a4eee113f Delete deprecated Aut

Re: [PR] Delete deprecated AutoML operators (inc. 'google-cloud-automl' dep.) and deprecate hook and links [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38418: URL: https://github.com/apache/airflow/pull/38418 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Improve logging readability with DockerOperator [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38008: URL: https://github.com/apache/airflow/pull/38008#discussion_r1536864652 ## airflow/providers/docker/operators/docker.py: ## @@ -423,13 +447,19 @@ def _run_image_with_mounts(self, target_mounts, add_tmp_variable: bool) -> list[ )

Re: [PR] Adding Task ids in DAG details API endpoint #37564 [airflow]

2024-03-24 Thread via GitHub
msoni1369 commented on PR #37866: URL: https://github.com/apache/airflow/pull/37866#issuecomment-2016869645 > If we call the field `tasks`, users will expect more information, not just `task_ids`. You may change the field to task_ids. In the future, we can deprecate that and have `tasks` fi

Re: [PR] Support adding custom TI Deps to help DagRun make more flexible TI scheduling decisions [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #37778: URL: https://github.com/apache/airflow/pull/37778#issuecomment-2016865953 Agree with TP that if we were to merge this one, we will need quite a bit more: * documentation with some examples * general use cases where it would be available * some perf

(airflow) branch main updated (81790aaaa0 -> d820f13e68)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 817900 Fix deprecated `DockerOperator` operator arguments in `MappedOperator` (#38379) add d820f13e68 Use `re

Re: [PR] Use `relative-imports (TID252)` instead of `pygrep` rule [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38428: URL: https://github.com/apache/airflow/pull/38428 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Fix deprecated `DockerOperator` operator arguments in `MappedOperator` (#38379)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 817900 Fix deprecated `DockerOperator` operator

Re: [PR] Fix deprecated `DockerOperator` operator arguments in `MappedOperator` [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38379: URL: https://github.com/apache/airflow/pull/38379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (0cc410b5b8 -> d3e9229105)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 0cc410b5b8 Updat build dependencies after hatchling update (#38439) add d3e9229105 Bugfix/update screenshots (#384

Re: [PR] Bugfix/update screenshots [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38438: URL: https://github.com/apache/airflow/pull/38438 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on code in PR #38437: URL: https://github.com/apache/airflow/pull/38437#discussion_r1536854627 ## airflow_pre_installed_providers.txt: ## @@ -1,7 +1,7 @@ # List of all the providers that are pre-installed when you run `pip install apache-airflow` without extra

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016858378 cc: @uranusjr -> I'd appreciate your comments here, I believe now we should be **really** compliant with the PEP expectations. I compared the generated METADATA in the .whl files

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (db109c8372 -> 8343dabf24)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard db109c8372 Turn optional-dependencies in pyproject.toml into dynamic propert

Re: [PR] Turn optional-dependencies in pyproject.toml into dynamic property [airflow]

2024-03-24 Thread via GitHub
potiuk commented on PR #38437: URL: https://github.com/apache/airflow/pull/38437#issuecomment-2016854440 After some recent (and looking and old) discussions, it turns out that we need one more thing to be fully PEP-517 compliant. While hatchling allows us (for now) to dynamically modiy proj

Re: [PR] changing dag_processing.processes from UpDownCounter to guage [airflow]

2024-03-24 Thread via GitHub
Bowrna closed pull request #38400: changing dag_processing.processes from UpDownCounter to guage URL: https://github.com/apache/airflow/pull/38400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] heartbeat recovery message [airflow]

2024-03-24 Thread via GitHub
Bowrna commented on code in PR #34457: URL: https://github.com/apache/airflow/pull/34457#discussion_r1536847700 ## airflow/jobs/job.py: ## @@ -131,22 +146,20 @@ def executors(self): def heartrate(self) -> float: return Job._heartrate(self.job_type) -def is_al

(airflow) branch turn-optional-dependencies-in-dynamic-metadata updated (ad76bf579b -> db109c8372)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch turn-optional-dependencies-in-dynamic-metadata in repository https://gitbox.apache.org/repos/asf/airflow.git discard ad76bf579b Turn optional-dependencies in pyproject.toml into dynamic propert

(airflow) branch main updated: Updat build dependencies after hatchling update (#38439)

2024-03-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0cc410b5b8 Updat build dependencies after hatchling

Re: [PR] Update build dependencies after hatchling update [airflow]

2024-03-24 Thread via GitHub
potiuk merged PR #38439: URL: https://github.com/apache/airflow/pull/38439 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

  1   2   >