codeant-ai-for-open-source[bot] commented on code in PR #41075: URL: https://github.com/apache/superset/pull/41075#discussion_r3572406843
########## tests/unit_tests/tasks/test_version_history_retention.py: ########## @@ -0,0 +1,200 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Unit tests for the operational instrumentation in +``superset.tasks.version_history_retention``. + +Covers the branches that emit statsd counters: the ``retention_days <= 0`` +short-circuit, incomplete shadow-table resolution, the ``OperationalError`` +retry path, and the terminal failure counter. The +"happy path" / SERIALIZABLE retry behaviour against a real database is +exercised by ``tests/integration_tests/versioning/retention_prune_tests.py``; +this file pins the metric-emission contract that is load-bearing for +operator alerting. +""" + +from __future__ import annotations + +from collections.abc import Iterator +from unittest.mock import MagicMock, patch + +import pytest +from sqlalchemy.exc import OperationalError +from sqlalchemy_continuum.exc import ClassNotVersioned + +from superset.tasks import version_history_retention + + [email protected](name="stats") +def _stats_fixture() -> Iterator[MagicMock]: + """Patch the shared stats logger so every test can assert on + emissions without standing up the real statsd backend.""" + with patch.object( + version_history_retention, "stats_logger_manager" + ) as mock_manager: + mock_manager.instance = MagicMock() + yield mock_manager.instance + + +def test_retention_disabled_emits_skipped_metric(stats: MagicMock) -> None: + """``retention_days <= 0`` is the documented "disable retention" + config. The early-return must emit ``superset.versioning.retention.skipped`` + so a dashboard can tell "operator disabled it" apart from "scheduler + isn't running".""" + result = version_history_retention._prune_old_versions_impl(retention_days=0) + assert result == {"skipped": 1} + stats.incr.assert_called_once_with("superset.versioning.retention.skipped") + stats.gauge.assert_not_called() + + +def test_task_normalizes_string_retention_config(stats: MagicMock) -> None: + """String values from custom config modules are normalized to integers.""" + mock_app = MagicMock() Review Comment: **Suggestion:** Add an explicit type annotation to this local test variable to comply with the required type-hints rule. [custom_rule] **Severity Level:** Minor ๐งน <details> <summary><b>Why it matters? โญ </b></summary> The file is new Python code, and this local variable is introduced without an explicit type hint. That matches the requirement to flag relevant variables in new or modified Python code that omit type hints. </details> <details> <summary><b>Rule source ๐ </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=3d99ecb1f69043218fe6e304effbfdd5&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=3d99ecb1f69043218fe6e304effbfdd5&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent ๐ค </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/tasks/test_version_history_retention.py **Line:** 65:65 **Comment:** *Custom Rule: Add an explicit type annotation to this local test variable to comply with the required type-hints rule. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=0094271ec723df6256a3c39eb7acb46cdf326a0da5ee0dd0115d2f0e6667510f&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=0094271ec723df6256a3c39eb7acb46cdf326a0da5ee0dd0115d2f0e6667510f&reaction=dislike'>๐</a> ########## tests/unit_tests/tasks/test_version_history_retention.py: ########## @@ -0,0 +1,200 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Unit tests for the operational instrumentation in +``superset.tasks.version_history_retention``. + +Covers the branches that emit statsd counters: the ``retention_days <= 0`` +short-circuit, incomplete shadow-table resolution, the ``OperationalError`` +retry path, and the terminal failure counter. The +"happy path" / SERIALIZABLE retry behaviour against a real database is +exercised by ``tests/integration_tests/versioning/retention_prune_tests.py``; +this file pins the metric-emission contract that is load-bearing for +operator alerting. +""" + +from __future__ import annotations + +from collections.abc import Iterator +from unittest.mock import MagicMock, patch + +import pytest +from sqlalchemy.exc import OperationalError +from sqlalchemy_continuum.exc import ClassNotVersioned + +from superset.tasks import version_history_retention + + [email protected](name="stats") +def _stats_fixture() -> Iterator[MagicMock]: + """Patch the shared stats logger so every test can assert on + emissions without standing up the real statsd backend.""" + with patch.object( + version_history_retention, "stats_logger_manager" + ) as mock_manager: + mock_manager.instance = MagicMock() + yield mock_manager.instance + + +def test_retention_disabled_emits_skipped_metric(stats: MagicMock) -> None: + """``retention_days <= 0`` is the documented "disable retention" + config. The early-return must emit ``superset.versioning.retention.skipped`` + so a dashboard can tell "operator disabled it" apart from "scheduler + isn't running".""" + result = version_history_retention._prune_old_versions_impl(retention_days=0) + assert result == {"skipped": 1} + stats.incr.assert_called_once_with("superset.versioning.retention.skipped") + stats.gauge.assert_not_called() + + +def test_task_normalizes_string_retention_config(stats: MagicMock) -> None: + """String values from custom config modules are normalized to integers.""" + mock_app = MagicMock() + mock_app.config = {"SUPERSET_VERSION_HISTORY_RETENTION_DAYS": "30"} + with ( + patch.object(version_history_retention, "current_app", mock_app), + patch.object( + version_history_retention, + "_prune_old_versions_impl", + return_value={"pruned_transactions": 0}, + ) as prune, + ): + result = version_history_retention.prune_old_versions() + + assert result == {"pruned_transactions": 0} + prune.assert_called_once_with(30) + stats.incr.assert_not_called() + + +def test_incomplete_shadow_table_resolution_fails_closed( + stats: MagicMock, +) -> None: + """Missing shadow metadata must abort before the destructive pass.""" + with ( + patch.object( + version_history_retention, + "_resolve_shadow_tables", + side_effect=RuntimeError("missing shadow"), + ), + pytest.raises(RuntimeError, match="missing shadow"), + ): + version_history_retention._prune_old_versions_impl(retention_days=30) + stats.incr.assert_not_called() + + +def test_resolve_shadow_tables_rejects_partial_registry() -> None: + """One missing versioned mapper makes the complete registry unsafe.""" + resolved_table = MagicMock() Review Comment: **Suggestion:** Add an explicit type annotation for this local variable so it is type-declared under the typing rule. [custom_rule] **Severity Level:** Minor ๐งน <details> <summary><b>Why it matters? โญ </b></summary> This is a newly added local variable in Python code with no type annotation, so it fits the custom rule requiring type hints on relevant variables. </details> <details> <summary><b>Rule source ๐ </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=4fee0039bbe84cfeb4c938202a6d66ce&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=4fee0039bbe84cfeb4c938202a6d66ce&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent ๐ค </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/tasks/test_version_history_retention.py **Line:** 100:100 **Comment:** *Custom Rule: Add an explicit type annotation for this local variable so it is type-declared under the typing rule. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=b28c6cb95122109ff29701fa32499558c901266ffc7bdf33fd058961729c5fe6&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=b28c6cb95122109ff29701fa32499558c901266ffc7bdf33fd058961729c5fe6&reaction=dislike'>๐</a> ########## tests/unit_tests/tasks/test_version_history_retention.py: ########## @@ -0,0 +1,200 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Unit tests for the operational instrumentation in +``superset.tasks.version_history_retention``. + +Covers the branches that emit statsd counters: the ``retention_days <= 0`` +short-circuit, incomplete shadow-table resolution, the ``OperationalError`` +retry path, and the terminal failure counter. The +"happy path" / SERIALIZABLE retry behaviour against a real database is +exercised by ``tests/integration_tests/versioning/retention_prune_tests.py``; +this file pins the metric-emission contract that is load-bearing for +operator alerting. +""" + +from __future__ import annotations + +from collections.abc import Iterator +from unittest.mock import MagicMock, patch + +import pytest +from sqlalchemy.exc import OperationalError +from sqlalchemy_continuum.exc import ClassNotVersioned + +from superset.tasks import version_history_retention + + [email protected](name="stats") +def _stats_fixture() -> Iterator[MagicMock]: + """Patch the shared stats logger so every test can assert on + emissions without standing up the real statsd backend.""" + with patch.object( + version_history_retention, "stats_logger_manager" + ) as mock_manager: + mock_manager.instance = MagicMock() + yield mock_manager.instance + + +def test_retention_disabled_emits_skipped_metric(stats: MagicMock) -> None: + """``retention_days <= 0`` is the documented "disable retention" + config. The early-return must emit ``superset.versioning.retention.skipped`` + so a dashboard can tell "operator disabled it" apart from "scheduler + isn't running".""" + result = version_history_retention._prune_old_versions_impl(retention_days=0) + assert result == {"skipped": 1} + stats.incr.assert_called_once_with("superset.versioning.retention.skipped") + stats.gauge.assert_not_called() + + +def test_task_normalizes_string_retention_config(stats: MagicMock) -> None: + """String values from custom config modules are normalized to integers.""" + mock_app = MagicMock() + mock_app.config = {"SUPERSET_VERSION_HISTORY_RETENTION_DAYS": "30"} + with ( + patch.object(version_history_retention, "current_app", mock_app), + patch.object( + version_history_retention, + "_prune_old_versions_impl", + return_value={"pruned_transactions": 0}, + ) as prune, + ): + result = version_history_retention.prune_old_versions() + + assert result == {"pruned_transactions": 0} + prune.assert_called_once_with(30) + stats.incr.assert_not_called() + + +def test_incomplete_shadow_table_resolution_fails_closed( + stats: MagicMock, +) -> None: + """Missing shadow metadata must abort before the destructive pass.""" + with ( + patch.object( + version_history_retention, + "_resolve_shadow_tables", + side_effect=RuntimeError("missing shadow"), + ), + pytest.raises(RuntimeError, match="missing shadow"), + ): + version_history_retention._prune_old_versions_impl(retention_days=30) + stats.incr.assert_not_called() + + +def test_resolve_shadow_tables_rejects_partial_registry() -> None: + """One missing versioned mapper makes the complete registry unsafe.""" + resolved_table = MagicMock() + + def resolve_version_class(model: type[object]) -> MagicMock: + if model.__name__ == "TableColumn": + raise ClassNotVersioned(model) + version_model = MagicMock() + version_model.__table__ = resolved_table + return version_model + + with patch("sqlalchemy_continuum.version_class", side_effect=resolve_version_class): + with pytest.raises(RuntimeError, match="TableColumn"): + version_history_retention._resolve_shadow_tables(MagicMock()) + + +def test_serialization_failure_then_success_increments_retried_once( + stats: MagicMock, +) -> None: + """A single ``OperationalError`` on attempt 1 should: + * fire ``.retried`` once (one retry happened), + * sleep for ``_RETRY_BACKOFF_BASE_SECONDS`` (patched away in tests), + * succeed on attempt 2 with ``stats["retried"] == 1``, + * fire ``.pruned_transactions`` gauge with the success count. + + The contract on ``.retried`` is "fires per retry attempt observed" + (per-attempt, not per-session). This test pins the per-attempt shape so + a future refactor doesn't silently change the metric semantics.""" + pass_fn = MagicMock( + side_effect=[ + OperationalError("SELECT 1", {}, Exception("could not serialize access")), + {"pruned_transactions": 7, "cutoff": "2026-01-01T00:00:00"}, + ] + ) + tables = version_history_retention.ShadowTables( + parent=[MagicMock()], child=[MagicMock()], m2m=None, transaction=MagicMock() + ) + with ( + patch.object( + version_history_retention, "_resolve_shadow_tables", return_value=tables + ), + patch.object(version_history_retention, "_run_prune_pass", pass_fn), + patch.object(version_history_retention.time, "sleep"), + ): + result = version_history_retention._prune_old_versions_impl(retention_days=30) + + assert result["retried"] == 1 + assert result["pruned_transactions"] == 7 + incr_calls = [call.args[0] for call in stats.incr.call_args_list] + assert incr_calls == ["superset.versioning.retention.retried"], ( + f"Expected exactly one .retried emission; got {incr_calls}" + ) + stats.gauge.assert_called_once_with( + "superset.versioning.retention.pruned_transactions", 7 + ) + + +def test_all_attempts_fail_reraises_after_max_retries(stats: MagicMock) -> None: + """When every attempt raises ``OperationalError``, the task re-raises + after ``_MAX_RETRY_ATTEMPTS`` so the outer Celery wrapper logs it. + The retry counter fires once per attempt that hit the exception.""" + exc = OperationalError("SELECT 1", {}, Exception("conflict")) + tables = version_history_retention.ShadowTables( + parent=[MagicMock()], child=[MagicMock()], m2m=None, transaction=MagicMock() + ) + with ( + patch.object( + version_history_retention, "_resolve_shadow_tables", return_value=tables + ), + patch.object(version_history_retention, "_run_prune_pass", side_effect=exc), + patch.object(version_history_retention.time, "sleep"), + pytest.raises(OperationalError), + ): + version_history_retention._prune_old_versions_impl(retention_days=30) + + incr_calls = [call.args[0] for call in stats.incr.call_args_list] + assert ( + incr_calls.count("superset.versioning.retention.retried") + == version_history_retention._MAX_RETRY_ATTEMPTS + ), ( + f"Expected {version_history_retention._MAX_RETRY_ATTEMPTS} " + f".retried emissions (one per attempt); got {incr_calls}" + ) + + +def test_terminal_failure_emits_failed_metric_and_swallows(stats: MagicMock) -> None: + """The Celery wrapper catches a terminal failure, returns ``{"error": 1}`` + (so the schedule isn't poisoned), AND emits a ``.failed`` counter so the + destructive job's primary failure mode is alertable, not just logged.""" + mock_app = MagicMock() Review Comment: **Suggestion:** Add an explicit type annotation to this second local app mock variable as well, for consistent compliance with the typing rule. [custom_rule] **Severity Level:** Minor ๐งน <details> <summary><b>Why it matters? โญ </b></summary> This is another newly added local variable without a type annotation, so it also violates the type-hint requirement. </details> <details> <summary><b>Rule source ๐ </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=8ed52698413b41469b17bee444ed00a5&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=8ed52698413b41469b17bee444ed00a5&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent ๐ค </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/tasks/test_version_history_retention.py **Line:** 187:187 **Comment:** *Custom Rule: Add an explicit type annotation to this second local app mock variable as well, for consistent compliance with the typing rule. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=0d503f67f6dcafd5416ea2f5eb53110ce79b9fb79836b322ce6a5dc5e42a4f25&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=0d503f67f6dcafd5416ea2f5eb53110ce79b9fb79836b322ce6a5dc5e42a4f25&reaction=dislike'>๐</a> ########## tests/unit_tests/tasks/test_version_history_retention.py: ########## @@ -0,0 +1,200 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Unit tests for the operational instrumentation in +``superset.tasks.version_history_retention``. + +Covers the branches that emit statsd counters: the ``retention_days <= 0`` +short-circuit, incomplete shadow-table resolution, the ``OperationalError`` +retry path, and the terminal failure counter. The +"happy path" / SERIALIZABLE retry behaviour against a real database is +exercised by ``tests/integration_tests/versioning/retention_prune_tests.py``; +this file pins the metric-emission contract that is load-bearing for +operator alerting. +""" + +from __future__ import annotations + +from collections.abc import Iterator +from unittest.mock import MagicMock, patch + +import pytest +from sqlalchemy.exc import OperationalError +from sqlalchemy_continuum.exc import ClassNotVersioned + +from superset.tasks import version_history_retention + + [email protected](name="stats") +def _stats_fixture() -> Iterator[MagicMock]: + """Patch the shared stats logger so every test can assert on + emissions without standing up the real statsd backend.""" + with patch.object( + version_history_retention, "stats_logger_manager" + ) as mock_manager: + mock_manager.instance = MagicMock() + yield mock_manager.instance + + +def test_retention_disabled_emits_skipped_metric(stats: MagicMock) -> None: + """``retention_days <= 0`` is the documented "disable retention" + config. The early-return must emit ``superset.versioning.retention.skipped`` + so a dashboard can tell "operator disabled it" apart from "scheduler + isn't running".""" + result = version_history_retention._prune_old_versions_impl(retention_days=0) + assert result == {"skipped": 1} + stats.incr.assert_called_once_with("superset.versioning.retention.skipped") + stats.gauge.assert_not_called() + + +def test_task_normalizes_string_retention_config(stats: MagicMock) -> None: + """String values from custom config modules are normalized to integers.""" + mock_app = MagicMock() + mock_app.config = {"SUPERSET_VERSION_HISTORY_RETENTION_DAYS": "30"} + with ( + patch.object(version_history_retention, "current_app", mock_app), + patch.object( + version_history_retention, + "_prune_old_versions_impl", + return_value={"pruned_transactions": 0}, + ) as prune, + ): + result = version_history_retention.prune_old_versions() + + assert result == {"pruned_transactions": 0} + prune.assert_called_once_with(30) + stats.incr.assert_not_called() + + +def test_incomplete_shadow_table_resolution_fails_closed( + stats: MagicMock, +) -> None: + """Missing shadow metadata must abort before the destructive pass.""" + with ( + patch.object( + version_history_retention, + "_resolve_shadow_tables", + side_effect=RuntimeError("missing shadow"), + ), + pytest.raises(RuntimeError, match="missing shadow"), + ): + version_history_retention._prune_old_versions_impl(retention_days=30) + stats.incr.assert_not_called() + + +def test_resolve_shadow_tables_rejects_partial_registry() -> None: + """One missing versioned mapper makes the complete registry unsafe.""" + resolved_table = MagicMock() + + def resolve_version_class(model: type[object]) -> MagicMock: + if model.__name__ == "TableColumn": + raise ClassNotVersioned(model) + version_model = MagicMock() + version_model.__table__ = resolved_table + return version_model + + with patch("sqlalchemy_continuum.version_class", side_effect=resolve_version_class): + with pytest.raises(RuntimeError, match="TableColumn"): + version_history_retention._resolve_shadow_tables(MagicMock()) + + +def test_serialization_failure_then_success_increments_retried_once( + stats: MagicMock, +) -> None: + """A single ``OperationalError`` on attempt 1 should: + * fire ``.retried`` once (one retry happened), + * sleep for ``_RETRY_BACKOFF_BASE_SECONDS`` (patched away in tests), + * succeed on attempt 2 with ``stats["retried"] == 1``, + * fire ``.pruned_transactions`` gauge with the success count. + + The contract on ``.retried`` is "fires per retry attempt observed" + (per-attempt, not per-session). This test pins the per-attempt shape so + a future refactor doesn't silently change the metric semantics.""" + pass_fn = MagicMock( Review Comment: **Suggestion:** Annotate this locally assigned callable mock with an explicit type to satisfy the type-hint requirement. [custom_rule] **Severity Level:** Minor ๐งน <details> <summary><b>Why it matters? โญ </b></summary> The local callable mock is introduced without an explicit annotation. Since the rule applies to relevant variables that can be annotated, this is a real violation. </details> <details> <summary><b>Rule source ๐ </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=ceeac0d2be3a4a27a6dd8c5da6f3ceba&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=ceeac0d2be3a4a27a6dd8c5da6f3ceba&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent ๐ค </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/tasks/test_version_history_retention.py **Line:** 126:126 **Comment:** *Custom Rule: Annotate this locally assigned callable mock with an explicit type to satisfy the type-hint requirement. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=3bd7f4f83ca832841970df69927cdaeda20b0f37b94d5316d4cc463f544251b8&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=3bd7f4f83ca832841970df69927cdaeda20b0f37b94d5316d4cc463f544251b8&reaction=dislike'>๐</a> ########## tests/unit_tests/tasks/test_version_history_retention.py: ########## @@ -0,0 +1,200 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Unit tests for the operational instrumentation in +``superset.tasks.version_history_retention``. + +Covers the branches that emit statsd counters: the ``retention_days <= 0`` +short-circuit, incomplete shadow-table resolution, the ``OperationalError`` +retry path, and the terminal failure counter. The +"happy path" / SERIALIZABLE retry behaviour against a real database is +exercised by ``tests/integration_tests/versioning/retention_prune_tests.py``; +this file pins the metric-emission contract that is load-bearing for +operator alerting. +""" + +from __future__ import annotations + +from collections.abc import Iterator +from unittest.mock import MagicMock, patch + +import pytest +from sqlalchemy.exc import OperationalError +from sqlalchemy_continuum.exc import ClassNotVersioned + +from superset.tasks import version_history_retention + + [email protected](name="stats") +def _stats_fixture() -> Iterator[MagicMock]: + """Patch the shared stats logger so every test can assert on + emissions without standing up the real statsd backend.""" + with patch.object( + version_history_retention, "stats_logger_manager" + ) as mock_manager: + mock_manager.instance = MagicMock() + yield mock_manager.instance + + +def test_retention_disabled_emits_skipped_metric(stats: MagicMock) -> None: + """``retention_days <= 0`` is the documented "disable retention" + config. The early-return must emit ``superset.versioning.retention.skipped`` + so a dashboard can tell "operator disabled it" apart from "scheduler + isn't running".""" + result = version_history_retention._prune_old_versions_impl(retention_days=0) + assert result == {"skipped": 1} + stats.incr.assert_called_once_with("superset.versioning.retention.skipped") + stats.gauge.assert_not_called() + + +def test_task_normalizes_string_retention_config(stats: MagicMock) -> None: + """String values from custom config modules are normalized to integers.""" + mock_app = MagicMock() + mock_app.config = {"SUPERSET_VERSION_HISTORY_RETENTION_DAYS": "30"} + with ( + patch.object(version_history_retention, "current_app", mock_app), + patch.object( + version_history_retention, + "_prune_old_versions_impl", + return_value={"pruned_transactions": 0}, + ) as prune, + ): + result = version_history_retention.prune_old_versions() + + assert result == {"pruned_transactions": 0} + prune.assert_called_once_with(30) + stats.incr.assert_not_called() + + +def test_incomplete_shadow_table_resolution_fails_closed( + stats: MagicMock, +) -> None: + """Missing shadow metadata must abort before the destructive pass.""" + with ( + patch.object( + version_history_retention, + "_resolve_shadow_tables", + side_effect=RuntimeError("missing shadow"), + ), + pytest.raises(RuntimeError, match="missing shadow"), + ): + version_history_retention._prune_old_versions_impl(retention_days=30) + stats.incr.assert_not_called() + + +def test_resolve_shadow_tables_rejects_partial_registry() -> None: + """One missing versioned mapper makes the complete registry unsafe.""" + resolved_table = MagicMock() + + def resolve_version_class(model: type[object]) -> MagicMock: + if model.__name__ == "TableColumn": + raise ClassNotVersioned(model) + version_model = MagicMock() + version_model.__table__ = resolved_table + return version_model + + with patch("sqlalchemy_continuum.version_class", side_effect=resolve_version_class): + with pytest.raises(RuntimeError, match="TableColumn"): + version_history_retention._resolve_shadow_tables(MagicMock()) + + +def test_serialization_failure_then_success_increments_retried_once( + stats: MagicMock, +) -> None: + """A single ``OperationalError`` on attempt 1 should: + * fire ``.retried`` once (one retry happened), + * sleep for ``_RETRY_BACKOFF_BASE_SECONDS`` (patched away in tests), + * succeed on attempt 2 with ``stats["retried"] == 1``, + * fire ``.pruned_transactions`` gauge with the success count. + + The contract on ``.retried`` is "fires per retry attempt observed" + (per-attempt, not per-session). This test pins the per-attempt shape so + a future refactor doesn't silently change the metric semantics.""" + pass_fn = MagicMock( + side_effect=[ + OperationalError("SELECT 1", {}, Exception("could not serialize access")), + {"pruned_transactions": 7, "cutoff": "2026-01-01T00:00:00"}, + ] + ) + tables = version_history_retention.ShadowTables( + parent=[MagicMock()], child=[MagicMock()], m2m=None, transaction=MagicMock() + ) + with ( + patch.object( + version_history_retention, "_resolve_shadow_tables", return_value=tables + ), + patch.object(version_history_retention, "_run_prune_pass", pass_fn), + patch.object(version_history_retention.time, "sleep"), + ): + result = version_history_retention._prune_old_versions_impl(retention_days=30) + + assert result["retried"] == 1 + assert result["pruned_transactions"] == 7 + incr_calls = [call.args[0] for call in stats.incr.call_args_list] + assert incr_calls == ["superset.versioning.retention.retried"], ( + f"Expected exactly one .retried emission; got {incr_calls}" + ) + stats.gauge.assert_called_once_with( + "superset.versioning.retention.pruned_transactions", 7 + ) + + +def test_all_attempts_fail_reraises_after_max_retries(stats: MagicMock) -> None: + """When every attempt raises ``OperationalError``, the task re-raises + after ``_MAX_RETRY_ATTEMPTS`` so the outer Celery wrapper logs it. + The retry counter fires once per attempt that hit the exception.""" + exc = OperationalError("SELECT 1", {}, Exception("conflict")) Review Comment: **Suggestion:** Add a concrete type annotation to this local exception variable to keep new code fully typed. [custom_rule] **Severity Level:** Minor ๐งน <details> <summary><b>Why it matters? โญ </b></summary> This local exception object is newly introduced and unannotated. The custom rule explicitly calls for type hints on relevant variables in new Python code. </details> <details> <summary><b>Rule source ๐ </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=6c8c377d556f4182aa32fe1ebf70061a&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=6c8c377d556f4182aa32fe1ebf70061a&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent ๐ค </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/tasks/test_version_history_retention.py **Line:** 159:159 **Comment:** *Custom Rule: Add a concrete type annotation to this local exception variable to keep new code fully typed. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=7da4a321497c663e2ce13cf5388f3261700d1e5cc7692d2221accabf641b8527&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41075&comment_hash=7da4a321497c663e2ce13cf5388f3261700d1e5cc7692d2221accabf641b8527&reaction=dislike'>๐</a> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
