codeant-ai-for-open-source[bot] commented on code in PR #39509: URL: https://github.com/apache/superset/pull/39509#discussion_r3559521599
########## tests/unit_tests/models/test_helpers_offset.py: ########## @@ -0,0 +1,93 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +import ast +from pathlib import Path + +HELPERS_PATH = ( + Path(__file__).resolve().parents[3] / "superset" / "models" / "helpers.py" +) Review Comment: **Suggestion:** Add an explicit type annotation for the module-level path constant so this new variable complies with the projectβs type-hint requirement. [custom_rule] **Severity Level:** Minor π§Ή <details> <summary><b>Why it matters? β </b></summary> The new module-level constant `HELPERS_PATH` is assigned without any type annotation, and it is a clearly annotatable Python variable. This matches the rule requiring type hints on new or modified Python code for relevant variables. </details> <details> <summary><b>Rule source π </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=bf88fca6f43242a6b7d87cee8dc77fd7&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=bf88fca6f43242a6b7d87cee8dc77fd7&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent π€ </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/models/test_helpers_offset.py **Line:** 20:22 **Comment:** *Custom Rule: Add an explicit type annotation for the module-level path constant so this new variable complies with the projectβs type-hint requirement. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=f22e33ec29431b8a873c0f08e5988c2f15d27294f081f5dbea11f5fc4b84d815&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=f22e33ec29431b8a873c0f08e5988c2f15d27294f081f5dbea11f5fc4b84d815&reaction=dislike'>π</a> ########## tests/unit_tests/views/datasource/test_utils.py: ########## @@ -0,0 +1,389 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from unittest.mock import MagicMock, patch + +import pytest + + [email protected] +def fake_datasource_factory(): + """Builds a MagicMock datasource whose db_engine_spec is configurable.""" + + def _build(supports_offset: bool) -> MagicMock: + datasource = MagicMock(name="SqlaTable") + datasource.type = "table" + datasource.id = 1 + datasource.columns = [] + datasource.database.db_engine_spec.supports_offset = supports_offset + return datasource + + return _build + + +def test_get_samples_uses_normal_path_when_engine_supports_offset( + fake_datasource_factory, +): Review Comment: **Suggestion:** Add type hints for the fixture parameter and an explicit return type for this test function. [custom_rule] **Severity Level:** Minor π§Ή <details> <summary><b>Why it matters? β </b></summary> This test function omits both a type annotation for the fixture parameter and a return type annotation, which matches the stated Python type-hint rule violation. </details> <details> <summary><b>Rule source π </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=0ef35284c0bd414ba824410358ea944e&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=0ef35284c0bd414ba824410358ea944e&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent π€ </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/views/datasource/test_utils.py **Line:** 37:39 **Comment:** *Custom Rule: Add type hints for the fixture parameter and an explicit return type for this test function. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=37131e9fd214cf940601bea6784f15c2caed6e373435ea31a9adad1bed51b92d&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=37131e9fd214cf940601bea6784f15c2caed6e373435ea31a9adad1bed51b92d&reaction=dislike'>π</a> ########## tests/unit_tests/views/datasource/test_utils.py: ########## @@ -0,0 +1,389 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from unittest.mock import MagicMock, patch + +import pytest + + [email protected] +def fake_datasource_factory(): Review Comment: **Suggestion:** Add a return type annotation to this fixture function to satisfy the required type-hint rule. [custom_rule] **Severity Level:** Minor π§Ή <details> <summary><b>Why it matters? β </b></summary> The fixture function is newly added and has no return type annotation, which violates the type-hint rule for Python functions that can be annotated. </details> <details> <summary><b>Rule source π </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=cbbc5cfb078b4f048e2c1e7ad1099529&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=cbbc5cfb078b4f048e2c1e7ad1099529&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent π€ </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/views/datasource/test_utils.py **Line:** 23:23 **Comment:** *Custom Rule: Add a return type annotation to this fixture function to satisfy the required type-hint rule. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=61889d7fffe0a935209a702886a35ef86ac5f9958e78e0063a3a3e02e0aa3815&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=61889d7fffe0a935209a702886a35ef86ac5f9958e78e0063a3a3e02e0aa3815&reaction=dislike'>π</a> ########## tests/unit_tests/views/datasource/test_utils.py: ########## @@ -0,0 +1,389 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from unittest.mock import MagicMock, patch + +import pytest + + [email protected] +def fake_datasource_factory(): + """Builds a MagicMock datasource whose db_engine_spec is configurable.""" + + def _build(supports_offset: bool) -> MagicMock: + datasource = MagicMock(name="SqlaTable") + datasource.type = "table" + datasource.id = 1 + datasource.columns = [] + datasource.database.db_engine_spec.supports_offset = supports_offset + return datasource + + return _build + + +def test_get_samples_uses_normal_path_when_engine_supports_offset( + fake_datasource_factory, +): + """ + Engines with supports_offset=True continue to use the existing + QueryContext/get_payload path. No cursor-method calls. + """ + from superset.views.datasource import utils + + datasource = fake_datasource_factory(supports_offset=True) + datasource.database.db_engine_spec.fetch_data_with_cursor = MagicMock() + + with ( + patch.object( + utils, "DatasourceDAO", MagicMock(get_datasource=lambda **kw: datasource) + ), + patch.object(utils, "QueryContextFactory") as qcf, + ): + samples_ctx = MagicMock() + samples_ctx.get_payload.return_value = { + "queries": [ + { + "data": [{"a": 1}], + "colnames": ["a"], + "coltypes": [], + "status": "success", + } + ] + } + count_ctx = MagicMock() + count_ctx.get_payload.return_value = { + "queries": [{"data": [{"COUNT(*)": 42}], "status": "success"}] + } + qcf.return_value.create.side_effect = [samples_ctx, count_ctx] + + result = utils.get_samples( + datasource_type="table", + datasource_id=1, + page=2, + per_page=50, + ) + + assert result["data"] == [{"a": 1}] + assert result["page"] == 2 + assert result["per_page"] == 50 + assert result["total_count"] == 42 + datasource.database.db_engine_spec.fetch_data_with_cursor.assert_not_called() + + +def test_get_samples_normal_path_cleans_count_cache_when_sample_data_failed( + fake_datasource_factory, +): Review Comment: **Suggestion:** Annotate the injected fixture argument and add an explicit return type annotation to this function. [custom_rule] **Severity Level:** Minor π§Ή <details> <summary><b>Why it matters? β </b></summary> The function signature lacks annotations for the injected fixture argument and the return type, so it is a real omission under the required type-hint rule. </details> <details> <summary><b>Rule source π </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=ceb841e44a2142c388fbd7cf7839f27b&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=ceb841e44a2142c388fbd7cf7839f27b&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent π€ </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/views/datasource/test_utils.py **Line:** 86:88 **Comment:** *Custom Rule: Annotate the injected fixture argument and add an explicit return type annotation to this function. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=7b243394b430d7a32229da87045a85b3ba9278a0b3ef732fdf337ac96e020a2b&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=7b243394b430d7a32229da87045a85b3ba9278a0b3ef732fdf337ac96e020a2b&reaction=dislike'>π</a> ########## tests/unit_tests/views/datasource/test_utils.py: ########## @@ -0,0 +1,389 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from unittest.mock import MagicMock, patch + +import pytest + + [email protected] +def fake_datasource_factory(): + """Builds a MagicMock datasource whose db_engine_spec is configurable.""" + + def _build(supports_offset: bool) -> MagicMock: + datasource = MagicMock(name="SqlaTable") + datasource.type = "table" + datasource.id = 1 + datasource.columns = [] + datasource.database.db_engine_spec.supports_offset = supports_offset + return datasource + + return _build + + +def test_get_samples_uses_normal_path_when_engine_supports_offset( + fake_datasource_factory, +): + """ + Engines with supports_offset=True continue to use the existing + QueryContext/get_payload path. No cursor-method calls. + """ + from superset.views.datasource import utils + + datasource = fake_datasource_factory(supports_offset=True) + datasource.database.db_engine_spec.fetch_data_with_cursor = MagicMock() + + with ( + patch.object( + utils, "DatasourceDAO", MagicMock(get_datasource=lambda **kw: datasource) + ), + patch.object(utils, "QueryContextFactory") as qcf, + ): + samples_ctx = MagicMock() + samples_ctx.get_payload.return_value = { + "queries": [ + { + "data": [{"a": 1}], + "colnames": ["a"], + "coltypes": [], + "status": "success", + } + ] + } + count_ctx = MagicMock() + count_ctx.get_payload.return_value = { + "queries": [{"data": [{"COUNT(*)": 42}], "status": "success"}] + } + qcf.return_value.create.side_effect = [samples_ctx, count_ctx] + + result = utils.get_samples( + datasource_type="table", + datasource_id=1, + page=2, + per_page=50, + ) + + assert result["data"] == [{"a": 1}] + assert result["page"] == 2 + assert result["per_page"] == 50 + assert result["total_count"] == 42 + datasource.database.db_engine_spec.fetch_data_with_cursor.assert_not_called() + + +def test_get_samples_normal_path_cleans_count_cache_when_sample_data_failed( + fake_datasource_factory, +): + """ + On the normal (non-cursor) path, a FAILED samples payload must evict the + count-star cache and raise DatasetSamplesFailedError β mirroring the + cursor path's failure handling. + """ + from superset.commands.dataset.exceptions import DatasetSamplesFailedError + from superset.constants import CacheRegion + from superset.views.datasource import utils + + datasource = fake_datasource_factory(supports_offset=True) + + samples_ctx = MagicMock() + samples_ctx.get_payload.return_value = { + "queries": [ + { + "status": "failed", + "error": "backend query failed", + } + ] + } + count_ctx = MagicMock() + count_ctx.get_payload.return_value = { + "queries": [ + { + "data": [{"COUNT(*)": 1}], + "status": "success", + "cache_key": "count-cache-key", + } + ] + } + + with ( + patch.object( + utils, "DatasourceDAO", MagicMock(get_datasource=lambda **kw: datasource) + ), + patch.object(utils, "QueryContextFactory") as qcf, + patch.object(utils, "QueryCacheManager") as cache_mgr, + ): + qcf.return_value.create.side_effect = [samples_ctx, count_ctx] + + with pytest.raises(DatasetSamplesFailedError) as excinfo: + utils.get_samples( + datasource_type="table", + datasource_id=1, + page=1, + per_page=50, + ) + + cache_mgr.delete.assert_called_once_with("count-cache-key", CacheRegion.DATA) + assert "backend query failed" in str(excinfo.value) + + +def test_get_samples_uses_cursor_path_when_engine_disallows_offset( + fake_datasource_factory, +): + """ + When the engine reports supports_offset=False and the requested + page is > 1, get_samples delegates to fetch_data_with_cursor with SQL + compiled statically (without executing the normal samples payload). + """ + from superset.views.datasource import utils + + datasource = fake_datasource_factory(supports_offset=False) + datasource.database.db_engine_spec.fetch_data_with_cursor.return_value = ( + [[99]], + ["a"], + ) + + samples_ctx = MagicMock() + samples_ctx.queries = [MagicMock()] + samples_ctx.datasource.get_query_str.return_value = "SELECT a FROM idx LIMIT 50" + count_ctx = MagicMock() + count_ctx.get_payload.return_value = { + "queries": [{"data": [{"COUNT(*)": 200}], "status": "success"}] + } + + with ( + patch.object( + utils, "DatasourceDAO", MagicMock(get_datasource=lambda **kw: datasource) + ), + patch.object(utils, "QueryContextFactory") as qcf, + ): + qcf.return_value.create.side_effect = [samples_ctx, count_ctx] + + result = utils.get_samples( + datasource_type="table", + datasource_id=1, + page=3, + per_page=50, + ) + + datasource.database.db_engine_spec.fetch_data_with_cursor.assert_called_once() + kwargs = datasource.database.db_engine_spec.fetch_data_with_cursor.call_args.kwargs + assert kwargs["page_index"] == 2 + assert kwargs["page_size"] == 50 + # The cursor path compiles SQL statically via get_query_str, without + # executing the normal samples payload; the engine spec is responsible + # for any sanitation (strip ``;``/``LIMIT``). + assert kwargs["sql"] == "SELECT a FROM idx LIMIT 50" + samples_ctx.get_payload.assert_not_called() + + assert result["data"] == [{"a": 99}] + assert result["colnames"] == ["a"] + assert result["page"] == 3 + assert result["per_page"] == 50 + assert result["total_count"] == 200 + + +def test_get_samples_cursor_path_infers_coltypes_from_cursor_rows( + fake_datasource_factory, +): + """ + coltypes on the cursor path are inferred from the returned rows via + extract_dataframe_dtypes β the same function the normal (non-cursor) + path uses to type page 1 β so SamplesPane's useGridColumns() picks up + type-based cell renderers on page 2+ without an extra query. + """ + from superset.utils.core import GenericDataType + from superset.views.datasource import utils + + datasource = fake_datasource_factory(supports_offset=False) + datasource.database.db_engine_spec.fetch_data_with_cursor.return_value = ( + [["x", 1]], + ["a", "b"], + ) + + samples_ctx = MagicMock() + samples_ctx.queries = [MagicMock()] + samples_ctx.datasource.get_query_str.return_value = "SELECT a, b FROM idx LIMIT 50" + count_ctx = MagicMock() + count_ctx.get_payload.return_value = { + "queries": [{"data": [{"COUNT(*)": 2000}], "status": "success"}] + } + + with ( + patch.object( + utils, "DatasourceDAO", MagicMock(get_datasource=lambda **kw: datasource) + ), + patch.object(utils, "QueryContextFactory") as qcf, + ): + qcf.return_value.create.side_effect = [samples_ctx, count_ctx] + + result = utils.get_samples( + datasource_type="table", + datasource_id=1, + page=2, + per_page=50, + ) + + assert result["coltypes"] == [GenericDataType.STRING, GenericDataType.NUMERIC] + assert result["colnames"] == ["a", "b"] + assert result["data"] == [{"a": "x", "b": 1}] + + +def test_get_samples_cursor_path_cleans_count_cache_on_failure( + fake_datasource_factory, +): + """ + Issue 2: if fetch_data_with_cursor raises, the count-star cache must be + evicted (mirroring the normal FAILED path) and the error re-raised as + DatasetSamplesFailedError. + """ + from superset.commands.dataset.exceptions import DatasetSamplesFailedError + from superset.constants import CacheRegion + from superset.views.datasource import utils + + datasource = fake_datasource_factory(supports_offset=False) + datasource.database.db_engine_spec.fetch_data_with_cursor.side_effect = ( + RuntimeError("boom: internal es stack trace details") + ) + + samples_ctx = MagicMock() + samples_ctx.queries = [MagicMock()] + samples_ctx.datasource.get_query_str.return_value = "SELECT a FROM idx LIMIT 50" + count_ctx = MagicMock() + count_ctx.get_payload.return_value = { + "queries": [ + { + "data": [{"COUNT(*)": 200}], + "status": "success", + "cache_key": "count-cache-key", + } + ] + } + + with ( + patch.object( + utils, "DatasourceDAO", MagicMock(get_datasource=lambda **kw: datasource) + ), + patch.object(utils, "QueryContextFactory") as qcf, + patch.object(utils, "QueryCacheManager") as cache_mgr, + ): + qcf.return_value.create.side_effect = [samples_ctx, count_ctx] + + with pytest.raises(DatasetSamplesFailedError) as excinfo: + utils.get_samples( + datasource_type="table", + datasource_id=1, + page=3, + per_page=50, + ) + + cache_mgr.delete.assert_called_once_with("count-cache-key", CacheRegion.DATA) + # Backend-internal error text must not leak into the user-facing message; + # only the original exception (chained via ``from exc``) retains the detail. + assert "internal es stack trace details" not in str(excinfo.value) + assert isinstance(excinfo.value.__cause__, RuntimeError) + + +def test_get_samples_cursor_path_raises_when_compiled_sql_is_empty( + fake_datasource_factory, +): + """ + If get_query_str compiles to an empty string, the cursor path has + nothing to submit. Fail fast with a descriptive error and evict the + count cache, instead of handing an empty statement to the engine driver. + """ + from superset.commands.dataset.exceptions import DatasetSamplesFailedError + from superset.constants import CacheRegion + from superset.views.datasource import utils + + datasource = fake_datasource_factory(supports_offset=False) + + samples_ctx = MagicMock() + samples_ctx.queries = [MagicMock()] + samples_ctx.datasource.get_query_str.return_value = "" + count_ctx = MagicMock() + count_ctx.get_payload.return_value = { + "queries": [ + { + "data": [{"COUNT(*)": 200}], + "status": "success", + "cache_key": "count-cache-key", + } + ] + } + + with ( + patch.object( + utils, "DatasourceDAO", MagicMock(get_datasource=lambda **kw: datasource) + ), + patch.object(utils, "QueryContextFactory") as qcf, + patch.object(utils, "QueryCacheManager") as cache_mgr, + ): + qcf.return_value.create.side_effect = [samples_ctx, count_ctx] + + with pytest.raises(DatasetSamplesFailedError): + utils.get_samples( + datasource_type="table", + datasource_id=1, + page=2, + per_page=50, + ) + + cache_mgr.delete.assert_called_once_with("count-cache-key", CacheRegion.DATA) + datasource.database.db_engine_spec.fetch_data_with_cursor.assert_not_called() + + +def test_get_samples_cursor_path_unused_for_page_one(fake_datasource_factory): Review Comment: **Suggestion:** Provide a type hint for the fixture argument and annotate the function as returning no value. [custom_rule] **Severity Level:** Minor π§Ή <details> <summary><b>Why it matters? β </b></summary> The test function has an unannotated fixture argument and no explicit return type, which is a direct match for the type-hint requirement. </details> <details> <summary><b>Rule source π </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=1142097dfde64434ade2bbe9f55eabfb&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=1142097dfde64434ade2bbe9f55eabfb&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent π€ </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/views/datasource/test_utils.py **Line:** 347:347 **Comment:** *Custom Rule: Provide a type hint for the fixture argument and annotate the function as returning no value. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=378285f31d2e8bc775c4a55f48950a8c8b41be28159f23269d37811b24fe6846&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=378285f31d2e8bc775c4a55f48950a8c8b41be28159f23269d37811b24fe6846&reaction=dislike'>π</a> ########## tests/unit_tests/views/datasource/test_utils.py: ########## @@ -0,0 +1,389 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from unittest.mock import MagicMock, patch + +import pytest + + [email protected] +def fake_datasource_factory(): + """Builds a MagicMock datasource whose db_engine_spec is configurable.""" + + def _build(supports_offset: bool) -> MagicMock: + datasource = MagicMock(name="SqlaTable") + datasource.type = "table" + datasource.id = 1 + datasource.columns = [] + datasource.database.db_engine_spec.supports_offset = supports_offset + return datasource + + return _build + + +def test_get_samples_uses_normal_path_when_engine_supports_offset( + fake_datasource_factory, +): + """ + Engines with supports_offset=True continue to use the existing + QueryContext/get_payload path. No cursor-method calls. + """ + from superset.views.datasource import utils + + datasource = fake_datasource_factory(supports_offset=True) + datasource.database.db_engine_spec.fetch_data_with_cursor = MagicMock() + + with ( + patch.object( + utils, "DatasourceDAO", MagicMock(get_datasource=lambda **kw: datasource) + ), + patch.object(utils, "QueryContextFactory") as qcf, + ): + samples_ctx = MagicMock() + samples_ctx.get_payload.return_value = { + "queries": [ + { + "data": [{"a": 1}], + "colnames": ["a"], + "coltypes": [], + "status": "success", + } + ] + } + count_ctx = MagicMock() + count_ctx.get_payload.return_value = { + "queries": [{"data": [{"COUNT(*)": 42}], "status": "success"}] + } + qcf.return_value.create.side_effect = [samples_ctx, count_ctx] + + result = utils.get_samples( + datasource_type="table", + datasource_id=1, + page=2, + per_page=50, + ) + + assert result["data"] == [{"a": 1}] + assert result["page"] == 2 + assert result["per_page"] == 50 + assert result["total_count"] == 42 + datasource.database.db_engine_spec.fetch_data_with_cursor.assert_not_called() + + +def test_get_samples_normal_path_cleans_count_cache_when_sample_data_failed( + fake_datasource_factory, +): + """ + On the normal (non-cursor) path, a FAILED samples payload must evict the + count-star cache and raise DatasetSamplesFailedError β mirroring the + cursor path's failure handling. + """ + from superset.commands.dataset.exceptions import DatasetSamplesFailedError + from superset.constants import CacheRegion + from superset.views.datasource import utils + + datasource = fake_datasource_factory(supports_offset=True) + + samples_ctx = MagicMock() + samples_ctx.get_payload.return_value = { + "queries": [ + { + "status": "failed", + "error": "backend query failed", + } + ] + } + count_ctx = MagicMock() + count_ctx.get_payload.return_value = { + "queries": [ + { + "data": [{"COUNT(*)": 1}], + "status": "success", + "cache_key": "count-cache-key", + } + ] + } + + with ( + patch.object( + utils, "DatasourceDAO", MagicMock(get_datasource=lambda **kw: datasource) + ), + patch.object(utils, "QueryContextFactory") as qcf, + patch.object(utils, "QueryCacheManager") as cache_mgr, + ): + qcf.return_value.create.side_effect = [samples_ctx, count_ctx] + + with pytest.raises(DatasetSamplesFailedError) as excinfo: + utils.get_samples( + datasource_type="table", + datasource_id=1, + page=1, + per_page=50, + ) + + cache_mgr.delete.assert_called_once_with("count-cache-key", CacheRegion.DATA) + assert "backend query failed" in str(excinfo.value) + + +def test_get_samples_uses_cursor_path_when_engine_disallows_offset( + fake_datasource_factory, +): Review Comment: **Suggestion:** Add missing type annotations for the function parameter and the return type on this test definition. [custom_rule] **Severity Level:** Minor π§Ή <details> <summary><b>Why it matters? β </b></summary> This test definition also omits type hints for its parameter and return value, so the rule violation is present in the existing code. </details> <details> <summary><b>Rule source π </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=57f8cddce4c946a4af18c03b1ac990c3&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=57f8cddce4c946a4af18c03b1ac990c3&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent π€ </b></summary> ```mdx This is a comment left during a code review. **Path:** tests/unit_tests/views/datasource/test_utils.py **Line:** 141:143 **Comment:** *Custom Rule: Add missing type annotations for the function parameter and the return type on this test definition. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=6ac572df966444a70e91e0d357cd661c69db04c61e0154355d3ad55cbcf81a8b&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=6ac572df966444a70e91e0d357cd661c69db04c61e0154355d3ad55cbcf81a8b&reaction=dislike'>π</a> ########## superset/db_engine_specs/base.py: ########## @@ -500,6 +500,12 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods allows_sql_comments = True allows_escaped_colons = True + # Whether the engine supports OFFSET in SQL queries. Defaults to True; + # engines like Elasticsearch SQL that do not support OFFSET set this to + # False and are expected to implement `fetch_data_with_cursor` for + # pagination via another mechanism (e.g. Elasticsearch's cursor API). + supports_offset = True Review Comment: **Suggestion:** Add an explicit type annotation to this new class attribute so it complies with the type-hint requirement for newly introduced variables. [custom_rule] **Severity Level:** Minor π§Ή <details> <summary><b>Why it matters? β </b></summary> The new class attribute is a Python variable that can be annotated, but it is defined without a type hint. This matches the type-hint requirement for newly introduced or modified Python code. </details> <details> <summary><b>Rule source π </b></summary> .cursor/rules/dev-standard.mdc (line 28) </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=25ca602c80ae47eeb601042f85fda4e3&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=25ca602c80ae47eeb601042f85fda4e3&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent π€ </b></summary> ```mdx This is a comment left during a code review. **Path:** superset/db_engine_specs/base.py **Line:** 507:507 **Comment:** *Custom Rule: Add an explicit type annotation to this new class attribute so it complies with the type-hint requirement for newly introduced variables. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=3f107cd267b4deebc8d15926363405b51d9109c59dfe9ecc2a063672d32bd526&reaction=like'>π</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F39509&comment_hash=3f107cd267b4deebc8d15926363405b51d9109c59dfe9ecc2a063672d32bd526&reaction=dislike'>π</a> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
