codeant-ai-for-open-source[bot] commented on code in PR #40180:
URL: https://github.com/apache/superset/pull/40180#discussion_r3555442882
##########
superset/models/helpers.py:
##########
@@ -2637,21 +2637,50 @@ def filter_values_handler( # pylint:
disable=too-many-arguments # noqa: C901
if values is None:
return None
- def handle_single_value(value: Optional[FilterValue]) ->
Optional[FilterValue]:
- if operator == utils.FilterOperator.TEMPORAL_RANGE:
- return value
+ temporal_comparison_operators: set[utils.FilterOperator] = {
+ utils.FilterOperator.EQUALS,
+ utils.FilterOperator.NOT_EQUALS,
+ utils.FilterOperator.IN,
+ utils.FilterOperator.NOT_IN,
+ utils.FilterOperator.GREATER_THAN,
+ utils.FilterOperator.LESS_THAN,
+ utils.FilterOperator.GREATER_THAN_OR_EQUALS,
+ utils.FilterOperator.LESS_THAN_OR_EQUALS,
+ }
+
+ def handle_temporal_value(value: FilterValue) -> FilterValue |
ColumnElement:
if (
- isinstance(value, (float, int))
- and target_generic_type == utils.GenericDataType.TEMPORAL
- and target_native_type is not None
- and db_engine_spec is not None
+ operator not in temporal_comparison_operators
+ or target_generic_type != utils.GenericDataType.TEMPORAL
+ or target_native_type is None
+ or db_engine_spec is None
):
- value = db_engine_spec.convert_dttm(
- target_type=target_native_type,
- dttm=datetime.utcfromtimestamp(value / 1000),
- db_extra=db_extra,
+ return value
+
+ if isinstance(value, (float, int)) and not isinstance(value, bool):
+ epoch_ms: float = value
+ elif isinstance(value, str) and re.fullmatch(r"[+-]?\d+", value):
+ epoch_ms = int(value)
+ else:
+ return value
+
+ try:
+ dttm = datetime.fromtimestamp(epoch_ms / 1000,
tz=timezone.utc).replace(
+ tzinfo=None
)
- value = literal_column(value)
+ except (OverflowError, OSError, ValueError):
+ return value
+
+ temporal_sql = db_engine_spec.convert_dttm(
+ target_type=target_native_type,
+ dttm=dttm,
+ db_extra=db_extra,
+ )
Review Comment:
**Suggestion:** The new temporal coercion path calls `convert_dttm` with
`db_extra`, but this function is currently invoked without passing database
extras, so `db_extra` is effectively always `None`. Engines that depend on
extras to format temporal literals correctly can emit the wrong SQL expression
and break temporal drill/cross-filter queries. Thread the dataset/database
extras from the caller into `filter_values_handler` so this conversion has the
same engine context as other datetime literal generation paths. [api mismatch]
<details>
<summary><b>Severity Level:</b> Major ⚠️</summary>
```mdx
- ⚠️ Temporal drill filters ignore extras for version-sensitive engines.
- ⚠️ Cross-filters may use less accurate datetime SQL.
- ⚠️ Inconsistent temporal literals vs `dttm_sql_literal` path using extras.
```
</details>
<details>
<summary><b>Steps of Reproduction ✅ </b></summary>
```mdx
1. Create a database using an engine spec whose `convert_dttm` reads
`db_extra`, for
example `ElasticsearchEngineSpec.convert_dttm` in
`superset/db_engine_specs/elasticsearch.py:12-38`, which accesses
`db_extra.get("version")` to decide between `DATETIME_PARSE` and `CAST`.
2. Configure that database’s `extra` JSON to include a `version` field (e.g.
`"version":
"7.8.0"`), so the engine spec has additional context; this is surfaced to
SQLA models via
`SqlaTable.db_extra` in `superset/connectors/sqla/models.py:11-13`.
3. Build a chart backed by this database and apply a cross-filter or
drill-to-detail on a
temporal column so the API constructs a filter object; this flows into the
ExploreMixin
query builder loop in `superset/models/helpers.py:50-88`, which calls `eq =
self.filter_values_handler(..., db_engine_spec=db_engine_spec)` without
passing
`db_extra`.
4. Inside `ExploreMixin.filter_values_handler` in
`superset/models/helpers.py:26-80`, the
new `handle_temporal_value` path converts epoch-millisecond values and calls
`db_engine_spec.convert_dttm(target_type=target_native_type, dttm=dttm,
db_extra=db_extra)` at `superset/models/helpers.py:75-78`, but `db_extra` is
always `None`
from this caller; as a result, engines like Elasticsearch cannot see the
configured
version and fall back to less accurate temporal literal SQL (e.g.
timezone-insensitive
`CAST`) for drill/cross-filter queries instead of their extras-aware
formatting.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=904f12a0cbe0495c87412518627baffe&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
[](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=904f12a0cbe0495c87412518627baffe&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/models/helpers.py
**Line:** 2674:2678
**Comment:**
*Api Mismatch: The new temporal coercion path calls `convert_dttm` with
`db_extra`, but this function is currently invoked without passing database
extras, so `db_extra` is effectively always `None`. Engines that depend on
extras to format temporal literals correctly can emit the wrong SQL expression
and break temporal drill/cross-filter queries. Thread the dataset/database
extras from the caller into `filter_values_handler` so this conversion has the
same engine context as other datetime literal generation paths.
Validate the correctness of the flagged issue. If correct, How can I resolve
this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask
user if the user wants to fix the rest of the comments as well. if said yes,
then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a
href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40180&comment_hash=368516fafb4158d2a1e34e9767f34a3f032b52dce7a4400e7accd248581ff9b8&reaction=like'>👍</a>
| <a
href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40180&comment_hash=368516fafb4158d2a1e34e9767f34a3f032b52dce7a4400e7accd248581ff9b8&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
