codeant-ai-for-open-source[bot] commented on code in PR #40180:
URL: https://github.com/apache/superset/pull/40180#discussion_r3252915070
##########
superset/models/helpers.py:
##########
@@ -2156,21 +2156,50 @@ def filter_values_handler( # pylint:
disable=too-many-arguments # noqa: C901
if values is None:
return None
- def handle_single_value(value: Optional[FilterValue]) ->
Optional[FilterValue]:
- if operator == utils.FilterOperator.TEMPORAL_RANGE:
- return value
+ temporal_comparison_operators = {
+ utils.FilterOperator.EQUALS,
+ utils.FilterOperator.NOT_EQUALS,
+ utils.FilterOperator.IN,
+ utils.FilterOperator.NOT_IN,
+ utils.FilterOperator.GREATER_THAN,
+ utils.FilterOperator.LESS_THAN,
+ utils.FilterOperator.GREATER_THAN_OR_EQUALS,
+ utils.FilterOperator.LESS_THAN_OR_EQUALS,
+ }
+
+ def handle_temporal_value(value: FilterValue) -> FilterValue |
ColumnElement:
if (
- isinstance(value, (float, int))
- and target_generic_type == utils.GenericDataType.TEMPORAL
- and target_native_type is not None
- and db_engine_spec is not None
+ operator not in temporal_comparison_operators
+ or target_generic_type != utils.GenericDataType.TEMPORAL
+ or target_native_type is None
+ or db_engine_spec is None
):
- value = db_engine_spec.convert_dttm(
- target_type=target_native_type,
- dttm=datetime.utcfromtimestamp(value / 1000),
- db_extra=db_extra,
+ return value
+
+ if isinstance(value, (float, int)) and not isinstance(value, bool):
+ epoch_ms: float = value
+ elif isinstance(value, str) and re.fullmatch(r"\d+", value):
+ epoch_ms = int(value)
Review Comment:
**Suggestion:** Temporal epoch strings are only coerced when they match
`\d+`, so negative epoch-millisecond values (valid for pre-1970 timestamps) are
skipped and left as raw strings; that causes temporal filters on historical
dates to compile incorrectly and fail on engines that require typed temporal
literals. Update the string check to accept signed integers before conversion.
[incorrect condition logic]
<details>
<summary><b>Severity Level:</b> Major ⚠️</summary>
```mdx
- ❌ ExploreMixin.get_sqla_query misbuilds filters for pre-1970 timestamps.
- ❌ BigQueryEngineSpec receives raw string instead of temporal literal.
- ⚠️ Drill-to-detail on historical rows may error or mis-filter.
```
</details>
<details>
<summary><b>Steps of Reproduction ✅ </b></summary>
```mdx
1. Create a test similar to
`test_temporal_epoch_string_filter_is_coerced_for_bigquery` in
`tests/unit_tests/models/helpers_test.py:8-25`, but call
`ExploreMixin.filter_values_handler` with `values=\"-1778630400000\"` (a
negative
epoch-millisecond string), `operator=FilterOperator.EQUALS`,
`target_generic_type=GenericDataType.TEMPORAL`,
`target_native_type=\"DATE\"`, and
`db_engine_spec=BigQueryEngineSpec`.
2. The call enters `ExploreMixin.filter_values_handler` in
`superset/models/helpers.py:2145-2155`, passes the non-None `values` through
to the inner
`handle_single_value` function defined at
`superset/models/helpers.py:81-109`.
3. Inside `handle_single_value`, `value` is a string, so it is stripped and
then passed to
`handle_temporal_value` at `superset/models/helpers.py:51-79`; in that
function, the `elif
isinstance(value, str) and re.fullmatch(r\"\\d+\", value):` condition at
line 2181 fails
for `\"-1778630400000\"` because the regex `\\d+` does not accept the
leading `-`, so
`handle_temporal_value` returns the original string instead of converting it
to a datetime
and then to an engine-specific temporal literal.
4. Back in `handle_single_value`, the unmodified string `\"-1778630400000\"`
becomes `eq`
and is later used in the comparison branch that calls
`db_engine_spec.handle_comparison_filter(sqla_col, op, eq)` at
`superset/models/helpers.py:3238-3254`, so BigQueryEngineSpec receives a raw
string
instead of a `ColumnElement` produced via `convert_dttm`; when this
propagates into an
actual query built by `get_sqla_query` (helpers.py:2689-2768), the WHERE
clause compares a
temporal column to a bare string literal, which for engines like BigQuery
that require
typed temporal literals will yield an invalid or mis-typed filter for
historical
(pre-1970) timestamps.
```
</details>
[Fix in
Cursor](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt=This%20is%20a%20comment%20left%20during%20a%20code%20review.%0A%0A%2A%2APath%3A%2A%2A%20superset%2Fmodels%2Fhelpers.py%0A%2A%2ALine%3A%2A%2A%202181%3A2182%0A%2A%2AComment%3A%2A%2A%0A%09%2AIncorrect%20Condition%20Logic%3A%20Temporal%20epoch%20strings%20are%20only%20coerced%20when%20they%20match%20%60%5Cd%2B%60%2C%20so%20negative%20epoch-millisecond%20values%20%28valid%20for%20pre-1970%20timestamps%29%20are%20skipped%20and%20left%20as%20raw%20strings%3B%20that%20causes%20temporal%20filters%20on%20historical%20dates%20to%20compile%20incorrectly%20and%20fail%20on%20engines%20that%20require%20typed%20temporal%20literals.%20Update%20the%20string%20check%20to%20accept%20signed%20integers%20before%20conversion.%0A%0AValidate%20the%20correctness%20of%20the%20flagged%20issue.%20If%20correct%2C%20How%20can%20I%20resolve%20this%3F%20If%20you%20propose%20a%20fix%2C%20implement%20it%20and%20please%20make%20it%20concise.%0AOnce
%20fix%20is%20implemented%2C%20also%20check%20other%20comments%20on%20the%20same%20PR%2C%20and%20ask%20user%20if%20the%20user%20wants%20to%20fix%20the%20rest%20of%20the%20comments%20as%20well.%20if%20said%20yes%2C%20then%20fetch%20all%20the%20comments%20validate%20the%20correctness%20and%20implement%20a%20minimal%20fix%0A)
| [Fix in VSCode
Claude](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt=This%20is%20a%20comment%20left%20during%20a%20code%20review.%0A%0A%2A%2APath%3A%2A%2A%20superset%2Fmodels%2Fhelpers.py%0A%2A%2ALine%3A%2A%2A%202181%3A2182%0A%2A%2AComment%3A%2A%2A%0A%09%2AIncorrect%20Condition%20Logic%3A%20Temporal%20epoch%20strings%20are%20only%20coerced%20when%20they%20match%20%60%5Cd%2B%60%2C%20so%20negative%20epoch-millisecond%20values%20%28valid%20for%20pre-1970%20timestamps%29%20are%20skipped%20and%20left%20as%20raw%20strings%3B%20that%20causes%20temporal%20filters%20on%20historical%20dates%20to%20compile%20incorrectly%20and%20fail%20on%20engines%20that%20re
quire%20typed%20temporal%20literals.%20Update%20the%20string%20check%20to%20accept%20signed%20integers%20before%20conversion.%0A%0AValidate%20the%20correctness%20of%20the%20flagged%20issue.%20If%20correct%2C%20How%20can%20I%20resolve%20this%3F%20If%20you%20propose%20a%20fix%2C%20implement%20it%20and%20please%20make%20it%20concise.%0AOnce%20fix%20is%20implemented%2C%20also%20check%20other%20comments%20on%20the%20same%20PR%2C%20and%20ask%20user%20if%20the%20user%20wants%20to%20fix%20the%20rest%20of%20the%20comments%20as%20well.%20if%20said%20yes%2C%20then%20fetch%20all%20the%20comments%20validate%20the%20correctness%20and%20implement%20a%20minimal%20fix%0A)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/models/helpers.py
**Line:** 2181:2182
**Comment:**
*Incorrect Condition Logic: Temporal epoch strings are only coerced
when they match `\d+`, so negative epoch-millisecond values (valid for pre-1970
timestamps) are skipped and left as raw strings; that causes temporal filters
on historical dates to compile incorrectly and fail on engines that require
typed temporal literals. Update the string check to accept signed integers
before conversion.
Validate the correctness of the flagged issue. If correct, How can I resolve
this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask
user if the user wants to fix the rest of the comments as well. if said yes,
then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a
href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40180&comment_hash=f3c2a98f2d2acdbd753678a3e7854056892223c000898bdf195918f1ec5a90d2&reaction=like'>👍</a>
| <a
href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40180&comment_hash=f3c2a98f2d2acdbd753678a3e7854056892223c000898bdf195918f1ec5a90d2&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
