codeant-ai-for-open-source[bot] commented on code in PR #45559:
URL: https://github.com/apache/superset/pull/45559#discussion_r4334507912
##########
superset/connectors/sqla/partition_mapping.py:
##########
@@ -1844,31 +1883,68 @@ def evaluate_transform(
if not all(
literal_round_trips(value, dialect, database.backend) for value in
distinct
):
+ _count("declined.value_does_not_round_trip")
logger.warning(
"A filter value is not written the same way by the driver and the "
"database, so the partition transform is not probed at it; this "
"query will not prune"
)
return None
- cache_key = _probe_cache_key(database, catalog, schema, transform,
distinct)
- cached = _cache_get(cache_key)
+ cached = _cached_probe(database, catalog, schema, transform, distinct,
errors)
if cached is None:
- cached = _run_probe(
- database, catalog, schema, transform, distinct, errors=errors
- )
- if cached is None:
- # Deliberately not cached: a transient engine blip would otherwise
- # keep the dataset pruning-free for the whole cache timeout.
- return None
- _cache_set(cache_key, cached)
+ return None
evaluated = dict(
zip((_hashable(value) for value in distinct), cached, strict=False)
)
return [evaluated[_hashable(value)] for value in values]
+def _cached_probe(
+ database: Database,
+ catalog: str | None,
+ schema: str | None,
+ transform: str,
+ distinct: list[Any],
+ errors: list[str] | None,
+) -> list[Any] | None:
+ """
+ `_run_probe` behind the probe cache, remembering failures as well.
+
+ A failure is remembered only briefly. Not at all, and a transform the
+ engine always rejects costs a warehouse round trip on every chart load for
+ as long as it is configured; for the full cache timeout, and one transient
+ engine blip keeps the dataset pruning-free for a day. The engine's own
+ messages are kept with it so the preview can still say why.
+ """
+ cache_key = _probe_cache_key(database, catalog, schema, transform,
distinct)
+ cached = _cache_get(cache_key)
+ if cached is not None:
+ _count("cache_hit")
+ return cached
+
+ failure = _cache_get(_failure_cache_key(cache_key))
+ if failure is not None:
+ _count("failure_cache_hit")
+ if errors is not None:
+ errors.extend(failure)
+ return None
+
+ _count("cache_miss")
+ probe_errors: list[str] = []
+ cached = _run_probe(
+ database, catalog, schema, transform, distinct, errors=probe_errors
Review Comment:
**Suggestion:** Concurrent requests can all miss this key and call
`_run_probe` before any result is stored, causing a cold probe to fan out to
the warehouse.
**Assessment:** ๐ `Major` ยท ๐ `Occurrence: Sometimes` ยท ๐ท๏ธ `Race condition`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill)
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=5381383ccb3b4eeeb296c030d3db6096&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
[](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=5381383ccb3b4eeeb296c030d3db6096&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/connectors/sqla/partition_mapping.py
**Line:** 1922:1937
**Comment:**
*Race Condition: Concurrent requests can all miss this key and call
`_run_probe` before any result is stored, causing a cold probe to fan out to
the warehouse.
Validate the correctness of the flagged issue. If correct, How can I resolve
this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask
user if the user wants to fix the rest of the comments as well. if said yes,
then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a
href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F45559&comment_hash=2056a68fac0be1fbc3cce731267e355a280927d0ad2ba96c5636a580c354704a&reaction=like'>๐</a>
| <a
href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F45559&comment_hash=2056a68fac0be1fbc3cce731267e355a280927d0ad2ba96c5636a580c354704a&reaction=dislike'>๐</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
