[
https://issues.apache.org/jira/browse/IGNITE-29102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Werner updated IGNITE-29102:
-----------------------------------
Labels: metrics-to-log (was: from-log-to-metric)
> Add a metric to track B+Tree cursor corruption failures during SQL query
> execution
> ----------------------------------------------------------------------------------
>
> Key: IGNITE-29102
> URL: https://issues.apache.org/jira/browse/IGNITE-29102
> Project: Ignite
> Issue Type: Task
> Reporter: Egor Baranov
> Assignee: Egor Baranov
> Priority: Major
> Labels: metrics-to-log
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *Problem:*
> When a B+Tree cursor encounters a broken invariant (typically an
> AssertionError in AbstractForwardCursor.find() or
> AbstractForwardCursor.fillFromBuffer()), the exception is wrapped into a
> CorruptedTreeException and propagated up the call chain to
> GridMapQueryExecutor. There, it is caught in a generic catch (Throwable e)
> block and logged as "Failed to execute local query" at ERROR/WARN level.
> There is currently *no metric* that tracks how often such corruption-related
> query failures occur. Operators can only detect this problem by parsing log
> files, which is impractical for:
> - Large clusters with high query throughput
> - Intermittent corruption events that are hard to spot in log noise
> - Automated alerting on infrastructure degradation
> Example stack trace:
> {code:text}
> java.lang.AssertionError: null
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.find(BPlusTree.java:5559)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.reinitialize(BPlusTree.java:5572)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5627)
> ...
> at
> org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest0(GridMapQueryExecutor.java:538)
> {code}
> {*}Solution{*}{*}{{*}}
> Add a dedicated long counter metric in the map query execution path that is
> incremented whenever a query fails due to a CorruptedTreeException.
> {*}Resulting Metric{*}{*}{{*}}
> ||*Registry*||*Name*||*Type*||*Description*||
> |sql.queries.treeCorruption|count|long|Number of queries that failed due to
> B+Tree cursor corruption (AssertionError during B+Tree page iteration)|
> *This metric enables:*
> - Automated alerting on tree corruption frequency
> - Correlation of corruption events with specific operations (rebalance,
> compaction, etc.)
> - Quantification of data reliability without log scraping
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
