[
https://issues.apache.org/jira/browse/CASSANDRA-21117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18094080#comment-18094080
]
Dmitry Konstantinov edited comment on CASSANDRA-21117 at 7/6/26 5:48 PM:
-------------------------------------------------------------------------
The fix in _putUnusedPortion_ looks good to me and explains the negative values
reported for both the overflow size and the overall size.
At the same time, I see two potential issues:
# The patch changes the metric exposed as size in nodetool info. Previously,
it represented the total pool size, whereas now it reports the used portion of
the pool.
The original metric also makes sense because it reflects the amount of memory
that has actually been allocated (from OS point of view), regardless of whether
it is currently in use or sitting idle in the pool. The new metric, on the
other hand, shows only the amount of memory currently taken from the pool. It
is hard to say which one is more appropriate, as it depends on the intended
semantics of the metric in the nodetool output.
# The original report also raises the concern that the reported size can be
"2× the heap size or larger." That aspect does not appear to be analyzed here,
so it is difficult to conclude that the issue has been fully resolved. As far
as I can tell, the patch addresses only the negative-value scenario. It would
be good to clarify whether the large reported size is actually a bug or simply
reflects real memory consumption, considering that the pool may allocate
off-heap memory.
was (Author: dnk):
The fix in _putUnusedPortion_ looks good to me and explains the negative values
reported for both the overflow size and the overall size.
At the same time, I see two potential issues:
# The patch changes the metric exposed as size in nodetool info. Previously,
it represented the total pool size, whereas now it reports the used portion of
the pool.
The original metric also makes sense because it reflects the amount of memory
that has actually been allocated (from OS point of view), regardless of whether
it is currently in use or sitting idle in the pool. The new metric, on the
other hand, shows only the amount of memory currently taken from the pool. It
is hard to say which one is more appropriate, as it depends on the intended
semantics of the metric.
# The original report also raises the concern that the reported size can be
"2× the heap size or larger." That aspect does not appear to be analyzed here,
so it is difficult to conclude that the issue has been fully resolved. As far
as I can tell, the patch addresses only the negative-value scenario. It would
be good to clarify whether the large reported size is actually a bug or simply
reflects real memory consumption, considering that the pool may allocate
off-heap memory.
> Nodetool info reports unreliable size in use
> --------------------------------------------
>
> Key: CASSANDRA-21117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21117
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
> Reporter: Brad Schoening
> Assignee: Stefan Miklosovic
> Priority: Low
> Fix For: 4.1.x, 5.0.x, 6.x, 7.x
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> We noticed 'nodetool info' may report the networking cache may be 2X the heap
> size or larger.
> {noformat}
> Node: 169.102.228.220
> Heap Memory (MB) : 9581.65 / 16384.00
> Off Heap Memory (MB) : 398.83
> Data Center : NA_NE_C01
> Rack : D
> Exceptions : 0
> Key Cache : entries 964712, size 100 MiB, capacity 100 MiB, 70259124 hits,
> 240261115 requests, 0.292 recent hit rate, 14400 save period in seconds
> Row Cache : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests,
> NaN recent hit rate, 0 save period in seconds
> Counter Cache : entries 0, size 0 bytes, capacity 50 MiB, 0 hits, 0 requests,
> NaN recent hit rate, 7200 save period in seconds
> Chunk Cache : entries 964502, size 6.1 GiB, capacity 9.97 GiB, 1515943127
> misses, 2414866720 requests, 0.372 recent hit rate, 96.680 microseconds miss
> latency
> Network Cache : size 30.9 GiB, overflow size: 30.15 GiB, capacity 768
> MiB{noformat}
> in other cases, it grew to a reported 75GB or even negative values
>
> {noformat}
> Network Cache : size 76.05 GiB, overflow size: 75.05 GiB, capacity 1 GiB
> Network Cache : size -154303849440 bytes, overflow size: -155377591264 bytes,
> capacity 1 GiB
> {noformat}
>
> The Cassandra version this is happening on all recent patch versions – so
> i.e. 5.0.4 - 4.1.9
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
