masaori335 opened a new pull request, #13841:
URL: https://github.com/apache/trafficserver/pull/13841
# Motivation
After a restart that reads the directory from disk,
`proxy.process.cache.volume_N.direntries.used` reads lower than the
real number of entries. If recovery clears most of a stripe, the gauge goes
below zero, and monitoring that reads it as
unsigned shows about 1.8e19. The global
`proxy.process.cache.direntries.used` is not affected.
For example,
```
proxy.process.cache.direntries.total 10248483768
proxy.process.cache.direntries.used 1081041961
proxy.process.cache.volume_1.direntries.total 10248156288
proxy.process.cache.volume_1.direntries.used 1043024692
proxy.process.cache.volume_2.direntries.total 327480
proxy.process.cache.volume_2.direntries.used 9177
```
The volume totals add up to the global total exactly. The volume `used`
gauges add up to 1,043,033,869, which is
38,008,092 (3.5%) less than the global gauge. After startup the global and
volume gauges get the same increments and
decrements, so this gap was set at startup and stays for the life of the
process.
# Approach
Recovery (`StripeSM::handle_recover_from_data()`) clears directory entries
before `CacheProcessor::cacheInitialized()`
seeds the gauges, and it decrements the gauges as it goes.
`cacheInitialized()` then `store`s the global gauge, which
discards those decrements. It `increment`s each volume gauge instead, which
keeps them. So each volume gauge starts at
the real count minus the entries recovery removed.
`cacheInitialized()` now sets the volume gauges to zero before adding up
each stripe's `entries_used()`. This is how
`CachePeriodicMetricsUpdate()` already handles the per-volume `bytes_used`.
`Directory::clear_range()` only runs during recovery, so its decrements were
always going to be overwritten. This PR
removes them. `cleanup()` still decrements during recovery, because
`agg_wrap()` and `freelist_clean()` use it at
runtime, so the zeroing is still required.
# Testing
- Cache unit tests (`ctest -R cache`) pass.
- No new unit test. The seeding code is inline in `cacheInitialized()` and
would have to be pulled out to test it on
its own.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
