Quanlong Huang created IMPALA-12666:
---------------------------------------
Summary: Statestore topic metrics are wrong when impalads are
shutdown
Key: IMPALA-12666
URL: https://issues.apache.org/jira/browse/IMPALA-12666
Project: IMPALA
Issue Type: Bug
Components: Backend
Reporter: Quanlong Huang
Assignee: Quanlong Huang
While looking into the statestore codes, I found we incorrectly update the
metrics in Statestore::Topic::DeleteIfVersionsMatch()
{code:cpp}
void Statestore::Topic::DeleteIfVersionsMatch(TopicEntry::Version version,
const Statestore::TopicEntryKey& key) {
// Acquire exclusive lock - we are modifying the topic.
lock_guard<shared_mutex> write_lock(lock_);
TopicEntryMap::iterator entry_it = entries_.find(key);
if (entry_it != entries_.end() && entry_it->second.version() == version) {
// Add a new entry with the the version history for this deletion and
remove the old
// entry
topic_update_log_.erase(version);
topic_update_log_.emplace(++last_version_, key);
value_size_metric_->Increment(entry_it->second.value().size()); // We
shouldn't bump this since nothing is added into the entrics_ map.
topic_size_metric_->Increment(entry_it->second.value().size()); // Same
here.
entry_it->second.SetDeleted(true);
entry_it->second.SetVersion(last_version_);
}
}{code}
value_size_metric_ and topic_size_metric_ are shown in the "/metrics" page as
"statestore.total-value-size-bytes" and "statestore.total-topic-size-bytes".
Each topic has two counters: total_key_size_bytes_ and total_value_size_bytes_.
value_size_metric_ tracks the sum of total_value_size_bytes_ across all topics.
We shouldn't bump the metric here since we just mark a topic entry as deleted
but not removing it. Note that in other places, we will always bump
total_value_size_bytes_ and value_size_metric_. This is the only place that we
just bump value_size_metric_ without total_value_size_bytes_ so it's wrong.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
