[
https://issues.apache.org/jira/browse/IMPALA-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802346#comment-17802346
]
ASF subversion and git services commented on IMPALA-12666:
----------------------------------------------------------
Commit fcda98ad99c13324e3ab09f2e92d331d0304bb8e in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=fcda98ad9 ]
IMPALA-12666: Fix incorrect statestore metrics when impalad is down
Statestore::Topic::DeleteIfVersionsMatch() is used when impalad is down.
It incorrectly bumps the metrics of total value size and total topic
size. This patch removes the codes.
In order to verify the metrics using the /topics page, this patch
changes the /topics URL to also return numeric values which can be used
in the e2e test to verify the /metrics page.
Tests
- Add e2e test
Change-Id: I3ffcfb45b7cde0b40a87c9ca410ec634cb31cefb
Reviewed-on: http://gerrit.cloudera.org:8080/20841
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Statestore topic metrics are wrong when impalads are shutdown
> -------------------------------------------------------------
>
> Key: IMPALA-12666
> URL: https://issues.apache.org/jira/browse/IMPALA-12666
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> While looking into the statestore codes, I found we incorrectly update the
> metrics in Statestore::Topic::DeleteIfVersionsMatch()
> {code:cpp}
> void Statestore::Topic::DeleteIfVersionsMatch(TopicEntry::Version version,
> const Statestore::TopicEntryKey& key) {
> // Acquire exclusive lock - we are modifying the topic.
> lock_guard<shared_mutex> write_lock(lock_);
> TopicEntryMap::iterator entry_it = entries_.find(key);
> if (entry_it != entries_.end() && entry_it->second.version() == version) {
> // Add a new entry with the the version history for this deletion and
> remove the old
> // entry
> topic_update_log_.erase(version);
> topic_update_log_.emplace(++last_version_, key);
> value_size_metric_->Increment(entry_it->second.value().size()); // We
> shouldn't bump this since nothing is added into the entrics_ map.
> topic_size_metric_->Increment(entry_it->second.value().size()); // Same
> here.
> entry_it->second.SetDeleted(true);
> entry_it->second.SetVersion(last_version_);
> }
> }{code}
> value_size_metric_ and topic_size_metric_ are shown in the "/metrics" page as
> "statestore.total-value-size-bytes" and "statestore.total-topic-size-bytes".
> Each topic has two counters: total_key_size_bytes_ and
> total_value_size_bytes_.
> value_size_metric_ tracks the sum of total_value_size_bytes_ across all
> topics. We shouldn't bump the metric here since we just mark a topic entry as
> deleted but not removing it. Note that in other places, we will always bump
> total_value_size_bytes_ and value_size_metric_. This is the only place that
> we just bump value_size_metric_ without total_value_size_bytes_ so it's wrong.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
