[
https://issues.apache.org/jira/browse/KAFKA-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976519#comment-14976519
]
Grant Henke commented on KAFKA-2588:
------------------------------------
Thanks for the feedback [~gwenshap]. I used the aggregation example as a way to
try and clarify my point briefly, but perhaps it muddied the water. The only
reason I bring it up is because I have seen a few people confused by what this
metric represents. Let me try and explain my understanding of the Kafka model
and if things still don't align we can leave this alone.
Note: This discuses the model at a user/logical level ignoring Java class names
and implementation details.
In Kafka there are *topics* which are a _container_ that contains feeds of
messages. Those topics have *partitions*. The partition is mainly a logical
concept owned by a topic. There is no physical/real partition that can be owned
by a broker. However, each partition has _n_ *replicas*, these replicas are are
physically hosted by brokers and may be in-sync or out-of-sync (arguably not a
replica anymore). Today, the logic that maps a physical replica to a logical
partition is if the replica is the leader. Therefore, that is the closet thing
to a partition that we can count on a per-broker basis. Otherwise describing
how many partitions a broker has does not make much sense. Because topics have
partitions not brokers.
This is why my change does not eliminate any useful metrics, but essentially
renames them to make their values more clear. Below is the metric names, what I
expect to see and what I currently see:
- replicaCount
-- currently: does not exists
-- with my change: added to contain what partitionCount used to. The count
of replicas on that broker.
- partitionCount
-- currently: the count of replicas on that broker
-- with my change: the count replicas be served on that broker (synonymous
with leaderCount)
- leaderCount
-- this did not change but partitionCount also holds this value
Based on your description above it looks like you expect partitionCount to
represent the same thing I do:
{quote}
PartitionCount shows number of *partitions served* by the broker.
{quote}
Though based on my summary above, I would correct this part about
leaders/followers to be:
{quote}
...and since each -partition- *replica* can potentially become leader or
follower...
{quote}
I hope this clarified my reasoning for the re-label of the metrics and why I
think some users were confused. I apologize for being long winded. I wanted to
be sure I was clear on my current understanding. Please don't hesitate to
correct my mistakes. We can drop this if it still does not align with others
perspectives.
> ReplicaManager partitionCount metric should actually be replicaCount
> --------------------------------------------------------------------
>
> Key: KAFKA-2588
> URL: https://issues.apache.org/jira/browse/KAFKA-2588
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8.2.1
> Reporter: Grant Henke
> Assignee: Grant Henke
>
> The metrics "partitionCount" in the ReplicaManager actually represents the
> count of replicas.
> As an example if I have a cluster with 1 topic with 1 partitions and a
> replication factor of 3. The metric (aggregated) would show a value of 3.
> There is a metric called "LeaderCount" that actually represents the
> "partitionCount". In my example above the metric (aggregated) would show a
> value of 1.
> We do need to consider compatibility of consuming systems. I think the most
> simple change would be to:
> - Adjust the "partitionCount" metric to be the same value as "LeaderCount"
> - Add a "replicaCount" metric which contains the values "partitionCount" does
> today
> - Leave "LeaderCount" in for compatibility
> Documentation will need to be updated as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
