zstan commented on code in PR #13130:
URL: https://github.com/apache/ignite/pull/13130#discussion_r3586973582
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
+
+Use several signals together.
+For example, `SELECT 1` can confirm that a client or SQL query path is
reachable, but it does not check user partitions, backup consistency, baseline
membership, or all server nodes.
+Similarly, `control.sh --cache idle_verify` is an important consistency check,
but it is not a complete health check.
+
+=== Check the Intended State and Node Membership
+Start with the link:monitoring-metrics/cluster-states[cluster state].
+
+Run:
+
+[source,shell]
+----
+./control.sh --state
+----
+
+Relevant output:
+
+[source,text]
+----
+Command [STATE] started
+Arguments: --state
+--------------------------------------------------------------------------------
+Cluster state: ACTIVE
+Command [STATE] finished with code: 0
+----
+
+`ACTIVE` is expected for normal read-write operation.
+`ACTIVE_READ_ONLY` is normal when read-only operation was intentionally
enabled.
+`INACTIVE` is acceptable only when it matches the current operation, for
example planned maintenance; an inactive cluster does not serve the data
workload.
+The criterion is whether the actual state matches the state that was
intentionally set for the deployment.
+
+If baseline topology is enabled, check the baseline as well:
+
+Run:
+
+[source,shell]
+----
+./control.sh --baseline
+----
+
+Relevant output for a cluster where all baseline nodes are online:
+
+[source,text]
+----
+Cluster state: ACTIVE
+Current topology version: 3
+Baseline auto adjustment disabled: softTimeout=300000
+
+Current topology version: 3 (Coordinator: ConsistentId=node-1, Order=1)
+
+Baseline nodes:
+ ConsistentId=node-1, State=ONLINE, Order=1
+ ConsistentId=node-2, State=ONLINE, Order=2
+ ConsistentId=node-3, State=ONLINE, Order=3
+--------------------------------------------------------------------------------
+Number of baseline nodes: 3
+
+Other nodes not found.
+----
+
+If an online server node has joined the cluster but is not in the baseline,
the command shows it under `Other nodes`:
+
+[source,text]
+----
+Other nodes:
+ ConsistentId=node-4, Order=4
+Number of other nodes: 1
+----
+
+The baseline contains server nodes that are intended to store data.
+Client nodes are not part of the baseline.
+An online server node in `Other nodes` is not always an error: the node may
have been prepared intentionally but not yet introduced into the data topology.
+If the node is expected to store data, first check the
link:clustering/baseline-topology#baseline-topology-autoadjustment[baseline
auto-adjustment policy] and the deployment plan, then use the documented
baseline change procedure.
+Changing the baseline can start link:data-rebalancing[rebalancing]. This is
often not an easy procedure for a cluster with data in persistence storage.
+
+Frequent unexpected topology changes are not normal and usually indicate node
instability, network issues, or misconfigured discovery. However, a temporary
topology change does not make the cluster unhealthy by itself.
+Server `JOIN`, `LEFT`, and `FAIL` events change cluster membership and the
major topology version.
+Cache start or stop, distributed cache creation, SQL table creation, and other
affinity-changing events can change the minor affinity topology version without
a node loss.
+Planned events are normal.
+Investigate unexpected repeated membership churn, node segmentation, network
failures, or a partition map exchange that does not finish.
Review Comment:
Just give me a clue here - this sentence is correct, but how it can be
detected ? I think we need to form this document from position that reader is
not deeply understant how it work and need to have some commands\logs how to
understand (split brain\PME) situations ?
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
+
+Use several signals together.
+For example, `SELECT 1` can confirm that a client or SQL query path is
reachable, but it does not check user partitions, backup consistency, baseline
membership, or all server nodes.
+Similarly, `control.sh --cache idle_verify` is an important consistency check,
but it is not a complete health check.
+
+=== Check the Intended State and Node Membership
+Start with the link:monitoring-metrics/cluster-states[cluster state].
+
+Run:
+
+[source,shell]
+----
+./control.sh --state
+----
+
+Relevant output:
+
+[source,text]
+----
+Command [STATE] started
+Arguments: --state
+--------------------------------------------------------------------------------
+Cluster state: ACTIVE
+Command [STATE] finished with code: 0
+----
+
+`ACTIVE` is expected for normal read-write operation.
+`ACTIVE_READ_ONLY` is normal when read-only operation was intentionally
enabled.
+`INACTIVE` is acceptable only when it matches the current operation, for
example planned maintenance; an inactive cluster does not serve the data
workload.
+The criterion is whether the actual state matches the state that was
intentionally set for the deployment.
+
+If baseline topology is enabled, check the baseline as well:
+
+Run:
+
+[source,shell]
+----
+./control.sh --baseline
+----
+
+Relevant output for a cluster where all baseline nodes are online:
+
+[source,text]
+----
+Cluster state: ACTIVE
+Current topology version: 3
+Baseline auto adjustment disabled: softTimeout=300000
+
+Current topology version: 3 (Coordinator: ConsistentId=node-1, Order=1)
+
+Baseline nodes:
+ ConsistentId=node-1, State=ONLINE, Order=1
+ ConsistentId=node-2, State=ONLINE, Order=2
+ ConsistentId=node-3, State=ONLINE, Order=3
+--------------------------------------------------------------------------------
+Number of baseline nodes: 3
+
+Other nodes not found.
+----
+
+If an online server node has joined the cluster but is not in the baseline,
the command shows it under `Other nodes`:
+
+[source,text]
+----
+Other nodes:
+ ConsistentId=node-4, Order=4
+Number of other nodes: 1
+----
+
+The baseline contains server nodes that are intended to store data.
+Client nodes are not part of the baseline.
+An online server node in `Other nodes` is not always an error: the node may
have been prepared intentionally but not yet introduced into the data topology.
+If the node is expected to store data, first check the
link:clustering/baseline-topology#baseline-topology-autoadjustment[baseline
auto-adjustment policy] and the deployment plan, then use the documented
baseline change procedure.
+Changing the baseline can start link:data-rebalancing[rebalancing]. This is
often not an easy procedure for a cluster with data in persistence storage.
+
+Frequent unexpected topology changes are not normal and usually indicate node
instability, network issues, or misconfigured discovery. However, a temporary
topology change does not make the cluster unhealthy by itself.
+Server `JOIN`, `LEFT`, and `FAIL` events change cluster membership and the
major topology version.
Review Comment:
Not clear, rewrite or expand it plz.
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
+
+Use several signals together.
+For example, `SELECT 1` can confirm that a client or SQL query path is
reachable, but it does not check user partitions, backup consistency, baseline
membership, or all server nodes.
+Similarly, `control.sh --cache idle_verify` is an important consistency check,
but it is not a complete health check.
+
+=== Check the Intended State and Node Membership
+Start with the link:monitoring-metrics/cluster-states[cluster state].
+
+Run:
+
+[source,shell]
+----
+./control.sh --state
+----
+
+Relevant output:
+
+[source,text]
+----
+Command [STATE] started
+Arguments: --state
+--------------------------------------------------------------------------------
+Cluster state: ACTIVE
+Command [STATE] finished with code: 0
+----
+
+`ACTIVE` is expected for normal read-write operation.
+`ACTIVE_READ_ONLY` is normal when read-only operation was intentionally
enabled.
+`INACTIVE` is acceptable only when it matches the current operation, for
example planned maintenance; an inactive cluster does not serve the data
workload.
+The criterion is whether the actual state matches the state that was
intentionally set for the deployment.
+
+If baseline topology is enabled, check the baseline as well:
+
+Run:
+
+[source,shell]
+----
+./control.sh --baseline
+----
+
+Relevant output for a cluster where all baseline nodes are online:
+
+[source,text]
+----
+Cluster state: ACTIVE
+Current topology version: 3
+Baseline auto adjustment disabled: softTimeout=300000
+
+Current topology version: 3 (Coordinator: ConsistentId=node-1, Order=1)
+
+Baseline nodes:
+ ConsistentId=node-1, State=ONLINE, Order=1
+ ConsistentId=node-2, State=ONLINE, Order=2
+ ConsistentId=node-3, State=ONLINE, Order=3
+--------------------------------------------------------------------------------
+Number of baseline nodes: 3
+
+Other nodes not found.
+----
+
+If an online server node has joined the cluster but is not in the baseline,
the command shows it under `Other nodes`:
+
+[source,text]
+----
+Other nodes:
+ ConsistentId=node-4, Order=4
+Number of other nodes: 1
+----
+
+The baseline contains server nodes that are intended to store data.
+Client nodes are not part of the baseline.
+An online server node in `Other nodes` is not always an error: the node may
have been prepared intentionally but not yet introduced into the data topology.
+If the node is expected to store data, first check the
link:clustering/baseline-topology#baseline-topology-autoadjustment[baseline
auto-adjustment policy] and the deployment plan, then use the documented
baseline change procedure.
Review Comment:
"deployment plan" - what does it mean ?
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
+
+Use several signals together.
+For example, `SELECT 1` can confirm that a client or SQL query path is
reachable, but it does not check user partitions, backup consistency, baseline
membership, or all server nodes.
+Similarly, `control.sh --cache idle_verify` is an important consistency check,
but it is not a complete health check.
+
+=== Check the Intended State and Node Membership
+Start with the link:monitoring-metrics/cluster-states[cluster state].
+
+Run:
+
+[source,shell]
+----
+./control.sh --state
+----
+
+Relevant output:
+
+[source,text]
+----
+Command [STATE] started
+Arguments: --state
+--------------------------------------------------------------------------------
+Cluster state: ACTIVE
+Command [STATE] finished with code: 0
+----
+
+`ACTIVE` is expected for normal read-write operation.
+`ACTIVE_READ_ONLY` is normal when read-only operation was intentionally
enabled.
+`INACTIVE` is acceptable only when it matches the current operation, for
example planned maintenance; an inactive cluster does not serve the data
workload.
+The criterion is whether the actual state matches the state that was
intentionally set for the deployment.
+
+If baseline topology is enabled, check the baseline as well:
Review Comment:
When it can be enabled\disabled and what user need to do - if it disabled ?
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
Review Comment:
From reader side this phrase is confusing, (you and me understand that it\`s
true but i don\`t like to show it for reader) we need to remove it at all or
extend this comment.
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
Review Comment:
what does it mean in common ? Probably we need to mention baseline,
partition loss ?
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
+
+Use several signals together.
+For example, `SELECT 1` can confirm that a client or SQL query path is
reachable, but it does not check user partitions, backup consistency, baseline
membership, or all server nodes.
Review Comment:
I know that this is a common practise (in some installations) but, in
common, it shows nothing, this command can be executed on client node without
cluster nodes interaction.
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
+
+Use several signals together.
+For example, `SELECT 1` can confirm that a client or SQL query path is
reachable, but it does not check user partitions, backup consistency, baseline
membership, or all server nodes.
+Similarly, `control.sh --cache idle_verify` is an important consistency check,
but it is not a complete health check.
+
+=== Check the Intended State and Node Membership
+Start with the link:monitoring-metrics/cluster-states[cluster state].
+
+Run:
+
+[source,shell]
+----
+./control.sh --state
+----
+
+Relevant output:
+
+[source,text]
+----
+Command [STATE] started
+Arguments: --state
+--------------------------------------------------------------------------------
+Cluster state: ACTIVE
+Command [STATE] finished with code: 0
+----
+
+`ACTIVE` is expected for normal read-write operation.
+`ACTIVE_READ_ONLY` is normal when read-only operation was intentionally
enabled.
+`INACTIVE` is acceptable only when it matches the current operation, for
example planned maintenance; an inactive cluster does not serve the data
workload.
+The criterion is whether the actual state matches the state that was
intentionally set for the deployment.
+
+If baseline topology is enabled, check the baseline as well:
+
+Run:
+
+[source,shell]
+----
+./control.sh --baseline
+----
+
+Relevant output for a cluster where all baseline nodes are online:
+
+[source,text]
+----
+Cluster state: ACTIVE
+Current topology version: 3
+Baseline auto adjustment disabled: softTimeout=300000
+
+Current topology version: 3 (Coordinator: ConsistentId=node-1, Order=1)
+
+Baseline nodes:
+ ConsistentId=node-1, State=ONLINE, Order=1
+ ConsistentId=node-2, State=ONLINE, Order=2
+ ConsistentId=node-3, State=ONLINE, Order=3
+--------------------------------------------------------------------------------
+Number of baseline nodes: 3
+
+Other nodes not found.
+----
+
+If an online server node has joined the cluster but is not in the baseline,
the command shows it under `Other nodes`:
+
+[source,text]
+----
+Other nodes:
+ ConsistentId=node-4, Order=4
+Number of other nodes: 1
+----
+
+The baseline contains server nodes that are intended to store data.
+Client nodes are not part of the baseline.
+An online server node in `Other nodes` is not always an error: the node may
have been prepared intentionally but not yet introduced into the data topology.
+If the node is expected to store data, first check the
link:clustering/baseline-topology#baseline-topology-autoadjustment[baseline
auto-adjustment policy] and the deployment plan, then use the documented
baseline change procedure.
+Changing the baseline can start link:data-rebalancing[rebalancing]. This is
often not an easy procedure for a cluster with data in persistence storage.
+
+Frequent unexpected topology changes are not normal and usually indicate node
instability, network issues, or misconfigured discovery. However, a temporary
topology change does not make the cluster unhealthy by itself.
+Server `JOIN`, `LEFT`, and `FAIL` events change cluster membership and the
major topology version.
+Cache start or stop, distributed cache creation, SQL table creation, and other
affinity-changing events can change the minor affinity topology version without
a node loss.
Review Comment:
What different between: 'Cache start' and 'distributed cache creation' ?
'SQL table creation', and what about drop ?
And over all - why this information is helpful in acpect of this document ?
Probably you want to say smth., but i miss it (
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
+
+Use several signals together.
+For example, `SELECT 1` can confirm that a client or SQL query path is
reachable, but it does not check user partitions, backup consistency, baseline
membership, or all server nodes.
+Similarly, `control.sh --cache idle_verify` is an important consistency check,
but it is not a complete health check.
+
+=== Check the Intended State and Node Membership
+Start with the link:monitoring-metrics/cluster-states[cluster state].
+
+Run:
+
+[source,shell]
+----
+./control.sh --state
+----
+
+Relevant output:
+
+[source,text]
+----
+Command [STATE] started
+Arguments: --state
+--------------------------------------------------------------------------------
+Cluster state: ACTIVE
+Command [STATE] finished with code: 0
+----
+
+`ACTIVE` is expected for normal read-write operation.
+`ACTIVE_READ_ONLY` is normal when read-only operation was intentionally
enabled.
+`INACTIVE` is acceptable only when it matches the current operation, for
example planned maintenance; an inactive cluster does not serve the data
workload.
+The criterion is whether the actual state matches the state that was
intentionally set for the deployment.
+
+If baseline topology is enabled, check the baseline as well:
+
+Run:
+
+[source,shell]
+----
+./control.sh --baseline
+----
+
+Relevant output for a cluster where all baseline nodes are online:
+
+[source,text]
+----
+Cluster state: ACTIVE
+Current topology version: 3
+Baseline auto adjustment disabled: softTimeout=300000
+
+Current topology version: 3 (Coordinator: ConsistentId=node-1, Order=1)
+
+Baseline nodes:
+ ConsistentId=node-1, State=ONLINE, Order=1
+ ConsistentId=node-2, State=ONLINE, Order=2
+ ConsistentId=node-3, State=ONLINE, Order=3
+--------------------------------------------------------------------------------
+Number of baseline nodes: 3
+
+Other nodes not found.
+----
+
+If an online server node has joined the cluster but is not in the baseline,
the command shows it under `Other nodes`:
Review Comment:
Plz append examples:
1. with one lost node
2. with partition loss
2a. if node will come back to cluster - it still has a partition loss and
need to be reset - expand this plz.
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
+
+Use several signals together.
+For example, `SELECT 1` can confirm that a client or SQL query path is
reachable, but it does not check user partitions, backup consistency, baseline
membership, or all server nodes.
+Similarly, `control.sh --cache idle_verify` is an important consistency check,
but it is not a complete health check.
+
+=== Check the Intended State and Node Membership
+Start with the link:monitoring-metrics/cluster-states[cluster state].
+
+Run:
+
+[source,shell]
+----
+./control.sh --state
+----
+
+Relevant output:
+
+[source,text]
+----
+Command [STATE] started
+Arguments: --state
+--------------------------------------------------------------------------------
+Cluster state: ACTIVE
+Command [STATE] finished with code: 0
+----
+
+`ACTIVE` is expected for normal read-write operation.
+`ACTIVE_READ_ONLY` is normal when read-only operation was intentionally
enabled.
+`INACTIVE` is acceptable only when it matches the current operation, for
example planned maintenance; an inactive cluster does not serve the data
workload.
+The criterion is whether the actual state matches the state that was
intentionally set for the deployment.
+
+If baseline topology is enabled, check the baseline as well:
+
+Run:
+
+[source,shell]
+----
+./control.sh --baseline
+----
+
+Relevant output for a cluster where all baseline nodes are online:
+
+[source,text]
+----
+Cluster state: ACTIVE
+Current topology version: 3
+Baseline auto adjustment disabled: softTimeout=300000
+
+Current topology version: 3 (Coordinator: ConsistentId=node-1, Order=1)
+
+Baseline nodes:
+ ConsistentId=node-1, State=ONLINE, Order=1
+ ConsistentId=node-2, State=ONLINE, Order=2
+ ConsistentId=node-3, State=ONLINE, Order=3
+--------------------------------------------------------------------------------
+Number of baseline nodes: 3
+
+Other nodes not found.
+----
+
+If an online server node has joined the cluster but is not in the baseline,
the command shows it under `Other nodes`:
+
+[source,text]
+----
+Other nodes:
+ ConsistentId=node-4, Order=4
+Number of other nodes: 1
+----
+
+The baseline contains server nodes that are intended to store data.
+Client nodes are not part of the baseline.
+An online server node in `Other nodes` is not always an error: the node may
have been prepared intentionally but not yet introduced into the data topology.
+If the node is expected to store data, first check the
link:clustering/baseline-topology#baseline-topology-autoadjustment[baseline
auto-adjustment policy] and the deployment plan, then use the documented
baseline change procedure.
+Changing the baseline can start link:data-rebalancing[rebalancing]. This is
often not an easy procedure for a cluster with data in persistence storage.
Review Comment:
"easy" - not an engeneering term ) what does it mean ? I\`d prefer to say
like : Baseline change procedure will provide 'rebalance' which means that
partitions will be rebased according to new affinity distribution.
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
+- Expected nodes are present.
+- No pending transactions and no split brain.
+- No lost partitions.
+- Transient activity such as rebalancing or queue growth converges instead of
accumulating.
+
+But the thing is, there is no universal single health metric.
+
+Use several signals together.
+For example, `SELECT 1` can confirm that a client or SQL query path is
reachable, but it does not check user partitions, backup consistency, baseline
membership, or all server nodes.
+Similarly, `control.sh --cache idle_verify` is an important consistency check,
but it is not a complete health check.
Review Comment:
control.**sh** - it\`s a linux specific
##########
docs/_docs/perf-and-troubleshooting/general-perf-tips.adoc:
##########
@@ -47,3 +47,345 @@ queries with JOINs at massive scale and expect significant
performance benefits.
* Adjust link:data-rebalancing[data rebalancing settings] to ensure that
rebalancing completes faster when your cluster topology changes.
+== How to assess cluster health
+
+Cluster health is a complex thing. Apache Ignite is capable of demonstrating
great performance across different scenarios with varying loads. Therefore, in
general terms, a healthy cluster is one whose behavior aligns with your
expectations. However, there are some universal aspects that apply to all
deployments and warrant attention.
+
+It is important to understand that a healthy cluster may undergo planned
topology changes or temporary load spikes.
+
+The key properties are:
+- The cluster serves operations allowed by its intended state.
+- Data remains consistent (`idle_verify` passes, meaning partitions are intact
and there are no conflicts).
Review Comment:
'idle_verify' need to be cross-linked with appropriate chapter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
