This is an automated email from the ASF dual-hosted git repository.
ethanfeng pushed a commit to branch branch-0.4
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/branch-0.4 by this push:
new 38391c11f [CELEBORN-1187][FOLLOWUP] Unify the size and file count of
active shuffle metrics for master and worker
38391c11f is described below
commit 38391c11fdb7ac2b5c0918d4fb99aa1c5a6be535
Author: SteNicholas <[email protected]>
AuthorDate: Fri Dec 22 18:09:39 2023 +0800
[CELEBORN-1187][FOLLOWUP] Unify the size and file count of active shuffle
metrics for master and worker
### What changes were proposed in this pull request?
Unify the size and file count of active shuffle metrics for `MasterSource`
and `WorkerSource`.
Follow up #2171.
### Why are the changes needed?
`MasterSource` uses `PartitionWritten` and `PartitionFileCount` metrics as
the size and file count of active shuffle for all workers. Meanwhile,
`WorkerSource` uses `ActiveShuffleSize` and `ActiveShuffleFileCount` metrics as
the size and file count of active shuffle for a worker including master replica
and slave replica. It's recommended to unify the size and file count of active
shuffle metrics between `MasterSource` and `WorkerSource`.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Internal tests.
Closes #2186 from SteNicholas/CELEBORN-1187.
Authored-by: SteNicholas <[email protected]>
Signed-off-by: mingji <[email protected]>
(cherry picked from commit 276ab979a4b0cf4ac82440d1529d28b4748dc4b6)
Signed-off-by: mingji <[email protected]>
---
assets/grafana/celeborn-dashboard.json | 54 ++--------------------------------
1 file changed, 2 insertions(+), 52 deletions(-)
diff --git a/assets/grafana/celeborn-dashboard.json
b/assets/grafana/celeborn-dashboard.json
index 653d5863e..8c9780d73 100644
--- a/assets/grafana/celeborn-dashboard.json
+++ b/assets/grafana/celeborn-dashboard.json
@@ -1792,32 +1792,7 @@
},
"unit": "decbytes"
},
- "overrides": [
- {
- "__systemRef": "hideSeriesFrom",
- "matcher": {
- "id": "byNames",
- "options": {
- "mode": "exclude",
- "names": [
-
"metrics_ActiveShuffleSize_Value{instance=\"core-1-1:9096\", job=\"RSS\",
role=\"Worker\"}"
- ],
- "prefix": "All except:",
- "readOnly": true
- }
- },
- "properties": [
- {
- "id": "custom.hideFrom",
- "value": {
- "legend": false,
- "tooltip": false,
- "viz": true
- }
- }
- ]
- }
- ]
+ "overrides": []
},
"gridPos": {
"h": 8,
@@ -1911,32 +1886,7 @@
]
}
},
- "overrides": [
- {
- "__systemRef": "hideSeriesFrom",
- "matcher": {
- "id": "byNames",
- "options": {
- "mode": "exclude",
- "names": [
-
"metrics_ActiveShuffleFileCount_Value{instance=\"core-1-1:9096\", job=\"RSS\",
role=\"Worker\"}"
- ],
- "prefix": "All except:",
- "readOnly": true
- }
- },
- "properties": [
- {
- "id": "custom.hideFrom",
- "value": {
- "legend": false,
- "tooltip": false,
- "viz": true
- }
- }
- ]
- }
- ]
+ "overrides": []
},
"gridPos": {
"h": 8,
