keith-turner commented on code in PR #4572:
URL: https://github.com/apache/accumulo/pull/4572#discussion_r1612384658
##########
server/tserver/src/main/java/org/apache/accumulo/tserver/metrics/TabletServerMetrics.java:
##########
@@ -36,6 +38,13 @@ public TabletServerMetrics(TabletServer tserver) {
@Override
public void registerMetrics(MeterRegistry registry) {
+ FunctionCounter
+ .builder(METRICS_COMPACTOR_ENTRIES_READ, null, o ->
FileCompactor.getTotalEntriesRead())
+ .description("Number of entries read").tag("type",
"tserver").register(registry);
Review Comment:
Could add a bit more to the description.
```suggestion
.description("Number of entries read by all compactions that have
run in the process").tag("type", "tserver").register(registry);
```
##########
server/base/src/main/java/org/apache/accumulo/server/compaction/FileCompactor.java:
##########
@@ -141,9 +153,61 @@ public synchronized String getCurrentLocalityGroup() {
return currentLocalityGroup;
}
- private void clearStats() {
- entriesRead.set(0);
- entriesWritten.set(0);
+ private void clearCurrentEntryCounts() {
+ currentEntriesRead.set(0);
+ currentEntriesWritten.set(0);
+ }
+
+ private void updateGlobalEntryCounts() {
+ updateTotalEntries(currentEntriesRead, lastRecordedEntriesRead,
totalEntriesRead);
+ updateTotalEntries(currentEntriesWritten, lastRecordedEntriesWritten,
totalEntriesWritten);
+ }
+
+ /**
+ * Updates the total count of entries by adding the difference between the
current count and the
+ * last recorded count to the total.
+ *
+ * @param current The current count of entries
+ * @param recorded The last recorded count of entries
+ * @param total The total count to add the difference to
+ */
+ private void updateTotalEntries(AtomicLong current, AtomicLong recorded,
LongAdder total) {
+ long currentCount = current.get();
+ long lastRecorded =
+ recorded.getAndUpdate(recordedValue -> Math.max(recordedValue,
currentCount));
+ if (lastRecorded < currentCount) {
+ total.add(currentCount - lastRecorded);
+ }
+ }
+
+ /**
+ * @return the total entries written by compactions over the lifetime of
this process.
+ */
+ public static long getTotalEntriesWritten() {
+ updateTotalEntries();
+ return totalEntriesWritten.sum();
+ }
+
+ /**
+ * @return the total entries read by compactions over the lifetime of this
process.
+ */
+ public static long getTotalEntriesRead() {
+ updateTotalEntries();
+ return totalEntriesRead.sum();
+ }
+
+ /**
+ * Updates total entries read and written for all currently running
compactions. Compactions will
+ * update the global stats when they finish. This can be called to update
them sooner. This method
+ * is rate limited, so it will not cause issues if called too frequently.
+ */
+ private static void updateTotalEntries() {
+ long currentTime = System.currentTimeMillis();
Review Comment:
could use nanotime here
##########
core/src/main/java/org/apache/accumulo/core/metrics/MetricsProducer.java:
##########
@@ -583,6 +583,8 @@ public interface MetricsProducer {
String METRICS_COMPACTOR_PREFIX = "accumulo.compactor.";
String METRICS_COMPACTOR_MAJC_STUCK = METRICS_COMPACTOR_PREFIX +
"majc.stuck";
+ String METRICS_COMPACTOR_ENTRIES_READ = METRICS_COMPACTOR_PREFIX +
"entries.read";
Review Comment:
These are good names for elasticity where there are no longer compaction
running in tservers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
