yesemsanthoshkumar commented on code in PR #6350:
URL: https://github.com/apache/hudi/pull/6350#discussion_r971631023
##########
website/docs/metrics.md:
##########
@@ -201,3 +201,32 @@ These `HoodieMetrics` can then be plotted on a standard
tool like grafana. Below
<figure>
<img className="docimage"
src={require("/assets/images/hudi_commit_duration.png").default}
alt="hudi_commit_duration.png" />
</figure>
+
+## List of metrics:
+
+The below metrics are available in all timeline operations that involves a
commit such as deltacommit, compaction, clustering and rollback.
+
+Name | Description
+--- | ---
+commitFreshnessInMs | Milliseconds from the commit end time and the maximum
event time of the incoming records
+commitLatencyInMs | Milliseconds from the commit end time and the minimum
event time of incoming records
+commitTime | Time of commit in epoch milliseconds
+duration | Total time taken for the commit/rollback in milliseconds
+numFilesDeleted | Number of files deleted during a clean/rollback
+numFilesFinalized | Number of files finalized in a write
+totalBytesWritten | Bytes written in a HoodieCommit
+totalCompactedRecordsUpdated | Number of records updated in a compaction
operation
+totalCreateTime | Time taken for file creation during a Hoodie Insert operation
+totalFilesInsert | Number of newly written files in a HoodieCommit
+totalFilesUpdate | Number of files updated in a HoodieCommit
+totalInsertRecordsWritten | Number of records inserted or converted to
updates(for small file handling) in a HoodieCommit
Review Comment:
@yihua This metric is emitted during both clustering and compaction op
inside SparkRDDWriteClient.
https://github.com/apache/hudi/blob/71b81740589c9c3ec1ee1bd4910342fde0656790/hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java#L320
for compaction reference.
Moreover, the
https://github.com/apache/hudi/blob/4f7ea8c79a9d13accf72d094296993c588d87beb/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/HoodieMetrics.java#L162
updatecommitmetrics function calls fetchtotalInsertRecordsWritten defined as
https://github.com/apache/hudi/blob/4f7ea8c79a9d13accf72d094296993c588d87beb/hudi-common/src/main/java/org/apache/hudi/common/model/HoodieCommitMetadata.java#L278
This inturn calls numInserts on the writestat.
numInserts is documented as
https://github.com/apache/hudi/blob/8ab6f17149e7edbdc80351bdf0d2e5c4c4a33908/hudi-common/src/main/java/org/apache/hudi/common/model/HoodieWriteStat.java#L69
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
