HeartSaVioR commented on a change in pull request #29269:
URL: https://github.com/apache/spark/pull/29269#discussion_r463897312
##########
File path: docs/web-ui.md
##########
@@ -426,11 +426,11 @@ queries. Currently, it contains the following metrics.
* **Batch Duration.** The process duration of each batch.
* **Operation Duration.** The amount of time taken to perform various
operations in milliseconds.
The tracked operations are listed as follows.
- * addBatch: Adds result data of the current batch to the sink.
- * getBatch: Gets a new batch of data to process.
- * latestOffset: Gets the latest offsets for sources.
- * queryPlanning: Generates the execution plan.
- * walCommit: Writes the offsets to the metadata log.
+ * addBatch: Time taken to read the micro-batch's input data from the
sources, process it, and write the batch's output to the sink. This should take
the bulk of the micro-batch's time.
+ * getBatch: Time taken to prepare the logical query to read the input of
the current micro-batch from the sources.
+ * getOffset: Time taken to query the maximum available offset for this
source.
Review comment:
What I meant about commenting latestOffset vs getOffset is, we are using
"both" as of now, which looks to be OK to consolidate into one. I prefer
`latestOffset`, given it will be picked up at least most of built-in data
source.
If you want to "just" change doc in this PR, you'd like to change this like
`* latestOffset & getOffset: ...`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
