loquisgon commented on code in PR #12488: URL: https://github.com/apache/druid/pull/12488#discussion_r869783472
########## docs/operations/metrics.md: ########## @@ -221,6 +221,41 @@ Note: If the JVM does not support CPU time measurement for the current thread, i |`worker/taskSlot/total/count`|Number of total task slots on the reporting worker per emission period. This metric is only available if the WorkerTaskCountStatsMonitor module is included.|category, version.|Varies.| |`worker/taskSlot/used/count`|Number of busy task slots on the reporting worker per emission period. This metric is only available if the WorkerTaskCountStatsMonitor module is included.|category, version.|Varies.| +## Batch ingestion metrics (Native parallel task) + +|Metric|Description|Dimensions|Normal Value| +|------|-----------|----------|------------| +|`ingest/batch/append/count`|Count of `1` every time a batch ingestion append job runs|dataSource, taskId, taskType|Always `1`.| +|`ingest/batch/append/segments/count`|Count of segments created |dataSource, taskId, taskType|At least `1`.| +|`ingest/batch/ovewrite/count`|Count of `1` every time a batch ingestion overwrite job runs|dataSource, taskId, taskType|Always `1`.| +|`ingest/batch/ovewrite/segments/count`|Count of segments created |dataSource, taskId, taskType|At least `1`.| +|`ingest/batch/replace/count`|Count of `1` every time a batch ingestion replace job runs|dataSource, taskId, taskType|Always `1`.| +|`ingest/batch/replace/segments/count`|Count of segments created |dataSource, taskId, taskType|At least `1`.| +|`ingest/batch/replace/tombstones/count`|Count of tombstones created in this replace job|dataSource, taskId, taskType|It can be zero when replace could not find any empty time chunks in the input intervals. It cannot be more than number of segments.| + +`APPEND`, `OVERWRITE`, and `REPLACE` are decided using the values +of the `isAppendToExisting` and `isDropExisting` flags in the +task's `IOConfig` as follows: + +|`isAppendToExisting` | `isdDropExisting` | mode | +|---------------------|-------------------|------| +`true` | `false` | `APPEND`| +`true` | `true ` | Invalid combination, exception thrown. | Review Comment: Yes it does but it was confusing where. Due to this great comment I decided to fully integrate the concept of `BatchIngestionMode` in the batch ingestion task internals replacing all the confusing tests for append or drop existing etc. There were many changes but they should straightforward to follow. The code is better now, IMO. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
