henriquekops opened a new issue, #12544: URL: https://github.com/apache/druid/issues/12544
### Affected Version Apache Druid v0.22.1. ### Description Hey Druids! I'm currently using Apache Druid 0.22.1 deployed on Kubernetes (v1.18) using [druid-operator](https://github.com/druid-io/druid-operator). Aiming to create Grafana dashboards over auto-compaction behavior, I have selected the following metrics [from the docs](https://druid.apache.org/docs/latest/operations/metrics.html#coordination): - `compact/task/count` - `compactTask/availableSlot/count` After setting auto-compaction on my data sources and configuring [prometheus-emitter](https://druid.apache.org/docs/latest/development/extensions-contrib/prometheus.html) extension I was able to capture metrics about ingestion, memory and others, but those referring to compaction tasks (listed above) were always returning the value `0`. The following print screen shows this strange behavior for `compactTask/availableSlot/count`. The same occurs for `compact/task/count` even when there are compaction tasks running.  To isolate the problem I have set `druid.emitter=logging`, but the same situation was found (logs from coordinator):  Taking a quick glance at the code I found out where those compaction metrics are included at coordinator status (at `src/main/java/org/apache/druid/server/coordinator/duty/CompactSegments.java`):  Later, those variables are emitted by a `ServiceEmitter` (at `src/main/java/org/apache/druid/server/coordinator/duty/EmitClusterStatsAndMetrics.java`):  Grepping the log right above the `globalStat` at `CompactSegments.java` I could get the value set for `numAvailableCompactionTaskSlots` and `compactionTaskCapacity` variables (logs from coordinator):  **Given this scenario, am I missing some configuration or this version is running into a race condition at coordinator's duties?** ### Appendix Leaving some additional info here to help debugging. #### 1. Auto-compaction config The auto-compaction config submitted to coordinators: ```json { "dataSource": "TEST", "taskPriority": 25, "inputSegmentSizeBytes": 524288000, "maxRowsPerSegment": null, "skipOffsetFromLatest": "P1D", "tuningConfig": { "maxRowsInMemory": null, "maxBytesInMemory": null, "maxTotalRows": null, "splitHintSpec": null, "partitionsSpec": { "type": "single_dim", "targetRowsPerSegment": 5000000, "maxRowsPerSegment": null, "partitionDimension": "nice_dimension", "assumeGrouped": false }, "indexSpec": null, "indexSpecForIntermediatePersists": null, "maxPendingPersists": null, "pushTimeout": null, "segmentWriteOutMediumFactory": null, "maxNumConcurrentSubTasks": 2, "maxRetry": null, "taskStatusCheckPeriodMs": null, "chatHandlerTimeout": null, "chatHandlerNumRetries": null, "maxNumSegmentsToMerge": null, "totalNumMergeTasks": 1, "forceGuaranteedRollup": true, "type": "index_parallel" }, "granularitySpec": null, "ioConfig": null, "taskContext": null } ``` #### 2. Compaction config returned by the coordinator Using route `/druid/coordinator/v1/config/compaction`, the coordinators have returned the following `maxCompactionSlots`: ```json { "compactionConfigs": [ { "dataSource": "TEST", ... } ], "compactionTaskSlotRatio": 0.5, "maxCompactionTaskSlots": 10 } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
