This is an automated email from the ASF dual-hosted git repository.
gian pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new c61313f4c4 Quieter streaming supervisors. (#13392)
c61313f4c4 is described below
commit c61313f4c4a396364aa19d63e68429c1a0cb277a
Author: Gian Merlino <[email protected]>
AuthorDate: Sun Nov 20 23:53:17 2022 -0800
Quieter streaming supervisors. (#13392)
Eliminates two common sources of noise with Kafka supervisors that have
large numbers of tasks and partitions:
1) Log the report at DEBUG rather than INFO level at each run cycle.
It can get quite large, and can be retrieved via API when needed.
2) Use log4j2.xml to quiet down the
org.apache.kafka.clients.consumer.internals
package. Avoids a log message per-partition per-minute as part of seeking
to the latest offset in the reporting thread. In the tasks, where this
sort of logging might be more useful, we have another log message with
the same information: "Seeking partition[%s] to[%s]".
---
examples/conf/druid/cluster/_common/log4j2.xml | 5 +++++
.../indexing/seekablestream/supervisor/SeekableStreamSupervisor.java | 4 +---
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/examples/conf/druid/cluster/_common/log4j2.xml
b/examples/conf/druid/cluster/_common/log4j2.xml
index 66dc13da4c..f55b70070c 100644
--- a/examples/conf/druid/cluster/_common/log4j2.xml
+++ b/examples/conf/druid/cluster/_common/log4j2.xml
@@ -82,5 +82,10 @@
<Logger name="com.sun.jersey.guice" level="warn" additivity="false">
<Appender-ref ref="FileAppender"/>
</Logger>
+
+ <!-- Quieter KafkaSupervisors -->
+ <Logger name="org.apache.kafka.clients.consumer.internals" level="warn"
additivity="false">
+ <Appender-ref ref="FileAppender"/>
+ </Logger>
</Loggers>
</Configuration>
diff --git
a/indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java
b/indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java
index 6f00ef7fbf..073749ca13 100644
---
a/indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java
+++
b/indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java
@@ -1508,9 +1508,7 @@ public abstract class
SeekableStreamSupervisor<PartitionIdType, SequenceOffsetTy
private void generateAndLogReport()
{
if (log.isDebugEnabled()) {
- log.debug(generateReport(true).toString());
- } else {
- log.info(generateReport(false).toString());
+ log.debug("%s", generateReport(true));
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
