[
https://issues.apache.org/jira/browse/KAFKA-14099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Garnier updated KAFKA-14099:
--------------------------------------
Description:
Prior to 2.2.1, when an REST API request was performed, there was a request log
in the log file:
{code:java}
[2022-07-23 07:18:16,128] INFO 172.18.0.1 - - [23/Jul/2022:07:18:16 +0000] "GET
/connectors HTTP/1.1" 200 2 "-" "curl/7.81.0" 66
(org.apache.kafka.connect.runtime.rest.RestServer:62)
{code}
Since 2.2.1, no more request logs
With a bisect, I found the problem comes from [PR
6651|https://github.com/apache/kafka/pull/6651] to fix KAFKA-8304
>From what I understand of the problem, the ContextHandlerCollection is added
>in the Server
>([https://github.com/dongjinleekr/kafka/blob/63a6130af30536d67fca5802005695a84c875b5e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java#L195])
> before handlers are really added in the ContextHandlerCollection
>([https://github.com/dongjinleekr/kafka/blob/63a6130af30536d67fca5802005695a84c875b5e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java#L296]).
I don't know the impact on other handlers, but clearly it doesn't work for the
RequestLogHandler.
A solution I found for the logging issue is to set the RequestLog directly in
the server without using an handlers:
{code:java}
diff --git
i/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
w/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
index ab18419efc..4d09cc0e6c 100644
---
i/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
+++
w/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
@@ -187,6 +187,11 @@ public class RestServer {
public void initializeServer() {
log.info("Initializing REST server");
+ Slf4jRequestLogWriter slf4jRequestLogWriter = new
Slf4jRequestLogWriter();
+
slf4jRequestLogWriter.setLoggerName(RestServer.class.getCanonicalName());
+ CustomRequestLog requestLog = new
CustomRequestLog(slf4jRequestLogWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT +
" %{ms}T");
+ jettyServer.setRequestLog(requestLog);
+
/* Needed for graceful shutdown as per `setStopTimeout` documentation
*/
StatisticsHandler statsHandler = new StatisticsHandler();
statsHandler.setHandler(handlers);
@@ -275,14 +280,7 @@ public class RestServer {
configureHttpResponsHeaderFilter(context);
}
- RequestLogHandler requestLogHandler = new RequestLogHandler();
- Slf4jRequestLogWriter slf4jRequestLogWriter = new
Slf4jRequestLogWriter();
-
slf4jRequestLogWriter.setLoggerName(RestServer.class.getCanonicalName());
- CustomRequestLog requestLog = new
CustomRequestLog(slf4jRequestLogWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT +
" %{ms}T");
- requestLogHandler.setRequestLog(requestLog);
-
contextHandlers.add(new DefaultHandler());
- contextHandlers.add(requestLogHandler);
handlers.setHandlers(contextHandlers.toArray(new Handler[0]));
try {
{code}
Same issue raised on StackOverflow:
[https://stackoverflow.com/questions/67699702/no-rest-api-logs-in-kafka-connect]
was:
Prior to 2.2.1, when an REST API request was performed, there was a request log
in the log file:
{code}
[2022-07-23 07:18:16,128] INFO 172.18.0.1 - - [23/Jul/2022:07:18:16 +0000] "GET
/connectors HTTP/1.1" 200 2 "-" "curl/7.81.0" 66
(org.apache.kafka.connect.runtime.rest.RestServer:62)
{code}
With a bisect, I found the problem comes from [PR
6651|https://github.com/apache/kafka/pull/6651] to fix KAFKA-8304
>From what I understand of the problem, the ContextHandlerCollection is added
>in the Server
>(https://github.com/dongjinleekr/kafka/blob/63a6130af30536d67fca5802005695a84c875b5e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java#L195)
> before handlers are really added in the ContextHandlerCollection
>(https://github.com/dongjinleekr/kafka/blob/63a6130af30536d67fca5802005695a84c875b5e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java#L296).
I don't know the impact on other handlers, but clearly it doesn't work for the
RequestLogHandler
A solution I found for the logging issue is to set the RequestLog directly in
the server without using an handlers:
{code}
diff --git
i/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
w/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
index ab18419efc..4d09cc0e6c 100644
---
i/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
+++
w/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
@@ -187,6 +187,11 @@ public class RestServer {
public void initializeServer() {
log.info("Initializing REST server");
+ Slf4jRequestLogWriter slf4jRequestLogWriter = new
Slf4jRequestLogWriter();
+
slf4jRequestLogWriter.setLoggerName(RestServer.class.getCanonicalName());
+ CustomRequestLog requestLog = new
CustomRequestLog(slf4jRequestLogWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT +
" %{ms}T");
+ jettyServer.setRequestLog(requestLog);
+
/* Needed for graceful shutdown as per `setStopTimeout` documentation
*/
StatisticsHandler statsHandler = new StatisticsHandler();
statsHandler.setHandler(handlers);
@@ -275,14 +280,7 @@ public class RestServer {
configureHttpResponsHeaderFilter(context);
}
- RequestLogHandler requestLogHandler = new RequestLogHandler();
- Slf4jRequestLogWriter slf4jRequestLogWriter = new
Slf4jRequestLogWriter();
-
slf4jRequestLogWriter.setLoggerName(RestServer.class.getCanonicalName());
- CustomRequestLog requestLog = new
CustomRequestLog(slf4jRequestLogWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT +
" %{ms}T");
- requestLogHandler.setRequestLog(requestLog);
-
contextHandlers.add(new DefaultHandler());
- contextHandlers.add(requestLogHandler);
handlers.setHandlers(contextHandlers.toArray(new Handler[0]));
try {
{code}
Same issue raised on StackOverflow:
https://stackoverflow.com/questions/67699702/no-rest-api-logs-in-kafka-connect
> No REST API request logs in Kafka connect
> -----------------------------------------
>
> Key: KAFKA-14099
> URL: https://issues.apache.org/jira/browse/KAFKA-14099
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 3.2.0
> Reporter: Alexandre Garnier
> Priority: Minor
>
> Prior to 2.2.1, when an REST API request was performed, there was a request
> log in the log file:
> {code:java}
> [2022-07-23 07:18:16,128] INFO 172.18.0.1 - - [23/Jul/2022:07:18:16 +0000]
> "GET /connectors HTTP/1.1" 200 2 "-" "curl/7.81.0" 66
> (org.apache.kafka.connect.runtime.rest.RestServer:62)
> {code}
> Since 2.2.1, no more request logs
>
> With a bisect, I found the problem comes from [PR
> 6651|https://github.com/apache/kafka/pull/6651] to fix KAFKA-8304
> From what I understand of the problem, the ContextHandlerCollection is added
> in the Server
> ([https://github.com/dongjinleekr/kafka/blob/63a6130af30536d67fca5802005695a84c875b5e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java#L195])
> before handlers are really added in the ContextHandlerCollection
> ([https://github.com/dongjinleekr/kafka/blob/63a6130af30536d67fca5802005695a84c875b5e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java#L296]).
> I don't know the impact on other handlers, but clearly it doesn't work for
> the RequestLogHandler.
>
> A solution I found for the logging issue is to set the RequestLog directly in
> the server without using an handlers:
> {code:java}
> diff --git
> i/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
>
> w/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
> index ab18419efc..4d09cc0e6c 100644
> ---
> i/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
> +++
> w/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
> @@ -187,6 +187,11 @@ public class RestServer {
> public void initializeServer() {
> log.info("Initializing REST server");
>
> + Slf4jRequestLogWriter slf4jRequestLogWriter = new
> Slf4jRequestLogWriter();
> +
> slf4jRequestLogWriter.setLoggerName(RestServer.class.getCanonicalName());
> + CustomRequestLog requestLog = new
> CustomRequestLog(slf4jRequestLogWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT
> + " %{ms}T");
> + jettyServer.setRequestLog(requestLog);
> +
> /* Needed for graceful shutdown as per `setStopTimeout`
> documentation */
> StatisticsHandler statsHandler = new StatisticsHandler();
> statsHandler.setHandler(handlers);
> @@ -275,14 +280,7 @@ public class RestServer {
> configureHttpResponsHeaderFilter(context);
> }
>
> - RequestLogHandler requestLogHandler = new RequestLogHandler();
> - Slf4jRequestLogWriter slf4jRequestLogWriter = new
> Slf4jRequestLogWriter();
> -
> slf4jRequestLogWriter.setLoggerName(RestServer.class.getCanonicalName());
> - CustomRequestLog requestLog = new
> CustomRequestLog(slf4jRequestLogWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT
> + " %{ms}T");
> - requestLogHandler.setRequestLog(requestLog);
> -
> contextHandlers.add(new DefaultHandler());
> - contextHandlers.add(requestLogHandler);
>
> handlers.setHandlers(contextHandlers.toArray(new Handler[0]));
> try {
> {code}
> Same issue raised on StackOverflow:
> [https://stackoverflow.com/questions/67699702/no-rest-api-logs-in-kafka-connect]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
