featzhang commented on code in PR #35:
URL:
https://github.com/apache/flink-connector-http/pull/35#discussion_r3015250572
##########
flink-connector-http/src/main/java/org/apache/flink/connector/http/config/HttpConnectorConfigConstants.java:
##########
@@ -164,4 +164,15 @@ public final class HttpConnectorConfigConstants {
SOURCE_RETRY_EXP_DELAY_PREFIX + "max-backoff";
public static final String SOURCE_RETRY_EXP_DELAY_MULTIPLIER =
SOURCE_RETRY_EXP_DELAY_PREFIX + "backoff-multiplier";
+
+ // -------------- Error logging configuration --------------
+
+ public static final String ERROR_LOG_LEVEL = FLINK_CONNECTOR_HTTP +
"error.log.level";
Review Comment:
Thanks for the review feedback. I've raised a Jira to document the
motivation behind this change.
To clarify why this introduces a new mechanism rather than extending
http.logging.level:
The existing http.logging.level controls content verbosity
(MIN/REQ_RESP/MAX) and is always emitted at DEBUG level. It is designed for
development-time
debugging and logs every request and response regardless of outcome. This
creates two gaps for production use:
1. In production, DEBUG logging is typically disabled. Errors become
invisible unless DEBUG is enabled for the entire connector, which floods logs
with
all successful requests as well.
2. There is no way to emit error information at ERROR or WARN level so
that log-based alerting systems (Splunk, Datadog, etc.) can detect failures.
The new http.error.log.* configuration addresses a different and
orthogonal concern: error severity. A user may want minimal content (no body)
but still
want errors surfaced at ERROR level for alerting. Or they may want MAX
content detail but only at DEBUG. These two dimensions — content verbosity and
error severity — are independent and should be configured separately.
One important behavioral detail: HttpErrorLogger respects
continue-on-error. When continue-on-error=true, errors are tolerated and only a
DEBUG message
is emitted, since logging at ERROR/WARN would be misleading. The
configured error log level only applies when the exception is actually thrown.
I've also split the concerns in the Jira description to make the
motivation clearer. Happy to discuss further if you have concerns about the
approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
