Jiří Smolík created LOG4J2-3425:
-----------------------------------
Summary: Syslog appender lacks the 'readTimeoutMillis' setting
Key: LOG4J2-3425
URL: https://issues.apache.org/jira/browse/LOG4J2-3425
Project: Log4j 2
Issue Type: Improvement
Components: Appenders
Affects Versions: 2.12.1
Reporter: Jiří Smolík
Recently, we've had an issue with our syslog server (collector), where it was
suddenly unable to forward messages sent to it. This resulted in flooding the
server and ultimately, exhausting its resources (memory and allocated temporary
storage).
However, more interestingly, Log4j2 appender behaved as follows:
# At the moment of exhaustion, one thread (T1) entered the synchronized block
in TcpSocketManager (see
[here](https://github.com/apache/logging-log4j2/blob/e2ee2f4dd9327eb1841023c3269ba12f5673e694/log4j-core/src/main/java/org/apache/logging/log4j/core/net/TcpSocketManager.java#L217))
and tried to write the message (see
[here]([https://github.com/apache/logging-log4j2/blob/e2ee2f4dd9327eb1841023c3269ba12f5673e694/log4j-core/src/main/java/org/apache/logging/log4j/core/net/TcpSocketManager.java#L253)).|https://github.com/apache/logging-log4j2/blob/e2ee2f4dd9327eb1841023c3269ba12f5673e694/log4j-core/src/main/java/org/apache/logging/log4j/core/net/TcpSocketManager.java#L219)),]
# The server didn't respond immediately - actually, it blocked T1 for hours
(it was most likely waiting for resources to be free).
# In the meantime, all threads attempting to write a message to the syslog
appender hanged at the synchronized block (they were waiting for T1 to finish).
Eventually, the connection to the server broke apart or some resources were
freed, and T1 finished. Interestingly, it didn't throw any exception, but that
is beside the point in this issue.
So, what happened is that the server got flooded and blocked one thread in our
application, but due to the way TcpSocketManager is implemented, our
application suffered a DoS.
As a precaution to such dreadful scenario, I would like to request adding a
read timeout setting to the TcpSocketManager (also syslog appender), which
would in case of the above described situation prevent T1 from blocking other
threads for a long time. At least, that's my idea of an easy fix. Connect and
reconnect timeout settings have a different purpose and were useless in this
case.
I am aware that the issue is also a matter of the syslog server configuration,
but application developers do not always have it under their control.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
