Dennis-Mircea Ciupitu created FLINK-40104:
---------------------------------------------
Summary: Logging framework JARs are placed under $FLINK_HOME/log
instead of the operator lib directory
Key: FLINK-40104
URL: https://issues.apache.org/jira/browse/FLINK-40104
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Affects Versions: kubernetes-operator-1.15.0
Reporter: Dennis-Mircea Ciupitu
h1. Summary
The Logback support added in FLINK-39501 (PR #1096) stores the
logging-framework JARs under {{$FLINK_HOME/log}} through a dedicated
{{OPERATOR_LOG}} environment variable, and the container entrypoint redundantly
redefines that same variable. Neither issue breaks the default behaviour, but
both are hygiene and robustness problems in the image layout.
h1. Background
To support switching the operator logging framework at runtime, the build
copies the SLF4J bindings into two selectable directories and the entrypoint
puts the chosen one on the classpath:
* {{log4j-slf4j-impl}} is copied into {{$FLINK_HOME/log/log4j}}
* {{logback-classic}} and {{logback-core}} are copied into
{{$FLINK_HOME/log/logback}}
The Dockerfile defines {{OPERATOR_LOG=$FLINK_HOME/log}} and
{{OPERATOR_LIB=$FLINK_HOME/operator-lib}}, and the entrypoint selects
{{$OPERATOR_LOG/log4j/\*}} or {{$OPERATOR_LOG/logback/\*}} based on the
{{LOGGING_FRAMEWORK}} value.
h1. Problem
h2. 1. Logging-framework JARs live under the conventional log-output directory
The contents of {{$FLINK_HOME/log/log4j}} and {{$FLINK_HOME/log/logback}} are
library JARs placed on the classpath, not log files. Storing them under
{{$FLINK_HOME/log}} is a category error, since {{$FLINK_HOME/log}} is the
conventional Flink log-output location. This has two consequences:
* It conflates classpath libraries with log output, which is confusing and
diverges from how the operator already isolates its own dependencies under
{{$FLINK_HOME/operator-lib}}.
* It is latently fragile. {{$FLINK_HOME/log}} is exactly where an operator
would mount a volume for log persistence. Such a mount would shadow the
{{log4j}} and {{logback}} subdirectories, remove the selected logging JARs from
the classpath, and break operator startup.
h2. 2. The entrypoint redundantly redefines OPERATOR_LOG
{{OPERATOR_LOG}} is already set as a Dockerfile {{ENV}} and is therefore
inherited by the entrypoint process. The entrypoint reassigns it to the same
effective value, so within the shipped image the line is a redundant overwrite
with no functional effect.
h1. Impact
Both problems are non-breaking under the default configuration, since the
operator logs to the console (no log files are written to {{$FLINK_HOME/log}})
and the shipped Helm chart does not mount a volume at {{/opt/flink/log}}. The
concern is the latent shadowing risk and the misleading image layout, which
make the logging setup fragile for users who add a log-persistence volume at
the conventional path and harder to reason about for maintainers.
h1. Expected outcome
The logging-framework JARs should live under the operator library area rather
than the log-output directory, so that {{$FLINK_HOME/log}} stays reserved for
log output consistent with Flink core, the selected framework JARs cannot be
shadowed by a log volume mount, and the redundant environment-variable
redefinition in the entrypoint is removed. The separation between the operator
library directory and Flink's own {{$FLINK_HOME/lib}} should be preserved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
