[
https://issues.apache.org/jira/browse/CASSSIDECAR-496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSSIDECAR-496:
-----------------------------------
Description:
{{HttpServerOptionsProvider}} line 58 hardcodes {{{}setLogActivity(true){}}},
installing
Netty's {{LoggingHandler}} on every HTTP connection with no way to disable it.
The only
hardcoded option in a method where everything else reads from
{{{}SidecarConfiguration{}}}.
On a 3-node cluster running the {{ghcr.io/apache/cassandra-sidecar}} image this
produced
8.6 million log lines in one hour, roughly 96% of all log volume, because
{{ByteBufUtil.appendPrettyHexDump}} turns a single 95,504-byte response into
about 6,000
lines. The dumps include request and response bodies in cleartext, and since
schema
endpoints return DDL, CQL ends up in the logs too. It only fires when the
process starts
without {{logback.configurationFile}} — supplied solely by the Gradle start
scripts — so a
directly-launched jar falls back to logback's default of root DEBUG and dumps
everything;
{{server/src/test/resources/logback-test.xml}} pins {{LoggingHandler}} to info,
which is
why CI never caught it.
Fix: disable, following the existing {{tcp_keep_alive}} pattern, plus
{{<logger name="io.netty" level="INFO"/>}} in {{conf/logback.xml}} as defence
in depth.
was:
{\{HttpServerOptionsProvider}} line 58 hardcodes \{{setLogActivity(true)}},
installing
Netty's \{{LoggingHandler}} on every HTTP connection with no way to disable it.
The only
hardcoded option in a method where everything else reads from
\{{SidecarConfiguration}}.
On a 3-node cluster running the \{{ghcr.io/apache/cassandra-sidecar}} image
this produced
8.6 million log lines in one hour, roughly 96% of all log volume, because
{\{ByteBufUtil.appendPrettyHexDump}} turns a single 95,504-byte response into
about 6,000
lines. The dumps include request and response bodies in cleartext, and since
schema
endpoints return DDL, CQL ends up in the logs too. It only fires when the
process starts
without \{{logback.configurationFile}} — supplied solely by the Gradle start
scripts — so a
directly-launched jar falls back to logback's default of root DEBUG and dumps
everything;
{\{server/src/test/resources/logback-test.xml}} pins \{{LoggingHandler}} to
info, which is
why CI never caught it.
Fix: make it configurable via \{{ServiceConfiguration}}, defaulting
to false, following the existing \{{tcp_keep_alive}} pattern, plus
{\{<logger name="io.netty" level="INFO"/>}} in \{{conf/logback.xml}} as defence
in depth.
> Sidecar unconditionally enables Netty wire logging, flooding logs with hex
> dumps of request payloads
> ------------------------------------------------------------------------------------------------------
>
> Key: CASSSIDECAR-496
> URL: https://issues.apache.org/jira/browse/CASSSIDECAR-496
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Configuration
> Reporter: Jon Haddad
> Assignee: Jon Haddad
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{HttpServerOptionsProvider}} line 58 hardcodes {{{}setLogActivity(true){}}},
> installing
> Netty's {{LoggingHandler}} on every HTTP connection with no way to disable
> it. The only
> hardcoded option in a method where everything else reads from
> {{{}SidecarConfiguration{}}}.
> On a 3-node cluster running the {{ghcr.io/apache/cassandra-sidecar}} image
> this produced
> 8.6 million log lines in one hour, roughly 96% of all log volume, because
> {{ByteBufUtil.appendPrettyHexDump}} turns a single 95,504-byte response into
> about 6,000
> lines. The dumps include request and response bodies in cleartext, and since
> schema
> endpoints return DDL, CQL ends up in the logs too. It only fires when the
> process starts
> without {{logback.configurationFile}} — supplied solely by the Gradle start
> scripts — so a
> directly-launched jar falls back to logback's default of root DEBUG and dumps
> everything;
> {{server/src/test/resources/logback-test.xml}} pins {{LoggingHandler}} to
> info, which is
> why CI never caught it.
> Fix: disable, following the existing {{tcp_keep_alive}} pattern, plus
> {{<logger name="io.netty" level="INFO"/>}} in {{conf/logback.xml}} as defence
> in depth.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
