Ma77Ball opened a new pull request, #5375:
URL: https://github.com/apache/texera/pull/5375
### What changes were proposed in this PR?
> **In one sentence:** The OpenTelemetry foundation every later
observability PR builds on: off by default and opt-in, and anything leaving the
process is filtered through an allowlist first.
Plumbing only. No dashboards, no metrics instrumentation, no API changes.
---
#### The three pieces
| Component | Job |
|---|---|
| `OtelInit` | The single entry point. Reads its `OTEL_*` settings from
`observability.conf`, validates them, builds the SDK, registers it globally. |
| `TexeraOtelLogAppender` | A Logback appender that mirrors app logs into
OTel. A thin shim, all security logic lives in the sanitizer. |
| `LogSanitizer` | Pure functions: redact secrets, strip control chars, cap
size, drop noisy MDC keys (and redact credential-named ones) before anything
ships. |
---
#### How a log record flows
```mermaid
flowchart TD
A["logger.info(...)"] --> B[Logback ROOT]
B --> C["console / file (always)"]
B --> D{appender bound?}
D -- no --> E[no-op]
D -- yes --> F[LogSanitizer]
F --> G["strip ctrl chars · redact secrets · cap 16 KiB · filter MDC"]
G --> H[OTLP exporter] --> I[collector]
```
If the appender is not bound (init off or not yet run), logs still flow to
stdout/file untouched.
---
#### Bootstrap, and how it stays safe
```mermaid
flowchart TD
A[Service.run] --> B{OTEL_SDK_DISABLED == false?}
B -- no --> C[INFO, no-op]
B -- yes --> D{endpoint valid?\nscheme + host allowlist}
D -- no --> E[one WARN, no-op]
D -- yes --> F[build SDK · allowlisted attrs only]
F --> G[attach appender · shutdown hook]
```
- **Default OFF.** Inert until explicitly enabled with
`OTEL_SDK_DISABLED=false`; while disabled no exporters start and nothing is
emitted (per #5367).
- **Fails quietly, never crashes.** Missing collector or bad config gives
one WARN and a no-op, never a startup exception.
- **No autoconfigure SPI**, on purpose: filtering must run *before* any
exporter is built, so we read every `OTEL_*` setting ourselves through
`observability.conf` (HOCON defaults with `${?OTEL_*}` overrides).
---
#### The reason it is its own PR: every output is allowlisted or bounded
| Guard | Blocks |
|---|---|
| Endpoint scheme + host allowlist | exfiltration to arbitrary
protocols/hosts (scheme is `http`/`https` only; extend hosts via
`TEXERA_OTEL_ALLOWED_HOSTS`) |
| Resource-attribute allowlist | leaking arbitrary env/host data onto every
record |
| MDC deny-list + credential-key redaction | noisy bridge fields and
secret-named values reaching exports |
| Secret redaction | Bearer tokens, `password=`, AWS keys printed in logs
(message and stack trace) |
| Control-char strip | CR/LF log forging from user strings |
| Body cap (16 KiB) + metric-interval fallback | runaway log lines /
busy-loop exporter |
Exception stack traces are appended to the log body, so Dropwizard 500s stay
diagnosable from the dashboard instead of vanishing behind `"Error handling a
request: <id>"`.
---
#### Supporting wiring
- OTel `1.50.0` deps pinned once in the `common/observability` module; every
service inherits them via `dependsOn(Observability)`.
- One-line `OtelInit.init(...)` added to each service entry point.
- The five `OTEL_*` knobs are exposed through `observability.conf`,
`EnvironmentalVariable`, and the single-node `.env` + k8s `values.yaml` so
operators can find and tune them.
- Framework log caps (`pekko`, `iceberg`, `hadoop`, `kafka`, `jetty`, ...)
pinned at `WARN` so future TRACE/DEBUG surfaces Texera code, not the framework
firehose.
### Any related issues, documentation, or discussions?
Closes: #5367
Part of #4070. Approach: https://github.com/apache/texera/discussions/5355
### How was this PR tested?
- Unit specs for all three components: `LogSanitizerSpec` (redaction,
stripping, truncation, MDC deny-list + credential-key redaction),
`OtelInitSpec` (endpoint validation, attr allowlist, interval fallback,
default-off, in-memory exporter), `TexeraOtelLogAppenderSpec` (throwable-body
redaction, severity mapping, MDC handling), plus `ObservabilityConfigSpec` for
the config defaults.
- `sbt scalafmtCheckAll` passes and the `Config` + `Observability` modules
compile; full compile and tests run in CI.
### Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.8 in compliance with ASF policy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
