aglinxinyuan opened a new pull request, #6797: URL: https://github.com/apache/texera/pull/6797
### What changes were proposed in this PR? **Root cause.** `AmberLogging` names each logger `"<actorId>] [<ClassName>"` so the logback pattern `[%logger]` renders both the actor id and the class. The side effect is that these logger names start with the **actor id** (`WF6-…`, `COORDINATOR`), not `org.apache…`, so the `<logger name="org.apache" level="WARN"/>` rule in `amber/src/main/resources/logback.xml` never matches the engine classes. They all fall through to the root logger and emit at `INFO`. In loop/integration tests, workers are recreated every iteration, so per-ECM / per-worker / per-region lines fire dozens of times and flood the CI console. This PR demotes those statements to `DEBUG` so they no longer print at the default `INFO` level (CI, local, and prod), and demotes three `WARN`s that fire on **normal** operation: | Message | Was | Source | |---|---|---| | `receive` / `process` / `send ECM` (2–3× per control message, full `ChannelIdentity` dump) | INFO | `DataProcessor.scala`, `main_loop.py` | | `register <id> -> Actor[…]` | INFO | `PekkoActorRefMappingService.scala` | | `…is not reachable anymore, it might have crashed` — fires on graceful teardown | WARN | `PekkoActorRefMappingService.scala` | | `unknown identifier` — registration race | WARN | `PekkoActorRefMappingService.scala` | | `worker replay log writing conf`, `DP thread started`/`exits`, `Starting the worker.` | INFO | `WorkflowActor`, `DPThread`, `StartHandler` | | `Region N successfully terminated.` | INFO | `RegionExecutionManager.scala` | | `…completed, # of input ports…` | INFO | `DataProcessor.scala` | | `client actor cannot handle …` — `//skip` fallthrough | WARN | `ClientActor.scala` | Genuine fault paths are **untouched**: `DP Thread exists unexpectedly` (ERROR), `Failed to fetch actorRef` (WARN), and `Error when terminating region` (WARN, the failure branch — distinct from the demoted `successfully terminated` success branch). As a CI backstop, `TEXERA_SERVICE_LOG_LEVEL=WARN` and `UDF_PYTHON_LOG_STREAMHANDLER_LEVEL=WARN` are pinned on the amber unit and integration test steps in `build.yml`, so any remaining `INFO` chatter is suppressed there even before the source demotions take effect. Net effect: the bulk of the per-iteration output from loop/integration specs disappears from the default-level log, while `DEBUG` keeps it available on demand. ### Any related issues, documentation, discussions? Closes #6796 ### How was this PR tested? This is a log-level-only change — no behavior changes — so it relies on the existing test suites run in CI. It was additionally verified by inspection: - **No test asserts on a demoted message.** Grepped all amber Scala and Python test sources for each demoted string; none is asserted. (`ReplayLogGeneratorSpec` matches `"cannot handle"`, but against an unrelated `RuntimeException` from `ReplayLogGenerator`, not the `ClientActor` log.) - **The CI `WARN` env does not break config specs.** `UdfConfigSpec`'s and `PekkoConfigSpec`'s default-value assertions are guarded (`ifUnset` / `!sys.env.contains(...)`), and `pekko.stdout-loglevel` is hardcoded `INFO` in `cluster.conf` (only `pekko.loglevel` is env-driven), so its unguarded assertion still passes under `WARN`. - **Lint clean.** No imports added (so `scalafixAll --check` is unaffected); the longest changed Scala line is 96 columns (< the 100 `maxColumn`), so `scalafmtCheckAll` stays green. A follow-up sweep for other hot-path `INFO`/`WARN` logs (e.g. `AsyncRPCClient` null control-reply, the range-shuffle partitioner, `Tuple` cast warnings) is captured in #6796 rather than expanded here. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8 [1M context]) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
