SEZ9 opened a new pull request, #11984:
URL: https://github.com/apache/seatunnel/pull/11984
### Purpose of this pull request
Items (b), (c) and (d) of #11981: bring runtime log levels to the servlet
REST plane, make an
override tell you it is an override, and make it revertible.
Today the only way to change a level while a cluster runs is the legacy
Hazelcast endpoint
`POST /hazelcast/rest/maps/log-level`. It applies a level and says nothing
else: not what the level
was before, not whether the current level came from `log4j2.properties`, and
not how to get back to
it. Reverting means editing the file on every node and waiting for the
monitor interval. Nothing is
written to the log either, so a level someone changed by hand during an
incident is indistinguishable
from a configured one afterwards.
This PR adds the same capability to the v2 (Jetty servlet) plane, with the
missing pieces:
| Endpoint | Purpose |
|----------|---------|
| `GET /loggers` | every logger of the running configuration, with its level
and origin |
| `GET /loggers/{name}` | the effective level of one logger, resolved
through its closest configured ancestor |
| `POST /loggers/{name}?level=DEBUG` | override a level (the level may also
be sent as `{"level":"DEBUG"}`) |
| `DELETE /loggers/{name}` | revert the override |
- **`origin`** is `file` for the level of the log4j2 configuration file and
`runtime-override` for a
level that was set through a log level endpoint (the legacy endpoint is
tracked too, since both go
through the same helper). An overridden logger that the file configures as
well also reports
`fileLevel`, the level `DELETE` puts back.
- **`DELETE`** restores exactly the pre-override state: the configured level
when the file configured
the logger, and otherwise it drops the `LoggerConfig` the override added
so the logger inherits
from its parent again. `status` is `NO_OVERRIDE`, and nothing is touched,
when the logger was never
overridden through an endpoint.
- **`?scope=cluster`** runs the same request on every member and answers
with one entry per member,
so a level does not have to be changed node by node. `status` is
`SUCCESS`, `PARTIAL_FAILURE` or
`FAILURE`, and a member that could not be reached is reported with its own
`status` and `error`
rather than failing the whole request.
- **Audit**: one `INFO` line per change with the logger, the old and the new
level, the scope and the
address of the caller.
- An unknown level or scope is rejected with `400` and the list of valid
values; a request without a
level or without a logger name is rejected as well instead of silently
doing nothing.
Members are reached over the REST port of their configuration through the
existing
`GetNodeHttpPortOperation` — the same mechanism `/logs` already uses to
collect log names — so no new
`IdentifiedDataSerializable` class ids are introduced. This shares the
existing limitation of
`/logs`: a member that took a different port through `enable-dynamic-port`
is not reached, which is
stated in the documentation. Making the bound port discoverable would change
`/logs` too and belongs
in its own PR.
**This PR is stacked on #11983** (which introduces the `LogLevels` helper
this reuses). The diff
shown here includes that commit; please merge #11983 first, or review only
the second commit.
Related to #11981
### Does this PR introduce _any_ user-facing change?
Yes, new REST endpoints on the v2 plane. No existing endpoint changes.
```
$ curl 'http://127.0.0.1:8080/loggers'
{"node":"127.0.0.1:8080","loggers":[{"name":"root","level":"INFO","origin":"file"}]}
$ curl -X POST
'http://127.0.0.1:8080/loggers/org.apache.seatunnel.connectors.seatunnel.jdbc?level=debug&scope=cluster'
{"scope":"cluster","status":"SUCCESS","name":"org.apache.seatunnel.connectors.seatunnel.jdbc","level":"DEBUG",
"nodes":[{"name":"...jdbc","level":"DEBUG","origin":"runtime-override","node":"127.0.0.1:8080",
"previousLevel":"INFO","status":"SUCCESS"}]}
$ curl -X DELETE
'http://127.0.0.1:8080/loggers/org.apache.seatunnel.connectors.seatunnel.jdbc?scope=cluster'
{"scope":"cluster","status":"SUCCESS","name":"org.apache.seatunnel.connectors.seatunnel.jdbc",
"nodes":[{"name":"...jdbc","level":"INFO","origin":"file","node":"127.0.0.1:8080",
"previousLevel":"DEBUG","status":"SUCCESS"}]}
```
The node log of every change:
```
Logger level changed: logger=org.apache.seatunnel.connectors.seatunnel.jdbc,
INFO -> DEBUG, scope=node, client=10.0.0.7
Logger level reverted:
logger=org.apache.seatunnel.connectors.seatunnel.jdbc, DEBUG -> INFO,
scope=node, client=10.0.0.7
```
### How was this patch tested?
Unit tests, `LogLevelsTest` in `seatunnel-engine-server` (extends the class
added by #11983):
- an override of a logger with no configuration of its own is tracked,
reports
`runtime-override`/no `fileLevel`, and after `reset` inherits its parent
level again;
- an override of a configured logger remembers the configured level, does
not forget it on a second
override, and restores it on `reset`;
- `reset` of a logger that was never overridden reports no change;
- the root logger can be overridden and reverted, and appears in the logger
list as `root` rather
than as the empty log4j2 name.
End-to-end, `RestApiIT#testLoggers` (two-node cluster):
- the logger list contains `root` with `origin=file`;
- an unknown level, a missing level, a missing logger name and an unknown
scope are each rejected
with `400`;
- `POST {"level":"debug"}` applies `DEBUG` and reports
`origin=runtime-override`, and a following
`GET` agrees;
- `DELETE` reports `previousLevel=DEBUG` and `origin=file`, and a second
`DELETE` reports
`NO_OVERRIDE`;
- `?scope=cluster` answers for every member on `GET`, `POST` and `DELETE`.
I could not build or run these locally (no JDK/Maven in my environment), so
CI is the first real
execution of both suites.
### Check list
* [x] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
— no new dependency.
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs — new
`/loggers` section in `docs/en|zh/engines/zeta/rest-api-v2.md`, and
`docs/en|zh/engines/zeta/logging.md` now spells out the
difference between editing `log4j2.properties` and calling the endpoint.
* [x] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR — not needed,
only new endpoints are added.
* [x] If you are contributing the connector code, please check that the
following files are updated — not a connector change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
