jerryshao opened a new pull request, #13275:
URL: https://github.com/apache/gravitino/pull/13275
### What changes were proposed in this pull request?
- Log4j2 templates (server, Iceberg REST, Lance REST): add `%i` to every
size-rolled archive name, set
`fileIndex = nomax`, and scope each `Delete` action to the log's own
archives (`IfFileName`,
`maxDepth = 1`). An archive is deleted when it is older than `logMaxAge`
(30d), or, oldest first, when
the log's archives exceed its total size cap. The caps are properties at
the top of each file: server 2GB,
audit 10GB, lineage 1GB, Iceberg REST 2GB, Lance REST 1GB. The roll size
goes from 10MB to 100MB;
lineage now also rolls by size.
- `.out` files: `bin/common.sh.template` adds `rotate_out_file`. Every
`start` rotates the file and keeps
`GRAVITINO_OUT_FILE_KEEP` (default 5) previous files.
- Helm charts (gravitino, iceberg-rest-server, lance-rest-server): the same
retention for the main appender,
with new values `logMaxAge`, `serverLogMaxTotalSize`,
`rollingAppenderStrategyFileIndex`, and
`rollingAppenderStrategyDeleteIfFileNameGlob`.
- Docs: a "Log rotation and retention" section in `how-to-install.md`, and
the audit retention example in
`gravitino-server-config.md`.
### Why are the changes needed?
Each 10MB roll overwrites the previous archive of the day, so most of each
day's logs are lost. The audit
log keeps at most 7 archives a day, `Delete` actions remove unrelated old
files under the log directory,
no log has a size limit, and `.out` files grow forever.
Fix: #13274
### Does this PR introduce _any_ user-facing change?
- Archive names become `<log>_<yyyyMMdd>.<index>.log.gz`, and the roll size
is 100MB.
- New log4j2 properties: `logMaxAge`, `serverLogMaxTotalSize`,
`auditLogMaxTotalSize`, `lineageLogMaxTotalSize`.
- New env var `GRAVITINO_OUT_FILE_KEEP`.
- New Helm values listed above. The chart defaults change the same way as
the tarball.
- `appender.audit_file.strategy.delete.ifAll.ifLastModified.age` no longer
exists. Log4j2 rejects a
configuration file that still sets it; the docs describe the replacement.
### How was this patch tested?
- `TestLog4j2RollingConfiguration` (10 cases): loads each template with a
1KB roll size and checks that no
message is lost, that only the log's own expired archives are deleted, and
that the oldest archives are
deleted beyond the cap.
- `TestOutFileRotation` (5 cases): covers `rotate_out_file` with the
default, custom, zero, and invalid limits.
- helm-unittest cases for the rendered retention defaults and overrides.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
