shreemaan-abhishek opened a new pull request, #13562:
URL: https://github.com/apache/apisix/pull/13562
### Description
When `loki-logger` is configured with a dynamic label value (e.g. a global
rule with `"log_labels": {"job": "apisix", "service_name": "$service_name"}`),
the label leaked across services: one service's `service_name` got pinned onto
every other service's logs, and the value froze after the first request.
Two defects combined:
1. **The shared plugin `conf.log_labels` was mutated in place.** While
resolving variables, the resolved literal was written back into
`conf.log_labels`. After the first request `"$service_name"` became a literal,
and `core.utils.resolve_var` returns `n_resolved == 0` for a literal, so it was
never re-resolved again, freezing the value.
2. **Labels were resolved once per batch processor and shared by all
entries.** The resolution and the flush closure sat *after* the
`batch_processor_manager:add_entry` early-return, so they ran only for the
request that first created the processor. The closure built a single Loki
stream and put every batched line into it under one label set.
This is worst with a **global rule**, which has exactly one `conf` (the
batch processor is keyed by `plugin.conf_version(conf)`), so one shared
processor per worker froze the label for all subsequent requests from any
service.
### Fix
- **Clone** `conf.log_labels` before resolving, so the shared conf is never
mutated.
- **Resolve per request, before the early-return**, and stamp the result
onto the entry (`entry.loki_labels`), mirroring the existing
`entry.loki_log_time`. The entry is the only channel through the batch
processor.
- At flush time, **group entries into multiple Loki streams** keyed by their
resolved label set, so each request is logged under its own labels. In Loki a
stream is one unique label set, and the push payload supports multiple stream
objects — the old code forced everything into one.
- The grouping key sorts label names and joins `key=value` pairs with a NUL
separator (avoids relying on JSON key ordering).
- Internal fields (`loki_log_time`, `loki_labels`) are stripped from the
entry before encoding the log line.
No schema change; pure runtime behavior.
### Tests
Added `t/plugin/loki-logger.t` TEST 17–20: one route with
`log_labels.service = "$http_x_service_name"`, hit twice with different header
values against the same conf, then assert Loki has a distinct stream per value
and each line sits under its own label. Fails on the old frozen-label code,
passes on the fix. These are integration tests needing a live Loki at
`127.0.0.1:3100` (like the existing TEST 4/7/14).
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [x] I have updated the documentation to reflect this change (no doc change
needed — the documented behavior was already correct; this fix makes the
implementation match it)
- [x] I have verified that the code passes lint (`luacheck`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
