mochengqian opened a new pull request, #966: URL: https://github.com/apache/dubbo-go-pixiu/pull/966
## Summary Closes #943. The LLM proxy cooldown store is bounded by `maxCooldownStoreEntries`. At capacity it evicted the oldest entry by scanning the whole map to find the minimum `lastFailure` — O(N) on the failure path. This PR replaces that scan with a standard-library `container/list` LRU so eviction is O(1), while preserving the existing key and TTL behavior. ## Change - `cooldownStore` now holds a `recencyOrder *list.List` (oldest at front, newest at back) and the map is keyed to `*list.Element`. - `cooldownEntry` carries its own `key`, so a front-pop eviction can delete the matching map entry without a reverse lookup. - `markFailure`: existing key → update entry + `MoveToBack`; new key → `evictOldestIfFullLocked` then `PushBack`. - `evictOldestIfFullLocked`: pops `recencyOrder.Front()` in O(1) (was a full map scan). - A shared `removeLocked(key, element)` helper keeps the map and list in sync on every delete / sweep / eviction, so no stale list element survives. Caller-visible cooldown semantics are unchanged: endpoint identity key (`newCooldownKey` untouched), TTL derivation, idempotent replay, and the lazy expiry sweep all behave exactly as before. No third-party dependency, no background goroutine. ## Tests Existing cooldown tests pass unchanged (eviction-at-capacity, address/credential isolation, lazy sweep across clusters). New tests cover the LRU-specific invariants the issue calls out: - `TestCooldownStoreRefreshUpdatesRecency` — re-failing an entry moves it to the newest side; a later eviction drops a genuinely older entry. - `TestCooldownStoreSweepRemovesMapAndListState` — the TTL sweep clears both the map and the recency list (asserts `recencyOrder.Len() == 0`). - `TestCooldownStoreDeleteExpiredLeavesNoStaleListElement` — clearing an expired cooldown via the request path leaves no stale list element. Each new test asserts `len(map) == recencyOrder.Len()` to pin map/list sync. ## Test plan - [x] `go test -race ./pkg/filter/llm/proxy/...` - [x] `go vet ./pkg/filter/llm/proxy/` clean - [x] `go build ./...` clean - [x] pre-push local-CI (gofmt/vet/tests) passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
