neoremind opened a new issue, #16507: URL: https://github.com/apache/lucene/issues/16507
### Description When `storeTermVectors` is enabled, `BytesRefHash.clear()` is called after every document via `TermVectorsConsumerPerField.finishDocument()` -> `reset()` -> `bytesHash.clear(false)`. Inside `clear()`, `shrink(count)` may allocate a brand new `int[] ids` if the previous document had way more unique terms than the current one. This means for workloads with var-len term-vectors-enabled fields, we can produce many short-lived `int[]` allocation per document. For example: - Doc N has 40 unique terms in a term-vector-enabled field -> `ids[]` size is 128 - Doc N+1 has only 14 unique terms -> `shrink()` allocates a new `int[32]`, the old `int[128]` gets GCed <img width="1329" height="618" alt="Image" src="https://github.com/user-attachments/assets/1f267ef4-2a3c-4bfc-87c0-37bdd842cf3c"; /> Over millions of documents, this may create GC pressure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
