One idea I've heard batted around is to override IndexSearcher.createWeight in a profiling IndexSearcher and then wrap Weights and finally Scorers in order to emit metrics on every advance or other low-level operation. This could help with search profiling I guess although of course it will slow everything down, it might be helpful for counting operations. But maybe your problem is during indexing?
On Wed, Mar 19, 2025 at 5:59 AM Robert Muir <rcm...@gmail.com> wrote: > > opentelemetry is anything but lightweight, not recommended. Don't > waste your time with it. if you want to expose 'metrics', it is really > up to the application to do that, lucene is just a library. I'd > recommend just exposing via HTTP and using prometheus & co. > > Debugging performance issues in the library is totally different and > telemetry as a word doesn't make sense for that You can setup 'perf' > to debug performance issues. You can also run with lightweight JFR > profiling built into the JVM. The nightly benchmarks already do that. > > On Wed, Mar 19, 2025 at 2:16 AM Vigya Sharma <vigya.w...@gmail.com> wrote: > > > > Hi All, > > > > I am debugging an unexpected throughput regression in my feature branch > > which has changes across multiple files and functions, and was wondering if > > there's a better way to identify the slowdown than adding elapsed time logs > > to infostream. > > > > Do we have support in Lucene for telemetry on internal operations? It would > > be nice to be able to add timers, counters, and metrics, to identify > > performance changes. This would hide disabled behind a gradle flag. We > > could enable it for benchmark runs and possibly chart these metrics in > > nightly runs. > > > > Has this been discussed before? I'm happy to explore this more if it sounds > > like a good idea. Any suggestions for lightweight, open-source telemetry > > libraries that I should look into? I'm considering OpenTelemetry but open > > to alternate suggestions and ideas. > > > > Vigya > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
