For the search side of things, this profiling already exists in sandbox: https://github.com/apache/lucene/blob/main/lucene/sandbox/src/java/org/apache/lucene/sandbox/search/QueryProfilerIndexSearcher.java
It was copied from the Elasticsearch profiler code. I believe Ankit Jain is looking into making it work well with concurrent search. On Wed, Mar 19, 2025 at 5:23 AM Michael Sokolov <msoko...@gmail.com> wrote: > One idea I've heard batted around is to override > IndexSearcher.createWeight in a profiling IndexSearcher and then wrap > Weights and finally Scorers in order to emit metrics on every advance > or other low-level operation. This could help with search profiling I > guess although of course it will slow everything down, it might be > helpful for counting operations. But maybe your problem is during > indexing? > > On Wed, Mar 19, 2025 at 5:59 AM Robert Muir <rcm...@gmail.com> wrote: > > > > opentelemetry is anything but lightweight, not recommended. Don't > > waste your time with it. if you want to expose 'metrics', it is really > > up to the application to do that, lucene is just a library. I'd > > recommend just exposing via HTTP and using prometheus & co. > > > > Debugging performance issues in the library is totally different and > > telemetry as a word doesn't make sense for that You can setup 'perf' > > to debug performance issues. You can also run with lightweight JFR > > profiling built into the JVM. The nightly benchmarks already do that. > > > > On Wed, Mar 19, 2025 at 2:16 AM Vigya Sharma <vigya.w...@gmail.com> > wrote: > > > > > > Hi All, > > > > > > I am debugging an unexpected throughput regression in my feature > branch which has changes across multiple files and functions, and was > wondering if there's a better way to identify the slowdown than adding > elapsed time logs to infostream. > > > > > > Do we have support in Lucene for telemetry on internal operations? It > would be nice to be able to add timers, counters, and metrics, to identify > performance changes. This would hide disabled behind a gradle flag. We > could enable it for benchmark runs and possibly chart these metrics in > nightly runs. > > > > > > Has this been discussed before? I'm happy to explore this more if it > sounds like a good idea. Any suggestions for lightweight, open-source > telemetry libraries that I should look into? I'm considering OpenTelemetry > but open to alternate suggestions and ideas. > > > > > > Vigya > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
