mikemccand commented on issue #15662:
URL: https://github.com/apache/lucene/issues/15662#issuecomment-3842918834
01/22 (good) run has these `perf stat` counters for one of the 20
`SearchPerfTest` JVMs:
```
Performance counter stats for '/usr/lib/jvm/java-25-openjdk/bin/java
--add-modules jdk.incubator.vector -Xmx32g -Xms32g -server -XX:+UseParallelGC
-Djava.io.tmpdir=/l/tmp --enable-native-access=ALL-UNNAMED -XX:StartFligh\
tRecording=dumponexit=true,maxsize=250M,settings=/l/util.nightly/src/python/profiling.jfc,filename=/l/logs.nightly/bench-search-nightly-nightly-9.jfr
-XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints -classpath /l/tr\
unk.nightly/lucene/core/build/libs/lucene-core-11.0.0-SNAPSHOT.jar:/l/trunk.nightly/lucene/sandbox/build/classes/java/main:/l/trunk.nightly/lucene/misc/build/classes/java/main:/l/trunk.nightly/lucene/facet/build/classes/j\
ava/main:/l/trunk.nightly/lucene/analysis/common/build/classes/java/main:/l/trunk.nightly/lucene/analysis/icu/build/classes/java/main:/l/trunk.nightly/lucene/queryparser/build/classes/java/main:/l/trunk.nightly/lucene/gro\
uping/build/classes/java/main:/l/trunk.nightly/lucene/suggest/build/classes/java/main:/l/trunk.nightly/lucene/highlighter/build/classes/java/main:/l/trunk.nightly/lucene/codecs/build/classes/java/main:/l/trunk.nightly/luc\
ene/queries/build/classes/java/main:/l/trunk.nightly/lucene/join/build/classes/java/main:/l/trunk.nightly/lucene/spatial3d/build/classes/java/main:/l/util.nightly/lib/HdrHistogram.jar:/l/util.nightly/src/main/build/classe\
s/java/main:/l/util.nightly/build perf.SearchPerfTest -dirImpl MMapDirectory
-indexPath
/l/indices/wikimedium.trunk.nightly.facets.taxonomy:Date.taxonomy:Month.taxonomy:DayOfYear.sortedset:Month.sortedset:DayOfYear.sorted\
set:Date.taxonomy:RandomLabel.sortedset:RandomLabel.Lucene90.Lucene104.dvfields.vectors=1024.nd27.625M
-facets taxonomy:Date;Date -facets taxonomy:Month;Month -facets
taxonomy:DayOfYear;DayOfYear -facets sortedset:Month;M\
onth -facets sortedset:DayOfYear;DayOfYear -facets sortedset:Date;Date
-facets taxonomy:RandomLabel;RandomLabel -facets
sortedset:RandomLabel;RandomLabel -analyzer StandardAnalyzerNoStopWords
-taskSource /l/util.nightly/t\
asks/wikinightly.tasks -numConcurrentQueries 1 -taskRepeatCount 50 -field
body -tasksPerCat 5 -searchConcurrency 8 -staticSeed -8035476 -seed -7082989
-similarity BM25Similarity -commit multi -hiliteImpl FastVectorHighlig\
hter -log /l/logs.nightly/nightly.nightly.9 -topN 100 -context -printHeap
-pk -vectorFile
/lucenedata/enwiki/cohere-v3/cohere-v3-wikipedia-en-scattered-1024d.queries.vec
-vectorDimension 1024 -pollute':
1,088,849.85 msec task-clock:u # 2.728 CPUs
utilized
0 context-switches:u # 0.000 /sec
0 cpu-migrations:u # 0.000 /sec
603,814 page-faults:u # 554.543 /sec
9,322,771,152,253 instructions:u # 2.13 insn per
cycle
# 0.04 stalled cycles
per insn (30.78%)
4,379,914,648,276 cycles:u # 4.023 GHz
(30.77%)
398,139,212,961 stalled-cycles-frontend:u # 9.09% frontend
cycles idle (30.76%)
1,777,065,993,098 branches:u # 1.632 G/sec
(30.76%)
23,633,297,198 branch-misses:u # 1.33% of all
branches (30.76%)
4,403,556,999,556 L1-dcache-loads:u # 4.044 G/sec
(30.76%)
39,427,726,537 L1-dcache-load-misses:u # 0.90% of all
L1-dcache accesses (30.77%)
622,042,664,608 L1-icache-loads:u # 571.284 M/sec
(30.77%)
1,693,096,314 L1-icache-load-misses:u # 0.27% of all
L1-icache accesses (30.77%)
1,829,585,485 dTLB-loads:u # 1.680 M/sec
(30.77%)
371,109,765 dTLB-load-misses:u # 20.28% of all
dTLB cache accesses (30.77%)
214,576,217 iTLB-loads:u # 197.067 K/sec
(30.78%)
23,720,429 iTLB-load-misses:u # 11.05% of all
iTLB cache accesses (30.78%)
399.117771506 seconds time elapsed
1035.914308000 seconds user
42.341318000 seconds sys
```
1/29 (bad) has these:
```
Performance counter stats for '/usr/lib/jvm/java-25-openjdk/bin/java
--add-modules jdk.incubator.vector -Xmx32g -Xms32g -server -XX:+UseParallelGC
-Djava.io.tmpdir=/l/tmp --enable-native-access=ALL-UNNAMED -XX:StartFligh\
tRecording=dumponexit=true,maxsize=250M,settings=/l/util.nightly/src/python/profiling.jfc,filename=/l/logs.nightly/bench-search-nightly-nightly-9.jfr
-XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints -classpath /l/tr\
unk.nightly/lucene/core/build/libs/lucene-core-11.0.0-SNAPSHOT.jar:/l/trunk.nightly/lucene/sandbox/build/classes/java/main:/l/trunk.nightly/lucene/misc/build/classes/java/main:/l/trunk.nightly/lucene/facet/build/classes/j\
ava/main:/l/trunk.nightly/lucene/analysis/common/build/classes/java/main:/l/trunk.nightly/lucene/analysis/icu/build/classes/java/main:/l/trunk.nightly/lucene/queryparser/build/classes/java/main:/l/trunk.nightly/lucene/gro\
uping/build/classes/java/main:/l/trunk.nightly/lucene/suggest/build/classes/java/main:/l/trunk.nightly/lucene/highlighter/build/classes/java/main:/l/trunk.nightly/lucene/codecs/build/classes/java/main:/l/trunk.nightly/luc\
ene/queries/build/classes/java/main:/l/trunk.nightly/lucene/join/build/classes/java/main:/l/trunk.nightly/lucene/spatial3d/build/classes/java/main:/l/util.nightly/lib/HdrHistogram.jar:/l/util.nightly/src/main/build/classe\
s/java/main:/l/util.nightly/build perf.SearchPerfTest -dirImpl MMapDirectory
-indexPath
/l/indices/wikimedium.trunk.nightly.facets.taxonomy:Date.taxonomy:Month.taxonomy:DayOfYear.sortedset:Month.sortedset:DayOfYear.sorted\
set:Date.taxonomy:RandomLabel.sortedset:RandomLabel.Lucene90.Lucene104.dvfields.vectors=1024.nd27.625M
-facets taxonomy:Date;Date -facets taxonomy:Month;Month -facets
taxonomy:DayOfYear;DayOfYear -facets sortedset:Month;M\
onth -facets sortedset:DayOfYear;DayOfYear -facets sortedset:Date;Date
-facets taxonomy:RandomLabel;RandomLabel -facets
sortedset:RandomLabel;RandomLabel -analyzer StandardAnalyzerNoStopWords
-taskSource /l/util.nightly/t\
asks/wikinightly.tasks -numConcurrentQueries 1 -taskRepeatCount 50 -field
body -tasksPerCat 5 -searchConcurrency 8 -staticSeed -8035476 -seed -7082989
-similarity BM25Similarity -commit multi -hiliteImpl FastVectorHighlig\
hter -log /l/logs.nightly/nightly.nightly.9 -topN 100 -context -printHeap
-pk -vectorFile
/lucenedata/enwiki/cohere-v3/cohere-v3-wikipedia-en-scattered-1024d.queries.vec
-vectorDimension 1024 -pollute':
1,566,239,675,863 task-clock:u # 2.851 CPUs
utilized
0 context-switches:u # 0.000 /sec
0 cpu-migrations:u # 0.000 /sec
722,319 page-faults:u # 461.180 /sec
9,378,561,411,799 instructions:u # 2.22 insn per
cycle
# 0.04 stalled cycles
per insn (30.77%)
4,228,223,018,839 cycles:u # 2.700 GHz
(30.78%)
393,219,880,128 stalled-cycles-frontend:u # 9.30% frontend
cycles idle (30.77%)
1,777,249,743,096 branches:u # 1.135 G/sec
(30.78%)
23,578,816,975 branch-misses:u # 1.33% of all
branches (30.77%)
4,449,403,141,862 L1-dcache-loads:u # 2.841 G/sec
(30.77%)
40,782,544,753 L1-dcache-load-misses:u # 0.92% of all
L1-dcache accesses (30.77%)
593,096,321,940 L1-icache-loads:u # 378.675 M/sec
(30.77%)
1,713,082,414 L1-icache-load-misses:u # 0.29% of all
L1-icache accesses (30.78%)
1,908,659,533 dTLB-loads:u # 1.219 M/sec
(30.79%)
349,067,129 dTLB-load-misses:u # 18.29% of all
dTLB cache accesses (30.78%)
179,164,433 iTLB-loads:u # 114.391 K/sec
(30.78%)
21,113,672 iTLB-load-misses:u # 11.78% of all
iTLB cache accesses (30.78%)
549.354958865 seconds time elapsed
1479.501112000 seconds user
64.359562000 seconds sys
```
I find the 4.023 GHz vs 2.700 GHz ("effective clock speed" of the CPU
disturbing -- the box just ran fewer instructions per second overall. Thermal
throttling? Different CPU governor? But @msokolov pointed out that e.g. more
blocking on IO, or thread lock contention, could also manifest this way maybe?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
