mikemccand commented on issue #15662:
URL: https://github.com/apache/lucene/issues/15662#issuecomment-3891212594
Another update, but still no smoking gun yet:
* I've made my iterating faster by just running the search tasks, and
reusing the very-slow-to-create fixed-segments index. This could in theory
hide the smoking gun if somehow it is pre-index but that's hard to believe
(some system package somehow caused index to be written in a mysterious
massive-slowdown-while-searching yet passes the smell tests we do (e.g. at
least confirming index has same segment geometry as prior night)). I'll at
least check whether size of index changed.
* One thing I discovered is the JFR profiling we turn on in the nightly
tasks is non-trivial cost: it brings total QPS of each JVM from ~41-42 down to
~37-38, maybe ~7-12% impact. I'll try turning it off once benchy is up and
running again.
* I think I finally got all packages downgraded to rough timestamp of my
whole system update prior to the 1/29 update, and ran again, and things were
still slow. I hit scary problems -- there were some "malformed" PGP signatures
on packages, a known issue that was maybe a tra nsient gnupg issue (I had also
hit this on upgrading, I think on 1/29, maybe prior, as well). I tried the
published workarounds and never succeeded and instead asked pacman to "screw it
and barrel on through" with a scary `/etc/pacman.conf` eidt (ignore the bad
sigs, I know, not secure, but....!). I also hit confilcting files in the
filesystem, some of which pacman knew were a given package, others were
probably my fault hand-rolling Python package installs into the system python
(also not good practice!). Anyway, the system was still slow, but I hadn't
rebooted, so I rebooted and box is not coming back up. I'll try to resuscitate
once I'm home tonite...sigh.
So net/net this remains mysterious ... in theory that last test I did should
be identical to 1/22 config: same Lucene, same kernel, same java, same all
other system packages (but not rebooted -- so maybe something in there cared
about reboot).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
