On Thu, Oct 24, 2024 at 9:07 AM Jason Gerlowski <gerlowsk...@gmail.com> wrote:
> Hi! > > I'm all for (eventually) nudging folks towards HttpCSP. It's a "Good > Thing", conceptually, to hide ZK! > > But it sounds like there are still some significant (if few) downsides > to using HttpCSP (the "all live nodes disappear" case being the > biggest, to my mind). IMO we should close the more significant of > those gaps before we start nudging folks in that direction. If I was > a client I'd be pretty ticked if I switched to the "recommended" > HttpCSP and had to restart my client pods after every > sufficiently-long network blip. Agreed, assuming this code inspection based finding is true. Probably needs a test reproducer first. > WDYT about closing some of those gaps > before tweaking example code, javadocs, etc.? > In a perfect world -- yes. Will I have the time for this or my colleagues.... mmm.... remains to be seen. Meanwhile, Zk ClusterStateProvider is not some masterpiece of engineering. At extreme collection scale it puts load on ZK, consumes memory & CPU synchronizing its state for collections it might actually never have to talk to. I will file a JIRA issue or two. I think only this risk of losing live nodes is the only potential blocker. > That said - really glad to hear about some of 9.8's HttpCSP > performance improvements! Kudos to Haythem and Aparna! Any chance > you have pointers to that work? (I grepped around a bit in > CHANGES.txt but didn't see anything at a glance...) Maybe your checkout is out of sync LOL? In 9.8 simply search for Aparna and you'll find her listed 3 times under the Optimizations section. Relevant to this: SOLR-14985 (the big most serious one), SOLR-17381. Haythem's role was longer ago separating out solrj-zookeeper so that users aren't forced to depend on it. Pretty soon it'll have yet another dependency -- Curator. > Are the perf > improvements enough to bring HttpCSP level with ZkClientCSP > performance wise, or is there still a gap? I think so. The perf issues observed were due to many extra needless requests to Solr that were avoidable. And some were returning too much. Now we actually have a test to ensure we don't have a regression of this nature. > I've been on teams in the > past that wanted to use HttpCSP but that were held back by some of the > performance issues.. Not surprised. ~ David
