I will not have the time for this today but took a quick look and I think these external dependencies on hppc can be removed after the work Bruno has done to port some of these utility classes to the core. I'd also move the entire Lucene hppc fork under internal and only expose it to other Lucene modules that need it - would have to verify that no class is part of the public API but I don't think it is (in spatial3d and spatial-extras).
Dawid On Sat, May 25, 2024 at 10:08 PM Dawid Weiss <dawid.we...@gmail.com> wrote: > > Hi Chris, > > Since Elasticsearch is deployed as a module, then we need to update to >> hppc 0.9.1 [2], but unfortunately this is not straightforward. In fact, >> Ryan has a PR open [3] for the past 2 years without completion! The >> iteration order of some collection types in hppc 0.9.x [*] is tickling some >> inadvertent order dependencies in Elasticsearch. It may take some time to >> track these down and fix them. >> > > I understand it's a pain if the order changes from run to run but I don't > see a way this can be avoided ([1] is the issue you mentioned on gh). Tests > (and code) shouldn't rely on map/set ordering, although I realize it may be > difficult to weed out in such a large codebase. > > For what it's worth, the next version of HPPC will be a proper module > (with com.carrotsearch.hppc id). Would it change anything/ make it easier > if I renamed it to just 'hppc'? > > I wonder if others may run into either or both of these issues, as we have >> in Elasticsearch, if we release 9.11 with this change? >> > > That's why I wasn't entirely sold on having HPPC as the dependency from > Lucene when Bruno mentioned it recently - the jar/module hell will surface > sooner than later... Maybe it'd be a better idea to just copy what's needed > to the core jar and expose those packages to other Lucene modules (so that > there is no explicit dependency on HPPC at all)? Bruno copied a lot of > those classes already anyway - don't know how much of it is left to copy to > drop the dependency. > > Dawid > > [1] https://github.com/carrotsearch/hppc/issues/228 > [2] > https://github.com/carrotsearch/hppc/commit/d569a8944091844c62349646f8eeaf35ebfb5ba6 > > >> >> -Chris. >> >> [1] https://github.com/apache/lucene/pull/13392 >> [2] https://github.com/elastic/elasticsearch/pull/109006 >> [3] https://github.com/elastic/elasticsearch/pull/84168 >> >> [*] HPPC-186: A different strategy has been implemented for collision >> avalanche avoidance. This results in removal of Scatter* maps and sets and >> their unification with their Hash* counterparts. This change should not >> affect any existing code unless it relied on static, specific ordering of >> keys. A side effect of this change is that key/value enumerators will >> return a different ordering of their container's values on each invocation. >> If your code relies on the order of values in associative arrays, it must >> order them after they are retrieved. (Bruno Roustant). >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >>
