Hi, If only TermsEnum array is not thread safe, maybe it's worth to consider sticking only it (or all other required arrays) to a thread? Heap dump report showed me that a lot of space is taken by versions NumericDocValues array.
-- Paweł W dniu poniedziałek, 16 marca 2015 00:03:59 UTC+1 użytkownik Jörg Prante napisał: > > It is not thread safe because of the TermsEnum array, which can not be > shared between threads. By not sharing, a thread can reuse the array, which > avoids expensive reinitialization. > > The utility class was introduced at > > https://github.com/elastic/elasticsearch/issues/6212 > > and from what I understand this replaced the previous version ID lookup by > bloom filters (which comes with a very noticeable RAM cost) > > Maybe you have lots of segments? > > Sometimes, ThreadLocals go crazy because of Java issues, and they are hard > to clean up. So I think if you can post some more detailed information > about what you have seen and what OS, JVM, and ES versions you use, it > would be helpful. > > Jörg > > On Sun, Mar 15, 2015 at 10:16 PM, Paweł Róg <[email protected] > <javascript:>> wrote: > >> Hi, >> Can anyone shortly describe why class PerThreadIDAndVersionLookup is not >> thread safe and what is needed to make it thread safe? I'm wondering if it >> is possible to keep only single instance of VersionLookup and make it not >> stick to a thread. I see waste of big chunk of memory in JVM only because >> of class PerThreadIDAndVersionLookup. >> >> Thanks a lot for any suggestions/advises. >> >> -- >> Paweł >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_oPJcs4LqyYA%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_oPJcs4LqyYA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9af61116-2abb-4995-aa0c-86a39b046337%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
