Sorry for being unclear, the TermsEnum array is one (the most important) of the arrays for iteration, the other arrays are also not thread safe - you can view all the private class variables as a thread-private cache. NumericDocValues is the key component for retrieving the version.
Jörg On Mon, Mar 16, 2015 at 2:29 PM, Paweł Róg <[email protected]> wrote: > Hi, > If only TermsEnum array is not thread safe, maybe it's worth to consider > sticking only it (or all other required arrays) to a thread? > Heap dump report showed me that a lot of space is taken by versions > NumericDocValues array. > > -- > Paweł > > W dniu poniedziałek, 16 marca 2015 00:03:59 UTC+1 użytkownik Jörg Prante > napisał: >> >> It is not thread safe because of the TermsEnum array, which can not be >> shared between threads. By not sharing, a thread can reuse the array, which >> avoids expensive reinitialization. >> >> The utility class was introduced at >> >> https://github.com/elastic/elasticsearch/issues/6212 >> >> and from what I understand this replaced the previous version ID lookup >> by bloom filters (which comes with a very noticeable RAM cost) >> >> Maybe you have lots of segments? >> >> Sometimes, ThreadLocals go crazy because of Java issues, and they are >> hard to clean up. So I think if you can post some more detailed information >> about what you have seen and what OS, JVM, and ES versions you use, it >> would be helpful. >> >> Jörg >> >> On Sun, Mar 15, 2015 at 10:16 PM, Paweł Róg <[email protected]> wrote: >> >>> Hi, >>> Can anyone shortly describe why class PerThreadIDAndVersionLookup is >>> not thread safe and what is needed to make it thread safe? I'm wondering if >>> it is possible to keep only single instance of VersionLookup and make it >>> not stick to a thread. I see waste of big chunk of memory in JVM only >>> because of class PerThreadIDAndVersionLookup. >>> >>> Thanks a lot for any suggestions/advises. >>> >>> -- >>> Paweł >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_ >>> oPJcs4LqyYA%40mail.gmail.com >>> <https://groups.google.com/d/msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_oPJcs4LqyYA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/9af61116-2abb-4995-aa0c-86a39b046337%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/9af61116-2abb-4995-aa0c-86a39b046337%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG41vXVG5q-ayQtSGVuj8zhkUe%3DXbavRxkH%2BQYP0HD%3Dig%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
