Sergi, However if we implement non snapshotable data structure to hold indexes (skip list or b-tree) then it will be feasible to optimize point 1) - working with off-heap data with BinaryObjectOffHeapImpl. Am I right?
— Denis > On Apr 21, 2016, at 5:27 PM, Sergi Vladykin <[email protected]> wrote: > > 1. In general it is not possible to keep BinaryObject in "attached" state, > because at the query end we release snapshots and the entry can be > concurrently removed with offheap memory deallocation, while result row is > still needed. > > 2. Of course there are multiple alternatives (skiplist or b+tree), but we'll > need to implement them over offheap memory, this will take some time... > > Sergi > > > Sergi > > 2016-04-21 16:48 GMT+03:00 Denis Magda <[email protected] > <mailto:[email protected]>>: > Sergi, Igniters, > > Big deployments store enormous amount of data in off-heap. If to keep using > such deployments under high load executing SQL queries and basic cache > operations in parallel then it’s easy to see that Java heap pressure grows > due to the bigger number of short live objects creation. > > I’ve been investigating Ignite code and noted the following > > 1) During SQL query execution off-heap data is unswapped and stored in > BinaryObjectImpl wrapper for columns comparison. Also as I see the same > happens when an index is updated. > > 2) Off-heap index tree is snapshotable. Taking into account hight update and > query rates, Java heap will be populated with much more short snapshots. > > Actually I think that we can try to lower the impact caused by 1) by trying > reusing BinaryObjectOffHeapImpl that allows to access fields directly working > with off-heap. What do you think, any obstacles? > > In regards to the second. Is there any alternatives to snapshotable tree in > general? > > — > Denis >
