Yes, such races during pre-loading are possible even if you configured 0 backups.
On Fri, Jun 30, 2017 at 7:31 PM, charly <[email protected]> wrote: > It helps a lot. Thanks for the detailed example. I have a last question. > Would it still be a potential issue to lower the queue size if we don't > have any backup? Like if a new node pre-loads from the primary, any new > changes will also be populated to the soon-to-be new primary? > > Sent from my iPhone > > On Jun 30, 2017, at 11:27 AM, Ilya Lantukh [via Apache Ignite Users] <[hidden > email] <http:///user/SendEmail.jtp?type=node&node=14183&i=0>> wrote: > > Hi Charly, > > There is no special thread in charge of clearing the queue. Entry is > cleared when you try to add new entry to the queue, but it is already full. > So, once queue becomes full, it will always stay full, and lowering queue > size obviously reduces heap consumption. If cache is destroyed, the queue > should be cleared immediately. > > However, if queue size is too low, it may lead to data inconsistency on > unstable topology. > > Here is an example: > Node A just joined the cluster, became primary node for some partitions > and started preloading from node B, which is backup. Node B sends value V > for key K, but before it was received and processed on node A user executed > cache.remove(K). > - If queue size is 0, node A now doesn't have any information that entry > was just removed and has to store (K, V) pair from node B. On the other > hand, node B will remove K and now primary and backup nodes have different > data. > - If queue size is N > 0, node A will save a tombstone entry for K. It > will have all necessary information to understand that (K, V) pair from > node B is older than current value and don't store it. Tombstone will be > cleared after N other keys are removed. > > Hope this helps. > > > On Thu, Jun 22, 2017 at 2:11 AM, charly <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=14182&i=0>> wrote: > >> Hey everyone, >> >> We use Ignite 1.8 and see a difference in heap used when lowering the >> value >> for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE. I'm not sure to understand >> what >> it's going to happen if we decrease that value on production. Could >> someone >> clear that up for me please? From what I understand, deleted entries go in >> that queue for cleaning, where one thread per cache is in charge of >> cleaning >> the queue. But what if the queue is full? I might have missed that >> information in the documentation, my bad if I did. >> >> Maybe on the same subject, is it going to slow down the free up of the >> heap >> after cache.destroy()? We have a use-case where we delete 4 caches at the >> same time containing ~500k entries heavily indexed. When doing so, it >> takes >> few minutes for the heap to decrease as you can see here >> http://imgur.com/a/kPsSc . We would not want to extend the time to free >> up >> memory even more. >> >> More information about our setup: >> - on heap (we'll eventually move to off heap but back then we could not >> make Ignite 1.8 to free up memory at all with our version of glibc. It >> would >> work with jmalloc but that was not permitted on production at that time). >> - no backup >> - atomicity mode: atomic >> - cache mode: partitioned >> - cluster of 4 nodes >> - jvm options: -J-DIGNITE_LONG_OPERATIONS_DUMP_TIMEOUT=300000 -J-Xms12g >> -J-Xmx12g -J-server -J-XX:+UseParNewGC -J-XX:+UseConcMarkSweepGC >> -J-XX:+UseTLAB -J-XX:NewSize=128m -J-XX:MaxNewSize=128m >> -J-XX:MaxTenuringThreshold=0 -J-XX:SurvivorRatio=1024 >> -J-XX:+UseCMSInitiatingOccupancyOnly -J-XX:CMSInitiatingOccupancyFr >> action=10 >> -J-XX:MaxGCPauseMillis=1000 -J-XX:InitiatingHeapOccupancyPercent=10 >> -J-XX:+UseCompressedOops -J-XX:ParallelGCThreads=8 -J-XX:ConcGCThreads=8 >> -J-XX:+PrintGCDetails -J-XX:+PrintGCTimeStamps -J-XX:+PrintGCDateStamps >> -J-XX:+UseGCLogFileRotation -J-XX:NumberOfGCLogFiles=10 >> -J-XX:GCLogFileSize=100M -J-Xloggc:/tmp/ignite-gc.log >> -J-Dcom.sun.management.jmxremote.port=49130 >> >> Thanks for your help, >> Charly >> >> >> >> >> >> -- >> View this message in context: http://apache-ignite-users.705 >> 18.x6.nabble.com/Small-value-for-IGNITE-ATOMIC-CACHE-DELETE >> -HISTORY-SIZE-tp14037.html >> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >> > > > > -- > Best regards, > Ilya > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://apache-ignite-users.70518.x6.nabble.com/Small- > value-for-IGNITE-ATOMIC-CACHE-DELETE-HISTORY-SIZE-tp14037p14182.html > To unsubscribe from Small value for IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE, > click here. > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > ------------------------------ > View this message in context: Re: Small value for > IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE > <http://apache-ignite-users.70518.x6.nabble.com/Small-value-for-IGNITE-ATOMIC-CACHE-DELETE-HISTORY-SIZE-tp14037p14183.html> > > Sent from the Apache Ignite Users mailing list archive > <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com. > -- Best regards, Ilya
