If Ignite can hold all the records in memory it won't be reading pages from disk (the page replacement). Thus, it's expected that the 12GB performs better than the 5GB case.
Btw, are you planning to reload the whole data set into Ignite on potential cluster restarts? Each node loses a subset of the data located in the swap space on restarts. - Denis On Thu, Aug 13, 2020 at 7:36 PM 38797715 <38797...@qq.com> wrote: > Hi Denis, > > We did a test, in the same environment (8G RAM, 10G swap partition) and > the same configuration (2G Heap, enable persistence, data volume is about > 6G), the only difference is that the maxSize size is different, which is > configured as 5G and 12G respectively. We found that the performance of the > scenario with maxSize = 12G is better than that of the scenario with > maxSize = 5G, and the write performance is improved by more than 10%. > > I suspect that if the memory data region is large enough to hold all the > data, ignite's page replacement might not be enabled. > > Our test scenarios are limited and may not be convincing. However, I think > that the lack of memory may be the norm. At this time, it may be a good > practice to make full use of the swap mechanism of the OS, which takes up > more disk space but achieves better performance. > 在 2020/8/14 上午8:22, Denis Magda 写道: > > Ignite swapping is based on the swapping mechanism of the OS. So, you > shouldn’t see any difference if enable the OS one directly some way. > > Generally, you should not use swapping of any form as a permanent > persistence layer due to the performance penalty. Once the swapping kicks > in, you should scale out your cluster and wait while the cluster rebalances > a part of the data to a new node. When the rebalancing completes, the > performance will be recovered and swapping won’t longer be needed. > > Denis > > On Thursday, August 13, 2020, 38797715 <38797...@qq.com> wrote: > >> Hi, >> >> We retested and found that if we configured swapPath, as the amount of >> data increased, the write speed was actually slower and slower. If the >> amount of data is large, on average, it is much slower than the scenario >> where native persistence is enabled and wal is disabled. >> >> In this way, the use of the swapPath property has no productive value, >> maybe it was an early development function, and now it is a bit out of date. >> >> What I want to ask is, in the case of small physical memory, turning on >> persistence, and then configuring a larger maxSize (using the swap >> mechanism of the OS), is this a solution? In other words, the swap >> mechanism of the OS and the page replacement of Ignite, which is better? >> 在 2020/8/6 下午9:23, Ilya Kasnacheev 写道: >> >> Hello! >> >> I think the performance of swap space should be on par with persistence >> with disabled WAL. >> >> You can submit suggested updates to the documentation if you like. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> ср, 5 авг. 2020 г. в 06:00, 38797715 <38797...@qq.com>: >> >>> Hi Ilya, >>> >>> If so, there are two ways to implement ignite's swap space: >>> 1. maxSize > physical memory, which will use the swap mechanism of the >>> OS, can be used *vm.swappiness* Adjust. >>> 2. Configure the *swapPath* property, which is implemented by Ignite >>> itself, is independent of the OS and has no optimization parameters. >>> There's a choice between these two models, right? Then I think there may >>> be many problems in the description of the document. I hope you can check >>> it again: >>> https://apacheignite.readme.io/docs/swap-space >>> >>> After our initial testing, the performance of swap space is much better >>> than native persistence, so I think this pattern is valuable in some >>> scenarios. >>> 在 2020/8/4 下午10:16, Ilya Kasnacheev 写道: >>> >>> Hello! >>> >>> From the docs: >>> >>> To avoid this situation with the swapping capabilities, you need to : >>> >>> - Set maxSize = bigger_ than_RAM_size, in which case, the OS will >>> take care of the swapping. >>> - Enable swapping by setting the DataRegionConfiguration.swapPath >>> property. >>> >>> >>> I actually think these are either-or. You should either do the first >>> (and configure OS swapping) or the second part. >>> >>> Having said that, I recommend setting proper Native Persistence instead. >>> >>> Regards, >>> -- >>> Ilya Kasnacheev >>> >>> >>> сб, 25 июл. 2020 г. в 04:49, 38797715 <38797...@qq.com>: >>> >>>> Hi, >>>> >>>> https://apacheignite.readme.io/docs/swap-space >>>> >>>> According to the above document, if the physical memory is small, you >>>> can solve this problem by opening the swap space,The specific method is to >>>> configure maxSize to a larger value (i.e. larger than the physical memory), >>>> and the swapPath property needs to be configured. >>>> >>>> But from the test results, the node is terminated. >>>> >>>> I think the correct result should be that even if the amount of data >>>> exceeds the physical memory, the node should still be able to run normally, >>>> but the data is exchanged to the disk. >>>> >>>> I want to know what parameters affect the behavior of this >>>> configuration? *vm.swappiness* or others? >>>> 在 2020/7/24 下午9:55, aealexsandrov 写道: >>>> >>>> Hi, >>>> >>>> Can you please clarify your expectations? You expected that JVM process >>>> will >>>> be killed instead of gracefully stopping? What you are going to achieve? >>>> >>>> BR, >>>> Andrei >>>> >>>> >>>> >>>> -- >>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >>>> >>>> > > -- > - > Denis > >
