Hm, I don't know, honestly. We need to look at the code or somebody else can give us a hint.
On Mon, Sep 19, 2022 at 2:55 PM 38797715 <[email protected]> wrote: > if native persistence enabled, the result is the same. > > It seems that there is a bug in the calculation of the "PagesFillFactor" > metrics? > 在 2022/9/19 17:30, Alexander Polovtcev 写道: > > Do you have native persistence enabled? > > On Mon, Sep 19, 2022 at 11:57 AM 38797715 <[email protected]> wrote: > >> 2.10 and 2.13, the same result. >> 在 2022/9/19 16:54, Alexander Polovtcev 写道: >> >> Which version of Ignite do you use? >> >> On Mon, Sep 19, 2022 at 11:38 AM 38797715 <[email protected]> wrote: >> >>> for example: >>> >>> start a node by ignite.sh. >>> >>> then: >>> >>> CREATE TABLE City ( >>> ID INT, >>> Name VARCHAR, >>> CountryCode CHAR(3), >>> District VARCHAR, >>> Population INT, >>> PRIMARY KEY (ID, CountryCode) >>> ) WITH "template=partitioned, backups=1, affinityKey=CountryCode, >>> CACHE_NAME=City, KEY_TYPE=demo.model.CityKey, VALUE_TYPE=demo.model.City"; >>> >>> >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (1,'Kabul','AFG','Kabol',1780000); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (2,'Qandahar','AFG','Qandahar',237500); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (3,'Herat','AFG','Herat',186800); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (4,'Mazar-e-Sharif','AFG','Balkh',127800); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (5,'Amsterdam','NLD','Noord-Holland',731200); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (6,'Rotterdam','NLD','Zuid-Holland',593321); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (7,'Haag','NLD','Zuid-Holland',440900); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (8,'Utrecht','NLD','Utrecht',234323); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (9,'Eindhoven','NLD','Noord-Brabant',201843); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (10,'Tilburg','NLD','Noord-Brabant',193238); >>> INSERT INTO City(ID, Name, CountryCode, District, Population) VALUES >>> (11,'Groningen','NLD','Groningen',172701); >>> >>> SELECT * FROM sys.metrics WHERE name LIKE 'io.dataregion.default%'; >>> >>> io.dataregion.default.OffHeapSize 104857600 >>> io.dataregion.default.PhysicalMemoryPages 2066 >>> io.dataregion.default.EmptyDataPages 0 >>> io.dataregion.default.UsedCheckpointBufferSize 0 >>> io.dataregion.default.TotalThrottlingTime 0 >>> io.dataregion.default.PagesReplaced 0 >>> io.dataregion.default.EvictionRate 0 >>> io.dataregion.default.InitialSize 104857600 >>> io.dataregion.default.DirtyPages 0 >>> io.dataregion.default.MaxSize 8589934592 >>> io.dataregion.default.PagesWritten 0 >>> io.dataregion.default.PagesReplaceRate 0 >>> io.dataregion.default.PagesRead 0 >>> io.dataregion.default.PagesFillFactor 0.9997031688690186 >>> io.dataregion.default.TotalAllocatedPages 2066 >>> io.dataregion.default.PhysicalMemorySize 8511920 >>> io.dataregion.default.PagesReplaceAge 0 >>> io.dataregion.default.AllocationRate 2066 >>> io.dataregion.default.OffheapUsedSize 8511920 >>> io.dataregion.default.LargeEntriesPagesCount 0 >>> io.dataregion.default.TotalAllocatedSize 8511920 >>> io.dataregion.default.CheckpointBufferSize 0 >>> >>> TotalUsedSize - (TotalAllocatedSize - PagesFillFactor * >>> TotalAllocatedSize) >>> >>> = 8511920 - (8511920 - 0.9997031688690186 * 8511920) = 8509392 >>> >>> >>> delete from city; >>> >>> SELECT * FROM sys.metrics WHERE name LIKE 'io.dataregion.default%'; >>> >>> io.dataregion.default.OffHeapSize 104857600 >>> io.dataregion.default.PhysicalMemoryPages 2075 >>> io.dataregion.default.EmptyDataPages 0 >>> io.dataregion.default.UsedCheckpointBufferSize 0 >>> io.dataregion.default.TotalThrottlingTime 0 >>> io.dataregion.default.PagesReplaced 0 >>> io.dataregion.default.EvictionRate 0 >>> io.dataregion.default.InitialSize 104857600 >>> io.dataregion.default.DirtyPages 0 >>> io.dataregion.default.MaxSize 8589934592 >>> io.dataregion.default.PagesWritten 0 >>> io.dataregion.default.PagesReplaceRate 0 >>> io.dataregion.default.PagesRead 0 >>> io.dataregion.default.PagesFillFactor 1.0 >>> io.dataregion.default.TotalAllocatedPages 2075 >>> io.dataregion.default.PhysicalMemorySize 8549000 >>> io.dataregion.default.PagesReplaceAge 0 >>> io.dataregion.default.AllocationRate 9 >>> io.dataregion.default.OffheapUsedSize 8549000 >>> io.dataregion.default.LargeEntriesPagesCount 0 >>> io.dataregion.default.TotalAllocatedSize 8549000 >>> io.dataregion.default.CheckpointBufferSize 0 >>> >>> >>> TotalUsedSize - (TotalAllocatedSize - PagesFillFactor * >>> TotalAllocatedSize) >>> >>> =8549000 - (8549000 - 1.0 * 8549000) = 8549000 >>> >>> Instead, the value becomes larger? >>> >>> >>> 在 2022/9/19 15:13, Alexander Polovtcev 写道: >>> >>> Sorry, I messed up the metric names a little bit, we should use the size >>> metrics, not the page metrics. So the correct formula would be: >>> `TotalUsedSize - (TotalAllocatedSize - PagesFillFactor * >>> TotalAllocatedSize)` >>> >>> On Sun, Sep 18, 2022 at 4:29 PM 38797715 <[email protected]> wrote: >>> >>>> I've checked TotalUsedPages in ignite.dataRegionMetrics(), it is found >>>> that the logic of this metric is consistent with OffheapUsedSize, that is, >>>> the value will not be updated after data deletion. >>>> 在 2022/9/18 19:42, Alexander Polovtcev 写道: >>>> >>>> Hello, you can check out "TotalUsedPages" and "TotalUsedSize" metrics >>>> that can be found in the Data Region metrics. However, please keep in mind >>>> that this does not take page fragmentation into account. >>>> >>>> On Sat, Sep 17, 2022 at 2:35 PM 38797715 <[email protected]> wrote: >>>> >>>>> Hi team, >>>>> >>>>> We found that if the delete operation is executed, the memory space >>>>> corresponding to the data will not be released, and this space will be >>>>> reused later. >>>>> >>>>> Therefore, metrics such as OffheapUsedSize will become inaccurate. >>>>> >>>>> So, how to calculate the exact amount of memory occupied by data in a >>>>> cache or a dataregion? >>>>> >>>> >>>> >>>> -- >>>> With regards, >>>> Aleksandr Polovtcev >>>> >>>> >>> >>> -- >>> With regards, >>> Aleksandr Polovtcev >>> >>> >> >> -- >> With regards, >> Aleksandr Polovtcev >> >> > > -- > With regards, > Aleksandr Polovtcev > > -- With regards, Aleksandr Polovtcev
