Hello!
I have ran your test and I don't observe any off-heap growth:
[18:17:14,655][INFO ][grid-timeout-worker-#24%ignite.GrowTest0%][GrowTest0]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=23ad15f7, name=ignite.GrowTest0, uptime=00:00:40.015]
^-- H/N/C [hosts=1, nodes=1, CPUs=8]
^-- CPU [cur=31,67%, avg=31,69%, GC=0,2%]
* ^-- PageMemory [pages=3314]*
^-- Heap [used=2419MB, free=66,03%, comm=2796MB]
^-- Non heap [used=48MB, free=-1%, comm=53MB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=6, qSize=0]
^-- System thread pool [active=0, idle=8, qSize=0]
and then after a few minutes:
[18:19:14,710][INFO ][grid-timeout-worker-#24%ignite.GrowTest0%][GrowTest0]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=23ad15f7, name=ignite.GrowTest0, uptime=00:02:40.071]
^-- H/N/C [hosts=1, nodes=1, CPUs=8]
^-- CPU [cur=31,93%, avg=31,67%, GC=0,2%]
* ^-- PageMemory [pages=3317]*
^-- Heap [used=1442MB, free=79,74%, comm=2828MB]
^-- Non heap [used=48MB, free=-1%, comm=53MB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=0, qSize=0]
^-- System thread pool [active=0, idle=6, qSize=0]
There's barely any growth at all.
Maybe it's something in your post-processing via annotations that add
something to classes that causes them to bloat in size?
Regards,
--
Ilya Kasnacheev
пт, 7 сент. 2018 г. в 10:39, Serg <[email protected]>:
> Hi All
>
> We have got a problem with growing off-heap on each sync data.
>
> *Precondition *
> We have ignite in memory cache. with key Integer and value Contact
> Each hour we sync full data in cache via api cache.put(key, value); (cache
> full load)
> Also we have increment update via the api cache.put(key, value);
> (streaming)
>
> Data:
>
> @Builder
> @Data
> @AllArgsConstructor
> @NoArgsConstructor
> private class Contact {
> private int id;
> private Map<String, <b>BinaryObject*> customFields;
> }
>
> @Data
> private class CustomInfo {
>
> private Date updateTime;
> private boolean deleted;
>
> }
>
> /if we do not use BinaryObject in Contact all works as expected./
>
> *Problem*
> Off-heap memory allocate time to time for the same key new memory pages.
> As a result, memory is constantly growing, but the data size is not
> increasing.
> If we stop ignite and load data again off-heap has normal size.
>
>
> Example to reproduce problem
>
> @Test
> public void testStreamAndUpdateContact() {
> HashMap<String, BinaryObject> baseCustomFields = new HashMap<>();
> baseCustomFields.put("customX",
>
> ignite.binary().builder(CustomInfo.class.getName()).build());
> baseCustomFields.put("customY",
> ignite.binary().builder(CustomInfo.class.getName()).build());
> baseCustomFields.put("customY1",
> ignite.binary().builder(CustomInfo.class.getName()).build());
> baseCustomFields.put("customY2",
> ignite.binary().builder(CustomInfo.class.getName()).build());
> baseCustomFields.put("customY3",
> ignite.binary().builder(CustomInfo.class.getName()).build());
>
> {
> CacheConfiguration<Integer, Contact> cfg = new
> CacheConfiguration<>("contactsEx");
> cfg.setCacheMode(CacheMode.PARTITIONED);
> cfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
> IgniteCache<Integer, Contact> cache =
> ignite.getOrCreateCache(cfg);
> }
>
> for (int z = 0; z < 50_000; z++) {
>
> //Fill base contacts
> {
> IgniteCache<Integer, Contact> cache =
> ignite.getOrCreateCache("contactsEx");
> IntStream.range(0, 10_000).forEach(counter -> {
> Lock lock = cache.lock(counter);
> lock.lock();
> try {
> cache.put(counter,
> Contact.builder()
> .id(counter)
> .customFields(baseCustomFields)
> .build());
> } finally {
> lock.unlock();
> }
>
> });
> }
>
> //////events
>
> ExecutorService executorService =
> Executors.newSingleThreadExecutor();
> executorService.execute(() -> {
> {
> IgniteCache<Integer, Contact> cache =
> ignite.getOrCreateCache("contactsEx");
> IntStream.range(0, 10_000).forEach(counter -> {
>
> Lock lock = cache.lock(counter);
> lock.lock();
> try {
>
> Contact c = cache.get(counter);
> Map<String, BinaryObject> customFields =
> c.getCustomFields();
> customFields.put("customY", ignite.binary()
>
> .builder(CustomInfo.class.getName()).setField("updateTime", new Date())
> .build());
> cache.put(counter, c);
>
> } finally {
> lock.unlock();
> }
>
> });
>
> }
> executorService.shutdown();
> });
>
>
> }
> }
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
