Re: Wrong off-heap size is reported for a node

2018-09-27 Thread Dmitriy Pavlov
Fixed, now the fix is in the ignite-2.7 branch. ср, 26 сент. 2018 г. в 23:18, Dmitriy Pavlov : > Oops, my fault. It seems I forgot to cherry-pick to 2.7, but I resolved > ticket with 2.7 assigned. > > I will fix it tomorrow. > > ср, 26 сент. 2018 г. в 21:52, Denis Magda : > >> Thanks, Pavel and t

Re: Wrong off-heap size is reported for a node

2018-09-26 Thread Dmitriy Pavlov
Oops, my fault. It seems I forgot to cherry-pick to 2.7, but I resolved ticket with 2.7 assigned. I will fix it tomorrow. ср, 26 сент. 2018 г. в 21:52, Denis Magda : > Thanks, Pavel and the rest of the Igniters involved. > > That simple usability improvement is a big deal for those who use Ignit

Re: Wrong off-heap size is reported for a node

2018-09-26 Thread Denis Magda
Thanks, Pavel and the rest of the Igniters involved. That simple usability improvement is a big deal for those who use Ignite in production. Are we getting it in 2.7? -- Denis On Wed, Sep 26, 2018 at 10:11 AM Dmitriy Pavlov wrote: > Hi Igniters, > > Thanks to everyone, who has participated in

Re: Wrong off-heap size is reported for a node

2018-09-26 Thread Dmitriy Pavlov
Hi Igniters, Thanks to everyone, who has participated in this discussion and shared their view and ideas. I've merged fix of changes related to logging only. Fixing of cluster metrics can be done in a separate ticket/discussion. Pavel, thank you for your contribution and for answering my questio

Re: Wrong off-heap size is reported for a node

2018-09-24 Thread Pavel Pereslegin
Andrei, I totally agree with you and I think that "ClusterMetrics" should also be fixed, I'm just not sure that we should include this change in the same ticket. пн, 24 сент. 2018 г. в 18:43, aealexsandrov : > > Hi, > > OK, the user can use it to calculate the off-heap. But I think that the > reaso

Re: Wrong off-heap size is reported for a node

2018-09-24 Thread aealexsandrov
Hi, OK, the user can use it to calculate the off-heap. But I think that the reason for your changes to fix the calculation of the nonHeap used in Ignite now. For example now REST return "-1" for nonHeapMemoryMaximum. I think that it can't be used somehow. So REST possible should be updated as you

Re: Wrong off-heap size is reported for a node

2018-09-24 Thread Pavel Pereslegin
Hello Andrei, All these metrics available through JMX (see "DataRegionMetrics" group) [1]. [1] https://apacheignite.readme.io/docs/memory-metrics пн, 24 сент. 2018 г. в 17:58, aealexsandrov : > > Hi Igniters, > > Small notes according to these fix. > > As I see that all logic of calculation off-n

Re: Wrong off-heap size is reported for a node

2018-09-24 Thread aealexsandrov
Hi Igniters, Small notes according to these fix. As I see that all logic of calculation off-neap max size at the moment located in the ackNodeMetrics method in IgniteKernal.java. I think that it isn't ok because this logic should be added to other functionality too. I think that will be better t

Re: Wrong off-heap size is reported for a node

2018-09-12 Thread Alex Plehanov
Hi Dmitriy, Yes, I will finish review tomorrow. ср, 12 сент. 2018 г. в 20:37, Dmitriy Pavlov : > Hi Igniters, > > It seems the issue is a blocker of 2.7. > > We need a volunteer here to run review. > > Alexey P. will you step in? > > Sincerely, > Dmitriy Pavlov > > ср, 29 авг. 2018 г. в 16:16, N

Re: Wrong off-heap size is reported for a node

2018-09-12 Thread Dmitriy Pavlov
Hi Igniters, It seems the issue is a blocker of 2.7. We need a volunteer here to run review. Alexey P. will you step in? Sincerely, Dmitriy Pavlov ср, 29 авг. 2018 г. в 16:16, Nikita Amelchev : > Pavel, > > I think that point 1 is the correct way to calculate the committed > size for a log. I

Re: Wrong off-heap size is reported for a node

2018-08-29 Thread Nikita Amelchev
Pavel, I think that point 1 is the correct way to calculate the committed size for a log. It is already calculated regardless of the metricsEnabled flag. In addition, I suggest more readable log format for data regions in issue comments [1]. [1] https://issues.apache.org/jira/browse/IGNITE-9305

Re: Wrong off-heap size is reported for a node

2018-08-21 Thread Pavel Pereslegin
Hello, Igniters. I assigned ticket [1] created by Denis and want to clarify how to log committed size. The metric offHeapSize (in DataRegionMetricsImpl) is always calculated, but getOffHeapSize returns zero if memory metrics are disabled for this data region. So I see the following options: 1. Mo

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Denis Magda
Vova, the things are even simpler - we have this ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the number equal/comparabel to pageNumber X pageSize. Igniters, if you believe that we need to do more work here then let's do it iteratively. Let's fix the off-heap occupied size the

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Dmitriy Pavlov
Yes, I agree. To calculate free (currently -1) we need to know total & used. Used=sysPageSize*usedPages, but overall should be obtained from all segments and chunks. So it would be a remarkable part of all mem.metrics migration (required by IGNITE-5583). пт, 17 авг. 2018 г. в 23:55, Alex Plehanov

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Alex Plehanov
To be more precise we need to multiply it by page size with system overhead (systemPageSize). If we want to print only used offheap memory, this will solve the problem. If we want to print, for example, currently allocated (commited) offheap memory (if persistence is disabled for data region then o

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Vladimir Ozerov
Folks, We already have this: >>> PageMemory [pages=6997377] Then we can multiply it by page size and get occupied memory. Am I wrong? On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov wrote: > Hi Maxim, > > thank you for stepping in and for finding these issues. Yes, these tickets > are correct.

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Dmitriy Pavlov
Hi Maxim, thank you for stepping in and for finding these issues. Yes, these tickets are correct. I can move https://issues.apache.org/jira/browse/IGNITE-5583 to unassigned if someone would like to implement this change. I will not have enough time to complete it in 1 month (before 2.7 release).

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Maxim Muzafarov
Igniters, Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap memory metrics to new page memory semantics` and related previous disscustions to it [4]. Also we have some additional improvements to CacheMetrics: IGNITE-5490 [2] - `Implement replacement for obsolete CacheMetrics#get

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Dmitriy Pavlov
Hi Igniters, It is not an easy fix, so I'm not sure it is possible to do in 2.7. Offheap size is not reported by VM (it returns -1). To implement it we need totally migrate off-heap memory metrics to durable memory data. I think this issue was reported and I'll find the duplicate. Sincerely, Dm

Re: Wrong off-heap size is reported for a node

2018-08-16 Thread Denis Magda
Yes, it was at the end of my wordy email :) https://issues.apache.org/jira/browse/IGNITE-9305 -- Denis On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan wrote: > Is there a blocker ticket for 2.7? > > On Thu, Aug 16, 2018, 19:59 Denis Magda wrote: > > > Igniters, > > > > Was troubleshooting a

Re: Wrong off-heap size is reported for a node

2018-08-16 Thread Dmitriy Setrakyan
Is there a blocker ticket for 2.7? On Thu, Aug 16, 2018, 19:59 Denis Magda wrote: > Igniters, > > Was troubleshooting an Ignite deployment today and couldn't find out from > the logs what was the actual off-heap space used. > > Those were the given memory resoures (Ignite 2.6): > > [2018-08-16 1

Wrong off-heap size is reported for a node

2018-08-16 Thread Denis Magda
Igniters, Was troubleshooting an Ignite deployment today and couldn't find out from the logs what was the actual off-heap space used. Those were the given memory resoures (Ignite 2.6): [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] Topology snapshot [ver=1, servers=1, clients=0, CP

[jira] [Created] (IGNITE-9305) Wrong off-heap size is reported for a node

2018-08-16 Thread Denis Magda (JIRA)
Denis Magda created IGNITE-9305: --- Summary: Wrong off-heap size is reported for a node Key: IGNITE-9305 URL: https://issues.apache.org/jira/browse/IGNITE-9305 Project: Ignite Issue Type: Task