> > This metric can be used only for local node, to get size of cache use > org.apache.ignite.internal.processors.cache.CacheMetricsImpl#getCacheSize.
Got it, agree. If there is a lot of data in node that can be rebuilt, percentage may > change very rarely and may not give an estimate of how much time is left. > If we see for example that 50_000 keys are rebuilt once a minute, and we > have 1_000_000_000 keys, then we can have an approximate estimate. What do > you think of that? If the progress is very slow, I don't see issues with tracking it if the percentage float has enough precision. Still, usability of the metric concerns me. In order to estimate remaining time of index rebuild, user should: 1. Gain an understanding that local cache size (CacheMetricsImpl#getCacheSize) should be used as a 100% milestone (it isn't mentioned neither in javadoc nor in JMX method description). 2. Manually calculate sum of all metrics and divide to sum of all cache sizes. On the other hand, % of index rebuild progress is self-descriptive. I don't understand why we tend to make user's life harder. -- Best regards, Ivan On Mon, Aug 10, 2020 at 8:53 PM ткаленко кирилл <tkalkir...@yandex.ru> wrote: > Hi, Ivan! > > For this you can use > org.apache.ignite.cache.CacheMetrics#IsIndexRebuildInProgress > > How can a local number of processed keys can help us to understand when > > index rebuild will be finished? > > This metric can be used only for local node, to get size of cache use > org.apache.ignite.internal.processors.cache.CacheMetricsImpl#getCacheSize. > > We can't compare metric value with cache.size(). First one is node-local, > > while cache size covers all partitions in the cluster. > > If there is a lot of data in node that can be rebuilt, percentage may > change very rarely and may not give an estimate of how much time is left. > If we see for example that 50_000 keys are rebuilt once a minute, and we > have 1_000_000_000 keys, then we can have an approximate estimate. What do > you think of that? > > I find one single metric much more usable. It would be perfect if metric > > value is represented in percentage, e.g. current progress of local node > > index rebuild is 60%. > > 10.08.2020, 19:11, "Ivan Rakov" <ivan.glu...@gmail.com>: > > Folks, > > > > Sorry for coming late to the party. I've taken a look at this issue > during > > review. > > > > How can a local number of processed keys can help us to understand when > > index rebuild will be finished? > > We can't compare metric value with cache.size(). First one is node-local, > > while cache size covers all partitions in the cluster. > > Also, I don't understand why we need to keep separate metrics for all > > caches. Of course, the metric becomes more fair, but obviously harder to > > make conclusions on whether "the index rebuild" process is over (and the > > cluster is ready to process queries quickly). > > > > I find one single metric much more usable. It would be perfect if metric > > value is represented in percentage, e.g. current progress of local node > > index rebuild is 60%. > > > > -- > > Best regards, > > Ivan > > > > On Fri, Jul 24, 2020 at 1:35 PM Stanislav Lukyanov < > stanlukya...@gmail.com> > > wrote: > > > >> Got it. I thought that index building and index rebuilding are > essentially > >> the same, > >> but now I see that they are different: index rebuilding cares about all > >> indexes at once while index building cares about particular ones. > >> > >> Kirill's approach sounds good. > >> > >> Stan > >> > >> > On 20 Jul 2020, at 14:54, Alexey Goncharuk < > alexey.goncha...@gmail.com> > >> wrote: > >> > > >> > Stan, > >> > > >> > Currently we never build indexes one-by-one - we always use a cache > data > >> > row visitor which either updates all indexes (see > >> IndexRebuildFullClosure) > >> > or updates a set of all indexes that need to catch up (see > >> > IndexRebuildPartialClosure). GIven that, I do not see any need for > >> > per-index rebuild status as this status will be updated for all > outdated > >> > indexes simultaneously. > >> > > >> > Kirill's approach for the total number of processed keys per cache > seems > >> > reasonable to me. > >> > > >> > --AG > >> > > >> > пт, 3 июл. 2020 г. в 10:12, ткаленко кирилл <tkalkir...@yandex.ru>: > >> > > >> >> Hi, Stan! > >> >> > >> >> Perhaps it is worth clarifying what exactly I wanted to say. > >> >> Now we have 2 processes: building and rebuilding indexes. > >> >> > >> >> At moment, we have some metrics for rebuilding indexes: > >> >> "IsIndexRebuildInProgress", "IndexBuildCountPartitionsLeft". > >> >> > >> >> I suggest adding another metric "Indexrebuildkeyprocessed", which > will > >> >> allow you to determine how many records are left to rebuild for > cache. > >> >> > >> >> I think your comments are more about building an index that may need > >> more > >> >> metrics, but I think you should do it in a separate ticket. > >> >> > >> >> 03.07.2020, 03:09, "Stanislav Lukyanov" <stanlukya...@gmail.com>: > >> >>> If multiple indexes are to be built "number of indexed keys" > metric may > >> >> be misleading. > >> >>> > >> >>> As a cluster admin, I'd like to know: > >> >>> - Are all indexes ready on a node? > >> >>> - How many indexes are to be built? > >> >>> - How much resources are used by the index building (how many > threads > >> >> are used)? > >> >>> - Which index(es?) is being built right now? > >> >>> - How much time until the current (single) index building finishes? > >> Here > >> >> "time" can be a lot of things: partitions, entries, percent of the > >> cache, > >> >> minutes and hours > >> >>> - How much time until all indexes are built? > >> >>> - How much does it take to build each of my indexes / a single > index of > >> >> my cache on average? > >> >>> > >> >>> I think we need a set of metrics and/or log messages to solve all > of > >> >> these questions. > >> >>> I imaging something like: > >> >>> - numberOfIndexesToBuild > >> >>> - a standard set of metrics on the index building thread pool (do > we > >> >> already have it?) > >> >>> - currentlyBuiltIndexName (assuming we only build one at a time > which > >> is > >> >> probably not true) > >> >>> - for the "time" metrics I think percentage might be the best as > it's > >> >> the easiest to understand; we may add multiple metrics though. > >> >>> - For "time per each index" I'd add detailed log messages stating > how > >> >> long did it take to build a particular index > >> >>> > >> >>> Thanks, > >> >>> Stan > >> >>> > >> >>>> On 26 Jun 2020, at 12:49, ткаленко кирилл <tkalkir...@yandex.ru> > >> >> wrote: > >> >>>> > >> >>>> Hi, Igniters. > >> >>>> > >> >>>> I would like to know if it is possible to estimate how much the > index > >> >> rebuild will take? > >> >>>> > >> >>>> At the moment, I have found the following metrics [1] and [2] and > >> >> since the rebuild is based on caches, I think it would be useful to > know > >> >> how many records are processed in indexing. This way we can > estimate how > >> >> long we have to wait for the index to be rebuilt by subtracting [3] > and > >> how > >> >> many records are indexed. > >> >>>> > >> >>>> I think we should add this metric [4]. > >> >>>> > >> >>>> Comments, suggestions? > >> >>>> > >> >>>> [1] - https://issues.apache.org/jira/browse/IGNITE-12184 > >> >>>> [2] - > >> >> > >> > > org.apache.ignite.internal.processors.cache.CacheGroupMetricsImpl#idxBuildCntPartitionsLeft > >> >>>> [3] - org.apache.ignite.cache.CacheMetrics#getCacheSize > >> >>>> [4] - org.apache.ignite.cache.CacheMetrics#getNumberIndexedKeys > >> >> >
