I seem to be in the minority here :)
Fine, let's make it as clear as possible which metric method
(localCacheSize) should be called in order to retrieve a 100% progress
milestone.
I've left comments in the PR.
On Tue, Aug 11, 2020 at 4:31 PM Nikolay Izhikov wrote:
> > I propose to stick with a
> I propose to stick with a cache-group level metric (e.g.
> getIndexBuildProgress)
+1
> that returns a float from 0 to 1, which is calculated as [processedKeys] /
> [localCacheSize].
From my point of view, we shouldn’t do calculations on the Ignite side if we
can avoid it.
I’d rather
>
> As a compromise, I can add jmx methods (rebuilding indexes in the process
> and the percentage of rebuilding) for the entire node, but I tried to find
> a suitable place and did not find it, tell me where to add it?
I have checked existing JMX beans. To be honest, I struggle to find a
Hi, Ivan!
What precision would be sufficient?
> If the progress is very slow, I don't see issues with tracking it if the
> percentage float has enough precision.
I think we can add a mention getting cache size.
> 1. Gain an understanding that local cache size
> (CacheMetricsImpl#getCacheSize)
>
> This metric can be used only for local node, to get size of cache use
> org.apache.ignite.internal.processors.cache.CacheMetricsImpl#getCacheSize.
Got it, agree.
If there is a lot of data in node that can be rebuilt, percentage may
> change very rarely and may not give an estimate of how
Hi, Ivan!
For this you can use
org.apache.ignite.cache.CacheMetrics#IsIndexRebuildInProgress
> How can a local number of processed keys can help us to understand when
> index rebuild will be finished?
This metric can be used only for local node, to get size of cache use
Folks,
Sorry for coming late to the party. I've taken a look at this issue during
review.
How can a local number of processed keys can help us to understand when
index rebuild will be finished?
We can't compare metric value with cache.size(). First one is node-local,
while cache size covers all
Stan,
Currently we never build indexes one-by-one - we always use a cache data
row visitor which either updates all indexes (see IndexRebuildFullClosure)
or updates a set of all indexes that need to catch up (see
IndexRebuildPartialClosure). GIven that, I do not see any need for
per-index rebuild
Hi, Stan!
Perhaps it is worth clarifying what exactly I wanted to say.
Now we have 2 processes: building and rebuilding indexes.
At moment, we have some metrics for rebuilding indexes:
"IsIndexRebuildInProgress", "IndexBuildCountPartitionsLeft".
I suggest adding another metric
If multiple indexes are to be built "number of indexed keys" metric may be
misleading.
As a cluster admin, I'd like to know:
- Are all indexes ready on a node?
- How many indexes are to be built?
- How much resources are used by the index building (how many threads are used)?
- Which index(es?)
Hi, Igniters.
I would like to know if it is possible to estimate how much the index rebuild
will take?
At the moment, I have found the following metrics [1] and [2] and since the
rebuild is based on caches, I think it would be useful to know how many records
are processed in indexing. This
11 matches
Mail list logo
