hll1213181368 commented on PR #3270: URL: https://github.com/apache/kvrocks/pull/3270#issuecomment-3596125472
> > @hll1213181368 What do we need to stop the key scanning when the master becomes the slave role? I feel it's a bit strange to do that. > > @git-hulk Because key scanning is performed slot-by-slot, it is time-consuming. For example, one four shard cluster, with each shard containing approximately 1TB of data, key scanning would take about 6 hours. We implemented parallelization for key scanning, achieving a parallelism equal to the number of CPU cores, which improved performance by 6 times, but increased CPU utilization to 90%. A parallelism of the half number of CPU cores improved performance by 3 times, but increased CPU utilization to 50%. > > Under normal scene, We execute key scans on slave nodes. To prevent excessive CPU load during master-slave failover if a switchover occurs while the new master node is still performing key scanning tasks. I will commit a pr to parallel execute key sanning. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
