vagetablechicken commented on issue #3624:
URL:
https://github.com/apache/incubator-doris/issues/3624#issuecomment-638000200
The problems can be divided into two separate parts
1. Generation strategy：
Highest permit task first, but we need to consider about dirs. So
1. Skip the dirs which are doing compaction
2. For one dir, we only gen one or X tasks in one round, so it's
probably better to gen CUMU 9 times out of 10 rounds, only 1 round to gen
BASE. The number can be adjusted.
2. Permits calc strategy：
_I only tested about AlphaRowsetReader's MemPool. The memory estimate model
is not perfect now. (Without considering the elusive memory and
BetaRowsetReader)
So the memory estimate model is temporarily unavailable._
**We can use scores as permit on the first step.**
(cumulative compaction has
config::max_cumulative_compaction_num_singleton_deltas. So the score of cumu
compaction task is greater than max_deltas, the permits can be set to
max_deltas.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
