keith-turner commented on issue #4076: URL: https://github.com/apache/accumulo/issues/4076#issuecomment-1858367368
Looking over the kubernetes issue, it seems like if this were done with metrics that maybe one could have a custom script that periodically updates the shutdown cost of pods using metrics data and HPA could use that to make decisions. However I am not sure I understood that correctly. One thing I like about the approach in this issue is that offers a way to scale down that does not kill compactors in the middle of a long compaction. If only using HPA and it scales down, then it may kill a compactor that has been working on a compaction for hours when it could have killed a compactor process that was doing nothing. Ideally we could use HPA+metrics to do this because it would operate within its min and max, but that may not be possible currently, not sure. Also using HPA+metrics puts all of the scaling config in one place instead of having some scaling config in Accumulo and some in Kubernetes. If in the furture we find a way to accomplish the goals in this PR with HPA+metrics we may end up with redundant code in Accumulo if this is added. So we could deprecate and remove these change then if we add them now. I don't think the lack of a minimum is big deal for compactors. If the number of compactor processes goes to zero, its not a big deal as long when compaction jobs is queued some compactors are started. I don't think anyone cares if there is some latency before a queued compaction starts. For scan servers and tablet servers though, I could see a desire to keep a minimum number of those processes running to make sudden queries on a system that was idle more responsive. I am in favor of the proposal in this issue for compactors as it offers a simple way to scale down that avoids killing compactors that are not idle. I am uncertain about the usefulness of the idle setting for tablet servers and scan servers though. It may be good to work through using this new property in K8s to scale compactors up and down and see if it works as desired. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@accumulo.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
