[
https://issues.apache.org/jira/browse/KUDU-607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Percy updated KUDU-607:
----------------------------
Parent: KUDU-518
> Flush MRS/DMS based on how much logs they retain
> ------------------------------------------------
>
> Key: KUDU-607
> URL: https://issues.apache.org/jira/browse/KUDU-607
> Project: Kudu
> Issue Type: Sub-task
> Components: tserver
> Affects Versions: M5
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
>
> The time it takes to bootstrap a tablet after a TS restart is directly
> dependant on amount of data we have in the logs. We currently have a process
> that GCs the logs whose data is fully flushed, up to a configurable retention
> level, meaning that by default we keep at least 2 log segments even if their
> data is flushed. The reason we want this is so that we some logs around when
> a follower slows down and we have to help them catch up (instead of kicking
> them out).
> This starts working against us when some DMS or the MRS only receives a
> trickle of inserts. We can add time-based flushing but it's not perfect.
> Instead, we can flush based on whoevers preventing us to GC the biggest
> amount of logs data. We can do this by using the oldest entry index from the
> MRS/DMS and check that against the max indexes in each log segments that we
> can't GC. We then modify the Maintenance Manager to be able to take that into
> account when selecting an operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
