nickva opened a new issue, #4239: URL: https://github.com/apache/couchdb/issues/4239
`cleanup_index_files` is currently disabled by default: https://github.com/apache/couchdb/blob/ebbcc7ec2a5bb6abf65738ff03f1be9f1be6017c/src/smoosh/src/smoosh_channel.erl#L569-L575 When enabled it can misbehave, especially in cases when there are lot of concurrent view compactions. As implemented currently `cleanup_index_files` spawns a simple, unmonitored process on each shard compaction start https://github.com/apache/couchdb/blob/ebbcc7ec2a5bb6abf65738ff03f1be9f1be6017c/src/smoosh/src/smoosh_channel.erl#L498-L501 : During cleanup it performs a cluster-wide fabric call to fetch all design docs and then does wildcard view index folder search for stale files. Since this happens at the cluster database level we end performing the exact same cleanup action for the same fabric db Q times. With the fetching all ddocs and the wildcard directory search this operation can quickly overwhelm the cluster, falling behind while creating these unbounded processes, often doing redundant work. At the same time, it would be nice to have the ability to cleanup stale view index files so they don't keep piling up and being able to default this setting to true. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
