sodonnel commented on PR #3286: URL: https://github.com/apache/ozone/pull/3286#issuecomment-1099593096
I like the idea from @errose28, where we check the quota as we allocate each block to an open key. This means each open key cannot exceed the total available quota, but also if someone is writing small keys, they can now write many concurrently with a small quota - before that would have failed as we pre-allocate 256mb * 3 for each open key, even if you only write 1MB. It still does not prevent someone (or a bad application) writing keys up to the quota, and then abandoning them, filling up the system. Also, what would happen if an open key fails to allocate a block due to exceeding the quota? Will it be left as an open key and have to wait for a cleanup job? I am still not sure on this though. The purpose of a quota is to prevent a user from consuming more than a certain amount of disk space on the system. In that respect, surely anything on disk owned by that user, either open, abandoned or committed should count toward the quota? Perhaps we need some sort of "open key + abandoned key" quota to go along with the committed quota. But that starts to make it harder to understand how the whole system works, which is bad for users too! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
