Hi, I have a Prometheus deployment for which we have only used size based retention, which we have set to 2.5T. We regularly see unexpectedly large blocks getting created, so that when we hit storage.tsdb.retention.size we see a huge drop in the metrics available for querying.
For instance we currently have a block which is 1.1T and contains data from Mon 1 Jun 01:00:00 BST 2020 to Sun 14 Jun 13:00:00 BST 2020, which is ~45% of our retention size! I can see from the docs that "Compaction will create larger blocks up to 10% of the retention time, or 31 days, whichever is smaller". I expected this to mean 10% of whatever retention I set, but from what I have seen it looks like this means the max block size will always be 31 days if you have storage.tsdb.retention.time=0s and are just using size based retention. We have elected to use sized based retention instead of time because we are in the process of onboarding application exporters so the number of days we can retain is decreasing frequently, but the amount of space we have available is not. In this situation what would be the best way to configure storage.tsdb.max-block-duration? Thanks, Dom -- This e-mail together with any attachments (the "Message") is confidential and may contain privileged information. If you are not the intended recipient or if you have received this e-mail in error, please notify the sender immediately and permanently delete this Message from your system. Do not copy, disclose or distribute the information contained in this Message. _Maven Investment Partners Ltd (No. 07511928), _Maven Investment Partners US Ltd (No. _11494299), Maven Europe Ltd (No. 08966), Maven Derivatives Asia Limited (No.10361312) & Maven Securities Holding Ltd (No. 07505438) are registered as companies in England and Wales and their registered address is Level 3, 6 Bevis Marks, London EC3A 7BA, United Kingdom. The companies’ VAT No. is 135539016. Maven Asia (Hong Kong) Ltd (No. 2444041) is registered in Hong Kong and its registered address is 20/F, Tai Tung Building, 8 Fleming Road, Wan Chai, Hong Kong. Maven Europe Ltd is authorised and regulated by the Financial Conduct Authority (FRN:770542). Maven Asia (Hong Kong) Ltd is registered and regulated by the Securities and Futures Commission (CE No: BJF060).___ -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/281cc4ae-f6f8-4677-8f15-e28e252b8721n%40googlegroups.com.
