I was able to get answer for above question from prometheus documentation. https://ganeshvernekar.com/blog/prometheus-tsdb-compaction-and-retention/#condition-2-preset-time-ranges
On Wednesday, 29 June 2022 at 14:52:12 UTC+5:30 Sriharsha B M wrote: > Adding some more details.. > During prometheus instaltion, max-block-duration and > storage.tsdb.retention.time is not set. 15d default value is picked > for RetentionDuration and max-block-duration is internally calulated as > 1d12h(10% of RetentionDuration) . Same can be found in logs as well as > prometus UI. > > caller=main.go:986 level=debug msg="TSDB options" MinBlockDuration=2h > MaxBlockDuration=1d12h MaxBytes=0B NoLockfile=false RetentionDuration=15d > WALSegmentSize=0B AllowOverlappingBlocks=false WALCompression=true > > Even though logs/UI says MaxBlockDuration is 1d12h but compaction is > happening only upto 18hr. This is not aligning with "Compaction will > create larger blocks containing data spanning up to 10% of the retention > time, or 31 days, whichever is smaller." Could you please clarify this. > > > On Sunday, 26 June 2022 at 23:37:11 UTC+5:30 Sriharsha B M wrote: > >> Hi, >> As per prometheus documentation, "Compaction will create larger blocks >> containing data spanning up to 10% of the retention time, or 31 days, >> whichever is smaller." >> >> We have set retention time as 4 days. 10% of retention time is 9.6h. If >> we align to the 2 hour boundary then it will become 8 hr. But prometheus >> is creating bigger blocks only upto 6 hour. Could you please let us know >> how it is calculated. >> >> Following is differnt test which we did with different >> retention time(10m, 1d, 2d, 3d, 4d, 7d, 15d). Please note retention size is >> not set. >> >> fifteen-day-cpro-server-0 >> total 60 >> drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found >> -rw-r--r--. 1 65534 65534 0 Jun 7 06:08 lock >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR5XPZW1SJ7ZSDKEA8GWG >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518P9M546ZMG7PC6XC435KQ >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8RBDRNCFK0ZX389R1WCG >> drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X914XWP2ZYXQ5JBCQTWR5 >> -rw-r--r--. 1 65534 65534 20001 Jun 8 15:06 queries.active >> four-day-cpro-server-0 >> total 68 >> drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found >> -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR4TKCW31TX7A988YXEH1 >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZFG1S11VXXJENB17B1QZY >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2KBZ4Q5QDKFQT5C5APZ9 >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NVEEQ58VNCY9YA2Q03SE >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QDYBJA81XWS9TSB6BDQ >> drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8ZWY7H9AWDE9SVH2CP5N >> -rw-r--r--. 1 65534 65534 20001 Jun 8 15:06 queries.active >> one-day-cpro-server-0 >> total 96 >> drwxrws---. 2 root 65534 16384 Jun 7 06:08 lost+found >> -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 15:00 01G4ZAW0WNFYMP5JK9K12XJ5Q3 >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHQR4N49YYM33NMT2NJCM8 >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 19:00 01G4ZRKFCMFPP15V6VBV5DNQQC >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZF6MS3JEF8B8K8CRA1G3X >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 23:00 01G506AXWP0R72GCFWRQTNHWHV >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 01:00 01G50D6N4NMA9VNVZKWB2W8G4Q >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2CDG33SSXC5MH04HXF36 >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 05:00 01G50TY3MQV7Z5A5T3Z00FWR4M >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 07:00 01G511STWNM6QFD7CJ4NDKE3ZZ >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NJ4NVWF5VYFBX170B74Z >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 11:00 01G51FH8Z4KH8P30EQC776WKMY >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 13:00 01G51PD0MNAM4YEYAV5HBXC68M >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QWNW7K0RG5WPEPA6J75 >> drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal >> -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active >> seven-day-cpro-server-0 >> total 68 >> drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found >> -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR3TQTJ3XRZRZQRQF3YNY >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZFH68NEQJK6T2F3F2XC3Y >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2KW3JRK2J7H0TZ0WGYGK >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NWQ2QX42XAYP46H31D92 >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8R6V0FVSNKQ8KTBXSANS >> drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X9056BT7YJEZBJZMSAEVV >> -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active >> ten-min-cpro-server-0 >> total 48 >> drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found >> -rw-rw-r--. 1 65534 65534 0 Jun 7 06:09 lock >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QJ2E6E92KDB8CX8FNWY >> drwxrwsr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head >> drwxrwsr-x. 3 65534 65534 4096 Jun 8 15:00 wal >> -rw-rw-r--. 1 65534 65534 20001 Jun 8 15:07 queries.active >> three-day-cpro-server-0 >> total 68 >> drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found >> -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR5PE5E5W0G8Y3KZHGZ59 >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZFJ9K8HBW0S9AXT07PTS0 >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2K3PQHG1XFRKZS9HWCST >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NVSZ5GKFCPYX4VK8ES5S >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8RBJQ68P709CMNKWHTSH >> drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X90VJ543W770RJQM7D81W >> -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active >> two-day-cpro-server-0 >> total 108 >> drwxrws---. 2 root 65534 16384 Jun 7 06:08 lost+found >> -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 09:09 01G4YPT28KE1ERW4K43AXWGYWM >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 11:00 01G4YX4HY0JZC3RFAHEQYNNQ31 >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 13:00 01G4Z40961X9HKT4WMHSSSXBN4 >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 15:00 01G4ZAW0E0YVYQJEHNNC7JZTR4 >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHQQP0W8DAZA3JRXAFPAQQ >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 19:00 01G4ZRKEY00KXCMX9X7SG7F5VR >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZF661NZV15S1DFKAFMMXY >> drwxr-sr-x. 3 65534 65534 4096 Jun 7 23:00 01G506AXE1H9WXG1E4F6ARF6B2 >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 01:00 01G50D6MP0FRJ8BEHMSB0SQEN0 >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2BY0N3MB37W5MR1NPWPT >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 05:00 01G50TY361R94D1SSD9GGWCNMP >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 07:00 01G511STE11YHBY842MG04T2T5 >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NHP1M4GM2J340CYM97F9 >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 11:00 01G51FH8Y0JTPDC3F0Q86JFWST >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 13:00 01G51PD060XKRD1GY3G1W149TJ >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QE1HTTVCDNJVEH4JYZC >> drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head >> drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal >> -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active >> >> >> >> >> >> ReplyForward >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/5514e570-ae41-417a-9cce-b1cd40bf1b34n%40googlegroups.com.
