Adding some more details.. During prometheus instaltion, max-block-duration and storage.tsdb.retention.time is not set. 15d default value is picked for RetentionDuration and max-block-duration is internally calulated as 1d12h(10% of RetentionDuration) . Same can be found in logs as well as prometus UI.
caller=main.go:986 level=debug msg="TSDB options" MinBlockDuration=2h MaxBlockDuration=1d12h MaxBytes=0B NoLockfile=false RetentionDuration=15d WALSegmentSize=0B AllowOverlappingBlocks=false WALCompression=true Even though logs/UI says MaxBlockDuration is 1d12h but compaction is happening only upto 18hr. This is not aligning with "Compaction will create larger blocks containing data spanning up to 10% of the retention time, or 31 days, whichever is smaller." Could you please clarify this. On Sunday, 26 June 2022 at 23:37:11 UTC+5:30 Sriharsha B M wrote: > Hi, > As per prometheus documentation, "Compaction will create larger blocks > containing data spanning up to 10% of the retention time, or 31 days, > whichever is smaller." > > We have set retention time as 4 days. 10% of retention time is 9.6h. If we > align to the 2 hour boundary then it will become 8 hr. But prometheus is > creating bigger blocks only upto 6 hour. Could you please let us know how > it is calculated. > > Following is differnt test which we did with different retention time(10m, > 1d, 2d, 3d, 4d, 7d, 15d). Please note retention size is not set. > > fifteen-day-cpro-server-0 > total 60 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:08 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR5XPZW1SJ7ZSDKEA8GWG > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518P9M546ZMG7PC6XC435KQ > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8RBDRNCFK0ZX389R1WCG > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X914XWP2ZYXQ5JBCQTWR5 > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:06 queries.active > four-day-cpro-server-0 > total 68 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR4TKCW31TX7A988YXEH1 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZFG1S11VXXJENB17B1QZY > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2KBZ4Q5QDKFQT5C5APZ9 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NVEEQ58VNCY9YA2Q03SE > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QDYBJA81XWS9TSB6BDQ > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8ZWY7H9AWDE9SVH2CP5N > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:06 queries.active > one-day-cpro-server-0 > total 96 > drwxrws---. 2 root 65534 16384 Jun 7 06:08 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 15:00 01G4ZAW0WNFYMP5JK9K12XJ5Q3 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHQR4N49YYM33NMT2NJCM8 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 19:00 01G4ZRKFCMFPP15V6VBV5DNQQC > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZF6MS3JEF8B8K8CRA1G3X > drwxr-sr-x. 3 65534 65534 4096 Jun 7 23:00 01G506AXWP0R72GCFWRQTNHWHV > drwxr-sr-x. 3 65534 65534 4096 Jun 8 01:00 01G50D6N4NMA9VNVZKWB2W8G4Q > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2CDG33SSXC5MH04HXF36 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 05:00 01G50TY3MQV7Z5A5T3Z00FWR4M > drwxr-sr-x. 3 65534 65534 4096 Jun 8 07:00 01G511STWNM6QFD7CJ4NDKE3ZZ > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NJ4NVWF5VYFBX170B74Z > drwxr-sr-x. 3 65534 65534 4096 Jun 8 11:00 01G51FH8Z4KH8P30EQC776WKMY > drwxr-sr-x. 3 65534 65534 4096 Jun 8 13:00 01G51PD0MNAM4YEYAV5HBXC68M > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QWNW7K0RG5WPEPA6J75 > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > seven-day-cpro-server-0 > total 68 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR3TQTJ3XRZRZQRQF3YNY > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZFH68NEQJK6T2F3F2XC3Y > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2KW3JRK2J7H0TZ0WGYGK > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NWQ2QX42XAYP46H31D92 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8R6V0FVSNKQ8KTBXSANS > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X9056BT7YJEZBJZMSAEVV > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > ten-min-cpro-server-0 > total 48 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-rw-r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QJ2E6E92KDB8CX8FNWY > drwxrwsr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxrwsr-x. 3 65534 65534 4096 Jun 8 15:00 wal > -rw-rw-r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > three-day-cpro-server-0 > total 68 > drwxrws---. 2 root 65534 16384 Jun 7 06:07 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHR5PE5E5W0G8Y3KZHGZ59 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZFJ9K8HBW0S9AXT07PTS0 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2K3PQHG1XFRKZS9HWCST > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NVSZ5GKFCPYX4VK8ES5S > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8RBJQ68P709CMNKWHTSH > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X90VJ543W770RJQM7D81W > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > two-day-cpro-server-0 > total 108 > drwxrws---. 2 root 65534 16384 Jun 7 06:08 lost+found > -rw-r--r--. 1 65534 65534 0 Jun 7 06:09 lock > drwxr-sr-x. 3 65534 65534 4096 Jun 7 09:09 01G4YPT28KE1ERW4K43AXWGYWM > drwxr-sr-x. 3 65534 65534 4096 Jun 7 11:00 01G4YX4HY0JZC3RFAHEQYNNQ31 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 13:00 01G4Z40961X9HKT4WMHSSSXBN4 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 15:00 01G4ZAW0E0YVYQJEHNNC7JZTR4 > drwxr-sr-x. 3 65534 65534 4096 Jun 7 17:00 01G4ZHQQP0W8DAZA3JRXAFPAQQ > drwxr-sr-x. 3 65534 65534 4096 Jun 7 19:00 01G4ZRKEY00KXCMX9X7SG7F5VR > drwxr-sr-x. 3 65534 65534 4096 Jun 7 21:00 01G4ZZF661NZV15S1DFKAFMMXY > drwxr-sr-x. 3 65534 65534 4096 Jun 7 23:00 01G506AXE1H9WXG1E4F6ARF6B2 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 01:00 01G50D6MP0FRJ8BEHMSB0SQEN0 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 03:00 01G50M2BY0N3MB37W5MR1NPWPT > drwxr-sr-x. 3 65534 65534 4096 Jun 8 05:00 01G50TY361R94D1SSD9GGWCNMP > drwxr-sr-x. 3 65534 65534 4096 Jun 8 07:00 01G511STE11YHBY842MG04T2T5 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 09:00 01G518NHP1M4GM2J340CYM97F9 > drwxr-sr-x. 3 65534 65534 4096 Jun 8 11:00 01G51FH8Y0JTPDC3F0Q86JFWST > drwxr-sr-x. 3 65534 65534 4096 Jun 8 13:00 01G51PD060XKRD1GY3G1W149TJ > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 01G51X8QE1HTTVCDNJVEH4JYZC > drwxr-sr-x. 2 65534 65534 4096 Jun 8 15:00 chunks_head > drwxr-sr-x. 3 65534 65534 4096 Jun 8 15:00 wal > -rw-r--r--. 1 65534 65534 20001 Jun 8 15:07 queries.active > > > > > > ReplyForward > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/eb5860cb-2175-461b-b686-108af952438dn%40googlegroups.com.
