prasad-reddyk opened a new issue #6639: The bookkeeper ledger no honoring the retention period and causing for the more disk usage. URL: https://github.com/apache/pulsar/issues/6639 **Describe the bug** The bookkeeper ledger no honoring the retention period and causing for the more disk usage. During my test execution i set **defaultRetentionTimeInMinutes is 6** and noticed that bookie ledger is no honoring the retention period and eating up more disk space and causing for failures due to "Not enough non-faulty bookie available". When drilled down to disk and see, most of message are retaining even after retention period is completed in ledger disk. The below are the list of broker paramter settins broker: configData: managedLedgerMinLedgerRolloverTimeMinutes: "1" managedLedgerCursorRolloverTimeInSeconds: "60" managedLedgerMaxLedgerRolloverTimeMinutes: "2" managedLedgerMaxEntriesPerLedger: "50000" defaultRetentionTimeInMinutes: "6" defaultRetentionSizeInMB: "1000" ttlDurationDefaultInSeconds: "3600" **To Reproduce** Steps to reproduce the behavior: 1. Use 2.5.0 Pulsar 2. Create 100 topics with three partitions and 100 producers and consumers 3. By using pulsar perf client send data for an hour continuously ( message size is 50 kb) and observe whether disk is flushing out or not after completion of the retention period 4. I got the below ERROR message in broker logs "Not enough non-faulty bookie available" **Expected behavior** As per retentionperiod the data should get flushed out and make disk space free **Screenshots** n/a **Desktop (please complete the following information):** - OS: CentOS 7.7 **Additional context** Currently we have 100 GB for each bookie Below is the disk calculation as per my messages/sec. a=msg/sec -844.023/sec=844.023*50 kb =42201.2 msgs/sec in KB b=a*retention period = 42201.2*360(6 minutes)=15192414 msg/sec =14.4 GB (3 bookies) is required for 6 minutes duration. c= 100 GB each bookie (300 GB for 3 bookies) so b<c as per above calculation. i have enough disk space but still facing issues
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
