Hi Please have a look at this old jira: https://issues.apache.org/jira/browse/NIFI-2203 I have had issues where a processor create a log message ever 10ms resulting in the disk is being full. For me it seems like the maxHistory settings only effect how many files defined by the rolling patten to be kept. If you have defined it like this: <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app%d{yyyy-MM-dd}.%i.log</fileNamePattern> MaxHistory only effect the days not the increments file %i per day. So you can stille have thousands of files in one day. The totalSizeCap will delete the oldes files if the total size hits the cap settings.
The totalSizeCap have been added in the logback.xml file for nifi-registry where it has been added inside the rollingPolicy section. I cound not get it to work inside the rollingPolicy section in nifi but just added in appender section. See my comment in the jira: https://issues.apache.org/jira/browse/NIFI-2203 Kind regards Jens M. Kofoed Den lør. 8. jul. 2023 kl. 04.27 skrev Mike Thomsen <mikerthom...@gmail.com>: > Yeah, I'm working through some of it where I have time. I plan to have a > Jira up this weekend. I'm wondering, though, if we shouldn't consider a > spike for switching to log4j2 in 2.X because I saw a lot of complaints > about logback being inconsistent in honoring its settings. > > On Fri, Jul 7, 2023 at 10:19 PM Joe Witt <joe.w...@gmail.com> wrote: > >> Hmmmm. Interesting. Can you capture these bits of fun in a jira? >> >> Thanks >> >> On Fri, Jul 7, 2023 at 7:17 PM Mike Thomsen <mikerthom...@gmail.com> >> wrote: >> >>> After doing some research, it appears that <maxFileSize/> is a wonky >>> setting WRT how well it's honored by logback. I let a GenerateFlowFile > >>> LogAttribute flow run for a long time, and it just kept filling up. When I >>> added <totalSizeCap/> that appeared to force expected behavior on total log >>> size. We might want to add the following: >>> >>> <cleanHistoryOnStart>true</cleanHistoryOnStart> >>> <totalSizeCap>50GB</totalSizeCap> >>> >>> On Fri, Jul 7, 2023 at 11:33 AM Michael Moser <moser...@gmail.com> >>> wrote: >>> >>>> Hi Mike, >>>> >>>> You aren't alone in experiencing this. I think logback uses a pattern >>>> matcher on filename to discover files to delete. If "something" happens >>>> which causes a gap in the date pattern, then the matcher will then fail to >>>> pick up and delete files on the other side of that gap. >>>> >>>> Regards, >>>> -- Mike M >>>> >>>> >>>> On Thu, Jul 6, 2023 at 10:28 AM Mike Thomsen <mikerthom...@gmail.com> >>>> wrote: >>>> >>>>> We are using the stock configuration, and have noticed that we have a >>>>> lot of nifi-app* logs that are well beyond the historic data cap of 30 >>>>> days >>>>> in logback.xml; some of those logs go back to April. We also have a bunch >>>>> of 0 byte nifi-user logs and some of the other logs are 0 bytes as well. >>>>> It >>>>> looks like logback is rotating based on time, but isn't cleaning up. Is >>>>> this expected behavior or a problem with the configuration? >>>>> >>>>> Thanks, >>>>> >>>>> Mike >>>>> >>>>
