Hi, I've deleted the files as suggested by James, but I would also like to fine tune so this doesn't happen again. However, I'm not sure what exactly happens with parameter: nifi.provenance.repository.max.storage.size
I haven't changed the default (1 GB), so what would happen when the limit is reached? Does NiFi stops storing the provenance and someone has to manually delete the files or does it cycle again over the 1 GB overwriting the existing files? Thanks, Pablo. On Wed, Dec 14, 2016 at 5:31 PM, James Wing <[email protected]> wrote: > Pablo, > > An extreme fix is to stop stop NiFi and completely delete the provenance > repository directory. You will get a new, empty provenance repository when > you restart NiFi. All data will be lost, but NiFi will work again. > > Obviously, Lee's advice is much better, but there are options if you need > them. > > James > > On Tue, Dec 13, 2016 at 10:23 PM, Lee Laim <[email protected]> wrote: > >> Pablo, >> >> You can decrease the following attributes in conf/nifi.properties. >> >> nifi.provenance.repository.max.storage.time >> >> The maximum amount of time to keep data provenance information. The >> default value is 24 hours. >> >> nifi.provenance.repository.max.storage.size >> >> The maximum amount of data provenance information to store at a time. The >> default is 1 GB. >> https://nifi.apache.org/docs/nifi-docs/html/administration-g >> uide.html#flowfile-repository >> >> >> Thanks, >> Lee >> >> >> >> On Tue, Dec 13, 2016 at 10:43 PM, Pablo Lopez < >> [email protected]> wrote: >> >>> Hi, >>> >>> We've been loading quite a few big files through NiFi, but now it looks >>> that the file disk is full and we get this error in most places and nothing >>> gets processed. >>> >>> "Failed to rollover Provenance Event Log due to java.io.IOException no >>> space left on device" >>> >>> Is there a way to manually delete all the provenance entries from the >>> repository so we can continue processing? >>> Also, is there a way to disable data provenance for big files? >>> >>> Thanks for your help, >>> Pablo. >>> >> >> > -- Pablo Lopez. Integration Architect Integrado Pty Ltd M: 044 84 52 479 [email protected]
