Perhaps you could delete the repository directories when you need to restart with no data?
On Thu, Jan 11, 2018, 9:16 PM 尹文才 <[email protected]> wrote: > Hi Mark, forgot to ask about VolatileFlowFileRepository you mentioned, if I > switch to use VolatileFlowFileRepository, will NiFi swap out all the other > FlowFiles to disk if a queue is already full? > Is it just simply keeping all FlowFiles in memory? > > Regards, > Ben > > 2018-01-12 12:07 GMT+08:00 尹文才 <[email protected]>: > > > Thanks Mark, my case is that I'm using NiFi to do some ETL work and it's > > possible that NiFi dies unexpectedly due to lack of system resources. > After > > NiFi restarts itself, > > I will re-extract all the data from database and re-perform all the > > operations, so I need to clear all possible FlowFiles that might exist in > > any queue. > > > > Regards, > > Ben > > > > 2018-01-12 11:49 GMT+08:00 Mark Payne <[email protected]>: > > > >> Ben, > >> > >> I have to admit - that’s kind of an odd request :) I’m curious what the > >> use case is, if you can share? > >> > >> Regardless, the easiest way would be to update nifi.properties so that > >> the FlowFile repo that is used is the VolatileFlowFileRepository. This > >> would avoid writing the FlowFile state to disk, so ok restart you will > lose > >> all FlowFiles. The content will still be present, but nifi will delete > it > >> all on startup because there is no FlowFile associated with it. > >> > >> I’m on my phone right now so can’t easily tell you the exact name of the > >> property to change but you’ll probably find it pretty quickly. The Admin > >> Guide may well explain the different repositories as well. > >> > >> Thanks > >> -Mark > >> > >> Sent from my iPhone > >> > >> > On Jan 11, 2018, at 10:31 PM, 尹文才 <[email protected]> wrote: > >> > > >> > Hi guys, I'm trying to clear all FlowFIles in all queues when NiFi is > >> > restarted, but I don't know the correct way to do this. I checked all > >> > NiFi's guide documentation, > >> > it seems there're 2 possible solutions: > >> > 1. write a custom notification service: a notification service could > be > >> > notified when NiFi is restarted and then inside the service, delete > all > >> the > >> > files inside content_repository, flowfile_repository and > >> > provenance_repository. > >> > I know there're now 2 existing services: email and http. But I'm not > >> > quite sure how to correctly write one and deploy it into my NiFi > >> > environment, is there a tutorial on writing one notification service? > >> > > >> > 2. I know from the developer guide that by using the annotation > >> @Shutdown > >> > in a custom processor, the method could be called when NiFi is > >> successfully > >> > shut down. The problem with this approach is the method could > >> > not be guaranteed to be called when NiFi dies unexpectedly. > >> > > >> > Does anyone know what is the correct way to implement it? Thanks. > >> > > >> > Regards, > >> > Ben > >> > > > > >
