Lars, Definitely look forward to understanding the mechanics here a bit better of what you're seeing and if you can provide something reproducible. Even if you have a custom processor the API/Process Session construct should protect from many of the things that can go wrong there. Now the content repo will likely be large empty as the data represents on 888KB of data and it is probably in a relative small number of files on disk.
Thanks joe On Tue, Feb 16, 2016 at 2:57 PM, Lars Francke <[email protected]> wrote: > Hi Matt, > > thanks for the quick response. It's late here so I'll try reproducing > tomorrow. > > Source and destination processors are custom processors. > This is Nifi 0.5.0 RC3 > > NiFi thinks all FlowFiles are still there: <http://imgur.com/isDlRk4> > > I'm looking at logs now no ERRORs or WARN that seem suspicious so far > > On Tue, Feb 16, 2016 at 8:46 PM, Matthew Clarke <[email protected]> > wrote: > >> Lars, >> What version of NiFi are you running? >> What type of processor was your source processor? >> What type of processor was the destination processor? >> I tried reproducing using a GenerateFlowFile to produce ~100k >> Flowfiles on a connection to an UpdateAttribute processor. I then stopped >> the GenerateFlowFile processor , added a funnel, and moved the connection. >> I also added another processor feeding that same funnel and routed the >> connection from the funnel back to the UpdateAttribute processor. The >> files moved as expected through the funnnel. >> >> Can you reproduce? Any other errors in your app log from prior to >> completing the connection? >> >> Thanks, >> Matt >> >> On Tue, Feb 16, 2016 at 1:15 PM, Lars Francke <[email protected]> >> wrote: >> >> > Hi, >> > >> > I'm trying to understand what happened and how I can prevent this in the >> > future. >> > >> > The outcome seems to be that all my FlowFiles which were sitting in a >> > connection have been deleted from disk. >> > >> > I had a flow with two processors connected via a single connection. >> > >> > What I did: >> > * Stop all Processors >> > * Add a Funnel >> > * Add a Processor >> > * Move destination end of existing connection to funnel (with the >> existing >> > FlowFiles) >> > * Connect new Processor to Funnel >> > * Connect Funnel to old destination Processor >> > >> > The connection between the Funnel and the Destination processor still >> shows >> > all 90k FlowFiles but the Processor fails on session.read with a >> > MissingFlowFileException. >> > >> > Sure enough my content_repository is mostly empty too. >> > >> > Now this isn't so bad because it's only a dev environment but I'd like to >> > understand how this could happen. Did I do something wrong? >> > >> > Any hints on what to search for in the logs or which place in the source >> > code to look? >> > >> > Cheers, >> > Lars >> > >>
