Re: how to edit queue content
If you know one of the supported scripting languages, you can probably do some of that with ExecuteScript. For example, if you wanted to drop every other flowfile in a block of 100, it'd be like this: def flowfiles = session.get(100) // Get up to 100 int index = 1 flowfiles?.each { flowFile -> if (index % 2 == 0) { session.remove(flowFile) } else { session.transfer(flowFile, REL_SUCCESS) } index++ } On Tue, Mar 27, 2018 at 12:06 AM, scottwrote: > Hi community, > > I've got a question about a feature I would find useful. I've been setting > up a lot of new flows and testing various configurations, and I thought it > would be really useful if I could edit the content of queues. For example, > I can examine each file in the queue, then decide I want to keep the second > one and the third one, then remove the rest before resuming my flow > testing. I know I can delete all files, but is there a way to have more > control over the queue content? Could I delete a specific file, or change > the order of the queue? > > Thanks for your time, > > Scott > >
Re: how to edit queue content
How is it going to work with e.g. 20GB of events in the queue? I'd be careful, as requirements blow up into a full db with indexes, search, and a UI on top. If one wanted to filter events, wouldn't a standard processor do the job better? Andrew On Tue, Mar 27, 2018, 12:11 AM Joe Wittwrote: > Scott > Yep definitely something we've talked about [1]. We've not pursued it > directly as of yet since it is indeed a queue and we're just letting > you peak into it. We dont have facilities built in to really alter > the queue in a particular position. Also, the complexity comes in > when folks want to have paging/selection of various items down the > list/etc.. (but it isn't a list - its a queue). > > If you could bound the range of what you'd expect to be able to do > that would probably help constrain into something reasonably > implemented. > > Thanks > > [1] > https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management > > On Tue, Mar 27, 2018 at 12:06 AM, scott wrote: > > Hi community, > > > > I've got a question about a feature I would find useful. I've been > setting > > up a lot of new flows and testing various configurations, and I thought > it > > would be really useful if I could edit the content of queues. For > example, I > > can examine each file in the queue, then decide I want to keep the second > > one and the third one, then remove the rest before resuming my flow > testing. > > I know I can delete all files, but is there a way to have more control > over > > the queue content? Could I delete a specific file, or change the order of > > the queue? > > > > Thanks for your time, > > > > Scott > > >
Re: how to edit queue content
Scott Yep definitely something we've talked about [1]. We've not pursued it directly as of yet since it is indeed a queue and we're just letting you peak into it. We dont have facilities built in to really alter the queue in a particular position. Also, the complexity comes in when folks want to have paging/selection of various items down the list/etc.. (but it isn't a list - its a queue). If you could bound the range of what you'd expect to be able to do that would probably help constrain into something reasonably implemented. Thanks [1] https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management On Tue, Mar 27, 2018 at 12:06 AM, scottwrote: > Hi community, > > I've got a question about a feature I would find useful. I've been setting > up a lot of new flows and testing various configurations, and I thought it > would be really useful if I could edit the content of queues. For example, I > can examine each file in the queue, then decide I want to keep the second > one and the third one, then remove the rest before resuming my flow testing. > I know I can delete all files, but is there a way to have more control over > the queue content? Could I delete a specific file, or change the order of > the queue? > > Thanks for your time, > > Scott >