Hey NiFi experts :-) I have started to work on the backend part of interactive queue management, which has several related issues: NIFI-99 (Review in flight flow file details) <https://issues.apache.org/jira/browse/NIFI-99>,NIFI-108 <https://issues.apache.org/jira/browse/NIFI-108>,NIFI-730 (Purge queue from UI) <https://issues.apache.org/jira/browse/NIFI-730>,NIFI-139 (Distribution of FlowFiles on a connection) <https://issues.apache.org/jira/browse/NIFI-139>. There is a feature proposal description [1], which helps you to get a quick overview.
I hope I made the first step to move forward with this topic in a "standard" and "good" direction. I tried to think generally, and making the changes considering all the requested improvements from backend perspective. The basic idea was to extend the web-api with a new endpoint for managing a connection queue (used e.g. by the UI). Based on the mentioned issues, I created the following new "methods": - Get the content of the connection queue *GET* http://your-host/nifi-api/controller/process-groups/{process-group-id}/connections/{connection-id}/queue Response: List of connection queue items - Clear (purge) the connection queue *DELETE* http://your-host/nifi-api/controller/process-groups/{process-group-id}/connections/{connection-id}/queue - Remove a single item from the connection queue *DELETE* http://your-host/nifi-api/controller/process-groups/{process-group-id}/connections/{connection-id}/queue/{flow-file-uuid} - Get a single item from the connection queue *GET* http://your-host/nifi-api/controller/process-groups/{process-group-id}/connections/{connection-id}/queue/{flow-file-uuid} Response: Single connection queue item The connection queue item looks like this (JSON): "connectionQueueItem": { "flowFileId": 16, "flowFileUuid": "92c74b41-005e-444d-8f9e-f9cbc01af5f2", "fileSize": "42 bytes", "fileSizeBytes": 42, "fileName": "filename.tsv", "entryDate": 1443546148763, "lineageStartDate": 1443546148763, "contentClaimSection": "1", "contentClaimContainer": "default", "contentClaimIdentifier": "1443546148763-1", "contentClaimOffset": 0 } If the flow file has a priority attribute, it is also included as a numeric value. It contains enough information for "View content" and "Download content" panels and from the frontend perspective, it could be implemented in a similar way. If you would like to purge the connection queue, or a single item, you just have to make an HTTP DELETE request. And off course your are able to make statistics and review the content of the queue with the first method. Updating an item (e.g. reorder the queue) can result a new method, or covered by a delete + put for a single item with a new priority value. You can find my commits in the following repo : https://github.com/ImpressTv/nifi in branch NIFI-108 <https://github.com/ImpressTV/nifi/tree/NIFI-108>. I do not want to make a pull request until the backend is not in a mergeable state. It is important to mention, that the backend code is not complete, and at some points it maybe requires some reshaping, but you can see the basic concept, and the direction. Before making any new commits, I wanted to share the current state with you, and start/initiate a conversation about the topic, and to get feedback, whether is it a good contribution, or not. So guys, the questions are open :-) Regards, Joe [1] https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management
