@Mark: I’m now able to query the REST-API. However, the call nifi-api/processors/<processor-id>/diagnostics does reply on all nodes “An unexpected error has occurred. Please check the logs for additional details.” If I remove the /diagnostics I’m getting a useful reply in JSON format. I tried on several nifi nodes, even on a not secure one (http://unsec.company.net:8080/nifi-api/processors/eeba4842-0166-1000-9f56-e59ad010d040/diagnostics ) , all show the same behavior with the error.
Example: curl 'https://abc.company.net:8443/nifi-api/processors/cea533b0-edb9-1d11-955f-7380c973ac62/diagnostics' -H 'Authorization: Bearer $TOKEN' --compressed –insecure An unexpected error has occurred. Please check the logs for additional details. The loadbalancing problem still exists on our cluster… Cheers Josef From: "Zahner Josef, GSB-LR-TRW-LI" <josef.zahn...@swisscom.com> Date: Friday, 16 November 2018 at 22:32 To: "users@nifi.apache.org" <users@nifi.apache.org> Subject: Re: NiFi 1.8 and stuck flowfile in Load Balanced enabled queue Looks pretty much the same in our case, just that we have a 8 node cluster (all nodes connected). * Flowfile expiration: 0 * Back Pressure Object Threshold: 1000000 * Size Threshold: 1 GB * Load Balance Strategy: Round Robin * Load Balance Compression: Do not compress We have on both sides a processor. Just tried to access the nifi-api for the destination, but we have a secured (ldap) cluster and it returns “Unknown user with identity 'anonymous'. Contact the system administrator.” Need to find out first how to access it… Cheers Josef From: dan young <danoyo...@gmail.com> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> Date: Friday, 16 November 2018 at 17:35 To: NiFi Mailing List <users@nifi.apache.org> Subject: Re: NiFi 1.8 and stuck flowfile in Load Balanced enabled queue Here's what we're running: - Load Balance: Round Robin - Compression: None - Nodes: 3 - Are all nodes connected? : Yes - Backpressure configuration: Default, not changed. Note, with our case there wasn't any other flowfiles in any queues. it was just stuck. The only resolution for use was a rolling restart each node in the cluster. Since then, I have configured the the FlowFile Expiration to 3600 seconds and we haven't seen any stuck Flowfiles....continuing to monitor. Regards, Dano On Fri, Nov 16, 2018 at 7:14 AM Mark Payne <marka...@hotmail.com<mailto:marka...@hotmail.com>> wrote: Hey Josef, So a few questions to help figure out what's going on here: - What is the Load Balance Strategy in use? (I.e., Round Robin, Partition by Attribute, etc.) - What compression is being used (None, Compress Attributes Only, Compress Contents and Attributes) - How many nodes in the cluster? - Are all nodes connected? - What is the backpressure configured to? Is the source or destination of the Connection a Processor? If so, we can get a lot of info by going to http://localhost:8080/nifi-api/processors/<processor id>/diagnostics Where http://localhost:8080/ would need to be changed to wherever your nifi instance is running and <processor id> is the UUID of the Processor that is either the source or destination of the connection. You can get the UUID of the Processor by clicking on it in the UI and then looking at the 'Operate' palette on the left-hand side of the screen. If you can get the result of going to the URL, it will likely be helpful, as it shows a lot of details about the processor, as well as all of its incoming and outgoing connections. Thanks -Mark On Nov 16, 2018, at 3:05 AM, <josef.zahn...@swisscom.com<mailto:josef.zahn...@swisscom.com>> <josef.zahn...@swisscom.com<mailto:josef.zahn...@swisscom.com>> wrote: Hi Mark We see the issue again, even after a fresh started cluster (where we started everything at the same time). The files stuck for multiple seconds/minutes in the queue and the light blue loadbalancing icon on the right side shows that it is actually loadbalancing the whole time (even if it is just 1 or 2 files). The log (with default log levels) show no WARN or ERRORs… Thanks in advance, Josef From: Mark Payne <marka...@hotmail.com<mailto:marka...@hotmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Monday, 12 November 2018 at 17:19 To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: NiFi 1.8 and stuck flowfile in Load Balanced enabled queue Hey Dan, Have looked through the logs to see if there are any WARN or ERROR's indicating what's going on? Thanks -Mark On Nov 12, 2018, at 9:06 AM, dan young <danoyo...@gmail.com<mailto:danoyo...@gmail.com>> wrote: Hello, We have two processor groups connected via the new Load Balancing/Round Robin queue. It seems that a flowfile is "stuck" in this queue. I've been watching it for some time now. Is there any way to trouble shoot what is stuck in the queue and why? or maybe remove it? I've tried to stop the PG and empty the queue, but always says emptied 0 out of 1 flowflies... Regards, Dano <Screen Shot 2018-11-12 at 7.00.37 AM.png>
smime.p7s
Description: S/MIME Cryptographic Signature
