Adding my response to Joe since I responded to the wrong thread: Joe,
Our current dev/cert/prod flows run on NiFi 1.2.0 and this is where we face this issue a lot (hence why we created the first version of the monitoring flow). We have stood up new clusters which are running NiFi 1.8.0. No users/flows are running on the new 1.8.0 clusters besides our Admin monitoring flows. So no we don’t face this issue on 1.8.0, but we also have no users yet. We are opening up the 1.8.0 clusters to our users this week for them to start porting over their existing flows to the 1.8.0 clusters. Jeremy, I have a reverse proxy setup and can use it successfully for the NiFi UI. But since the disconnected node is technically still up and running, I still get sent to the disconnected node. So the API call to the reverse proxy will still sometimes give me the “Only a node connected to a cluster can process the request.”” message since it will get sent to the disconnected node sometimes. But it sounds like I might be going down a path that is not worth the effort, since you guys are saying 1.8.0 shouldn’t give me these issues anymore. I can hold off on future development of this and reach back out if I do face the disconnecting issue in 1.8.0 as well. For my knowledge, is it something in 1.8.0 specifically that addresses/fixes this problem, or just overall improvements since 1.2.0? Thanks, Chad From: Jeremy Dyer <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Monday, December 10, 2018 at 10:26 AM To: "[email protected]" <[email protected]> Subject: Re: How to get Node ID of disconnected node to be used in a cluster/node monitoring flow Chad - Ahh I see what you are saying now. To Joe's point the problems really shouldn't be there anymore. However to resolve what you are seeing the best practice is to setup the cluster behind a reverse proxy [1]. In this configuration you are not accessing the REST API on an explicit node and the issue you explained will not occur. [1] - https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration<https://urldefense.proofpoint.com/v2/url?u=https-3A__nifi.apache.org_docs_nifi-2Ddocs_html_administration-2Dguide.html-23proxy-5Fconfiguration&d=DwMFaQ&c=gJN2jf8AyP5Q6Np0yWY19w&r=MJ04HXP0mOz9-J4odYRNRx3ln4A_OnHTjJvmsZOEG64&m=Fu5GvEY5W6MSQUzs1kOAlkIcfLFh4ZZCXJPyGx-7AKQ&s=E4jDUra2uIvmsWV4bPBLS5unz_Jl6yUgPvD-ShNcLMI&e=> On Mon, Dec 10, 2018 at 10:23 AM Joe Witt <[email protected]<mailto:[email protected]>> wrote: chad in 1.8.0 you should not see much need to script that. are you still seeing disconnects in 1.8.0 that it doesnt restore on its own. thanks On Mon, Dec 10, 2018, 10:08 AM Woodhead, Chad <[email protected]<mailto:[email protected]> wrote: Hey Jeremy! We occasionally see a node disconnect from the cluster (not disappear from the cluster) so I’m building this flow just to reconnect a disconnected node. The cluster still shows the node, but just that it is in a disconnected state. As far as the API call to /nifi-api/controller/cluster, yes I get the same result if I call the API on one of the connected nodes. If I call the API of a disconnected node, I get “Only a node connected to a cluster can process the request.” And since the flowfile is sometimes run and processed on the disconnected node, using ${hostname(true)} in the API to /nifi-api/controller/cluster won’t work. I am also using NiFi 1.8.0 for reference as well. -Chad From: Jeremy Dyer <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Monday, December 10, 2018 at 9:12 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: How to get Node ID of disconnected node to be used in a cluster/node monitoring flow *External Message* - Use caution before opening links or attachments Chad - The API call to /nifi-api/controller/cluster will return all of the nodes even if they are disconnected. The JSON response will indicate which node is "disconnected". I have included a screenshot of a simple environment I have setup for this where I just tested it. However I'm thinking since you are performing monitoring here you might have a deeper issue you are trying to work to monitor for. Are you experiencing situations where you are seeing nodes completely disappear from the cluster and not just be in the "disconnected" state? If so that is a different issue but if they are just "disconnected" you should be able to see that as the screenshot below illustrates. I am using NiFi 1.8.0 just for reference. On Mon, Dec 10, 2018 at 8:47 AM Woodhead, Chad <[email protected]<mailto:[email protected]>> wrote: I am working on a cluster/node monitoring flow and was looking for some guidance. I currently have a working flow that monitors if a node gets disconnected from the cluster and sends an email if a node disconnects. Here is the flow: (A SiteToSiteBulletinReportingTask sends to the ClusterMonitoring input port) Error! Filename not specified. The CheckAPIForDisconnectedNode process group makes an API call to NiFi confirm the node is still disconnected before sending the email. This flow works as expected. I am now wanting to update the flow to automatically attempt to reconnect the node itself. When a node disconnects, I capture the server name of that node. To reconnect a node, you can call the API with PUT /nifi-api/controller/cluster/nodes/{id}, but the problem is I don’t know the Node ID. The only place I have found I can get the Node ID is via a GET /nifi-api/controller/cluster, but that API only works on a node that is connected to the cluster and the way this flow works, the processing can and sometimes does actually run on the node that has been disconnected. Is there a way to force it so the processing is only done on a cluster connected node and then use ${hostname(true)} in the API call, or is there another way I can get the Node ID, even on a disconnected node? I don’t want to have to hard code all of my Node ID’s somewhere. Thanks, Chad Woodhead
