[
https://issues.apache.org/jira/browse/NIFI-7467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17127080#comment-17127080
]
ASF subversion and git services commented on NIFI-7467:
-------------------------------------------------------
Commit 845b66ab9204cc4e8f2418ee6fd154191c3c3941 in nifi's branch
refs/heads/master from Andy LoPresto
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=845b66a ]
NIFI-7467 Refactored S2S peer selection logic.
Removed list structure for peer selection as it was unnecessary and often
wasteful (most clusters are 3 - 7 nodes, the list was always 128 elements).
Changed integer percentages to double to allow for better normalization.
Removed 80% cap on remote peers as it was due to legacy requirements.
Added unit tests for non-deterministic distribution calculations.
Added unit tests for edge cases due to rounding errors, single valid remotes,
unbalanced clusters, and peer queue consecutive selection tracking.
Migrated all legacy PeerSelector unit tests to new API.
Removed unused System time manipulation as tests no longer need it.
Added class-level Javadoc to PeerSelector.
Removed S2S details request replication, as the responses were not being
merged, which led to incorrect ports being returned and breaking S2S peer
retrieval.
Fixed copy/paste error where input ports were being listed as output ports
during remote flow refresh.
Fixed comments and added unbalanced cluster test scenarios.
Removed unnecessary marker interface.
Removed commented code.
Changed weighting & penalization behavior.
Changed dependency scope to test.
This closes #4289.
Signed-off-by: Mark Payne <[email protected]>
> Improve S2S peer retrieval process
> ----------------------------------
>
> Key: NIFI-7467
> URL: https://issues.apache.org/jira/browse/NIFI-7467
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
> Affects Versions: 1.11.4
> Reporter: Andy LoPresto
> Assignee: Andy LoPresto
> Priority: Major
> Labels: cluster, peer, security, site-to-site, subjectAltName
> Time Spent: 2h
> Remaining Estimate: 0h
>
> During investigation for NIFI-7407, [~thenatog] and I discovered a scenario
> where site to site peer retrieval was sub-optimal. Some of this was related
> to hosting a secure cluster with multiple nodes on the same physical/virtual
> server, introducing hostname and SAN resolution problems. In other instances,
> the retrieval has a nested {{NullPointerException}}.
> {code}
> 2020-05-14 18:44:39,140 INFO [Clustering Tasks Thread-2]
> o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2020-05-14
> 18:44:39,124 and sent to node3.nifi:11443 at 2020-05-14 18:44:39,140; send
> took 15 millis
> 2020-05-14 18:44:41,789 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> node1.nifi:9443 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for <node1.nifi>
> doesn't match any of the subject alternative names: [node3.nifi]
> 2020-05-14 18:44:41,789 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@57dfcccd Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster
> 2020-05-14 18:44:44,159 INFO [Clustering Tasks Thread-2]
> o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2020-05-14
> 18:44:44,146 and sent to node3.nifi:11443 at 2020-05-14 18:44:44,159; send
> took 13 millis
> 2020-05-14 18:44:46,791 WARN [Timer-Driven Process Thread-10]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> node1.nifi:9443 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for <node1.nifi>
> doesn't match any of the subject alternative names: [node3.nifi]
> 2020-05-14 18:44:46,791 WARN [Timer-Driven Process Thread-10]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@57dfcccd Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster
> 2020-05-14 18:44:46,791 INFO [Timer-Driven Process Thread-10]
> o.a.nifi.remote.client.http.HttpClient Couldn't find a valid peer to
> communicate with.
> 2020-05-14 18:44:46,817 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> node1.nifi:9443 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for <node1.nifi>
> doesn't match any of the subject alternative names: [node3.nifi]
> 2020-05-14 18:44:46,817 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@57dfcccd Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster
> 2020-05-14 18:44:49,178 INFO [Clustering Tasks Thread-2]
> o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2020-05-14
> 18:44:49,164 and sent to node3.nifi:11443 at 2020-05-14 18:44:49,178; send
> took 13 millis
> 2020-05-14 18:44:51,332 INFO [Timer-Driven Process Thread-6]
> o.a.n.remote.StandardRemoteProcessGroup Successfully refreshed Flow Contents
> for RemoteProcessGroup[https://node1.nifi:9441/nifi]; updated to reflect 1
> Input Ports [InputPort[name=From Self,
> targetId=15f64e5b-0172-1000-ffff-fffff134169a]] and 0 Output Ports
> [OutputPort[name=From Self, targetId=15f64e5b-0172-1000-ffff-fffff134169a]]
> 2020-05-14 18:44:51,833 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector Could not communicate with
> node1.nifi:9443 to determine which nodes exist in the remote NiFi cluster,
> due to javax.net.ssl.SSLPeerUnverifiedException: Certificate for <node1.nifi>
> doesn't match any of the subject alternative names: [node3.nifi]
> 2020-05-14 18:44:51,833 WARN [Http Site-to-Site PeerSelector]
> o.apache.nifi.remote.client.PeerSelector
> org.apache.nifi.remote.client.PeerSelector@57dfcccd Unable to refresh Remote
> Group's peers due to Unable to communicate with remote NiFi cluster in order
> to determine which nodes exist in the remote cluster
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
