markap14 opened a new pull request #5514:
URL: https://github.com/apache/nifi/pull/5514
…a VersionedDataflow using JSON as well as XML, and prefers the JSON
representation on load, if it's available. This also results in the need for
the cluster protocol to exchange its representation of the dataflow to using
JSON. Rather than re-implementing all of the complex logic of Flow
Fingerprinting, updated to just inherit the cluster's flow.
- Note that while the following list of changes may at first seem to be a
long list of unrelated improvements, refactorings all gathered into a single
PR, it needed to be done this way. Changing flow persistence from XML to JSON
meant that the Cluster Coordinator will now provide the flow as a JSON instead
of an XML. This breaks our Flow Fingerprint mechanism for determining if a node
should be allowed to join a cluster. Rather than re-implement all of the logic
for Flow Fingerprinting based on JSON it made more sense and is simpler to just
use the synchronization logic we already have for dealing with Versioned
Components. These changes made it easier then to inherit a cluster flow, even
if there are conflicts because this synchronization logic is more robust and
includes starting/stopping components as necessary. Because of this change, now
nodes will be able to rejoin even with conflicts so it didn't make sense to
continue the requirement that the cluster cannot be updated if a
node is not part of the cluster, so that check was removed. I looked at
separating these changes out into separate Jiras/PRs but doing so would have
greatly complicated things because of all of the interdependencies of these
tasks, so I kept it all as a single PR. Below is a more comprehensive list of
changes/improvements/refactorings that were necessary as part of this changeset:
- Moved logic to synchronize Process Group with Versioned Process Group into
a new ProcessGroupSynchronizer class instead of having all of the logic within
StandardProcessGroup. This significantly improves readability and avoids
importing things into the StandardProcessGroup that are not actually part of
the ProcessGroup.
- Reworked versioned components to use an instance id. This is important
when synching an existing component to a cluster's component rather than a
Versioned Flow.
- Renamed StandardFlowSynchronizer to XmlFlowSynchronizer; introduced new
StandardFlowSynchronizer that delegates to the appropriate (Xml or
Versioned)FlowSynchronzer
- Updated to allow import of VersionedProcessGroup even if not all bundles
are available - will now use ghost components
- Introduced a VersionedDataflow object to hold controller-level services,
reporting tasks, parameter contexts, templates, etc.
- Allow mutable requests to be made while nodes are disconnected. Also fixed
issue in AbstractPolicyBasedAuthorizer that caused ClassNotFoundException /
NoClassDefFoundError if the authorizations were changed and then a node
attempts to rejoin the cluster. The Authorizer was attempting to use XmlUtils,
which is in nifi-security-utils and so so by madking nifi-security-utils a
provided dependency of nifi-framework-api, but this doesn't work, because
nifi-framework-api is loaded by a higher-level classloader, so the classloader
that loads AbstractPolicyBasedAuthorizer will never have the appropriate
classloader to provide nifi-security-utils. Addressed this by copying the code
for creating a safe document builder from XmlUtils to
AbstractPolicyBasedAuthorizer.
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
#### Description of PR
_Enables X functionality; fixes bug NIFI-YYYY._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
