Otto,
I am not currently proposing that there be a unified flow comparison capability
- or that there not be
one. I think that's more of an implementation detail. I do believe there's a
good chance that we can
map a FlowSnippetDTO into a VersionedProcessGroup object, in which case we
could use
Thanks, all, for the great feedback so far. I did leave out any mention of how
I would envision
handling the authorizations.xml and users.xml in the new approach, but yes, as
has been suggested
I do believe that it will be important to also inherit those from the cluster.
Bryan also brought up a
Mark
I definitely think it is time to move on from the fingerprinting
model. I recall the conversations long ago that led us down this path
and ultimately the thing that mattered most was ensuring data loss
cases were prevented. This still addresses that, reduces a ton of
code, and simplifies th
Great ideas Mark. Another addition for ease/convenience of a node joining a
cluster is related to the authorizations.xml and users.xml. In the case of
the flow.xml.gz, if the file is missing, it will obtain a copy from the
cluster. The same should be true of authorizations.xml and users.xml files,
Using the versioned flow logic seems like a good idea.
Would the authorizer fingerprints still be checked as part of joining
the cluster?
Currently that is appended to the overall fingerprint to ensure each
node has the same users/policies, or at least same config (i.e. LDAP).
Would be nice if a
Great write up.
While I am not an expert on clustering, it would seem that having one
method of comparing flows, perhaps
with different strategies within that would be more maintainable as well.
Are you proposing that there is a unified flow comparison
capability/implementation/service that is sh
Hi Mark,
Thanks for the detailed explanations!
The suggested approach makes total sense to me and it'll greatly improve
the user experience as I often see a node unable to join the cluster
because of a discrepancy in some component configuration... Right now, the
only solution is to do as you said
Hi Mark,
Thanks for the great write-up! I support your proposal. It seems a logical
improvement based on your description of how fingerprinting works today, the
original problem it was trying to solve, and the proposed alternative.
I am not an expert on how NiFi clustering is implemented, so I'
Hi all,
Over the past couple of months, I have been doing a lot of testing with large
scale flows and
talking to others who are using large scale flows in production. ("Large scale"
flows in this case
means several thousand to tens of thousands of Processors). While NiFi does a
really good job
