Another thing I'd mention about JOLT is that it works by matching
"nodes" as it walks the "tree". That means in certain cases you can
include fields in the spec that may not match a node in one input, but
would match in another. So in the fullName case, you might be able to
write one spec that work
We've had some more clustering issues, and found that some nodes are running
out of memory when we have unexpected spikes in data, then we run into a GC
stop-the-world event... We lowered our thread count, and that has allowed the
cluster to stabilize for the time being.
Our hardware is pretty
How are you currently defining schemas and transforms? or are you free
to choose depending what works best for building the data flow?
If you are willing to use JOLT to define the transform then you have a
few options...
1) As Joe mentioned, you can send the GUID in a header which becomes a
flow
Joe/Carlos,
Thanks for the replies. Here is a little more information that hopefully
clarifies the problem statement. We have System-A and System-B. Each time
we bring on a new User, they get a new Organization (this will be the
unique id) in both System-A and System-B. The new User will go in and
Ryan
I am not entirely sure I fully understand the scenario so read the
following with a bit of caution.
But the gist I think i'm reading is that system A generates data
against a given schema referenceable with some guid say 'GUID1'. For
every GUID1 from System A there is a similar/but possibly
Hi Ryan,
If I understand well, you have a jolt map specification for each user. In that
case you can put the relation guid, map_spec in some table of a external
database.
After that in your flow before JoltTransformJSON you create a custom processor
to read the configuration from the table a
Hi All,
I have been working on an integration between two systems with NiFi in the
middle as the integration point. Basically when an event happens on
System-A, such as a record update, those changes need to be propagated to
System-B. For the first use case, I have set up a data flow that listens
Scott,
I would think that you would upgrade dev to the latest version, in
this case 1.7.1, then make sure all your flows are working properly
and commit any updates to the registry. For example any of the process
groups that involved PutSFTP would need to be committed because of the
new properties
I was wanting to reach out and find out how people are upgrading Nifi
environments while still using the registry to upgrade flows.
We have a particular issue occurring between nifi environments and a process
flow. We run three environments Dev, Staging, and Production. We are on a
weekly upgra