Mark Payne created NIFI-11237:
---------------------------------
Summary: Large bulletin messages should be truncated
Key: NIFI-11237
URL: https://issues.apache.org/jira/browse/NIFI-11237
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Mark Payne
Bulletins are kept in memory for 5 minutes and are returned via the REST API
whenever a user navigates to a Process Group, etc. Under most conditions, this
works well. But if a bulletin is particularly large, this can be problematic.
Take, for example, EvaluateJsonPath. There are cases in which it logs an error
and includes the entire JSON Payload, which can be many MB or more. This is all
stored in a bulletin currently. This not only uses massive amounts of heap but
causes very poor UI performance.
Consider a 10-node cluster, in which case each node does this. The Cluster
Coordinator must then parse all 10 nodes' worth of bulletins, merge them
together, and re-serialize it. Then, this is transferred to the browser, and
the browser must then parse the result. This can easily take many seconds,
causing a very poor user experience.
While we can fix the EvaluateJsonPath processor, it makes more sense to simply
truncate any large messages on any bulletins, instead of trying to play
whack-a-mole with specific log messages.
Any bulletin over 1,000 characters (or perhaps 4,000 characters?) should be
truncated. This will improve resilience and reliability and provide a better
user experience.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
