markobean commented on pull request #5324: URL: https://github.com/apache/nifi/pull/5324#issuecomment-1033290551
@markap14 I do not disagree with any points your have made about the dangers of using large attributes. Also, thanks for detailing the reasons of why this can be a problem for other users who may not be aware. Rest assured, these were all considerations and well within the understanding and scope of the use case which drove this ticket. We have never had OOM issues, and the usage is for approximately 200 FlowFiles per day with reasonably sized XML attributes. I appreciate the necessity to prevent “bad things” happening as much as possible. While I respectfully disagree with limiting the platform out of fear of self-induced problems, I understand such problems do occur in the real world. Since NiFi is easily extensible, we will apply these changes to a custom processor to satisfy the specific case without injecting it to the NiFi community. @joewitt I hear what you’re saying about being inconsistent. A prime example related to this PR for processing XML in attributes is available right in Expression Language. It has functions such as (un)escapeXml, and also (un)escapeJson and (un)escapeHtml. It seems the availability of these functions promotes placing XML (or JSON or HTML) into attributes equally as much as this PR, or even more since EL can be applied to a wide variety of processors. EL even has other jsonPath* functions whose simple examples in the EL Guide include 300-500 character JSON strings. Perhaps these examples should be updated to simpler JSON more consistent with the recommended limitation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
