joewitt commented on pull request #5324: URL: https://github.com/apache/nifi/pull/5324#issuecomment-1031990263
So a summary of sorts: Improving validation of XML in content is good/great/useful and safe. Creating or increasing validation of XML in the context of attributes is far less desirable as having such payloads in attributes should not be encouraged given the design reasons MarkP noted. I do agree. To add to this I just spent 45 minutes on a call with a great long time user of nifi, they know what they're doing, and they're getting bit by the usage of large memory caused by very similar circumstances. And had they looked at the docs it would have called out how to avoid getting into the memory exhausting situation but these are rarely sufficient guard rails particularly as the user base has expanded well beyond software developers the past few years. MarkB in an earlier JIRA comment said " I find it overly limiting to prevent responsible usage only for fear it may be abused." Of course many of us would agree with that statement on its own. But we do need put in reasonable effort to not lean into features which encourage dangerous usage patterns OR we need to include features that by default prevent exposure the danger of the pattern. In this case we'd be hard pressed to put an artificial limit on attribute size now that we didnt from the get go. On the other hand it isn't a great idea to encourage large attributes by adding capabilities to operate on them. Ultimately this part of the community job is unpleasant. Saying no to someone doing work is not fun especially since we know YOU know how to use it safely. Saying yes to this example is also not fun because it means we're implicitly adding support for a usage pattern we know we have not designed for and for which we see users run into trouble all the time. We have many cases where we've been inconsistent on such things and this is by no means the straw that would break our back. That said we do need to push back where we can and the review process is a helpful way. MarkB you certainly could put up a processor such as ValidateXMLAttribute and offer that via a github repository. We could link to such custom processors as we have done in other cases. I think that is the intent of a wiki page like this https://cwiki.apache.org/confluence/display/NIFI/Community+Contributions -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
