"Changing the default value to an unlimited number would negate the protection that the new restriction provides.”
Yes, requesting to negate it. If a dev needs to reduce the bounds, then provide a mechanism for this in the processors in a future Nifi release. From: https://github.com/FasterXML/jackson-core/issues/1014#issuecomment-1581657373 "One takeway for me, too, is that I should have heeded my own "never add a feature just for sake of consistency; only implement things that have been specifically requested". That'd have worked well here, addressing just number length & nesting depth -- both of which have specific known threats.” So there is (currently) no security reason for this change. Worth noting is that number length and nesting depth limits have been introduced as well. > On Sep 19, 2023, at 5:03 PM, David Handermann <[email protected]> > wrote: > > Changing the default value to an unlimited number would negate the protection > that the new restriction provides. > > It should not be necessary to make global changes, but focusing on Processor > and Controller Service references would be a good starting point for > introducing configurable sizes. > > Regards, > David Handermann > > On Tue, Sep 19, 2023 at 3:45 PM Chris Sampson <[email protected] > <mailto:[email protected]>> wrote: > Probably worth noting that the Jackson ObjectMapper is used in lots of places > across the code base - 182 hits in GitHub on current main branch (some of > those are duplicates within classes). > > So it could be a significant change to update and test all places where it's > currently used. The string length limit might not be limited to the > ObjectMapper either, so that would need checking and working through. > > A more centralised configuration might be worthwhile if possible. > > > On Tue, 19 Sept 2023, 20:42 Gregory M. Foreman, > <[email protected] <mailto:[email protected]>> > wrote: > Defaulting maxStringLength to Integer.MAX_VALUE would be my vote. It would > work as it had previously and as expected. I do not currently see a use case > where my client would want to restrict the size of a json string…using > fileSize to route is adequate. > >> On Sep 18, 2023, at 4:53 PM, Bryan Bende <[email protected] >> <mailto:[email protected]>> wrote: >> >> It looks like Jackson 2.15 introduced a max length for a single string value >> inside of the json document which defaults to 20MB. >> >> It can be configured on the ObjectMapper's factory [1] like: >> >> objectMapper.getFactory().setStreamReadConstraints(StreamReadConstraints.builder().maxStringLength(40000000).build())); >> >> If we want to make this configurable, we'd probably have to expose a >> property in any controller-service/processor that does json parsing with an >> ObjectMapper. >> >> [1] >> https://github.com/spring-projects/spring-boot/issues/34709#issuecomment-1482939940 >> >> <https://github.com/spring-projects/spring-boot/issues/34709#issuecomment-1482939940> >> On Mon, Sep 18, 2023 at 4:23 PM Gregory M. Foreman >> <[email protected] <mailto:[email protected]>> >> wrote: >> Yes >> >> Sent from my iPhone >> >>> On Sep 18, 2023, at 2:29 PM, Greene (US), Geoffrey N via users >>> <[email protected] <mailto:[email protected]>> wrote: >>> >>> >>> Is this what you are referring to? 20MB string values vs 20 MB document >>> size… >>> >>> >>> >>> https://github.com/FasterXML/jackson-core/issues/1082 >>> <https://github.com/FasterXML/jackson-core/issues/1082> >>> >>> >>> >>> >>> From: Joe Witt <[email protected] <mailto:[email protected]>> >>> Sent: Monday, September 18, 2023 2:25 PM >>> To: [email protected] <mailto:[email protected]> >>> Subject: [EXTERNAL] Re: Large json string field errors >>> >>> >>> >>> EXT email: be mindful of links/attachments. >>> >>> >>> >>> Greg >>> >>> >>> >>> Can you share details on how you're seeing this limit? Is there a >>> resulting stack trace or other output for it in the nifi-app.log? What >>> version of NiFi are you on? >>> >>> >>> >>> Thanks >>> >>> >>> >>> On Mon, Sep 18, 2023 at 11:19 AM Gregory M. Foreman >>> <[email protected] <mailto:[email protected]>> >>> wrote: >>> >>> Hello: >>> >>> It appears Jackson recently set a 20mb upper boundary on json string field >>> length. Is there a way to remove/override this limit in Nifi? >>> >>> Thanks, >>> Greg >>> >
