All, I added more tests and found that most the optimizations were not happening due to buffering.
Essentially there are two buffers between Snippet.Buffer and Snippet.SnippetOutputStream. The SnippetOutputStream had the responsibility to tell the code up the stack when we've reached the max snippet length. Since all the bytes were buffered, it would see nothing until the very end and we'd end up serializing the full json text anyway. One is the 64k buffer in JsonGeneratorImpl and the other is an 8k buffer in the JVM implementation code of OutputStreamWriter. Since the OutputStreamWriter buffer is hardcoded, we can't solve this by adjusting buffer sizes and have no choice but to aggressively call flush() to ensure SnippetOutputStream has the bytes and can do its job. I've reworked the code and greatly expanded the tests so we're not just asserting the resulting text but the details in how we get there. We could potentially have Snippet override the buffer setting of JsonGeneratorFactoryImpl so the buffer is of snippetMaxLength. I'm curious as to what others think. On one hand, we could fit 1310 snippet buffers of 50 bytes into the size of just one 64k default buffer. On the other hand, we're coping config maps and essentially building a new factory to get there. I'm not sure if I like that idea. > On Apr 21, 2022, at 10:39 PM, Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > > Le ven. 22 avr. 2022 à 01:00, David Blevins <david.blev...@gmail.com> a > écrit : >> Where do we document these kinds of parameters? If you have any pointers >> I'll try to work that into the next PR along with using this in a few more >> places. >> > > We use maven site. so > https://github.com/apache/johnzon/blob/master/src/site/markdown/index.md > It is not great in terms of browsing but the place we have as of today. > Feel free to break it in several places if you think it is better. Great, thanks. Looks like we haven't yet started documenting the config properties that can be passed into JsonbConfig. I'll probably defer that then and move on to updating all the error messages in a second PR. Getting proper documentation for our JsonbConfig properties is likely to involve a larger effort to compile a list of all the properties and their defaults. That will definitely be work unto itself. -David
smime.p7s
Description: S/MIME cryptographic signature
