On Wed, Mar 10, 2021 at 12:26 PM 'Kireet Reddy' via jackson-user <[email protected]> wrote: > > Thanks for the reply! > > I am pretty sure the parser gets closed, we only have one place where we do > smile format deserialization and it's called like this: > smileObjectMapper.readValue(in, clazz);
Ok. That will close the `in` if (but only if) `StreamReadFeature.AUTO_CLOSE_SOURCE` (or its deprecated predecessor in `JsonParser.Feature`) is enabled. > where in is an InputStream. The jackson code seems to close the create > JsonParser in a try/resource block. I can still try to disable > SmileGenerator.Feature.CHECK_SHARED_NAMES. Would it be ok to do this if we've > already serialized data with this enabled, or do we need to regenerate all of > our data? CHECK_SHARED_NAMES does change output (generally reducing size), so to see any effect you would need to regenerate. > Regarding JsonFactory.Feature.CANONICALIZE_FIELD_NAMES, we disabled this a > long time ago as we had memory issues with it enabled. We have some > structures with Maps with user generated keys, and I guess the > canonicalization table may have gotten too big? That is possible -- but disabling this feature will definitely reduce performance since all the keys will need to be re-created all the time for every document. Maximum symbol table size is bound to something like 8k or 16k entries but I can see how that might still be too big for some cases. One key question would then be whether there is just one (or small number) `SmileFactory` (and typically `ObjectMapper` that references it), or more: symbol tables are shared by all parsers created by the same factory. But not across factories. Canonicalization has similar effect for JSON and Smile, although probably more drastic for JSON actually. So that alone should not explain difference between formats. But lack of canonicalization might explain something about locking/thread-sync: maybe JSON alternative has bit more optimized handling there (I recall working through issues in that area at some point; and code for Smile does differ a bit), -+ Tatu +- > On Wednesday, March 10, 2021 at 12:04:20 PM UTC-8 Tatu Saloranta wrote: >> >> On Wed, Mar 10, 2021 at 11:55 AM 'Kireet Reddy' via jackson-user >> <[email protected]> wrote: >> > >> > Hello, >> > We recently switched a lot of our jackson data to smile format. I did some >> > benchmarks on some of our data and we saw good deserialization speedup of >> > around 30% when using smile rather than text. We are mostly concerned with >> > deserialization as we do many more of those ops. >> >> Ok good, that makes sense. >> >> One option you may want to try out, if deserialization matters more, is >> >> SmileGenerator.Feature.CHECK_SHARED_STRING_VALUES >> >> which defaults to false. Enabling that would typically reduce size, >> speed up deserialization, if there are repeated not-super-long String >> values in content. >> >> > However when I actually enabled it for all our data, we noticed things >> > actually running slower. It seems one of our objects in particular got >> > about 10% slower. I captured a java flight recording and it appears the >> > hotspot is in ByteQuadsCanonicalizer._verifySharing (the Arrays.copyOf >> > operations). I also enabled the afterburner module -- not sure if that >> > would have any adverse impacts. >> >> I'll have to look into this to say for sure but one possibility is >> that if `SmileParser` instances were not closed after use, this could >> be a symptom. >> But I'll have better idea later on. >> >> As to Afterburner: no, it should not matter here I think (although >> with HotSpot/JIT it is hard to rule out anything). >> >> > We also have a time of day when one of our objects gets larger. We did see >> > slower performance for this object during that time, which was expected. >> > But we also noticed a slow down during that time in other objects that >> > don't change size. Is there any shared resource that could explain that? >> > >> > Is it expected that smile could be slower for some data structures? I had >> > assumed it would just generally be faster. >> >> It should be, yes. One other thing you coudl try disabling (enabled by >> default) would be >> >> SmileGenerator.Feature.CHECK_SHARED_NAMES >> >> which should not cause the issue, but could give some more information >> about the issue. >> >> > I think our configuration is pretty straightforward, we do have some >> > custom serializers but otherwise it's just: >> > >> > SmileFactoryBuilder factoryBuilder = new SmileFactoryBuilder(new >> > SmileFactory()) >> > .disable(JsonFactory.Feature.CANONICALIZE_FIELD_NAMES) >> >> Ohhhhh. No, you should probably NOT disable this ^^^^ >> >> > .disable(JsonFactory.Feature.INTERN_FIELD_NAMES) >> > .disable(StreamWriteFeature.AUTO_CLOSE_TARGET); >> > >> > ObjectMapper m = new ObjectMapper(factoryBuilder.build()); >> > m.configure(MapperFeature.USE_GETTERS_AS_SETTERS, false); >> > m.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false); >> > m.configure(SerializationFeature.CLOSE_CLOSEABLE, false); >> > m.setSerializationInclusion(JsonInclude.Include.NON_NULL); >> > m.registerModule(new AfterburnerModule()); >> >> Yes, these make sense. >> >> -+ Tatu +- >> >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "jackson-user" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to [email protected]. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msgid/jackson-user/a3433732-9b29-4399-99ab-fd099da4f2d4n%40googlegroups.com. > > -- > You received this message because you are subscribed to the Google Groups > "jackson-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jackson-user/1f03dc85-f42c-4c35-86da-6117212039d6n%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "jackson-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAL4a10hStSbwSCAHkJ2f0Pqg%2B8nNoPnFCg7tvT0WxF0ppOCisA%40mail.gmail.com.
