On Wed, Mar 10, 2021 at 8:42 PM Tatu Saloranta <[email protected]> wrote: > > On Wed, Mar 10, 2021 at 5:09 PM 'Kireet Reddy' via jackson-user > <[email protected]> wrote: > > > > This explanation would make a lot of sense. I tested with some of our data > > where the keys are fixed, so it would explain why our benchmarks showed the > > expected speedup, but not all of our data did in production. Also you can > > ignore my time of day comment, I think we happened to have a code > > deployment around that time which would better explain the temporary > > slowdown. > > > > Thanks so much for your help and being such a great steward of this project! > > You are welcome! This is an interesting problem to tackle (Smile and > CBOR backends are something I like working on more than many other > components), and I created 2 issues wrt ability (or lack thereof) to > turn off canonicalization: > > * https://github.com/FasterXML/jackson-dataformats-binary/issues/252 (smile) > * https://github.com/FasterXML/jackson-dataformats-binary/issues/253 (cbor)
Quick update here: I implemented both for 2.13(.0-SNAPSHOT) so canonicalization can be turned off for both Smile and CBOR for Jackson 2.13. There is some performance overhead for that, for common use cases, but it may be worthwhile for specific kinds of data. Interesting enough, 3.0.0-SNAPSHOT has very little performance degradation for POJO use case: if we ever get there, this will be pretty useful setting there -- one can effectively have BOTH stellar performance for bounded key sets (POJOs) AND no overhead for unbounded cases (Map<UUID, POJO> values....)! -+ Tatu +- > > -+ Tatu +- > > > > > > On Wed, Mar 10, 2021 at 4:52 PM Tatu Saloranta <[email protected]> wrote: > >> > >> On Wed, Mar 10, 2021 at 12:59 PM Tatu Saloranta <[email protected]> wrote: > >> > > >> > On Wed, Mar 10, 2021 at 12:26 PM 'Kireet Reddy' via jackson-user > >> > <[email protected]> wrote: > >> > >> Ok, so looking at this specifically: > >> > >> ... > >> > >> > However when I actually enabled it for all our data, we noticed > >> > >> > things actually running slower. It seems one of our objects in > >> > >> > particular got about 10% slower. I captured a java flight recording > >> > >> > and it appears the hotspot is in > >> > >> > ByteQuadsCanonicalizer._verifySharing (the Arrays.copyOf > >> > >> > operations). I also enabled the afterburner module -- not sure if > >> > >> > that would have any adverse impacts. > >> .... > >> > >> It is a bit odd. Copy operation would occur only if the table is > >> shared and presumably would be a fairly big symbol table for it to > >> take time > >> (there is no synchronization in there). But if CANONICALIZATION was > >> disabled, it should not happen. > >> > >> But now that I am looking through bootstrapping, it looks like this > >> setting only affects JsonParsers but not Smile or CBOR parsers. > >> So I don't think disabling here has any effect. Given this it sounds > >> like your content does get a lot of unique keys and this is why > >> every time a copy is likely made. > >> > >> I think this may explain what you are seeing, assuming documents have > >> a big number of distinct unique keys: with JSON, canonicalization > >> being disabled, there is more constant steady overhead. With Smile > >> this is not the case at the moment, and for larger content negative > >> impact probably grows with the size of document (which has a larger > >> set of unique names?). > >> > >> Ideally Smile and CBOR parsers would allow disabling of > >> canonicalization. I think I will file 2 issues; while not trivial to > >> implement > >> (for JSON things are a bit easier as it can fall back to using > >> `Writer`-based parser), should be relatively straight-forward. > >> Downside is that this probably has to go in 2.13 and not in 2.12 > >> patch, depending on how intrusive changes are. > >> > >> So it looks like I got some work on my plate. :) > >> > >> -+ Tatu +- > >> > >> -- > >> You received this message because you are subscribed to a topic in the > >> Google Groups "jackson-user" group. > >> To unsubscribe from this topic, visit > >> https://groups.google.com/d/topic/jackson-user/azIcJNCUEkE/unsubscribe. > >> To unsubscribe from this group and all its topics, send an email to > >> [email protected]. > >> To view this discussion on the web visit > >> https://groups.google.com/d/msgid/jackson-user/CAL4a10ha9J0oUXz7D5ekc%3D%3DtdzeP0W91b70xmheEDM4MOKx%3DQw%40mail.gmail.com. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "jackson-user" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/jackson-user/CACkKG4g%2BY0qSLdaXySbjEneDQqCgNZ2MFYJx%3DJrGMubsvqC96g%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "jackson-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAL4a10jx8Qq5FD6H8Qu%3DtJyHp17wji7JbfEq%3DhGkJJP3WJeWaA%40mail.gmail.com.
