On segunda-feira, 22 de janeiro de 2018 16:56:56 PST Thiago Macieira wrote: > After fixing the converter example to properly use mmap in all three cases, > plus refactoring the CBOR parser to operate on a pre-loaded array and use > larger buffer than single-digit byte counts, the numbers are:
A major difference between the three is the memory allocation strategy. The Binary JSON needs none, since it uses an in-memory format for the on-disk format. So the application runs with 53880 kB of the mmapped file in a total of 64384 kB RSS. The parsed QJsonDocument uses the same allocation strategy, which one if its greatest advantages. It allocates a buffer equal to the size of the JSON input, so after unloading the 55872 kB of the source file, it keeps a huge block of 55792 kB allocated, in a total of 65928 kB RSS. A major drawback of this is that the Binary JSON memory format is limited to 128 MB in total. There's a bug report about this and changing the format to raise this limit, even for in-memory, will change the numbers above. QCborValue does no such thing, but it's also not limited. Since it uses regular memory allocation, after unloading the 40244 kB file, it needs 110356 kB of heap, for a total of 120428 kB RSS. For comparison, loading the same data from a QDataStream and keeping in a QVariant requires 233944 kB of heap, for a total of 244088 kB RSS. The QDataStream file is 76156 kB and that includes all strings stored as UTF-16, so it's not the UTF-16 vs UTF-8 vs Latin1 that accounts for the majority of the overhead. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center _______________________________________________ Development mailing list [email protected] http://lists.qt-project.org/mailman/listinfo/development
