On Thu, Jul 4, 2019 at 11:32 PM Inada Naoki <songofaca...@gmail.com> wrote:
>
> On Thu, Jul 4, 2019 at 8:09 PM Antoine Pitrou <solip...@pitrou.net> wrote:
> >
> > Ah, interesting. Were you able to measure the memory footprint as well?
> >
>
> Hmm, it is not good. mimalloc uses MADV_FREE so it may affect to some
> benchmarks. I will look it later.
>
> ```
> $ ./python -m pyperf compare_to pymalloc-mem.json mimalloc-mem.json -G
> Slower (60):
> - logging_format: 10.6 MB +- 384.2 kB -> 27.2 MB +- 21.3 kB: 2.58x
> slower (+158%)
> - logging_simple: 10028.4 kB +- 371.2 kB -> 22.2 MB +- 24.9 kB: 2.27x
> slower (+127%)
I think I understand why the mimalloc uses more than twice memory than the
pymalloc + glibc malloc in logging_format and logging_simple benchmarks.
These two benchmarks does like this:
buf = [] # in StringIO
for _ in range(10*1024):
buf.append("important: some important information to be logged")
s = "".join(buf) # StringIO.getvalue()
s.splitlines()
mimalloc uses size segregated allocator for ~512KiB. And size class
is determined
top three bits.
On the other hand, list increases capacity by 9/8. It means next size
class is used
on each realloc. At last, all size classes has1~3 used/cached memory blocks.
This is almost worst case for mimalloc. In more complex application,
there may be
more chance to reuse memory blocks.
In complex or huge application, this overhead will become relatively small.
It's speed is attractive.
But for memory efficiency, pymalloc + jemalloc / tcmalloc may be better for
common cases.
Regards,
--
Inada Naoki <songofaca...@gmail.com>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/ORVLH5FAEO7LVE7SK44TQR6XK4YRRZ7L/
