On Tue, Mar 31, 2020 at 7:04 AM Paul Sokolovsky <pmis...@gmail.com> wrote:
> for i in range(50000):
> v = u"==%d==" % i
> # All individual strings will be kept in the list and
> # can't be GCed before teh final join.
> sz += sys.getsizeof(v)
> sb.append(v)
> s = "".join(sb)
> sz += sys.getsizeof(sb)
> sz += sys.getsizeof(s)
> print(sz)
>
> ... about order of magnitude more memory ...
I suspect you may be multiply-counting some of your usage here. Rather
than this, it would be more reliable to use the resident set size (on
platforms where you can query that).
if "strio" in sys.argv: strio()
else: listjoin()
print("Max RSS:", resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
Based on that, I find that it's at worst a 4:1 difference. Plus, I
couldn't see any material difference - the numbers were within half a
percent, basically just noise - until I upped your loop counter to
400,000, nearly ten times as much as you were doing. (At that point it
became a 2:1 difference. The 4:1 didn't show up until a lot later.) So
you have to be working with a *ridiculous* number of strings before
there's anything to even consider.
And even then, it's only notable if the individual strings are short
AND all unique. Increasing the length of the strings basically made it
a wash. Consider:
for i in range(1000000):
sb.write(u"==%d==" % i + "*"*1024)
Max RSS: 2028060
for i in range(1000000):
v = u"==%d==" % i + "*"*1024
Max RSS: 2104204
So at this point, the string join is slightly faster and takes
slightly more memory - within 20% on the time and within 5% on the
memory.
ChrisA
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/253V2BEV5UTMBKGRFOWM4Z4OOTJIALN7/
Code of Conduct: http://python.org/psf/codeofconduct/
