[Python-ideas] Re: Fwd: Re: Experimenting with dict performance, and an immutable dict

2020-09-16 Thread Marco Sulla
Well, I simply forgot vectorcall. I've done a test and it seems the speedup is about 25%. Unluckily, now I abandoned definitively the hope of a big big resize, but it seems I don't resize correctly, since now I have again an assert error on dk_usable. What is the difference between dk_usable and

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Rob Cliffe via Python-ideas
On 14/09/2020 17:36, Christopher Barker wrote: There seems to be a fair bit of support for this idea. Will it need a PEP ? -CHB If I've understood correctly (far from certain) the existing json.dumps and json.loads functions are permissive (allow some constructions that are not part of

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Christopher Barker
Is that suggested code? I don't follow. But if it is, no. personally, I think ANY use of system settings is a bad idea [*]. But certainly no need to even think about it for JSON. -CHB * have we not learned that in the age of the internet the machine the code happens to be running on has nothing

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Christopher Barker
On Wed, Sep 16, 2020 at 12:59 AM Rob Cliffe via Python-ideas < python-ideas@python.org> wrote: > On 14/09/2020 17:36, Christopher Barker wrote: > > nstructions that are not part of the JSON spec) but the proposed new > functions will be strict. > as it looks like I maybe the one to write the

[Python-ideas] Re: 'Infinity' constant in Python

2020-09-16 Thread Christopher Barker
Could you all please start another thread if you want to discuss possible changes to Error handling for floats. Or anything that isn't strictly adding some names to builtins. There's been ongoing confusion from the expansion of the original topic here. Thanks, -CHB On Wed, Sep 16, 2020 at

[Python-ideas] Re: 'Infinity' constant in Python

2020-09-16 Thread Stephen J. Turnbull
Paul Moore writes: > > > And as soon as we start considering integer division, we're talking > > > about breaking a *vast* amount of code. > > > > Yeah, I'm ok with *not* breaking that code. > > You may have misunderstood me - when I said "integer division", I > meant "division of two

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread lammenspaolo
Maybe unrelated, but the same goes for `pickle.load` and `pickle.dump`. For consistencies, any changes made to `json.load` and `json.dump` (e.g. adding `json.loadf` and `json.dumpf` or accepting a path like as argument) should be also applied equivalently to `pickle.load` and `pickle.dump`.

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Wes Turner
On Wed, Sep 16, 2020, 5:18 PM Christopher Barker wrote: > On Tue, Sep 15, 2020 at 5:26 PM Wes Turner wrote: > >> On Tue, Sep 15, 2020 at 9:09 AM Wes Turner wrote: >> >>> json.load and json.dump already default to UTF8 and already have parameters for json loading and dumping. >>> > so

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Christopher Barker
I believe Sergie already suggested pickle and marshall, and I guess we can add plistlib to those. Personally, I'm not so sure it should be added to all these. I see why the same API was used for all of them, but they really are fairly different beasts. So if they have a function with the same

[Python-ideas] Re: Fwd: Re: Experimenting with dict performance, and an immutable dict

2020-09-16 Thread Wes Turner
Would an e.g. bm_dict.py in [1] be a good place for a few benchmarks of dict; or is there a more appropriate project for authoritatively measuring performance regressions and optimizations of core {cpython,} data structures? [1]

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Inada Naoki
On Thu, Sep 17, 2020 at 6:54 AM Wes Turner wrote: > > > Why would we impose UTF-8 when the spec says UTF-8, UTF-16, or UTF-32? Obsolete JSON spec said UTF-8, UTF-16, and UTF-32. Current spec says UTF-8. See https://tools.ietf.org/html/rfc8259#section-8.1 So `dumpf` must use UTF-8, although

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Wes Turner
https://docs.python.org/3/library/os.html#os.fspath *__fspath__ On Wed, Sep 16, 2020, 5:53 PM Wes Turner wrote: > > > On Wed, Sep 16, 2020, 5:18 PM Christopher Barker > wrote: > >> On Tue, Sep 15, 2020 at 5:26 PM Wes Turner wrote: >> >>> On Tue, Sep 15, 2020 at 9:09 AM Wes Turner wrote: >>>

[Python-ideas] Re: Fwd: Re: Experimenting with dict performance, and an immutable dict

2020-09-16 Thread Inada Naoki
On Thu, Sep 17, 2020 at 8:03 AM Marco Sulla wrote: > > Well, it seems ok now: > https://github.com/python/cpython/compare/master...Marco-Sulla:master > > I've done a quick speed test and speedup is quite high for a creation > using keywods or a dict with "holes": about 30%: 30% on microbenchmark

[Python-ideas] Re: Fwd: Re: Experimenting with dict performance, and an immutable dict

2020-09-16 Thread Marco Sulla
Well, it seems ok now: https://github.com/python/cpython/compare/master...Marco-Sulla:master I've done a quick speed test and speedup is quite high for a creation using keywods or a dict with "holes": about 30%: python -m timeit -n 2000 --setup "from uuid import uuid4 ; o =

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Christopher Barker
On Tue, Sep 15, 2020 at 5:26 PM Wes Turner wrote: > On Tue, Sep 15, 2020 at 9:09 AM Wes Turner wrote: > >> json.load and json.dump already default to UTF8 and already have >>> parameters for json loading and dumping. >>> >> so it turns out that loads(), which optionally takes a bytes or

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Chris Angelico
On Thu, Sep 17, 2020 at 9:53 AM wrote: > > Maybe unrelated, but the same goes for `pickle.load` and `pickle.dump`. For > consistencies, any changes made to `json.load` and `json.dump` (e.g. adding > `json.loadf` and `json.dumpf` or accepting a path like as argument) should be > also applied

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Christopher Barker
On Wed, Sep 16, 2020 at 2:53 PM Wes Turner wrote: > So I was not correct: dump does not default to UTF-8 (and does not accept > an encoding= parameter) > > >> I think dumpf() should use UTF-8, and that's it. If anyone really wants >> something else, they can get it by providing an open text file

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread William Pickard
os.fspath exists for a reason. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at

[Python-ideas] Re: Fwd: Re: Experimenting with dict performance, and an immutable dict

2020-09-16 Thread Wes Turner
That sounds like a worthwhile optimization. FWIW, is this a bit simpler but sufficient?: python -m timeit -n 2000 --setup "from uuid import uuid4; \ o = {uuid4().hex: i for i in range(1)}" \ "dict(**o)" Is there a preferred tool to comprehensively measure the performance impact of a

[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf

2020-09-16 Thread Wes Turner
Is all of this locale/encoding testing necessary (or even sufficient)? ```python import json import locale import os def get_default_encoding(): """ TODO XXX: ??? """ default_encoding = locale.getdefaultlocale()[1] if default_encoding.startswith("UTF-"): return