[Python-Dev] Make HAMT available to python script

[zhang kai]

Hi,

HAMT is a very useful immutable mapping type. Currently CPython use it 
internally to implement contextvar. Considering immutable data structure is 
very useful I hope we can make it available to python script(maybe via 
collections module).

Immutable data structures are fundamental parts of our project. Currently we 
use a full-featured python immutable data library called pyrsistent. Pyrsistent 
is very powerful, however the map type in it is implemented in python script 
not in C. It becomes slow when the data set is relatively large.

On the other hand, CPython now already has an immutable mapping type in it. I 
think maybe it’s a good idea to make it public? Many projects can benefit from 
it I believe.

Here is a talk given by the author of javascript immutable-js library explain 
why immutable data structures are powerful: 
https://www.youtube.com/watch?v=I7IdS-PbEgI

Pyristent: https://github.com/tobgu/pyrsistent

What do you think?

Cheers,
Kai
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2WYPX44WBFS2ZMZFNMDFMRPROB7G34JQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Make HAMT available to python script

[zhang kai]

Thanks Victor and Pablo. I will check the discussion of PEP 603. It's a
little weird to use the immutables library when it's code in already in
CPython but I'm glad it's an option.

Kai

On Fri, Apr 1, 2022 at 6:14 PM Pablo Galindo Salgado 
wrote:

> You may want to check PEP 603, which more or less proposes this (the
> author of the pep is the author of the HAMT code)
>  check https://peps.python.org/pep-0603/
>
> Alternatively, there is already a pypi package with this code:
>
> https://pypi.org/project/immutables/
>
> Regards from cloudy London,
> Pablo
>
> On Fri, 1 Apr 2022, 10:34 zhang kai,  wrote:
>
>> Hi,
>>
>> HAMT is a very useful immutable mapping type. Currently CPython use it
>> internally to implement contextvar. Considering immutable data structure is
>> very useful I hope we can make it available to python script(maybe via
>> collections module).
>>
>> Immutable data structures are fundamental parts of our project. Currently
>> we use a full-featured python immutable data library called pyrsistent.
>> Pyrsistent is very powerful, however the map type in it is implemented in
>> python script not in C. It becomes slow when the data set is relatively
>> large.
>>
>> On the other hand, CPython now already has an immutable mapping type in
>> it. I think maybe it’s a good idea to make it public? Many projects can
>> benefit from it I believe.
>>
>> Here is a talk given by the author of javascript immutable-js library
>> explain why immutable data structures are powerful:
>> https://www.youtube.com/watch?v=I7IdS-PbEgI
>>
>> Pyristent: https://github.com/tobgu/pyrsistent
>>
>> What do you think?
>>
>> Cheers,
>> Kai
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/2WYPX44WBFS2ZMZFNMDFMRPROB7G34JQ/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RAD37ETHP5BXR4QZQKZIBVLHPEGK2HYO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Make HAMT available to python script

[zhang kai]

Hi, Steve. Thanks for your detailed explanation. Indeed I already saw the
API discussion in PEP 603. It's much easier to make the decision in a
third-party library. I think we will be fine with the immutables library.

On Fri, Apr 1, 2022 at 7:05 PM Steve Dower  wrote:

> On 4/1/2022 11:48 AM, zhang kai wrote:
> > Thanks Victor and Pablo. I will check the discussion of PEP 603. It's a
> > little weird to use the immutables library when it's code in already in
> > CPython but I'm glad it's an option.
>
> The main difference is that 'immutables' offers you a stable/versioned
> interface to use it, while the one that's in CPython is an internal
> implementation detail. If one day we find a better design, we can just
> switch to it, while 'immutables' probably can't. If we've exposed as a
> public interface in the core runtime, it's much more complicated.
>
> (For what it's worth, the major thing that held up contextvars in the
> first place was justifying why it needed a new data structure that
> wasn't already in the core runtime. So we didn't adopt it lightly, and
> making sure we kept the freedom to change it was an important compromise.)
>
> There are plenty of other things in this same category, and because we
> want to keep things as stable as possible while also improving
> performance and reliability, we have to keep pretty tight limits on what
> we promise will remain stable. Most of our discussions are about finding
> this balance ;)
>
> Cheers,
> Steve
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4OZ2DNZ3Q3LY6CUGJEUTXGZRNIX73FYQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Make HAMT available to python script

[zhang kai]

>
> All of that kind of adds up to make it a distraction that's likely worse
> than a dict subclass with __setitem__ overridden (if that's the
> behaviour you want). It's a fantastic data structure for when you need
> it, but it's one that deserves to be researched and understood before
> simply dropping it into your code. An optional package is a perfectly
> good place for it.
>

I agree with this. I just want to explain a little bit about why we want to
mutate an immutable map.

Immutable data structures have some nice features. Because of its immutable
nature, immutable data structures are inherently thread-safe so much safer
to use in a multi-thread environment. And it can be much faster to find the
differences between two immutable data structures because we can replace !=
with is not , therefore quickly rule out all unchanged data.

The immutable nature is actually very powerful when you want to change a
map(especially with nested maps in it) often, then need to find the
differences in other places. It sometimes reminds me of git. With immutable
data, we can keep every snapshot of data history in memory. Reduced memory
usage is also important.

I think immutable map and frozendict are actually two different data types
because they have different intentions.

The author of immutable.js gives a great talk about how immutable data
structure is implemented and why it’s powerful. React.js actually relies a
lot on immutable data structures. Here is the youtube link to the talk:

https://www.youtube.com/watch?v=I7IdS-PbEgI&t=1005s
There is also a CppCon talk about immutable data structure:

https://www.youtube.com/watch?v=sPhpelUfu8Q&t=412s
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HJUY734PUZ3CMHF7SMLM27D63JRWH7RQ/
Code of Conduct: http://python.org/psf/codeofconduct/