Re: [Python-ideas] Allow mutable builtin types (optionally)
Mostly historical reasons I guess, we started with static types because most class extension examples were using it, and it worked for all we did at the time (including the __class__ assign trick). We then got hit by the change, and solved the issue by patching python. Now keeping our own patched CPython is still perfectly viable, but there are disadvantages to not using the off-the-self interpreter: it prevent interacting with other proprietary libs within a single interpreter. So that would be plan C (BTW having a common interpreter is a necessary condition, but far from enough...) Plan A was to push the idea that extension type behavior could be more tunable: ATM you choose between static and heap type, and it comes with a bunch of other non obvious differences, whose combination is not necessarily the best for your use case or preferences (see for example http://grokbase.com/t/python/python-dev/097twacntz/py-tpflags-heaptype-too-overloaded ). I still think it's s good idea to allow for more tunable behavior, and certainly not use heap type as a marker for things not obviously related, but as it did not gather traction 10y ago, nor this time either, so I will not push for plan A. Time has teached me it is better to wait for the proposal come back from another source, than invest too much time in it if support, or at least discussion is not strong from the start. It will probably come back and pass later, in a form or another ( PEP225/465 ;-p ) Plan B is to see how much additional work it will be to use heap type, and if we do not suffer from it's other properties. If we end up requiring another patch, bye bye plan B and back to the original patch. Le mer. 27 juin 2018 à 19:46, Guido van Rossum a écrit : > So the question remains -- why not use a heap type? > > On Wed, Jun 27, 2018 at 2:05 AM Greg wrote: > >> As I introduced (a long time ago) this demand, let me add my grain of >> salt here. >> >> The use case is pretty simple, and somewhat common when writing manually >> C extension class: The reason to write extension class is usually >> performance, or link into an existing library. >> When doing this manually (instead of using automatic python-wrapping >> tools like boost, swig,...) you try to wrap the minimum amount of >> methods/accessors/...to your underlying c/c++ class, and replicate >> non-critical methods in python. >> Moreover, extending your class by adding new methods is usually much more >> easy in Python, especially if it involve complex but not >> performance-bounded python-data manipulation. >> Problem is to make those python-implemented methods avaible to instances >> of your extension class, especially when those instances are returned by >> the C layer of your extension. >> >> The solution we choose was to change the __class__ of each extension >> type instance to the python derived newclass implementing all those >> extra-methods.Not too difficult, a simple encapsulation of all methods >> returning extension-class instances is enough, and can be automated. >> This solution is quite common I think, it translate something you do for >> python-class instances, but then you get the __class__ assignment: only >> for heap types error. >> >> The argument about sub-interpreters is a good one, but not really >> applicable for this use case: we really want to have one extension type (or >> a hierarchy of it) shared across all interpreter importing the extension, >> it just happen that instead of being implemented in pure C/C++, the >> extension is implemented in C/C++ and Python. The fact that the python >> parts will be seen everywhere is a feature, not a problem: you expect the >> replacement of C-implemented methods by Python-implemented method to be as >> transparent as possible. >> >> Alternatives would be to use a heap type for our C extensions classes >> (we need to check what it would imply, but it may be quite painless) >> or use some form or delegation instead of assigning to __class__. >> The later is not really painless, AFAIK, in term of coding complexity and >> possibly performance (extra lookups steps needed). >> >> If there are other solutions or if delegation can be made as >> simple/efficient as the __class__ mechanism, it would be good to know, and >> it is I think valuable info for many people writing extension classes. >> Anyway, my personal position on this has not changed in 10y and is in >> line with Eloi: I think that beeing a heaptype and allowing assigment to >> the __class__ attribute of instances is indeed quite orthogonal.. >> >> >> >> . >> >> >> >> >> >> >> ___ >> Python-ideas mailing list >> Python-ideas@python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > -- > --Guido van Rossum (python.org/~guido) > ___ Python-ideas mailing list Python-ideas@python.org
Re: [Python-ideas] Allow mutable builtin types (optionally)
So the question remains -- why not use a heap type? On Wed, Jun 27, 2018 at 2:05 AM Greg wrote: > As I introduced (a long time ago) this demand, let me add my grain of salt > here. > > The use case is pretty simple, and somewhat common when writing manually C > extension class: The reason to write extension class is usually > performance, or link into an existing library. > When doing this manually (instead of using automatic python-wrapping tools > like boost, swig,...) you try to wrap the minimum amount of > methods/accessors/...to your underlying c/c++ class, and replicate > non-critical methods in python. > Moreover, extending your class by adding new methods is usually much more > easy in Python, especially if it involve complex but not > performance-bounded python-data manipulation. > Problem is to make those python-implemented methods avaible to instances > of your extension class, especially when those instances are returned by > the C layer of your extension. > > The solution we choose was to change the __class__ of each extension type > instance to the python derived newclass implementing all those > extra-methods.Not too difficult, a simple encapsulation of all methods > returning extension-class instances is enough, and can be automated. > This solution is quite common I think, it translate something you do for > python-class instances, but then you get the __class__ assignment: only > for heap types error. > > The argument about sub-interpreters is a good one, but not really > applicable for this use case: we really want to have one extension type (or > a hierarchy of it) shared across all interpreter importing the extension, > it just happen that instead of being implemented in pure C/C++, the > extension is implemented in C/C++ and Python. The fact that the python > parts will be seen everywhere is a feature, not a problem: you expect the > replacement of C-implemented methods by Python-implemented method to be as > transparent as possible. > > Alternatives would be to use a heap type for our C extensions classes (we > need to check what it would imply, but it may be quite painless) > or use some form or delegation instead of assigning to __class__. > The later is not really painless, AFAIK, in term of coding complexity and > possibly performance (extra lookups steps needed). > > If there are other solutions or if delegation can be made as > simple/efficient as the __class__ mechanism, it would be good to know, and > it is I think valuable info for many people writing extension classes. > Anyway, my personal position on this has not changed in 10y and is in line > with Eloi: I think that beeing a heaptype and allowing assigment to the > __class__ attribute of instances is indeed quite orthogonal.. > > > > . > > > > > > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow mutable builtin types (optionally)
Hi Brett, Sorry about that, I did not mean to be rude. What I wanted to says is: 1. That I relied on such a feature 2. Other people on this mailing-list already asked something similar at several occasions 3. HEAPTYPE would not always be a solution 4. Then I thought this was something that would indeed need more discussion and might get acceptance if discussed once again. Eloi From: Brett Cannon Sent: Tuesday, June 26, 2018 8:53 PM To: Eloi Gaudry Cc: encu...@gmail.com; python-ideas@python.org Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) On Thu, Jun 21, 2018, 12:27 Eloi Gaudry, mailto:eloi.gau...@fft.be>> wrote: This request didn't have a lot of traction, but I still consider this is something that would need to be supported Please be careful about using the word "need" as it comes off as demanding instead of as a suggestion. -Brett (2 lines of code to be changed; no regression so far with python 2 and python 3). My main points are: - HEAP_TYPE is not really used (as anyone being using it ?) - HEAP_TYPE serves other purposes - extension would benefit for allowing direct access to any of its type attributes Petr, what do you think ? Eloi From: Python-ideas mailto:fft...@python.org>> on behalf of Eloi Gaudry mailto:eloi.gau...@fft.be>> Sent: Tuesday, May 8, 2018 9:26:47 AM To: encu...@gmail.com<mailto:encu...@gmail.com>; python-ideas@python.org<mailto:python-ideas@python.org> Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) On Mon, 2018-05-07 at 15:23 -0400, Petr Viktorin wrote: > On 05/07/18 11:37, Eloi Gaudry wrote: > > I mean, to my knowledge, there is no reason why a type should be > > allocated on the heap (https://docs.python.org/2/c-api/typeobj.html > > ) to > > be able to change its attributes at Python level. > > One reason is sub-interpreter support: you can have multiple > interpreters per process, and those shouldn't influence each other. > (see https://docs.python.org/3/c-api/init.html#sub-interpreter-suppor > t) > > With heap types, each sub-interpreter can have its own copy of the > type > object. But with builtins, changes done in one interpreter would be > visible in all the others. Yes, this could be a reason, but if you don't rely on such a feature neither implicitly nor explicitly ? I mean, our types are built-in and should be considered as immutable across interpreters. And we (as most users I guess) are only running one interpreter. In case several intepreters are used, it would make sense to have a non-heap type that would be seen as a singleton across all of them, no ? ___ Python-ideas mailing list Python-ideas@python.org<mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org<mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow mutable builtin types (optionally)
As I introduced (a long time ago) this demand, let me add my grain of salt here. The use case is pretty simple, and somewhat common when writing manually C extension class: The reason to write extension class is usually performance, or link into an existing library. When doing this manually (instead of using automatic python-wrapping tools like boost, swig,...) you try to wrap the minimum amount of methods/accessors/...to your underlying c/c++ class, and replicate non-critical methods in python. Moreover, extending your class by adding new methods is usually much more easy in Python, especially if it involve complex but not performance-bounded python-data manipulation. Problem is to make those python-implemented methods avaible to instances of your extension class, especially when those instances are returned by the C layer of your extension. The solution we choose was to change the __class__ of each extension type instance to the python derived newclass implementing all those extra-methods.Not too difficult, a simple encapsulation of all methods returning extension-class instances is enough, and can be automated. This solution is quite common I think, it translate something you do for python-class instances, but then you get the __class__ assignment: only for heap types error. The argument about sub-interpreters is a good one, but not really applicable for this use case: we really want to have one extension type (or a hierarchy of it) shared across all interpreter importing the extension, it just happen that instead of being implemented in pure C/C++, the extension is implemented in C/C++ and Python. The fact that the python parts will be seen everywhere is a feature, not a problem: you expect the replacement of C-implemented methods by Python-implemented method to be as transparent as possible. Alternatives would be to use a heap type for our C extensions classes (we need to check what it would imply, but it may be quite painless) or use some form or delegation instead of assigning to __class__. The later is not really painless, AFAIK, in term of coding complexity and possibly performance (extra lookups steps needed). If there are other solutions or if delegation can be made as simple/efficient as the __class__ mechanism, it would be good to know, and it is I think valuable info for many people writing extension classes. Anyway, my personal position on this has not changed in 10y and is in line with Eloi: I think that beeing a heaptype and allowing assigment to the __class__ attribute of instances is indeed quite orthogonal.. . ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow mutable builtin types (optionally)
Hi Guido, I would like to be sure that the lack of support would not be the result of my inability to sum-up my use case. This is why I gave some links to illustrate -the reason why the behavior was changed a decade ago -that such a possibility was actually needed by other extension developers (where their built-in types would benefit from being able to redefine some method dynamically) -that python core developers and python extension developers can have different needs and objectives (which was the main reason why I was submitting this to the mailing-list again) I feel sorry if that only resulted in looking like I was repeating myself. Have a good day, Eloi From: Guido van Rossum Sent: Tuesday, June 26, 2018 7:20 PM To: Eloi Gaudry Cc: Python-Ideas ; Serhiy Storchaka Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) Hey Eloi, I think you need to just give up on this. Nobody here seems to support or understand your use case. At this point you are repeating yourself (again claiming there is no good reason for the prohibition and that it's only a few lines of code to change) and you can be assured that the response will also be the same. --Guido On Tue, Jun 26, 2018 at 8:00 AM Eloi Gaudry mailto:eloi.gau...@fft.be>> wrote: the origin of this feature disappearing for built-in types: http://bugs.jython.org/issue1058 ''' object.__set/delattr__ allow modification of built in types, this is known as the Carlo Verre hack: Jython 2.3a0+ (trunk:4630:4631M, Jun 14 2008, 20:07:38) [Java HotSpot(TM) Client VM (Apple Inc.)] on java1.5.0_13 Type "help", "copyright", "credits" or "license" for more information. >>> object.__setattr__(str, 'lower', str.upper) >>> 'dammit Carlo!'.lower() 'DAMMIT CARLO!' ''' but I do not see any reason why having an explicit flag for python extensions written in C to declare their types as static struct, and still be able to change their __setattr__, __getattr__, etc. slots would not make sense. extensions and core types have not the same constraints and purposes, this should be reflected on the capabilities the first would have somewhere then. From: Eloi Gaudry Sent: Tuesday, June 26, 2018 4:27:18 PM To: python-ideas@python.org<mailto:python-ideas@python.org> Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) some literature: https://mail.python.org/pipermail/python-dev/2008-February/077180.html https://mail.python.org/pipermail/python-dev/2008-February/077169.html where it is stated that python C struct type should not be able to have their attributes changed. but the extension needs is clearly not taken into account. From: Python-ideas mailto:fft...@python.org>> on behalf of Eloi Gaudry mailto:eloi.gau...@fft.be>> Sent: Thursday, June 21, 2018 5:26:37 PM To: python-ideas@python.org<mailto:python-ideas@python.org>; encu...@gmail.com<mailto:encu...@gmail.com> Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) This request didn't have a lot of traction, but I still consider this is something that would need to be supported (2 lines of code to be changed; no regression so far with python 2 and python 3). My main points are: - HEAP_TYPE is not really used (as anyone being using it ?) - HEAP_TYPE serves other purposes - extension would benefit for allowing direct access to any of its type attributes Petr, what do you think ? Eloi From: Python-ideas mailto:fft...@python.org>> on behalf of Eloi Gaudry mailto:eloi.gau...@fft.be>> Sent: Tuesday, May 8, 2018 9:26:47 AM To: encu...@gmail.com<mailto:encu...@gmail.com>; python-ideas@python.org<mailto:python-ideas@python.org> Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) On Mon, 2018-05-07 at 15:23 -0400, Petr Viktorin wrote: > On 05/07/18 11:37, Eloi Gaudry wrote: > > I mean, to my knowledge, there is no reason why a type should be > > allocated on the heap (https://docs.python.org/2/c-api/typeobj.html > > ) to > > be able to change its attributes at Python level. > > One reason is sub-interpreter support: you can have multiple > interpreters per process, and those shouldn't influence each other. > (see https://docs.python.org/3/c-api/init.html#sub-interpreter-suppor > t) > > With heap types, each sub-interpreter can have its own copy of the > type > object. But with builtins, changes done in one interpreter would be > visible in all the others. Yes, this could be a reason, but if you don't rely on such a feature neither implicitly nor explicitly ? I mean, our types are built-in and should be considered as immutable across interpreters. And we (as most users I g
Re: [Python-ideas] Allow mutable builtin types (optionally)
some literature: https://mail.python.org/pipermail/python-dev/2008-February/077180.html <https://mail.python.org/pipermail/python-dev/2008-February/077180.html>https://mail.python.org/pipermail/python-dev/2008-February/077169.html where it is stated that python C struct type should not be able to have their attributes changed. but the extension needs is clearly not taken into account. From: Python-ideas on behalf of Eloi Gaudry Sent: Thursday, June 21, 2018 5:26:37 PM To: python-ideas@python.org; encu...@gmail.com Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) This request didn't have a lot of traction, but I still consider this is something that would need to be supported (2 lines of code to be changed; no regression so far with python 2 and python 3). My main points are: - HEAP_TYPE is not really used (as anyone being using it ?) - HEAP_TYPE serves other purposes - extension would benefit for allowing direct access to any of its type attributes Petr, what do you think ? Eloi From: Python-ideas on behalf of Eloi Gaudry Sent: Tuesday, May 8, 2018 9:26:47 AM To: encu...@gmail.com; python-ideas@python.org Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) On Mon, 2018-05-07 at 15:23 -0400, Petr Viktorin wrote: > On 05/07/18 11:37, Eloi Gaudry wrote: > > I mean, to my knowledge, there is no reason why a type should be > > allocated on the heap (https://docs.python.org/2/c-api/typeobj.html > > ) to > > be able to change its attributes at Python level. > > One reason is sub-interpreter support: you can have multiple > interpreters per process, and those shouldn't influence each other. > (see https://docs.python.org/3/c-api/init.html#sub-interpreter-suppor > t) > > With heap types, each sub-interpreter can have its own copy of the > type > object. But with builtins, changes done in one interpreter would be > visible in all the others. Yes, this could be a reason, but if you don't rely on such a feature neither implicitly nor explicitly ? I mean, our types are built-in and should be considered as immutable across interpreters. And we (as most users I guess) are only running one interpreter. In case several intepreters are used, it would make sense to have a non-heap type that would be seen as a singleton across all of them, no ? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow mutable builtin types (optionally)
On Thu, Jun 21, 2018, 12:27 Eloi Gaudry, wrote: > This request didn't have a lot of traction, but I still consider this is > something that would need to be supported > Please be careful about using the word "need" as it comes off as demanding instead of as a suggestion. -Brett (2 lines of code to be changed; no regression so far with python 2 and > python 3). > > > My main points are: > > - HEAP_TYPE is not really used (as anyone being using it ?) > > - HEAP_TYPE serves other purposes > > - extension would benefit for allowing direct access to any of its > type attributes > > Petr, what do you think ? > > Eloi > -- > *From:* Python-ideas > on behalf of Eloi Gaudry > *Sent:* Tuesday, May 8, 2018 9:26:47 AM > *To:* encu...@gmail.com; python-ideas@python.org > *Subject:* Re: [Python-ideas] Allow mutable builtin types (optionally) > > On Mon, 2018-05-07 at 15:23 -0400, Petr Viktorin wrote: > > On 05/07/18 11:37, Eloi Gaudry wrote: > > > I mean, to my knowledge, there is no reason why a type should be > > > allocated on the heap (https://docs.python.org/2/c-api/typeobj.html > > > ) to > > > be able to change its attributes at Python level. > > > > One reason is sub-interpreter support: you can have multiple > > interpreters per process, and those shouldn't influence each other. > > (see https://docs.python.org/3/c-api/init.html#sub-interpreter-suppor > > t) > > > > With heap types, each sub-interpreter can have its own copy of the > > type > > object. But with builtins, changes done in one interpreter would be > > visible in all the others. > > Yes, this could be a reason, but if you don't rely on such a feature > neither implicitly nor explicitly ? > > I mean, our types are built-in and should be considered as immutable > across interpreters. And we (as most users I guess) are only running > one interpreter. > > In case several intepreters are used, it would make sense to have a > non-heap type that would be seen as a singleton across all of them, no > ? > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow mutable builtin types (optionally)
Hey Eloi, I think you need to just give up on this. Nobody here seems to support or understand your use case. At this point you are repeating yourself (again claiming there is no good reason for the prohibition and that it's only a few lines of code to change) and you can be assured that the response will also be the same. --Guido On Tue, Jun 26, 2018 at 8:00 AM Eloi Gaudry wrote: > the origin of this feature disappearing for built-in types: > > http://bugs.jython.org/issue1058 > > > ''' > > object.__set/delattr__ allow modification of built in types, this is > known as the Carlo Verre hack: > > Jython 2.3a0+ (trunk:4630:4631M, Jun 14 2008, 20:07:38) > [Java HotSpot(TM) Client VM (Apple Inc.)] on java1.5.0_13 > Type "help", "copyright", "credits" or "license" for more information. > >>> object.__setattr__(str, 'lower', str.upper) > >>> 'dammit Carlo!'.lower() > 'DAMMIT CARLO!' > ''' > > but I do not see any reason why having an explicit flag for > python extensions written in C to declare their types as static struct, and > still be able to change their __setattr__, __getattr__, etc. slots would > not make sense. > > extensions and core types have not the same constraints and purposes, this > should be reflected on the capabilities the first would have somewhere then. > > > > > -------------- > *From:* Eloi Gaudry > *Sent:* Tuesday, June 26, 2018 4:27:18 PM > *To:* python-ideas@python.org > *Subject:* Re: [Python-ideas] Allow mutable builtin types (optionally) > > > some literature: > > https://mail.python.org/pipermail/python-dev/2008-February/077180.html > > <https://mail.python.org/pipermail/python-dev/2008-February/077180.html> > https://mail.python.org/pipermail/python-dev/2008-February/077169.html > > > where it is stated that python C struct type should not be able to have > their attributes changed. > > but the extension needs is clearly not taken into account. > > -------------- > *From:* Python-ideas > on behalf of Eloi Gaudry > *Sent:* Thursday, June 21, 2018 5:26:37 PM > *To:* python-ideas@python.org; encu...@gmail.com > *Subject:* Re: [Python-ideas] Allow mutable builtin types (optionally) > > > This request didn't have a lot of traction, but I still consider this is > something that would need to be supported (2 lines of code to be changed; > no regression so far with python 2 and python 3). > > > My main points are: > > - HEAP_TYPE is not really used (as anyone being using it ?) > > - HEAP_TYPE serves other purposes > > - extension would benefit for allowing direct access to any of its > type attributes > > > Petr, what do you think ? > > Eloi > -- > *From:* Python-ideas > on behalf of Eloi Gaudry > *Sent:* Tuesday, May 8, 2018 9:26:47 AM > *To:* encu...@gmail.com; python-ideas@python.org > *Subject:* Re: [Python-ideas] Allow mutable builtin types (optionally) > > On Mon, 2018-05-07 at 15:23 -0400, Petr Viktorin wrote: > > On 05/07/18 11:37, Eloi Gaudry wrote: > > > I mean, to my knowledge, there is no reason why a type should be > > > allocated on the heap (https://docs.python.org/2/c-api/typeobj.html > > > ) to > > > be able to change its attributes at Python level. > > > > One reason is sub-interpreter support: you can have multiple > > interpreters per process, and those shouldn't influence each other. > > (see https://docs.python.org/3/c-api/init.html#sub-interpreter-suppor > > t) > > > > With heap types, each sub-interpreter can have its own copy of the > > type > > object. But with builtins, changes done in one interpreter would be > > visible in all the others. > > Yes, this could be a reason, but if you don't rely on such a feature > neither implicitly nor explicitly ? > > I mean, our types are built-in and should be considered as immutable > across interpreters. And we (as most users I guess) are only running > one interpreter. > > In case several intepreters are used, it would make sense to have a > non-heap type that would be seen as a singleton across all of them, no > ? > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow mutable builtin types (optionally)
the origin of this feature disappearing for built-in types: http://bugs.jython.org/issue1058 ''' object.__set/delattr__ allow modification of built in types, this is known as the Carlo Verre hack: Jython 2.3a0+ (trunk:4630:4631M, Jun 14 2008, 20:07:38) [Java HotSpot(TM) Client VM (Apple Inc.)] on java1.5.0_13 Type "help", "copyright", "credits" or "license" for more information. >>> object.__setattr__(str, 'lower', str.upper) >>> 'dammit Carlo!'.lower() 'DAMMIT CARLO!' ''' but I do not see any reason why having an explicit flag for python extensions written in C to declare their types as static struct, and still be able to change their __setattr__, __getattr__, etc. slots would not make sense. extensions and core types have not the same constraints and purposes, this should be reflected on the capabilities the first would have somewhere then. From: Eloi Gaudry Sent: Tuesday, June 26, 2018 4:27:18 PM To: python-ideas@python.org Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) some literature: https://mail.python.org/pipermail/python-dev/2008-February/077180.html <https://mail.python.org/pipermail/python-dev/2008-February/077180.html>https://mail.python.org/pipermail/python-dev/2008-February/077169.html where it is stated that python C struct type should not be able to have their attributes changed. but the extension needs is clearly not taken into account. From: Python-ideas on behalf of Eloi Gaudry Sent: Thursday, June 21, 2018 5:26:37 PM To: python-ideas@python.org; encu...@gmail.com Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) This request didn't have a lot of traction, but I still consider this is something that would need to be supported (2 lines of code to be changed; no regression so far with python 2 and python 3). My main points are: - HEAP_TYPE is not really used (as anyone being using it ?) - HEAP_TYPE serves other purposes - extension would benefit for allowing direct access to any of its type attributes Petr, what do you think ? Eloi From: Python-ideas on behalf of Eloi Gaudry Sent: Tuesday, May 8, 2018 9:26:47 AM To: encu...@gmail.com; python-ideas@python.org Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) On Mon, 2018-05-07 at 15:23 -0400, Petr Viktorin wrote: > On 05/07/18 11:37, Eloi Gaudry wrote: > > I mean, to my knowledge, there is no reason why a type should be > > allocated on the heap (https://docs.python.org/2/c-api/typeobj.html > > ) to > > be able to change its attributes at Python level. > > One reason is sub-interpreter support: you can have multiple > interpreters per process, and those shouldn't influence each other. > (see https://docs.python.org/3/c-api/init.html#sub-interpreter-suppor > t) > > With heap types, each sub-interpreter can have its own copy of the > type > object. But with builtins, changes done in one interpreter would be > visible in all the others. Yes, this could be a reason, but if you don't rely on such a feature neither implicitly nor explicitly ? I mean, our types are built-in and should be considered as immutable across interpreters. And we (as most users I guess) are only running one interpreter. In case several intepreters are used, it would make sense to have a non-heap type that would be seen as a singleton across all of them, no ? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow mutable builtin types (optionally)
This request didn't have a lot of traction, but I still consider this is something that would need to be supported (2 lines of code to be changed; no regression so far with python 2 and python 3). My main points are: - HEAP_TYPE is not really used (as anyone being using it ?) - HEAP_TYPE serves other purposes - extension would benefit for allowing direct access to any of its type attributes Petr, what do you think ? Eloi From: Python-ideas on behalf of Eloi Gaudry Sent: Tuesday, May 8, 2018 9:26:47 AM To: encu...@gmail.com; python-ideas@python.org Subject: Re: [Python-ideas] Allow mutable builtin types (optionally) On Mon, 2018-05-07 at 15:23 -0400, Petr Viktorin wrote: > On 05/07/18 11:37, Eloi Gaudry wrote: > > I mean, to my knowledge, there is no reason why a type should be > > allocated on the heap (https://docs.python.org/2/c-api/typeobj.html > > ) to > > be able to change its attributes at Python level. > > One reason is sub-interpreter support: you can have multiple > interpreters per process, and those shouldn't influence each other. > (see https://docs.python.org/3/c-api/init.html#sub-interpreter-suppor > t) > > With heap types, each sub-interpreter can have its own copy of the > type > object. But with builtins, changes done in one interpreter would be > visible in all the others. Yes, this could be a reason, but if you don't rely on such a feature neither implicitly nor explicitly ? I mean, our types are built-in and should be considered as immutable across interpreters. And we (as most users I guess) are only running one interpreter. In case several intepreters are used, it would make sense to have a non-heap type that would be seen as a singleton across all of them, no ? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow mutable builtin types (optionally)
On Mon, 2018-05-07 at 15:23 -0400, Petr Viktorin wrote: > On 05/07/18 11:37, Eloi Gaudry wrote: > > I mean, to my knowledge, there is no reason why a type should be > > allocated on the heap (https://docs.python.org/2/c-api/typeobj.html > > ) to > > be able to change its attributes at Python level. > > One reason is sub-interpreter support: you can have multiple > interpreters per process, and those shouldn't influence each other. > (see https://docs.python.org/3/c-api/init.html#sub-interpreter-suppor > t) > > With heap types, each sub-interpreter can have its own copy of the > type > object. But with builtins, changes done in one interpreter would be > visible in all the others. Yes, this could be a reason, but if you don't rely on such a feature neither implicitly nor explicitly ? I mean, our types are built-in and should be considered as immutable across interpreters. And we (as most users I guess) are only running one interpreter. In case several intepreters are used, it would make sense to have a non-heap type that would be seen as a singleton across all of them, no ? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow mutable builtin types (optionally)
On 05/07/18 11:37, Eloi Gaudry wrote: Hi, I'd like to bring back this discussion (from 2005, by Greg): https://bugs.python.org/issue1229239 Briefly, non-heap types cannot have their attributes changed by Python code. This makes sense for python builtin types, but not for the types defined in extension/modules. As we have been using this patch for the very same reasons and for more than 10 years, I think it might make sense to reconsider the discussion that Greg started. The main question at that time was "why not using a heap type instead ?" (because heap-type do not have this limitation). But I think that the right question could have been "why imposing such a restriction on non-heap types as defined in (non Python core) extensions ?". I mean, to my knowledge, there is no reason why a type should be allocated on the heap (https://docs.python.org/2/c-api/typeobj.html) to be able to change its attributes at Python level. One reason is sub-interpreter support: you can have multiple interpreters per process, and those shouldn't influence each other. (see https://docs.python.org/3/c-api/init.html#sub-interpreter-support) With heap types, each sub-interpreter can have its own copy of the type object. But with builtins, changes done in one interpreter would be visible in all the others. I'm not saying that all non-heap types should be able to do so, just that it would make sense to allow this behavior, as an option (bit flag). At the implementation level, the changes needed are really limited (about a few lines): - Include/object.h - Objects/typeobject.c: Eloi ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
