Re: Copy-on-write friendly Python garbage collection (Posting On Python-List Prohibited)

2018-01-01 Thread Niles Rogoff
> Niles, if you want to claim wxjmfauth is right, you'll have to present > some actual evidence.  He's claimed for years that Python's Unicode > support is buggy (as he does here), without ever demonstrating a bug. > We've long ago tired of trying to reason with him. > > The tradeoffs of memory

Re: Copy-on-write friendly Python garbage collection (Posting On Python-List Prohibited)

2018-01-01 Thread Ned Batchelder
On 1/1/18 1:49 PM, Niles Rogoff wrote: On Mon, 01 Jan 2018 10:42:58 -0800, breamoreboy wrote: On Monday, January 1, 2018 at 10:14:59 AM UTC, wxjm...@gmail.com wrote: Le lundi 1 janvier 2018 08:35:53 UTC+1, Lawrence D’Oliveiro a écrit : On Monday, January 1, 2018 at 7:52:48 AM UTC+13, Paul

Re: Copy-on-write friendly Python garbage collection

2018-01-01 Thread breamoreboy
On Monday, January 1, 2018 at 12:53:03 PM UTC, Wu Xi wrote: > breamoreboy: > > On Sunday, December 31, 2017 at 6:19:13 PM UTC, Wu Xi wrote: > >> breamoreboy: > >>> An interesting write up on something that is incorporated into Python 3.7 > >>> https

Re: Copy-on-write friendly Python garbage collection (Posting On Python-List Prohibited)

2018-01-01 Thread Niles Rogoff
On Mon, 01 Jan 2018 10:42:58 -0800, breamoreboy wrote: > On Monday, January 1, 2018 at 10:14:59 AM UTC, wxjm...@gmail.com wrote: >> Le lundi 1 janvier 2018 08:35:53 UTC+1, Lawrence D’Oliveiro a écrit : >> > On Monday, January 1, 2018 at 7:52:48 AM UTC+13, Paul Rubin wrote: >> > > I wonder if

Re: Copy-on-write friendly Python garbage collection

2018-01-01 Thread Wu Xi
breamore...@gmail.com: > On Sunday, December 31, 2017 at 6:19:13 PM UTC, Wu Xi wrote: >> breamoreboy: >>> An interesting write up on something that is incorporated into Python 3.7 >>> https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6

Re: Copy-on-write friendly Python garbage collection

2018-01-01 Thread breamoreboy
On Sunday, December 31, 2017 at 6:19:13 PM UTC, Wu Xi wrote: > breamoreboy: > > An interesting write up on something that is incorporated into Python 3.7 > > https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf > > Appearantly, Er

Re: Copy-on-write friendly Python garbage collection

2018-01-01 Thread INADA Naoki
FYI: https://bugs.python.org/issue31558 INADA Naoki <songofaca...@gmail.com> On Mon, Jan 1, 2018 at 12:39 AM, <breamore...@gmail.com> wrote: > An interesting write up on something that is incorporated into Python 3.7 > https://engineering.instagram.com/copy-on-write-frien

Re: Copy-on-write friendly Python garbage collection

2017-12-31 Thread Wu Xi
breamore...@gmail.com: > An interesting write up on something that is incorporated into Python 3.7 > https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf Appearantly, Erlang is the way to go, when it comes to web frameworks. -- https://mail.pyth

Re: Copy-on-write friendly Python garbage collection

2017-12-31 Thread Wu Xi
they said they run the largest deployment of Django world-wide. be it as it may... many still consider the web guys to be the "funny people". Why did they not switch over to Erlang ? -- https://mail.python.org/mailman/listinfo/python-list

Re: Copy-on-write friendly Python garbage collection

2017-12-31 Thread Wu Xi
breamore...@gmail.com: > An interesting write up on something that is incorporated into Python 3.7 > https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf kewl py 3.7 does not fully make install here, but it built and works, as far as I ca

Copy-on-write friendly Python garbage collection

2017-12-31 Thread breamoreboy
An interesting write up on something that is incorporated into Python 3.7 https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf -- Kindest regards. Mark Lawrence. -- https://mail.python.org/mailman/listinfo/python-list

[issue22003] BytesIO copy-on-write

2015-03-04 Thread David Wilson
David Wilson added the comment: Hi Piotr, There wasn't an obvious fix that didn't involve changing the buffer interface itself. There is presently ambiguity in the interface regarding the difference between a read only buffer and an immutable buffer, which is crucial for its use in this

[issue22003] BytesIO copy-on-write

2015-03-04 Thread Piotr Dobrogost
Piotr Dobrogost added the comment: This new patch abandons the buffer interface and specializes for Bytes per the comments on this issue. Why does it abandon buffer interface? Because of the following? Thanks for digging here. As much as I'd love to follow this interpretation, it simply

[issue22003] BytesIO copy-on-write

2015-02-14 Thread Roundup Robot
Roundup Robot added the comment: New changeset 7ae156f07a90 by Berker Peksag in branch 'default': Add a whatsnew entry for issue #22003. https://hg.python.org/cpython/rev/7ae156f07a90 -- ___ Python tracker rep...@bugs.python.org

[issue22003] BytesIO copy-on-write

2015-02-09 Thread David Wilson
David Wilson added the comment: Attached trivial patch for whatsnew.rst. -- Added file: http://bugs.python.org/file38058/whatsnew.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003 ___

[issue22003] BytesIO copy-on-write

2015-02-09 Thread Mikhail Korobov
Mikhail Korobov added the comment: Shouldn't this fix be mentioned in https://docs.python.org/3.5/whatsnew/3.5.html#optimizations ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003

[issue22003] BytesIO copy-on-write

2014-07-29 Thread David Wilson
David Wilson added the comment: I suspect it's all covered now, but is there anything else I can help with to get this patch pushed along its merry way? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003

[issue22003] BytesIO copy-on-write

2014-07-29 Thread Roundup Robot
Roundup Robot added the comment: New changeset 79a5fbe2c78f by Antoine Pitrou in branch 'default': Issue #22003: When initialized from a bytes object, io.BytesIO() now http://hg.python.org/cpython/rev/79a5fbe2c78f -- nosy: +python-dev ___ Python

[issue22003] BytesIO copy-on-write

2014-07-29 Thread Antoine Pitrou
Antoine Pitrou added the comment: The latest patch is good indeed. Thank you very much! -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003

[issue22003] BytesIO copy-on-write

2014-07-28 Thread David Wilson
David Wilson added the comment: Newest patch incorporates Antoine's review comments. The final benchmark results are below. Just curious, what causes e.g. telco to differ up to 7% between runs? That's really huge Report on Linux k2 3.14-1-amd64 #1 SMP Debian 3.14.9-1 (2014-06-30) x86_64 Total

[issue22003] BytesIO copy-on-write

2014-07-28 Thread Stefan Krah
Stefan Krah added the comment: Just curious, what causes e.g. telco to differ up to 7% between runs? That's really huge. telco.py always varies a lot between runs (up to 10%), even in the big version telco.py full: http://bytereef.org/mpdecimal/quickstart.html#telco-benchmark Using the

[issue22003] BytesIO copy-on-write

2014-07-28 Thread Stefan Krah
Stefan Krah added the comment: So I wonder why the benchmark suite says that the telco slowdown is significant. :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003 ___

[issue22003] BytesIO copy-on-write

2014-07-27 Thread David Wilson
David Wilson added the comment: Hey Antoine, Thanks for the link. I'm having trouble getting reproducible results at present, and running out of ideas as to what might be causing it. Even after totally isolating a CPU for e.g. django_v2 and with frequency scaling disabled, numbers still jump

[issue22003] BytesIO copy-on-write

2014-07-27 Thread Antoine Pitrou
Antoine Pitrou added the comment: Even after totally isolating a CPU for e.g. django_v2 and with frequency scaling disabled, numbers still jump around for the same binary by as much as 3%. That's expected. If the difference doesn't go above 5-10%, then you IMO can pretty much consider

[issue22003] BytesIO copy-on-write

2014-07-25 Thread Antoine Pitrou
Antoine Pitrou added the comment: It doesn't seem likely this patch would introduce severe performance troubles elsewhere, but I'd like to trying it out with some example heavy BytesIO consumers (any suggestions? Some popular template engine?) I don't have any specific suggestions, but you

[issue22003] BytesIO copy-on-write

2014-07-25 Thread Antoine Pitrou
Changes by Antoine Pitrou pit...@free.fr: -- stage: needs patch - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003 ___ ___

[issue22003] BytesIO copy-on-write

2014-07-24 Thread David Wilson
David Wilson added the comment: This new patch abandons the buffer interface and specializes for Bytes per the comments on this issue. Anyone care to glance at least at the general structure? Tests could probably use a little more work. Microbenchmark seems fine, at least for construction.

[issue22003] BytesIO copy-on-write

2014-07-22 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: See also issue15381. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003 ___ ___ Python-bugs-list mailing

[issue22003] BytesIO copy-on-write

2014-07-22 Thread Antoine Pitrou
Antoine Pitrou added the comment: There's also the following code in numpy's getbuffer method: /* * If a read-only buffer is requested on a read-write array, we return a * read-write buffer, which is dubious behavior. But that's why this call * is guarded by

[issue22003] BytesIO copy-on-write

2014-07-22 Thread Stefan Krah
Stefan Krah added the comment: Actually we have an extra safety net in memory_hash() apart from the readonly check: We also check if the underlying object is hashable. This might be applicable here, too. Unfortunately mmap objects *are* hashable, leading to some funny results: import mmap

[issue22003] BytesIO copy-on-write

2014-07-22 Thread Stefan Krah
Stefan Krah added the comment: I think the mmap behavior is probably worse than the NumPy example. I assume that in the example the exporter sets view.readonly=0. mmap objects set view.readonly=1 and can still be mutated. -- ___ Python tracker

[issue22003] BytesIO copy-on-write

2014-07-22 Thread David Wilson
David Wilson added the comment: Stefan, I like your new idea. If there isn't some backwards compatibility argument about mmap.mmap being hashable, then it could be considered a bug, and fixed in the same hypothetical future release that includes this BytesIO change. The only cost now is that

[issue22003] BytesIO copy-on-write

2014-07-22 Thread Antoine Pitrou
Antoine Pitrou added the comment: I don't like the idea of trying to hash the object. It may be a time-consuming operation, while the result will be thrown away. I think restricting the optimization to bytes objects is fine. We can whitelist other types, such as memoryview. --

[issue22003] BytesIO copy-on-write

2014-07-21 Thread Antoine Pitrou
Antoine Pitrou added the comment: Pretty sure this approach is broken. What about the alternative approach of specializing for Bytes? That would certainly sound good enough, to optimize the common case. Also, it would be nice if you could add some tests to the patch (e.g. to stress the

[issue22003] BytesIO copy-on-write

2014-07-21 Thread Antoine Pitrou
Antoine Pitrou added the comment: As for whether the checking for a readonly view approach is broken, I don't know: that part of the buffer API is still mysterious to me. Stefan, would you have some insight? -- nosy: +skrah ___ Python tracker

[issue22003] BytesIO copy-on-write

2014-07-21 Thread Stefan Krah
Stefan Krah added the comment: I think checking for a readonly view is fine. The protocol is this: 1) Use the PyBUF_WRITABLE flag in the request. Then the provider must either have a writable buffer or else deny the request entirely. 2) Omit the PyBUF_WRITABLE flag in the request.

[issue22003] BytesIO copy-on-write

2014-07-21 Thread Stefan Krah
Stefan Krah added the comment: The original wording in the PEP is this: readonly an integer variable to hold whether or not the memory is readonly. 1 means the memory is readonly, zero means the memory is writable. To me this means that a hypothetical compiler that could

[issue22003] BytesIO copy-on-write

2014-07-21 Thread David Wilson
David Wilson added the comment: Stefan, Thanks for digging here. As much as I'd love to follow this interpretation, it simply doesn't match existing buffer implementations, including within the standard library. For example, mmap.mmap(..., flags=mmap.MAP_SHARED, prot=mmap.PROT_READ) will

[issue22003] BytesIO copy-on-write

2014-07-21 Thread David Wilson
David Wilson added the comment: I'm not sure how much work it would be, or even if it could be made sufficient to solve our problem, but what about extending the buffers interface to include a int stable flag, defaulting to 0? It seems though, that it would just be making the already arcane

[issue22003] BytesIO copy-on-write

2014-07-21 Thread Stefan Krah
Stefan Krah added the comment: I'm sure many exporters aren't setting the right flags; on the other hand we already hash memoryviews based on readonly buffers, assuming they are immutable. -- ___ Python tracker rep...@bugs.python.org

[issue22003] BytesIO copy-on-write

2014-07-21 Thread David Wilson
Changes by David Wilson d...@botanicus.net: Added file: http://bugs.python.org/file36016/cow4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003 ___

[issue22003] BytesIO copy-on-write

2014-07-21 Thread David Wilson
David Wilson added the comment: Hi Stefan, How does this approach in reinit() look? We first ask for a writable buffer, and if the object obliges, immediately copy it. Otherwise if it refused, ask for a read-only buffer, and this time expect that it will never change. This still does not

[issue22003] BytesIO copy-on-write

2014-07-20 Thread David Wilson
David Wilson added the comment: This version is tidied up enough that I think it could be reviewed. Changes are: * Defer `buf' allocation until __init__, rather than __new__ as was previously done. Now upon completion, BytesIO.__new__ returns a valid, closed BytesIO, whereas previously a

[issue22003] BytesIO copy-on-write

2014-07-20 Thread David Wilson
David Wilson added the comment: New patch also calls unshare() during getbuffer() -- Added file: http://bugs.python.org/file36005/cow3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003

[issue22003] BytesIO copy-on-write

2014-07-20 Thread David Wilson
David Wilson added the comment: I'm not sure the read only buffer test is strong enough: having a readonly view is not a guarantee that the data in the view cannot be changed through some other means, i.e. it is read-only, not immutable. Pretty sure this approach is broken. What about the

[issue22003] BytesIO copy-on-write

2014-07-18 Thread David Wilson
David Wilson added the comment: Good catch :( There doesn't seem to be way a to ask for an immutable buffer, so perhaps it could just be a little more selective. I think the majority of use cases would still be covered if the sharing behaviour was restricted only to BytesType. In that case

[issue22003] BytesIO copy-on-write

2014-07-18 Thread Stefan Behnel
Stefan Behnel added the comment: Even if there is no way to explicitly request a RO buffer, the Py_buffer struct that you get back actually tells you if it's read-only or not. Shouldn't that be enough to enable this optimisation? Whether or not implementors of the buffer protocol set this

[issue22003] BytesIO copy-on-write

2014-07-18 Thread Mikhail Korobov
Changes by Mikhail Korobov kmik...@gmail.com: -- nosy: +kmike ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003 ___ ___ Python-bugs-list mailing

[issue22003] BytesIO copy-on-write

2014-07-17 Thread David Wilson
://bugs.python.org/issue15381 .. it's not clear to me which is better. -- components: Library (Lib) files: cow.patch keywords: patch messages: 223383 nosy: dw priority: normal severity: normal status: open title: BytesIO copy-on-write type: performance versions: Python 3.5 Added file: http

[issue22003] BytesIO copy-on-write

2014-07-17 Thread David Wilson
David Wilson added the comment: Submitted contributor agreement. Please consider the demo patch licensed under the Apache 2 licence. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003

[issue22003] BytesIO copy-on-write

2014-07-17 Thread Antoine Pitrou
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +benjamin.peterson, hynek, pitrou, stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22003 ___

[issue22003] BytesIO copy-on-write

2014-07-17 Thread Antoine Pitrou
Antoine Pitrou added the comment: Be careful what happens when the original object is mutable: b = bytearray(babc) bio = io.BytesIO(b) b[:] = bdefghi bio.getvalue() b'abc' I don't know what your patch does in this case. -- nosy: +serhiy.storchaka stage: - needs patch

Re: copy on write

2012-02-06 Thread Rick Johnson
On Feb 2, 11:28 pm, Chris Angelico ros...@gmail.com wrote: On Fri, Feb 3, 2012 at 4:04 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: No matter what Python did, somebody would complain. +1 This is, I think, the ultimate truth of the matter. People would not complain if

Re: copy on write

2012-02-06 Thread Wolfram Hinderer
On 3 Feb., 11:47, John O'Hagan resea...@johnohagan.com wrote: But isn't it equally true if we say that z = t[1], then t[1] += x is syntactic sugar for z = z.__iadd__(x)? Why should that fail, if z can handle it? It's more like syntactic sugar for y = t; z = y.__getitem__(1); z.__iadd__(x);

Re: copy on write

2012-02-03 Thread Antoon Pardon
On 02/03/2012 06:04 AM, Steven D'Aprano wrote: Ultimately, there is no right answer, because the multitude of requirements are contradictory. No matter what Python did, somebody would complain. Which makes me wonder why it was introduced at all, or at least so fast If you see the difference in

Re: copy on write

2012-02-03 Thread John O'Hagan
On 03 Feb 2012 05:04:39 GMT Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Fri, 03 Feb 2012 14:08:06 +1100, John O'Hagan wrote: I think we're 12 years late on this one. It's PEP 203 from 2000 and the key phrase was: The in-place function should always return a new

Re: copy on write

2012-02-03 Thread OKB (not okblacke)
Steven D'Aprano wrote: Perhaps lists shouldn't define += at all, but then people will complain that mylist += another_list is slow. Telling them to use mylist.extend instead just makes them cranky. After all, mylist + another_list works, so why shouldn't += work? It would work, it

Re: copy on write

2012-02-03 Thread 88888 Dihedral
在 2012年1月14日星期六UTC+8上午6时48分29秒,Evan Driscoll写道: On 01/13/2012 03:20 PM, Neil Cerutti wrote: They perform the same action, but their semantics are different. operator+ will always return a new object, thanks to its signature, and operator+= shall never do so. That's the main difference I

Re: copy on write

2012-02-02 Thread John O'Hagan
On Thu, 2 Feb 2012 01:34:48 -0500 Devin Jeanpierre jeanpierr...@gmail.com wrote: On Wed, Feb 1, 2012 at 10:18 PM, John O'Hagan resea...@johnohagan.com wrote: On Fri, 13 Jan 2012 10:40:47 -0800 Ethan Furman et...@stoneleaf.us wrote: Steven D'Aprano wrote: Normally this is harmless,

Re: copy on write

2012-02-02 Thread Steven D'Aprano
On Thu, 02 Feb 2012 19:11:53 +1100, John O'Hagan wrote: You're right, in fact, for me the surprise is that t[1] += is interpreted as an assignment at all, given that for lists (and other mutable objects which use +=) it is a mutation. Although as Steven says elsewhere, it actually is an

Re: copy on write

2012-02-02 Thread Thomas Rachel
Am 13.01.2012 13:30 schrieb Chris Angelico: It seems there's a distinct difference between a+=b (in-place addition/concatenation) and a=a+b (always rebinding), There is indeed. a = a + b is a = a.__add__(b), while a += b is a = a.__iadd__(b). __add__() is supposed to leave the original

Re: copy on write

2012-02-02 Thread Hrvoje Niksic
Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes: Perhaps you are thinking that Python could determine ahead of time whether x[1] += y involved a list or a tuple, and not perform the finally assignment if x was a tuple. Well, maybe, but such an approach (if possible!) is fraught

Re: copy on write

2012-02-02 Thread 88888 Dihedral
在 2012年1月14日星期六UTC+8上午6时48分29秒,Evan Driscoll写道: On 01/13/2012 03:20 PM, Neil Cerutti wrote: They perform the same action, but their semantics are different. operator+ will always return a new object, thanks to its signature, and operator+= shall never do so. That's the main difference I

Re: copy on write

2012-02-02 Thread John O'Hagan
On 02 Feb 2012 09:16:40 GMT Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Thu, 02 Feb 2012 19:11:53 +1100, John O'Hagan wrote: You're right, in fact, for me the surprise is that t[1] += is interpreted as an assignment at all, given that for lists (and other mutable

Re: copy on write

2012-02-02 Thread MRAB
On 02/02/2012 10:53, Hrvoje Niksic wrote: Steven D'Apranosteve+comp.lang.pyt...@pearwood.info writes: Perhaps you are thinking that Python could determine ahead of time whether x[1] += y involved a list or a tuple, and not perform the finally assignment if x was a tuple. Well, maybe, but

Re: copy on write

2012-02-02 Thread Devin Jeanpierre
On Thu, Feb 2, 2012 at 11:28 AM, MRAB pyt...@mrabarnett.plus.com wrote: Should failed assignment be raising TypeError? Is it really a type error? A failed setitem should be a TypeError as much as a failed getitem should. Should 1[0] be a TypeError? -- Devin --

Re: copy on write

2012-02-02 Thread Terry Reedy
On 2/2/2012 9:17 AM, John O'Hagan wrote: It's not so much about the type of x but that of x[1]. Wouldn't it be possible to omit the assignment simply if the object referred to by x[1] uses += without creating a new object? That way, some_tuple[i] += y will succeed if some_tuple[i] is a list but

Re: copy on write

2012-02-02 Thread Evan Driscoll
On 01/-10/-28163 01:59 PM, 8 Dihedral wrote: If you're working in C++ and overload your operators so that 'a +=' and 'a = + b' have different observable behaviors (besides perhaps time), then either your implementation is buggy or your design is very bad-mannered. Evan Do you mean the

Re: copy on write

2012-02-02 Thread John O'Hagan
On Thu, 02 Feb 2012 12:25:00 -0500 Terry Reedy tjre...@udel.edu wrote: On 2/2/2012 9:17 AM, John O'Hagan wrote: It's not so much about the type of x but that of x[1]. Wouldn't it be possible to omit the assignment simply if the object referred to by x[1] uses += without creating a new

Re: copy on write

2012-02-02 Thread Steven D'Aprano
On Fri, 03 Feb 2012 14:08:06 +1100, John O'Hagan wrote: I think we're 12 years late on this one. It's PEP 203 from 2000 and the key phrase was: The in-place function should always return a new reference, either to the old `x' object if the operation was indeed performed in-place, or to a

Re: copy on write

2012-02-02 Thread Chris Angelico
On Fri, Feb 3, 2012 at 4:04 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: No matter what Python did, somebody would complain. +1 This is, I think, the ultimate truth of the matter. ChrisA -- http://mail.python.org/mailman/listinfo/python-list

Re: copy on write

2012-02-01 Thread John O'Hagan
On Fri, 13 Jan 2012 10:40:47 -0800 Ethan Furman et...@stoneleaf.us wrote: Steven D'Aprano wrote: Normally this is harmless, but there is one interesting little glitch you can get: t = ('a', [23]) t[1] += [42] Traceback (most recent call last): File stdin, line 1, in module

Re: copy on write

2012-02-01 Thread Rick Johnson
On Jan 13, 10:48 am, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Fri, Jan 13, 2012 at 10:13 AM, Grant Edwards inva...@invalid.invalid wrote: On 2012-01-13, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico ros...@gmail.com wrote: There's a

Re: copy on write

2012-02-01 Thread Steven D'Aprano
On Wed, 01 Feb 2012 19:51:13 -0800, Rick Johnson wrote: Yeah there's a word for that; INTUITIVE, And I've been preaching its virtues (sadly in vain it seems!) to these folks for some time now. Intuitive to whom? Expert Python programmers? VB coders? Perl hackers? School children who have

Re: copy on write

2012-02-01 Thread Devin Jeanpierre
On Wed, Feb 1, 2012 at 10:18 PM, John O'Hagan resea...@johnohagan.com wrote: On Fri, 13 Jan 2012 10:40:47 -0800 Ethan Furman et...@stoneleaf.us wrote: Steven D'Aprano wrote: Normally this is harmless, but there is one interesting little glitch you can get: t = ('a', [23]) t[1] +=

Re: copy on write

2012-01-13 Thread Eduardo Suarez-Santana
El 13/01/12 11:33, Eduardo Suarez-Santana escribió: I wonder whether this is normal behaviour. Even simpler: $ python Python 2.7.2 (default, Oct 31 2011, 11:54:55) [GCC 4.5.3] on linux2 Type help, copyright, credits or license for more information. r={'a':1}; d={}; d['x']=r; d['y']=r;

copy on write

2012-01-13 Thread Eduardo Suarez-Santana
I wonder whether this is normal behaviour. I would expect equal sign to copy values from right to left. However, it seems there is a copy-on-write mechanism that is not working. Anyone can explain and provide a working example? Thanks, -Eduardo $ python Python 2.7.2 (default, Oct 31 2011

Re: copy on write

2012-01-13 Thread Jean-Michel Pichavant
Eduardo Suarez-Santana wrote: El 13/01/12 11:33, Eduardo Suarez-Santana escribió: I wonder whether this is normal behaviour. Even simpler: $ python Python 2.7.2 (default, Oct 31 2011, 11:54:55) [GCC 4.5.3] on linux2 Type help, copyright, credits or license for more information. r={'a':1};

Re: copy on write

2012-01-13 Thread Steven D'Aprano
On Fri, 13 Jan 2012 11:33:24 +, Eduardo Suarez-Santana wrote: I wonder whether this is normal behaviour. I would expect equal sign to copy values from right to left. Assignment in Python never copies values. However, it seems there is a copy-on-write mechanism that is not working

Re: copy on write

2012-01-13 Thread Chris Angelico
On Fri, Jan 13, 2012 at 11:10 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: z = [x, y]  # z is a list containing the same sublist twice z[0].append(23) print z [[42, 23], [42, 23]] When you work with floats, ints or strings, you don't notice this because those types are

Re: copy on write

2012-01-13 Thread Steven D'Aprano
On Fri, 13 Jan 2012 23:30:56 +1100, Chris Angelico wrote: It seems there's a distinct difference between a+=b (in-place addition/concatenation) and a=a+b (always rebinding), Actually, both are always rebinding. It just happens that sometimes a+=b rebinds to the same object that it was

Re: copy on write

2012-01-13 Thread Devin Jeanpierre
On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico ros...@gmail.com wrote: It seems there's a distinct difference between a+=b (in-place addition/concatenation) and a=a+b (always rebinding), which is sorely confusing to C programmers. But then, there's a lot about Python that's sorely confusing

Re: copy on write

2012-01-13 Thread Grant Edwards
On 2012-01-13, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico ros...@gmail.com wrote: It seems there's a distinct difference between a+=b (in-place addition/concatenation) and a=a+b (always rebinding), which is sorely confusing to C programmers.

Re: copy on write

2012-01-13 Thread Devin Jeanpierre
On Fri, Jan 13, 2012 at 10:13 AM, Grant Edwards invalid@invalid.invalid wrote: On 2012-01-13, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico ros...@gmail.com wrote: It seems there's a distinct difference between a+=b (in-place

Re: copy on write

2012-01-13 Thread Neil Cerutti
On 2012-01-13, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Fri, Jan 13, 2012 at 10:13 AM, Grant Edwards invalid@invalid.invalid wrote: On 2012-01-13, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico ros...@gmail.com wrote: It seems there's

Re: copy on write

2012-01-13 Thread Grant Edwards
On 2012-01-13, Neil Cerutti ne...@norwich.edu wrote: If you've ever implemented operator=, operator+, and operator+= in C++ you'll know how and why they are different. That assumes that C++ programmers understand C++. ;) A C++ programmer would be wondering how either can work on immutable

Re: copy on write

2012-01-13 Thread Chris Angelico
On Sat, Jan 14, 2012 at 5:15 AM, Grant Edwards invalid@invalid.invalid wrote: That assumes that C++ programmers understand C++. I understand C++ very well. That's why I use Python or Pike. (With apologies to Larry Wall) ChrisA -- http://mail.python.org/mailman/listinfo/python-list

Re: copy on write

2012-01-13 Thread Grant Edwards
On 2012-01-13, Chris Angelico ros...@gmail.com wrote: On Sat, Jan 14, 2012 at 5:15 AM, Grant Edwards invalid@invalid.invalid wrote: That assumes that C++ programmers understand C++. I understand C++ very well. That's why I use Python or Pike. (With apologies to Larry Wall) Were one

Re: copy on write

2012-01-13 Thread Ethan Furman
Steven D'Aprano wrote: Normally this is harmless, but there is one interesting little glitch you can get: t = ('a', [23]) t[1] += [42] Traceback (most recent call last): File stdin, line 1, in module TypeError: 'tuple' object does not support item assignment t ('a', [23, 42]) There is

Re: copy on write

2012-01-13 Thread Neil Cerutti
On 2012-01-13, Grant Edwards invalid@invalid.invalid wrote: On 2012-01-13, Chris Angelico ros...@gmail.com wrote: On Sat, Jan 14, 2012 at 5:15 AM, Grant Edwards invalid@invalid.invalid wrote: That assumes that C++ programmers understand C++. I understand C++ very well. That's why I use

Re: copy on write

2012-01-13 Thread Evan Driscoll
On 01/13/2012 10:54 AM, Neil Cerutti wrote: If you've ever implemented operator=, operator+, and operator+= in C++ you'll know how and why they are different. At the same time, you'd also know that that implementing them in such a way that 'a += b' does *not* perform the same action as 'a = a

Re: copy on write

2012-01-13 Thread Neil Cerutti
On 2012-01-13, Evan Driscoll edrisc...@wisc.edu wrote: On 01/13/2012 10:54 AM, Neil Cerutti wrote: If you've ever implemented operator=, operator+, and operator+= in C++ you'll know how and why they are different. At the same time, you'd also know that that implementing them in such a way

Re: copy on write

2012-01-13 Thread 88888 Dihedral
Ethan Furman於 2012年1月14日星期六UTC+8上午2時40分47秒寫道: Steven D'Aprano wrote: Normally this is harmless, but there is one interesting little glitch you can get: t = ('a', [23]) t[1] += [42] Traceback (most recent call last): File stdin, line 1, in module TypeError: 'tuple' object does

Re: copy on write

2012-01-13 Thread Evan Driscoll
On 01/13/2012 03:20 PM, Neil Cerutti wrote: They perform the same action, but their semantics are different. operator+ will always return a new object, thanks to its signature, and operator+= shall never do so. That's the main difference I was getting at. I was talking about the combination of

Re: Copy-on-write when forking a python process

2011-04-12 Thread jac
the child processes do want to mutate the dictionary, but do not want the changes to show up in the parent process' dictionary. If shared memory is used, then a more complicated algorithm would be needed to get the desired behavior. Long story short, shared-memory is great for some things. Copy on write

Copy-on-write when forking a python process

2011-04-08 Thread John Connor
Hi all, Long time reader, first time poster. I am wondering if anything can be done about the COW (copy-on-write) problem when forking a python process. I have found several discussions of this problem, but I have seen no proposed solutions or workarounds. My understanding of the problem

Re: Copy-on-write when forking a python process

2011-04-08 Thread Heiko Wundram
Am 08.04.2011 18:14, schrieb John Connor: Has anyone else looked into the COW problem? Are there workarounds and/or other plans to fix it? Does the solution I am proposing sound reasonable, or does it seem like overkill? Does anyone foresee any problems with it? Why'd you need a fix like

Re: Copy-on-write when forking a python process

2011-04-08 Thread jac
clone, it is an optimization that works at the memory page level, not at the memory image level. In other words, if I write to a copy-on-write page, only that page is copied into my process' address space, not the entire parent image. To the best of my knowledge by preventing the child process

Re: Copy-on-write when forking a python process

2011-04-08 Thread Heiko Wundram
Am 08.04.2011 20:34, schrieb jac: I disagree with your statement that COW is an optimization for a complete clone, it is an optimization that works at the memory page level, not at the memory image level. In other words, if I write to a copy-on-write page, only that page is copied into my