Re: [Python-Dev] tracemalloc: add an optional memory limit
09.12.13 03:56, Victor Stinner написав(ла): On Linux, it's possible to limit globally the "address space" of a process, but this value is not convinient. For example, the address space includes shared memory and read-only memory mappings of files, whereas this memory should not reduce the memory available for other applications. tracemalloc only counts memory directly allocated by Python and so private memory which reduces directly the memory available for other applications. But tracemalloc doesn't count memory allocated besides Python allocators (e.g. memory for executable, static variables and stack, memory allocated by extensions and C lib, unallocated but not returned to OS dynamical memory). When you want investigate how your program works on low memory, e.g. 500 MB, ulimit -v 50 is much closer to real life than tracemalloc. Limiting memory in tracemalloc can be used only for testing obscure corner cases in Python interpreter itself. It will be very rarely used since all bugs will be fixed first time (and thank you for your current work). So there is more sense implemented it as auxiliary utility than as feature of the stdlib module. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] tracemalloc: add an optional memory limit
Hi,
2013/12/9 Serhiy Storchaka :
> But tracemalloc doesn't count memory allocated besides Python allocators
> (e.g. memory for executable, static variables and stack, memory allocated by
> extensions and C lib, unallocated but not returned to OS dynamical memory).
> When you want investigate how your program works on low memory, e.g. 500 MB,
> ulimit -v 50 is much closer to real life than tracemalloc.
Well, both limits are useful, but also (completly?) different.
"memory for executable, static variables and stack," : these values
are fixed, so they don't matter at all. Substract them from the RSS
when you choose the memory limit for tracemalloc.
"memory allocated by extensions" : I started to modify some Python
extensions (ex: zlib, bzip2, lzma) to reuse Python memory allocation,
so the memory is also traced by tracemalloc. Memory allocated by other
extensions is not traced.
"and C lib," : correct, this memory is not traced by tracemalloc and
so not counted in the memory limit.
"unallocated but not returned to OS dynamical memory" : it's really
hard to estimate and understand the fragmentation of the heap memory
:-( But this may only concern short peak, and the free memory can be
reused later. If the peak is still lower than the limit, the
fragmentation does not matter
> Limiting memory in tracemalloc can be used only for testing obscure corner
> cases in Python interpreter itself.
In my previous job in embedded device, we hard an *hard* limit for
memory. Python had its own dedicated memory: an hardcoded 30 MB chunk
of memory, reserved for Python. Only memory directly allocated by
Python was stored in this 30 MB chunk. Other allocations ("memory
allocated by extensions, C lib" in your list) was allocated in the
system memory. And I was only concerned of memory directly allocated
by Python.
So being able to run the application on a PC and set a "Python" memory
limit of 30 MB makes sense in such use case.
> It will be very rarely used since all
> bugs will be fixed first time (and thank you for your current work). So
> there is more sense implemented it as auxiliary utility than as feature of
> the stdlib module.
The memory limit is used to test if the application doesn't leak
memory nor allocate more than the limit. Said differently, it's a
convinient tool to simulate an embeded device :-)
Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 답장: Add Gentoo packagers of external modules to Misc/ACKS
Please add access permissions on the Python bug tracker for the seotaewong40 account. For what it's worth, you need to add Marien Zwart to Misc/ACKS (apparently he has contributed to the GNU Emacs mode for Python). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Sort error in Misc/ACKS
There is a sorting error: The name Jonas Wagner comes before Zachary Ware and Barry Warsaw. Please correct sorting error so that his name appears between Sjoerd de Vries and Niki W. Waibel. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] tracemalloc: add an optional memory limit
On Mon, 9 Dec 2013 10:28:17 +0100 Victor Stinner wrote: > Hi, > > 2013/12/9 Serhiy Storchaka : > > But tracemalloc doesn't count memory allocated besides Python allocators > > (e.g. memory for executable, static variables and stack, memory allocated by > > extensions and C lib, unallocated but not returned to OS dynamical memory). > > When you want investigate how your program works on low memory, e.g. 500 MB, > > ulimit -v 50 is much closer to real life than tracemalloc. > > Well, both limits are useful, but also (completly?) different. > > "memory for executable, static variables and stack," : these values > are fixed, so they don't matter at all. Substract them from the RSS > when you choose the memory limit for tracemalloc. Stack consumption is O(number of threads). > "unallocated but not returned to OS dynamical memory" : it's really > hard to estimate and understand the fragmentation of the heap memory > :-( But this may only concern short peak, and the free memory can be > reused later. If the peak is still lower than the limit, the > fragmentation does not matter I don't understand the statement: "if the peak is still lower than the limit, the fragmentation does not matter". Serhiy's point is precisely that the peak's computation doesn't account for OS-level fragmentation. That said, I agree the feature can be useful. Unlike ulimit, it's a best-effort thing, but it also works at a much more useful level than "size of the virtual address space". Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sort error in Misc/ACKS
On Dec 09, 2013, at 04:00 PM, TaeWong wrote: >The name Jonas Wagner comes before Zachary Ware and Barry Warsaw. Maybe it's time to give up on trying to sort by last name, and just sort by first character? No, this is not a bold attempt to jump higher up in the list. -AABarry ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sort error in Misc/ACKS
On Mon, Dec 9, 2013 at 8:57 AM, Barry Warsaw wrote: > On Dec 09, 2013, at 04:00 PM, TaeWong wrote: > >>The name Jonas Wagner comes before Zachary Ware and Barry Warsaw. > > Maybe it's time to give up on trying to sort by last name, and just sort by > first character? > > No, this is not a bold attempt to jump higher up in the list. > -AABarry Hey, I'm far enough down the list already... More seriously, we do specify "rough alphabetical order[ing]"; there's no need to point out this kind of thing. (Although it might be worth it to change that to "no particular order" so that adding someone is as simple as `echo "Some Human" >> Misc/ACKS`. We could always run random.shuffle() on the current list so new additions don't look out of place ;) -- Zach ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add Gentoo packagers of external modules to Misc/ACKS
Looks like a human to me instead, or some kind of smart AI. look he just responded again. On Sun, Dec 8, 2013 at 5:29 AM, R. David Murray wrote: > As far as we have been able to determine, Tae Wong is in fact a bot > (note the 'seo' in the email address...a tip of the hand, as far as > I can see). We have removed all access permissions (including email) > from the related account on the bug tracker already. IMO this address > should be blocked from posting to all python lists. > > --David > > On Sun, 08 Dec 2013 11:07:10 +0100, Dirkjan Ochtman > wrote: > > Tae Wong, > > > > Please don't speak on behalf the Gentoo Python team. > > > > Everyone else, sorry for this, they are definitely not actually > > connected to our Gentoo team. > > > > Cheers, > > > > Dirkjan > > ___ > > Python-Dev mailing list > > [email protected] > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/rdmurray%40bitdance.com > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/gokoproject%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add Gentoo packagers of external modules to Misc/ACKS
Please look up his name in the archives. We've gone through this before. He sends posts like this to many lists and there is never any value to it (but it's hard to prove since usually 1-2 of the people in his list really did contribute or are at least known to the target audience). His motivations are unclear unless it is just to attract links to his email profile. Please ban and ignore. On Mon, Dec 9, 2013 at 7:22 AM, John Wong wrote: > Looks like a human to me instead, or some kind of smart AI. look he just > responded again. > > > On Sun, Dec 8, 2013 at 5:29 AM, R. David Murray > wrote: >> >> As far as we have been able to determine, Tae Wong is in fact a bot >> (note the 'seo' in the email address...a tip of the hand, as far as >> I can see). We have removed all access permissions (including email) >> from the related account on the bug tracker already. IMO this address >> should be blocked from posting to all python lists. >> >> --David >> >> On Sun, 08 Dec 2013 11:07:10 +0100, Dirkjan Ochtman >> wrote: >> > Tae Wong, >> > >> > Please don't speak on behalf the Gentoo Python team. >> > >> > Everyone else, sorry for this, they are definitely not actually >> > connected to our Gentoo team. >> > >> > Cheers, >> > >> > Dirkjan >> > ___ >> > Python-Dev mailing list >> > [email protected] >> > https://mail.python.org/mailman/listinfo/python-dev >> > Unsubscribe: >> > https://mail.python.org/mailman/options/python-dev/rdmurray%40bitdance.com >> ___ >> Python-Dev mailing list >> [email protected] >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/gokoproject%40gmail.com > > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add Gentoo packagers of external modules to Misc/ACKS
On Mon, 09 Dec 2013 10:22:18 -0500, John Wong wrote: > Looks like a human to me instead, or some kind of smart AI. look he just > responded again. Could be. He's never responded to a direct question on the bug tracker, though, just posts a new comment with some loose connection to the first comment, as happened here. So human or not, his "contributions" on the bug tracker did not have any utility. He's also never talked about anything other than the ACKS file, and given the absence of response to questions on the existing issue, we viewed the opening of new issues as spam, and therefore disabled the account. Someone from another project reported similar problems on that project, and a google search did not turn up any *meaningful* contributions anywhere that I could find. If he'll engage us in a real conversation, this could of course be changed :) --David ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add Gentoo packagers of external modules to Misc/ACKS
On Sun, 2013-12-08 at 05:29 -0500, R. David Murray wrote: > As far as we have been able to determine, Tae Wong is in fact a bot > (note the 'seo' in the email address...a tip of the hand, as far as > I can see). We have removed all access permissions (including email) > from the related account on the bug tracker already. IMO this address > should be blocked from posting to all python lists. FWIW the address has also been posting to the gcc lists "helpfully" asking for spam posts to be removed (with *links* to the posts), plus some (apparently) random-harvested paragraphs of text from various other mailing lists, presumably to try to get past filters. See e.g. the URL obtained by running: echo uggc://tpp.tah.bet/zy/tpp/2013-12/zft00097.ugzy | rot13 Hope this is constructive Dave ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sort error in Misc/ACKS
> We could always run random.shuffle() on the current list > so new additions don't look out of place ;) Wouldn't that bloat the repository with diffs and make merges more difficult? Skip ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sort error in Misc/ACKS
Am i the one who feels we do not add any value by doing this? Best Regards, Srinivas Reddy Thatiparthy 9703888668. "Anyone who has never made a mistake has never tried anything new !!! " --Albert Einstein On Mon, Dec 9, 2013 at 11:43 PM, Skip Montanaro wrote: > > We could always run random.shuffle() on the current list > > so new additions don't look out of place ;) > > Wouldn't that bloat the repository with diffs and make merges more > difficult? > > > > Skip > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/thatiparthysreenivas%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sort error in Misc/ACKS
The troll is successful already. Read the other thread. :-( On Mon, Dec 9, 2013 at 10:37 AM, Sreenivas Reddy T wrote: > Am i the one who feels we do not add any value by doing this? > > Best Regards, > Srinivas Reddy Thatiparthy > 9703888668. > > "Anyone who has never made a mistake has never tried anything new !!! " > --Albert Einstein > > > On Mon, Dec 9, 2013 at 11:43 PM, Skip Montanaro wrote: >> >> > We could always run random.shuffle() on the current list >> > so new additions don't look out of place ;) >> >> Wouldn't that bloat the repository with diffs and make merges more >> difficult? >> >> >> >> Skip >> ___ >> Python-Dev mailing list >> [email protected] >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/thatiparthysreenivas%40gmail.com > > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] One-line abstractmethod function?
I just wanted to sum up this thread of discussion.
Proposal:
A function in abc to provide a default abstract method
implementation.
foo = make_abstractmethod('foo', ['self'])
is the same as
@abc.abstractmethod
def foo(self):
pass
Details:
Default behavior, if implemented, should probably be empty/pass/return
None. How to handle docstrings? Either attribute docstring (a separate
discussion) or a parameter in the function call.
Pros:
Save a lot of lines defining interface-like ABCs, especially in small
scripts without docstrings (bad practice, but I do it often =))
Cons:
Do we need it?
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] One-line abstractmethod function?
On 12/09/2013 11:52 AM, Allen Li wrote: Pros: Save a lot of lines defining interface-like ABCs, especially in small scripts without docstrings (bad practice, but I do it often =)) If the only pro is encouraging bad practice, it's not going to happen. ;) Cons: Do we need it? In my opinion, no. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
