Re: [Python-ideas] Membership of infinite iterators

On Tue, Oct 17, 2017 at 9:44 PM, Brendan Barnwell wrote: > On 2017-10-17 07:26, Serhiy Storchaka wrote: > >> 17.10.17 17:06, Nick Coghlan пише: >> >>> >Keep in mind we're not talking about a regular loop you can break out of >>> >with Ctrl-C here - we're talking about a tight loop inside the >>>

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 03:39, Koos Zevenhoven wrote: > On Tue, Oct 17, 2017 at 5:26 PM, Serhiy Storchaka > wrote: > >> 17.10.17 17:06, Nick Coghlan пише: >> >>> Keep in mind we're not talking about a regular loop you can break out of >>> with Ctrl-C here - we're talking about a tight loop inside

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 10:56, Koos Zevenhoven wrote: > I'm unable to reproduce the "uninterruptible with Ctrl-C" problem with > infinite iterators. At least itertools doesn't seem to have it: > import itertools for i in itertools.count(): > ... pass > ... > ^CTraceback (most recent ca

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 19:56, Koos Zevenhoven wrote: > I'm unable to reproduce the "uninterruptible with Ctrl-C"​ problem with > infinite iterators. At least itertools doesn't seem to have it: > > >>> import itertools > >>> for i in itertools.count(): > ... pass > ... > That's interrupting th

Re: [Python-ideas] Membership of infinite iterators

On Oct 18, 2017 13:29, "Nick Coghlan" wrote: On 18 October 2017 at 19:56, Koos Zevenhoven wrote: > I'm unable to reproduce the "uninterruptible with Ctrl-C"​ problem with > infinite iterators. At least itertools doesn't seem to have it: > > >>> import itertools > >>> for i in itertools.count():

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 20:39, Koos Zevenhoven wrote: > On Oct 18, 2017 13:29, "Nick Coghlan" wrote: > > On 18 October 2017 at 19:56, Koos Zevenhoven wrote: > >> I'm unable to reproduce the "uninterruptible with Ctrl-C"​ problem with >> infinite iterators. At least itertools doesn't seem to have

[Python-ideas] Memory limits [was Re: Membership of infinite iterators]

On Wed, Oct 18, 2017 at 01:39:28PM +0300, Koos Zevenhoven wrote: > I'm writing from my phone now, cause I was dumb enough to try list(count()) You have my sympathies -- I once, due to typo, accidentally ran something like range(10**100) in Python 2. > But should it be fixed in list or in coun

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 2:08 PM, Nick Coghlan wrote: > On 18 October 2017 at 20:39, Koos Zevenhoven wrote: > >> On Oct 18, 2017 13:29, "Nick Coghlan" wrote: >> >> On 18 October 2017 at 19:56, Koos Zevenhoven wrote: >> >>> I'm unable to reproduce the "uninterruptible with Ctrl-C"​ problem with

Re: [Python-ideas] Memory limits [was Re: Membership of infinite iterators]

On 18 October 2017 at 21:38, Steven D'Aprano wrote: > > But should it be fixed in list or in count? > > Neither. There are too many other places this can break for it to be > effective to try to fix each one in place. > > e.g. set(xrange(2**64)), or tuple(itertools.repeat([1])) > A great many o

Re: [Python-ideas] Memory limits [was Re: Membership of infinite iterators]

On Wed, Oct 18, 2017 at 10:43:57PM +1000, Nick Coghlan wrote: > Per-process memory quotas *can* help avoid this, but enforcing them > requires that every process run in a resource controlled sandbox. Hence, > it's not a coincidence that mobile operating systems and container-based > server environm

Re: [Python-ideas] Membership of infinite iterators

18.10.17 13:22, Nick Coghlan пише: 2.. These particular cases can be addressed locally using existing protocols, so the chances of negative side effects are low Only the particular case `count() in count()` can be addressed without breaking the following examples: >>> class C: ... def __

Re: [Python-ideas] Memory limits [was Re: Membership of infinite iterators]

On Wed, Oct 18, 2017 at 2:38 PM, Steven D'Aprano wrote: > On Wed, Oct 18, 2017 at 01:39:28PM +0300, Koos Zevenhoven wrote: > > > I'm writing from my phone now, cause I was dumb enough to try > list(count()) > > You have my sympathies -- I once, due to typo, accidentally ran > something like range

Re: [Python-ideas] Memory limits [was Re: Membership of infinite iterators]

On 18 October 2017 at 22:51, Stefan Krah wrote: > On Wed, Oct 18, 2017 at 10:43:57PM +1000, Nick Coghlan wrote: > > Per-process memory quotas *can* help avoid this, but enforcing them > > requires that every process run in a resource controlled sandbox. Hence, > > it's not a coincidence that mobi

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 22:53, Serhiy Storchaka wrote: > 18.10.17 13:22, Nick Coghlan пише: > >> 2.. These particular cases can be addressed locally using existing >> protocols, so the chances of negative side effects are low >> > > Only the particular case `count() in count()` can be addressed wit

Re: [Python-ideas] Membership of infinite iterators

Hi all, FWIW, I just tried the list(count()) experiment on my phone (Termux Python interpreter under Android). Python 3.6.2 (default, Sep 16 2017, 23:55:07) [GCC 4.2.1 Compatible Android Clang 5.0.300080 ] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 22:36, Koos Zevenhoven wrote: > On Wed, Oct 18, 2017 at 2:08 PM, Nick Coghlan wrote: > >> That one can only be fixed in count() - list already checks >> operator.length_hint(), so implementing itertools.count.__length_hint__() >> to always raise an exception would be enough

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 5:48 PM, Nick Coghlan wrote: > On 18 October 2017 at 22:36, Koos Zevenhoven wrote: > >> On Wed, Oct 18, 2017 at 2:08 PM, Nick Coghlan wrote: >> >>> That one can only be fixed in count() - list already checks >>> operator.length_hint(), so implementing itertools.count.__l

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 16:27, Koos Zevenhoven wrote: > So you're talking about code that would make a C-implemented Python iterable > of strictly C-implemented Python objects and then pass this to something > C-implemented like list(..) or sum(..), while expecting no Python code to be > run or sign

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 6:36 PM, Paul Moore wrote: > On 18 October 2017 at 16:27, Koos Zevenhoven wrote: > > So you're talking about code that would make a C-implemented Python > iterable > > of strictly C-implemented Python objects and then pass this to something > > C-implemented like list(..)

Re: [Python-ideas] Membership of infinite iterators

OK, looks like I've lost track of what this thread is about then. Sorry for the noise. Paul On 18 October 2017 at 16:40, Koos Zevenhoven wrote: > On Wed, Oct 18, 2017 at 6:36 PM, Paul Moore wrote: >> >> On 18 October 2017 at 16:27, Koos Zevenhoven wrote: >> > So you're talking about code that w

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 6:56 PM, Paul Moore wrote: > OK, looks like I've lost track of what this thread is about then. > Sorry for the noise. > Paul > > ​No worries. I'm not sure I can tell what this thread is about either. Different people seem to have different ideas about that. My most recent

Re: [Python-ideas] Membership of infinite iterators

On 2017-10-18 15:48, Nick Coghlan wrote: On 18 October 2017 at 22:36, Koos Zevenhoven > wrote: On Wed, Oct 18, 2017 at 2:08 PM, Nick Coghlan mailto:ncogh...@gmail.com>> wrote: That one can only be fixed in count() - list already checks operator.leng

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 9:24 PM, MRAB wrote: > > The re module increments a counter on each iteration and checks for > signals when the bottom 12 bits are 0. > > The regex module increments a 16-bit counter on each iteration and checks > for signals when it wraps around to 0. > Then I​'d say tha

Re: [Python-ideas] Membership of infinite iterators

18.10.17 17:48, Nick Coghlan пише: 1. It will make those loops slower, due to the extra overhead of checking for signals (even the opcode eval loop includes all sorts of tricks to avoid actually checking for new signals, since doing so is relatively slow) 2. It will make those loops harder to m

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 10:13 PM, Serhiy Storchaka wrote: > 18.10.17 17:48, Nick Coghlan пише: > >> 1. It will make those loops slower, due to the extra overhead of checking >> for signals (even the opcode eval loop includes all sorts of tricks to >> avoid actually checking for new signals, since

Re: [Python-ideas] Membership of infinite iterators

18.10.17 22:21, Koos Zevenhoven пише: ​Nice! Though I'd really like a general ​solution that other code can easily adopt, even third-party extension libraries. What is the more general solution? For interrupting C code you need to check signals manually, either in every loop, or in every itera

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 10:21 PM, Koos Zevenhoven wrote: > On Wed, Oct 18, 2017 at 10:13 PM, Serhiy Storchaka > wrote: > >> 18.10.17 17:48, Nick Coghlan пише: >> >>> 1. It will make those loops slower, due to the extra overhead of >>> checking for signals (even the opcode eval loop includes all

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 10:30 PM, Serhiy Storchaka wrote: > 18.10.17 22:21, Koos Zevenhoven пише: > >> ​Nice! Though I'd really like a general ​solution that other code can >> easily adopt, even third-party extension libraries. >> > > What is the more general solution? For interrupting C code you

Re: [Python-ideas] PEP draft: context variables

On Sun, Oct 15, 2017 at 8:15 AM, Paul Moore wrote: > On 13 October 2017 at 23:30, Yury Selivanov wrote: >> At this point of time, there's just one place which describes one well >> defined semantics: PEP 550 latest version. Paul, if you have >> time/interest, please take a look at it, and say wh

Re: [Python-ideas] Membership of infinite iterators

Nick Coghlan wrote: since breaking up the current single level loops as nested loops would be a pre-requisite for allowing these APIs to check for signals while they're running while keeping the per-iteration overhead low Is there really much overhead? Isn't it just checking a flag? -- Greg _

Re: [Python-ideas] Memory limits [was Re: Membership of infinite iterators]

On Wed, Oct 18, 2017 at 02:51:37PM +0200, Stefan Krah wrote: > $ softlimit -m 10 python3 [...] > MemoryError > > > People who are worried could make a python3 alias or use Ctrl-\. I just tried that on two different Linux computers I have, and neither have softlimit. Nor (presumably)

Re: [Python-ideas] Membership of infinite iterators

On 19 October 2017 at 08:34, Greg Ewing wrote: > Nick Coghlan wrote: > >> since breaking up the current single level loops as nested loops would be >> a pre-requisite for allowing these APIs to check for signals while they're >> running while keeping the per-iteration overhead low >> > > Is there

Re: [Python-ideas] Membership of infinite iterators

Nick Coghlan wrote: having checks in both the producer & the consumer merely means that you'll be checking for signals twice every 65k iterations, rather than once. Here's a possible reason for wanting checks in the producers: If your producer happens to take a long time per iteration, and the