[Python-ideas] Re: Support for atomic types in Python

On Mon, Sep 9, 2019, 09:27 Vinay Sharma via Python-ideas < python-ideas@python.org> wrote: > Also, I didn't mean simple variables to be atomic. I just mean that > some objects which are frequently being used across processes in > multiprocessing can have some atomic operations/methods. > So, as

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

On Wed, Jul 18, 2018 at 2:38 PM MRAB wrote: > What if an object is not going to be shared, but instead "moved" from > one subinterpreter to another? The first subinterpreter would no longer > have a reference to the object. > > If the object's refcount is 1 and the object doesn't refer to any

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

On Wed, Jul 18, 2018 at 12:49 PM Stephan Houben wrote: > Antoine said that what I proposed earlier was very similar to what Eric > is trying to do, but from the direction the discussion has taken so far > that appears not to be the case. It looks like we are after the same thing actually. :)

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

On Wed, Jul 18, 2018 at 1:37 AM Barry Scott wrote: > Let me try a longer answer. The inc+test and dec+test do not require a > lock if coded correctly. All OS and run times have solved this to provide > locks. All processors provide the instructions that are the building blocks > for lock

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

On Wed, Jul 18, 2018 at 3:31 AM Antoine Pitrou wrote: > Please read in context: we are not talking about making all refcounts > atomic, only a couple refcounts on shared objects (which probably > won't be Python objects, actually). I have no plans to use refcounts for shared data (outside of

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

On Tue, Jul 17, 2018 at 1:44 PM Barry wrote: > The decrement itself is not the problem, that can be made thread safe. Yeah, by using the GIL. Otherwise, please elaborate. My understanding is that if the decrement itself were not the problem then we'd have gotten rid of the GIL already. > Do

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

On Mon, Jul 16, 2018 at 11:08 AM Antoine Pitrou wrote: > On Mon, 16 Jul 2018 18:00:37 +0100 > MRAB wrote: > > Could you explicitly share an object in a similar way to how you > > explicitly open a file? > > > > The shared object's refcount would be incremented and the sharing > > function would

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

On Sat, Jul 14, 2018 at 3:41 AM Antoine Pitrou wrote: > Davin has been mostly inactive. I'm the de facto maintainer for > multiprocessing. Ah, that's great to know. Sorry about the confusion. -eric ___ Python-ideas mailing list

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

On Tue, Jul 10, 2018 at 8:32 AM David Foster wrote: > > I was not aware of PyParallel. The PyParellel "parallel thread" > line-of-execution implementation is pretty interesting. Trent, big kudos > to you on that effort. +1 It's a neat project. Trent's pretty smart. :) > Since you're speaking

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

On Sun, Jul 8, 2018 at 12:30 PM David Foster wrote: > In the past I have personally viewed Python as difficult to use for > parallel applications, which need to do multiple things simultaneously > for increased performance: > > * The old Threads, Locks, & Shared State model is inefficient in

Re: [Python-ideas] Let try-except check the exception instance

On Wed, May 30, 2018 at 10:47 PM, Danilo J. S. Bellini wrote: try: > ... # [...] > ... session.commit() # Here it raises! > ... # [...] > ... except DatabaseError as exc: > ... msg = get_db_error_msg_from_exception(exc) > ... if msg == "beyond_limit": > ... #

Re: [Python-ideas] __dir__ in which folder is this py file

On Mon, May 7, 2018 at 7:14 AM, Serhiy Storchaka wrote: > * Additional burden on maintainers of import machinery. It is already too > complex, and __file__ is set in multiple places. Don't forgot about > third-party implementations. > > See also issue33277: "Deprecate

Re: [Python-ideas] Allow multiple imports from a package while preserving its namespace

On Fri, Apr 27, 2018 at 5:14 AM, Nick Coghlan wrote: > Taking this idea in a completely different direction: what if we were > to take advantage of PEP 451 __spec__ attributes to enhance modules to > natively support implicit on-demand imports before they give up and > raise

Re: [Python-ideas] Allow multiple imports from a package while preserving its namespace

On Thu, Apr 26, 2018 at 7:51 AM, Nick Coghlan wrote: > On 26 April 2018 at 23:37, Paul Moore wrote: >> What are the benefits of this over a simple "import "? > > Forcing submodule imports would be the main thing, as at the moment, > you have to choose

Re: [Python-ideas] sys.py

On Thu, Sep 14, 2017 at 4:26 PM, Victor Stinner wrote: > How do you crash Python? See https://bugs.python.org/issue31404. > Can't we fix the interpreter? I'm looking into it. In the meantime I've split the original branch up into 3. The first I've already landed.

Re: [Python-ideas] sys.py

On Tue, Sep 12, 2017 at 9:30 PM, Guido van Rossum wrote: > I find this a disturbing trend. Which trend? Moving away from "consenting adults"? In the case of sys.modules, the problem is that assigning a bogus value (e.g. []) can cause the interpreter to crash. It wasn't a

Re: [Python-ideas] LOAD_NAME/LOAD_GLOBAL should be use getattr()

On Sep 12, 2017 10:17 AM, "Neil Schemenauer" wrote: Introducing another special feature of modules to make this work is not the solution, IMHO. We should make module namespaces be more like instance namespaces. We already have a mechanism and it is getattr on

Re: [Python-ideas] lazy import via __future__ or compiler analysis

On Sep 11, 2017 2:32 PM, "Neil Schemenauer" wrote: On 2017-09-11, Neil Schemenauer wrote: > A module can be a singleton instance of a singleton ModuleType > instance. Maybe more accurate to say each module would have its own unique __class__ associated with it.

[Python-ideas] sys.py

The sys module is a rather special case as far as modules go. It is effectively a "console" into the interpreter's internal state and that includes some mutable state. Since it is a module, we don't have much of an opportunity to: * validate values assigned to its attributes [1] * issue

Re: [Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

On Sun, Sep 10, 2017 at 12:14 PM, Antoine Pitrou wrote: > What could improve performance significantly would be to share objects > without any form of marshalling; but it's not obvious it's possible in > the subinterpreters model *if* it also tries to remove the GIL. Yep.

Re: [Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

On Sun, Sep 10, 2017 at 7:52 AM, Koos Zevenhoven wrote: > I assume the concept of a main interpreter is inherited from the previous > levels of support in the C API, but what exactly is the significance of > being "the main interpreter"? Instead, could they just all be >

Re: [Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

to discuss this. -eric On Sun, Sep 10, 2017 at 3:18 AM, Ronald Oussoren <ronaldousso...@mac.com> wrote: > >> On 8 Sep 2017, at 05:11, Eric Snow <ericsnowcurren...@gmail.com> wrote: > >> On Thu, Sep 7, 2017 at 3:48 PM, Nathaniel Smith <n...@pobox.com> wrote: >>

Re: [Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

On Thu, Sep 7, 2017 at 11:19 PM, Nathaniel Smith <n...@pobox.com> wrote: > On Thu, Sep 7, 2017 at 8:11 PM, Eric Snow <ericsnowcurren...@gmail.com> wrote: >> My concern is that this is a chicken-and-egg problem. The situation >> won't improve until subinterpreters

Re: [Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

On Thu, Sep 7, 2017 at 12:44 PM, Paul Moore <p.f.mo...@gmail.com> wrote: > On 7 September 2017 at 20:14, Eric Snow <ericsnowcurren...@gmail.com> wrote: >> I didn't include such a queue in this proposal because I wanted to >> keep it as focused as possible. I'll

Re: [Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

First of all, thanks for the feedback and encouragement! Responses in-line below. -eric On Thu, Sep 7, 2017 at 3:48 PM, Nathaniel Smith wrote: > My concern about this is the same as it was last time -- the work > looks neat, but right now, almost no-one uses subinterpreters >

Re: [Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

On Thu, Sep 7, 2017 at 1:14 PM, Sebastian Krause wrote: > How is the GIL situation with subinterpreters these days, is the > long-term goal still "solving multi-core Python", i.e. using > multiple CPU cores from within the same process? Or is it mainly > used for

Re: [Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

On Thu, Sep 7, 2017 at 12:44 PM, Paul Moore wrote: > Ah, OK. so if I create a new interpreter, none of the classes, > functions, or objects defined in my calling code will exist within the > target interpreter? That makes sense, but I'd missed that nuance from > the

Re: [Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

On Thu, Sep 7, 2017 at 11:52 AM, Paul Moore wrote: > The only quibble I have is that I'd prefer it if we had a > run(callable, *args, **kwargs) method. Either instead of, or as well > as, the run(string) one here. > > Is there any reason why passing a callable and args is

[Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

554 Title: Multiple Interpreters in the Stdlib Author: Eric Snow <ericsnowcurren...@gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2017-09-05 Python-Version: 3.7 Post-History: Abstract This proposal introduces the stdlib "interpreters" mod

Re: [Python-ideas] New PEP 550: Execution Context

On Aug 11, 2017 16:38, "Yury Selivanov" wrote: Hi, This is a new PEP to implement Execution Contexts in Python. Nice! I've had something like this on the back burner for a while as it helps solve some problems with encapsulating the import state (e.g. PEP 408).

Re: [Python-ideas] a new namedtuple

On Mon, Jul 17, 2017 at 6:01 PM, Ethan Furman wrote: > Guido has decreed that namedtuple shall be reimplemented with speed in mind. FWIW, I'm sure that any changes to namedtuple will be kept as minimal as possible. Changes would be limited to the underlying implementation,

Re: [Python-ideas] Security: remove "." from sys.path?

On Thu, Jun 1, 2017 at 10:30 AM, Victor Stinner wrote: > Hi, > > Perl 5.26 succeeded to remove the current working directory from the > default include path (our Python sys.path): > >

Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

On Thu, May 25, 2017 at 11:55 AM, Brett Cannon wrote: > I'm +1 on Nick's idea of the low-level, private API existing first to > facilitate testing, but putting off any public API until we're sure we can > make it function in a way we're happy with to more generally expose. Same

Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

On Thu, May 25, 2017 at 11:19 AM, Nathaniel Smith wrote: > My impression is that the code to support them inside CPython is fine, but > they're broken and not very useful in the sense that lots of C extensions > don't really support them, so in practice you can't reliably use them

[Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

Although I haven't been able to achieve the pace that I originally wanted, I have been able to work on my multi-core Python idea little-by-little. Most notably, some of the blockers have been resolved at the recent PyCon sprints and I'm ready to move onto the next step: exposing multiple

Re: [Python-ideas] AtributeError inside __get__

On Sat, Dec 31, 2016 at 12:33 AM, Nick Coghlan wrote: > On 31 December 2016 at 05:53, Ethan Furman wrote: >> On 12/30/2016 07:10 AM, Chris Angelico wrote: >>> What module would be appropriate, though? >> >> Well, DynamicClassAttribute is kept in the types

Re: [Python-ideas] Null coalescing operator

On Fri, Oct 28, 2016 at 7:13 AM, Barry Warsaw wrote: > You could of course support exactly the same syntax being proposed as a > language change, e.g. > > from operator import attrgetter > r = attrgetter('b?.x?.z') > > and then you wouldn't even need the `coalesce`

Re: [Python-ideas] Fwd: Null coalescing operator

On Mon, Sep 12, 2016 at 1:05 AM, Michel Desmoulin wrote: > There is also an alternative to this operator, and it's allowing a > shortcut to do: > > try: > val = do_thing() > except ThingError: > val = "default" > > In the form of: > > val = do_thing() except