Re: [Python-ideas] Provide a way to import module without exec body

[MRAB]

On 2017-12-01 22:46, Steven D'Aprano wrote:

On Fri, Dec 01, 2017 at 10:23:37AM -0500, brent bejot wrote:


I have found myself implementing something like this before.  I was working
on a command-line tool with nested sub-commands.  Each sub-command would
import a script and execute something out of it.  I ended up moving the
importing of those little scripts into the functions that called them
because importing all of them was slowing things down.  A built-in lazy
importer would have made for a better solution.


If I understand your use-case, you have a bunch of functions like this:

def spam_subcommand():
 import spam
 spam.command()

def eggs_subcommand():
 import eggs
 eggs.command()


With lazy importing, you might have something like this:

spam = lazy_import('spam')
eggs = lazy_import('eggs')

def spam_subcommand():
 load(spam)
 spam.command()

def eggs_subcommand():
 load(eggs)
 eggs.command()


I don't see the benefit for your use-case. How would it be better? Have
I missed something?

You don't think you'd need the 'load'; you'd delay execution of the 
module's code until the first attribute access.


All of the script's module dependencies would be listed at the top, but 
you could avoid most of the cost of importing a module until you know 
that you need the module's functionality.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Provide a way to import module without exec body

[Steven D'Aprano]

On Fri, Dec 01, 2017 at 10:23:37AM -0500, brent bejot wrote:

> I have found myself implementing something like this before.  I was working
> on a command-line tool with nested sub-commands.  Each sub-command would
> import a script and execute something out of it.  I ended up moving the
> importing of those little scripts into the functions that called them
> because importing all of them was slowing things down.  A built-in lazy
> importer would have made for a better solution.

If I understand your use-case, you have a bunch of functions like this:

def spam_subcommand():
import spam
spam.command()

def eggs_subcommand():
import eggs
eggs.command()


With lazy importing, you might have something like this:

spam = lazy_import('spam')
eggs = lazy_import('eggs')

def spam_subcommand():
load(spam)
spam.command()

def eggs_subcommand():
load(eggs)
eggs.command()


I don't see the benefit for your use-case. How would it be better? Have 
I missed something?


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Provide a way to import module without exec body

[Brett Cannon]

On Fri, 1 Dec 2017 at 10:11 Neil Schemenauer  wrote:

> On 2017-12-01, Chris Angelico wrote:
> > Can you elaborate on where this is useful, please?
>
> Introspection tools, for example, might want to look at the module
> without executing it.  Also, it is a building block to make lazy loading
> of modules work.  As Nick points out, importlib can do this already.
>
> Currently, the IMPORT_NAME both loads the code for a module and also
> executes it.  The exec happens fairly deep in the guts of importlib.
> This makes import.c and ceval.c mutually recursive.  The locking gets
> complicated.  There are hacks like _call_with_frames_removed() to hide
> the recursion going on.
>
> Instead, we could have two separate opcodes, one that gets the module
> but does not exec it (i.e. a function like __import__() that returns a
> future) and another opcode that actually does the execution.  Figuring
> out all the details is complicated.
>
> Possible benefits:
>
> - importlib is simpler
>
> - reduce the amount of stack space used (removing recursion by
>   "continuation passing style").
>
> - makes profiling Python easier.  Tools like valgrind get confused
>   by call cycle between ceval.c and import.c.
>
> - easier to implement lazy loading of modules (not necessarily a
>   standard Python feature but will make 3rd party implementations
>   cleaner)
>
> I'm CCing Brett as I'm sure he has thoughts on this, given his intimate
> knowledge of importlib.  To me, it seems like __import__() has a
> terribly complicated API because it does so many different things.
>

I have always assumed the call signature for __import__() was because the
import-related opcodes pushed so much logic into the function instead of
doing it in opcodes (I actually blogged about this at
https://snarky.ca/if-i-were-designing-imort-from-scratch/). Heck, the thing
takes in locals() and yet never uses them (and its use of globals() is
restricted to specific values so it really doesn't need to be quite so
broad). Basically I wished __import__() looked like
importlib.import_module().


>
> Maybe two opcodes is not even enough.  Maybe we should have one to
> resolve relative imports (i.e. import.c:resolve_name), one to load but
> not exec a module given its absolute name (i.e.  _find_and_load()
> without the exec), one to exec a loaded module, one or more to handle
> the horror of "fromlist" (i.e.  _handle_fromlist()).
>

I have always wanted to at least break up getting the module and fromlist
as separate opcodes, so +1 for that. Name resolution could potentially be
done as an opcode as it relies on execution state pulled from the globals
of the module, but the logic also isn't difficult so +0 for that (i.e.
making an opcode that calls something more like importlib.import_module()
is more critical to me than eliminating the 'package' argument to that
call, but I don't view it as a bad thing to have another opcode for that
either).

As for the completely separating the loading and execution, I don't have a
need for what's being proposed so I don't have an opinion. I basically made
sure Eric Snow structured specs so that lazy loading as currently supported
works so I got what I wanted for basic lazy importing (short of the PyPI
package I keep talking about writing to add a nicer API around lazy
importing :) .

-Brett



>
> Regards,
>
>   Neil
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

[C Anthony Risinger]

On Nov 29, 2017 10:09 PM, "Steven D'Aprano"  wrote:

> On Thu, Nov 30, 2017 at 11:21:48AM +1300, Greg Ewing wrote:
>
> > It seems that many people think about unpacking rather
> > differently from the way I do. I think the difference
> > is procedural vs. declarative.
> >
> > To my way of thinking, something like
> >
> >a, b, c = x
> >
> > is a pattern-matching operation. It's declaring that
> > x is a sequence of three things, and giving names to
> > those things. It's not saying to *do* anything to x.
>
> I hadn't thought of that interpretation before, but now that Greg
> mentions it, its so obvious-in-hindsight that I completely agree with
> it. I think that we should promote this as the "one obvious"
> interpretation.
>
> Obviously as a practical matter, there are some x (namely iterators)
> where you cannot extract items without modifying x, but in all other
> cases I think that the pattern-matching interpretation is superiour.
>

This conversation about suitcases, matching, and language assumptions is
interesting. I've realized two concrete things about how I understand
unpacking, and perhaps, further explain the dissonance we have here:

* Unpacking is destructuring not pattern matching.
* Tuple syntax is commas, paren, one, or both.

For the former, destructuring, this reply conveys my thoughts verbatim:
https://groups.google.com/forum/#!topic/clojure/SUoThs5FGvE

"There are two different concerns in what people refer to as "pattern
matching": binding and flow-control. Destructuring only addresses binding.
Pattern matching emphasizes flow control, and some binding features
typically come along for free with whatever syntax it uses. (But you could
in principle have flow control without binding.)"

The only part of unpacking that is 'pattern matching' is the fact that it
blows up spectacularly when the LHS doesn't perfectly match the length of
RHS, reversing flow via exception:

>>> 0,b = 0,1
  File "", line 1
SyntaxError: can't assign to literal

If Python really supported pattern matching (which I would 100% love! yes
please), and unpacking was pattern matching, the above would succeed
because zero matches zero. Pattern matching is used extensively in
Erlang/Elixir for selecting between various kinds of clauses (function,
case, etc), but you also see *significant* use of the `[a, b | _] = RHS`
construct to ignore "the remainder" because 99% of the time what you really
want is to [sometimes!] match a few things, bind a few others, and ignore
what you don't understand or need. This is why destructuring Elixir maps or
JS objects never expect (or even support AFAIK) exact-matching the entire
object... it would render this incredible feature next to useless!
*Destructuring is opportunistic if matching succeeds*.

For the latter, tuples-are-commas-unless-they-are-parens :-), I suspect I'm
very much in the minority here. While Python is one of my favorite
languages, it's only 1 of 10, and I didn't learn it until I was already 4
languages deep. It's easy to forget how odd tuples are because they are so
baked in, but I've had the "well, ehm, comma is the tuple constructor...
usually" or "well, ehm, you are actually returning 1 tuple... not 2 things"
conversation with *dozens* of seasoned developers. Even people
professionally writing Python for months or more. Other languages use more
explicit, visually-bounded object constructors. This makes a meaningful
difference in how a human's intuition interprets the meaning of a new
syntax. *Objects start and end but commas have no inherent boundaries*.

These two things combined lead to unpacking problems because I look at all
assignments through the lens of destructuring (first observation)
and unpacking almost never uses parentheses (second observation). To
illustrate this better, the following is how my mind initially parses
different syntax contrasted with what's actually happening (and thus the
correction I've internalized over a decade):

>>> a, b = 5, 6
CONCEPT ... LHS[0] `a` bound to RHS[0] `5`, LHS[1] `b` bound to RHS[1] `6`.
REALITY ... LHS[:] single tuple destructured to RHS[:] single tuple.

>>> a, b = 5, 6, 7
CONCEPT ... LHS[0] `a` bound to RHS[0] `5`, LHS[1] `b` bound to RHS[1]
`6`, RHS[2] `6` is unbound expression.
REALITY ... LHS[:] single tuple destructured to RHS[:] single tuple, bad
match, RAISE ValueError!

>>> (a, b) = 5, 6
>>> [a, b] = 5, 6
CONCEPT ... LHS[0] `(a, b)` bound to RHS[0] `5`, bad match, RAISE TypeError!
REALITY ... LHS[:] single tuple destructured to RHS[:] single tuple.

>>> a, b = it
>>> [a], b = it
CONCEPT ... LHS[0] `a` bound with RHS[0] `it`, LHS[1] is bad match, RAISE
UnboundLocalError/NameError!
REALITY ... `a` bound to `it[0]` and `b` bound to `it[1]` (`it[N]` for
illustration only!)

The tuple thing in particular takes non-zero time to internalize. I
consider it one of Python's warts, attributed to times explained and
comparisons with similar languages. Commas at the top-level, with no other

Re: [Python-ideas] Provide a way to import module without exec body

[Neil Schemenauer]

On 2017-12-01, Chris Angelico wrote:
> Can you elaborate on where this is useful, please?

Introspection tools, for example, might want to look at the module
without executing it.  Also, it is a building block to make lazy loading
of modules work.  As Nick points out, importlib can do this already.

Currently, the IMPORT_NAME both loads the code for a module and also
executes it.  The exec happens fairly deep in the guts of importlib.
This makes import.c and ceval.c mutually recursive.  The locking gets
complicated.  There are hacks like _call_with_frames_removed() to hide
the recursion going on.

Instead, we could have two separate opcodes, one that gets the module
but does not exec it (i.e. a function like __import__() that returns a
future) and another opcode that actually does the execution.  Figuring
out all the details is complicated.

Possible benefits:

- importlib is simpler

- reduce the amount of stack space used (removing recursion by
  "continuation passing style").

- makes profiling Python easier.  Tools like valgrind get confused
  by call cycle between ceval.c and import.c.

- easier to implement lazy loading of modules (not necessarily a
  standard Python feature but will make 3rd party implementations
  cleaner)

I'm CCing Brett as I'm sure he has thoughts on this, given his intimate
knowledge of importlib.  To me, it seems like __import__() has a
terribly complicated API because it does so many different things.

Maybe two opcodes is not even enough.  Maybe we should have one to
resolve relative imports (i.e. import.c:resolve_name), one to load but
not exec a module given its absolute name (i.e.  _find_and_load()
without the exec), one to exec a loaded module, one or more to handle
the horror of "fromlist" (i.e.  _handle_fromlist()).

Regards,

  Neil
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505 vs matrix multiplication

[Chris Barker]

On Fri, Dec 1, 2017 at 6:16 AM, Paul Moore  wrote:

> > I genuinely don't think these kinds of operators are all that useful
> > outside the specific domain of working with semi-structured
> > hierarchical data stored in graph databases and document stores like
> > MongoDB, ElasticSearch, and PostgreSQL JSONB columns, or else piping
> > data between such stores and JSON consuming clients.
>
> In that case, surely there are 3rd party libraries that help with
> extracting such data from raw objects?


Sure -- it's handled by validation libraries like Colander, for instance:

https://docs.pylonsproject.org/projects/colander/en/latest/

And I'd be shocked if there weren't similar functionality built in to
PyMongo and such.

Which makes a lot of sense -- if you want to enforce any kind of schema,
you need to specify which fields are optional, and handle those cases.

So this is perhaps largely about making it easier to write such libraries.

Though the other really common use case is None default parameters:

def fun(self, par1, par2, par3=None):
if par3 is None:
self.par3 = something_mutable

is a really common idiom -- though not all the verbose.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 447: Adding type.__getdescriptor__

[Ronald Oussoren]

> On 1 Dec 2017, at 12:29, Nick Coghlan  wrote:
> 
> On 1 December 2017 at 21:04, Ronald Oussoren  wrote:
>> The second question is more a design question: what’s the better design, 
>> having __getdescriptor__ as a class method on classes or as method on 
>> metaclasses? Either one would work, but a class method appears to be easier 
>> to use and with the introduction of __init_subclass__ there is a precedent 
>> for going for a class method.
>> 
>> The  current PEP claims that a method on a metaclass would be better to 
>> avoid subtle problems, but ignores the conceptual cost of adding a 
>> metaclass. The subtle problem is that a class can have two direct 
>> superclasses with a __getdescriptor__ when using multiple inheritance, but 
>> that can already be an issue for other methods and that currently includes 
>> __getattribute__ for most of not all usecases where __getdescriptor__ would 
>> be useful.
> 
> I think it's having it being a method on the metaclass that creates
> the infinite regress Mark was worried about: since type's metaclass
> *is* type, if "__getdescriptor__" is looked up as a regular descriptor
> in its own right, then there's no base case to terminate the recursive
> lookup.

But type.__getattribute__ is already special, it cannot be the same as its 
superclass implementation (because that’s object), and object.__getattribute__ 
logically uses type.__getattribute__ to get at type.__dict__. Adding 
__getdescriptor__ in the mix makes type.__getattribute__ a bit messier, but not 
by much.

> 
> By contrast, defining it as a class method opens up two options:
> 
> 1. Truly define it as a class method, and expect implementors to call
> super().__getdescriptor__() if their own lookup fails. I think this
> will be problematic and a good way to get the kinds of subtle problems
> that prompted you to initially opt for the metaclass method.

The only subtle problem is having a class using multiple inheritance that uses
two __getdescriptor__ implementations from two superclasses, where both
do something beyond looking up the name in __dict__. 

Failing to call super().__getdescriptor__() is similar to failing to do so for
other methods. 

> 
> 2. Define it as a class method, but have the convention be for the
> *caller* to worry about walking the MRO, and hence advise class
> implementors to *never* call super() from __getdescriptor__
> implementations (since doing so would nest MRO walks, and hence
> inevitably have weird outcomes). Emphasise this convention by passing
> the current base class from the MRO as the second argument to the
> method.

But that’s how I already define the method, that is the PEP proposes to change
the MRO walking loop to:
 
   for cls in mro_list:
try:
return cls.__getdescriptor__(name)  # was cls.__dict__[name]
except AttributeError:  # was KeyError
pass

Note that classes on the MRO control how to try to fetch the name at that 
level. The code is the same for __getdescriptor__ as a classmethod and as a 
method on the  metaclass.

I don’t think there’s a good technical reason to pick either option, other than 
that the metaclass option forces an exception when creating a class that 
inherits (using multiple inheritance) from two classes that have a custom 
__getdescriptor__. I’m not convinced that this is a good enough reason to go 
for the metaclass option.

> 
> The reason I'm liking option 2 is that it leaves the existing
> __getattribute__ implementations fully in charge of the MRO walk, and
> *only* offers a way to override the "base.__dict__[name]"  part with a
> call to "base.__dict__['__getdescriptor__'](cls, base, name)" instead.

Right. That’s why I propose __getdescriptor__ in the first place. This allows 
Python coders to do extra work (or different work) to fetch an attribute of a 
specific class in the MRO and works both with regular attribute lookup as well 
as lookup through super(). 

The alternative I came up with before writing the PEP is to add a special API 
that can be used by super(), but that leads to more code duplication as coders 
would have to implement both __getattribute__ and this other method.

I guess another options would be a method that does the work including walking 
the MRO, but that leads to more boilerplate for users of the API. 

BTW. I haven’t had a lot of time to work on the implementation. The code in 
typeobject.c has changed enough that this needs more work than tweaking the 
patch until it applies cleanly.

Ronald

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Provide a way to import module without exec body

[brent bejot]

I have found myself implementing something like this before.  I was working
on a command-line tool with nested sub-commands.  Each sub-command would
import a script and execute something out of it.  I ended up moving the
importing of those little scripts into the functions that called them
because importing all of them was slowing things down.  A built-in lazy
importer would have made for a better solution.

On Fri, Dec 1, 2017 at 5:36 AM, Nick Coghlan  wrote:

> On 1 December 2017 at 20:17, Steven D'Aprano  wrote:
> > If that's what you mean, then no, I haven't wished for that. Unless I'm
> > missing something, it seems pointless. When, and why, would I want to
> > import an empty module?
>
> Having access to something along these lines is the core building
> block for lazy loading. You figure out everything you need to actually
> load the module up front (so you still get an immediate ImportError if
> the module doesn't even exist), but then defer actually finishing the
> load to the first __getattr__ invocation (so if you never actually use
> the module, you avoid any transitive imports, as well as any other
> costs of initialising it).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add a dict with the attribute access capability

[Ivan Pozdeev via Python-ideas]

On 01.12.2017 1:19, Greg Ewing wrote:

Ivan Pozdeev via Python-ideas wrote:
I needed to hold an external function reference in an object instance 
(if I assigned it to an attribute, it was converted into an instance 
method).


No, that only happens to functions stored in *class* attributes,
not instance attributes.

>>> class A:
...    pass
...
>>> a = A()
>>>
>>> def f():
...    print("I'm just a function")
...
>>> a.x = f
>>> a.x()
I'm just a function

Well, yes, that was a singleton class, so I kept data in the class 
object. Now I can simplify the code by only keeping the instance 
reference in the class, thank you. (Without knowing this, that bore no 
visible benefits.)


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505 vs matrix multiplication

[Paul Moore]

On 1 December 2017 at 13:40, Nick Coghlan  wrote:
> I genuinely don't think these kinds of operators are all that useful
> outside the specific domain of working with semi-structured
> hierarchical data stored in graph databases and document stores like
> MongoDB, ElasticSearch, and PostgreSQL JSONB columns, or else piping
> data between such stores and JSON consuming clients.

In that case, surely there are 3rd party libraries that help with
extracting such data from raw objects? Or if not, how seriously has
anyone looked at developing one? With the ability to create
specialised getattr and getitem behaviour, is it really so difficult
to produce a class that allows users to extract hierarchical data? I
know it probably couldn't do as good a job as if there were dedicated
syntax, but as a basis for a proposal that said "current best practice
(using module XXX) looks like this, but would be improved with the
following language support" it would help to ground the discussion in
real use cases. In the context of comparisons with matrix
multiplication, PEP 465 put a lot of time into explaining how all the
ways of approaching the problem short of a language change had been
tried and found wanting. Maybe PEP 505 should be held to a similar
standard?

At the moment, 99% of the discussion seems rooted in generalised "it
would help a lot of code" with readability arguments based on
artificial examples, and that's not really helping move the discussion
forward.

To be clear, I understand the problem of reading semi-structured data.
I've hit it myself and been frustrated by it. But my reaction was "why
am I not able to find a library that does this?", and when I couldn't
find such a library, my assumption was that people in general don't
find the current behaviour sufficiently frustrating to do anything
about it. And I was in the same situation - it annoys me, but not
enough to write a helper module (and certainly not enough that I'm
crying out for a language change). So I do appreciate the need, I just
don't think "language change" should be the first thing that's
suggested.

Paul

PS Some of the above may have been covered in the PEPs and previous
discussions. I haven't reread them - but any serious reboot of the
discussion should probably start with a summary of where we're up to.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505 vs matrix multiplication

[Nick Coghlan]

On 1 December 2017 at 04:49, Antoine Pitrou  wrote:
> On Wed, 29 Nov 2017 18:14:36 +1000
> Nick Coghlan  wrote:
>>
>> As far as utility goes, I put it in a similar category to matrix
>> multiplication: if you don't need it, you don't need it, but when you
>> do need it, you need it a *lot*.
>
> As someone who appreciates both the matrix multiplication operator and
> "async/await", I really don't think PEP 505-style operators (regardless
> of their spellings) fall into the same conceptual bucket.
>
> There's no risk of matrix multiplication operators bleeding into
> non-domain specific code, and the readers of domain specific code
> already know about the matrix multiplication operator and what it
> does (or they should anyway, since it's so damn useful).  It's like
> "async/await": you won't find them in regular non-async code, so the
> mental burden only falls on specialists who write and read event-driven
> networking code (mostly, even though Guido would like to see parsers
> based on the idiom too :-)). Conversely, PEP 505-style operators may
> appear in everyday code regardless of their application domain or
> target.  This in turn increases the mental burden for *everyone*.

I genuinely don't think these kinds of operators are all that useful
outside the specific domain of working with semi-structured
hierarchical data stored in graph databases and document stores like
MongoDB, ElasticSearch, and PostgreSQL JSONB columns, or else piping
data between such stores and JSON consuming clients.

If there was a high change of their being broadly adopted outside
those domains, I don't think we'd be seeing the sharp division of
opinion that we see between folks that consider these operators to be
obviously useful, and those that are honestly befuddled as to why on
earth anyone would ever want them.

It's just that where matrix multiplication and async programming have
rich vocabularies and computer science foundations to draw on, the
None-aware and None-severing operators in different languages arise
more from the pragmatic hackery of working with semi-structured data
for tasks that are essentially a matter of reshaping blobs of JSON
from one system to feed into another (it's an imperative approach to
the kind of work that XSLT does for XML in a more declarative way).

The closest mathematical equivalent is a quiet NaN, but the PEP
already discusses some of the limitations of pursuing that approach
for algorithmic operations in Python:
https://www.python.org/dev/peps/pep-0505/#haskell-style-maybe

I think the PEP as currently written honestly goes too far into
symbolic magic, and hence doesn't give a reader enough hints to
plausibly guess what "?." means if they've never seen it before:

return jsonify(
first_seen=site.first_seen?.isoformat(),
id=site.id,
is_active=site.is_active,
last_seen=site.last_seen?.isoformat(),
url=site.url.rstrip('/')
)

Thus the idea of possibly using "??" as a pronoun symbol (akin to "_"
at the interactive prompt) to allow both the condition and the RHS in
a conditional expression to refer to the LHS:

return jsonify(
first_seen = site.first_seen if ?? is None else ??.isoformat(),
id=site.id,
is_active = site.is_active,
last_seen = site.last_seen if ?? is None else ??.isoformat(),
url = site.url.rstrip('/')
)

Here, even someone who's never seen "??" before has at least some
chance of guessing "OK, it looks like some kind of implicitly defined
variable reference. What might it be referring to? Well, the code
calls a method on it if it isn't None, so perhaps it means the LHS of
the conditional expression it appears in?".

And the transcription to English would probably use an actual pronoun:

"We set first_seen to site.first_seen if that's None, otherwise we
set it to the result of site.first_seen's isoformat() method"

Further suggesting a potential name for the symbol: a "that"
reference. (Where precisely what "that" refers to will depend on where
the symbol appears, similar to regular pronoun usage in English).

It isn't the same way that languages that use " ?  :
" for their conditional expression syntax do things, but
spelling conditional expressions as " if  else " is
already pretty unique in its own right :)

"That" references could also be expanded to comprehensions and
generator expressions in a fairly useful way:

[f(x) for x in iterable if ?? is not None]

Pronounced something like "f of x, if that's not None, for x in iterable".

Cheers,
Nick.

P.S. As previously noted, I don't think we should rush into anything
for 3.7 on this point, hence my deferral of all the related PEPs,
rather than requesting pronouncement. I do think the symbolic pronoun
idea is potentially worth exploring further for 3.8 though.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia

Re: [Python-ideas] PEP 447: Adding type.__getdescriptor__

[Nick Coghlan]

On 1 December 2017 at 21:04, Ronald Oussoren  wrote:
> The second question is more a design question: what’s the better design, 
> having __getdescriptor__ as a class method on classes or as method on 
> metaclasses? Either one would work, but a class method appears to be easier 
> to use and with the introduction of __init_subclass__ there is a precedent 
> for going for a class method.
>
> The  current PEP claims that a method on a metaclass would be better to avoid 
> subtle problems, but ignores the conceptual cost of adding a metaclass. The 
> subtle problem is that a class can have two direct superclasses with a 
> __getdescriptor__ when using multiple inheritance, but that can already be an 
> issue for other methods and that currently includes __getattribute__ for most 
> of not all usecases where __getdescriptor__ would be useful.

I think it's having it being a method on the metaclass that creates
the infinite regress Mark was worried about: since type's metaclass
*is* type, if "__getdescriptor__" is looked up as a regular descriptor
in its own right, then there's no base case to terminate the recursive
lookup.

By contrast, defining it as a class method opens up two options:

1. Truly define it as a class method, and expect implementors to call
super().__getdescriptor__() if their own lookup fails. I think this
will be problematic and a good way to get the kinds of subtle problems
that prompted you to initially opt for the metaclass method.

2. Define it as a class method, but have the convention be for the
*caller* to worry about walking the MRO, and hence advise class
implementors to *never* call super() from __getdescriptor__
implementations (since doing so would nest MRO walks, and hence
inevitably have weird outcomes). Emphasise this convention by passing
the current base class from the MRO as the second argument to the
method.

The reason I'm liking option 2 is that it leaves the existing
__getattribute__ implementations fully in charge of the MRO walk, and
*only* offers a way to override the "base.__dict__[name]"  part with a
call to "base.__dict__['__getdescriptor__'](cls, base, name)" instead.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 447: Adding type.__getdescriptor__

[Ronald Oussoren]

> On 1 Dec 2017, at 11:32, Nick Coghlan  wrote:
> 
> On 1 December 2017 at 19:15, Ronald Oussoren  wrote:
> 
>> Maybe, but how would this work with super()? Super walks the MRO of type of
>> the instance, but skips the class on the MRO.  This is not equivalent to
>> walking the MRO of the second class on the MRO when you use multiple
>> inheritance,
>> 
>> This also has some other disadvantages. The first is that
>> tp.__getdescriptor__ would replace the default behaviour for the entire MRO
>> and it would be possible to have different behavior for classes on the MRO.
>> The second, minor. one is that __getdescriptor__ would have to reimplement
>> the default logic of walking the MRO, but that logic is fairly trivial.
> 
> I believe those can both be addressed by structuring the override a
> little differently, and putting it at the level of individual
> attribute retrieval on a specific class (rather than on all of its
> subclasses):
> 
>def _PyType_Lookup(tp, name):
>mro = tp.mro()
>assert isinstance(mro, tuple)
> 
>for base in mro:
>   assert isinstance(base, type)
> 
>   try:
>   getdesc = base.__dict__["__getdescriptor__"]
>   except KeyError:
>   try:
>   return base.__dict__[name]
>   except KeyError:
>   pass
>   else:
>   try:
>   return getdesc(tp, base, name)
>   except AttributeError:
>   pass
> 
>return None
> 
> In that version, the __getdescriptor__ signature would be:
> 
>def __getdescriptor__(dynamic_cls, base_cls, attr):
>…

That’s basically what’s in the PEP, except that the PEP says that type will 
implement __getdescriptor__ to make cooperative subclassing easier.  The 
current patch inlines type.__getdescriptor__ in the lookup code for efficiency 
reasons (that is, the runtime cost for not using this feature is basically a 
pointer test instead of a function call), but would work just as well without 
inlining.

I’m pretty sure that the first concern isn’t really there, in the end 
attribute/descriptor resolution ends up in object and type whose implementation 
must be magic in some way, even without this PEP. 

The second question is more a design question: what’s the better design, having 
__getdescriptor__ as a class method on classes or as method on metaclasses? 
Either one would work, but a class method appears to be easier to use and with 
the introduction of __init_subclass__ there is a precedent for going for a 
class method.

The  current PEP claims that a method on a metaclass would be better to avoid 
subtle problems, but ignores the conceptual cost of adding a metaclass. The 
subtle problem is that a class can have two direct superclasses with a 
__getdescriptor__ when using multiple inheritance, but that can already be an 
issue for other methods and that currently includes __getattribute__ for most 
of not all usecases where __getdescriptor__ would be useful.

Ronald


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

[Steven D'Aprano]

On Thu, Nov 30, 2017 at 08:02:08PM +0300, Ivan Pozdeev via Python-ideas wrote:

> My experience with these operators in C# says:
> * They do save "more than a few keystrokes". Even more importantly, they 
> allow to avoid double evaluation or the need for a temporary variable 
> workaround that are inherent in " if  else "
>     * (An alternative solution for the latter problem would be an 
> assignment expression, another regularly rejected proposal.)
> * They make it temptingly easy and implicit to ignore errors.

How? 

> * They are alien to Python's standard semantics on search failure which 
> is to raise an exception rather than return None

Alien, like this?

py> mo = re.match(r'\d:', 'abc')
py> mo is None
True


Besides, who said this is limited to searching? I don't remember even a 
single example in the PEP being about searching.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Provide a way to import module without exec body

[Nick Coghlan]

On 1 December 2017 at 20:17, Steven D'Aprano  wrote:
> If that's what you mean, then no, I haven't wished for that. Unless I'm
> missing something, it seems pointless. When, and why, would I want to
> import an empty module?

Having access to something along these lines is the core building
block for lazy loading. You figure out everything you need to actually
load the module up front (so you still get an immediate ImportError if
the module doesn't even exist), but then defer actually finishing the
load to the first __getattr__ invocation (so if you never actually use
the module, you avoid any transitive imports, as well as any other
costs of initialising it).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 447: Adding type.__getdescriptor__

[Nick Coghlan]

On 1 December 2017 at 19:15, Ronald Oussoren  wrote:

> Maybe, but how would this work with super()? Super walks the MRO of type of
> the instance, but skips the class on the MRO.  This is not equivalent to
> walking the MRO of the second class on the MRO when you use multiple
> inheritance,
>
> This also has some other disadvantages. The first is that
> tp.__getdescriptor__ would replace the default behaviour for the entire MRO
> and it would be possible to have different behavior for classes on the MRO.
> The second, minor. one is that __getdescriptor__ would have to reimplement
> the default logic of walking the MRO, but that logic is fairly trivial.

I believe those can both be addressed by structuring the override a
little differently, and putting it at the level of individual
attribute retrieval on a specific class (rather than on all of its
subclasses):

def _PyType_Lookup(tp, name):
mro = tp.mro()
assert isinstance(mro, tuple)

for base in mro:
   assert isinstance(base, type)

   try:
   getdesc = base.__dict__["__getdescriptor__"]
   except KeyError:
   try:
   return base.__dict__[name]
   except KeyError:
   pass
   else:
   try:
   return getdesc(tp, base, name)
   except AttributeError:
   pass

return None

In that version, the __getdescriptor__ signature would be:

def __getdescriptor__(dynamic_cls, base_cls, attr):
...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Stub class for Generic to further improve PEP 560

[Ivan Levkivskyi]

On 1 December 2017 at 00:34, Ilya Kulakov  wrote:

> Anyway, my expectation is that going along this way (i.e. removing all
> runtime API apart from a necessary minimum)
> will give a minor speed-up as compared to PEP 560 at the cost of a
> breaking change (even for small number of developers).
>
>
> I don't think the change will be breaking: usage of this class will be
> entirely voluntarily and does not replace typing.Generic
>
>
If you propose an additional class, then yes, it is non-breaking, but I
still don't see much value given a minor performance improvement.


> PEP 560 already gives overhead of 80% as compared to normal classes in
> worst case scenario
> (empty body with a single generic base). This is actually less than for
> ABCs (they can give up to 120% in worst case scenario).
>
>
> GenericMeta inherits from ABCMeta. Do you mean that it will be removed
> after 560 lands?
>
>
Yes, GenericMeta will be removed.


> Moreover, performance is not a single motivation for PEP 560, there are
> other arguments such as metaclass conflicts which will
> not be solved without the machinery proposed by the PEP.
>
>
> Perhaps you can consider designing Generic / GenericMeta in a way that
> will allow end user to create GenericStub-alike class without much trouble?
>

This can be done, but the hardest part here is not to make the runtime
changes, but to get the support of all
static type checkers (they need to understand what GenericStub means and
add all the necessary special casing).

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Provide a way to import module without exec body

[Steven D'Aprano]

On Fri, Dec 01, 2017 at 02:13:37AM -0600, Neil Schemenauer wrote:
> I have been working on reducing Python statup time.  It would be
> nice if there was some way to load a module into memory without exec
> of its body code.  I'm sure other people have wished for this.

I don't understand why you would want to do this. Given a source file:


# module.py
spam = 1
eggs = 2


if you import the module without executing the code in the module, 
surely you'll get a bare module with nothing in it? Then:

module.spam
module.eggs

will both fail with AttributeError.

If that's what you mean, then no, I haven't wished for that. Unless I'm 
missing something, it seems pointless. When, and why, would I want to 
import an empty module?


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

[Paul Moore]

On 1 December 2017 at 09:48, Kirill Balunov  wrote:
> Probably, some time ago it was necessary to split this thread into two
> questions:
> 1. Philosophical question regarding sequences and iterators. In particular,
> should they behave differently depending on the context,
> or, in other words, whether to emphasize their different nature as
> fixed-size containers and those that are lazily produce values on demand.
> 2. Additional syntax in the assignment statement for partial extraction of
> values from the iterable.

That's a good summary of the two elements of the discussion here.

On (1), I'd say that Python should *not* have context-dependent
semantics like this. It's something Perl was famous for (list and
scalar contexts) and IMO makes for pretty unreadable code. Python's
Zen here is "Explicit is better than implicit". Specifically, having
the semantics of the assignment statement vary depending on the type
of the value being assigned seems like a very subtle distinction, and
not in line with any other statement in the language.

On (2), that's something that is relatively simple to debate - all of
the normal rules for new syntax proposals apply - what problem does it
solve, how much of an improvement over existing ways of solving the
problem does the proposal give, how easy is it for beginners to
understand and for people encountering it to locate the documentation,
does it break backward compatibility, etc... Personally I don't think
it's a significant enough benefit but I'm willing to be swayed if good
enough arguments are presented (currently the "a, b, ... = value"
syntax is my preferred proposal, but I don't think there's enough
benefit to justify implementing it).

> 2017-11-30 22:19 GMT+03:00 Paul Moore :
>>
>>
>> Mostly corner cases, and I don't believe there have been any
>> non-artificial examples posted in this thread.
>>
>> Certainly no-one has offered a real-life code example that is made
>> significantly worse by
>> the current semantics, and/or which couldn't be easily worked around
>> without needing a language change.
>
>
> Yes, in fact, this is a good question, is whether that is sufficiently
> useful to justify extending the syntax. But it is not about corner cases, it
> is rather usual situation.
> Nevertheless, this is the most difficult moment for Rationale. By now, this
> feature does not give you new opportunities for solving problems. It's more
> about expressiveness and convenience. You can write:
>
> x, y, ... = iterable
>
> or,
>
> it = iter(iterable)
> x, y = next(it), next(it)
>
> or,
>
> from itertools import isclice
> x, y = islice(iterable, 2)
>
> or,
> x, y = iterable[:2]
>
> and others, also in some cases when you have infinite generator or iterator,
> you should use 2nd or 3rd.

It's significant to me that you're still only able to offer artificial
code as examples. In real code, I've certainly needed this type of
behaviour, but it's never been particularly problematic to just use

first_result = next(it)
second_result - next(it)

Or if I have an actual sequence, x, y = seq[:2]

The next() approach actually has some issues if the iterator
terminates early - StopIteration is typically not the exception I
want, here. But all that means is that I should use islice more. The
reason i don't think to is because I need to import it from itertools.
But that's *not* a good argument - we could use the same argument to
make everything a builtin. Importing functionality from modules is
fundamental to Python, and "this is a common requirement, so it should
be a builtin" is an argument that should be treated with extreme
suspicion. What I *don't* have a problem with is the need to specify
the number of items - that seems completely natural to me, I'm
confirming that I require an iterable that has at least 2 elements at
this point in my code.

The above is an anecdotal explanation of my experience with real code
- still not compelling, but hopefully better than an artificial
example with no real-world context :-)


> In fact, this has already been said and probably
> I will not explain it better:
>
> 2017-11-28 1:40 GMT+03:00 Greg Ewing :
>>
>> Guido van Rossum wrote:
>>>
>>> Is this problem really important enough that it requires dedicated
>>> syntax? Isn't the itertools-based solution good enough?
>>
>>
>> Well, it works, but it feels very clumsy. It's annoying to
>> have to specify the number of items in two places.
>>
>> Also, it seems perverse to have to tell Python to do *more*
>> stuff to mitigate the effects of stuff it does that you
>> didn't want it to do in the first place.
>>
>> Like I said, I'm actually surprised that this doesn't already
>> work. To me it feels more like filling in a piece of
>> functionality that was overlooked, rather than adding a
>> new feature. Filling in a pothole in the road rather than
>> bulding a new piece of road.
>>
>> (Pushing the road analogy maybe 

Re: [Python-ideas] How assignment should work with generators?

[Kirill Balunov]

2017-11-29 22:33 GMT+03:00 Steve Barnes :

>
> Just a thought but what about a syntax something along the lines of:
>
> a, b, *remainder = iterable
>
> Where remainder becomes the iterable with the first two values consumed
> by assigning to a & b. If the iterator has less than 2 values, (in the
> above case), remaining it should error, if it has exactly 2 then
> remainder would become an exhausted iterable. Of course the user could
> also use:
>
> a, b, *iterable = iterable
>
> Others may differ but this syntax has a lot of similarity to the f(a, b,
> *args) syntax, possibly enough that most users could understand it.
>


Before I started this thread, not so long ago, I have already asked a
question about this semantics [1
].

But it appears to be very ambiguous in practice for the various rhs:

...
x, *y, z = some_iter
*x , y, z = some_iter
x, y, *z = some_iter

And only for the last case it will mean something special. In addition, it
is a huge backward compatibility break.

Probably, some time ago it was necessary to split this thread into two
questions:
1. Philosophical question regarding sequences and iterators. In particular,
should they behave differently depending on the context,
or, in other words, whether to emphasize their different nature as
fixed-size containers and those that are lazily produce values on demand.
2. Additional syntax in the assignment statement for partial extraction of
values from the iterable.

2017-11-30 22:19 GMT+03:00 Paul Moore :

>
> Mostly corner cases, and I don't believe there have been any non-artificial
> examples posted in this thread.

Certainly no-one has offered a real-life code example that is made
> significantly worse by
> the current semantics, and/or which couldn't be easily worked around
> without needing a language change.


Yes, in fact, this is a good question, is whether that is sufficiently
useful to justify extending the syntax. But it is not about corner cases,
it is rather usual situation.
Nevertheless, this is the most difficult moment for Rationale. By now, this
feature does not give you new opportunities for solving problems. It's more
about expressiveness and convenience. You can write:

x, y, ... = iterable

or,

it = iter(iterable)
x, y = next(it), next(it)

or,

from itertools import isclice
x, y = islice(iterable, 2)

or,
x, y = iterable[:2]

and others, also in some cases when you have infinite generator or
iterator, you should use 2nd or 3rd. In fact, this has already been said
and probably I will not explain it better:

2017-11-28 1:40 GMT+03:00 Greg Ewing :

> Guido van Rossum wrote:
>
>> Is this problem really important enough that it requires dedicated
>> syntax? Isn't the itertools-based solution good enough?
>>
>
> Well, it works, but it feels very clumsy. It's annoying to
> have to specify the number of items in two places.
>
> Also, it seems perverse to have to tell Python to do *more*
> stuff to mitigate the effects of stuff it does that you
> didn't want it to do in the first place.
>
> Like I said, I'm actually surprised that this doesn't already
> work. To me it feels more like filling in a piece of
> functionality that was overlooked, rather than adding a
> new feature. Filling in a pothole in the road rather than
> bulding a new piece of road.
>
> (Pushing the road analogy maybe a bit too far, the current
> itertools solution is like digging *more* potholes to make
> the road bumpy enough that you don't notice the first
> pothole.)
>
> (Or failing that, couldn't we add something to itertools to make it more
>> readable rather than going straight to new syntax?)
>>
>
> I'm not sure how we would do that. Even if we could, it
> would still feel clumsy having to use anything from itertools
> at all.


 With kind regards, -gdg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 447: Adding type.__getdescriptor__

[Ronald Oussoren]

> On 1 Dec 2017, at 07:01, Nick Coghlan  wrote:
> 
> On 1 December 2017 at 01:23, Ronald Oussoren  wrote:
>> 1) Last time around Mark Shannon worried that this introduces infinite
>> recursion in the language itself (in my crummy summary, please read this
>> message to get the real concern
>> ).  Is
>> this truly a problem?  I don’t think there is a problem, but I’m worried
>> that I don’t fully understand Mark’s concerns.
>> 
>> 2) PEP 487 introduced __init_subclass__ as a class method to avoid having to
>> write a metaclass for a number of use cases.  My PEP currently does require
>> a metaclass, but it might be nicer to switch to a regular class method
>> instead (like __init_subclass__).
> 
> I think the second point there may actually allow you to resolve the
> first one, by way of making `__getdescriptor__` an optional override
> of the typical lookup algorithm. That is:
> 
>def _PyType_Lookup(tp, name):
> 
># New optional override for descriptor lookups
>try:
># Ordinary attribute lookup, *NOT* a descriptor lookup
>getdesc = tp.__getdescriptor__
>except AttributeError:
>pass
>else:
>return getdesc(name)
> 
># Default algorithm used in the absence of an override
>mro = tp.mro()
>assert isinstance(mro, tuple)
> 
>for base in mro:
>   assert isinstance(base, type)
> 
>   try:
>   return base.__dict__[name]
>   except KeyError:
>   pass
> 
>return None
> 
> If you did go this way, then we'd want to add a
> "types.getdescriptor(cls, name)" API to expose _PyType_Lookup at the
> Python layer, since "getattr(type(obj), name)" wouldn't be an accurate
> emulation of the algorithm any more.

Maybe, but how would this work with super()? Super walks the MRO of type of the 
instance, but skips the class on the MRO.  This is not equivalent to walking 
the MRO of the second class on the MRO when you use multiple inheritance,

This also has some other disadvantages. The first is that tp.__getdescriptor__ 
would replace the default behaviour for the entire MRO and it would be possible 
to have different behavior for classes on the MRO. The second, minor. one is 
that __getdescriptor__ would have to reimplement the default logic of walking 
the MRO, but that logic is fairly trivial.

BTW. getattr(type(obj), name) is not an accurate emulation of _PyType_Lookup 
even now, thanks to metaclasses. In particular:

```
class A_meta (type):
@property
def description(self): return "meta description"

@description.setter
def description(self, v):
raise RuntimeError

class A (metaclass=A_meta):

@property
def description(self): return "description"

@description.setter
def description(self, v):
raise RuntimeError


a = A()

print(A.description)# prints “meta description"
print(a.description)# prints “description"
print(getattr(type(a), 'description’))  # prints “meta description"
```

The setter definitions are necessary to ensure that the properties are data 
descriptors, which are handled differently than function descriptors by 
__getattribute__.

Ronald

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Provide a way to import module without exec body

[Nick Coghlan]

On 1 December 2017 at 18:37, Nick Coghlan  wrote:
> On 1 December 2017 at 18:13, Neil Schemenauer
>  wrote:
>> I have been working on reducing Python statup time.  It would be
>> nice if there was some way to load a module into memory without exec
>> of its body code.  I'm sure other people have wished for this.
>>
>> Perhaps there could be a new special function, similar to __import__
>> for this purpose.  E.g.  __load_module__().  To actually execute the
>> module, I had the idea to make module objects callable, i.e. tp_call
>> for PyModule_Type.  That's a little too cute though and will cause
>> confusion.  Maybe instead, add a function attribute to modules, e.g.
>> mod.__exec__().
>>
>> I have a little experimental code, just a small step:
>>
>> https://github.com/nascheme/cpython/tree/import_defer_exec
>>
>> We need importlib to give us the module object and the bytecode
>> without doing the exec().
>
> What does actually doing the load give that simply calling
> https://docs.python.org/3/library/importlib.html#importlib.util.find_spec
> doesn't?
>
> At that point, you know the module exists, and how to load it, which
> is all a lazy loading implementations really needs to be confident
> that a subsequent actual execution attempt will be able to start.

After posting this, and while filing
https://bugs.python.org/issue32192, I double checked how
"importlib.util.module_from_spec" works, and it turns out that already
handle the main part of what you're after: it creates the module
without executing it.

The actual execution is then handled by running
"module.__spec__.loader.exec_module(module)".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Provide a way to import module without exec body

[Nick Coghlan]

On 1 December 2017 at 18:13, Neil Schemenauer
 wrote:
> I have been working on reducing Python statup time.  It would be
> nice if there was some way to load a module into memory without exec
> of its body code.  I'm sure other people have wished for this.
>
> Perhaps there could be a new special function, similar to __import__
> for this purpose.  E.g.  __load_module__().  To actually execute the
> module, I had the idea to make module objects callable, i.e. tp_call
> for PyModule_Type.  That's a little too cute though and will cause
> confusion.  Maybe instead, add a function attribute to modules, e.g.
> mod.__exec__().
>
> I have a little experimental code, just a small step:
>
> https://github.com/nascheme/cpython/tree/import_defer_exec
>
> We need importlib to give us the module object and the bytecode
> without doing the exec().

What does actually doing the load give that simply calling
https://docs.python.org/3/library/importlib.html#importlib.util.find_spec
doesn't?

At that point, you know the module exists, and how to load it, which
is all a lazy loading implementations really needs to be confident
that a subsequent actual execution attempt will be able to start.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Provide a way to import module without exec body

[Chris Angelico]

On Fri, Dec 1, 2017 at 7:13 PM, Neil Schemenauer
 wrote:
> I have been working on reducing Python statup time.  It would be
> nice if there was some way to load a module into memory without exec
> of its body code.  I'm sure other people have wished for this.

I haven't. Can you elaborate on where this is useful, please?

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Provide a way to import module without exec body

[Neil Schemenauer]

I have been working on reducing Python statup time.  It would be
nice if there was some way to load a module into memory without exec
of its body code.  I'm sure other people have wished for this.

Perhaps there could be a new special function, similar to __import__
for this purpose.  E.g.  __load_module__().  To actually execute the
module, I had the idea to make module objects callable, i.e. tp_call
for PyModule_Type.  That's a little too cute though and will cause
confusion.  Maybe instead, add a function attribute to modules, e.g.
mod.__exec__().

I have a little experimental code, just a small step:

https://github.com/nascheme/cpython/tree/import_defer_exec

We need importlib to give us the module object and the bytecode
without doing the exec().  My hackish solution is to set properties
on __spec__ and then have PyImport_ImportModuleLevelObject() do the
exec().
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/