[Python-ideas] An itertools.interleave / itertools.join function
Hi I like using itertools for creating long strings while not paying the cost of intermediate strings (by eventually calling str.join on the whole iterator). However, one missing feature is to mimic the behavior of str.join as an iterator: an iterator that returns the items of an iterable, separated by the separator. I suggest name "interleave" or "join" (whichever is the most clear / least ambigous). def interleave(sep, iterable): """ Makes an iterator that returns elements from an iterable, separated by the separator. """ notfirst = False for i in iterable: if notfirst: yield sep else: notfirst = True yield i Could imagine a more elaborate implementation that can take several iterators, and would be equivalent to lambda chain_zip_interleave sep, *iterables: itertools.chain.from_iterable(interleave((sep,), zip(*iterables))) But that may be seriously overkill, and I have hard time describing it. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YWT5BVGPNO3UBW4DZYYPXVCJY2JH7B4H/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: __init__ in module names
On Tue, Dec 08, 2020 at 03:07:48PM -0800, Gregory Szorc wrote:
> We're talking about "__init__" being the exact name of a module component.
> `"__init__" in fullname.split(".")`, as I wrote in my initial email.
I'm sorry, you've just added more confusion to me, rather than less :-(
What's a module component? Do you mean a directory name? If I have a
package:
mypackage/subpackage/__init__/stuff/things/spam.py
never mind that this is an unusual naming convention, but according to
your test given above, that means that importing
mypackage.subpackage.__init__.stuff.things.spam
should be treated according to your proposal.
I *thought* you meant only modules called literally "__init__.py" (or
equivalent .pyc etc), and only if they are part of a package, but your
test above suggests that this is not the case.
For example, if I have a non-package module called "__init__.py" with no
package structure, what happens? Right now I can import that easily, and
because there is no package structure, it's just a module with an
unusual name.
So that's two cases that your "in fullname.split" test above seems to
mishandle:
- bare modules (not part of a package) called "__init__";
- (sub)packages called "__init__", corresponding to a directory called
"__init__".
There may be other cases.
I think your proposal needs to be more specific about *exactly* what
cases you are handling. I think it is *only* the case that you have a
package and the __init__.py file inside the package is imported
directly, but your test for that case is too broad.
(Modulo file extensions of course.)
> "__init__ as a substring - as weird as that may be - should be allowed.
> This is because it is only the exact "__init__" filename that is treated
> specially by the filename resolver.
Okay, that's what I thought you meant, but your insistence on this test:
"__init__" in fullname.split(".")
[Me]
> > I think that you would need to demonstrate that:
> >
> > (1) people are inadvertly importing "__init__", not realising the
> > consequences;
> >
>
> > (2) leading to bugs in their code;
> >
> > (3) that this happens *more often* than people intentionally and
> > knowingly importing "__init__";
> >
> > (4) and that there is a work-around for those intentionally importing
> > "__init__".
> >
[Gregory]
> I can't speak for the people practicing this pattern because I'm not one of
> them. However, I'm willing to bet a lot of them are either cargo culting
> the practice or thinking "oh, this __init__.py file exists, '__init__' must
> be the module name." The importer/code works and they run with it.
So do you have any examples of *actual* bugs caused by this feature, or
is this is a hypothetical problem?
> I'm also willing to wager that people engaged in this practice (who
> apparently don't fully understand how the importer works otherwise they
> wouldn't be using "__init__" in module names) don't realize that this
> practice results in multiple module objects. I'm willing to wager that a
> subset of these people have seen weird bugs or undesired behavior due to
> the existence of multiple module objects (e.g. 2 instances of a supposed
> module singleton).
>
> I wish I could find stronger evidence here, but I don't have anything
> concrete, just a GitHub search showing code in the wild, likely authored by
> people who aren't Python experts.
Or maybe they are experts who are doing exactly what they need to do,
e.g. see MAL's comment about using the existence of an __init__ module
to distinguish a namespace package from a regular package.
As the person proposing a backwards-compatibility breaking change, the
onus is on you (or somebody supporting your proposal) to demonstrate
that breaking people's code, even with just a warning, is *less bad*
than the status quo.
"I feel, I expect, I would wager" etc is not evidence for this being an
actual genuine problem that needs fixing. I would need to see more than
just your gut feeling to support changing this behaviour. As MAL says,
this behaviour apparently goes back to Python 1.5 days and the number of
bug reports caused by it is approximately zero.
I agree with you that importing __init__ directly *feels* weird, it's a
code smell, but then I feel the same about people who call dunders
directly, and even referring to __doc__ directly feels weird. But that
doesn't mean it is broken.
--
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/YPQEEVNL4W47GU2ONQP45J5NL2PPGQFK/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Lazy Type Casting
On Tue, Dec 08, 2020 at 11:46:59AM -, Mathew Elman wrote: > I would like to propose adding lazy types for casting builtins in a > lazy fashion. e.g. `lazy_tuple` which creates a reference to the > source iterable and a morally immutable sequence but only populates > the tupular container when it or the source is used. What are your use-cases for this? Does this include things like `lazy_list`, `lazy_float`, `lazy_bool`, `lazy_str`, `lazy_bytearray` etc? > An alternative to adding lazy types / lazy type casting (making it > possible to implement these oneself) would be to add method call hooks > to python, since this would allow having a "freeze value" callback > hooked into the __setitem__ and __getitem__ methods. Hooks may be a > more useful solution for wider use cases as well. Nope, sorry, I don't see how that would work. Here I have a list: L = [50, 40, 30, 20, 10] Suppose these hooks exist. I want to make a "lazy tuple": t = lazy_tuple(L) How do these hooks freeze the list? What if I have more than one lazy object pointing at the same source? s = lazy_str(L) And then follow with a different method call? L.insert(2, "surprise!") I just can't see how this will work. -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/6OLOHQBMYVPQSDAI5HRTRBVJRKDYB4QM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: __init__ in module names
On Tue, Dec 8, 2020 at 2:44 PM Steven D'Aprano wrote:
> On Tue, Dec 08, 2020 at 11:47:22AM -0800, Gregory Szorc wrote:
>
> > It was recently brought to my attention via
> > https://github.com/indygreg/PyOxidizer/issues/317 that "__init__" in
> module
> > names is something that exists in Python code in the wild.
>
> Can we be clear whether you are talking about "__init__" **in** module
> names (a substring, like "my__init__module.py") or "__init__" **as** a
> module name (not a substring, "__init__.py" exactly)?
>
We're talking about "__init__" being the exact name of a module component.
`"__init__" in fullname.split(".")`, as I wrote in my initial email.
"__init__ as a substring - as weird as that may be - should be allowed.
This is because it is only the exact "__init__" filename that is treated
specially by the filename resolver.
>
> My guess is that you are only talking about the second case, can you
> confirm please?
>
>
> > In that GitHub issue and https://bugs.python.org/issue42564, I
> discovered
> > that what's happening is the stdlib PathFinder meta path importer is
> "dumb"
> > and doesn't treat "__init__" in module names specially.
>
> I would hope and expect that it doesn't. If somebody explicitly asks to
> do something, Python should do what they ask, and not something
> different.
>
> Analogy: if I explicitly call `someobject.__init__(*args)` then I would
> expect Python to call that method, and not to translate that into a call
> to `type(someobject).__new__(*args)` because "__init__ is special".
>
> The interpreter should do as its told and not try to guess what I meant.
>
Then this is an argument against silent normalization of the module name. I
buy that argument.
>
>
> > If someone uses
> > syntax like "import foo.__init__" or "from .__init__ import foo",
> > PathFinder operates on "__init__" like any other string value and
> proceeds
> > to probe the filesystem for the relevant {.py, .pyc, .so, etc} files. The
> > "__init__" files do exist in probed locations and PathFinder summarily
> > constructs a new module object, albeit with "__init__" in its name. The
> end
> > result is you have 2 module objects and sys.modules entries referring to
> > the same file, keyed to different names (e.g. "foo" and "foo.__init__").
>
> Right. But given that the caller has *explicitly* asked for
> "foo.__init__" to be imported, presumably that is exactly the
> behaviour they want.
>
> Are there cases where people inadvertly import "foo.__init__" and are
> then surprised to get a different module from "foo" alone?
>
> Personally, I think this is a case for education. If you are explicitly
> touching *any* dunder name, it is up to you to know what you are doing.
>
>
> > There is a strong argument to be made that "__init__" in module names
> > should be treated specially. It seems wrong to me that you are allowed to
> > address the same module/file through different names
>
> Can you make that strong argument please? "It seems wrong to me" is a
> very weak argument.
>
>
> > (let's pretend filesystem path normalization doesn't exist)
>
> Let's not pretend, because it does exist.
>
> There is also the "module importing itself" issue, and hard links, and
> I'm sure that there are other clever ways to get two module objects out
> of a single module file. Deep copying doesn't work, but modules are very
> simple objects and you can copy them by hand:
>
> import spam
> eggs = type(spam)("eggs", vars(spam).copy())
>
>
> > and that the filesystem
> > encoding of Python module files/names is addressable through the importer
> > names. This feels like a bug that inadvertently shipped.
>
> Not to me. The current behaviour is exactly what I would expect.
>
>
> > However, code in the wild is clearly relying on "__init__" in module
> names
> > being allowed. And changing the behavior is backwards incompatible and
> > could break this code.
>
> Right, so "it feels wrong" is not a sufficient reason to make that
> breaking change.
>
> I think that you would need to demonstrate that:
>
> (1) people are inadvertly importing "__init__", not realising the
> consequences;
>
> (2) leading to bugs in their code;
>
> (3) that this happens *more often* than people intentionally and
> knowingly importing "__init__";
>
> (4) and that there is a work-around for those intentionally importing
> "__init__".
>
I can't speak for the people practicing this pattern because I'm not one of
them. However, I'm willing to bet a lot of them are either cargo culting
the practice or thinking "oh, this __init__.py file exists, '__init__' must
be the module name." The importer/code works and they run with it.
I'm also willing to wager that people engaged in this practice (who
apparently don't fully understand how the importer works otherwise they
wouldn't be using "__init__" in module names) don't realize that this
practice results in multiple module objects. I'm willing to wager that a
subset of these people have seen weird bugs or
[Python-ideas] Re: __init__ in module names
On Tue, Dec 08, 2020 at 11:47:22AM -0800, Gregory Szorc wrote:
> It was recently brought to my attention via
> https://github.com/indygreg/PyOxidizer/issues/317 that "__init__" in module
> names is something that exists in Python code in the wild.
Can we be clear whether you are talking about "__init__" **in** module
names (a substring, like "my__init__module.py") or "__init__" **as** a
module name (not a substring, "__init__.py" exactly)?
My guess is that you are only talking about the second case, can you
confirm please?
> In that GitHub issue and https://bugs.python.org/issue42564, I discovered
> that what's happening is the stdlib PathFinder meta path importer is "dumb"
> and doesn't treat "__init__" in module names specially.
I would hope and expect that it doesn't. If somebody explicitly asks to
do something, Python should do what they ask, and not something
different.
Analogy: if I explicitly call `someobject.__init__(*args)` then I would
expect Python to call that method, and not to translate that into a call
to `type(someobject).__new__(*args)` because "__init__ is special".
The interpreter should do as its told and not try to guess what I meant.
> If someone uses
> syntax like "import foo.__init__" or "from .__init__ import foo",
> PathFinder operates on "__init__" like any other string value and proceeds
> to probe the filesystem for the relevant {.py, .pyc, .so, etc} files. The
> "__init__" files do exist in probed locations and PathFinder summarily
> constructs a new module object, albeit with "__init__" in its name. The end
> result is you have 2 module objects and sys.modules entries referring to
> the same file, keyed to different names (e.g. "foo" and "foo.__init__").
Right. But given that the caller has *explicitly* asked for
"foo.__init__" to be imported, presumably that is exactly the
behaviour they want.
Are there cases where people inadvertly import "foo.__init__" and are
then surprised to get a different module from "foo" alone?
Personally, I think this is a case for education. If you are explicitly
touching *any* dunder name, it is up to you to know what you are doing.
> There is a strong argument to be made that "__init__" in module names
> should be treated specially. It seems wrong to me that you are allowed to
> address the same module/file through different names
Can you make that strong argument please? "It seems wrong to me" is a
very weak argument.
> (let's pretend filesystem path normalization doesn't exist)
Let's not pretend, because it does exist.
There is also the "module importing itself" issue, and hard links, and
I'm sure that there are other clever ways to get two module objects out
of a single module file. Deep copying doesn't work, but modules are very
simple objects and you can copy them by hand:
import spam
eggs = type(spam)("eggs", vars(spam).copy())
> and that the filesystem
> encoding of Python module files/names is addressable through the importer
> names. This feels like a bug that inadvertently shipped.
Not to me. The current behaviour is exactly what I would expect.
> However, code in the wild is clearly relying on "__init__" in module names
> being allowed. And changing the behavior is backwards incompatible and
> could break this code.
Right, so "it feels wrong" is not a sufficient reason to make that
breaking change.
I think that you would need to demonstrate that:
(1) people are inadvertly importing "__init__", not realising the
consequences;
(2) leading to bugs in their code;
(3) that this happens *more often* than people intentionally and
knowingly importing "__init__";
(4) and that there is a work-around for those intentionally importing
"__init__".
--
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/QJD4YIWUE6L6NTLA3YP7F6FCZ5MKZRXG/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: __init__ in module names
On Tue, Dec 08, 2020 at 08:06:09PM +, Filipe Laíns wrote: > I support this change. Can you explain why you support this breaking change? I am especially interested in cases where people accidentally, or inadvertently, imported "package.__init__" and then used it without realising that it is distinct from `import package` alone. -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WRVUD3WVJYXNV242AZMF35HXCL55SBVR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: __init__ in module names
On Tue, 2020-12-08 at 11:47 -0800, Gregory Szorc wrote:
> PyOxidizer's pure Rust implementation of a meta path importer
> (https://pyoxidizer.readthedocs.io/en/stable/oxidized_importer_oxidized_finder.html
> ) has been surprisingly effective at finding corner cases and behavior quirks
> in Python's importing mechanisms.
>
> It was recently brought to my attention via
> https://github.com/indygreg/PyOxidizer/issues/317 that "__init__" in module
> names is something that exists in Python code in the wild. (See
> https://github.com/search?l=Python=%22from+.__init__+import%22=Code for
> some examples.)
>
> In that GitHub issue and https://bugs.python.org/issue42564, I discovered that
> what's happening is the stdlib PathFinder meta path importer is "dumb" and
> doesn't treat "__init__" in module names specially. If someone uses syntax
> like "import foo.__init__" or "from .__init__ import foo", PathFinder operates
> on "__init__" like any other string value and proceeds to probe the filesystem
> for the relevant {.py, .pyc, .so, etc} files. The "__init__" files do exist in
> probed locations and PathFinder summarily constructs a new module object,
> albeit with "__init__" in its name. The end result is you have 2 module
> objects and sys.modules entries referring to the same file, keyed to different
> names (e.g. "foo" and "foo.__init__").
>
> There is a strong argument to be made that "__init__" in module names should
> be treated specially. It seems wrong to me that you are allowed to address the
> same module/file through different names (let's pretend filesystem path
> normalization doesn't exist) and that the filesystem encoding of Python module
> files/names is addressable through the importer names. This feels like a bug
> that inadvertently shipped.
>
> However, code in the wild is clearly relying on "__init__" in module names
> being allowed. And changing the behavior is backwards incompatible and could
> break this code.
>
> Anyway, I was encouraged by Brett Cannon to email this list to assess the
> appetite for introducing a backwards incompatible change to this behavior. So
> here's my strawman/hardline proposal:
>
> 1. 3.10 introduces a DeprecationWarning for "__init__" appearing as any module
> part component (`"__init__" in fullname.split(".")`).
> 2. Some future release (I'm unsure which) turns it into a hard error.
>
> (A less aggressive proposal would be to normalize "__init__" in module names
> to something more reasonable - maybe stripping trailing ".__init__" from
> module names. But I'll start by proposing the stricter solution.)
>
> What do others think we should do?
>
> Gregory
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UEHUJOH4WDYLH4QPN7RC3HXCEUK3OAMO/
> Code of Conduct: http://python.org/psf/codeofconduct/
I support this change.
It would also be fairly trivial to make the hard error be descriptive and user-
friendly, like the Python 3 does for Python 2 style print statements, so it's
not like code would just stop working in a obscure way.
Cheers,
Filipe Laíns
signature.asc
Description: This is a digitally signed message part
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/JVVZJHCIYZ4X4IZ7BTWWVGQJTABJMNRJ/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: __init__ in module names
On 08.12.2020 20:47, Gregory Szorc wrote:
> Anyway, I was encouraged by Brett Cannon to email this list to assess the
> appetite for introducing a backwards incompatible change to this behavior. So
> here's my strawman/hardline proposal:
>
> 1. 3.10 introduces a DeprecationWarning for "__init__" appearing as any module
> part component (`"__init__" in fullname.split(".")`).
> 2. Some future release (I'm unsure which) turns it into a hard error.
-1 on this proposal. We don't want to needlessly break code just because
they use a feature of the existing implementation, which has been around
for decades.
Moreover, if you use namespace packages, a module __init__.py does not
have to exist in the directory, so importing pkg.__init__ is a way
to test for such a case.
> (A less aggressive proposal would be to normalize "__init__" in module names
> to
> something more reasonable - maybe stripping trailing ".__init__" from module
> names. But I'll start by proposing the stricter solution.)
-0 on this, since it may break code. If done, Python should issue
a warning to flag the issue.
Third solution: leave things as they are and document it.
+1 on this one, since it's been like this for ages (going way back to
the Python 1.x days).
Cheers,
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Experts (#1, Dec 08 2020)
>>> Python Projects, Coaching and Support ...https://www.egenix.com/
>>> Python Product Development ...https://consulting.egenix.com/
::: We implement business ideas - efficiently in both time and costs :::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
https://www.egenix.com/company/contact/
https://www.malemburg.com/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/Y7LB7BWSBV42VAKQELAONHILK2BSNGZP/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] __init__ in module names
PyOxidizer's pure Rust implementation of a meta path importer (
https://pyoxidizer.readthedocs.io/en/stable/oxidized_importer_oxidized_finder.html)
has been surprisingly effective at finding corner cases and behavior quirks
in Python's importing mechanisms.
It was recently brought to my attention via
https://github.com/indygreg/PyOxidizer/issues/317 that "__init__" in module
names is something that exists in Python code in the wild. (See
https://github.com/search?l=Python=%22from+.__init__+import%22=Code
for some examples.)
In that GitHub issue and https://bugs.python.org/issue42564, I discovered
that what's happening is the stdlib PathFinder meta path importer is "dumb"
and doesn't treat "__init__" in module names specially. If someone uses
syntax like "import foo.__init__" or "from .__init__ import foo",
PathFinder operates on "__init__" like any other string value and proceeds
to probe the filesystem for the relevant {.py, .pyc, .so, etc} files. The
"__init__" files do exist in probed locations and PathFinder summarily
constructs a new module object, albeit with "__init__" in its name. The end
result is you have 2 module objects and sys.modules entries referring to
the same file, keyed to different names (e.g. "foo" and "foo.__init__").
There is a strong argument to be made that "__init__" in module names
should be treated specially. It seems wrong to me that you are allowed to
address the same module/file through different names (let's pretend
filesystem path normalization doesn't exist) and that the filesystem
encoding of Python module files/names is addressable through the importer
names. This feels like a bug that inadvertently shipped.
However, code in the wild is clearly relying on "__init__" in module names
being allowed. And changing the behavior is backwards incompatible and
could break this code.
Anyway, I was encouraged by Brett Cannon to email this list to assess the
appetite for introducing a backwards incompatible change to this behavior.
So here's my strawman/hardline proposal:
1. 3.10 introduces a DeprecationWarning for "__init__" appearing as any
module part component (`"__init__" in fullname.split(".")`).
2. Some future release (I'm unsure which) turns it into a hard error.
(A less aggressive proposal would be to normalize "__init__" in module
names to something more reasonable - maybe stripping trailing ".__init__"
from module names. But I'll start by proposing the stricter solution.)
What do others think we should do?
Gregory
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/UEHUJOH4WDYLH4QPN7RC3HXCEUK3OAMO/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make for/while loops nameable.
On Tue, Dec 8, 2020 at 10:42 AM David Mertz wrote: > On Tue, Dec 8, 2020 at 3:31 PM Ricky Teachey wrote: > >> I agree with the person who called this a brilliant solution. Here is the >> code from the link for completeness: >> > > I'm not diss'ing the approach. But it doesn't really save that much over > a sentinel variable `break_to_middle`. And if you wanted to break to any > of several levels from the innermost, you'd need multiple context managers. > > >> Fair enough. For me the "brilliance" comes in how readable it comes out to be; I find it far easier to scan through and understand (without even looking at the brk() function to see what it does) the first time through then the sentinel approach. My syntax suggestion came from thinking about how to make it even more readable (providing one can swallow the "this decorator automatically becomes a context manager" part-- which might be too hard to swallow I don't know). --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LAKWE3MDR75OEOC7SBEOWYVTSFMVPGSS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make for/while loops nameable.
On Tue, Dec 8, 2020 at 3:31 PM Ricky Teachey wrote:
> I agree with the person who called this a brilliant solution. Here is the
> code from the link for completeness:
>
I'm not diss'ing the approach. But it doesn't really save that much over a
sentinel variable `break_to_middle`. And if you wanted to break to any of
several levels from the innermost, you'd need multiple context managers.
> for outer in range(5):
> with breakable() as brk:
> for middle in range(5):
> for inner in range(10):
> print(outer, middle, inner)
> if middle == 1 and inner == 5:
> brk()
> print(f"end middle loop #{middle}")
> print(f"end outer loop #{outer}")
> print("done")
>
--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/DGK3IH7CVZSFR2G45THGVKIFTDOGVLGQ/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make for/while loops nameable.
On Mon, Dec 7, 2020 at 6:27 PM Daniel Moisset wrote:
> For the likely rare situation where I'd want to do this rather than
> refactoring into a function, I might try with something like this without
> requiring changes to the language:
>
> from contextlib import contextmanager
>
> @contextmanager
> def breakable():
> class Break(Exception): pass
> def breaker(): raise Break
> try:
> yield breaker
> except Break:
> pass
>
> This allows defining a block that can be "aborted" (which can have a loop
> or any other statement block within); you abort it by calling the resulting
> value of the context manager. An example of use is available at
> https://gist.github.com/dmoisset/55f5916f9f339a143b6f3d155de8706e
>
>
I agree with the person who called this a brilliant solution. Here is the
code from the link for completeness:
for outer in range(5):
with breakable() as brk:
for middle in range(5):
for inner in range(10):
print(outer, middle, inner)
if middle == 1 and inner == 5:
brk()
print(f"end middle loop #{middle}")
print(f"end outer loop #{outer}")
print("done")
Iterating on that idea a bit and taking advantage of the walrus, what if
syntax were created allowing us to remove the extra indentation level, and
also allowing us to contextualize any loop without having to employ the
with statement? I imagine is looking something like this:
for outer in range(5):
@breakable
for middle in loopX := range(5):
for inner in range(10):
print(outer, middle, inner)
if middle == 1 and inner == 5:
loopX.brk()
print(f"end middle loop #{middle}")
print(f"end outer loop #{outer}")
print("done")
Above, breakable will be some decorator that returns a context manager, and
the "loopX" loop happens in the context of that context manager (with the
name "loopX" assigned to the context object).
Here is some more detail about what I am imagining happening with this new
syntax:
1. the name `loopX` is assigned to the iterable in the for statement, as
usual
2. the first time the for statement ` for middle in loopX := range(5):
` is interpreted/excuted, the presence of the decorator causes this to
happen before iteration starts:
loopX = breakable(loopX)
with loopX as loopX:
for middle in loopX := range(5):
# the "loopX" loop is now executed within the loopX context
3. the utility of this syntax would not be limited to the breaking of
named loops. with this syntax in place, one could write any kind of context
manager they want for contextualizing the execution of loops (both for
loops and while loops).
4. last note: if no name is supplied to the iterable object using the
walrus, then applying the decorator to the loop would be a syntax error
What do you think?
---
Ricky.
"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
[Python-ideas] Re: Thinking about dead simple import hooks...
Hello, On Wed, 9 Dec 2020 02:01:58 +1100 Chris Angelico wrote: > On Tue, Dec 8, 2020 at 11:05 PM Paul Sokolovsky > wrote: > > > It'd be best to > > > just write what you want directly, using the tools in importlib. > > > > Of course, the implementation of the above for CPython would be > > based on importlib. And other implementations could implement that > > "directly", for the background idea of it being how to implement > > import hooks as frugally as possible, and easily allow to do most > > of what could be wanted of single-file import hooking. > > Have you tried to just write this as-is? Python already offers a > number of tools. Try writing it using the existing tools and see if > this even needs to be a proposal. I have written this (what my initial mail talks about) for both Pycopy and CPython. Now I wonder if I shouldn't really follow "the most minimal possible implementation" (the usual Pycopy criteria) and upgrade API a just little bit. Likely, the code byte count will still decide, but I'd like to get a second opinion from usability PoV, etc. > > ChrisA -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IY56ESZKSRJVQWOXDQRS3HII7TZ2EBAP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Thinking about dead simple import hooks...
On Tue, Dec 8, 2020 at 11:05 PM Paul Sokolovsky wrote: > > It'd be best to > > just write what you want directly, using the tools in importlib. > > Of course, the implementation of the above for CPython would be based on > importlib. And other implementations could implement that "directly", > for the background idea of it being how to implement import hooks as > frugally as possible, and easily allow to do most of what could be > wanted of single-file import hooking. Have you tried to just write this as-is? Python already offers a number of tools. Try writing it using the existing tools and see if this even needs to be a proposal. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RJOZIMWN7KXOFSTDT73BGWZ3HJ2E2PUL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Lazy Type Casting
Hello, On Tue, 08 Dec 2020 11:46:59 - "Mathew Elman" wrote: > I am not sure if this has been suggested before, so my apologies if > it has. > > I would like to propose adding lazy types for casting builtins in a > lazy fashion. e.g. `lazy_tuple` which creates a reference to the > source iterable and a morally immutable sequence but only populates > the tupular container when it or the source is used. > > Note that if the original object is not a built-in, will not be used > after casting _or_ is immutable, this is trivial to implement. The > difficulty arises when this it not the case, since the lazy object > needs to also freeze any values that are mutated in the original to > avoid side-effects. You pinpointed the problem exactly. So, the only reliable way to create "lazy" version is to store a copy *eagerly*, which makes the point moot. Lazy-evaluation languages usually deal with that problem by disallowing mutation in the first place (i.e. being purely functional). You can use that solution in Python too. (Without "no mutation" error checking, but it should be possible to implement the alternative API which is "immutable", and I bet someone even did that (PoC-style of course).) > An alternative to adding lazy types / lazy type casting (making it > possible to implement these oneself) would be to add method call > hooks to python, since this would allow having a "freeze value" > callback hooked into the __setitem__ and __getitem__ methods. Hooks > may be a more useful solution for wider use cases as well. [] -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MV4RZSAC3GK65SICBUIOMNNQDDFIK5MC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Thinking about dead simple import hooks...
Hello, On Tue, 8 Dec 2020 22:32:42 +1100 Chris Angelico wrote: > On Tue, Dec 8, 2020 at 9:54 PM Paul Sokolovsky > wrote: > > > > Hello, > > > > On Tue, 8 Dec 2020 07:29:15 -0300 > > "Joao S. O. Bueno" wrote: > > > > > And how would Python compute the "full basename of the file to be > > > imported"? > > > > The way it does it usually. > > > > That involves a number of searches for exact file and/or directory > names, so you can't just search a path arbitrarily. > > I would suggest that this is simultaneously too generic to be > meaningful AND insufficiently generic to be useful. I usually like such statements myself, but that one made me wonder... Do you have an example (in the "single file == single module" domain) where it will be "insufficiently generic to be useful"? > It'd be best to > just write what you want directly, using the tools in importlib. Of course, the implementation of the above for CPython would be based on importlib. And other implementations could implement that "directly", for the background idea of it being how to implement import hooks as frugally as possible, and easily allow to do most of what could be wanted of single-file import hooking. In that regard, the question is again about API such import hooks offer. For example, how do you feel about "The hook should be aware that there might have been previous hook(s), and it's current hook's responsibility to call them." clause? > > ChrisA -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OBJ6CDGYYDQES6NQ53LSKGVXZNMAMEP5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Lazy Type Casting
I am not sure if this has been suggested before, so my apologies if it has. I would like to propose adding lazy types for casting builtins in a lazy fashion. e.g. `lazy_tuple` which creates a reference to the source iterable and a morally immutable sequence but only populates the tupular container when it or the source is used. Note that if the original object is not a built-in, will not be used after casting _or_ is immutable, this is trivial to implement. The difficulty arises when this it not the case, since the lazy object needs to also freeze any values that are mutated in the original to avoid side-effects. An alternative to adding lazy types / lazy type casting (making it possible to implement these oneself) would be to add method call hooks to python, since this would allow having a "freeze value" callback hooked into the __setitem__ and __getitem__ methods. Hooks may be a more useful solution for wider use cases as well. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SDAUO33DVPISBFQ52M2N4WEWPLJ4UDIS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Thinking about dead simple import hooks...
On Tue, Dec 8, 2020 at 9:54 PM Paul Sokolovsky wrote: > > Hello, > > On Tue, 8 Dec 2020 07:29:15 -0300 > "Joao S. O. Bueno" wrote: > > > And how would Python compute the "full basename of the file to be > > imported"? > > The way it does it usually. > That involves a number of searches for exact file and/or directory names, so you can't just search a path arbitrarily. I would suggest that this is simultaneously too generic to be meaningful AND insufficiently generic to be useful. It'd be best to just write what you want directly, using the tools in importlib. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QWEY7OJTHBTJVGD4EDKPR2FCQNV7GB6C/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Thinking about dead simple import hooks...
Hello, On Tue, 8 Dec 2020 07:29:15 -0300 "Joao S. O. Bueno" wrote: > And how would Python compute the "full basename of the file to be > imported"? The way it does it usually. > How could it guess among all directories on sys.path the one > containing the "file", and check if it is a file or a package without > going through the existing mechanism? It would not guess anything, it would iterate thru sys.path, as it usually does, and would give the hook a chance to see if the needed file exists. This proposal is also about *modules* and treats packages just as "hierarchical structure of modules" (Python does more than that, but this is "dead simple import hooks"). > Maybe this proposal is good - but possibly, just cutingt the "full > basename" and pass > to the "simple hook" whatever string is on the import statement would > be better. That's already possible, just hook into __import__() (or one of the other extension points provided by importlib). It's exactly the "convert full hierarchical module name into the actual file path" what's the boring part of it, and what I want to leave to the existing import system. [] -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BKNB3GRSKJZHBX5LUBBIFJJZ25I3PRQ7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Thinking about dead simple import hooks...
And how would Python compute the "full basename of the file to be
imported"?
How could it guess among all directories on sys.path the one containing
the "file", and check if it is a file or a package without going through
the existing mechanism?
Maybe this proposal is good - but possibly, just cutingt the "full
basename" and pass
to the "simple hook" whatever string is on the import statement would be
better.
On Tue, 8 Dec 2020 at 06:30, Paul Sokolovsky wrote:
> Hello,
>
> It would be nice to have a simple and easy way to override module
> importing, instead of that babylon which CPython has in that regard.
> (Even if initially "limited" to filesystem importing, most people
> won't ever find that limiting at all.)
>
> The desired semantics would be:
>
> For "import foo", before "/path1/foo.py", "/path2/foo.py" (where
> /path1:/path2 come from sys.path) is tried to be imported, a
> user-defined hook function can be called. It should take just one param
> - full basename of the module to be imported, e.g. "/path1/foo",
> "/path2/foo" for the cases above. The hook should either return a
> module object, the result of the import, or None to signal that standard
> processing should take place. The hook should be aware that there might
> have been previous hooks, and it's current hook's responsibility to call
> them.
>
> Example of a hook:
>
> def import_hook(path):
> if os.path.exists(path + ".my_ext"):
> mod = imp.new_module("")
> # populate mod here
> return mod
>
> A crucial semantic detail is that all possible module types from one
> sys.path entry should be tried first, before going to the next entry.
> (CPython import extensibility instead seems to be built around idea
> that it's fine to do multiple passes over sys.path, loading only a
> particular extension module type on each pass, which is not the desired
> semantics).
>
> Questions:
>
> 1. Does anybody know such thing to already exist?
> 2. Would you tweak anything in the API/boilerplate semantics of the
> above?
>
>
> --
> Best regards,
> Paul mailto:pmis...@gmail.com
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/NIQQVA3OJFUHL3INBMBUBMTP24W74XEO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/NKMLGALBWLXEFKKRXM6MY3ISE64F4AAA/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Thinking about dead simple import hooks...
Hello,
It would be nice to have a simple and easy way to override module
importing, instead of that babylon which CPython has in that regard.
(Even if initially "limited" to filesystem importing, most people
won't ever find that limiting at all.)
The desired semantics would be:
For "import foo", before "/path1/foo.py", "/path2/foo.py" (where
/path1:/path2 come from sys.path) is tried to be imported, a
user-defined hook function can be called. It should take just one param
- full basename of the module to be imported, e.g. "/path1/foo",
"/path2/foo" for the cases above. The hook should either return a
module object, the result of the import, or None to signal that standard
processing should take place. The hook should be aware that there might
have been previous hooks, and it's current hook's responsibility to call
them.
Example of a hook:
def import_hook(path):
if os.path.exists(path + ".my_ext"):
mod = imp.new_module("")
# populate mod here
return mod
A crucial semantic detail is that all possible module types from one
sys.path entry should be tried first, before going to the next entry.
(CPython import extensibility instead seems to be built around idea
that it's fine to do multiple passes over sys.path, loading only a
particular extension module type on each pass, which is not the desired
semantics).
Questions:
1. Does anybody know such thing to already exist?
2. Would you tweak anything in the API/boilerplate semantics of the
above?
--
Best regards,
Paul mailto:pmis...@gmail.com
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/NIQQVA3OJFUHL3INBMBUBMTP24W74XEO/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make for/while loops nameable.
Daniel Moisset writes: > For the likely rare situation where I'd want to do this rather than > refactoring into a function, I might try with something like this without > requiring changes to the language: > @contextmanager > def breakable(): Of course this doesn't address Serhiy's case where he cares about the performance hit of the try, but it sure looks nicer than some of the other alternatives! ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/W73VBWL3AHUT4Y7MMD6HHJTIYTNKBQZC/ Code of Conduct: http://python.org/psf/codeofconduct/
