Re: [Python-ideas] Allow manual creation of DirEntry objects

[Serhiy Storchaka]

On 16.08.16 22:35, Brendan Moloney wrote:

I have a bunch of functions that operate on DirEntry objects, typically
doing some sort of filtering
to select the paths I actually want to process. The overwhelming
majority of the time these functions
are going to be operating on DirEntry objects produced by the scandir
function, but there are some
cases where the user will be supplying the path themselves (for example,
the root of a directory tree
to process). In my current code base that uses the scandir package I
just wrap these paths in a
'GenericDirEntry' object and then pass them through the filter functions
the same as any results
coming from the scandir function.


You can just create an object that duck-types DirEntry. See for example 
_DummyDirEntry in the os module.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "Immutable Builder" Pattern and Operator

[Serhiy Storchaka]

On 23.01.17 00:45, Soni L. wrote:

I've been thinking of an Immutable Builder pattern and an operator to go
with it. Since the builder would be immutable, this wouldn't work:

long_name = mkbuilder()
long_name.seta(a)
long_name.setb(b)
y = long_name.build()


I think the more pythonic way is:

y = build(a=a, b=b)

A Builder pattern is less used in Python due to the support of keyword 
arguments.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "Immutable Builder" Pattern and Operator

[Serhiy Storchaka]

On 23.01.17 01:30, Soni L. wrote:

On 22/01/17 08:54 PM, Serhiy Storchaka wrote:

On 23.01.17 00:45, Soni L. wrote:

I've been thinking of an Immutable Builder pattern and an operator to go
with it. Since the builder would be immutable, this wouldn't work:

long_name = mkbuilder()
long_name.seta(a)
long_name.setb(b)
y = long_name.build()


I think the more pythonic way is:

y = build(a=a, b=b)

A Builder pattern is less used in Python due to the support of keyword
arguments.


I guess you could do something like this, for an IRC bot builder:

fnircbotbuilder = mkircbotbuilder(network="irc.freenode.net", port=6697,
ssl=true)
mainbot = mkircbotbuilder(parent=fnircbotbuilder,  # ???
  channels=["#bots"]).build()
fndccbotbuilder = mkircbotbuilder(parent=fnircbotbuilder, dcc=true,
channeldcc=true)
dccbot = mkircbotbuilder(parent=fndccbotbuilder,
channels=["#ctcp-s"]).build()
otherircbotbuilder = mkircbotbuilder(parent=fndccbotbuilder,
network="irc.subluminal.net")  # because we want this whole network
otherbot = mkircbotbuilder(parent=otherircbotbuilder,
channels=["#programming"]).build()# to use DCC and channel DCC

But this would be cleaner:

botbuilder =
mkircbotbuilder().network("irc.freenode.net").port(6697).ssl(true)
mainbot = botbuilder.channels(["#bots"]).build()
botbuilder .= dcc(true).channeldcc(true)
dccbot = botbuilder.channels(["#ctcp-s"]).build()
botbuilder .= network("irc.subluminal.net")
otherbot = botbuilder.channels(["#programming"]).build()


In Python you can save common options in a dict and pass them as 
var-keyword argument. Or use functools.partial. In any case you don't 
need a builder class with the build method and a number of configuring 
methods. It can be just a function with optional keyword parameters.


A Builder pattern is often used in languages that don't support passing 
arguments by keyword and partial functions. Python rarely needs the 
purposed class implementing a Builder pattern. Actually a Builder 
pattern is built-in in the language as a part of syntax.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Optional parameters without default value

[Serhiy Storchaka]

Function implemented in Python can have optional parameters with default 
value. It also can accept arbitrary number of positional and keyword 
arguments if use var-positional or var-keyword parameters (*args and 
**kwargs). But there is no way to declare an optional parameter that 
don't have default value. Currently you need to use the sentinel idiom 
for implementing this:


_sentinel = object()
def get(store, key, default=_sentinel):
if store.exists(key):
return store.retrieve(key)
if default is _sentinel:
raise LookupError
else:
return default

There are drawback of this:

* Module's namespace is polluted with sentinel's variables.

* You need to check for the sentinel before passing it to other function 
by accident.


* Possible name conflicts between sentinels for different functions of 
the same module.


* Since the sentinel is accessible outside of the function, it possible 
to pass it to the function.


* help() of the function shows reprs of default values. "foo(bar=object at 0xb713c698>)" looks ugly.



I propose to add a new syntax for optional parameters. If the argument 
corresponding to the optional parameter without default value is not 
specified, the parameter takes no value. As well as the "*" prefix means 
"arbitrary number of positional parameters", the prefix "?" can mean 
"single optional parameter".


Example:

def get(store, key, ?default):
if store.exists(key):
return store.retrieve(key)
try:
return default
except NameError:
raise LookupError

Alternative syntaxes:

* "=" not followed by an expression: "def get(store, key, default=)".

* The "del" keyword: "def get(store, key, del default)".

This feature is orthogonal to supporting positional-only parameters. 
Optional parameters without default value can be positional-or-keyword, 
keyword-only or positional-only (if the latter is implemented).


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Optional parameters without default value

[Serhiy Storchaka]

On 02.03.17 14:20, Paul Moore wrote:

So I guess I'm +0.5 on the proposed "positional only parameters"
syntax, and -1 on any form of new language-defined sentinel value.


My proposition is not about "positional-only parameters".


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Serhiy Storchaka]

On 13.10.16 17:50, Chris Angelico wrote:

Solution: Abolish most of the control characters. Let's define a brand
new character encoding with no "alphabetical garbage". These
characters will be sufficient for everyone:

* [2] Formatting characters: space, newline. Everything else can go.
* [8] Digits: 01234567
* [26] Lower case Latin letters a-z
* [2] Vital social media characters: # (now officially called "HASHTAG"), @
* [2] Can't-type-URLs-without-them: colon, slash (now called both
"SLASH" and "BACKSLASH")

That's 40 characters that should cover all the important things anyone
does - namely, Twitter, Facebook, and email. We don't need punctuation
or capitalization, as they're dying arts and just make you look
pretentious.


https://en.wikipedia.org/wiki/DEC_Radix-50


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Show more info when `python -vV`

[Serhiy Storchaka]

On 14.10.16 10:40, INADA Naoki wrote:

When reporting issue to some project and want to include
python version in the report, python -V shows very limited information.

$ ./python.exe -V
Python 3.6.0b2+

sys.version is more usable, but it requires one liner.

$ ./python.exe -c 'import sys; print(sys.version)'
3.6.0b2+ (3.6:86a1905ea28d+, Oct 13 2016, 17:58:37)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)]

How about `python -vV` shows sys.version?


Are there precedences of combining verbose and version options in other 
programs?


PyPy just outputs sys.version for the --version option.

$ pypy -V
Python 2.7.10 (5.4.1+dfsg-1~ppa1~ubuntu16.04, Sep 06 2016, 23:11:39)
[PyPy 5.4.1 with GCC 5.4.0 20160609]

I think it would not be large breakage if new releases of CPython become 
outputting extended version information by default.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add sorted (ordered) containers

[Serhiy Storchaka]

On 13.10.16 23:36, Марк Коренберг wrote:

I think it should be one standardized implementation of such containers
in CPython.

For example, C++ has both ordered_map and unorderd_map.

Instead of trees, implementation may use SkipList structure, but this is
just implementation details.

Such structres imply fast insertion and deletion, ability to iterate,
and also memory efficiency.


I recommend to read thorough review articles written by Andrew Barnert:

http://stupidpythonideas.blogspot.com/2013/07/sorted-collections-in-stdlib.html

http://stupidpythonideas.blogspot.com/2014/04/sortedcontainers.html


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Enhancing vars()

[Serhiy Storchaka]

On 13.12.16 01:45, Steven D'Aprano wrote:

One of the lesser-known ones is vars(obj), which should be used in place
of obj.__dict__.

Unfortunately, vars() is less useful than it might be, since not all
objects have a __dict__. Some objects have __slots__ instead, or even
both. That is considered an implementation detail of the object.


http://bugs.python.org/issue13290
http://mail.python.org/pipermail/python-dev/2012-October/122011.html


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What about regexp string litterals : re".*" ?

[Serhiy Storchaka]

On 27.03.17 18:17, Simon D. wrote:

After some french discussions about this idea, I subscribed here to
suggest adding a new string litteral, for regexp, inspired by other
types like : u"", r"", b"", br"", f""…

The regexp string litteral could be represented by : re""

It would ease the use of regexps in Python, allowing to have some regexp
litterals, like in Perl or JavaScript.


There are several regular expression libraries for Python. One of them 
is included in the stdlib, but this is not the first regular expression 
library in the stdlib and may be not the last. Particular project can 
choose using an alternative regular expression library (because it has 
additional features or is faster for particular cases).



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add pathlib.Path.write_json and pathlib.Path.read_json

[Serhiy Storchaka]

On 27.03.17 15:50, Ram Rachum wrote:

Hi guys,

What do you think about adding methods pathlib.Path.write_json and
pathlib.Path.read_json , similar to write_text, write_bytes, read_text,
read_bytes?

This would make writing / reading JSON to a file a one liner instead of
a two-line with clause.


Good try, but you have published this idea 5 days ahead of schedule.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Thread-safe generators

[Serhiy Storchaka]

On 15.04.17 01:42, Greg Ewing wrote:

Serhiy Storchaka wrote:

but should not affect performance since locking is used only when you
faced with a generator running in other thread.


I don't think that's true, because the first thread to use a
generator has to lock it as well. And even if there is only
one thread in existence when __next__ is called, another one
could be created before it finishes and try to use the
generator.


The first thread just sets the running flag (as in current code). Due to 
GIL this doesn't need additional synchronization. Other threads check 
this flag and sleep rather than raising an exception. After finishing 
the generator the first thread checks if there are other threads 
awaiting and awake a one of them.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Thread-safe generators

[Serhiy Storchaka]

On 15.04.17 11:55, Stephen J. Turnbull wrote:

Serhiy Storchaka writes:

 > The first thread just sets the running flag (as in current code). Due to
 > GIL this doesn't need additional synchronization.

Can we assume this lack of additional synchronization for other
implementations?  If not, do we care?


Other implementations should have atomic test-and-set operations for the 
running flag. Or other ways to prevent a race condition. So yes, we can 
assume this.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Thread-safe generators

[Serhiy Storchaka]

When use a generator from different threads you can get a ValueError 
"generator already executing". Getting this exception with the single 
thread is a programming error, it in case of different threads it could 
be possible to wait until other thread finish executing the generator. 
The generator can be made thread-safe after wrapping it in a class that 
acquire a lock before calling the generator's __next__ method (for 
example see [1]). But this is not very efficient of course.


I wondering if it is worth to add support of thread-safe generators in 
the stdlib. Either by providing standard decorator (written in C for 
efficiency), or adding threading support just in the generator object. 
The latter may need increasing the size of the generator object for a 
lock and thread identifier (but may be GIL is enough), but should not 
affect performance since locking is used only when you faced with a 
generator running in other thread.


This topic already was raised on Python-Dev [2] but didn't moved too 
much. There are a number of StackOverflow questions about threads and 
generators. We have already encountered this issue in the stdlib. Once 
in regrtest with the -j option ([3], [4]), other time after 
reimplementing tempfile._RandomNameSequence as a generator [5].


[1] http://anandology.com/blog/using-iterators-and-generators/
[2] https://mail.python.org/pipermail/python-dev/2004-February/042390.html
[3] https://bugs.python.org/issue7996
[4] https://bugs.python.org/issue15320
[5] https://bugs.python.org/issue30030

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Positional-only parameters

[Serhiy Storchaka]

On 28.02.17 23:17, Victor Stinner wrote:

My question is: would it make sense to implement this feature in
Python directly? If yes, what should be the syntax? Use "/" marker?
Use the @positional() decorator?


I'm strongly +1 for supporting positional-only parameters. The main 
benefit to me is that this allows to declare functions that takes 
arbitrary keyword arguments like Formatter.format() or 
MutableMapping.update(). Now we can't use even the "self" parameter and 
need to use a trick with parsing *args manually. This harms clearness 
and performance.


The problem with the "/" marker is that it looks ugly. There was an 
excuse for the "*" marker -- it came from omitting the name in "*args". 
The "*" prefix itself means an iterable unpacking, but "/" is not used 
neither as prefix nor suffix.



Do you see concrete cases where it's a deliberate choice to deny
passing arguments as keywords?


dict.__init__(), dict.update(), partial.__new__() and partial.__call__() 
are obvious examples. There are others.


And there was performance reason. Just making the function supporting 
keyword arguments added an overhead even to calls with only positional 
arguments. This was changed recently, but I didn't checked whether some 
overhead is left.



Don't you like writing int(x="123") instead of int("123")? :-) (I know
that Serhiy Storshake hates the name of the "x" parameter of the int
constructor ;-))


I believe weird names like "x" was added when the support of "base" 
keyword was added due to the limitation of 
PyArg_ParseTupleAndKeywords(). All or nothing, either builtin function 
didn't support keyword arguments, or it supported passing by keyword for 
all arguments.


But now it is possible to support passing by keyword only the part of 
parameters. I want to propose to deprecate badly designed keyword names 
of builtins.



By the way, I read that "/" marker is unknown by almost all Python
developers, and [...] syntax should be preferred, but
inspect.signature() doesn't support this syntax. Maybe we should fix
signature() and use [...] format instead?


[...] is not Python syntax too. And it is orthogonal to positional-only 
parameters. [...] doesn't mean that parameters are positional-only. They 
can be passed by keyword, but just don't have default value. On other 
side, mandatory parameters can be positional-only.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Mimetypes Include application/json

[Serhiy Storchaka]

09.08.17 21:17, Brett Cannon пише:
On Wed, 9 Aug 2017 at 10:43 Nate. 
> wrote:

A friend and I have hit a funny situation with the `mimetypes.py`
library
guessing the type for a '.json' file. Is there a reason why '.json'
hasn't been
added to the mapping?


Probably no one thought about it since the module was added in 1997 
which is only 2 years after the creation of JavaScript itself. :)


No one proposed a patch.

Feel free to file a bug at bugs.python.org  and 
if you aren't too bothered then submit a PR to github.com/python/cpython 
 (https://devguide.python.org/ has all 
the details).


https://bugs.python.org/issue30824

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] tempfile.TemporaryDirectory() should be able to create temporary directory at a given arbitrary place

[Serhiy Storchaka]

18.07.17 14:55, George Fischhof пише:
I used tempfile.TemporaryDirectory(). On first usage it was good, but on 
second one there was a need to create tempopray directory and files in 
it a given place. (It needed for a test).


And I found that TemporaryDirectory() is not able to do this. So my idea 
is to implement this behaviour with an addittional path parameter in it.


You can pass the dir argument to TemporaryDirectory().

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Serhiy Storchaka]

20.07.17 04:35, Alexander Belopolsky пише:

On Wed, Jul 19, 2017 at 9:08 PM, Guido van Rossum  wrote:

The proposal in your email seems incomplete


The proposal does not say anything about type((x=1, y=2)).  I assume
it will be the same as the type currently returned by namedtuple(?, 'x
y'), but will these types be cached? Will type((x=1, y=2)) is
type((x=3, y=4)) be True?.


Yes, this is the key problem with this idea.

If the type of every namedtuple literal is unique, this is a waste of 
memory and CPU time. Creating a new type is much more slower than 
instantiating it, even without compiling. If support the global cache of 
types, we have problems with mutability and life time. If types are 
mutable (namedtuple classes are), setting the __doc__ or __name__ 
attributes of type((x=1, y=2)) will affect type((x=3, y=4)). How to 
create two different named tuple types with different names and 
docstrings? In Python 2 all types are immortal, in python 3 they can be 
collected as ordinary objects, and you can create types dynamically 
without a fear of spent too much memory. If types are cached, we should 
take care about collecting unused types, this will significantly 
complicate the implementation.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Tighten up the formal grammar and parsing a bit?

[Serhiy Storchaka]

On 15.05.17 16:00, Steven D'Aprano wrote:

There's also cases where

if x > y:
pass
else:
code

is *not necessarily* the same as

if not (x > y):
code


This is not true.

if not cond:
stmt1
else:
stmt2

always is equivalent to

if cond:
stmt2
else:
stmt1


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Tighten up the formal grammar and parsing a bit?

[Serhiy Storchaka]

On 15.05.17 18:46, Ryan Gonzalez wrote:

I guess maybe if you overload the operators to return broken objects,
maybe then they would be different?


No. The compiler generates an equivalent bytecode for both cases.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Serhiy Storchaka]

26.06.17 13:47, Joao S. O. Bueno пише:
On 25 June 2017 at 20:55, Danilo J. S. Bellini > wrote:


On Sun, Jun 25, 2017 at 3:06 PM, lucas via Python-ideas
>wrote:

I often use generators, and itertools.chain on them.
What about providing something like the following:

 a = (n for n in range(2))
 b = (n for n in range(2, 4))
 tuple(a + b)  # -> 0 1 2 3


You know you can do `tuple(*a, *b)` , right?

The problem with the "*" notation is that it will actually render the 
iterable

contents eagerly - unlike something that would just chain them.
But for creating tuples, it just works.


Even the tuple constructor is not needed.

>>> *a, *b
(0, 1, 2, 3)

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Serhiy Storchaka]

25.06.17 14:58, Markus Meskanen пише:
I'm a huge fan of the do...while loop in other languages, and it would 
often be useful in Python too, when doing stuff like:


while True:
 password = input()
 if password == ...:
 break


In this particular case you could write:

for password in iter(input, secret_password):
...

In more complex cases you can either write more complex generator or 
just use conditional break. There is nothing wrong with this.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Serhiy Storchaka]

25.06.17 15:06, lucas via Python-ideas пише:

I often use generators, and itertools.chain on them.
What about providing something like the following:

 a = (n for n in range(2))
 b = (n for n in range(2, 4))
 tuple(a + b)  # -> 0 1 2 3

This, from user point of view, is just as how the
__add__ operator works on lists and tuples.
Making generators works the same way could be a great way to avoid calls
to itertools.chain everywhere, and to limits the differences between
generators and other "linear" collections.

I do not know exactly how to implement that (i'm not that good at C, nor
CPython source itself), but by seeing the sources,
i imagine that i could do something like the list_concat function at
Objects/listobject.c:473, but in the Objects/genobject.c file,
where instead of copying elements i'm creating and initializing a new
chainobject as described at Modules/itertoolsmodule.c:1792.

(In pure python, the implementation would be something like `def
__add__(self, othr): return itertools.chain(self, othr)`)


It would be weird if the addition is only supported for instances of the 
generator class, but not for other iterators. Why (n for n in range(2)) 
+ (n for n in range(2, 4)) works, but iter(range(2)) + iter(range(2, 4)) 
and iter([0, 1]) + iter((2, 3)) don't? itertools.chain() supports 
arbitrary iterators. Therefore you will need to implement the __add__ 
method for *all* iterators in the world.


However itertools.chain() accepts not just *iterators*. It works with 
*iterables*. Therefore you will need to implement the __add__ method 
also for all iterables in the world. But __add__ already is implemented 
for list and tuple, and many other sequences, and your definition 
conflicts with this.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] tweaking the file system path protocol

[Serhiy Storchaka]

23.05.17 20:04, Brett Cannon пише:
On Tue, 23 May 2017 at 03:13 Wolfgang Maier 
> wrote:

My proposal is to change this to:
1) check whether the type of the argument is str or bytes *exactly*; if
so, return the argument unchanged
2) check wether __fspath__ can be called on the type and returns an
instance of str, bytes, or any subclass (just like in the current
version)
3) check whether the type is a subclass of str or bytes and, if so,
return it unchanged

What exactly is the performance issue you are having that is leading to 
this proposal?


It seems to me that the purpose of this proposition is not performance, 
but the possibility to use __fspath__ in str or bytes subclasses. 
Currently defining __fspath__ in str or bytes subclasses doesn't have 
any effect.


I don't know a reasonable use case for this feature. The __fspath__ 
method of str or bytes subclasses returning something not equivalent to 
self looks confusing to me.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] tweaking the file system path protocol

[Serhiy Storchaka]

29.05.17 00:33, Wolfgang Maier пише:
The path protocol does 
*not* use __fspath__ as an indicator that an object's str-representation 
is intended to be used as a path. If you had wanted this, the PEP should 
have defined __fspath__ not as a method, but as a flag and have the 
protocol check that flag, then call __str__ if appropriate.


__fspath__ is a method because there is a need to support bytes paths. 
__fspath__() can return a bytes object, str() can't.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] π = math.pi

[Serhiy Storchaka]

03.06.17 03:31, Guido van Rossum пише:
OK, I think this discussion is pretty much dead then. We definitely 
shouldn't allow math operators in identifiers, otherwise in Python 4 or 
5 we couldn't introduce them as operators.


Sorry. I proposed this idea as a joke. math.π is useless, but mostly 
harmless. But I don't want to change Python grammar.


The rule for Python identifiers already is not easy, there is no simple 
regular expression for them, and I'm sure most tools proceeding Python 
sources (even the tokenize module and IDLE) do not handle all Python 
identifier correctly. For example they don't recognize the symbol ℘ 
(U+2118, SCRIPT CAPITAL P) as a valid identifier.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] π = math.pi

[Serhiy Storchaka]

03.06.17 20:45, Brett Cannon пише:
Since this regularly comes up, why don't we add a note to the math 
module that you can do the above import(s) to bind various mathematical 
constants to their traditional symbol counterparts? The note can even 
start off with something like "While Python's standard library only uses 
ASCII characters to maximize ease of use and contribution, individuals 
are allowed to use various Unicode characters for variable names." This 
would also help with making sure people don't come back later and say, 
"why don't you just add the constants to the module?"


Nice idea!

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] π = math.pi

[Serhiy Storchaka]

01.06.17 10:03, David Mertz пише:
It's awfully easy to add in your own code. Since they are simply 
aliases, I don't see why bother put the duplication in the standard 
library. You can even monkey patch if you want it in the 'math' 
namespace. I'd prefer a bare 'π' in my own code though.


As well as adding tau = 2*math.pi in your own code. But this deserved 
the whole PEP and was added as a feature in 3.6.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] π = math.pi

[Serhiy Storchaka]

What you are think about adding Unicode aliases for some mathematic 
names in the math module? ;-)


math.π = math.pi
math.τ = math.tau
math.Γ = math.gamma
math.ℯ = math.e

Unfortunately we can't use ∞, ∑ and √ as identifiers. :-(

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] π = math.pi

[Serhiy Storchaka]

01.06.17 10:53, Stephan Houben пише:

Tau was kind of a joke.


math.π is a kind of joke too.

Honest, it is strange, that Python allows Unicode identifiers, but 
doesn't have the one in its stdlib!


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] π = math.pi

[Serhiy Storchaka]

01.06.17 12:49, Victor Stinner пише:

2017-06-01 8:47 GMT+02:00 Serhiy Storchaka <storch...@gmail.com>:

What you are think about adding Unicode aliases for some mathematic names in
the math module? ;-)

math.π = math.pi


How do you write π (pi) with a keyboard on Windows, Linux or macOS?


This shouldn't be a problem for Greek users. ;-)

Personally I copied them from a character table.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] π = math.pi

[Serhiy Storchaka]

01.06.17 12:32, Stephan Houben пише:

math.π is a kind of joke too.


The point is that tau, being a joke, should not be considered as
setting a precedent.


If add one joke feature per release, this one looks enough harmless.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] π = math.pi

[Serhiy Storchaka]

07.06.17 08:34, Greg Ewing пише:

Mikhail V wrote:

I find actually symbols ≤ ≥ (inclusive comparison) nice.


Yes, there are a few symbols it would be nice to have.
A proper ≠ symbol would have avoided the wars between
<> and !=. :-)


But this would start the war between ≤ and ⩽ (symbols used by 
mathematicians of different countries for less-or-equal).


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Run length encoding

[Serhiy Storchaka]

11.06.17 05:20, Neal Fultz пише:
I am very new to this, but on a different  forum and after a couple 
conversations, I really wished Python came with run-length encoding 
built-in; after all, it ships with zip, which is much more complicated :)


The general idea is to be able to go back and forth between two 
representations of a sequence:


|[1,1,1,1,2,3,4,4,3,3,3]|

and

|[(1, 4), (2, 1), (3, 1), (4, 2), (3, 3)]|

where the first element is the data element, and the second is how many 
times it is repeated.


I wrote an encoder/decoder in about 20 lines ( 
https://github.com/nfultz/rle.py/blob/master/rle.py ) and would like to 
offer it for the next version; I think it might fit in nicely in the 
itertools module, for example. I am curious about your thoughts.


RLE is just a general idea. Concrete implementations in file formats and 
protocols have different limitations and peculiarities. Different 
schemes are used for encoding lengths and values, short repetition 
sequences usually are not encoded with RLE, as well as repetition 
sequences of specific values, there are limitations on the maximal 
length. The implementation of the general idea is simple, but is not 
helpful in concrete cases.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Run length encoding

[Serhiy Storchaka]

11.06.17 09:17, Neal Fultz пише:
   * other people have been false positive and wanted a SQL-type group 
by, but got burned

* hence the warnings in the docs.


This wouldn't help if people don't read the docs.

Also, if someone rewrote zip in pure python, would many people actually 
notice a slow down vs network latency, disk IO,  etc?


Definitely yes.


RLE is a building block just like bisect.


This is very specific building block. And if ZIP compression be rewrote 
in pure Python it wouldn't use


FYI, there are multiple compression methods supported in ZIP files, but 
the zipmodule module implements not all of them. In particular simple 
RLE based methods are not implemented (they almost not used in real 
world now). I suppose that if the zipmodule module implements these 
algorithms it wouldn't use any general RLE implementation.


https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Add shutil.get_dir_size

[Serhiy Storchaka]

On 02.05.17 22:07, Ram Rachum wrote:

I have a suggestion: Add a function shutil.get_dir_size that gets the
size of a directory, including all the items inside it recursively. I
currently need this functionality and it looks like I'll have to write
my own function for it.


The comprehensive implementation should take into account hard links, 
mount points, variable block sizes, sparse files, transparent files and 
blocks compression, file tails packing, blocks deduplication, additional 
file streams, file versioning, and many many other FS specific features. 
If you implement a module providing this feature, publish it on PyPI and 
prove its usefulness for common Python user, it may be considered for 
inclusion in the Python standard library.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add shutil.ignore_patterns() to shutil.rmtree()

[Serhiy Storchaka]

On 05.05.17 10:58, George Fischhof wrote:

I have a task to synchronize folders but some files should be remained
untouched.
I think this is a very common task.

I found that shutil.copytree() has ignore_patterns() but rmtree() has not.

So here comes my idea: add ignore_patterns() to rmtree() it is a good
feature and makes the functions symmetric.


You can't remove a tree when remain its branches. Thus rmtree() with 
ignore_patterns will always fail.


For your particular case I suggest to use os.walk(). You can implement 
arbitrary application specific conditions.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] LOAD_NAME/LOAD_GLOBAL should be use getattr()

[Serhiy Storchaka]

13.09.17 23:07, Lucas Wiman пише:
On Wed, Sep 13, 2017 at 11:55 AM, Serhiy Storchaka 
<storch...@gmail.com 
<mailto:storch...@gmail.com>> wrote:


[...] Calling __getattr__() will slow down the access to builtins.
And there is a recursion problem if module's __getattr__() uses
builtins.


  The first point is totally valid, but the recursion problem doesn't 
seem like a strong argument. There are already lots of recursion 
problems when defining custom __getattr__ or __getattribute__ methods, 
but on balance they're a very useful part of the language.


In normal classes we have the recursion problem in __getattr__() only 
with accessing instance attributes. Builtins (like isinstance, getattr, 
AttributeError) can be used without problems. In module's __getattr__() 
all this is a problem.


Module attribute access can be implicit. For example comparing a string 
with a byte object in __getattr__() can trigger the lookup of 
__warningregistry__ and the infinity recursion.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] LOAD_NAME/LOAD_GLOBAL should be use getattr()

[Serhiy Storchaka]

12.09.17 19:17, Neil Schemenauer пише:

This is my idea of making module properties work.  It is necessary
for various lazy-loading module ideas and it cleans up the language
IMHO.  I think it may be possible to do it with minimal backwards
compatibility problems and performance regression.

To me, the main issue with module properties (or module __getattr__)
is that you introduce another level of indirection on global
variable access.  Anywhere the module.__dict__ is used as the
globals for code execution, changing LOAD_NAME/LOAD_GLOBAL to have
another level of indirection is necessary.  That seems inescapable.

Introducing another special feature of modules to make this work is
not the solution, IMHO.  We should make module namespaces be more
like instance namespaces.  We already have a mechanism and it is
getattr on objects.


There is a difference between module namespaces and instance namespaces. 
LOAD_NAME/LOAD_GLOBAL fall back to builtins if the name is not found in 
the globals dictionary. Calling __getattr__() will slow down the access 
to builtins. And there is a recursion problem if module's __getattr__() 
uses builtins.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Hexadecimal floating literals

[Serhiy Storchaka]

21.09.17 18:23, Victor Stinner пише:

My vote is now -1 on extending the Python syntax to add hexadecimal
floating literals.

While I was first in favor of extending the Python syntax, I changed
my mind. Float constants written in hexadecimal is a (very?) rare use
case, and there is already float.fromhex() available.

A new syntax is something more to learn when you learn Python. Is it
worth it? I don't think so. Very few people need to write hexadecimal
constants in their code.


Initially I was between -0 and +0. The cost of implementing this feature 
is not zero, but it looked harmless (while almost useless). But after 
reading the discussion (in particular the comments of proponents) I'm 
closer to -1.


This feature can be useful for very few people. And they already have 
float.fromhex(). Taking to account the nature of Python the arguments 
for literals are weaker than in case of statically compiled languages. 
For the rest of users it rather adds confusion and misunderstanding. And 
don't forgot about non-zero cost. You will be impressed by the number of 
places just in the CPython core and stdlib that should be updated for 
supporting a new type of literals.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add pop_callback to deque signature

[Serhiy Storchaka]

04.10.17 07:58, Nick Coghlan пише:

For deque specifically, I like Steven D'Aprano's suggestion of a
"__dropped__" or "__discard__" subclassing API that makes it
straightforward to change the way that queue overruns are handled
(especially if raising an exception from the new subclass method can
prevent the collection modification entirely - that way you could
readily change the deque semantics in a subclass such that if the
queue fills up, submitters start getting errors instead of silently
discarding older messages, allowing backpressure to be more easily
propagated through a system of queues).


Wouldn't this harm performance? Looking up the attribute of the type is 
more costly than pushing/popping the item in the deque.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Membership of infinite iterators

[Serhiy Storchaka]

17.10.17 09:42, Nick Coghlan пише:
On 17 October 2017 at 16:32, Nick Coghlan 
> wrote:


So this sounds like a reasonable API UX improvement to me, but you'd
need to ensure that you don't inadvertently change the external
behaviour of *successful* containment tests.


I should also note that there's another option here beyond just 
returning "False": it would also be reasonable to raise an exception 
like "RuntimeError('Attempted negative containment check on infinite 
iterator')".


What about other operations with infinite iterators? min(count()), 
max(count()), all(count(1))? Do you want to implement special cases for 
all of them?


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Membership of infinite iterators

[Serhiy Storchaka]

17.10.17 17:06, Nick Coghlan пише:
Keep in mind we're not talking about a regular loop you can break out of 
with Ctrl-C here - we're talking about a tight loop inside the 
interpreter internals that leads to having to kill the whole host 
process just to get out of it.


And this is the root of the issue. Just let more tight loops be 
interruptible with Ctrl-C, and this will fix the more general issue.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add time.time_ns(): system clock with nanosecond resolution

[Serhiy Storchaka]

13.10.17 17:12, Victor Stinner пише:

I would like to add new functions to return time as a number of
nanosecond (Python int), especially time.time_ns().

It would enhance the time.time() clock resolution. In my experience,
it decreases the minimum non-zero delta between two clock by 3 times,
new "ns" clock versus current clock: 84 ns (2.8x better) vs 239 ns on
Linux, and 318 us (2.8x better) vs 894 us on Windows, measured in
Python.

The question of this long email is if it's worth it to add more "_ns"
time functions than just time.time_ns()?

I would like to add:

* time.time_ns()
* time.monotonic_ns()
* time.perf_counter_ns()
* time.clock_gettime_ns()
* time.clock_settime_ns()

time(), monotonic() and perf_counter() clocks are the 3 most common
clocks and users use them to get the best available clock resolution.
clock_gettime/settime() are the generic UNIX API to access these
clocks and so should also be enhanced to get nanosecond resolution.


I don't like the idea of adding a parallel set of functions.

In the list of alternatives in PEP 410 there is no an idea about fixed 
precision float type with nanoseconds precision. It can be implemented 
internally as a 64-bit integer, but provide all methods required for 
float-compatible number. It would be simpler and faster than general 
Decimal.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Hexadecimal floating literals

[Serhiy Storchaka]

The support of hexadecimal floating literals (like 0xC.68p+2) is 
included in just released C++17 standard. Seems this becomes a mainstream.


In Python float.hex() returns hexadecimal string representation. Is it a 
time to add more support of hexadecimal floating literals? Accept them 
in float constructor and in Python parser? And maybe add support of 
hexadecimal formatting ('%x' and '{:x}')?


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] allow overriding files used for the input builtin

[Serhiy Storchaka]

29.09.17 08:53, Wren Turkal пише:

This is meant to turn code like the following:

orig_stdin = sys.stdin

orig_stdout = sys.stdout

with open('/dev/tty', 'r+') as f:

     sys.stdin = f

     sys.stdout = f

     name = input('Name? ')

sys.stdin = orig_stdin

sys.stdout = orig_stdout

print(name)


into something more like this:

with open('/dev/tty', 'r+') as f:

     name = input('Name? ', fin=f, fout=f)

print(name)


Why not use just the following two lines?

f.write('Name? ')
name = f.readline()

This falls to me in the category "not every two lines of the code should 
be added as a builtin".


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] tarfile.extractall progress

[Serhiy Storchaka]

01.09.17 14:50, Tarek Ziadé пише:

For large archives, I want to display a progress bar while the archive
is being extracted with:

https://docs.python.org/3/library/tarfile.html#tarfile.TarFile.extractall

I could write my own version of extractall() to do this, or maybe we
could introduce a callback option that gets called
everytime .extract() is called in extractall()

The callback can receive the tarinfo object and where it's being
extracted. This is enough to plug a progress bar
and avoid reinventing .extractall()


This is not enough if extract large files. In that case you perhaps want 
to update the progress bar more often.


If add this feature to tarfile, it may be worth to add it to zipfile and 
shutil functions (copytree, rmtree). And if call a callback for every 
extracted/copied entity, it may be worth to use its result for filtering.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Membership of infinite iterators

[Serhiy Storchaka]

18.10.17 13:22, Nick Coghlan пише:
2.. These particular cases can be addressed locally using existing 
protocols, so the chances of negative side effects are low


Only the particular case `count() in count()` can be addressed without 
breaking the following examples:


>>> class C:
... def __init__(self, count):
... self.count = count
... def __eq__(self, other):
... print(self.count, other)
... if not self.count:
... return True
... self.count -= 1
... return False
...
>>> import itertools
>>> C(5) in itertools.count()
5 0
4 1
3 2
2 3
1 4
0 5
True
>>> it = itertools.cycle([C(5)]); it in it
5 
4 
3 
2 
1 
0 
True
>>> it = itertools.repeat(C(5)); it in it
5 repeat(<__main__.C object at 0x7f65512c5dc0>)
4 repeat(<__main__.C object at 0x7f65512c5dc0>)
3 repeat(<__main__.C object at 0x7f65512c5dc0>)
2 repeat(<__main__.C object at 0x7f65512c5dc0>)
1 repeat(<__main__.C object at 0x7f65512c5dc0>)
0 repeat(<__main__.C object at 0x7f65512c5dc0>)
True

3. The total amount of code involved is likely to be small (a dozen or 
so lines of C, a similar number of lines of Python in the tests) in 
well-isolated protocol functions, so the chances of introducing future 
maintainability problems are low


It depends on what you want to achieve. Just prevent an infinity loop in 
`count() in count()`, or optimize `int in count()`, or optimize more 
special cases.


4. We have a potential contributor who is presumably offering to do the 
work (if that's not the case, then the question is moot anyway until a 
sufficiently interested volunteer turns up)


Maintaining is more than writing an initial code.

If we were to do that, then we *could* make the solution to the reported 
problem more general by having all builtin and standard library 
operations that expect to be working with finite iterators (the 
containment testing fallback, min, max, sum, any, all, functools.reduce, 
etc) check for a length hint, even if they aren't actually 
pre-allocating any memory.


This will add a significant overhead for relatively short (hundreds of 
items) sequences. I already did benchmarking for similar cases in the past.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Membership of infinite iterators

[Serhiy Storchaka]

18.10.17 17:48, Nick Coghlan пише:
1. It will make those loops slower, due to the extra overhead of 
checking for signals (even the opcode eval loop includes all sorts of 
tricks to avoid actually checking for new signals, since doing so is 
relatively slow)
2. It will make those loops harder to maintain, since the high cost of 
checking for signals means the existing flat loops will need to be 
replaced with nested ones to reduce the per-iteration cost of the more 
expensive checks
3. It means making the signal checking even harder to reason about than 
it already is, since even C implemented methods that avoid invoking 
arbitrary Python code could now still end up checking for signals


I have implemented signals checking for itertools iterators. [1] The 
overhead is insignificant because signals are checked only for every 
0x1-th item (100-4000 times/sec). The consuming loops are not 
changed because signals are checked on the producer's side.


[1] https://bugs.python.org/issue31815

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Membership of infinite iterators

[Serhiy Storchaka]

18.10.17 22:21, Koos Zevenhoven пише:
​Nice! Though I'd really like a general ​solution that other code can 
easily adopt, even third-party extension libraries.


What is the more general solution? For interrupting C code you need to 
check signals manually, either in every loop, or in every iterator. It 
seems to me that the number of loops is larger than the number of iterators.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

[Serhiy Storchaka]

28.11.17 15:36, Nick Coghlan пише:

* I'm opposed to making assert substantially different from the way it works now


If sometimes we will change assert I would prefer to make it more 
similar to assert in pytest. Output values of the final and intermediate 
expressions. The hardest problem -- the repr can be very long and 
expensive, we need an alternate protocol for producing a shortened 
representation.



* I'm in favour of adding a new "ensure()" builtin that encapsulates
the check-and-raise logic


Then may be add a new "die()" buildin? ;-)

0 <= r < abs(y) or die()

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Logging Levels

[Serhiy Storchaka]

28.11.17 21:45, Guido van Rossum пише:
These look like good improvements. I think you should make an issue on 
bugs.python.org  describing your proposal and if 
you can submit a PR that implements it.


See https://bugs.python.org/issue31732

It was discussed and rejected. Citing Raymond: "Overall, this seems 
rehash and second guess the discussions and decisions made 15 years ago 
when PEP 282 was accepted."


The set of logging levels is not closed. The user can extend it to cover 
more specialized uses by logging.addLevelName(). There are disadvantages 
of having too much standard names for logging levels (as we can see in 
Java).


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: allow length_hint to specify infinite iterators

[Serhiy Storchaka]

29.11.17 03:34, Steven D'Aprano пише:

This wastes the opportunity to fail fast on operations which cannot
possibly succeed, e.g. list(count()) must eventually fail with
MemoryError. Or worse: if the OS starts thrashing trying to meet the
memory requests, you can lock up the computer.

I propose that we:

(1) extend the __length_hint__ protocol to allow iterators to report
that they are infinite;

(2) and recommend that consumers of iterators (such as list) that
require finite input should fail fast in the event of an infinite
iterator.


Infinite iterators are rare. And count() is even more special in the 
sense that iterating it doesn't have side effects and is not interruptible.



(c) take the convention that returning sys.maxint means infinity;


Returning sys.maxsize will likely lead to failing fast with MemoryError.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

[Serhiy Storchaka]

29.11.17 08:08, Steven D'Aprano пише:

Perl is hardly the only language with null-coalescing operators -- we
might better describe ?? as being familiar to C#, PHP, Swift and Dart.
That's two mature, well-known languages and two up-and-coming languages.


What is the syntax of the ternary operator in these languages?

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add a dict with the attribute access capability

[Serhiy Storchaka]

In 3.7 I have removed an old-deprecated plistlib.Dict. [1] Actually it 
already was deprecated when the plistlib module was added to the regular 
stdlib in Python 2.6.


This is a dict subclass which allows to access items as attributes.

d = plistlib.Dict()
d['a'] = 1
assert d.a == 1
d.b = 2
assert d['b'] == 2

Raymond noticed that that capability seemed nice to have.

What do you think about reviving this type as general purpose type in 
collections or types? Perhaps it can be convenient for working with 
JSON, plists, configuration files, databases and in other cases that 
need a dict with string keys.


If reintroduce it, there are open questions.

1. The name of the type.

2. The location of the type. collections or types? Or other variants?

3. How it will collaborate with OrderedDict, defaultdict, etc?

4. Should it be a dict subclass, or a mixin, or a proxy? Or add several 
types?



[1] https://bugs.python.org/issue29196

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

[Serhiy Storchaka]

29.11.17 15:01, Stephan Houben пише:

What about a more general:

A if  B else C

which would allow

A if is not None else C

but also e.g.

A if >= 1 else 0


This look the most "natural" to me. I.e. the least "unnatural". If we 
even will introduce a new special syntax I will prefer this syntax.


The only disadvantage is that this will prevent introducing "angular 
parenthesis" in future: A if  == B else C.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

[Serhiy Storchaka]

29.11.17 14:39, Nick Coghlan пише:

 "a if def else b" -> pronounced "a if defined, else b"


I understand "a if defined, else b" as

try:
result = a
except (NameError, AttributeError, LookupError):
result = b

The problem is that None is not undefined. This is a regular value. 
Seems it is not so special in Python as NULL or undef in other 
languages. Python even don't have a special purposed 
NullPointerException (or NoneError).


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

[Serhiy Storchaka]

29.11.17 11:45, Steven D'Aprano пише:

On Wed, Nov 29, 2017 at 09:14:12AM +0200, Serhiy Storchaka wrote:

29.11.17 08:08, Steven D'Aprano пише:

Perl is hardly the only language with null-coalescing operators -- we
might better describe ?? as being familiar to C#, PHP, Swift and Dart.
That's two mature, well-known languages and two up-and-coming languages.


What is the syntax of the ternary operator in these languages?


All four use:

 condition ? first : second

for the ternary if operator.


If all four use ?, it is natural that in operators which are shortcuts 
of the ternary operator they use ?. But in Python the bar of introducing 
? is higher.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] How assignment should work with generators?

[Serhiy Storchaka]

27.11.17 23:24, Guido van Rossum пише:
Is this problem really important enough that it requires dedicated 
syntax? Isn't the itertools-based solution good enough? (Or failing 
that, couldn't we add something to itertools to make it more readable 
rather than going straight to new syntax?)


I want to remind PEP 204 and PEP 212. The special purposed syntaxes were 
proposed to help solving much more common problems. They were rejected 
in favor of builtins range() and enumerate(). And we are happy with 
these builtins.


The function for solving this problem already exists. It's 
itertools.islice(). It isn't builtin, but this problem is much less 
common than use cases for range() and enumerate().


If we don't have special non-function syntax for range(), enumerate(), 
zip(), itertools.chain(), itertools.count(), itertools.repeat(), etc, I 
don't think we should have a special syntax for one particular case of 
using itertools.islice().


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repr of lambda

[Serhiy Storchaka]

17.12.17 22:55, Terry Reedy пише:
What if include the signature and the expression of the lambda in its 
repr?


 >>> lambda x: x**2



Printing the return value requires adding a code or function attribute.


Yes, this requires adding an optional constant code attribute.

The return expression(s), possibly None, would be just as useful for 
named functions.


In case of named functions the body is not an expression and usually is 
multiline.


But there is a problem with default values. Their reprs can be very 
long, especially in the following case with mutable default value:


I would not expect the representation to change; just use the expression.


This is a solution, thanks.


Maybe the placeholder should be always used instead of default values.


Do you mean 'a placeholder'?


Yes, sorry.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Repr of lambda

[Serhiy Storchaka]

Currently repr of doesn't contain much of information besides that it is 
a lambda.


>>> lambda x: x**2
 at 0x7f3479b74488>

All lambdas have the same repr, different only by unreadable hexadecimal 
address.


What if include the signature and the expression of the lambda in its repr?

>>> lambda x: x**2


This would return an old feature of Python 0.9.1 
(https://twitter.com/dabeaz/status/934835068043956224).


Repr of function could contain just the signature.



But there is a problem with default values. Their reprs can be very 
long, especially in the following case with mutable default value:


def foo(x, _cache={}):
try:
return _cache[x]
except KeyError:
pass
_cache[x] = y = expensive_calculation(x)
return y

Maybe the placeholder should be always used instead of default values.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Ignorable whitespaces in the re.VERBOSE mode

[Serhiy Storchaka]

17.11.17 00:09, MRAB пише:

On 2017-11-16 21:44, Serhiy Storchaka wrote:

16.11.17 19:38, Guido van Rossum пише:
Who would benefit from changing this? Let's not change things just 
because we can, or because Perl 6 does it.


I don't know. I know the disadvantages of making this change, and I ask
what is the benefit. If there is a benefit, and it is important for
Python, I could implement this feature in re and regex.

You could see what some more languages, e.g. C#, do. If there isn't a 
consensus of some kind, it's best to leave it.


I haven't found this in the documentation, but according to the sources 
it uses only 5 ASCII whitespaces (exluding \v).


Java uses 6 ASCII whitespaces.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 562

[Serhiy Storchaka]

10.09.17 21:48, Ivan Levkivskyi пише:

   # lib.py

   from warnings import warn

   deprecated_names = ["old_function", ...]

   def _deprecated_old_function(arg, other):
       ...

   def __getattr__(name):
       if name in deprecated_names:
           warn(f"{name} is deprecated", DeprecationWarning)
           return globals()[f"_deprecated_{name}"]
       raise AttributeError(f"module {__name__} has no attribute {name}")

   # main.py

   from lib import old_function  # Works, but emits the warning


I think the PEP should provide better examples, because they will be 
copied in third-party code.


For keeping all functionality (in particularly be pickleable) the 
deprecated function should preserve its name.


   def old_function(arg, other):
   ...
   _deprecated_old_function = old_function
   del old_function

or

   def _deprecated_old_function(arg, other):
   ...
   _deprecated_old_function.__name__ = 'old_function'
   _deprecated_old_function.__qualname__ = 'old_function'

(I prefer the former variant.)

I'm wondering if it is worth to provide a special helper that will 
rename the deprecated function and create the corresponding __getattr__ 
function.


It could be possible to create helpers that implement module-level 
properties too.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Serhiy Storchaka]

21.11.17 05:16, Mikhail V пише:

my·variable
myᝍvariable
myㅡvariable
myⵧvariable
myㄧvariable
myㆍvariable

^ Is this good idea *for Python*? I mean this is not Python that
I knew. I don't know how it is possible. Looks like a result of
some unlucky nuclear experment. Might be it will not cause any possible
confusion, or less than a hyphen and a minus.


Yes, it causes less confusion that changing meaning of a minus.

And yes, it can cause confusion if misused. As well as using the 
following variables:


мyvariable
mуvariable
myvаriable
myvarіable
myvariаble
myvariaЬle
myvariab1e
myvariablе

But the name моязмінна doesn't cause any confusion if used in an 
appropriate context (for example in a lesson for young Ukrainian 
children). I believe the above dot- and hyphen-like characters don't 
cause confusion if used as letters in an appropriate language context.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding a new function "zip_flat" to itertools (Re: Rewriting the "roundrobin" recipe in the itertools documentation)

[Serhiy Storchaka]

21.11.17 01:00, Terry Reedy пише:

On 11/20/2017 4:57 PM, Steven D'Aprano wrote:

Is there a reason for calling reversed() instead of reversing the order
of range in the first place?

range(len(iterables)-1, -1, -1)


Readability.  Accurately and automatically reversing ranges was one of 
the use cases motivating the addition of 'reversed'.  Ranges have a 
__reversed__ method which returns a iterator for a reversed range.


Agree. You also can write this as range(1, len(iterables))[::-1]. This 
is shorter and (I suppose) slightly faster. But reverse() can be better 
choice for a recipe, if it looks less confusing for beginners.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Should Python have user-defined constants?

[Serhiy Storchaka]

21.11.17 08:33, Saeed Baig пише:
Hey guys I am thinking of perhaps writing a PEP to introduce 
user-defined constants to Python. Something along the lines of Swift’s 
“let” syntax (e.g. “let pi = 3.14”).


Do you guys think it would be a good idea? Why or why not? Do you think 
there’s a better way to do it?


To do what? What problem do you need to solve?

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Ignorable whitespaces in the re.VERBOSE mode

[Serhiy Storchaka]

21.11.17 04:20, Stephen J. Turnbull пише:

Serhiy Storchaka writes:
  > I agree. But if there is a special part of the Unicode standard for
  > Pattern White Spaces which includes non-ASCII characters, perhaps there
  > is a need in them. I asked for the case if Python developers with very
  > different cultures have need in additional whitespaces in regular
  > expressions, but I don't know. Seems nobody has claimed their need.

I doubt that Japanese would want it.  I do use \N{IDEOGRAPHIC SPACE} a
bit as a *target* of regular expressions, but I would never want it as
non-syntactic in re.VERBOSE.  (Of course, I'm not a native Japanese, but
I have never heard a Japanese developer wish for use of that character
in any programming language, outside of literal strings.)

  > In particularly I don't know how helpful would be supporting
  > right-to-left and left-to-right marks in verbose regular expressions

That's a good question.  Interpretation and display of R2L in
programming constructs came up briefly in the discussions about BIDI
on the emacs-devel list.  I'll ask Eli Zaretskii, who implemented it
for Emacs.


Thank you Stephen. I would prefer to not change anything (because 
supporting additional whitespaces will complicate and slow down the 
code, and can add subtle bugs, add likely will add a confusion for 
users). But I want to know whether there is a real need in supporting 
additional whitespaces and rtl and ltr marks in regular expressions and 
Python syntax.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Star assignment in iterator way?

[Serhiy Storchaka]

21.11.17 10:54, Kirill Balunov пише:

Of course this version has drawbacks (the first that come to mind):
1. Will *b see change if rhs is some muttable sequence?
2. Will *b one way iterator or somethong like range?

But still it seems to me that the "iterator way" has more useful 
applications.


Your implementation iterates seq multiple times. But iterable unpacking 
syntax works with an arbitrary iterable, and iterates it only once.


Changing the result of iterable unpacking will break existing code that 
depends on the result been a list.


And you already have mentioned a question about mutable sequence.

If these conditions and restrictions suit you, you can use your 
good_star_exp() in your code or share it with others. But the semantic 
of iterable unpacking can't be changed.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Rewriting the "roundrobin" recipe in the itertools documentation

[Serhiy Storchaka]

The roundrobin() implementation in recipes has quadratic time for large 
number of iterables. As well as all other proposed implementations. This 
is a problem if you use it with hundreds or thousands of iterables. For 
example:


list(roundrobin(*([[1]]*1000)))
next(roundrobin(*([[]]*1000 + [[1]]])))

The following implementation has linear time.

def roundrobin(*iterables):
"roundrobin('ABC', 'D', 'EF') --> A D E B F C"
nexts = [iter(it).__next__ for it in iterables]
while nexts:
next_nexts = []
append = next_nexts.append
for next in nexts:
try:
yield next()
except StopIteration:
pass
else:
append(next)
nexts = next_nexts

Actually it expands cycle() in Python. And this makes it slower for 
smaller number of iterations.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Star assignment in iterator way?

[Serhiy Storchaka]

21.11.17 11:27, Kirill Balunov пише:

Your implementation iterates seq multiple times. But iterable
unpacking syntax works with an arbitrary iterable, and iterates it
only once.


Oh sorry, I know that my implementation iterates seq multiple times, I 
only provide this to show the idea. It can be much optimized at C level. 
I just want to understand if it's worth the time and effort.


You can implement the first case, but for other cases you will need a 
storage for saving intermediate items. And using a list is a good option.



And you already have mentioned a question about mutable sequence.

If these conditions and restrictions suit you, you can use your
good_star_exp() in your code or share it with others. But the
semantic of iterable unpacking can't be changed.


And how do you look at something like this (deferred star evaluation)?:

a, ?*b, c, d = something_iterable


This will be not different from

a, *b, c, d = something_iterable
b = iter(b)

There is nothing deferred here.

The only possible benefit can be in the case

a, b, ?*c = something_iterable

But I have doubts that this special case deserves introducing a new syntax.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Should Python have user-defined constants?

[Serhiy Storchaka]

21.11.17 11:34, Saeed Baig пише:
Serhiy asked, in relation to constants, “To do what? What problem do you 
need to solve?”. No problem in particular, to be honest. I just thought 
they’d be nice since they’d increase confidence that my 
variables-intended-to-be-constants wouldn’t get reassigned, and just to 
increase readability/explicitness to other programmers.


For increasing readability/explicitness you can use a name convention 
(UPPER_CASE), comments and documentation.


As for increasing confidence that your 
variables-intended-to-be-constants wouldn’t get reassigned, you can use 
read-only properties. This is a tricky to implement read-only properties 
for modules, but see PEP 562 which should help to do this.


Please don't use GIGANTIC font in your messages.

And please don't create a new thread when answer.

And *please* don't include unrelated text in your messages.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Should Python have user-defined constants?

[Serhiy Storchaka]

21.11.17 11:47, Paul Moore пише:

-1. I don't see how this would improve any programs I've written or
seen. Tools like mypy or linters might benefit from a feature to track
constants and ensure they don't get changed, but I don't think it's
needed as a language feature. Seriously, has anyone ever written
"math.pi = 5" and been surprised that their code broke? Or even
modified an application constant like BUFFER_LIMIT? Do you have any
evidence that this would avoid bugs in real-world code?


Actually the possibility of modifying application constants like 
BUFFER_LIMIT can be useful.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Rewriting the "roundrobin" recipe in the itertools documentation

[Serhiy Storchaka]

21.11.17 11:44, Serhiy Storchaka пише:
The roundrobin() implementation in recipes has quadratic time for large 
number of iterables. As well as all other proposed implementations. This 
is a problem if you use it with hundreds or thousands of iterables. For 
example:


     list(roundrobin(*([[1]]*1000)))
     next(roundrobin(*([[]]*1000 + [[1]]])))

The following implementation has linear time.

def roundrobin(*iterables):
     "roundrobin('ABC', 'D', 'EF') --> A D E B F C"
     nexts = [iter(it).__next__ for it in iterables]
     while nexts:
     next_nexts = []
     append = next_nexts.append
     for next in nexts:
     try:
     yield next()
     except StopIteration:
     pass
     else:
     append(next)
     nexts = next_nexts

Actually it expands cycle() in Python. And this makes it slower for 
smaller number of iterations.


Yet one natural implementation with linear time is:

from collections import deque
def roundrobin(*iterables):
"roundrobin('ABC', 'D', 'EF') --> A D E B F C"
nexts = deque(iter(it).__next__ for it in iterables)
popleft = nexts.popleft
append = nexts.append
while nexts:
next = popleft()
try:
yield next()
except StopIteration:
pass
else:
append(next)

As the previous example it doesn't have any relation to itertools.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Serhiy Storchaka]

19.11.17 04:01, Mikhail V пише:

Python allows underscore character as a separator in variables.
This is better than nothing, still it does not make the look much better.

**Proposal**: allow additional separator, namely hyphen character.


You already can use "separators" different from the underscore.

my·variable
myᝍvariable
myㅡvariable
myⵧvariable
myㄧvariable
myㆍvariable

Be lucky to distinguish one "separator" from other.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Star assignment in iterator way?

[Serhiy Storchaka]

21.11.17 13:53, Kirill Balunov пише:

If I can not copy at Python level, I can 'tee' when 'star_pos' is reached.


And tee() uses a real RAM for saving items.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Serhiy Storchaka]

21.11.17 22:03, Stephan Houben пише:

If anybody is still worried about this,
here is a 29-line proof-of-concept code checker which warns if your 
source file

contains identifiers which are different but look the same.

https://gist.github.com/stephanh42/61eceadc2890cf1b53ada5e48ef98ad1


Ha! I wanted to look at a 29-line code that is able to detect all 
homoglyphs. But actually it just imports a third-party module. :-(


In any case this is a nice code.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Ignorable whitespaces in the re.VERBOSE mode

[Serhiy Storchaka]

16.11.17 19:38, Guido van Rossum пише:
Who would benefit from changing this? Let's not change things just 
because we can, or because Perl 6 does it.


I don't know. I know the disadvantages of making this change, and I ask 
what is the benefit. If there is a benefit, and it is important for 
Python, I could implement this feature in re and regex.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Ignorable whitespaces in the re.VERBOSE mode

[Serhiy Storchaka]

Currently the re module ignores only 6 ASCII whitespaces in the 
re.VERBOSE mode:


 U+0009 CHARACTER TABULATION
 U+000A LINE FEED
 U+000B LINE TABULATION
 U+000C FORM FEED
 U+000D CARRIAGE RETURN
 U+0020 SPACE

Perl ignores characters that Unicode calls "Pattern White Space" in the 
/x mode. It ignores additional 5 non-ASCII characters.


 U+0085 NEXT LINE
 U+200E LEFT-TO-RIGHT MARK
 U+200F RIGHT-TO-LEFT MARK
 U+2028 LINE SEPARATOR
 U+2029 PARAGRAPH SEPARATOR

The regex module just ignores characters for which str.isspace() returns 
True. It ignores additional 20 non-ASCII whitespace characters, 
including characters U+001C..001F whose classification as whitespaces is 
questionable, but doesn't ignore LEFT-TO-RIGHT MARK and RIGHT-TO-LEFT MARK.


 U+001C [FILE SEPARATOR]
 U+001D [GROUP SEPARATOR]
 U+001E [RECORD SEPARATOR]
 U+001F [UNIT SEPARATOR]
 U+00A0 NO-BREAK SPACE
 U+1680 OGHAM SPACE MARK
 U+2000 EN QUAD
 U+2001 EM QUAD
 U+2002 EN SPACE
 U+2003 EM SPACE
 U+2004 THREE-PER-EM SPACE
 U+2005 FOUR-PER-EM SPACE
 U+2006 SIX-PER-EM SPACE
 U+2007 FIGURE SPACE
 U+2008 PUNCTUATION SPACE
 U+2009 THIN SPACE
 U+200A HAIR SPACE
 U+202F NARROW NO-BREAK SPACE
 U+205F MEDIUM MATHEMATICAL SPACE
 U+3000 IDEOGRAPHIC SPACE

Is it worth to extend the set of ignored whitespaces to "Pattern 
Whitespaces"? Would it add any benefit? Or add confusion? Should this 
depend on the re.ASCII mode? Should the byte b'\x85' be ignorable in 
verbose bytes patterns?


And there is a similar question about the Python parser. If Python uses 
Unicode definition for identifier, shouldn't it accept non-ASCII 
"Pattern Whitespaces" as whitespaces? There will be technical problems 
with supporting this, but are there any benefits?



https://perldoc.perl.org/perlre.html
https://www.unicode.org/reports/tr31/tr31-4.html#Pattern_Syntax
https://unicode.org/L2/L2005/05012r-pattern.html

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Modules as global namespaces rather than dicts

[Serhiy Storchaka]

14.11.17 22:34, Neil Schemenauer пише:

This is an idea I have been playing with and seems to hold some
promise.  I think we should use a module instance as the standard
global namespace rather than directly using its dict.  I have a
prototype version of CPython that does this, not working 100% yet
though.


Wouldn't this harm a performance? Especially after implementing PEP 562.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] About the efficiency of range()

[Serhiy Storchaka]

05.11.17 20:49, Juancarlo Añez пише:

I found this interesting:

https://stackoverflow.com/a/46996392 


Please explain your suggestion.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal to change Python version release cycle

[Serhiy Storchaka]

05.11.17 21:53, Stephan Houben пише:

After python39.dll I'd expect python3A.dll .


The problem will return in 3.36.


Or possibly python3⑽.dll

 >>> len("python3⑽.dll") == len("python39.dll")
True

No worries, then.


But len(os.fsencode("python3⑽.dll")) != len(os.fsencode("python39.dll")).

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal to change Python version release cycle

[Serhiy Storchaka]

04.11.17 17:29, Wolfgang пише:

A good point but two digits minor version numbers have the possibility
to break a lot code. There is a lot of stuff out where a single digit
major version is assumed. Even the official Python build for windows
with python27.dll, python36.dll can be problematic because the dot
is omitted between numbers.
Other do the same for compilation they concatenate only majorminor for a
name.
Then version 3.10 is the same as 31.0.


Actually this is yet one argument against your idea. With your proposal 
27 will be the same as 2.7.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] array.array.index() and Sequence ABC

[Serhiy Storchaka]

06.11.17 11:17, Niki Spahiev пише:
I noticed that array.index() method takes single argument in python 3.6, 
while Sequence protocol specifies 2 optional arguments after the first.


example from list:

L.index(value, [start, [stop]]) -> integer -- return first index of value.

I propose adding start and stop arguments to array.index()


Open an issue on the bug tracker and provide a patch.

See also:

https://bugs.python.org/issue28197
https://bugs.python.org/issue31942

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Any chance on (slowly) deprecating `eval` and `exec` as builtins?

[Serhiy Storchaka]

07.11.17 16:41, Steven D'Aprano пише:

On Tue, Nov 07, 2017 at 03:35:58PM +0200, Serhiy Storchaka wrote:

07.11.17 12:29, אלעזר пише:

Also, it is unfortunate that `ast.literal_eval` is less accessible than
`builtins.eval`. Giving it an alias in builtins might make it easier for
programmers (and less scary - "ast" might sound like I need a PhD to use
it).


ast.literal_eval is not so safe as you think. Malicious input can cause
a stack overflow in your program. [1]

[1] https://bugs.python.org/issue31113



I don't see anything about literal_eval in that bug report.


Sorry, this particular issue isn't related to literal_eval. There was 
other recently fixed issue, but I forgot its number.


But the point is that the compiler is recursive, and processing nested 
constructs consumes the C stack. There are some guards against too deep 
recursion (2.7 has less guards and more vulnerable), but it is hard to 
prove that all vulnerabilities are fixed.


Your method (limiting the size of the input) helps against some attacks. 
Other methods -- restricting the set of characters and the number of 
parenthesis, braces and brackets.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Any chance on (slowly) deprecating `eval` and `exec` as builtins?

[Serhiy Storchaka]

07.11.17 12:29, אלעזר пише:
Also, it is unfortunate that `ast.literal_eval` is less accessible than 
`builtins.eval`. Giving it an alias in builtins might make it easier for 
programmers (and less scary - "ast" might sound like I need a PhD to use 
it).


ast.literal_eval is not so safe as you think. Malicious input can cause 
a stack overflow in your program. [1]


[1] https://bugs.python.org/issue31113

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Is there a reason some of the PyLong_As* functions don't call an object's __int__?

[Serhiy Storchaka]

08.12.17 13:36, Antoine Pitrou пише:

On Fri, 8 Dec 2017 13:26:48 +0200
Serhiy Storchaka <storch...@gmail.com>
wrote:



Currently the following functions fall back on __int__ where available:

PyLong_AsLong
PyLong_AsLongAndOverflow
PyLong_AsLongLong
PyLong_AsLongLongAndOverflow
PyLong_AsUnsignedLongMask
PyLong_AsUnsignedLongLongMask


I think this should be deprecated (and there should be an open issue for
this). Calling __int__ is just a Python 2 legacy.


I think that's a bad idea.  There are widely-used int-like classes out
there and it will break actual code:


import numpy as np
x = np.int64(5)
isinstance(x, int)

False

x.__int__()

5


NumPy integers implement __index__.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Is there a reason some of the PyLong_As* functions don't call an object's __int__?

[Serhiy Storchaka]

08.12.17 12:41, Erik Bray пише:

IIUC, it seems to be carry-over from Python 2's PyLong API, but I
don't see an obvious reason for it.  In every case there's an explicit
PyLong_Check first anyways, so not calling __int__ doesn't help for
the common case of exact int objects; adding the fallback costs
nothing in that case.


There is also a case of int subclasses. It is expected that 
PyLong_AsLong is atomic, and calling __int__ can lead to crashes or 
similar consequences.



I ran into this because I was passing an object that implements
__int__ to the maxlen argument to deque().  On Python 2 this used
PyInt_AsSsize_t which does fall back to calling __int__, whereas
PyLong_AsSsize_t does not.


PyLong_* functions provide an interface to PyLong objects. If they don't 
return the content of a PyLong object, how can it be retrieved? If you 
want to work with general numbers you should use PyNumber_* functions.


In your particular case it is more reasonable to fallback to __index__ 
rather than __int__. Unlikely maxlen=4.2 makes sense.



Currently the following functions fall back on __int__ where available:

PyLong_AsLong
PyLong_AsLongAndOverflow
PyLong_AsLongLong
PyLong_AsLongLongAndOverflow
PyLong_AsUnsignedLongMask
PyLong_AsUnsignedLongLongMask


I think this should be deprecated (and there should be an open issue for 
this). Calling __int__ is just a Python 2 legacy.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] a sorting protocol dunder method?

[Serhiy Storchaka]

04.12.17 01:06, Chris Barker пише:

So: has this already been brought up and rejected?


https://bugs.python.org/issue20632


Am I imagining the performance benefits?


This will add an additional overhead. This will be even slower than 
passing the key function, since you will need to look up the __key__ 
method in every item. And there will be an overhead even in the case 
when the __key__ method is not defined.



Is sorting-related functionally too special-case to deserve a protocol?


Yes, it is too special-case. I don't see any advantages in comparison 
with defining the __lt__ method. It will be rather confusing if 
different methods of sorting produce inconsistent order. If the first 
item of the sorted list is not the smallest item of the list.


But the idea of the class decorator looks more sane to me.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] a sorting protocol dunder method?

[Serhiy Storchaka]

04.12.17 13:06, Antoine Pitrou пише:

On Mon, 4 Dec 2017 08:45:55 +0200
Serhiy Storchaka <storch...@gmail.com>
wrote:

04.12.17 01:06, Chris Barker пише:

So: has this already been brought up and rejected?


https://bugs.python.org/issue20632


Am I imagining the performance benefits?


This will add an additional overhead. This will be even slower than
passing the key function, since you will need to look up the __key__
method in every item.


That is a reasonable objection.  However, looking up a tp_XXX slot is
very cheap.


But introducing a new slot is not easy. This will increase the size of 
the type object, break binary compatibility. According to Stefan 
Behnel's researches (https://bugs.python.org/issue31336) the main time 
of the creation of a new type is spent on initializing slots. This cost 
will pay every Python program, even if it doesn't use the __key__ method.


There are many more common and important methods that don't have the 
corresponding slot (__length_hint__, __getstate__, __reduce__, __copy__, 
keys).



Yes, it is too special-case. I don't see any advantages in comparison
with defining the __lt__ method.


There are definitely advantages.  Sorting calls __lt__ for each
comparison (that is, O(n log n) times) while __key__ would only be
called once per item at the start (that is, O(n) times).


It will call __lt__ for each key comparison (the same O(n log n) times), 
but *in addition* it will call __key__ O(n) times. You can get the 
benefit only when times of calling __key__ is much smaller than the 
difference between times of calling item's __lt__ and key's __lt__, and 
maybe only for large lists. But why not just pass the key argument when 
you sort the large list?



It will be rather confusing if
different methods of sorting produce inconsistent order.


If __key__ is inconsistent with __lt__, it is the same error as making
__lt__ inconsistent with __gt__.


If __key__ is consistent with __lt__, then we can just use __lt__, and 
don't introduce new special methods.



And there could be a decorator that generates all comparison methods
from __key__.


The decorator idea LGTM. But it doesn't need the special __key__ method. 
Just pass the key function as a decorator argument.


This idea was proposed more than 3.5 years ago. Is there a PyPI package 
that implements it?


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] a sorting protocol dunder method?

[Serhiy Storchaka]

04.12.17 14:25, Steven D'Aprano пише:

On Mon, Dec 04, 2017 at 08:45:55AM +0200, Serhiy Storchaka wrote:

But the idea of the class decorator looks more sane to me.


The purpose of __key__ is to define a key function (not a comparison
operator) for classes that aren't orderable and don't have __lt__.

If you're going to then go ahead and define __lt__ and the other
comparison operators, there's no point to __key__.


Right. The only benefit of this decorator is that it could avoid writing 
a boilerplate code for simple cases. Just add 
@ordered_by_key(attrgetter('name', 'weight')).


__key__ is not needed, just pass the key function as an argument of the 
decorator.


Of course if it can be useful this doesn't mean that it should be 
included in the stdlib. It could live on PyPI.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Provide a way to import module without exec body

[Serhiy Storchaka]

01.12.17 20:12, Neil Schemenauer пише:

On 2017-12-01, Chris Angelico wrote:

Can you elaborate on where this is useful, please?


Introspection tools, for example, might want to look at the module
without executing it.  Also, it is a building block to make lazy loading
of modules work.  As Nick points out, importlib can do this already.

Currently, the IMPORT_NAME both loads the code for a module and also
executes it.  The exec happens fairly deep in the guts of importlib.
This makes import.c and ceval.c mutually recursive.  The locking gets
complicated.  There are hacks like _call_with_frames_removed() to hide
the recursion going on.

Instead, we could have two separate opcodes, one that gets the module
but does not exec it (i.e. a function like __import__() that returns a
future) and another opcode that actually does the execution.  Figuring
out all the details is complicated.


The IMPORT_NAME opcode is highly optimized. In most cases it just looks 
up in sys.modules and check that the module is not imported right now.
I suppose two opcodes will hit performance. And I don't see how this 
could simplify the code.


I suppose the existing importlib machinery already supports loading 
modules without executing them. Maybe not with a single function, but 
with a combination of 2-3 methods. But what you want to get? The source? 
The code object? What about modules implemented in C?


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] f-string literals by default?

[Serhiy Storchaka]

05.12.17 23:22, Joseph Jevnik пише:

This would break code that uses str.format everywhere for very little benefit.


And many regular expressions. And string.Template patterns. And 
docstrings (silently). And ast.literal_eval, shelve, doctest.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Inline assignments using "given" clauses

[Serhiy Storchaka]

04.05.18 15:06, Nick Coghlan пише:
Recapping the use cases where the inline assignment capability received 
the most agreement regarding being potentially more readable than the 
status quo:


Sorry, but these examples don't look as good examples for inline 
assignments to me. I think that all these cases can be written better 
without using the inline assignment.


1. Making an "exactly one branch is executed" construct clearer than is 
the case for nested if statements:


     if m := pattern.search(data):
     ...
     elif m := other_pattern.search(data):
     ...
     else:
     ...


This case can be better handled by combining patterns in a single 
regular expression.


pattern = re.compile('(?Ppattern1)|(?Ppattern2)|...')
m = pattern.search(data)
if not m: # this can be omitted if the pattern is always found
...
elif m.group('foo'):
...
elif m.group('bar'):
...

See for example gettext.py where this pattern is used.


2. Replacing a loop-and-a-half construct:

     while m := pattern.search(remaining_data):
     ...


This case can be better handled by re.finditer().


for m in pattern.finditer(remaining_data):
...

In more complex cases it is handy to write a simple generator function 
and iterate its result.


The large number of similar cases are covered by a two-argument iter().

3. Sharing values between filtering clauses and result expressions in 
comprehensions:


     result = [(x, y, x/y) for x in data if (y := f(x))]


There are a lot of ways of writing this. PEP 572 mentions them. 
Different ways are used in real code depending on preferences of the 
author. Actually the number of these cases is pretty low in comparison 
with the total number of comprehensions.


It is possible to express an assignment in comprehensions with the "for 
var in [value]" idiom, and this idiom is more powerful than PEP 572 in 
this case because it allows to perform an assignment before the first 
'for'. But really complex comprehensions could be better written as 
'for' statements with explicit adding to the collection or yielding.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Pattern Matching Syntax

[Serhiy Storchaka]

05.05.18 09:23, Tim Peters пише:

[Tim]

... I liked the way he _reached_ that conclusion:  by looking at real-
life Python code that may have been written instead to use constructs
"like this".  I find such examination far more persuasive than abstract
arguments or made-up examples.


[Serhiy Storchaka <storch...@gmail.com>]

I would like to see such examination for PEP 572. And for all other syntax
changing ideas.


I did it myself for 572, and posted several times about what I found.


Could you please give links to these results? It is hard to find 
something in hundreds of messages.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Runtime assertion with no overhead when not active

[Serhiy Storchaka]

05.05.18 15:54, Eloi Gaudry пише:

I meant avoiding the overhead of the expression evaluation enclosed in the 
assert itself, not the active/disabled state (that comes at virtually no cost).
ex:

runtime_assert( { i:None for i in range( 1000 ) } )


By using the syntax you describe ('boolean and not expr'), you loose all the 
benefit of the Python grammar:
- self-contained


Could you please explain what you mean?


- able to catch semantic/syntax bugs

ex:

f=1
runtime_assert( f==1 )
runtime_assert( f=1 )

   File "", line 1
 runtime_assert( f=1 )
  ^
SyntaxError: invalid syntax


Until inline assignment expression is implemented, this is an error too:

>>> if debug and not f=1:
  File "", line 1
if debug and not f=1:
  ^
SyntaxError: invalid syntax

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Runtime assertion with no overhead when not active

[Serhiy Storchaka]

07.05.18 12:58, Eloi Gaudry пише:
I didn't mean to replace the current (debug) assert but I wanted to add 
another one that would allow to be switch on or off on production builds.


The need for a new keyword (not syntax) comes from this difference.


  I cannot think of another example that would convince you of the 
benefit of having a specific keyword for such a runtime assert.  I do 
believe that having such a feature in non-debug build is more than 
interesting but indeed needed.


I just don't understand why you need a new keyword for writing runtime 
checks.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] __dir__ in which folder is this py file

[Serhiy Storchaka]

06.05.18 09:53, Yuval Greenfield пише:

I often need to reference a script's current directory. I end up writing:

import os
SRC_DIR = os.path.dirname(__file__)


But I would prefer to have a new dunder for that. I propose: "__dir__". 
I was wondering if others would find it convenient to include such a 
shortcut.


Here are some examples of dirname(__file__) in prominent projects.

https://github.com/tensorflow/models/search?l=Python=dirname=
https://github.com/django/django/search?l=Python=dirname=
https://github.com/nose-devs/nose/search?l=Python=dirname=

Reasons not to add __dir__:
* There already is one way to do it and it's clear and fairly short..
* Avoid the bikeshed discussion of __dir__, __folder__, and other 
candidates.


* Additional burden on maintainers of import machinery. It is already 
too complex, and __file__ is set in multiple places. Don't forgot about 
third-party implementations.


See also issue33277: "Deprecate __loader__, __package__, __file__, and 
__cached__ on modules" (https://bugs.python.org/issue33277).


* More complex user code, because you have to handle different cases:

  - __file__ is set, but __dir__ is not set.
  - __file__ and __dir__ are set, but are not consistent.


Reasons to add it:


Are you aware of importlib.resources?

https://docs.python.org/3.7/whatsnew/3.7.html#importlib-resources

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] __dir__ in which folder is this py file

[Serhiy Storchaka]

07.05.18 17:42, Eric Snow пише:

I'm not necessarily saying we should add ModuleSpec.dirname(), but it
(or something like it) is what I'd advocate for *if* we were to add a
convenient shortcut to the directory a module is in.  FWIW, I'd
probably use it.


The question is *why* you need the absolute path to the directory the 
module is in? Taking into account the availability of 
importlib.resources etc.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Serhiy Storchaka]

09.05.18 15:39, Facundo Batista пише:

authors = ["John", "Mary", "Estela"]
"Authors: {:, j}".format(authors)

'Authors: John, Mary, Estela'


In the case of the list of book or article authors it would be better to 
get "John, Mary and Estela" or "John, Mary, and Estela". In other cases 
"John, Mary & Estela" can be more appropriate.


I mean that in real case you will need to use an external function that 
implements more complex algorithm for formatting a list.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Runtime assertion with no overhead when not active

[Serhiy Storchaka]

07.05.18 20:52, Guido van Rossum пише:

He basically wants a macro so that

   runtime_assert()

expands to

   if  and ():
   raise AssertionError

In Lisp this would be easy. :-)


Python is not Lisp (still). But there is the MacroPy project. And at end 
you always can use an external tool for code generation. For example the 
old good cpp.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/