[issue34447] ttk.TreeView (and maybe other functions) is overzealous in converting item values to ints

2018-08-20 Thread Andrew Barnert


New submission from Andrew Barnert :

See this StackOverflow question for a repro case: 
https://stackoverflow.com/questions/51941260/

If you store a string that looks like an int in a TreeView's values, then call 
widget.item(row), the dict's 'values' value has that string converted to an int.

This seems reasonable, because it's legal to store an int in a TreeView, and if 
tkinter didn't convert the values, you'd store 123 and get back '123'.

However, I think it should only be converting strings that could be a valid Tcl 
int. If you store the int 123_456, Tcl of course gives you back '123456', not 
'123_456', so there's no reason for tkinter to convert '123_456' to an int. But 
it does, because ttk._tclobj_to_py just does a try: return int(s).

---

Maeanwhile, if you instead call widget.item(row, options='values'), the string 
is left alone. 

At first glance, that made sense. But looking into what tkinter is getting back 
from Tcl, there's really no difference here. The tuple ('180518-23', 
'23/06/18') is no more or less conversion-worthy than the same tuple inside the 
list (, '', , '', , ('180518-23', '23/06/18'), , 0, 
, ''). It's just that ttk._val_or_dict doesn't call 
_tclobj_to_py on the value when it gets back a key-value pair, and I don't see 
why it shouldn't.

However, that turns out to be a handy workaround for the Stack Overflow user 
who ran into this problem, so maybe it's better not to change that.

--
components: Tkinter
messages: 323819
nosy: abarnert
priority: normal
severity: normal
status: open
title: ttk.TreeView (and maybe other functions) is overzealous in converting 
item values to ints
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue34447>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26300] "unpacked" bytecode

2016-02-06 Thread Andrew Barnert

Andrew Barnert added the comment:

Reading more about wpython (slide 23 of 
https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/wpython2/Cleanup%20and%20new%20optimizations%20in%20WPython%201.1.pdf),
 one of his optimizations was moving the peephole optimizer into the compiler, 
so it could just use the linked list of block objects of arrays of instruction 
objects, instead of raw bytecode.

Obviously that idea isn't compatible with PEP 511. But on the off chance that 
PEP 511 founders, that might be the simplest answer to this problem.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26300>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26300] "unpacked" bytecode

2016-02-05 Thread Andrew Barnert

New submission from Andrew Barnert:

Currently, the compiler starts with a list of arrays of instructions, packs 
them to 1/3/6-bytes-apiece bytecodes, fixes up all the jumps, and then calls 
PyCode_Optimize on the result. This makes the peephole optimizer much more 
complicated. Assuming PEP 511 is accepted, it will also make plug-in bytecode 
optimizers much more complicated (and probably wasteful--they'll each be 
repeating the same work to re-do the fixups).

The simplest alternative (as suggested by Serhiy on -ideas) is to expose an 
"unpacked" bytecode to the optimizer (in the code parameter and return value 
and lnotab_obj in-out parameter for PyCode_Optimize, and similarly for PEP 511) 
where each instruction takes a fixed 4 bytes. This is much easier to process. 
After the optimizer returns, the compiler packs opcodes into the usual 
1/3/6-byte format, removing NOPs, retargeting jumps, and adjusting the lnotab 
as it goes. (Note that it already pretty much has code to do all of this except 
the NOP removal; it's just doing it before the optimizer instead of after.)

Negatives:

 * Arguments can now only go up to 2**23 instead of 2**31. I don't think that's 
a problem (has anyone ever created a code object with 4 million instructions?).

 * A bit more work for the compiler; we'd need to test to make sure there's no 
measurable performance impact.

We could also expose this functionality through C API PyCode_Pack/Unpack and 
Python dis.pack_code/unpack_code functions (and also make the dis module know 
how to parse unpacked code), which would allow import hooks, post-processing 
decorators, etc. to be simplified as well. This would remove some, but not all, 
of the need for things like byteplay. I think this may be worth doing, but I'm 
not sure until I see how complicated it is.

We could even allow code objects with unpacked bytecode to be executed, but I 
think that's unnecessary complexity. Nobody should want to do that 
intentionally, and if an optimizer lets such code escape by accident, a 
SystemError is fine.

MRAB implied an alternative: exposing some slightly-higher-level label-based 
format. That would be even nicer to work with. But it's also more complicated 
for the compiler and for the API, and I think it's already easy enough to 
handle jumps with fixed-width instructions.

--
components: Interpreter Core
messages: 259693
nosy: abarnert, benjamin.peterson, georg.brandl, haypo, pitrou, 
serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: "unpacked" bytecode
type: enhancement

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26300>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26267] UUID docs should say how to get "standard form"

2016-02-02 Thread Andrew Barnert

New submission from Andrew Barnert:

Most real-world code that uses the UUID module wants either the standard format 
'{12345678-1234-5678-1234-567812345678}', or the same thing without the braces.

There are a number of different documented accessors, but none of them give you 
either of these. The simplest way I can think of to guarantee the standard 
format from what's documented is '{%08x-%04x-%04x-%02x%02x-%12x}' % u.fields.

It might be nice to add accessors for standard form and braceless standard 
form, but that probably isn't necessary--as long as there's documentation 
saying that __str__ returns the braceless standard form.

The example code does say that, but I don't think people can trust that a 
comment in an example is binding documentation--plus, plenty of people don't 
read the examples looking for more information about things that aren't 
documented. And I've seen people come up with buggy versions of the format 
string that miss leading zeros, or horrible things like repr(u)[6:42].

--
assignee: docs@python
components: Documentation
messages: 259414
nosy: abarnert, docs@python
priority: normal
severity: normal
status: open
title: UUID docs should say how to get "standard form"
type: enhancement

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26267>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26267] UUID docs should say how to get "standard form"

2016-02-02 Thread Andrew Barnert

Andrew Barnert added the comment:

> The docs seem to clearly show a str(x) as the way to produce the desired form 
> (without braces). https://docs.python.org/3/library/uuid.html#example

Please read the last paragraph of the original message.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26267>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26267] UUID docs should say how to get "standard form"

2016-02-02 Thread Andrew Barnert

Andrew Barnert added the comment:

No, I suggest we add something for users who don't think that a comment in an 
example is the place to look for documentation, but do think that actual 
documentation of a class or method is.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26267>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2016-02-01 Thread Andrew Barnert

Andrew Barnert added the comment:

> I did actually mean a version changed notice for the data model change. I see 
> this as a small expansion of the Python object API. Previously, 
> __reversed__() had to be a function, now you are also allowed to set it to 
> None. The collections ABCs are just catching up with the API change.

Are you suggesting that in Python 3.5, it's not defined what happens if you set 
__reversed__ = None on a sequence-like object and then call reversed()? I think 
any implementation has to raise a TypeError; the only difference is that we're 
now documenting the behavior directly, rather than forcing you to infer it from 
reading a half-dozen different parts of the docs and tracing through the 
implied behavior. It's already the best way to do it in 3.5, or even 2.5; the 
problem is that it isn't _obviously_ the best way. And I think a 
version-changed would send the wrong message: someone might think, "Oh, I can't 
use None here because I can't require 3.6, so what should I do? Write a method 
that raises TypeError when called?" (Which will work, but will then make it 
very hard to write a Reversible implicit ABC if they later want to.)

> Imagine someone using an Orderable virtual base class that tested for 
> __gt__() etc.

That's part of what we're trying to solve. Do you test that with hasattr, like 
Sized, or do you test for getattr with default None, like Hashable?

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2016-02-01 Thread Andrew Barnert

Andrew Barnert added the comment:

> This is not really my area of expertise, but I would have thought if you 
> defined a __special__ method to something illegal (non-callable, or wrong 
> signature) it would be reasonable for Python to raise an error at class 
> definition (or assignment) time, not just later when you try to use it.

As Guido pointed out on the -ideas thread, defining __spam__ = None to block 
inheritance of a superclass implementation has long been the standard way to 
mark a class unhashable. So, any Python that raised such an error would break a 
lot of code. 

> Somewhere I think the documentation says you are only allowed to use these 
> names as documented.

I can't find anything that says that. Any idea where to look? That might be 
worth adding, but if we add it at the same time as (or after) we explicitly 
document the None behavior, that's not a problem. :)

By the way, did you not review my last patch because you didn't get an email 
for it? I think Rietveld #439 
(http://psf.upfronthosting.co.za/roundup/meta/issue439) may be causing issues 
to get stalled in the patch stage because people are expecting to get flagged 
when a new patch goes up but never see it (unless they're on the same mail 
domain as the patcher).

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26225] New misleading wording in execution model documenation

2016-01-27 Thread Andrew Barnert

New submission from Andrew Barnert:

In #24129, the wording describing class local bindings in 4.2.2 "Resolution of 
names" was changed for Python 3.4, 3.5, and 3.6. The new version is a lot 
clearer for classes--but now it's misleading for `exec`/`eval`.

---

> Class definition blocks and arguments to exec() and eval() are
> special in the context of name resolution. A class definition is...

... and then proceeds to explain how class lookup works, without ever 
mentioning `exec` and `eval`. This implies that they work the same way as 
classes, but of course that's not true:

i = 'global'
def f():
i = 'nonlocal'
class C:
print(i)
i = 'local'
print(i)
f()

That prints `global`, then `local`. But with `exec`:

i = 'global'
def f():
i = 'nonlocal'
exec("print(i)\ni = 'local'\nprint(i)\n")
f()

That prints `nonlocal` then `local`.

I think just putting a paragraph break between the first sentence and the rest 
of the paragraph might be sufficient to avoid the confusion here. Or just 
removing any mention of `eval` and `exec`. If not, this probably needs a new 
one-liner paragraph saying something like "Arguments to `exec()` and `eval()` 
are also special, as described later."

---

Meanwhile, if you keep reading, you'll eventually find that `exec` is described 
in a later section, 4.2.4 "Interaction with dynamic features", but that's 
_also_ misleading:

> The eval() and exec() functions do not have access to the full
> environment for resolving names. Names may be resolved in the
> local and global namespaces of the caller. Free variables are not
> resolved in the nearest enclosing namespace, but in the global
> namespace.

If that were true, the `exec` example would have printed `global`, right?

I'm pretty sure that what's going on here is that `exec` implicitly calls 
`locals()` (or, rather, the C-API equivalent), which constructs a locals dict 
on demand, which, only if you're inside a function block, includes not just the 
currently-bound fast locals, but _also_ the cell_contents of the 
currently-bound free variables. So, as far as `exec` is concerned, `i` is not 
an unbound local, or a free variable, but a local, which is bound to the 
`'nonlocal'` cell value of `i` at the time `exec` was called.

Which means the following actually _does_ print `global`:

i = 'global'
def f():
exec("print(i)\ni = 'local'\nprint(i)\n")
i = 'nonlocal'
f()

I have no idea how to make this clear. Maybe the simplest is to not try to give 
a full explanation here, and instead punt to the `locals()` function 
definition? Maybe something like this:

> The `eval()` and `exec()` functions do not have access to the full 
> environment for resolving names, but rather to the approximation of that 
> environment as constructed by the `locals()` function. Free variables that 
> are not captured as locals are not resolved in the nearest enclosing 
> namespace, but in the global...

... and from there, the same as the current paragraph.

--
assignee: docs@python
components: Documentation
messages: 259073
nosy: abarnert, docs@python
priority: normal
severity: normal
status: open
title: New misleading wording in execution model documenation

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26225] New misleading wording in execution model documenation

2016-01-27 Thread Andrew Barnert

Changes by Andrew Barnert <abarn...@yahoo.com>:


--
type:  -> enhancement

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26152] A non-breaking space in a source

2016-01-20 Thread Andrew Barnert

Andrew Barnert added the comment:

> Assuming Andrew is correct, it sounds like the tokenizer is treating the NBSP 
> and the “2” as part of the same token, because NBSP is non-ASCII.

It's more complicated than that. When you get an invalid character, it splits 
the token up. So, in this case, you get a separate `ERRORTOKEN` from cols 2-3 
and `NUMBER` token from cols 3-4. Even in the case of `1, a\xa0\xa02`, you get 
a `NAME` token for the `a`, a separate `ERRORTOKEN` for each nbsp, and a 
`NUMBER` token for the `2`.

But I think the code that generates the `SyntaxError` must be trying to 
re-generate the "intended token" from the broken one. For example:

>>> eval('1\xa0\xa02a')
File "", line 1
  1  2a
      ^
SyntaxError: invalid character in identifier

And if you capture the error and look at it, `e.args[1][1:3]` is 1, 5, which 
matches what you see.

But if you tokenize it (e.g., 
`list(tokenize.tokenize(io.BytesIO('1\xa0\xa02a'.encode('utf-8')).readline))`, 
but you'll probably want to wrap that up in a function if you're playing with 
it a lot...), you get a `NUMBER` from 0-1, an `ERRORTOKEN` from 1-2, another 
`ERRORTOKEN` from 2-3, a `NUMBER` from 3-4, and a `NAME` from 4-5. So, why does 
the `SyntaxError` point at the `NAME` instead of the first `ERRORTOKEN`? 
Presumably there's some logic that tries to work out that the two 
`ERRORTOKEN`s, `NUMBER`, and `NAME` were all intended to be one big identifier 
and points at that instead.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26152>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26152] A non-breaking space in a source

2016-01-19 Thread Andrew Barnert

Andrew Barnert added the comment:

Ultimately, this is because the tokenizer works byte by byte instead of 
character by character, as far as possible. Since any byte >= 128 must be part 
of some non-ASCII character, and the only legal use for non-ASCII characters 
outside of quotes and comments is as part of an identifier, the tokenizer 
assumes (see the macros at the top of tokenizer.c, and the top of the again 
block in tok_get) that any byte >= 128 is part of an identifier, and then 
checks the whole string with PyUnicode_IsIdentifier at the end.

This actually gives a better error for more visible glyphs, especially ones 
that look letter-like but aren't in XID_Continue, but it is kind of weird for a 
few, like non-break space.

If this needs to be fixed, I think the simplest thing is to special-case 
things: if the first non-valid-identifier character is in category Z, set an 
error about invalid whitespace instead of invalid identifier character. (This 
would probably require adding a PyUnicode_CheckIdentifier that, instead of just 
returning 0 for failure as PyUnicode_IsIdentifier, returns -n for 
non-identifier character with code point n.)

--
nosy: +abarnert

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26152>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2016-01-18 Thread Andrew Barnert

Andrew Barnert added the comment:

Style changes based on Martin's review

--
Added file: http://bugs.python.org/file41650/patch5.diff

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2016-01-18 Thread Andrew Barnert

Changes by Andrew Barnert <abarn...@yahoo.com>:


Removed file: http://bugs.python.org/file41650/patch5.diff

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19251] bitwise ops for bytes of equal length

2016-01-10 Thread Andrew Barnert

Andrew Barnert added the comment:

On Jan 10, 2016, at 06:48, cowlicks <rep...@bugs.python.org> wrote:
> 
> 
> Personally this came up while I was playing with toy crypto problems. I 
> expected to already be part of the language, but it wasn't. I think this is a 
> natural expectation.

Maybe if you're coming to Python from APL or R or something, but C, C#, Scala, 
Ruby, Haskell--almost any other language you pick, there's no bitwise 
operations on (byte) strings. (And if you _are_ coming to Python from something 
like R, you definitely should be using NumPy.)
> 
> And I do not understand how bitwise operations work on arbitrary precision 
> integers.

It's obvious once you think of them as infinite-sized fixed-size ints: 0x27 is 
the same number as 0x0027, so 0x27 & 0x0134 is 0x0024. (Of course not and 
negation aren't quite as obvious, but once you think about it, there's only one 
sensible thing 2's complement could do, and only two sensible things 1's 
complement could do, and Python is sensible, so it's not surprising once you 
try it out.)

> Some folks have suggested using NumPy, but that is a very heavy dependency, 
> and not useful outside of cpython.

It's useful outside of CPython. While NumPyPy isn't 100% yet, it's usable 
enough for many projects.

More to the point, if you're looking for arrays that have really fast and 
highly readable elementwise operations, that's exactly what NumPy is all about. 
Sure, you can get bits and pieces of similar functionality without it, but if 
you're thinking about your code in NumPy's terms (elementwise operations), you 
really do want to think about NumPy.

Meanwhile, have you looked around PyPI at the various bitarray, bitstring, etc. 
libraries? Are they too slow, too heavyweight, or too inconveniently-API'd? I 
know whenever I want to play with things like Huffman coding or the underlying 
bit representation of IEEE floats or anything else bitwise besides basic C 
integer stuff, I reach for one of those libraries, rather than trying to hack 
things up around bytes strings. (Except for that example I posted 
above--clearly for some reason I _did_ hack things up around bytes strings that 
time--but it only took me that simple class to make things convenient and 
efficient.)

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue19251>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19251] bitwise ops for bytes of equal length

2016-01-08 Thread Andrew Barnert

Andrew Barnert added the comment:

There are a number of existing libraries on PyPI for bit arrays and bit sets. 
There are a lot more API choices than you'd think of in advance, and between 
them they explore most of the useful space. And I don't think any of them need 
to be in the stdlib.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue19251>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19251] bitwise ops for bytes of equal length

2016-01-07 Thread Andrew Barnert

Andrew Barnert added the comment:

For what it's worth, I looked at some old code (a clean re-implementation of a 
crypto algorithm in Python, used as a unit test for production code in C++) and 
found this:

class BytesBits(bytes):
# from a quick test, 1.12us in NumPy, 1.39us in C, 2.55us this way, 46.1us 
with bytes(genexpr), so we don't need numpy
def _bitwise(self, other, op):
iself = int.from_bytes(self, 'little')
iother = int.from_bytes(other, 'little')
b = op(iself, iother)
return b.to_bytes(len(self), 'little')
def __or__(self, other):
return self._bitwise(other, int.__or__)
__ror__ = __or__
def __and__(self, other):
return self._bitwise(other, int.__and__)
__rand__ = __and__
def __xor__(self, other):
return self._bitwise(other, int.__xor__)
__rxor__ = __xor__

It doesn't do as much error checking as you want, but it was good enough for my 
purposes.

At any rate, the fact that it's trivial to wrap this up yourself (and even more 
so if you just write functions called band/bor/bxor instead of wrapping them up 
in a subclass) implies to me that, if it's not used all that often, it doesn't 
need to be on the builtin types.

--
nosy: +abarnert

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue19251>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2016-01-07 Thread Andrew Barnert

Changes by Andrew Barnert <abarn...@yahoo.com>:


Added file: http://bugs.python.org/file41527/patch4a.diff

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26019] collections.abc documentation incomplete

2016-01-05 Thread Andrew Barnert

Andrew Barnert added the comment:

The attached patch is, I think, the smallest change that includes all the 
necessary information.

Of course it will need to be redone if #25958 goes through, with a 
version-changed note explaining that prior to 3.6 some of the implicit ABCs 
(maybe with a list) treated None as an abstract method implementation while 
others didn't. So, this patch is basically a throwaway. But any review comments 
would still be useful, of course.

--
keywords: +patch
Added file: http://bugs.python.org/file41513/patch26019.diff

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26019>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2016-01-05 Thread Andrew Barnert

Andrew Barnert added the comment:

The second patch takes into account all the issues raised by Martin and Guido, 
as well as some other changes that didn't make it into the first patch because 
Windows hates me. And it should be flattened into a single commit, and 
therefore should hopefully work with Rietveld out of the box. It passes the 
test suite on Windows and Cygwin (and on OS X, but that's a slightly older 
version of the changes than what's in this flattened patch).

I think it's still an open question whether Reversible should inherit Iterable. 
In the current patch, as in the first, it does.

I'll go over Serhiy's Reitveld comments to see if there's anything I missed, 
and, if so, address it in a third attempt.

On Serhiy's test suggestions:

 * I don't think we need to repeat the same tests on every ABC. That seems more 
likely to introduce copy-paste bugs than to catch anything, especially since 
the ABCs now all share the same code, but the tests couldn't.

 * Likewise for adding repetitive tests on more __spam__, __ispam__, and 
__rspam__ methods.

 * However, he's definitely right that there are other kinds of fallback worth 
testing, like __ne__ -> __eq__. I'll review the different kinds of fallback 
more carefully and make sure we have tests for each case.

--
Added file: http://bugs.python.org/file41509/patch2.diff

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22268] add dedicated functions mrohasattr and mrogetattr

2016-01-05 Thread Andrew Barnert

Andrew Barnert added the comment:

See #25958, which incorporates a similar refactoring in service of making all 
of the subclasshooks in collections.abc handle None the same way.

We definitely do want something public in the abc module like this, rather than 
the private function in collections.abc that I wrote, especially given the 
discussion about moving a bunch of the ABCs to other modules.

--
nosy: +abarnert

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue22268>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26019] collections.abc documentation incomplete

2016-01-05 Thread Andrew Barnert

New submission from Andrew Barnert:

Some of the modules in collections.abc have a subclass hook that implicitly 
registers any type that declares the right methods, like Iterator. Others do 
not, like Sequence. For those that do have the hook, it's not always obvious 
what methods are tested. And some of them test the methods for truthiness, 
others only for existence (although #25958 will fix that last bit).

The documentation doesn't even mention this, much less describe which ABCs are 
of which kind.

For someone who just wants to know how to write isinstance(arg, Iterable), 
that's fine. But anyone who wants to create new classes, or integrate 
third-party classes that weren't ABC-friendly, has to read the collections.abc 
module source to figure out what they want to do.

--
assignee: docs@python
components: Documentation
messages: 257577
nosy: abarnert, docs@python
priority: normal
severity: normal
status: open
title: collections.abc documentation incomplete
type: enhancement
versions: Python 3.6

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26019>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2016-01-04 Thread Andrew Barnert

Andrew Barnert added the comment:

> My main concern about the documentation was that in your patch you say _all_ 
> special methods are now allowed to be None, but in your code you only check 
> __iter__(). This is adding new undocumented inconsistencies e.g. with 
> Iterable vs Container.

No it isn't. We already have undocumented inconsistencies between, e.g., 
Hashable vs. Container; we're just moving Iterable from one set to the other. 
(Notice that there's an even larger inconsistency between, e.g., Hashable and 
Container vs. Sequence and Set. As Guido suggests, that probably _does_ need to 
be fixed, but not as part of this bug, or #25958.)

And, as for the docs, it's already true that you can block fallback and 
inheritance of special methods by assigning them to None, but that isn't 
documented anywhere.

So, this bug doesn't add fix any of those inconsistencies, but it doesn't add 
any new ones, either. If you think we actually _need_ to fix them, see #25958 
as at least a starting point. (Notice that Guido seems to want that one fixed, 
so, assuming I can write a decent patch for it, this bug would then become a 
one-liner: "__reversed__ = None" inside Mapping.)

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2016-01-04 Thread Andrew Barnert

Andrew Barnert added the comment:

Is an hg export patch usable? If not, let me know and I'll flatten it.

Anyway, the attached patch fixes #25987 and #25864 as well as this one, as 
follows:

 * Every ABC in collections.abc that has a subclass hook now treats None as 
blocking, instead of a few checking falsiness and others not checking anything. 
(And all the hooks are now identical, which eliminates any unintended 
inconsistencies.)

 * collections.abc.Mapping.__reversed__ = None (fixes #25864).

 * collections.abc.Reversible added (fixes #25987).

 * iter(), reversed(), and in operator all have custom text in their TypeError 
instead of the generic "'NoneType' object is not callable".

 * Unit tests for all of the above, plus that None blocking also works 
appropriately (i.e., same as in CPython 2.3-3.6) for __spam__ -> __rspam__, 
__ispam__ -> __spam__.

  * I didn't try to write tests for _everything_ with fallback. For example, 
blocking fallback from __str__ to __repr__, or anything to do with __*attr*__, 
does work, but I can't think of any conceivable case where you'd ever do such a 
thing. (The fact that you can write a class that isn't repr-able isn't a 
feature of Python we want to guarantee, it's just a side-effect of other stuff 
that will hopefully never come up, so it seems wrong to write a unit test to 
guarantee it.)

 * Data Model docs document that you can set a special method to None to block 
behavior (including blocking fallback to old-style sequence protocol, 
inheritance from superclass, etc.).

(Note that this patch is not yet fully tested, because my Windows and Cygwin 
builds are for some reason taking forever. But if I don't post an update 
tomorrow, that will mean they passed the overnight tests. I doubt anyone was 
going to commit this tonight anyway, but, just in case...)

--
keywords: +patch
Added file: http://bugs.python.org/file41502/patch.diff

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2016-01-04 Thread Andrew Barnert

Andrew Barnert added the comment:

> A Mercurial export patch should work with the Reitveld review thing if it is 
> a single revision (and based on a public revision).

Well, it's 7 separate commits on a work branch, so I could check in each piece 
and test it separately, and then I just compared the branch to its parent 
(default) for the export. But, as I said, I can flatten it into a single diff 
if the review system can't handle this way.

Sorry for making the first patch hard to review, and thanks for reviewing it 
anyway; I'll fix that with the next one.

> You have a stray print() call, probably in Hashable.__subclasshook__().

Thanks; fixed for the next patch.

> For a the do-nothing __reversed__() implementation, do you think “yield from 
> ()” would be clearer than “while False”? Or even “return iter(())”?

Maybe, but this way is identical to the definition for the do-nothing 
Iterable.__iter__.

> What’s the advantage of having Reversed inherit Iterable?

This was discussed briefly... somewhere, maybe the -ideas thread? 

Pro: If they were independent types, more people would probably register 
with/inherit from Reversible but not Iterable in error than intentionally.

Pro: One less thing for Sequence to inherit from. This was mentioned in the 
context of adding both Reversible and HalfSequence (or whatever it ends up 
being called): "class Sequence(Iterable, Reversible, HalfSequence, Sized, 
Container) seems a little much; "class Sequence(Reversible, HalfSequence, 
Sized, Container)" is slightly better.

Con: It's not completely impossible that a type could be reversible but not 
iterable. For example, you might have a reversible, and even 
indexable-with-negative-indicies-only, infinite collection of all the negative 
numbers. (But nobody could think of an actual _useful_ example of such a type.)

Those are all pretty trivial points. When Guido said someone needs to work out 
where Reversible fits into the hierarchy, and there'd been enough talk without 
a patch. So I looked at where the discussion seemed to be leaning, couldn't 
find any additional problems with it when I looked into it further, and did it 
that way. Guido said he liked it on the Reversible bug, so I did it the same 
way here.

So, no strong arguments or even feelings from anyone; if you've got one, 
definitely chime in.

> How does the subclass hook interact with classes that implement 
> __reversed__() but not __iter__()? I don’t see how the 
> self.assertFalse(issubclass(K, Reversible)) test could pass.

Exactly the same way Iterator's handles a class that defines __next__ but not 
__iter__.

I'll double-check that it actually is implemented right before the next patch 
(and verify that the tests run), but it should be "return _check_methods(C, 
"__reversed__", "__iter__").

> Should the new Reversed class be excluded from collections.__all__, or is it 
> not worth it?

It's Reversible, not Reversed. It's in __all__, and I'm pretty sure it should 
be there--if it's not worth exporting, it's not worth creating in the first 
place, or documenting, right?

> I find the lambda generator a bit quirky in test_Reversible(). Maybe it would 
> be better as an ordinary non-lambda generator with a yield statement.

This is taken from an identical test in test_Iterable, in the same file.

> It would be good to have a versionadded notice for Reversible. 

Definitely; thanks for the catch. I'll add that for the next version of the 
patch.

> I think there should also be one for allowing None for all special methods.

I'm assuming you mean in collections.abc, not in the data model, right?

The problem is where to put it. The collections.abc docs don't even mention 
that some of the ABCs have subclass hooks that detect "implicit subclasses", 
much less tell you which ones do and don't, much less tell you which among 
those that do treat None specially. Until we add that documentation, there's 
really nothing to contrast with.

Maybe this means we need another bug where we rewrite the collections.abc docs 
to include all that information, and in the new documentation we can note what 
prior versions of Python did (or maybe prior versions of CPython--I doubt any 
major implementations were different, but without checking them all I wouldn't 
want to guarantee that)?

> Instead of self.assertEqual(list(reversed(R())), list(reversed([]))), why not 
> check it is equal to an empty list directly?

This might be a case of following the parallelism with test_Iterable too far; I 
can change it.

> In test_contains.py, I would either write lambda: 0 in bc, or use the “with 
> self.assertRaises()” form.

That makes sense; I'll do it.

> Finally, if setting a binary operator method to None means that operation is 
> not available, I find it a bit surprising that doing this prevent the other 
> operand’s method from being tried. 

[issue25958] Implicit ABCs have no means of "anti-registration"

2016-01-01 Thread Andrew Barnert

Andrew Barnert added the comment:

> I don’t think you need to define __len__() to get an iterable, only 
> __getitem__().

The "old-style sequence protocol" means having a __getitem__ that works for 
values from 0 to __len__() and raises IndexError at __len__(). You don't need 
to be a complete old-style sequence to be iterable; just having __getitem__ 
makes you iterable (without being a collections.abc.Iterable or a 
typing.Iterable), and having __getitem__ and __len__ makes you reversible 
(without being a typing.Reversible). At any rate, this bug isn't about avoiding 
false negatives for the implicit ABCs, but false positives: defining __iter__ = 
None blocks the old-style sequence protocol, but makes isinstance(Iterable) 
true.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25987] collections.abc.Reversible

2016-01-01 Thread Andrew Barnert

New submission from Andrew Barnert:

This came up as a side issue in the -ideas discussion on deprecating the 
old-style sequence protocol that came out of Guido's suggestion on 
https://github.com/ambv/typehinting/issues/170 
(http://article.gmane.org/gmane.comp.python.ideas/37599):

> I also think it's fine to introduce Reversible as another ABC and carefully 
> fit it into the existing hierarchy. It should be a one-trick pony and be 
> another base class for Sequence; it should not have a default implementation. 
> (But this has been beaten to death in other threads -- it's time to just file 
> an issue with a patch.)

I'll file a patch this weekend. But in case there's anything to bikeshed, here 
are the details:

* Reversible is a subclass of Iterable.
 * It has a single abstract method, __reversed__, with no default 
implementation.
 * Its subclass hook that checks for __reversed__ existing and not being None.
* Sequence is a subclass of Reversible, Sized, and Container rather than 
directly of Iterable, Sized, and Container.

Builtins tuple and list, and any subclasses of them, will be Reversible because 
they register with Sequence or MutableSequence. Subclasses of 
collections.abc.Sequence will be Reversible (and should be, as they inherit 
Sequence.__reversed__). Custom old-style sequences will not be Reversible, even 
though reversed works on them.

Builtins dict, set, and frozenset, and any subclasses of them, will not be 
Reversible (unless they add a __reversed__ method, as OrderedDict does). 
Subclasses of collections.abc.Mapping will not be Reversible (and should not 
be, as, assuming #25864 goes through, they inherit Mapping.__reversed__=None) 
(unless they add a __reversed__ method, as most third-party sorted-dict types 
do).

I'll include tests for all of those things.

I believe this is all exactly parallel with collections.abc.Iterable, and will 
make collections.abc.Reversible compatible with typing.Reversible[...] in 
exactly the same way collections.abc.Iterable is compatible with 
typing.Iterable[...].

Alternatives: We could make Reversible independent of Iterable. Alternatively, 
we could make it subclass both Iterable and Sized instead of just Iterable. But 
I think this is the simplest place to slot it in.

--
components: Library (Lib)
messages: 257310
nosy: abarnert
priority: normal
severity: normal
status: open
title: collections.abc.Reversible
type: enhancement

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25987>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2016-01-01 Thread Andrew Barnert

Andrew Barnert added the comment:

> Also, if I understand your problem, Container would also be susceptible in 
> theory

You're right, but not in the details.

Being iterable (whether via __iter__ or via the old-style sequence protocol) 
makes you a container. But, again, false negatives for Container aren't a 
problem.

But blocking that by setting __contains__ = None makes you a Container but not 
a container, the same kind of false positive as #25864. That's exactly why I 
split off this bug from that one--that one only fixes __iter__ and 
__reversed__, but it's possible that a more general solution is needed. (Or, of 
course, maybe we don't need anything more general, we just need to expand it to 
__iter__, __reversed__, and __contains__.)

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2016-01-01 Thread Andrew Barnert

Andrew Barnert added the comment:

> If this patch goes ahead, I think the ABC documentation should clarify which 
> methods are checked for None and which aren’t. 

That seems fair. 

Also, as you pointed out on #25958, at least one other ABC has the same problem 
as Iterable: you can block the "in" operator by setting __contains__=None, but 
you'll still be a Container. So, do we want to go through all of the existing 
ABCs and make sure they all do this negative check, instead of just Iterable?

> Also, what is the point of the odd __getitem__() method in test_enumerate.py? 
> Maybe you should use assertRaisesRegex() to check that the intended TypeError 
> is actually raised.

If an implementation doesn't raise a TypeError there, that's a failure. If it 
raises one with a different (possibly less helpful) message, I think that's 
just a quality-of-implementation issue, isn't it?

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25988] collections.abc.Indexable

2016-01-01 Thread Andrew Barnert

New submission from Andrew Barnert:

In an -ideas thread, Guido suggested 
(http://article.gmane.org/gmane.comp.python.ideas/37599):

> If we want some way to turn something that just defines __getitem__ and 
> __len__ into a proper sequence, it should just be made to inherit from 
> Sequence, which supplies the default __iter__ and __reversed__. (Registration 
> is *not* good enough here.) If we really want a way to turn something that 
> just supports __getitem__ into an Iterable maybe we can provide an additional 
> ABC for that purpose; let's call it a HalfSequence until we've come up with a 
> better name. (We can't use Iterable for this because Iterable should not 
> reference __getitem__.)

Later in the thread, Nick Coghlan suggested 
(http://article.gmane.org/gmane.comp.python.ideas/37614):

> Perhaps collections.abc.Indexable would work? Invariant:

> for idx, val in enumerate(container):
> assert container[idx] is val

> That is, while enumerate() accepts any iterable, Indexable containers
have the additional property that the contained values can be looked
up by their enumeration index. Mappings (even ordered ones) don't
qualify, since they offer a key:value lookup, but enumerating them
produces an index:key relationship.

So, in particular:

* Indexable is a subclass of Iterable.
* Sequence is a subclass of Indexable, Sized, and Container instead of 
Iterable, Sized, and Container. (Or, if #25987 also goes through, of 
Reversible, Indexable, Sized, and Container.)
* The abstract method __getitem__ and the concrete __iter__ implementation get 
moved up from Sequence to Indexable.
* Indexable does _not_ have a subclass hook (to avoid making every Mapping, 
generic type, etc. accidentally Indexable).

This means that you can write this (borrowing an example from Steven D'Aprano 
in http://article.gmane.org/gmane.comp.python.ideas/23369/):

class Squares(collections.abc.Indexable):
def __getitem__(self, index):
return index**2

Because this no longer depends on the old-style sequence protocol, testing it 
with ABCs will work as expected.

For related issues, see #25987, #25864, #25958, and 
https://github.com/ambv/typehinting/issues/170

--
components: Library (Lib)
messages: 257311
nosy: abarnert
priority: normal
severity: normal
status: open
title: collections.abc.Indexable
type: enhancement

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25988>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2015-12-28 Thread Andrew Barnert

Andrew Barnert added the comment:

Hashable and Awaitable already treat None (actually any falsey value) as not 
implementing the special method, and blocking any superclass implementation, in 
their __subclasshook__. (This only blocks implicit subclassing--anything that 
actually directly or indirectly inherits from or registers with Hashable is 
still Hashable even if it sets __hash__ = None.)

Coroutine, AsyncIterable, AsyncIterator, Iterable, Iterator, Generator, Sized, 
Container, and Callable treat None as an implementation.

http://bugs.python.org/file41440/cpython-iter-patch.diff for #25864 changes 
Iterable to work like Hashable, but doesn't touch any of the other types.

To fix this in general, just apply the same change to the other implicit ABCs 
(and probably factor out the logic instead of repeating it that many more 
times).

But I'm not sure a general fix is needed here. The special methods for those 
other types don't have a base class and/or fallback implementation to block the 
way __hash__, __iter__, and __reversed__ do (except for __call__ on 
metaclasses, but I can't imagine why you'd want a metaclass that isn't 
callable), so there's no need to set them to None to block that, so the problem 
should never come up.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-28 Thread Andrew Barnert

Andrew Barnert added the comment:

The attached patch does the following:

* collections.abc.Mapping.__reversed__ = None.

* collections.abc.Iterable.__subclasshook__ checks for None the same way 
Hashable does:

  * This tests for any falsey value, not just None. I'm not sure this is ideal, 
but it's consistent with Hashable, and it's unlikely to ever matter.

  * The test will only block implicit subclassing. If a class, e.g., inherits 
from tuple (which is explicitly registered with Sequence, which inherits from 
Iterable), it's Iterable, even if it sets __iter__ = None. I think this is the 
right behavior, and it's consistent with Hashable.

* iter and reversed add checks for None, which raise a TypeError with the 
appropriate message (instead of "'NoneType' is not callable").

* datamodel.rst section "Special method names" includes a paragraph on setting 
special methods to None.

* Tests for changes to reversed (in test_enumerate.py), iter (in test_iter.py), 
Iterable (in test_collections.py), and Mapping (in collections.py).

--
keywords: +patch
Added file: http://bugs.python.org/file41440/cpython-iter-patch.diff

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-26 Thread Andrew Barnert

Andrew Barnert added the comment:

> Perhaps there is a bug in typing.Reversible. It doesn't accept all types 
> supported by reversed().

> ... And accepts types that don't work with reversed().

The problem is the way the two are defined:

* Reversible is true if you implement __reversed__
* reverse works if you implement __reversed__ or implement the old-style 
sequence protocol.

That explains why it doesn't work on tuple, bytearray, etc. Iterable actually 
has the exact same problem, but, because it's a supertype of Sequence, and we 
have explicit Sequence.register(tuple) and MutableSequence.register(bytearray) 
in collections.abc, and typing.Iterable specifies collections.abc.Iterable as 
its "extra", it all works out.

We could do the same for Reversible: add a collections.abc.Reversible, make it 
a subtype of Iterable and make Sequence a subtype of Reversible instead of 
Iterable, and make that the extra for typing.Reversible. Then it would work for 
all of those builtin types (and many third-party types that explicitly register 
with Sequence), just as Iterable does.

But that only solves the problem in one direction. To solve it in the other 
direction, we'd need some way to either explicitly mark a method as not 
implemented (maybe setting it to None, or to any non-callable, or any data 
descriptor?) that ABC subclass hooks and/or typing checks are expected to 
understand, or unregister a class with an ABC so that it isn't a subtype even 
if it passes the implicit hooks.

Or... could we just drop Reversible as an implicit protocol? The lack of an 
explicit "deny" mechanism for implicit protocols and ABCs is a more general 
problem, but if this is the only actual instance of that problem in real life, 
do we need to solve the general problem? If not, there's no obvious way to 
define typing.Reversible that isn't wrong, it doesn't have a corresponding ABC, 
it doesn't seem like it will be useful often enough to be worth the problems it 
causes, and I doubt there's much real-life code out there already depending on 
it, so that seems a lot easier.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2015-12-26 Thread Andrew Barnert

Andrew Barnert added the comment:

As Guido pointed out on -ideas, hashing already uses the convention of 
`__hash__ is None` to declare a type unhashable, and 
`collections.abc.Hashable.__subclasshook__` already checks for that.

Meanwhile, setting `__iter__` and `__reversed__` to `None` already raises a 
`TypeError` on `iter` and `reversed` (although not with the most helpful 
description).

So, maybe that should be documented as the standard way to unimplement 
`__iter__` and `__reversed__`, and `collections.abc.Iterable` should check that 
the looked-up `__iter__` is not `None` (and presumably typecheckers doing the 
equivalent for both `Iterable` and `Reversible`)?

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-26 Thread Andrew Barnert

Andrew Barnert added the comment:

As mentioned in #25958, Guido pointed out on -ideas that `__hash__ = None` is 
already the standard way to declare a class unhashable, and it's recognized by 
`collections.abc.Hashable`.

Doing `__reversed__ = None` does make `reversed(m)` raise a `TypeError` 
(although with a description saying "'NoneType' is not callable", which isn't 
quite as nice a description, but probably good enough).

So, I think `Mapping` should set `__reversed__ = None`, rather than setting it 
to a method that raises `TypeError`. (If we need something more general, that's 
for #25958 and/or Serhiy's typechecking bug.)

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-26 Thread Andrew Barnert

Andrew Barnert added the comment:

> This sounds good. Also, reversed() could then be modified to produce a
better error.

Should `iter` also be modified to produce a better error if `__iter__` is None?

Also, should this be documented? Maybe a sentence in the "Special method names" 
section of the "Data model" chapter, like this:

> To indicate that some syntax is not supported, set the corresponding special 
> method name to None. For example, if __iter__ is None, the class is not 
> iterable, so iter() will not look for __getitem__.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-26 Thread Andrew Barnert

Andrew Barnert added the comment:

Serhiy already filed the typing.Reversible bug on the separate typehinting 
tracker (https://github.com/ambv/typehinting/issues/170). So, unless fixing 
that bug requires some changes back to collections.abc or something else in the 
stdlib, I think the only issue here is the original one, on Mapping.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2015-12-26 Thread Andrew Barnert

New submission from Andrew Barnert:

Serhiy Storchaka raised an issue (http://bugs.python.org/msg256910) with static 
type hints on #25864 (http://bugs.python.org/issue25864), which I believe also 
applies to runtime ABCs.

Consider this class, which uses `[]` in a way similar to generic types:

class Thing:
def __getitem__(self, specialization):
return type(self)._specialize(specialization)(self)
def __len__(self):
return len(self._specializations)

Because this type looks like it supports the old-style sequence protocol, 
calling either `iter` or `reversed` on an instance will successfully return a 
useless iterator. (You may get an error when you start trying to iterate it, 
but you may not even then.) You don't want that, so you add either this:

__iter__ = None
__reversed__ = None

... or this:

def __iter__(self): raise TypeError('not iterable')
def __reversed__(self): raise TypeError('not iterable')

Unfortunately, doing either means that `issubclass(Thing, 
collections.abc.Iterable)` now returns true. Which is the exact opposite of the 
intention of that check. (The same is true for `typing.Iterable` and 
`typing.Reversible`.) So, fixing the problem for duck typing creates the 
equivalent problem for explicit typing.

There are a few possible solutions here:

1. Maybe document it, otherwise do nothing.

2. Change the ABCs to check that the dunder method exists and is not None (or 
is callable, or is a non-data descriptor). Then, the one way to opt out is to 
assign `__iter__ = __reversed__ = None`.

3. Add an `ABC.unregister` method that can be used to explicitly state that 
this type does not support that ABC, regardless of what its `__subclasshook__` 
says.

Possible argument for #1: `Iterable` rarely has a problem. (Types that use 
`__getitem__` for something completely un-sequence-like, like `typing`'s 
generic types, usually don't have `__len__`. Types that have both `__getitem__` 
and `__len__`, like mappings, usually have a reasonable alternative `__iter__` 
to offer.) `Reversible` would have a problem if there was such an ABC, but 
there isn't. Off the top of my head, I can't think of any of the other implicit 
ABCs that are susceptible to this problem.

The counter-argument is that static typehinting definitely does have this 
problem (https://github.com/ambv/typehinting/issues/170), and, depending on how 
that's solved, it may well make sense to use the same solution here.

If we do need a solution, #2 seems better than #3 (or anything else I could 
think up). The only problem there is that `def __iter__(self): raise 
TypeError('not iterable')` gives you a nicer error than `__iter__ = None`.

--
components: Library (Lib)
messages: 257052
nosy: abarnert
priority: normal
severity: normal
status: open
title: Implicit ABCs have no means of "anti-registration"
versions: Python 3.6

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-26 Thread Andrew Barnert

Andrew Barnert added the comment:

Also, I filed #25958 as an ABC equivalent to Serhiy's typehinting problem. I 
don't know if that actually needs to be solved, but that definitely takes it 
out of the way for this issue.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25958] Implicit ABCs have no means of "anti-registration"

2015-12-26 Thread Andrew Barnert

Andrew Barnert added the comment:

As Serhiy pointed out on -ideas, there's no reason `reversed` couldn't add 
special handling for `__reversed__ is None`, akin to `hash`'s special handling 
for `__hash__ is None`, to produce whatever error message we wanted in the 
`TypeError`, instead of "'NoneType' object is not callable". So, retract that 
argument against #2.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-16 Thread Andrew Barnert

Andrew Barnert added the comment:

@Swati Jaiswal:
> But the work around suggested here ...  is also not a general solution, i.e. 
> ... for any other mapping which does not have 0 as a key, it results in 
> KeyError.

You're missing the point. The workaround isn't intended to be a general 
solution to making mappings reversible. It's intended to produce the exact same 
behavior as the current design, for any code that somehow depends on that. So, 
any mapping that happens to be reversible by luck is reversible with the 
workaround; any mapping that successfully produces odd nonsense produces the 
same odd nonsense; any mapping that raises a KeyError(0) will raise the same 
KeyError(0).

In the incredibly vast majority of cases (probably 100%) you will not want that 
workaround; you will want the new behavior that raises a TypeError instead. I 
don't think the workaround needs to be mentioned in the documentation or 
anything; I just produced it to prove that, on the incredibly unlikely chance 
that the change is a problem for someone, the workaround to restore the old 
behavior is trivial.

Meanwhile, your general solution takes linear space, and linear up-front work, 
which makes it unacceptable for a general __reversed__ implementation. When you 
actually want, you can do it manually and explicitly in a one-liner, as already 
explained in the docs.

If you're still not getting this, pretend I never mentioned the workaround. It 
really doesn't matter.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-16 Thread Andrew Barnert

Andrew Barnert added the comment:

@R. David Murray:

> What is it that makes reversed raise a typeerror on dict here?

There are separate slots for tp_as_sequence and tp_as_mapping, so a C type can 
be (and generally is) one or the other, not both. 

But for Python types, anything that has __getitem__ is both a sequence and a 
mapping at the C level. (It's one of the few minor inconsistencies between C 
and Python types left, like having separate slots for nb_add and sq_concat in C 
but only __add__ for both in Python.)

> Not that we can change it at this point, but reversed blindly using len and 
> __getitem__ for user classes but not on dict is rather inconsistent.  

But it's consistent with iter blindly using __len__ and __getitem__ if __iter__ 
is not present on Python classes but not doing that on C classes. That's how 
"old-style sequences" work, and I don't think we want to get rid of those (and, 
even if we did, I'm pretty sure that would require at least a thread on -dev or 
-ideas...).

> I suppose the dict TypeError special case catches common mistakes?

Yes. That's probably not why it was implemented (it's easier for a C type to 
_not_ fake being a broken sequence than to do so), but it has that positive 
effect.

> In which case adding a __reversed__ that raises a TypeError to Mapping seems 
> to make sense for the same reason.

Exactly.

I think Raymond's point is that, while it does make sense, it may still not be 
important enough to be worth even two lines of code. Hopefully we can get more 
than two opinions (his and mine) on that question; otherwise, at least as far 
as I'm concerned, he trumps me.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25866] Reference 3. Data Model: miscellaneous minor cleanups on the word "sequence".

2015-12-15 Thread Andrew Barnert

New submission from Andrew Barnert:

None of the below are very serious or likely to mislead anyone using or 
implementing Python, but...

---

3.3.2. Customizing attribute access

The docs for `__dir__` say:

> A sequence must be returned. dir() converts the returned sequence to a list 
> and sorts it.

At least in CPython and PyPy, this isn't enforced; any iterable can be returned 
(and it's then converted to a list and sorted). You can even make `__dir__` a 
generator function. (I've never seen that in the wild--but I have seen code 
that returns a `map` iterator.)

I think it would be better to say "An iterable must be returned. dir() converts 
the returned iterable to a list and sorts it."

---

3.3.2.3. __slots__

> This class variable can be assigned a string, iterable, or sequence of 
> strings...

While it's true that you can assign any iterable, it's at best misleading to 
assign an iterator. For example, if you use `__slots__ = iter('abc')`, you will 
get a class with descriptors named `a`, `b`, and `c`, but with an empty 
iterator in `__slots__`. There's probably no reason to actually outlaw that, 
but it might be worth noting in 3.3.2.3.1 along with the other weird 
possibilities like assigning a mapping.

---

3.3.6. Emulating container types

The docs still say that the ABCs are in `collections` rather than 
`collections.abc`.

> ... for mappings, __iter__() should be the same as keys()

No; `__iter__()` should be an iterator over the keys; `keys()` _can_ be an 
iterator, but generally should be a view instead.

> The membership test operators (in and not in) are normally implemented as an 
> iteration through a sequence. However, container objects can supply the 
> following special method with a more efficient implementation, which also 
> does not require the object be a sequence.

I don't think this should say "through a sequence". They're implemented as 
iteration through whatever the container is, whether it's a sequence, a set, a 
linked list, or anything else. The documentation on `__contains__` immediately 
below clarifies this, but it's probably better to be clear from the start.

--
assignee: docs@python
components: Documentation
messages: 256445
nosy: abarnert, docs@python
priority: normal
severity: normal
status: open
title: Reference 3. Data Model: miscellaneous minor cleanups on the word 
"sequence".
type: enhancement

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25866>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-14 Thread Andrew Barnert

New submission from Andrew Barnert:

Example:

class MyDict(collections.abc.Mapping):
def __init__(self, d): self.d = d
def __len__(self): return len(self.d)
def __getitem__(self, key): return self.d[key]
def __iter__(self): return iter(self.d)

d = {1:2, 3:4}
m = MyDict({1: 2, 3: 4})

If you do `reversed(d)`, you get a nice `TypeError: argument to reversed() must 
be a sequence`. But if you do `reversed(m)`, you get a `reversed` iterator. And 
when you iterate it, presumably expecting to get 0 and 1 in some arbitrary 
order, you instead get 3, and then a `KeyError: 0`.

Of course it's obvious why this happens once you think about it: in order to 
handle types that implement the old-style sequence protocol (just respond to 
`__getitem__` for all integers from 0 to `len(self)`), `reversed` has to fall 
back to trying `__getitem__` for all integers from `len(d)-1` to 0.

If you provide a `__reversed__` method, it just calls that. Or, if you're a 
C-API mapping like `dict`, `PySequence_Check` will return false and it'll raise 
a `TypeError`. But for a Python mapping, there's no way `PySequence_Check` or 
anything else can know that you're not actually a sequence (after all, you 
implement `__getitem__` and `__len__`), so it tries to use you as one, and 
confusion results.

I think trying to fix this for _all_ possible mappings is a non-starter.

But fixing it for mappings that use `collections.abc.Mapping` is easy: just 
provide a default implementation of `collections.abc.Mapping.__reversed__` that 
just raises a `TypeError`.

I can't imagine this would break any working code. If it did, the workaround 
would be simple: just implement `def __reversed__(self): return (self[k] for k 
in reversed(range(len(self`.

--
components: Library (Lib)
messages: 256434
nosy: abarnert
priority: normal
severity: normal
status: open
title: collections.abc.Mapping should include a __reversed__ that raises 
TypeError
type: behavior

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-14 Thread Andrew Barnert

Andrew Barnert added the comment:

> It seems no more interesting or problematic than sequence argument unpacking 
> working with dictionaries, "a, b = {'one': 1, 'two': 2}".

Dictionaries (including dicts, dict subclasses, and custom Mappings) are 
iterables. People use that fact every time they write `for key in d`. So it's 
not at all problematic that they work with iterable unpacking. Especially since 
here, custom Mappings work exactly the same way as dicts, dict subclasses, 
custom Sets, iterators, and every other kind of iterable.

Dictionaries are not sequences. People never write code expecting them to be. 
So it is problematic that they work with sequence reversing. Especially since 
here, custom Mappings do _not_ work the same way as dicts, dict subclasses, 
custom Sets, iterators, and other non-sequences, all of which raise a TypeError.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25865] 7.2 Assignment statements documentation is vague and slightly misleading

2015-12-14 Thread Andrew Barnert

New submission from Andrew Barnert:

>From 
>https://docs.python.org/3/reference/simple_stmts.html#assignment-statements

> If the target list contains one target prefixed with an asterisk, called a 
> “starred” target: The object must be a sequence with at least as many items 
> as there are targets in the target list, minus one. The first items of the 
> sequence are assigned, from left to right, to the targets before the starred 
> target. The final items of the sequence are assigned to the targets after the 
> starred target. A list of the remaining items in the sequence is then 
> assigned to the starred target (the list can be empty).
> Else: The object must be a sequence with the same number of items as there 
> are targets in the target list, and the items are assigned, from left to 
> right, to the corresponding targets.

The word "sequence" is used here multiple times. But the object being assigned 
to the target list can be any iterable (including an iterator, a set, some 
weird custom iterable you invent, ...).

> If the target is a target list enclosed in parentheses or in square brackets: 
> The object must be an iterable with the same number of items as there are 
> targets in the target list, and its items are assigned, from left to right, 
> to the corresponding targets.

This doesn't mention that you can have a starred target inside the 
parenthesized or bracketed target list. More importantly, it covers the exact 
same case already covered above. There's semantics for "assignment of an object 
to a target list, optionally enclosed in parentheses or square brackets", and 
then semantics for "Assignment of an object to a single target... If the target 
is a target list enclosed in parentheses or in square brackets".

According to the grammar, a single target (no commas, parens, or brackets) is a 
target list, but according to the previous paragraphs it seems like it isn't.

Finally, there's nothing that explains what distinguishes between target list 
assignment and single target assignment. As far as I can tell, it's treated as 
a target list iff there's a comma, or there's square brackets around it 
(similar to the fact that tuples are defined by commas, but lists by square 
brackets--but without the exception for `()`); that should probably be 
documented.

There's also some needless repetition there.

I think better wording might be something like this:

> Assignment is recursively defined as follows.

> * If the target list is a comma-separated list of two or more targets, 
> optionally enclosed in parentheses, or a comma-separated list of zero or more 
> targets enclosed in square brackets:

(It might be worth having a note here pointing out that these are almost the 
same rules for non-comprehension tuple and list displays, except that () is a 
tuple but not a target list. Or would that just be confusing?)

(It might also be worth mentioning that this is the same feature referred to as 
"iterable unpacking", "sequence unpacking", and/or "tuple unpacking" in PEP 
3132, the official tutorial, and lots of non-official materials?)

> ** If the target list contains one target prefixed with an asterisk, called a 
> “starred” target: The object must be an iterable with at least as many items 
> as there are targets in the target list, minus one. The first items of the 
> iterable are assigned, from left to right, to the targets before the starred 
> target. The final items of the iterable are assigned to the targets after the 
> starred target. A list of the remaining items in the iterable is then 
> assigned to the starred target (the list can be empty).

> ** Else: The object must be an iterable with the same number of items as 
> there are targets in the target list, and the items are assigned, from left 
> to right, to the corresponding targets.

> * Otherwise, the target list is treated as a single target. 

> ** If the target is an identifier (name): ...

(No section on bracketed target list here; that's already covered above.)

> ** If the target is an attribute reference: ...

> ** ...

--
assignee: docs@python
components: Documentation
messages: 256443
nosy: abarnert, docs@python
priority: normal
severity: normal
status: open
title: 7.2 Assignment statements documentation is vague and slightly misleading

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25865>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25865] 7.2 Assignment statements documentation is vague and slightly misleading

2015-12-14 Thread Andrew Barnert

Andrew Barnert added the comment:

As a side note, why isn't () allowed as an empty target list, like []? Then the 
rules for target lists vs. single targets would be exactly parallel to the 
rules for tuple and list displays. And the error message `can't assign to ()` 
seems a bit weird--you can't really assign to the tuple `a, b, c` or `(a, b, 
c)`, but that's not what you're doing; you're just specifying a target list.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25865>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25864] collections.abc.Mapping should include a __reversed__ that raises TypeError

2015-12-14 Thread Andrew Barnert

Andrew Barnert added the comment:

> What are we exactly expecting here?

Well, naively, I was expecting a TypeError, just as you get for dict, or a 
subclass of dict, or a custom extension type that implements the C-API mapping 
protocol.

Once you understand how reversed works, you can understand why it gives you a 
nonsensical and useless iterator instead. But nobody would actually _want_ that.

So, as I proposed in the initial description, and the title, what we should be 
doing is raising a TypeError.

> How can a dictionary be reversed?

Assuming this is a pointless rhetorical question: That's exactly the point. 
There's no sensible meaning to reversing a dictionary, so it should raise a 
TypeError. Exactly as it already does for dict, subclasses of dict, and C-API 
mappings.

If this wasn't rhetorical: I guess you could argue that any arbitrary order in 
reverse is any arbitrary order, so even returning iter(m) would be acceptable. 
Or maybe reversed(list(m)) would be even better, if it didn't require O(N) 
space. But in practice, nobody will ever expect that--again, they don't get it 
from dict, subclasses, C-API mappings--so why go out of our way to implement 
it? So, again, it should raise a TypeError.

> This seems to make no difference. I still got the KeyError.

Of course. Again, the current behavior is nonsensical, will almost always raise 
a KeyError at some point, and will never be anything a reasonable person wants. 
So a workaround that restores the current behavior will also be nonsensical, 
almost always raise a KeyError at some point, and never be anything a 
reasonable person wants.

But, if you happen to be unreasonably unreasonable--say, you created a mapping 
with {2:40, 0:10, 1:20} and actually wanted reversed(m) to confusingly give you 
40, 20, 10--and this change did break your code, the workaround would restore 
it.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25864>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24113] shlex constructor unreachable code

2015-05-03 Thread Andrew Barnert

Andrew Barnert added the comment:

Presumably ESR thought that information was useful enough to send Guido a patch 
to add it. But apparently nobody's missed it in the 15 years it's been checked 
in but unreachable. If anyone needed to know the `instream` and `lineno` right 
after initialization, presumably they just printed them directly. So, yeah, 
seems unnecessary to me.

--
nosy: +abarnert

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24113
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23525] isbuiltin, isroutine, etc.

2015-02-25 Thread Andrew Barnert

New submission from Andrew Barnert:

The documentation and implementation for `inspect.isbuiltin` and related 
functions do not match. While #4968 attempted to clarify things to make the 
documentation match the behavior, the actual committed changes are still wrong.

`isbuiltin` says Return true if the object is a built-in function or a bound 
built-in method, but it returns false for bound built-in special methods 
(e.g., `[].__add__`). That's because it's implemented as 
`isinstance(types.BuiltinFunctionType)`, but bound special methods are a 
different type (`method-wrapper`). Terry Reedy's suggested fix from #4968 (... 
or a bound built-in non-special method) would fix this, but the committed 
change didn't.

The docstring has a further problem; it says that anything that passes will 
have a `__self__` with instance to which a method is bound, or None, but for 
functions, the `__self__` is usually the module the `PyMethodDef` comes from, 
not `None`. (For example, `sum.__self__` is the`'builtins` module.)

`isroutine` says Return true if the object is a user-defined or built-in 
function or method. (The docstring instead says … any kind of function or 
method.) But it returns false for built-in bound special methods 
(`[].__add__`). That's because it's implemented as `isbuiltin` or `isfunction` 
or `ismethod` or `ismethoddescriptor`; while `isbuiltin` picks up functions and 
bound non-special methods, and `ismethoddescriptor` picks up unbound special 
and non-special methods, nothing picks up bound special methods.

A doc-only fix for this, along with Terry's suggested fix for `isbuiltin`, 
would be Return true if the object is a user-defined function or method, or a 
built-in function, unbound method, or bound non-special method. That sounds 
horrible, but that's because it actually is a horrible test in the first place.

A better fix would be to make `isbuiltin` handle both kinds of bound method. 
Then `isroutine` is correct as written, and both are documented correctly. 
(Note that there is no type in `types` corresponding to `method-wrapper`, so 
either `types` or `inspect` would have to add something like 
`BuiltinSpecialMethodType = type([].__add__)`.)

It would be even better to fix this at the source and clean up the types of 
builtin functions and methods, but that boat sailed with 3.0, so...

--
assignee: docs@python
components: Documentation
messages: 236649
nosy: abarnert, docs@python
priority: normal
severity: normal
status: open
title: isbuiltin, isroutine, etc.
versions: Python 3.4, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23525
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23226] Add float linspace recipe to docs

2015-01-13 Thread Andrew Barnert

Andrew Barnert added the comment:

So something like the first version below? Or should even the example be 
inline, as in the second version below? (Apologies if the formatting gets 
screwed up pasting from docs to bugs.)

---

Range objects are inappropriate for non-integral types, especially inexact 
types such as :class:`float`.  In most cases, for such types, it is better to 
specify a count of numbers or intervals instead of a step size.  For example:

start, stop, num = 0, 10, 11
step = (stop-start)/(num-1)
[start + i*step for i in range(num)]
   [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0]

.. seealso::

   * `linspace recipe http://code.activestate.com/recipes/579000/`_
 for an example lazy sequence built on this algorithm, and links
 to alternatives.

---

Range objects are inappropriate for non-integral types, especially inexact 
types such as :class:`float`.  In most cases, it is better to specify a count 
of numbers or intervals instead of a step size, as in `[start + 
i*(stop-start)/(num-1) for i in range(num)]`.

.. seealso::

   * `linspace recipe http://code.activestate.com/recipes/579000/`_
 for an example lazy sequence built on this algorithm, and links
 to alternatives.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23226
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23226] Add float linspace recipe to docs

2015-01-12 Thread Andrew Barnert

New submission from Andrew Barnert:

In a recent thread on python-ideas 
(https://mail.python.org/pipermail/python-ideas/2015-January/030817.html), it 
was concluded that Python should not have a range-like type for floats in the 
stdlib, but there should be some simple discussion of the alternatives in the 
docs.

The attached diff describes the issues, demonstrates the simplest reasonable 
algorithm, and links to a recipe on ActiveState (which builds the same 
algorithm into a lazy sequence, and links to other recipes with different 
alternatives).

--
assignee: docs@python
components: Documentation
files: stdtypes.rst.diff
keywords: patch
messages: 233893
nosy: abarnert, docs@python
priority: normal
severity: normal
status: open
title: Add float linspace recipe to docs
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37684/stdtypes.rst.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23226
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23226] Add float linspace recipe to docs

2015-01-12 Thread Andrew Barnert

Andrew Barnert added the comment:

As suggested by the review: removing unnecessary parenthetical, changing ulp 
to digit, and fixing the recipe link.

--
Added file: http://bugs.python.org/file37685/stdtypes.rst.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23226
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22755] contextlib.closing documentation should use a new example

2014-10-28 Thread Andrew Barnert

Andrew Barnert added the comment:

It may be pretty hard to come up with a good example for closing in the stdlib 
that shouldn't just be turned into a context manager… 

There are some types where using it in a with statement is sensible, but 
unusual enough that you might want to make it explicit (maybe 
logging.Handler?), but does that make a good example?

--
nosy: +abarnert

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22755
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1152248] Add support for reading records with arbitrary separators to the standard IO stack

2014-07-27 Thread Andrew Barnert

Andrew Barnert added the comment:

Akira, your patch does this:

-self._writetranslate = newline != ''
-self._writenl = newline or os.linesep
+self._writetranslate = newline in (None, '\r', '\r\n')
+self._writenl = newline if newline is not None else os.linesep

Any reason you made the second change? Why change the value assigned to 
_writenl for newline='\n' when you don't want to actually change the behavior 
for those cases? Just so you can double-check at write time that 
_writetranslate is never set unless _writenl is '\r', '\r\n', or os.linesep?

--
versions: +Python 3.4 -Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1152248
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1152248] Add support for reading records with arbitrary separators to the standard IO stack

2014-07-20 Thread Andrew Barnert

Changes by Andrew Barnert abarn...@yahoo.com:


Added file: http://bugs.python.org/file36008/pep-newline.txt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1152248
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1152248] Add support for reading records with arbitrary separators to the standard IO stack

2014-07-20 Thread Andrew Barnert

Changes by Andrew Barnert abarn...@yahoo.com:


Added file: http://bugs.python.org/file36009/pep-peek.txt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1152248
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1152248] Add support for reading records with arbitrary separators to the standard IO stack

2014-07-19 Thread Andrew Barnert

Andrew Barnert added the comment:

http://thread.gmane.org/gmane.comp.python.ideas/28310 discusses the same idea.

Guido raised a serious problem with either adding an argument to readline and 
friends, or adding new methods readrecord and friends: It means the hundreds of 
existing file-like objects that exist today no longer meet the file API.

Putting the separator in the constructor call solves that problem. Construction 
is not part of the file API, and different file-like objects' constructors are 
already wildly different. It also seems to fit in better with what perl, awk, 
bash, etc. do (whether you either set something globally, or on the file, 
rather than on the line-reading mechanism). And it seems conceptually cleaner; 
a file shouldn't be changing line-endings in mid-stream—and if it does, that's 
similar to changing encodings.

Whether this should be done by reusing newline, or by adding another new 
parameter, I'm not sure. The biggest issue with reusing newline is that it has 
a meaning for write mode, not just for read mode (either terminal \n 
characters, or all \n characters, it's not entire clear which, are replaced 
with newline), and I'm not sure that's appropriate here. (Or, worse, maybe it's 
appropriate for text files but not binary files?)

R. David Murray's patch doesn't handle binary files, or _pyio, and IIRC from 
testing the same thing there was one more problem to fix for text files as 
well… but it's not hard to complete. If I have enough free time tomorrow, I'll 
clean up what I have and post it.

--
nosy: +abarnert

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1152248
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1152248] Add support for reading records with arbitrary separators to the standard IO stack

2014-07-19 Thread Andrew Barnert

Andrew Barnert added the comment:

While we're at it, Douglas Alan's solution wouldn't be an ideal solution even 
if it were a builtin. A fileLineIter obviously doesn't support the stream API. 
It means you end up with two objects that share the same file, but have 
separate buffers and out-of-sync file pointers. And it's a lot slower.

That being said, I think it may be useful enough to put in the stdlib—even more 
so if you pull the resplit-an-iterator-of-strings code out:

def resplit(strings, separator):
partialLine = None
for s in strings:
if partialLine:
partialLine += s
else:
partialLine = s
if not s:
break
lines = partialLine.split(separator)
partialLine = lines.pop()
yield from lines
if partialLine:
yield partialLine

Now, you can do this:

with open('rdm-example') as f:
chunks = iter(partial(f.read, 8192), '')
lines = resplit(chunks, '\0')
lines = (line + '\n' for line in lines)

# Or, if you're just going to strip off the newlines anyway:
with open('file-0-example') as f:
chunks = iter(partial(f.read, 8192), '')
lines = resplit(chunks, '\0')

# Or, if you have a binary file:
with open('binary-example, 'rb') as f:
chunks = iter(partial(f.read, 8192), b'')
lines = resplit(chunks, b'\0')

# Or, if I understand ysj.ray's example:
with open('ysj.ray-example') as f:
chunks = iter(partial(f.read, 8192), '')
lines = resplit(chunks, '\r\n')
records = resplit(lines, '\t')

# Or, if you have something that isn't a file at all:
lines = resplit((packet.body for packet in packets), '\n')

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1152248
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1152248] Add support for reading records with arbitrary separators to the standard IO stack

2014-07-19 Thread Andrew Barnert

Andrew Barnert added the comment:

One last thing, a quick  dirty solution that works today, if you don't mind 
accessing private internals of stdlib classes, and don't mind giving up the 
performance of _io for _pyio, and don't need a solution for binary files:

class MyTextIOWrapper(_pyio.TextIOWrapper):
def readrecord(self, sep):
readnl, self._readnl = self._readnl, sep
try:
return self.readline()
finally:
self._readnl = readnl

Or, if you prefer:

class MyTextIOWrapper(_pyio.TextIOWrapper):
def __init__(self, *args, separator, **kwargs):
super().__init__(*args, **kwargs)
self._readnl = separator

For binary files, there's no solution quite as simple; you need to write your 
own readline method by copying and pasting the one from _pyio.RawIOBase, and 
the modifications to use an arbitrary separator aren't quite as trivial as they 
look at first (at least if you want multi-byte separators).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1152248
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22004] io documentation refers to newline as newlines

2014-07-18 Thread Andrew Barnert

New submission from Andrew Barnert:

In at least one place in the io module documentation (io.IOBase.readline), and 
in the corresponding docstring, the newline parameter to open (and io.open, and 
io.Foo.__init__) is referred to as newlines:

 The line terminator is always b'\n' for binary files; for text files, 
 the newlines argument to open() can be used to select the line 
 terminator(s) recognized.

(The newline parameter is closely related to the newlines attribute of the 
TextIOWrapper that gets created by the open call, but they're not the same 
thing, and I think were named differently intentionally.)

--
assignee: docs@python
components: Documentation
messages: 223398
nosy: abarnert, docs@python
priority: normal
severity: normal
status: open
title: io documentation refers to newline as newlines
type: behavior
versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22004
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22004] io documentation refers to newline as newlines

2014-07-18 Thread Andrew Barnert

Andrew Barnert added the comment:

Searching the source and the help page, it looks like the one example I gave is 
the only place it's wrong in each of the two, not one of multiple places.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22004
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1820] Enhance Object/structseq.c to match namedtuple and tuple api

2014-06-11 Thread Andrew Barnert

Andrew Barnert added the comment:

Hi, Stephan. Sorry, for some reason Yahoo was sending updates from the tracker 
to spam again, so I missed this. I'd be glad to sign a contributor agreement if 
it's still relevant, but it looks like there's a later patch that does what 
mine did and more?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1820
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1820] Enhance Object/structseq.c to match namedtuple and tuple api

2014-06-11 Thread Andrew Barnert

Andrew Barnert added the comment:

Sorry, Stefan, not Stephan. Anyway, I've signed the agreement.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1820
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20250] defaultdict docstring neglects the *args

2014-01-13 Thread Andrew Barnert

New submission from Andrew Barnert:

The docstring for defaultdict shows only a single argument, default_factory, 
and gives no clue that you can pass additional arguments:

|  defaultdict(default_factory) -- dict with default factory
|
|  The default factory is called without arguments to produce
|  a new value when a key is not present, in __getitem__ only.
|  A defaultdict compares equal to a dict with the same items.

The docs, by contrast, say:

 class collections.defaultdict([default_factory[, ...]])

 … The first argument provides the initial value for the default_factory 
 attribute; it defaults to None. All remaining arguments are treated the same 
 as if they were passed to the dict constructor, including keyword arguments.

I think the docstring should have the same information—or at least have the 
correct signature, which would hopefully be enough to prompt people to look at 
the docs.

I'm not sure whether the correct signature is (default_factory, *args, 
**kwargs), like OrderedDict, or (default_factory[, ...]), like deque, but I 
don't think either one would confuse anyone.

--
components: Library (Lib)
files: collectionsmodule.diff
keywords: patch
messages: 208071
nosy: abarnert
priority: normal
severity: normal
status: open
title: defaultdict docstring neglects the *args
type: enhancement
Added file: http://bugs.python.org/file33456/collectionsmodule.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20250
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20230] structseq types should expose _fields

2014-01-12 Thread Andrew Barnert

New submission from Andrew Barnert:

A PyStructSequence type doesn't expose its field names in any way to Python 
code (except indirectly, via the repr, which you could parse if you really 
wanted to…).

The docs claim that Struct sequence objects are the C equivalent of 
namedtuple() objects, so it seems like the obvious way to expose them is the 
exact same way namedtuple types do—as a class attribute named _fields.

Also, in #7796, and in a recent thread on python-ideas, multiple people 
suggested that named tuple is a protocol, not a type, and duck typing on 
_fields is the obvious way to detect types matching that protocol.

Only the sequence fields need to be exposed—fields that don't match up to 
sequence indices aren't named tuple fields, they're just extra instance 
attributes (that don't even show up in the repr).

Patch included. For the tests, I tested the fields in struct_time and 
stat_result in effectively the same way test_repr tests their repr, so it 
should pass on all the same platforms/builds.

--
files: structseq.diff
keywords: patch
messages: 207991
nosy: abarnert
priority: normal
severity: normal
status: open
title: structseq types should expose _fields
type: enhancement
Added file: http://bugs.python.org/file33436/structseq.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20230
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue7796] No way to find out if an object is an instance of a namedtuple

2014-01-12 Thread Andrew Barnert

Andrew Barnert added the comment:

I believe the structseq issues are a lot easier to solve than the appear. See 
#20230, which adds a _fields member to every structseq type. Having done that, 
a NamedTuple ABC works on them just fine. And of course duck typing them 
without an ABC does too.

--
nosy: +abarnert

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7796
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20230] structseq types should expose _fields

2014-01-12 Thread Andrew Barnert

Andrew Barnert added the comment:

After more searching (I should have searched on structseq instead of 
PyStructSequence…), this might be a dup of #1820. If so, apologies.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20230
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1820] Enhance Object/structseq.c to match namedtuple and tuple api

2014-01-12 Thread Andrew Barnert

Andrew Barnert added the comment:

See issue20230, which includes a patch implementing just the first part of part 
2 (adding _fields, but not _asdict or _replace).

Since this one has been open for 5+ years, if it's not going to be done soon, 
any chance of the easy (_fields) change being committed and putting off the 
harder parts until later?

--
nosy: +abarnert

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1820
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue19019] Investigate using Apple clang for building OS X installers

2013-10-24 Thread Andrew Barnert

Andrew Barnert added the comment:

Now that Xcode 5 is out, and does not have anything named gcc-4.2 (it _does_ 
still have something called gcc, which is now a wrapper around clang instead 
of a wrapper around llvm-gcc-4.2), this seems a lot more critical, because (as 
far as I can tell) nobody with Xcode 5 can build C extensions for python.org 
Python installations.

And that also implies that any changes need to be backported to 2.7 and 3.3, 
unless the plan is that only Python 3.4+ should work with Xcode 5.

--
nosy: +abarnert

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19019
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com