Re: [Python-Dev] for...else

[Isaac Morland]

The way I remember it is to observe that the following are *almost* exactly
the same thing:

if C:
T
else:
E

while C:
T
else:
E

The *only* differences are:

1) where execution jumps if it reaches the end of the T: in the "while", it
jumps back to the while itself, resulting in the condition being rechecked,
whereas in the "if" execution skips over the "else" to whatever follows; and

2) in the "while", inside the T "break" and "continue" relate to this
control structure rather than to a containing loop.

(At least I don't think I'm missing any other differences!)

Seen this way, the meaning of the "else" is easy to understand and to
remember.

And the for loop else is like the while loop else.


On 24 July 2017 at 11:35, Kiuhnm via Python-Dev <python-dev@python.org>
wrote:

> Hello,
>
> I think that the expression "for...else" or "while...else" is completely
> counter-intuitive. Wouldn't it be possible to make it clearer? Maybe
> something like
>
> break in for i in range(n):
>   ...
>   if cond:
> break
> else:
>   ...
>
> I'm not an English native speaker so I don't know whether "break in" is
> acceptable English in this context or can only mean "to get into a building
> by force".
>
> Kiuhnm
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/isaac.
> morland%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Impact of Namedtuple on startup time

[Isaac Morland]

My apologies, I misunderstood what had been proposed (and rejected).

So it sounds like the _source is a pre-requisite for the current exec-based
implementation, but the proposal is to replace with a non-exec-based
implementation, meaning _source would no longer be needed for the module to
work and might be eliminated. But _source could continue to be generated
lazily (and cached if thought helpful) using an @property, so even the
(apparently rare) uses of _source would continue to work.

This would in some sense be a DRY violation, but of a very pragmatic
Pythonic sort, where we have two implementations, one for documentation and
one for efficiency. How different would this be from all those modules that
have both Python and C implementations?


On 17 July 2017 at 09:31, Antoine Pitrou <anto...@python.org> wrote:

>
> Le 17/07/2017 à 15:26, Isaac Morland a écrit :
> >
> > I think I understand well enough to say something intelligent…
> >
> > While actual references to _source are likely rare (certainly I’ve never
> > used it), my understanding is that the way namedtuple works is to
> > construct _source, and then exec it to create the class. Once that is
> > done, there is no significant saving to be had by throwing away the
> > constructed _source value.
>
> The proposed resolution on https://bugs.python.org/issue28638 is to
> avoid exec() on most parts of the namedtuple class, hence speeding up
> the class creation.
>
> > I come from
> > a non-Pythonic background so use of exec still feels a bit weird to me
> > but I absolutely love namedtuple and use it constantly.
>
> I think for most Python programmers, it still feels a bit un-Pythonic.
> While exec() is part of Python, it's generally only used in fringe cases
> where nothing else works.
>
> Regards
>
> Antoine.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> isaac.morland%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Impact of Namedtuple on startup time

[Isaac Morland]

On 17 July 2017 at 08:43, Antoine Pitrou  wrote:

>
> Hello,
>
> Cost of creating a namedtuple has been identified as a contributor to
> Python startup time.  Not only Python core and the stdlib, but any
> third-party library creating namedtuple classes (there are many of
> them).  An issue was created for this:
> https://bugs.python.org/issue28638
>
> Raymond decided to close the issue because:
>
> 1) the proposed resolution makes the "_source" attribute empty (or, at
> least, something else than it currently is).  Raymond claims the
> "_source" attribute is an essential feature of namedtuples.
>

I think I understand well enough to say something intelligent…

While actual references to _source are likely rare (certainly I’ve never
used it), my understanding is that the way namedtuple works is to construct
_source, and then exec it to create the class. Once that is done, there is
no significant saving to be had by throwing away the constructed _source
value.

When namedtuple was being considered for inclusion, I actually went so far
as to write a proof-of-concept version that worked by creating a class,
creating attributes on it, etc. I don’t remember how far I got but the exec
version is the version included in the stdlib. I come from a non-Pythonic
background so use of exec still feels a bit weird to me but I absolutely
love namedtuple and use it constantly. I don't know whether a polished and
completed version of my idea could be faster than using exec, but I
wouldn't expect a major saving — a whole bunch of code has to run either
way.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does base64 return bytes?

[Isaac Morland]

On Wed, 15 Jun 2016, Greg Ewing wrote:


Simon Cross wrote:

If we only support one, I would prefer it to be bytes since (bytes ->
bytes -> unicode) seems like less overhead and slightly conceptually
clearer than (bytes -> unicode -> bytes),


Whereas bytes -> unicode, followed if needed by unicode -> bytes,
seems conceptually clearer to me. IOW, base64 is conceptually a
bytes-to-text transformation, and the usual way to represent
text in Python 3 is unicode.


And in CPython, do I understand correctly that the output text would be 
represented using one byte per character?  If so, would there be a way of 
encoding that into UTF-8 that re-used the raw memory that backs the 
Unicode object?  And, therefore, avoids almost all the inefficiency of 
going via Unicode?  If so, this would be a win - proper use of Unicode to 
represent a text string, combined with instantaneous conversion into a 
bytes object for the purpose of writing to the OS.


Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)

[Isaac Morland]

On Tue, 12 Apr 2016, Jon Ribbens wrote:


This is still a massive game of whack-a-mole.


No, it still isn't. If the names blacklist had to keep being extended
then you would be right, but that hasn't happened so far. Whitelists
by definition contain only a small, limited number of potential moles.

The only thing you found above that even remotely approaches an
exploit is the decimal.getcontext() thing, and even that I don't
think you could use to do any code execution.


"I don't think"?

Where's the formal proof?

Without a proof, this is indeed just a game of whack-a-mole.

I don't "think" Python is a suitable foundation for a sandboxing system 
intended for security purposes, but my "think" won't lead to security 
holes whereas yours will.  So, I would respectfully suggest that unless 
you increase the rigour of your effort substantially, it is not 
worthwhile.  Python is great for lots of applications already - there is 
no need to force it into unsuitable problem domains.


Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)

[Isaac Morland]

On Mon, 11 Apr 2016, Victor Stinner wrote:


2016-04-10 18:43 GMT+02:00 Jon Ribbens <jon+python-...@unequivocal.co.uk>:


That's the opposite of my approach though - I'm starting small and
adding things, not starting with everything and removing stuff. Even
if what we end up with is an extremely restricted subset of Python,
there are still cases where that could be a useful tool to have.


You design rely on the assumption that CPython is only pure Python.
That's wrong. A *lot* of Python features are implemented in C and
"ignore" your sandboxing code. Quick reminder: 50% of CPython is
written in the C language.

It means that your protections like hiding builtin functions from the
Python scope don't work. If an attacker gets access to a C function
giving access to the hidden builtin, the game is over.

[]

Non-Python core developer, non-expert-specifically-in-computer-security 
here, so won't take up much room on this list.


I know enough about almost everything in Computer Science to know just how 
ignorant I am about almost everything in Computer Science.


But I would not use for security purposes a Python sandbox that was not 
formally verified to be correct and unbreakable.  Of course in order for 
this to be possible, there first has to be a formal semantics for Python. 
Has anybody made a formal semantics for Python?  If not, then this project 
is missing a pretty important pre-requisite.


Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third milestone of FAT Python

[Isaac Morland]

On Fri, 4 Dec 2015, MRAB wrote:


Constant folding is when, say, "1 + 2" replaced by "2".


Isn't that called backspacing?  ;-)

Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

[Isaac Morland]

On Mon, 20 Apr 2015, Paul Moore wrote:


On 20 April 2015 at 19:41, Barry Warsaw ba...@python.org wrote:

tldr; type hints in python source are scary. Would reserving them for stub
files be better?


I think so.  I think PEP 8 should require stub files for stdlib modules and
strongly encourage them for 3rd party code.


Agreed. I have many of the same concerns as Harry, but I wouldn't have
expressed them quite as well. I'm not too worried about actually
removing annotations from the core language, but I agree that we
should create a strong culture of type hints go in stub files to
keep source files readable and clean.

On that note, I'm not sure stub files is a particularly good name.
Maybe type files would be better? Something that emphasises that
they are the correct place to put type hints, not a workaround.


How about header files?

(ducks...)

Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Emit SyntaxWarning on unrecognized backslash escapes?

[Isaac Morland]

On Tue, 24 Feb 2015, Ethan Furman wrote:


On 02/24/2015 10:49 AM, Guido van Rossum wrote:

On Tue, Feb 24, 2015 at 10:47 AM, Ethan Furman wrote:


I can attest from my impoverished Windows programming days that looking at

  -- os.listdir('c:\temp')
  SomeErrorMessage about syntax 'c:\temp'

is not very helpful.  There is zero visual indication that the \ and the t
are one character, not two.  Changing that error message to:

  SomeErrorMessage about syntax 'c:[\t]emp'

or

  SomeErrorMessage about syntax 'c:\x07emp'

or something that shouts out, hey!  one character in this 
location! would be a good thing.


I like the \x07 solution.


So final question is do we take the easy road and change the repr for 
str to always use hex or unicode escapes instead of simple 
backslash-escapes (and then run for our lives), or do we just hunt down 
and change the appropriate error messages for files (and possibly re) ?


Just a data point from a random programmer:

I like the \x07 solution for the error message as it draws attention to 
the character at issue, but I also like to see \n, \t etc. in the result 
of repr because it is more readable for the common cases.


Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] surrogatepass - she's a witch, burn 'er! [was: Cleaning up ...]

[Isaac Morland]

On Fri, 29 Aug 2014, M.-A. Lemburg wrote:


On 29.08.2014 02:41, Stephen J. Turnbull wrote:
Since Python allows working with lone surrogates in Unicode (they
are valid code points) and we're using UTF-8 for marshal, we needed
a way to make sure that Python 3 also optionally supports working
with lone surrogates in such UTF-8 streams (nowadays called CESU-8:
http://en.wikipedia.org/wiki/CESU-8).


If I want that wouldn't I specify cesu-8 as the encoding?

i.e., instead of .decode ('utf-8') I would use .decode ('cesu-8').  Right 
now, trying this I get that cesu-8 is an unknown encoding but that could 
be changed without affecting the behaviour of the utf-8 codec.


It seems to me that .decode ('utf-8') should decode exactly and only valid 
utf-8, including the non-use of surrogate pairs as an intermediate 
encoding step.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path support

[Isaac Morland]

On Sat, 23 Aug 2014, Marko Rauhamaa wrote:


Isaac Morland ijmor...@uwaterloo.ca:


 HTTP/1.1 200 OK
 Content-Type: text/html; charset=ISO-8859-1

 !DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01 Transitional//EN
 html
 head
 meta http-equiv=Content-Type content=text/html; charset=utf-16


For HTML it's not quite so bad.  According to the HTML 4 standard:
[...]

The Content-Type header takes precedence over a meta element. I
thought I read once that the reason was to allow proxy servers to
transcode documents but I don't have a cite for that. Also, the meta
element must only be used when the character encoding is organized
such that ASCII-valued bytes stand for ASCII characters so the
initial UTF-16 example wouldn't be conformant in HTML.


That's not how I read it:

  The META declaration must only be used when the character encoding is
  organized such that ASCII characters stand for themselves (at least
  until the META element is parsed). META declarations should appear as
  early as possible in the HEAD element.

  URL: http://www.w3.org/TR/1998/REC-html40-19980424/charset.ht
  ml#doc-char-set

IOW, you must obey the HTTP character encoding until you have parsed a
conflicting META content-type declaration.



From the same document:


--
To sum up, conforming user agents must observe the following priorities 
when determining a document's character encoding (from highest priority to 
lowest):


An HTTP charset parameter in a Content-Type field.
A META declaration with http-equiv set to Content-Type and a value 
set for charset.
The charset attribute set on an element that designates an external 
resource. 
--


(In the original they are numbered)

This is a priority list - if the Content-Type header gives a charset, it 
takes precedence, and all other sources for the encoding are ignored.  The 
charset= on an img or similar is only used if it is the only source 
for the encoding.


The at least until the META element is parsed bit allows for the use of 
encodings which make use of shifting.  So maybe they start out 
ASCII-compatible, but after a particular shift byte is seen those bytes 
now stand for Japanese Kanji characters until another shift byte is seen. 
This is allowed by the specification, as long as none of the 
non-ASCII-compatible stuff is seen before the META element.



The author of the standard keeps a straight face and continues:


I like your way of putting this - straight face indeed.  The third 
option really is a hack to allow working around nonsensical situations 
(and even the META tag is pretty questionable).  All this complexity 
because people can't be bothered to do things properly.



  For cases where neither the HTTP protocol nor the META element
  provides information about the character encoding of a document, HTML
  also provides the charset attribute on several elements. By combining
  these mechanisms, an author can greatly improve the chances that,
  when the user retrieves a resource, the user agent will recognize the
  character encoding.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path support

[Isaac Morland]

On Sat, 23 Aug 2014, Marko Rauhamaa wrote:


Stephen J. Turnbull step...@xemacs.org:


Just read as bytes and decode piecewise in one way or another. For
Oleg's HTML case, there's a well-understood structure that can be used
to determine retry points


HTML and XML are interesting examples since their encoding is initially
unknown:

 ?xml version=1.0?
 ^
 +--- Now I know it is UTF-8

 ?xml version=1.0 encoding=UTF-16?
 ^
 +--- Now I know it was UTF-16
  all along!

Then we have:


 HTTP/1.1 200 OK
 Content-Type: text/html; charset=ISO-8859-1

 !DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01 Transitional//EN
 html
 head
 meta http-equiv=Content-Type content=text/html; charset=utf-16

See how deep you have to parse the TCP stream before you realize the
content encoding is UTF-16.


For HTML it's not quite so bad.  According to the HTML 4 standard:

http://www.w3.org/TR/html4/charset.html

The Content-Type header takes precedence over a meta element.  I thought 
I read once that the reason was to allow proxy servers to transcode 
documents but I don't have a cite for that.  Also, the meta element 
must only be used when the character encoding is organized such that 
ASCII-valued bytes stand for ASCII characters so the initial UTF-16 
example wouldn't be conformant in HTML.


In HTML 5 it allows non-ASCII-compatible encodings as long as U+FEFF (byte 
order mark) is used:


http://www.w3.org/TR/html-markup/syntax.html#encoding-declaration

Not sure about XML.

Of course this whole area is a bit of an arms race between programmers 
competing to get away with being as sloppy as possible and other 
programmers who have to deal with their mess.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path support

[Isaac Morland]

On Thu, 21 Aug 2014, Chris Barker wrote:


so they are just byte strings, oh, except that you can't have a  null, and
the slash had better be code 47 (and vice versa). How is that different
than bytes-in-some-arbitrary-encoding-where-at-least
the-slash-character-is-ascii-compatible?


Actually, slash doesn't need to be code 47.  But no matter what code 47 
means outside of the context of a filename, it is the path arc separator 
byte (not character).


In fact, this isn't even entirely academic.  On a Mac OS X machine, go 
into Finder and try to create a directory called :.  You'll get an error 
saying 'The name “:” can’t be used.'.  Now create a directory called /. 
No problem, raising the question of what is going on at the filesystem 
level?


Answer:

$ ls -al
total 0
drwxr-xr-x   3 ijmorlan  staff   102 21 Aug 18:57 ./
drwxr-xr-x+ 80 ijmorlan  staff  2720 21 Aug 18:57 ../
drwxr-xr-x   2 ijmorlan  staff68 21 Aug 18:57 :/

And of course in shell one would remove the directory with this:

rm -rf :

not:

rm -rf /

So in effect the file system path arc encoding on Mac OS X is UTF-8 
*except* that : is outlawed and / is encoded as \x3A rather than the usual 
\x2F.  Of course, the path arc separator byte (not character) remains \x2F 
as always.


Just for fun, there are contexts in which one can give a full path at the 
GUI level, where : is used as the path separator.  This is for historical 
reasons and presumably is the reason for the above-noted behaviour.


I think the real tension here is between the POSIX level where filenames 
are byte strings (except for \x00, which is reserved for string 
termination) where \x2F has special interpretation, and absolutely every 
application ever written, in every language, which wants filenames to be 
character strings.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reviving restricted mode?

[Isaac Morland]

On Mon, 11 Aug 2014, Skip Montanaro wrote:


On Mon, Aug 11, 2014 at 12:42 PM, matsjoyce matsjo...@gmail.com wrote:

There maybe some holes in my approach, but I can't find them.


There's the rub. Given time, I suspect someone will discover a hole or two.


Schneier's Law:

Any person can invent a security system so clever that she or he can't
think of how to break it.

While I would not claim a Python sandbox is utterly impossible, I'm 
suspicious that the whole consenting adults approach in Python is 
incompatible with a sandbox.  The whole idea of a sandbox is to absolutely 
prevent people from doing things even if they really want to and know what 
they are doing.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reviving restricted mode?

[Isaac Morland]

On Thu, 14 Aug 2014, Steven D'Aprano wrote:


On Thu, Aug 14, 2014 at 02:26:29AM +1000, Chris Angelico wrote:

On Wed, Aug 13, 2014 at 11:11 PM, Isaac Morland ijmor...@uwaterloo.ca wrote:

While I would not claim a Python sandbox is utterly impossible, I'm
suspicious that the whole consenting adults approach in Python is
incompatible with a sandbox.  The whole idea of a sandbox is to absolutely
prevent people from doing things even if they really want to and know what
they are doing.


The point of a sandbox is that I, the consenting adult writing the
application in the first place, may want to allow *untrusted others* to
call Python code without giving them control of the entire application.
The consenting adults rule applies to me, the application writer, not
them, the end-users, even if they happen to be writing Python code. If
they want unrestricted access to the Python interpreter, they can run
their code on their own machine, not mine.


Yes, absolutely, and I didn't mean to contradict what you are saying. 
What I am suggesting is that the basic design of Python isn't a good 
starting point for imposing mandatory restrictions on what code can do. 
By contrast, take something like Safe Haskell.  I'm not absolutely certain 
that it really is safe as promised, but it's starting from a very 
different language in which the compiler performs extremely sophisticated 
type checking and simply won't compile programs that don't work within the 
type system.


This isn't a knock on Python (which I love using, by the way), just being 
realistic about what the existing language is likely to be able to 
support.  Having said that, I'll be very interested if somebody does come 
up with a restricted mode Python that is widely accepted as being secure - 
that would be a real achievement.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 463: Exception-catching expressions

[Isaac Morland]

On Sun, 23 Feb 2014, Nick Coghlan wrote:


Note that mandatory parentheses means we can duck the precedence
question entirely, which I count as another point in favour of
requiring them :)


Careful, if you take that too far then Python 4 will have to be Scheme. ;-)

Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] python 3 niggle: None 1 raises TypeError

[Isaac Morland]

On Fri, 14 Feb 2014, Chris Barker wrote:


This is actually a pretty common case -- the question here is: Is this
really None? or does None have a special meaning. It sounds like this is a
wrapper for a PostgreSQL range object, in which case None isn't really
right, there must be a +-infinity concept there -- how does postgress handle
it? (is this a generic range, or a datetime range? I'm not up on it).
If this means what I think it does, None is really a hack, both because of
al lteh specia case code required for sorting, comparing, etc, but also
because it's missing information -- the two ends of the range should really
have different meanings.


Postgres range types are a separate kind of type which can be created 
using CREATE TYPE:


http://www.postgresql.org/docs/current/interactive/sql-createtype.html

Several handy range types are built in, although not as many as one might 
expect.


Postgres ranges can be made infinite by specifying NULL as a bound. 
There isn't a separate special value for this.  This has somewhat strange 
effects when the base type of the range has its own infinity value:


= select upper_inf (daterange (current_date, 'infinity'));
 upper_inf
---
 f
(1 row)

= select isfinite ('infinity'::date);
 isfinite
--
 f
(1 row)

= select upper_inf (daterange (current_date, null));
 upper_inf
---
 t
(1 row)

In other words, a range with non-NULL bounds is finite according to the 
range functions even if the bound is the infinity value of the base type.


Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 461 - Adding % and {} formatting to bytes

[Isaac Morland]

On Wed, 15 Jan 2014, Antoine Pitrou wrote:


On Wed, 15 Jan 2014 15:47:43 + (UTC)
Neil Schemenauer n...@arctrix.com wrote:


Objects that implement __str__ can also implement __bytes__ if they
can guarantee that ASCII characters are always returned, no matter
what the *value*


I think that's a slippery slope. __bytes__ should mean that the object
has a well-known bytes equivalent or encoding, not that its __str__
happens to be pure ASCII.


+1


(for example, it would be fine for a HTTP message class to define a
__bytes__ method)

Also, consider that if e.g. float had a __bytes__ method, then
bytes(2.0) would start returning b'2.0', while bytes(2) would still
need to return b'\x00\x00'.


Not actually suggesting the following for a number of reasons including
but not limited to the consistency of floating point formats across
different implementations, but it would make more sense for bytes (2.0) to
return the 8-byte IEEE representation than for it to return the ASCII
encoding of the decimal representation of the number.

Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python3 complexity (was RFC: PEP 460: Add bytes...)

[Isaac Morland]

On Wed, 8 Jan 2014, Kristján Valur Jónsson wrote:


Believe it or not, sometimes you really don't care about encodings.


Sometimes you just want to parse text files.  Python 3 forces you to 
think about abstract concepts like encodings when all you want is to 
open that .txt file on the drive and extract some phone numbers and 
merge in some email addresses.  What encoding does the file have?  Do I 
care?  Must I care?


Mostly staying out of this, but I need to say something here.

If you don't know what encoding the file has, you don't know what bytes 
correspond to phone numbers.  So yes, you must care, or else you simply 
cannot write your code.


Of course, in practice, it's probably encoded in an ASCII-compatible 
encoding, so '0' encodes as the single byte 0x30.  Whether it's UTF-8, 
ISO-8859-1, or something else that is ASCII-compatible doesn't really 
matter.


So, as a practical matter, you can just use ISO-8859-1, even though in 
principal this is totally wrong.  Then ASCII is one byte per character as 
you expect, and all other bytes will round-trip unchanged.  Just don't do 
any non-trivial processing on non-ASCII characters.


I don't see how it could be made any simpler without going back to making 
it easy for people to pretend the issue doesn't exist at all and bringing 
back the attendant confusion and problems.


I have lots of little utilities, to help me with day to day stuff like 
this.  One fine morning I decided to start usnig Python 3 for the job. 
Imagine my surprise when it turned out to make my job more complicated, 
not easier.  Suddenly I had to start thining about stuff that hadn't 
mattered at all, and still didn't really matter.  All it did was 
complicate things for no benefit.

[]


All of this talk is positive, though.  The fact that these topics have 
finally reached the halls of python-dev are indication that people out 
there are _trying_ to move to 3.3 :)


Agreed.

Isaac Morland   CSCF Web Guru
DC 2619, x36650 WWW Software Specialist___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] python symbolizition

[Isaac Morland]

On Fri, 14 Jun 2013, R. David Murray wrote:


This discussion is better suited for python-ideas than it
is for python-dev.


Better yet, perl-ideas.


On Fri, 14 Jun 2013 16:50:53 +0200, Markus Unterwaditzer 
mar...@unterwaditzer.net wrote:

But why?

-- Markus (from phone)

Pynix Wang pynix.w...@gmail.com wrote:

1.lambda expression

c#
a.(x, y) = x == y
b.() = SomeMethod()

ruby:
- {|msg| puts msg}

python can use c# like and remove lambda keyword.

2.global variable

ruby
$glo_var

python can use $ or @ or another and remove global.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

[Isaac Morland]

On Thu, 25 Apr 2013, Lennart Regebro wrote:


On Thu, Apr 25, 2013 at 4:22 PM, MRAB pyt...@mrabarnett.plus.com wrote:

The JSON specification says that it's text. Its string literals can
contain Unicode codepoints. It needs to be encoded to bytes for
transmission and storage, but JSON itself is not a bytestring format.


OK, fair enough.


base64 is a way of encoding binary data as text.


It's a way of encoding binary data using ASCII. There is a subtle but
important difference.


It is a way of encoding arrays of 8-bit bytes as arrays of characters that 
are part of the printable, non-whitespace subset of the ASCII repertoire. 
Since the ASCII repertoire is now simply the first 128 code points in the 
Unicode repertoire, it is equally correct to say that base64 is a way of 
encoding binary data as Unicode text.



In Python 3 we're trying to stop mixing binary data (bytestrings) with
text (Unicode strings).


Yup. And that's why a byte64 encoding shouldn't return Unicode strings.


That is exactly why it should return Unicode strings.  What bytes should 
get sent if base64 is used to send a byte array over an EBCDIC link? [*]


Having said that, there may be other reasons for base64 encoding to return 
bytes - I can conceive of arguments involving efficiency, or practicality, 
or the most common use cases.  So I can't say for sure what base64 
encoding actually ought to return in Python.  But the purist stance should 
be that base64 encoding should return text, i.e. a string, i.e. unicode.


[*] I apologize to anybody who just ate.

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Updated PEP 362 (Function Signature Object)

[Isaac Morland]

On Wed, 6 Jun 2012, Nick Coghlan wrote:


2. Signature.bind introduces the ability to split the bind arguments
to parameters operation from the call object operation


Has anybody considered calling bind __call__?  That is, the result of 
calling the signature of a procedure instead of the procedure itself is 
the locals() dictionary the procedure would start with (except presumably 
missing non-parameter local variables).


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library

[Isaac Morland]

On Sat, 11 Feb 2012, Steven D'Aprano wrote:

Arguably, the canonical test for whether a package is provisional or not 
should be the existence of __provisional__:



for package in packages:
   if hasattr(package, '__provisional__')
   assert package documentation includes boilerplate
   else:
   assert package documentation does not includes boilerplate


Could the documentation generator simply insert the boilerplate if and 
only if the package has the __provisional__ attribute?  I'm not an expert 
in Python documentation but isn't it generated from properly-formatted 
comments within the Python source?


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 393 Summer of Code Project

[Isaac Morland]

On Thu, 25 Aug 2011, Guido van Rossum wrote:


I'm not sure what should happen with UTF-8 when it (in flagrant
violation of the standard, I presume) contains two separately-encoded
surrogates forming a valid surrogate pair; probably whatever the UTF-8
codec does on a wide build today should be good enough. Similarly for
encoding to UTF-8 on a wide build if one managed to create a string
containing a surrogate pair. Basically, I'm for a
garbage-in-garbage-out approach (with separate library functions to
detect garbage if the app is worried about it).


If it's called UTF-8, there is no decision to be taken as to decoder 
behaviour - any byte sequence not permitted by the Unicode standard must 
result in an error (although, of course, *how* the error is to be reported 
could legitimately be the subject of endless discussion).  There are 
security implications to violating the standard so this isn't just 
legalistic purity.


Hmmm, doesn't look good:

Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type help, copyright, credits or license for more information.

'\xed\xb0\x80'.decode ('utf-8')

u'\udc00'




Incorrect!  Although this is a narrow build - I can't say what the wide 
build would do.


For reasons of practicality, it may be appropriate to provide easy access 
to a CESU-8 decoder in addition to the normal UTF-8 decoder, but it must 
not be called UTF-8.  Other variations may also find use if provided.


See UTF-8 RFC: http://www.ietf.org/rfc/rfc3629.txt

And CESU-8 technical report: http://www.unicode.org/reports/tr26/

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] more timely detection of unbound locals

[Isaac Morland]

On Mon, 9 May 2011, Eli Bendersky wrote:


It's a known Python gotcha (*) that the following code:

x = 5
def foo():
   print(x)
   x = 1
   print(x)
foo()

Will throw:

  UnboundLocalError: local variable 'x' referenced before assignment

On the usage of 'x' in the *first* print. Recently, while reading the
zillionth question on StackOverflow on some variation of this case, I
started thinking whether this behavior is desired or just an implementation
artifact.

IIUC, the reason it behaves this way is that the symbol table logic goes
over the code before the code generation runs, sees the assignment 'x = 1`
and marks 'x' as local in foo. Then, the code generator generates LOAD_FAST
for all loads of  'x' in 'foo', even though 'x' is actually bound locally
after the first print. When the bytecode is run, since it's LOAD_FAST and no
store was made into the local 'x', ceval.c then throws the exception.

On first sight, it's possible to signal that 'x' truly becomes local only
after it's bound in the scope (and before that LOAD_NAME can be generated
for it instead of LOAD_FAST). To do this, some modifications to the symbol
table creation and usage are required, because we can no longer say x is
local in this block, but rather should attach scope information to each
instance of x. This has some overhead, but it's only at the compilation
stage so it shouldn't have a real effect on the runtime of Python code. This
is also less convenient and clean than the current approach - this is why
I'm wondering whether the behavior is an artifact of the implementation.


x = 5
def foo ():
print (x)
if bar ():
x = 1
print (x)

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] more timely detection of unbound locals

[Isaac Morland]

On Mon, 9 May 2011, Eli Bendersky wrote:


x = 5
def foo ():
   print (x)
   if bar ():
   x = 1
   print (x)



I wish you'd annotate this code sample, what do you intend it to
demonstrate?

It probably shows the original complaint even more strongly. As for being a
problem with the suggested solution, I suppose you're right, although it
doesn't make it much different. Still, before a *possible* assignment to
'x', it should be loaded as LOAD_NAME since it was surely not bound as
local, yet.


Extrapolating from your suggestion, you're saying before a *possible* 
assignment it will be treated as global, and after a *possible* assignment 
it will be treated as local?


But surely:

print (x)
if False:
x = 1
print (x)

should always print the same thing twice (in the absence of actions taken 
by other threads)!


Replace False by something that is usually (but not always) True, and 
print (x) by something that actually does something, and you had best 
put on your helmet because it's going to be a fun ride.


But I won't be on it.

The idea that the same name within the same scope always refers to the 
same value is an idea from functional programming and not part of Python; 
but surely the same name within the same scope should at least always 
refer to the same variable!


If something is to be done here, it occurs to me that the same parser that 
decides that the initial reference to x should use the local x could 
conceivably issue an error right away - local variable can never be 
assigned before use rather than waiting until runtime.  But even if I 
haven't confused myself about the possibility of this raising a false 
positive (and it certainly could in the presence of dead code), it 
wouldn't catch cases of conditional premature use of a local variable. I 
think in those cases people would still ask the same questions they do 
with the existing implementation.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyObject_RichCompareBool identity shortcut

[Isaac Morland]

On Wed, 27 Apr 2011, Alexander Belopolsky wrote:


High performance applications that rely on non-reflexivity will still
have an option of using ctypes.c_float type or NumPy.


Python could also provide IEEE-754 equality as a function (perhaps in 
math), something like:


def ieee_equal (a, b):
return a == b and not isnan (a) and not isnan (b)

Of course, the definition of math.isnan cannot then be by checking its 
argument by comparison with itself - it would have to check the 
appropriate bits of the float representation.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyObject_RichCompareBool identity shortcut

[Isaac Morland]

On Wed, 27 Apr 2011, Antoine Pitrou wrote:


Isaac Morland ijmor...@uwaterloo.ca wrote:


Python could also provide IEEE-754 equality as a function (perhaps in
math), something like:

def ieee_equal (a, b):
return a == b and not isnan (a) and not isnan (b)


+1 (perhaps call it math.eq()).


Alexander Belopolsky pointed out to me (thanks!) that isnan is implemented 
in C so my caveat about the implementation of isnan is not an issue.  But 
then that made me realize the ieee_equal (or just eq if that's 
preferable) probably ought to be implemented in C using a floating point 
comparison - i.e., use the processor implementation of the comparison 
operation..


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Support the /usr/bin/python2 symlink upstream

[Isaac Morland]

On Fri, 4 Mar 2011, R. David Murray wrote:


Nit pick: Change he to they to be gender neutral.


Nit pick: Change they to he to be grammatically correct. If we
really have to be gender neutral, change he to he or she.


English is evolving.  I vote for they.


Sorry, can't resist a further nitpick:  English has not been evolving in 
this particular case, in the sense that they has been used as a singular 
since before English was English:


http://motivatedgrammar.wordpress.com/2009/09/10/singular-they-and-the-many-reasons-why-its-correct/

Also interesting:

http://en.wikipedia.org/wiki/Singular_they

Attention he or sheists:  Singular they won before any of us was born. 
You may want to divert your energies to a more worthy cause, such as 
ensuring proper use of whom.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] constant/enum type in stdlib

[Isaac Morland]

On Tue, 23 Nov 2010, Antoine Pitrou wrote:


We already have a bunch of bizarrely unrelated stuff in collections
(such as Callable), so we could put enum there too.


Why not just enum (i.e., from enum import [...] or import 
enum.[...])?  Enumerations are one of the basic kinds of types overall 
(speaking informally and independent of any specific language) - they 
aren't at all exotic.  And Flat is better than nested, after all.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] constant/enum type in stdlib

[Isaac Morland]

On Tue, 23 Nov 2010, Antoine Pitrou wrote:


Le mardi 23 novembre 2010 à 12:32 -0500, Isaac Morland a écrit :

On Tue, 23 Nov 2010, Antoine Pitrou wrote:


We already have a bunch of bizarrely unrelated stuff in collections
(such as Callable), so we could put enum there too.


Why not just enum (i.e., from enum import [...] or import
enum.[...])?  Enumerations are one of the basic kinds of types overall
(speaking informally and independent of any specific language) - they
aren't at all exotic.


Enumerations aren't a type at all (they have no distinguishing
property).


Each enumeration is a type (well, OK, not in every language, presumably, 
but certainly in many languages).  The word basic is more important than 
types in my sentence - the point is that an enumeration capability is a 
very common one in a type system, and is very general, not specific to any 
particular application.



And Flat is better than nested, after all.


Not when it means creating a separate module for every micro-feature.


Classes have their own keyword.  I don't think it's disproportionate to 
give enums a top-level module name.


Having said that, I understand we're trying to have a not-too-flat module 
namespace and I can see the sense in putting it in collections.  But I 
think the idea that enumerations are of very wide applicability and hence 
deserve a shorter name should be seriously considered.


I'll leave it at that, except for:

Hey, how about this syntax:

enum Colors:
red = 0
green = 10
blue

(blue gets the value 11)

;-)

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] constant/enum type in stdlib

[Isaac Morland]

On Tue, 23 Nov 2010, Bill Janssen wrote:


The main purpose of that is to be able to catch type mismatches with
static typing, though.  Seems kind of pointless for Python.


The concept can work dynamically.  In fact, the flufl.enum package which 
has been discussed here makes each enumeration into a separate class so 
many of the advantages of catching type mismatches are obtained.



Hey, how about this syntax:

enum Colors:
red = 0
green = 10
blue


Why not

 class Color:
red = (255, 0, 0)
green = (0, 255, 0)
blue = (0, 0, 255)

Seems to handle the situation OK.


Yes, this looks almost exactly like flufl.enum syntax.  In any case my 
suggestion of a new keyword was not meant to be taken seriously.  If I 
ever think I have a good reason to suggest a new keyword I'll sleep on it, 
take a vacation, and then if I still think a new keyword is justified I 
will specifically disclaim any possibility of the suggestion being a joke.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Set the namespace free!

[Isaac Morland]

On Thu, 22 Jul 2010, Bartosz Tarnowski wrote:

[]

My proposal: let's make a syntax change.


I'm pretty sure this belongs on python-ideas.

Let all reserved words be preceded with some symbol, i.e. ! (exclamation 
mark). This goes also for standard library global identifiers.


!for boo in foo:
   !if boo is !None:
   !print(hoo)
   !else:
   !return !sorted(woo)


Is today April 1st?

Seriously, an identifier-quoting capability like PostgreSQL has wouldn't 
necessarily be a bad idea, but would be a topic for python-ideas, not 
here on python-dev.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] mkdir -p in python

[Isaac Morland]

On Tue, 20 Jul 2010, Steven D'Aprano wrote:


I tend to agree. Perhaps all we need is a recipe in the docs:

try:
   os.makedirs(path)
except OSError, e:
   if e.errno != 17:
   raise


What if the path or a parent of it already exists as a file?  If one has 
requested -p I believe one typically wants this to be an error.  And I 
assume that one would not use the literal 17 in production code.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible patch for functools partial - Interested?

[Isaac Morland]

On Fri, 7 May 2010, VanL wrote:


I make no effort to look for functional equality between two different
functions. too many side effects could go off trying to make that work,
and it would still be only inconsistently right. (IIRC, doing it
completely is NP-hard.)


Actually, doing it completely is flat-out impossible.

Godel, Halting Problem, and all that.

So you don't need to apologize for not doing it ;-)

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3147 ready for pronouncement and merging

[Isaac Morland]

On Tue, 13 Apr 2010, Barry Warsaw wrote:

I am attaching the latest revision of PEP 3147 to this message, which is 
also available here:


http://www.python.org/dev/peps/pep-3147/

[]

PEP: 3147
Title: PYC Repository Directories

[]

Further, pyc files will contain a magic string that differentiates the
Python version they were compiled for.  This allows multiple byte
compiled cache files to co-exist for a single Python source file.

This scheme has the added benefit of reducing the clutter in a Python
package directory.

When a Python source file is imported for the first time, a
`__pycache__` directory will be created in the package directory, if
one does not already exist.  The pyc file for the imported source will
be written to the `__pycache__` directory, using the magic-tag
formatted name.  If either the creation of the `__pycache__` directory
or the pyc file inside that fails, the import will still succeed, just
as it does in a pre-PEP-3147 world.

[]

Thank you for doing the work on this improvement.

I have one wording suggestion which I hope isn't bikeshedding: up above, I 
think the sentence containing pyc files will contain a magic string 
would be clearer if it made it clear that the file *names*, not (just?) 
the file contents, will contain the magic tag.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3147, __pycache__ directorie s and umask

[Isaac Morland]

On Tue, 23 Mar 2010, Antoine Pitrou wrote:


Greg Ewing greg.ewing at canterbury.ac.nz writes:

The main point of the __pycache__ proposal is to solve the needs of
Ubuntu/Debian packagers. If you are developing (rather than deploying or
building packages), you shouldn't have these needs AFAICT.


Maybe it's one point, but I'm not sure it's the *main* one.


It's the only reason the PEP was originally designed, and proposed.


At least one additional use case has appeared.  Actually, my use case was 
mentioned long ago, but I didn't really push (e.g. by writing a patch) and 
nobody jumped on it.  But this PEP solves my case too, so it should not be 
ignored just because the immediate impetus for the PEP is another case.



Personally I would benefit most from it during development.


Why? What benefit would it bring to you?


I'm sure Greg will jump in if I'm wrong about what he is saying, but the 
benefit to me and to Greg and to others writing .py code is that our 
directories will contain *.py and __pycache__, rather than *.py and *.pyc. 
So it will be much easier to see what is actually there.


Or if we're using SVN and we do svn status, the only spurious result 
will be ? __pycache__ rather than ? X.pyc for every X.py in the 
directory.


Or whatever other good effects come from having less junk in our source 
directories.


Directory tidiness is a positive general feature with at least a few 
specific benefits.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __pycache__ creation

[Isaac Morland]

On Mon, 22 Mar 2010, Antoine Pitrou wrote:


Oh, and by the way, there can be a race condition between __pycache__
creation and deletion (if it fails the test), where an attacker can stuff
a hostile pyc file in the directory in the meantime (and the deletion
then fails because the directory isn't empty).


Would creating it under a different name and then renaming help with this?


IMO, all these issues militate for putting __pycache__ creation out of
the interpreter core, and in the hands of third-party package-time/
install-time tools (or distutils).


Speaking only for myself, but really for anybody who likes tidy source 
directories, I hope some version of the __pycache__ proposal becomes part 
of standard Python, by which I ideally mean it's enabled by default but if 
that is just not a good idea then at most it should be required to set a 
command-line option to get this feature.


If I just want to write some .py code and run it, I don't see why my 
directories need to clutter up with .pyc files.  I've previously suggested 
a Python version of javac's -d (destination directory) option, but 
putting all the .pyc's in a __pycache__ directory per source directory is 
good enough to make me happy (and is Pythonically simple, in my opinion).


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Oddity PEP 0 key

[Isaac Morland]

On Sat, 2 May 2009, MRAB wrote:


Alexander Belopolsky wrote:

..

leaving just 'Rejected' and 'Replaced' to be disambiguated.


'X' or 'Z' for Rejected?  Looks like a perfect start for a bikeshed
discussion. :-)


Are there Unicode codepoints for smilies? I'm thinking of :-) for
'Accepted' and :-( for 'Rejected'. :-)


U+2639 WHITE FROWNING FACE
U+263A WHITE SMILING FACE

Also, U+2694 CROSSED SWORDS for vehement discussion on mailing list, 
U+2696 SCALES for BDFL is considering, and U+2678 BLACK UNIVERSAL 
RECYCLING SYMBOL for proposal previously rejected is being re-proposed 
due to changed circumstances.


For code don't forget great math operator symbols like U+2264 
LESS-THAN OR EQUAL TO and U+222A UNION.  But I doubt if anybody would want 
to bake in an absolute requirement for Unicode support in order to be able 
to read or write Python code.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: new list function: pack

[Isaac Morland]

On Fri, 20 Mar 2009, paul bedaride wrote:


I propose a new function for list for pack values of a list and
sliding over them:

then we can do things like this:
for i, j, k in pack(range(10), 3, partialend=False):
   print i, j, k

I propose this because i need a lot of times pack and slide function
over list and this one
combine the two in a generator way.


See the Python documentation for zip():

http://docs.python.org/library/functions.html#zip

And this article in which somebody independently rediscovers the idea:

http://drj11.wordpress.com/2009/01/28/my-python-dream-about-groups/

Summary: except for the partialend parameter, this can already be done 
in a single line.  It is not for me to say whether this nevertheless would 
be useful as a library routine (if only perhaps to make it easy to specify 
partialend explicitly).


It seems to me that sometimes one would want izip instead of zip.  And I 
think you could get the effect of partialend=True in 2.6 by using 
izip_longest (except with an iterator result rather than a list).



def pack(l, size=2, slide=2, partialend=True):
   lenght = len(l)
   for p in range(0,lenght-size,slide):
   def packet():
   for i in range(size):
   yield l[p+i]
   yield packet()
   p = p + slide
   if partialend or lenght-p == size:
   def packet():
   for i in range(lenght-p):
   yield l[p+i]
   yield packet()


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Isaac Morland]

On Thu, 11 Dec 2008, Ulrich Eckhardt wrote:


On Thursday 11 December 2008, Steve Holden wrote:

Ulrich Eckhardt wrote:
Seems to me this just threatens to add to the confusion.

If you know what your filesystem produces, you can take the appropriate
action to convert it into a type that makes sense to the user. If you
don't, then at least if you have the string in its bytes form you can

  ^^^

There are operating systems that don't use bytes to represent a file path,
namely all the MS Windows variants. Even worse, when you use a byte string
there, it typically means that you want to use the obsolete encoding that is
based on codepages.

Why can we not preserve the representation of a path as it is? Why do we
_have_ to convert it to anything at all, without even knowing if this
conversion is needed? I just want to do something to a file's content, why
does its path have to be converted to something and then be converted back in
order for the system to digest it?


re-present it to the filesystem to manipulate the file. What are we
supposed to do with the special type?


You receive from readdir() and pass it to stat(), simple as that. No
conversions from the native representation needed. If you need a textual
representation, then you have to convert it and you have to do so explicitly
according to whatever logic your application requires.


Not only would this address the issue with the local filesystem, it would 
also provide a principled way to deal with remote filesystems.  For 
example, an FTP interface library for Python could use this type to 
returns paths of the sort actually supported by the raw FTP protocol.


Thinking of the filesystem is actually a misconception - always 
referring to a filesystem opens up all sorts of possibilities.  There is 
a lot of coding to do to allow this, but allowing programs to work with 
paths and files in the local filesystem, remote filesystems, and 
filesystems constructed from others (e.g., by expanding symlinks, changing 
the root similar to chroot, or encoding/unencoding pathnames) would open 
up lots of possibilities, including better test environments.


This is an interesting case of separating byte strings from character 
strings.  As long as the two are conflated, everything appears simple. 
But when they are separated, not only are there two types where before 
there was only one, it turns out that which type is correct in some 
circumstances depends on the platform.  Also, many objects which are byte 
strings at the protocol level are usually or always meant to be character 
strings of some sort, but how to translate them simply cannot be nailed 
down once and for all.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] fileobj.read(float): warning or error?

[Isaac Morland]

On Tue, 2 Sep 2008, Jesus Cea wrote:


Indeed. read(0) is quite often generated as an edge case when one is
computing buffer sizes, and returning an empty string is most
definitely the right thing to do here (otherwise some application code
becomes more complex by having to avoid calling read(0) at all).


How do you differenciate between that empty string (when doing
read(0)), from EOF (that is signaled by an empty string)?.


Why would you expect a difference between reading 0 bytes at EOF and 
reading 0 bytes anywhere else?  If you read(4) when at offset 996 in a 
1000-byte file I doubt you expect any special notification that you are 
now at EOF.


The Unix read() system call doesn't treat EOF as special other than it 
won't return bytes from beyond EOF and therefore even when reading a 
regular file could return fewer (including 0) bytes than asked for in the 
call.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] confusing exec error message in 3.0

[Isaac Morland]

On Fri, 29 Aug 2008, Greg Ewing wrote:


Steven D'Aprano wrote:

I don't think M.__file__ should lie and say it was loaded from a file that 
it wasn't loaded from. It's useful to be able to look at a module and see 
what file it was actually loaded from.


On the other hand, it could be useful to be able to find
the source file for a module, regardless of whether it
was loaded from a .py, .pyc or .pyo.

Maybe there should be a __source__ attribute for that?

Or is os.path.splitext(M.__file__)[0] + .py considered
good enough for that?


I think .__source__ is a much better idea.  It's more explicit to start 
with, and on top of that it can simply be None when there is no source 
file available, which is much more explicit than file not found means no 
source file available.  Also, if a feature is ever added to allow telling 
Python to put the compiled files in a separate tree (like javac -d 
directory in the Java universe) then it will be even more clear that 
computing the source location from __file__ is not the way to go.


PS, in case it's not clear, I want that feature, in order to keep .pyc 
files out of my SVN checkouts.  I hate having automatically generated 
files in my workspace.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] String concatenation

[Isaac Morland]

On Sat, 23 Aug 2008, Fredrik Lundh wrote:

removing it is a bad idea for the reasons already given, but requiring 
parentheses could help.


that is, the following would result in a warning or an error:

   L = [first, second third]

but the following wouldn't:

   L = [first, (second third)]

   T = (This is a line of text.\n
This is another line of text.\n)


This would avoid accidentally leaving out commas in list construction, but 
tuple construction would still have the same problem.  And it's still a 
change in the language which would probably affect lots of existing code. 
I doubt if there is any problem-free way of trying to address this issue 
by changing the language.


One suggestion to help minimize problems when writing code would be always 
to put the optional trailing comma:


[
'a',
'b',
'c',
]

which is also a revision-control-friendly practice, and in the tuple 
constuction context avoids the possibility of removing an item from a 
two-tuple and ending up with not a one-tuple but instead just the item 
itself.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Slice as a copy... by design?

[Isaac Morland]

On Thu, 22 May 2008, Christian Heimes wrote:


The buffer interface was designed for the slice-as-copy use case:


a = abcdefg
b = buffer(a, 2, 3)
b

read-only buffer for 0x839c2e0, size 3, offset 2 at 0x8391c40

str(b)

'cde'

[]

This answers my musing about shared slices.  But it points me at another 
question: why is buffer() listed in Non-essential Built-in Functions? 
While it is obviously not essential like str() or list(), it isn't 
deprecated like apply().  On the other hand, some other built-in functions 
listed under Built-in Functions are probably also not essential (I'm not 
going to go any further out on the limb by giving an example!).


Perhaps I misunderstand the intent of this manual page.

http://docs.python.org/lib/non-essential-built-in-funcs.html#l2h-88

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Conditional For Statements

[Isaac Morland]

On Sun, 18 May 2008, Ryan Hitchman wrote:


I'd like to propose an addition to the for statement's syntax:

for {variable} in {iterable} if {condition}:
   {block}

which is equivalent to

for {variable} in {iterable}:
   if not {condition}:
   continue
   {block}

and

for {variable} in filter(lambda: {condition}, iterable):
   {block}

This would make the syntax closer to that of generators, which have
'for variable in iterable if condition', and would improve code
clarity by increased brevity and not negating boolean expressions.


http://mail.python.org/pipermail/python-dev/2006-May/065090.html

This was proposed and rejected before, after which it was again proposed 
and rejected.  I think it was also proposed and rejected sometime in 2007, 
but I can't seem to find it in the archives so my memory may be faulty.


The idea is apparently not dead yet.  Hang around a couple of minutes, 
though.  It won't be long.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On quote styles

[Isaac Morland]

On Sun, 11 May 2008, Greg Ewing wrote:


While Python doesn't have a char type (yet), I still find the distinction
between 'c' and abc useful to show intent (especially given my C
background


The way I tend to use them is that xxx is for data
operated on by the program and seen by the user,
and 'xxx' is for things that are only used internally,
e.g. enumerated type values and attribute names for use
by getattr().


I find myself doing something similar.  I use '' for anything that is a 
dictionary key and  for things that are strings not meant to be used for 
indexing, in particular the things that are looked up.  Except when I 
don't.


When I am generating HTML using the ll.xist library, which uses constructs 
like this:


html.a (u'link text', href=http://url.goes.here/;)

I have just started using '' for text element contents and  for 
attribute values.  This way the attribute values (keyword parameters) look 
almost the exact same as in actual HTML (only difference is the commas) 
(and the escaping), and my editor seems to be designed for C-style 
languages where '' are for character constants and  are for strings so 
it highlights them differently.


My personal opinion is that one should not use the different quoting 
styles at random, but I am inclined to believe that there is no single 
guideline that can apply to everybody.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problems with the new super()

[Isaac Morland]

On Fri, 2 May 2008, Greg Ewing wrote:


Guido van Rossum wrote:

The
alternative would be to make it a keyword, which seemed excessive
(plus, it would be odd if super() were a keyword when self is not).


If it's really such a useful thing as to warrant so much
magic to support it, then I think it deserves to have a
keyword.

Conversely, I would say that if it doesn't deserve a
keyword, it also doesn't deserve that much magic.


One might even go further and say that if it's that magic, then it's a 
keyword by definition.


Certainly I would want a syntax-highlighting editor to highlight it 
somehow, and if the editor has a rename feature to rename a variable and 
replace all references to it (but not other uses of the same identifiers 
in different scope), then it probably should at least alert the programmer 
before allowing a rename to or from the magic name.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A smarter shutil.copytree ?

[Isaac Morland]

On Sun, 20 Apr 2008, Steven Bethard wrote:


On Sun, Apr 20, 2008 at 4:15 PM, Tarek Ziadé [EMAIL PROTECTED] wrote:

I have submitted a patch for review here: http://bugs.python.org/issue2663

 glob-style patterns or a callable (for complex cases) can be provided
 to filter out files or directories.


I'm not a big fan of the sequence-or-callable argument. Why not just
make it a callable argument, and supply a utility function so that you
can write something like::

   exclude_func = shutil.excluding_patterns('*.tmp', 'test_dir2')
   shutil.copytree(src_dir, dst_dir, exclude=exclude_func)


Even if a glob pattern filter is considered useful enough to be worth 
special-casing, the glob capability should also be exposed via something 
like your excluding_patterns constructor and additionally as a function 
that can be called by another function intended for use as a callable 
argument.


If it is not, then doing something like files matching these glob 
patterns except for those matching this non-glob-expressible condition and 
also those files matching this second non-glob-expressible condition 
becomes painful because the glob part essentially needs to be 
re-implemented.


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] string representation of range in 3.0

[Isaac Morland]

On Wed, 16 Apr 2008, David Wolever wrote:

 On 16-Apr-08, at 9:37 AM, Isaac Morland wrote:
 On Wed, 16 Apr 2008, Paul Moore wrote:
 On 16/04/2008, Armin Rigo [EMAIL PROTECTED] wrote:
 What about the less confusing and more readily generalizable:
  range object 0, 1, ..., 9
 
 It would also be helpful IMHO to use this kind of repr for most built-in
 iterators and iterables, instead of their mosty-useless default repr.
 
 I quite like this. But as a non-beginner, I'll freely admit that my
 intuitions are suspect :-)
 
 I like this too.  For iterators, though, would you always show the next
 couple of elements?  The values contained in the iterator will change as
 the iterator iterates.  Alternatively, the representation could be
 frozen to reflect the values originally pending in the iterator, but
 then the representation wouldn't show anything about the current state of
 the iterator.
 So would you mean something like:
 generator object __main__.Foo instance at 0x835d0, __main__.Foo instance 
 at 0x83620, ...
 Or maybe:
 generator __main__.Foo instance at 0x835d0, __main__.Foo instance at 
 0x83620, ... 

 While I agree in theory, I'm not sure I like the looks of it in practise.

I was mostly responding to what I saw as a suggestion to change the 
representation of existing iterators.  It's been pointed out in a previous 
reply to my message that obtaining values from a general iterator for use 
in the representation is problematic at best, and in general I don't think 
it can be done in an acceptable fashion, because I can imagine code 
depending on values not being obtained from an iterator before they are 
explicitly requested by next().  We wouldn't want a call to __repr__() to 
change the operation of the iterator, so any idea to pull two values and 
store them somewhere isn't generally acceptable.

For some specific iterators which have access to all the underlying 
information, an informative representation is possible and frequently 
feasible.  My question simply concerned what it would look like.  Would it 
show the next couple of items, or the first couple of items (or something 
else)?

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Introducing the ``make check`` command

[Isaac Morland]

On Tue, 18 Mar 2008, Brett Cannon wrote:

 Lastly, I would like to have it strip trailing whitespace in C files.
 The only problem is that I don't know if removing trailing whitespace
 could possibly cause a problem or not. Anyone know?

I would be worried about the sequence backslash-space-newline.  Off the 
top of my head I can't see why that would occur in valid code, but 
dropping the space would give you an escaped newline which could be 
different from the original.  I'd be worried about some weird case related 
to macro expansion or definition.

http://gcc.gnu.org/onlinedocs/cppinternals/Lexer.html has some information 
related to the Gnu C preprocessor which may be relevant.

Have you considered also forcing Mac \r and DOS \r\n line endings to 
Unix \n style?

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New math functions

[Isaac Morland]

On Fri, 15 Feb 2008, Mark Dickinson wrote:

 On Tue, Feb 12, 2008 at 1:52 AM, Raymond Hettinger [EMAIL PROTECTED] wrote:

 Also, it would be useful to have a new method, float.is_integer().  This
 would be better than the current approach where we make the
 test:  if x == floor(x).

 How common is this test?  Given the inexact nature of floating-point
 arithmetic, checking whether a float is exactly an integer seems like
 something that one might actually want to discourage, just like comparing
 two floats for equality.

Depending on the nature of the computations that produced the float 
result, there may be no problem with the result being different from the 
mathematical result.  For example, if you are adding up a bunch of 
multiples of 1/2, a sufficient condition for an exact result is that the 
(mathematical) total of the absolute values of the numbers isn't too big.

After such a computation, meeting the given condition, checking for an 
integer result is guaranteed to be meaningful and to match what is 
mathematically wanted.

I can't say how often one needs to check a float for being an integer.  I 
believe that x == floor(x) is equivalent to not (x % 1) which also 
generalizes to checking for an exact multiple of values other than 1.

I suspect, but do not have evidence, that circumstances in which it is 
useful to check for an integer are likely to be ones in which there is a 
better-than-usual chance of getting precise float results.  How often are 
you going to check the output of sin() for being an integer?  But on the 
other hand pitfalls will occur if we, for example, replace 1/2 in my 
example with 1/10, since these are not generally representable in float 
format.

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Incorrect documentation of the raw_input built-in function

[Isaac Morland]

On Thu, 24 Jan 2008, [EMAIL PROTECTED] wrote:

Mike 2. Is this really the hard-coded behavior we want?  I don't think
Mikemy use-case is that odd; in fact, what I find very odd is that
Mikethe prompt output is send to stderr.  I mean, I'm printing the
Mikeprompt for a question, not some error message. Can there not at
Mikeleast be an optional parameter to indicate that you want the
Mikeoutput sent to stdout rather than stderr?

 I can think of situations where you don't want the output to go to stdout
 either (suppose it's the regular output of the file you want to save to a
 file).  Having a choice seems the best route.

What about an option (maybe even a default) to send the prompt to stdin?

The Postgres command line interface psql appears to do this:

$ psql 21 /dev/null
Password:
$

(I typed my password and then I quit by typing ^D; if I type the wrong 
password, it looks the same on screen but it quits right away without 
waiting for ^D)

I think ssh also does this when it needs to prompt for a password.

Really the prompt is part of the input, not part of the output, in a 
certain sense.  Have people actually verified that the prompt is really 
sent to stderr right now by using 2/dev/null to attempt to suppress it?

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Extracting variables from string.Template objects

[Isaac Morland]

I found myself wanting to obtain the variables from a string.Template 
object.  The situation is that part of input consists of a filename 
template, and I want to make sure that all the parameter values I have are 
actually represented in the filename template.  So:

def templateVariables (template):
 Extract the variable names from a string.Template.

 Returns a tuple of all variable names found in the template, in the order
 in which they occur.  If an invalid escape sequence occurs, the same
 error will be raised as if an attempt was made to expand the template.
 
 result = []
 for match in template.pattern.finditer (template.template):
 if match.group ('invalid') is not None:
 # Raises ValueError
 template._invalid (match)
 if match.group ('escaped') is not None:
 continue
 # The or None should be moot.  It is there to ensure equivalent
 # treatment for an empty 'named' and an empty 'braced'.
 result.append (match.group ('named') or match.group ('braced') or None)
 return tuple (result)

Note that almost all the work is actually done by calling in to parts of 
the template object, so I think this should work properly with subclasses 
and other less common cases.

I would like to add this as a method of string.Template, which I think 
amounts to changing template to self and putting it in the Template 
class in string.py rather than on its own.  If the general idea is 
approved I would be happy to make a patch.

Also, on a related issue, does it make sense to scan the template string 
for invalid escape sequences in Template.__init__?  For the applications I 
can imagine of string.Template, I would prefer to get an error upon 
creating the Template object rather than arbitrarily later when I try to 
.substitute with it.

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).

[Isaac Morland]

On Mon, 10 Dec 2007, Guido van Rossum wrote:

 I think we successfully resolved this. Adam Olsen will produce a patch
 that allows one to specify a single file descriptor to which a zero
 byte will be written by the C-level signal handler. Twisted and PyGTK

I'm surprised that the byte is zero, rather than being the signal number. 
This would reflect the way C-level signal handlers work, where they are I 
believe called with the signal number.  Is any discussion of this point 
conveniently available online?  I ask for my own understanding, and 
definitely not out of a desire to re-open any settled issues!

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [poll] New name for __builtins__

[Isaac Morland]

On Thu, 29 Nov 2007, Graham Horler wrote:

 Perhaps someone here can draw some inspiration from __monty__ python's
 flying __circus__.  It would be nice to have a name with a pythonic
 __ground__.

 Unfortunately that show is not my __staple__ entertainment, and although
 I have a __general__ idea what the show's about, it needs a __canonic__
 understanding of its __founding__ __elements__ in order to be used for
 __primary__ ideas.

For my part, I like __universal__.  I was going to suggest 
__cosmological__ to leave space in case we need to put something between 
__global__ and (what is now called) __builtin__, but then I remembered the 
Galaxy Song, and thought of __galactic__ for that eventuality.

I wonder how much you could sell the naming rights for?  i.e. call it 
__[name of sponsor]__.  Python's pretty popular, such advertising should 
be worth something

PS: I actually do like __universal__.  The rest may be somewhat addled.

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] .pyc location?

[Isaac Morland]

On Fri, 23 Nov 2007, Andreas Raab wrote:

 I'd like to be able to specify an explicit location for the .pyc files.

Me too, although for different reasons.  I don't like my working 
directories being filled with clutter by programs.  When I work in Java I 
always have javac aliased to use the -d option to put the .class files in 
a specific directory, and I set my CLASSPATH to refer to that directory. 
I end up with a java-src directory with a tree of .java files under it and 
a java-cls directory with .class files under it.  I also get java-doc 
containing the output of javadoc.

Something similar for Python would be very helpful for me.

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Library Addition: First-class Procedure Signatures

[Isaac Morland]

On Wed, 14 Nov 2007, Brett Cannon wrote:

 As Collin already pointed out, it sounds like you want PEP 362 to get
 into the stdlib.  I have not made a big push to try to get my existing
 implementation into Python 2.6/3.0, but I plan to at some point.

Yes, it had not occurred to me to check the existing PEPs for my PEP 
before proposing it.  Now that I've read it I have a couple of comments, 
but it is close to what I'm looking for.

 I've put the code below, but I wonder if the real solution is just to create 
 an
 interface to already-existing capability?  It occurs to me that the
 implementation is likely to be in the interpreter itself and not written in
 Python.

 I don't see why a Python implementation is bad.  If you make this
 information lazy then it is not such a big deal to have it take a
 little bit longer than if it was implemented in C.

I don't have any problem with a Python implementation.  In particular, I'm 
not concerned in this case about the performance.  Rather what I actually 
wanted was a way to just bind arguments and then get the resulting 
dictionary (what would usually become locals()).  I realized that I could 
create a Signature object with a binding method, but that I would just be 
duplicating part of the Python interpreter.

I haven't studied the Python interpreter so I don't know if it is feasible 
to re-use that (presumably highly optimized for actually calling 
procedures, not just binding arguments) code or if it makes more sense to 
simply re-implement it.

 One possible improvement (and I'm not sure it's better, so I'm just putting 
 it
 out there): perhaps expand_args should be renamed to __call__. Then 
 essentially
 a Signature object would be a procedure whose body is just return locals 
 ().

 __call__ is already used a method name for objects that can be called.

Yes, that is why I used that name.  The idea is that a Signature object be 
callable, have itself as signature, and return the dictionary of locals 
resulting from the procedure call argument binding process.

You can also think of (my idea of) Signature objects as providing a simple 
way to create lots of special-case dictionary constructors.

More (semi-)formally, if the body of a procedure p is return locals (), 
then p(...) is the same as p.__signature__(...).

A couple of comments about PEP-362:

1. For the name attribute of the Parameter object, I think it needs to 
be str | tuple(str) | tuple(tuple(str)) | :

 def a ((b,c),(d,(e,f))):
... print b,c,d,e,f
...
 a
function a at 0x7899b0
 a((1,2),(3,(4,5)))
1 2 3 4 5


2. For position of keyword-only parameters, are they allowed to conflict 
with each other, or does each parameter get a unique value?  +1 on not 
using -1 as a special value.  Python is not COBOL.

3. (My apologies if any of these have already been discussed)  Under 
Implementation, could __signature__ just be a property of callable 
objects?  Not saying anything about implementation, but just being able to 
say formataddr.__signature__ feels nicely minimal (to me).

4. Signature.bind - for what I'm doing, I definitely want what would 
become locals() in a procedure call, i.e. keys in the resulting dictionary 
are strings.  But I can see the other behaviour being useful in other 
circumstances so maybe there should be bind and bindp, or (see above) 
__call__ and bind, or something else.

5. var_args... default to None.  +10 from me on this one - this is 
*exactly* what None is, as far as I can tell.  I'm new enough that this 
should probably count for at most +0.1 though.

6. The PEP doesn't say anything about building Signature objects from 
scratch, and the code does not reassure me.  I would like to be able to 
build a Signature from a sequence of strings for positional parameter 
names, for example, and provide default values for some parameters.  My 
solution started off:

class Signature (object):
  def __init__ (self, argnames,
   excessargs=None, excesskeys=None, defaults=None):
  self.__argnames = tuple (argnames)
  self.__excessargs = excessargs
  self.__excesskeys = excesskeys
  if defaults is None:
  defaults = {}
  self.__defaults = dict (defaults)
[]

Thanks for the responses.  I hope the above is at least in part not a 
re-hash of old discussions.

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Library Addition: First-class Procedure Signatures

[Isaac Morland]

On Thu, 15 Nov 2007, Isaac Morland wrote:

 1. For the name attribute of the Parameter object, I think it needs to
 be str | tuple(str) | tuple(tuple(str)) | :

No, that's still wrong.  I think it needs to be T,
where T == str | tuple(T).

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Suggestions for Improvements to namedtuple

[Isaac Morland]

.__reprtemplate__ = %s(%s) % (name,
  , .join (%s=%%r % name for name in fields))
 result.__indices__ = dict ((field, i) for i, field in enumerate (fields))
 return result

from email.utils import formataddr

class test (namedtuple (test, (realname, email), {'realname': None})):
 @property
 def valid (self):
 return self.email.find (@) = 0

 __str__ = formataddr

if __name__ == __main__:
 e1 = test ((Smith, John, [EMAIL PROTECTED]))
 print e1.realname =, e1.realname
 print e1.email =, e1.email
 print e1 =, repr (e1)
 print str(e1) =, str (e1)

 e2 = test.__fromvalues__ (email=[EMAIL PROTECTED])
 print e2 =, repr (e2)
 print str(e2) =, str (e2)

Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python Library Addition: First-class Procedure Signatures

[Isaac Morland]

For another project (see my previous email on named tuples), I needed to 
represent procedure signatures, and use them to expand arguments into the 
dictionary of values that exists when execution of a procedure starts.  To my 
surprise, this capability didn't seem to be provided by the Python library, 
even though it clearly is present within the Python system somewhere.

So I wrote a Signature class.  Instances of the class represent all the 
information present between the parentheses of a procedure definition. 
Properties are provided to get the information out, and an expand_args method 
can be called to expand arguments into a dictionary.  This expand_args method 
implements (if I've done it right) the argument conversion part of section 
5.3.4 of the Python Reference Manual (http://docs.python.org/ref/calls.html).

I've put the code below, but I wonder if the real solution is just to create an 
interface to already-existing capability?  It occurs to me that the 
implementation is likely to be in the interpreter itself and not written in 
Python.

One possible improvement (and I'm not sure it's better, so I'm just putting it 
out there): perhaps expand_args should be renamed to __call__. Then essentially 
a Signature object would be a procedure whose body is just return locals ().

class Signature (object):
 def __init__ (self, argnames,
  excessargs=None, excesskeys=None, defaults=None):
 self.__argnames = tuple (argnames)
 self.__excessargs = excessargs
 self.__excesskeys = excesskeys
 if defaults is None:
 defaults = {}
 self.__defaults = dict (defaults)

 @property
 def argnames (self):
 return self.__argnames

 @property
 def excessargs (self):
 return self.__excessargs

 @property
 def excesskeys (self):
 return self.__excesskeys

 def defaults (self):
 return dict (self.__defaults)

 def expand_args (self, *args, **keys):
 # Start with defaults
 result = self.defaults ()

 # Assign positional arguments
 for i in range (min (len (args), len (self.argnames))):
 result[self.argnames[i]] = args[i]

 # Assign keyword arguments
 for arg in self.argnames:
 if arg in keys:
 if arg in result:
 raise TypeError
 result[arg] = keys[arg]
 del keys[arg]

 # Check for missing arguments
 for i in range (len (args), len (self.argnames)):
 if not self.argnames[i] in result:
 raise TypeError

 # Excess positional arguments (*args parameter)
 if self.excessargs is None:
 if len (args)  len (self.argnames):
 raise TypeError
 else:
 result[self.excessargs] = args[len (self.argnames):]

 # Excess keyword arguments (**keys parameter)
 if self.excesskeys is None:
 if keys:
 raise TypeError
 else:
 result[self.excesskeys] = keys

 return result


Isaac Morland   CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com