from:"Ned Batchelder"

[Python-Dev] Re: Typing syntax and ecosystem, take 2

2021-04-13 Thread Ned Batchelder


On 4/13/21 7:40 PM, Hugh Fisher wrote:

From: Ned Batchelder 

[ munch ]

This is very similar to statically typed languages. They also have two
steps:

   * There is the first step that checks the types but does not run the
 program. In C/C++, this is the compiler, in Python it is "mypy".
   * There is the second step that runs the program but does not check
 the types. In C/C++, this is running the executable, in Python it is
 "python".

In Python, we can skip the first step, and perhaps that is the source of
your concern.  I doubt anyone that has adopted type hinting in Python
will forget to run mypy, especially in a CI environment like you consider.

In the current Python implementation, we can't skip the first step.
Type hints have a significant performance impact on the Python interpreter,
hence PEP 649. If we used the TypeScript approach, the typing hints would
be removed before execution by the Python interpreter.


In another message, you alluded to PEP 649 being a solution to run-time 
type checking.  Maybe I'm misunderstanding something: type annotations 
never do type checking at runtime.  As I understand it, PEP 649 is about 
avoiding the overhead of evaluating the type expression itself, not 
about checking the types of values.




As for "forget to run mypy", no-one sets out to do the wrong thing. The
difference here is that with TypeScript -> JavaScript, if you forget to run
the translation step, the program won't run because TypeScript is not
valid JavaScript. Same was true for the old CFront C++ -> C translator.
Errors are automatically detected.

If you forget to run mypy, you've got a program which *looks* correct
but might not be.

Yes, absolutely.  But who is running mypy manually, and therefore could 
forget it? It will be a step in a Makefile, or a tox.ini, or a CI script.


You are right that this approach to type checking can surprise people 
coming from other languages.  And absolutely, you should not use type 
annotations without checking them.  People will learn to do it right.


--Ned.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WOKTMKWLZC7ZFM7D4TLRHBEG3AY7J527/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Typing syntax and ecosystem, take 2

2021-04-13 Thread Ned Batchelder


On 4/12/21 7:43 PM, Hugh Fisher wrote:

Having a type checker run before the Python interpreter in our
current day continuous build/integration environment adds a
second step


This is very similar to statically typed languages. They also have two 
steps:


 * There is the first step that checks the types but does not run the
   program. In C/C++, this is the compiler, in Python it is "mypy".
 * There is the second step that runs the program but does not check
   the types. In C/C++, this is running the executable, in Python it is
   "python".

In Python, we can skip the first step, and perhaps that is the source of 
your concern.  I doubt anyone that has adopted type hinting in Python 
will forget to run mypy, especially in a CI environment like you consider.


--Ned.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/STHJTBL6RL2JNKS47IST3K6EFN2WIMSV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Ned Batchelder


On 3/17/21 6:41 PM, MRAB wrote:

On 2021-03-17 22:10, Skip Montanaro wrote:

I stumbled on this while trying to generate a line number table in my 
side project register VM. As I understand it, the line number delta 
in the output table is supposed to always be >= 0. In my code I'm 
using dis.findlinestarts() to determine the line numbers for each 
block. Perhaps I should be modifying its results. OTOH, maybe it's a 
bug. (If that's the consensus, I will create an issue.)


co_lnotab has had negative deltas since 3.6.

--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EK2O6SE6J6LROGNHKPQYTQNJH5VFE2M4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please do not remove random bits of information from the tutorial

2020-11-14 Thread Ned Batchelder

On 11/9/20 2:25 PM, Raymond Hettinger wrote:

On Nov 7, 2020, at 9:51 AM, Riccardo Polignieri via Python-Dev
wrote:

My concern here is that if you start removing or simplifying some
"too-difficult-for-a-tutorial" bits of information on an occasional basis, and
without too much scrutiny or editorial guidance, you will end up loosing something
precious.

I concur with you sentiments and do not want the tutorial to be dumbed down.

I feel like we are missing a key element of Riccardo's point: "without
editorial guidance." Changes are being made without first having an
agreement about what the tutorial should be.

Personally, I would rather the tutorial did not try to cover
everything. If something is only useful to 10% of the readers, I'd
prefer to cover it elsewhere. Adding tutorial-style content throughout
the rest of the reference documentation seems a fine solution to me. As
an example, I tried to use ContextVars for the first time the other day,
and would have welcomed a description on their page about how they might
be used, where they are useful, and so on.

Here are a few thoughts on the subject:

* The word "tutorial" does not imply "easy". Instead it is a self-paced, example driven
walk-through of the language. That said, if the word "tutorial" doesn't sit well, then just rename
the guide.

* The world is full of well-written guides for beginners. The variety is especially important because "beginner" means many different
things: "never programmed before", "casually checking out what the language offers", "expert in some other language",
"is a student in elementary school", "is a student in high school", "is an electrical engineer needing write scripts",
etc.

* One thing that makes the current tutorial special is that much of it was
written by Guido. Delete this text and you lose one of the few places where
his voice comes through.

* There is value in having non-trivial coverage of the language. When people
ask how __cause__ works, we can link to the tutorial. Otherwise, we have to
throw them to the wolves by linking to the unfriendly, highly technical
reference guide or to a PEP.
Linking to content is a great use-case. There's no need for __cause__
to be covered in a linear front-to-back tutorial just so we can link to
it. When someone asks how __cause__ works, we could link to the (not
yet written) walkthrough about __cause__ elsewhere.

* For many people, our tutorial serves as the only systematic walk-through of
the language. If you decide to drop the mention of complex numbers, the odds
of a person ever finding about that capability drop to almost zero.

As a compromise, we could include full text for the 80% features in the
tutorial, and in each section, include links to "other topics" or
"advanced features" in other sections. This would leave the
front-to-back readers of the tutorial with a good 80% overview of the
language, but also let readers find the tangents that are of interest to
them.

This is the sort of overall design of the tutorial we should be undertaking.

* My suggestion is that we add a section to the beginning of the tutorial with external
links elsewhere, "If you are ten years old, go here. If have never programmed
before, go here, etc"

* If you think the word tutorial implies fluffy and easy, then let's just rename it to
"Language walk-through with examples" or some such.

* FWIW, I've closely monitored the bug tracker daily for almost two decades.
We almost never get a user complaint that the tutorial is too advanced. For
the most part, it has long been of good service to users. Almost certainly it
can be improved, but hopefully not be dropping content.

Bug reports on bpo are a really bad way to gauge how the tutorial is
working for people. Most people looking for a "tutorial" are not going
to think to write bug reports like you are expecting, much less find
bpo, or go to the trouble of creating an account to write one.

--Ned.

Raymond
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/CYFDV4ZYGUFGCYUI5HPTF66UNZ4FXO2M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please do not remove random bits of information from the tutorial

2020-11-14 Thread Ned Batchelder

I think when Riccardo said "The PEPs are the worst," he meant that PEPs
do not work well as documentation for features, because it was not their
purpose. PEPs are designed to be proposals, and then summaries of
decisions. I agree with him that linking to PEPs should be for
supporting information about the rationale of a feature; they should
only be used as the documentation of a feature as a last resort.

--Ned.

On 11/12/20 3:45 PM, Guido van Rossum wrote:
The correct place for the docs for __cause__ and __context__ is in the
section in the library reference about exceptions. There's quite a bit
about them there already. That's where the tutorial should link as well.

And now I ask you to stop complaining (your "the PEPs are the worst"
does not help your cause).

On Thu, Nov 12, 2020 at 12:30 PM Riccardo Polignieri via Python-Dev
mailto:python-dev@python.org>> wrote:

> There is value in having non-trivial coverage of the language.
When people ask how
> __cause__ works, we can link to the tutorial.

I don't necessarily agree with the rest, but I think this is very
important - at least, in the
current situation. Maybe in the future we will be able to
rearrange and move some parts
of the tutorial into the "how to" section, and/or to write more
introductions to the various
sections of the documentation... but for now, the tutorial is all
we have.

> Otherwise, we have to throw them to the
> wolves by linking to the unfriendly, highly technical reference
guide or to a PEP.

Indeed. The PEPs are the worst, more often than not. I remember
when I first learned Pyhton,
every time the documentation linked to a PEP, I felt a shiver down
my spine.

r.
___
Python-Dev mailing list -- python-dev@python.org

To unsubscribe send an email to python-dev-le...@python.org

https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/LHGZNTRKU2Z2OD3W7DPFEDS274OBFE7F/

Code of Conduct: http://python.org/psf/codeofconduct/

--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/ILOS6URJIPNVV7XKBVPUR6FYEBVLZGIZ/
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/KUZNJ6ZGPI2U6NWA6K7CE6WFI2VEULIT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 626: Precise line numbers for debugging and other tools.

2020-07-22 Thread Ned Batchelder


On 7/17/20 10:48 AM, Mark Shannon wrote:

Hi all,

I'd like to announce a new PEP.

It is mainly codifying that Python should do what you probably already 
thought it did :)


Should be uncontroversial, but all comments are welcome.


Thanks for thinking about these aspects of the interpreter, and for 
using the PEP process to work them out before implementation.


In the PEP, you mention, "some bytecodes will need to be marked as 
artificial, and not have a meaningful line number" (twice), but there's 
no example of what this means.  Can you elaborate?


--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OTYRLXBR74IGQVEPEZIG272SMKZ4J2SZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 626: Precise line numbers for debugging and other tools.

2020-07-22 Thread Ned Batchelder

On 7/22/20 6:42 AM, Inada Naoki wrote:
On Wed, Jul 22, 2020 at 6:12 PM Antoine Pitrou > wrote:

>
> >
> > But if we merge two equal code blocks, we can not produce precise line
> > numbers, can we?
> > Is this inconsistent microoptimization that real optimization harder?
> > This optimization must be prohibited in future Python?
>
> All attempts to improve Python performance by compile-time
> bytecode optimizations have more or less failed (the latter was
> Victor's, AFAIR).  Is there still interest in pursuing that avenue?
>
> Regards
>
> Antoine.
>

I don't think all attempts are failed.  Note that current CPython includes
some optimization already. If they are all failed, we must remove them
 to make compiler simple.

And I think there are some potential optimization if we can limit some
debugging/introspecting features, like some C variables are "optimzed 
away" in gdb when

we use -O option.

We seem to like following the C model when it comes to implementing 
optimizations, and then skip the part where C developers can disable all 
optimizations when reasoning about code is more important than speed.  I 
am fine with any optimizations at all, as long as there is a simple and 
supported way to ask that they be disabled.

--Ned.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HBNCQPU7ECM5EE3O2MDMEQDKX5D4L6DW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 626: Precise line numbers for debugging and other tools.

2020-07-22 Thread Ned Batchelder


On 7/21/20 5:04 PM, Gregory P. Smith wrote:
Given it is impossible for tools doing passive inspection of Python VM 
instances to execute code, co_linetable's exact format will be 
depended on just as co_lnotab was.  co_lnotab was only 
quasi-"officially" documented in the Python docs, it's spec lives in 
https://github.com/python/cpython/blob/master/Objects/lnotab_notes.txt (pointed 
to by a couple module's docs). The lnotab format "changed" once, in 
3.6, an unsigned delta was changed to signed (but I don't believe 
anything beyond some experiments ever actually used negatives?).


Negatives definitely happen.  When I comment out the line in coverage.py 
that deals with negative deltas, 34 of my tests fail.


For example:

   a = (
    1
   )

With 3.8 compiles to:

  2   0 LOAD_CONST   0 (1)

  1   2 STORE_NAME   0 (a)
  4 LOAD_CONST   1 (None)
  6 RETURN_VALUE

With an lnotab of "02 ff".

When executed, this produces these trace events:

   call on line 1
   line on line 2
   line on line 1
   return on line 1

--Ned.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CSMZXXEYISMTVDPTNOG4J65IZJ76OI2Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 626: Precise line numbers for debugging and other tools.

2020-07-17 Thread Ned Batchelder


https://www.python.org/dev/peps/pep-0626/ :)

--Ned.

On 7/17/20 10:48 AM, Mark Shannon wrote:

Hi all,

I'd like to announce a new PEP.

It is mainly codifying that Python should do what you probably already 
thought it did :)


Should be uncontroversial, but all comments are welcome.

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BMX32UARJFY3PZZYKRANS6RCMR2XBVVM/

Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SC5TWSCODSUZRMGNCXBBCV7AOS2LM2FO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Comments on PEP 554 (Multiple Interpreters in the Stdlib)

2020-04-22 Thread Ned Batchelder



On 4/21/20 12:32 PM, Mark Shannon wrote:

Hi,

I'm generally in favour of PEP 554, but I don't think it is ready to 
be accepted in its current form.



BTW, thanks for including the name of the PEP in the subject.  As a 
casual reader of this list, it's very helpful to have more than just the 
number, so I can decide whether to read into the deeper details.


--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5J27XI3ILH3YARS6DZKXZDDS2DRF3BRY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Any thoughts about a control flow optimizer for CPython?

2020-04-06 Thread Ned Batchelder

BTW, so that we don't have to completely retrace our steps, this topic
was also discussed in depth on Python-Ideas in May 2014:
https://mail.python.org/archives/list/python-id...@python.org/thread/X6VCB2E4EHOLUWHO42FYUT6VDAFNUHJF/

--Ned.

On 4/6/20 6:56 AM, Ned Batchelder wrote:

On 4/6/20 5:41 AM, Mark Shannon wrote:

Hi,

On 05/04/2020 12:47 pm, Ned Batchelder wrote:

On 4/3/20 11:13 AM, joannah nanjekye wrote:

Hey all,
From my CS theory, a control flow graph models a program flow and
one of its main characteristics is it has one entry and exit point.
IIRC, CPython’s compilation process involves generation of a
control flow graph.

Contrary to peephole optimizations, optimizations on the control
flow graph are more global allowing us to have complex and global
optimizations like branch and checkpoint eliminations etc.

I have seen several implementations of control flow optimizations.
The one I am familiar with is the V8 control flow optimizer.

I tried to investigate this for one of my directed courses last
fall but I want to know if there are people who have been thinking
about this for CPython and what their thoughts are.

Please make it possible to disable any optimizations. Sometimes
programs are run to understand the program, not for speed (for
example, during debugging, or coverage measurement.)

I have to disagree. The behavior of a program should not depend on
whether a flag is set or not. It makes debugging harder, IMO, if the
behavior changes depending on some flag.
I agree. But perhaps we need to be clearer what we mean by behavior.
I don't think an optimizer should change the behavior of a program,
that is, the results it produces. If an optimizer meets that
criterion, then enabling or disabling it doesn't affect the behavior
of the program.

For example, `pass` statements should never be executed.
This is an "optimization", but it is easier to understand if `pass`
is never executed. It becomes confusing it "executes" some of the
time, depending on some compiler flag.

IMO, a coverage tool should be not rely on `pass`, `while True:` and
other similarly trivial statements existing in the bytecode.
By "trivial statements", I mean any unconditional control flow
statements, conditional control flow statements with a constant test,
"pass", and any expression statements that are constant and without
side effect.

The results of a coverage tool depend very much on exactly which lines
get executed. If an optimizer decides it can skip a certain line, then
the coverage results will show that line as unexecuted. This alarms
users, and causes bug reports. See https://bugs.python.org/issue2506
for an example that has caused me headaches since 2008.

When you work in C, do you ever use the -O0 flag to disable
optimizations? If so, why do you use it? Why does Python not have the
same problems, and why does it not deserve a similar solution?

E.g. the following statements are "trivial":
try:
while False:
pass
1 + 1

On the other hand, other more powerful optimization should always
preserve the observable behavior of the program.
Since the observable behavior is unchanged, there would be no need to
turn them off.

My point is that coverage measurement is a different kind of
observation. Please don't overlook developers' needs in this regard.

--Ned

Cheers,
Mark.

[Python-Dev] Re: Any thoughts about a control flow optimizer for CPython?

2020-04-06 Thread Ned Batchelder

On 4/6/20 5:41 AM, Mark Shannon wrote:

Hi,

On 05/04/2020 12:47 pm, Ned Batchelder wrote:

On 4/3/20 11:13 AM, joannah nanjekye wrote:

Contrary to peephole optimizations, optimizations on the control
flow graph are more global allowing us to have complex and global
optimizations like branch and checkpoint eliminations etc.

I have seen several implementations of control flow optimizations.
The one I am familiar with is the V8 control flow optimizer.

I tried to investigate this for one of my directed courses last fall
but I want to know if there are people who have been thinking about
this for CPython and what their thoughts are.

Please make it possible to disable any optimizations. Sometimes
programs are run to understand the program, not for speed (for
example, during debugging, or coverage measurement.)

I have to disagree. The behavior of a program should not depend on
whether a flag is set or not. It makes debugging harder, IMO, if the
behavior changes depending on some flag.
I agree. But perhaps we need to be clearer what we mean by behavior. I
don't think an optimizer should change the behavior of a program, that
is, the results it produces. If an optimizer meets that criterion, then
enabling or disabling it doesn't affect the behavior of the program.

For example, `pass` statements should never be executed.
This is an "optimization", but it is easier to understand if `pass` is
never executed. It becomes confusing it "executes" some of the time,
depending on some compiler flag.

The results of a coverage tool depend very much on exactly which lines
get executed. If an optimizer decides it can skip a certain line, then
the coverage results will show that line as unexecuted. This alarms
users, and causes bug reports. See https://bugs.python.org/issue2506 for
an example that has caused me headaches since 2008.

When you work in C, do you ever use the -O0 flag to disable
optimizations? If so, why do you use it? Why does Python not have the
same problems, and why does it not deserve a similar solution?

E.g. the following statements are "trivial":
try:
while False:
pass
1 + 1

On the other hand, other more powerful optimization should always
preserve the observable behavior of the program.
Since the observable behavior is unchanged, there would be no need to
turn them off.

My point is that coverage measurement is a different kind of
observation. Please don't overlook developers' needs in this regard.

--Ned

Cheers,
Mark.

[Python-Dev] Re: Any thoughts about a control flow optimizer for CPython?

2020-04-05 Thread Ned Batchelder


On 4/3/20 11:13 AM, joannah nanjekye wrote:

Hey all,
From my CS theory, a control flow graph models a program flow and one 
of its main characteristics is it has one entry and exit point. IIRC, 
CPython’s compilation process involves generation of a control flow graph.


Contrary to peephole optimizations, optimizations on the  control flow 
graph are more global allowing us to have complex and global 
optimizations like branch and checkpoint eliminations etc.



I have seen several implementations of control flow optimizations. The 
one I am familiar with is the V8 control flow optimizer.



I tried to investigate this for one of my directed courses last fall 
but I want to know if there are people who have been thinking about 
this for CPython and what their thoughts are.




Please make it possible to disable any optimizations.  Sometimes 
programs are run to understand the program, not for speed (for example, 
during debugging, or coverage measurement.)


--Ned.


--
//Best,
Joannah Nanjekye
/"You think you know when you learn, are more sure when you can write, 
even more when you can teach, but certain when you can program."

Alan J. Perlis/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I376QAAWUHVLO5WGFFNOBTZAUPAVZKCB/
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UGEVQQWB32DQHBONJKTL2Q5K2YPCPIN2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-21 Thread Ned Batchelder

On 3/21/20 12:51 PM, Rob Cliffe via Python-Dev wrote:

On 21/03/2020 16:15, Eric V. Smith wrote:

On 3/21/2020 11:20 AM, Ned Batchelder wrote:

On 3/20/20 9:34 PM, Cameron Simpson wrote:

On 20Mar2020 13:57, Eric Fahlgren wrote:
On Fri, Mar 20, 2020 at 11:56 AM Dennis Sweeney

wrote:

If ``s`` is one these objects, and ``s`` has ``pre`` as a prefix,
then
``s.cutprefix(pre)`` returns a copy of ``s`` in which that prefix
has

been removed. If ``s`` does not have ``pre`` as a prefix, an
unchanged copy of ``s`` is returned. In summary,
``s.cutprefix(pre)``
is roughly equivalent to ``s[len(pre):] if s.startswith(pre) else
s``.

The second sentence above unambiguously states that cutprefix
returns 'an
unchanged *copy*', but the example contradicts that and shows that
'self'

may be returned and not a copy. I think it should be reworded to
explicitly allow the optimization of returning self.

My versions of these (plain old functions) return self if
unchanged, and are explicitly documented as doing so.

This has the concrete advantage that one can test for nonremoval if
the suffix with "is", which is very fast, instead of == which may
not be.

So one writes (assuming methods):

prefix = cutsuffix(s, 'abc')
if prefix is s:
... no change
else:
... definitely changed, s != prefix also

I am explicitly in favour of returning self if unchanged.

Why be so prescriptive? The semantics of these functions should be
about what the resulting string contains. Leave it to implementors
to decide when it is OK to return self or not.

The only reason I can think of is to enable the test above: did a
suffix/prefix removal take place? That seems like a useful thing. I
think if we don't specify the behavior one way or the other, people
are going to rely on Cpython's behavior here, consciously or not.

Is there some python implementation that would have a problem with
the "is" test, if we were being this prescriptive? Honest question.

Of course this would open the question of what to do if the suffix is
the empty string. But since "'foo'.startswith('')" is True, maybe
we'd have to return a copy in that case. It would be odd to have
"s.startswith('')" be true, but "s.cutprefix('') is s" also be True.
Or, since there's already talk in the PEP about what happens if the
prefix/suffix is the empty string, and if we adopt the "is" behavior
we'd add more details there. Like "if the result is the same object
as self, it means either the suffix is the empty string, or self
didn't start with the suffix".

Eric

*If* no python implementation would have a problem with the "is" test
(and from a position of total ignorance I would guess that this is the
case :-)), then it would be a useful feature and it is easier to
define it now than try to force conformance later. I have no problem
with 's.startswith("") == True and s.cutprefix("") is s'. YMMV.

Why take on that "*If*" conditional? We're constantly telling people
not to compare strings with "is". So why define how "is" will behave in
this PEP? It's the implementation's decision whether to return a new
immutable object with the same value, or the same object.

As Steven points out elsewhere in this thread, Python's builtins'
behavior differ, across methods and versions, in this regard. I
certainly didn't know that, and it was probably news to you as well. So
why do we need to nail it down for suffixes and prefixes?

There will be no conformance to force later, because if the value
doesn't change, then it doesn't matter whether it's a new string or the
same string.

--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/S3ILKW2HBJWFO4FCHEXITTP445URVHEG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-21 Thread Ned Batchelder


On 3/20/20 9:34 PM, Cameron Simpson wrote:

On 20Mar2020 13:57, Eric Fahlgren  wrote:
On Fri, Mar 20, 2020 at 11:56 AM Dennis Sweeney 


wrote:


If ``s`` is one these objects, and ``s`` has ``pre`` as a prefix, then
``s.cutprefix(pre)`` returns a copy of ``s`` in which that prefix has
been removed.  If ``s`` does not have ``pre`` as a prefix, an
unchanged copy of ``s`` is returned.  In summary, ``s.cutprefix(pre)``
is roughly equivalent to ``s[len(pre):] if s.startswith(pre) else s``.



The second sentence above unambiguously states that cutprefix returns 
'an
unchanged *copy*', but the example contradicts that and shows that 
'self'

may be returned and not a copy.  I think it should be reworded to
explicitly allow the optimization of returning self.


My versions of these (plain old functions) return self if unchanged, 
and are explicitly documented as doing so.


This has the concrete advantage that one can test for nonremoval if 
the suffix with "is", which is very fast, instead of == which may not be.


So one writes (assuming methods):

   prefix = cutsuffix(s, 'abc')
   if prefix is s:
   ... no change
   else:
   ... definitely changed, s != prefix also

I am explicitly in favour of returning self if unchanged.


Why be so prescriptive? The semantics of these functions should be about 
what the resulting string contains.  Leave it to implementors to decide 
when it is OK to return self or not.


--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/G3VIIQIGC4LTCEEV43S44RVDCGJ7AV42/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

2019-12-04 Thread Ned Batchelder


On 12/3/19 1:07 PM, Ethan Furman wrote:

On 12/03/2019 09:16 AM, Serhiy Storchaka wrote:

The 'u" string literal prefix was removed in 3.0 and reintroduced in 
3.3 to help writing the code compatible with Python 2 and 3 [1]. 
After the dead of Python 2.7 we will remove some deprecated features 
kept for compatibility with 2.7. When we are going to deprecate and 
remove the "u" prefix?


Can we gather all the 2.7 compatibility shims and remove them in 3.9?  
If not, I would say let's deprecate whatever needs deprecating and 
remove them in 3.10.  I don't think the u'' prefix needs the normal 
2-cyle deprecation since it was added specifically for 2/3 
cross-version code, and, as you say, "the dead of Python 2.7" has 
occurred (at least for us).


A few people in this thread have reasoned, "since we added it just to 
help people with 2.7, we can remove it now."  But that's exactly why it 
will be a significant cost to the community: lots of people made use of 
this useful needed thing. Removing the prefix will be a speed bump for 
people's use of Python, and will slow down people who want to upgrade to 
newer versions of Python.


--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6WX5GANERILH4TAMJAB2I55HRW5VTT6G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

2019-12-04 Thread Ned Batchelder

On 12/4/19 3:11 AM, Serhiy Storchaka wrote:

04.12.19 04:41, Ned Batchelder пише:

On 12/3/19 8:13 PM, Inada Naoki wrote:

I think it is too early to determine when to remove it.
Even only talking about it causes blaming war.

Has anyone yet given a reason to remove it? It will change working
code into broken code. Why do that?

Why the "<>" operator and the "L" suffix was removed?

Is this a serious question? Many things were removed in moving from
Python 2 to Python 3. It was explicitly decided that 2->3 would contain
breaking changes. If you recall, this caused a large amount of
controversy. Why bring that on ourselves again?

Maybe I missed something. Python used to pride itself on keeping old
code working. When hash randomization was introduced, it was decided to
be off by default in Python 2 because even though people shouldn't have
counted on the order of dicts, they were counting on it, and turning on
hash randomization would break code. So there we decided to keep things
insecure because otherwise code would break, even "wrong" code.

Now just because "it isn't needed anymore" we're going to break working
code? I don't understand the shift in philosophy.

--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/OGM2SHWH6ZJ434YVHGX2DXTG3HHSUZ6U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

2019-12-03 Thread Ned Batchelder

On 12/3/19 8:13 PM, Inada Naoki wrote:

I think it is too early to determine when to remove it.
Even only talking about it causes blaming war.

Has anyone yet given a reason to remove it? It will change working code
into broken code. Why do that?

--Ned.

BTW, I think 2to3 can help to move from 2&3 code to 3-only code.

* "future" fixer can be remove legacy futures. But it seems to remove
all futures,
including "annotations". It should be kept.
* "unicode" fixer can be used to remove u-prefix. But I'm not sure yet.

Are there any other things which are used for writing 2&3 code and
will be removed
someday in the future?
And is there any tool that can convert code using the "six" library to
normal Python 3 code?

Regards,

On Wed, Dec 4, 2019 at 2:29 AM Serhiy Storchaka wrote:

The 'u" string literal prefix was removed in 3.0 and reintroduced in 3.3
to help writing the code compatible with Python 2 and 3 [1]. After the
dead of Python 2.7 we will remove some deprecated features kept for
compatibility with 2.7. When we are going to deprecate and remove the
"u" prefix?

[1] https://www.python.org/dev/peps/pep-0414/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/EMFDKQ57JVWUZ6TPZM5VTFW7EUKVYAOY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

2019-12-03 Thread Ned Batchelder


On 12/3/19 12:16 PM, Serhiy Storchaka wrote:
The 'u" string literal prefix was removed in 3.0 and reintroduced in 
3.3 to help writing the code compatible with Python 2 and 3 [1]. After 
the dead of Python 2.7 we will remove some deprecated features kept 
for compatibility with 2.7. When we are going to deprecate and remove 
the "u" prefix?


[1] https://www.python.org/dev/peps/pep-0414/


Can you clarify this?: "we will remove some deprecated features kept for 
compatibility with 2.7"  Where is the discussion of the features being 
removed?


--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PY24FGY3GLHZP6X4G2UPYEAD5INVEHHV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

2019-12-03 Thread Ned Batchelder

On 12/3/19 2:41 PM, Christian Heimes wrote:

On 03/12/2019 19.09, Barry Warsaw wrote:

On Dec 3, 2019, at 09:16, Serhiy Storchaka
wrote:

The 'u" string literal prefix was removed in 3.0 and reintroduced
in 3.3 to help writing the code compatible with Python 2 and 3
[1]. After the dead of Python 2.7 we will remove some deprecated
features kept for compatibility with 2.7. When we are going to
deprecate and remove the "u" prefix?

I think removing or even deprecating it will cause unnecessary code
churn in third party libraries, for very little value. Let’s just
call it a wart until Python 4000.

I'm 100% with Barry. We can certainly document the "u" string prefix
as deprecated. I'm strongly against removing it from Python 3 or even
raising a deprecation warning. Let's leave it to linters and maybe
consider deprecation for Python 4. Or maybe not.

Yes, please, please, don't remove this prefix. It will cause needless
churn, and force people to revisit code that works just fine. We should
try to keep old code working unless there is a clear benefit in making a
breaking change. I haven't heard a benefit to removing this prefix.

"We added it just to help people migrate" is not a reason to remove it now.

--Ned.

Christian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/4GGYZ6XNEH5OQDYC6VLH2MADUVD5LGF3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The Python 2 death march

2019-09-10 Thread Ned Batchelder


On 9/10/19 11:06 AM, Benjamin Peterson wrote:

On Tue, Sep 10, 2019, at 15:54, Ned Batchelder wrote:

Maybe I'm not involved enough in the release process, but this seems
confusing to me.  On the same day that the PSF put up a page about the
1/1/2020 date, we choose April 2020 as the last release?  Why?  I
thought the point was to save core devs efforts.  Is this an unofficial
grace period?  Will there be a public document that announces the April
date?

This mail is a public document.

The thinking is that ±2 months is rounding error in Python 2's lifetime, so why not move 
it to a significant community time? (There was never going to be a release exactly on 
January 1 simply because it's an inconvenient time of the year for "work".) I 
suppose practically it amounts to a small grace period if core devs willing to merge 2.7 
changes post 2020-01-01 can be found.



I'm not looking forward to answering questions from the public about why 
the PSF is writing dire and specific warnings like "We have decided that 
January 1, 2020, will be the day that we sunset Python 2," while the 
core devs are planning a release four months after that.  It won't help 
Python's credibility, and may convince some people that they don't have 
to take the date seriously..


--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZTXTUW2WGSZSXP2LCQFUVHSBJYBZUHJQ/

[Python-Dev] Re: The Python 2 death march

2019-09-10 Thread Ned Batchelder

Maybe I'm not involved enough in the release process, but this seems 
confusing to me.  On the same day that the PSF put up a page about the 
1/1/2020 date, we choose April 2020 as the last release?  Why?  I 
thought the point was to save core devs efforts.  Is this an unofficial 
grace period?  Will there be a public document that announces the April 
date?


What does the "official EOL date" mean if there's a release in April?

--Ned.

On 9/9/19 9:29 AM, Benjamin Peterson wrote:

Hi all,
It's finally time to schedule the last releases in Python 2's life. There will 
be two more releases of Python 2.7: Python 2.7.17 and Python 2.7.18.

Python 2.7.17 release candidate 1 will happen on October 5th followed by the 
final release on October 19th.

I'm going to time Python 2.7.18 to coincide with PyCon 2020 in April, so 
attendees can enjoy some collective catharsis. We'll still say January 1st is 
the official EOL date.

Thanks to Sumana Harihareswara, there's now a FAQ about the Python 2 sunset on 
the website: https://www.python.org/doc/sunset-python-2/

Regards,
Benjamin
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/APWHFYQDKNVYQAK3HZMBGQIZHAVRHCV2/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TVIMZGRZQXDAXKARJBUEZJJ7YSTNWMKA/

[Python-Dev] Re: Removing dead bytecode vs reporting syntax errors

2019-07-05 Thread Ned Batchelder

I can understand the desire for correctness.  I do have to wonder 
though: has anyone *other* than language implementors noticed this issue 
with SyntaxErrors not being reported?

Perhaps we should remember "Practicality beats purity"?

--Ned.

On 7/5/19 6:14 PM, Pablo Galindo Salgado wrote:

> Until a solution which makes everyone happy can be found, I suggest to
move back to the status quo: revert the change.

>More people seems to expect "if 0: ..." to be removed, than people who
care of syntax errors on "if 0".

I don't think this is that clear. As Paul wrote on the issue this is 
the  result of fixing a bug that has been open since
2008 (11 years), which itself was noticed independently, also in 2008 
(#1875 and #1920, respectively).
He also independently discovered the same issue last year when writing 
some tests for IPython.

https://bugs.python.org/msg347394

On Fri, 5 Jul 2019 at 23:10, Victor Stinner <mailto:vstin...@redhat.com>> wrote:

Hi,

Until a solution which makes everyone happy can be found, I suggest to
move back to the status quo: revert the change.

More people seems to expect "if 0: ..." to be removed, than people who
care of syntax errors on "if 0".

--

Would it be possible to detect if the "if 0" block would raise a
syntax error, and only remove it if it doesn't raise a syntax error?

That's the approach I chose in my fatoptimizer project which is
implemented as an AST optimizer:

https://github.com/vstinner/fatoptimizer/blob/master/fatoptimizer/dead_code.py

See the tests to see which cases are *not* optimized:

https://github.com/vstinner/fatoptimizer/blob/master/test_fatoptimizer.py#L2428

Some examples (the "dead code elimitaiton" is not only about "if 0",
but also "while 0", dead code after return, etc.):

    self.check_dont_optimize("""
        def func():
            if 0:
                yield
    """)

    self.check_dont_optimize("while 1: x = 1")

    self.check_dont_optimize("""
        def func(obj):
            return
            if 0:
                yield from obj
    """)

    self.check_dont_optimize("""
        try:
            pass
        except Exception:
            yield 3
    """)

See also the doc:
https://fatoptimizer.readthedocs.io/en/latest/optimizations.html#dead-code

--

About code coverage, it seems like -X noopt would help:
https://github.com/python/cpython/pull/13600

But I'm not sure anymore after Ned Batchelder wrote:

"The real-word implications from my world are this: if your code has
"if 0:" clauses in it, and you measure its coverage, then because the
lines have not been optimized away, coverage.py will think the lines
are a possible execution path, and will be considered a miss because
they are not executed.  This will reduce your coverage percentage."
https://bugs.python.org/issue37500#msg347362

Does it mean that coverage.py will report even more "false positive"
using -X noopt?

Victor

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LWZPZQ2IL67DPX3RC342TTOVCLKDSBTJ/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4VORK4Q2BNHN6SQUU423TODCXCWN77NT/

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-21 Thread Ned Batchelder


On 5/21/19 8:37 AM, Christian Heimes wrote:

On 21/05/2019 14.06, Anders Munch wrote:

Fra: Python-Dev [mailto:python-dev-bounces+ajm=flonidan...@python.org] På vegne 
af Christian Heimes

* The removed modules will be available through PyPI.

Will they?  That's not the impression I got from the PEP.

It's all open source. It's up to the Python community to adopt packages and 
provide them on PyPI.

Python core will not maintain and distribute the packages. I'll merely provide 
a repository with packages to help kick-starting the process.



This statement should be in the PEP in some form. For example: "The 
modules could be made available on PyPI. The Python core team will not 
publish or maintain the packages."


--Ned.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Ned Batchelder


On 3/19/19 4:13 AM, Serhiy Storchaka wrote:

19.03.19 00:41, Raymond Hettinger пише:
3) Add a standards compliant canonicalization tool (see 
https://en.wikipedia.org/wiki/Canonical_XML ).  This is likely to be 
the right-way-to-do-it but takes time and energy.


4) Fix the tests in the third-party modules to be more focused on 
their actual test objectives, the semantics of the generated XML 
rather than the exact serialization.  This option would seem like the 
right-thing-to-do but it isn't trivial because the entire premise of 
the existing test is invalid.  For every case, we'll actually have to 
think through what the test objective really is.


I think the combination of options 3 and 4 is the right thing. Not 
always the stable output is needed, in these cases option 4 should be 
considered. But there are valid use cases for the stable output, in 
these cases we need to provide an alternative in the stdlib. I am 
working on this.


Option 4 is misleading.  Is anyone here really offering to "fix the 
tests in third-party modules"?  Option 4 is actually, "do nothing, and 
let a multitude of projects figure out how to fix their tests, slowing 
progress in those projects as they try to support Python 3.8."


In my case, the test code has a generic function to compare an actual 
directory of files to an expected directory of files, so it isn't quite 
as simple as "just use the right XML comparison."  And I support Python 
2.7, 3.5, etc, so tests still need to work under those versions.  None 
of this is impossible, but please try not to preach to us maintainers 
that we are doing it wrong, that it will be easy to fix, etc.  Using 
language like "the entire premise of the test is invalid" seems 
needlessly condescending.


As one of the suggested solutions, a DOM comparison is not enough. I 
don't just want to know that my actual XML is different than my expected 
XML.  I want to know where and how it differs.


Textual comparison may be the "wrong" way to check XML, but it gives me 
many tools for working with the test results.  It was simple and it 
worked.  Now in Python 3.8, because Python doesn't want to add an 
optional flag to continue doing what it has always done, I need to 
re-engineer my tests.


--Ned.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20

2018-04-28 Thread Ned Batchelder


On 4/27/18 5:28 PM, Tres Seaver wrote:

On 04/27/2018 05:11 PM, Tim Peters wrote:


In this specific case, line-oriented coverage tools have missed
accounting for all possible code paths since day #1; e.g.,

x = f() or g()

You don't need to reply to messages so obviously irrelevant to the PEP
unless you want to.  It's not like Guido will read them and go "oh!  a
binding expression in a ternary conditional is a fundamentally new
potential problem for a line-oriented coverage tool!  that's fatal" ;-)

FWIW, Ned Batchelder's 'coverage.py' does a good job with branch coverage.
I haven't seen anything in this discussion which indicates that binding
expressions will change that at all.




Coverage.py can measure branch coverage, but it is still line-oriented.  
There's no support for conditionals and branches within a line (though 
I've done some wicked hacks to experiment with it: 
https://nedbatchelder.com/blog/200804/wicked_hack_python_bytecode_tracing.html).


It's entirely true that binding expressions don't change this situation 
at all, EXCEPT that the entire point of binding expressions is to be 
able to express in one statement what used to take more than one.  With 
binding expressions, actions may be coverage-hidden within one statement 
that without them would have been coverage-visible in more than one 
statement.


I'm not sure that's an argument against binding expressions, but we 
should at least acknowledge that the motivation for them is to provide 
the option to write fewer, longer statements.  That option is not always 
a good idea, for a variety of reasons.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

2017-12-29 Thread Ned Batchelder


On 12/29/17 1:59 PM, Guido van Rossum wrote:
Regarding whether this should live on PyPI first, in this case that 
would not be helpful, since attrs is already the category killer on 
PyPI. So we are IMO taking the best course possible given that we want 
something in the stdlib but not exactly attrs.


It always seemed to me that the reason to recommend putting something on 
PyPI first wasn't so that it would climb up some kind of leaderboard, 
but so that people could get real-world experience with it before 
freezing it into the stdlib.  If we think people won't start using data 
classes from PyPI, why do we think it's important to get into the stdlib?


It still seems to me like there are open questions about how data 
classes should work. Getting people using it will be a good way to get 
the best design before our hands are tied with backward compatibility in 
the stdlib.  What is the rush to put a new design into the stdlib?  
Presumably it is better than attrs (or we would have simply adopted 
attrs).  Having data classes on PyPI will be a good way to gauge acceptance.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

2017-12-26 Thread Ned Batchelder


On 12/26/17 1:49 PM, Chris Barker wrote:
On Sat, Dec 23, 2017 at 5:54 PM, Nick Coghlan > wrote:



I still wonder about the "fields *must* be annotated" constraint
though. I can understand a constraint that the style be
*consistent* (i.e. all fields as annotations, or all fields as
field instances), since that's needed to determine the field
order, but I don't see the problem with the "no annotations" style
otherwise.


IIUC, without annotations, there is no way to set a field with no default.

And supporting both approaches violates "only one way to do it" in, I 
think, a confusing manner -- particularly if you can't mix and match them.


Also, could does using class attributes without annotations make a 
mess when subclassing? -- no I haven't thought that out yet.





I have not been following the design of dataclasses, and maybe I'm 
misunderstanding the state of the work.  My impression is that attrs was 
a thing, and lots of people loved it, so we wanted something like it in 
the stdlib.  Data Classes is that thing, but it is a new thing being 
designed from scratch.  There are still big questions about how it 
should work, but it is already a part of 3.7.


Often when people propose new things, we say, "Put it on PyPI first, and 
let's see how people like it."  Why isn't that the path for Data 
Classes?  Why are they already part of 3.7 when we have no practical 
experience with them?  Wouldn't it be better to let the design mature 
with real experience?  Especially since some of the questions being 
asked are about how it interrelates with another large new feature with 
little practical use yet (typing)?


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 unittest does not support namespace packages for discovering

2017-03-23 Thread Ned Batchelder

On 3/23/17 3:14 PM, Robert Collins wrote:
> On 24 March 2017 at 04:59, INADA Naoki  wrote:
>> And this issue is relating to it too: http://bugs.python.org/issue29716
>>
>> In short, "namespace package" is for make it possible to `pip install
>> foo_bar foo_baz`,
>> when foo_bar provides `foo.bar` and foo_baz provides `foo.baz`
>> package.  (foo is namespace package).
>>
>> If unittests searches normal directly, it may walk deep into very
>> large tree containing
>> millions of directories.  I don't like it.
> That is a risk, OTOH I think the failure to do what folk expect is a
> bigger risk.

The issue here is, what do folks expect?  PEP 420 is pretty clear on its
purpose. The first sentence of the abstract:

> Namespace packages are a mechanism for splitting a single Python
package across multiple directories on disk.

And the first sentence of the specification:

> Regular packages will continue to have an __init__.py and will reside
in a single directory.

PEP 420 is not meant to make all __init__.py files optional. It has a
specific purpose.  These proposed changes are not in support of that
purpose.  We should not bend over backwards to support getting rid of
__init__.py files just because people don't like empty __init__.py
files. That's not what PEP 420 is for.

--Ned.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Recent changes to PyCodeObject

2016-11-17 Thread Ned Batchelder

On 11/17/16 10:09 AM, Cody Piersall wrote:
> On Wed, Nov 16, 2016 at 6:18 PM, Ned Batchelder <n...@nedbatchelder.com> 
> wrote:
>> When I added Python 3.6 support to coverage.py, I posted a Mac wheel to
>> PyPI: https://pypi.python.org/pypi/coverage/  That wheel was built
>> against 3.6a3, the latest version at the time.  When I use it now on
>> 3.6b3, it doesn't work right.  The reason is that the co_firstlineno
>> field in PyCodeObject moved in September, as part of the PEP 523 work:
>> https://github.com/python/cpython/commit/f1e6d88b3ca2b56d51d87b6b38ea1870c5d9490c
>>
>> The docs say that PyCodeObject can change at any time, but I don't see
>> why the field had to move in the first place.  Was this needed?
> Christian Heimes had the idea so that memory consumption wouldn't be
> impacted by adding co_extra to the struct:
> https://mail.python.org/pipermail/python-dev/2016-August/145961.html
>

Thanks, I should have searched :)

--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Recent changes to PyCodeObject

2016-11-16 Thread Ned Batchelder

When I added Python 3.6 support to coverage.py, I posted a Mac wheel to
PyPI: https://pypi.python.org/pypi/coverage/  That wheel was built
against 3.6a3, the latest version at the time.  When I use it now on
3.6b3, it doesn't work right.  The reason is that the co_firstlineno
field in PyCodeObject moved in September, as part of the PEP 523 work:
https://github.com/python/cpython/commit/f1e6d88b3ca2b56d51d87b6b38ea1870c5d9490c

The docs say that PyCodeObject can change at any time, but I don't see
why the field had to move in the first place.  Was this needed?

Am I doing the wrong thing by using PyCodeObject fields directly in the
coverage.py C trace function?  It seems like this was an unnecessary
breaking change, even if it is a non-guaranteed interface.

BTW, there are some comments that should be updated based on that
commit, I can submit a patch to fix them.

--Ned.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is there a reference manual for Python bytecode?

2015-12-27 Thread Ned Batchelder




On 12/26/15 10:19 PM, Nick Coghlan wrote:

On 27 December 2015 at 12:23, Guido van Rossum  wrote:

On Sat, Dec 26, 2015 at 6:06 PM, Brett Cannon  wrote:

Ned also neglected to mention his byterun project which is a pure Python
implementation of the CPython eval loop: https://github.com/nedbat/byterun

 From the commit log it looks like it's a co-production between Ned and
Allison Kaptur (who gave the talk I mentioned).
Yes, Allison was very helpful in pushing it forward.  And I should also 
mention that I started with a dormant project that Paul Swartz wrote.


And: it doesn't work completely.  There are things it doesn't handle 
properly, and I turned to other projects some time ago.  If someone 
wants to pick it up, that would be cool.

It occurred to me that "byterun" would make a good see-also link from
the dis module docs, and looking into that idea brought me to this
article Allison wrote about it for the "500 lines" project:
http://aosabook.org/en/500L/a-python-interpreter-written-in-python.html

For a detailed semantic reference, byterun's eval loop is likely one
of the most readable sources of information:
https://github.com/nedbat/byterun/blob/master/byterun/pyvm2.py
I started working on byterun because I knew I didn't understand bytecode 
well enough to implement branch coverage measurement in coverage.py.  
Its primary goal was teaching (me!) how the bytecode worked.


Recently though, I've started a new implementation of branch coverage 
based on the ast rather than the bytecode.  This was prompted by the 
"async" keywords in 3.5.  "async for" and "for" compile very differently 
to bytecode, which caused headaches for a bytecode-based understanding 
of flow, so I'm trying out an ast-based understanding.


--Ned.

In terms of formal documentation, the main problem with providing
reference bytecode tables is keeping them up to date as the eval loop
changes. However, it would theoretically be possible to create a
custom Sphinx directive that uses the dis module to generate the
tables automatically during the docs build process, rather than
maintaining them by hand - something like that could be experimented
with outside CPython, and potentially incorporated into the dis module
docs if folks are able to figure out something that works well.

Regards,
Nick.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is there a reference manual for Python bytecode?

2015-12-26 Thread Ned Batchelder


On 12/26/15 6:13 PM, Erik wrote:

On 26/12/15 23:10, Joe Jevnik wrote:
All arguments are 2 bytes, if there needs to be more, EXTENDED_ARG is 
used


OK, got it - many thanks.
One thing to understand that may not be immediately apparent: the byte 
code can (and does) change between versions, so Python 2.7 doesn't have 
the exact same byte code as 3.4, which is also different from 3.5.


--Ned.


E.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython (3.5): remove STORE_MAP, since it's unused

2015-05-28 Thread Ned Batchelder


On 5/28/15 3:52 PM, Serhiy Storchaka wrote:

On 28.05.15 22:40, benjamin.peterson wrote:

https://hg.python.org/cpython/rev/ac891c518d4e
changeset:   96342:ac891c518d4e
branch:  3.5
parent:  96339:6f05f83c7010
user:Benjamin Peterson benja...@python.org
date:Thu May 28 14:40:08 2015 -0500
summary:
   remove STORE_MAP, since it's unused


Wouldn't it break support of .pyc files compiled with older versions 
of Python 3?
.pyc files are not compatible across versions of Python.  3.5 changed 
the meaning of the BUILD_MAP opcode, for example.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Issue 24285: regression for importing extensions in packages

2015-05-27 Thread Ned Batchelder

This issue has been fixed, but a day or two late for 3.5b1.  It prevents 
loading the coverage.py extension.  It'd be great to get a new beta 
release soon. :)


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The pysandbox project is broken

2013-11-12 Thread Ned Batchelder


On 11/12/13 6:48 PM, Terry Reedy wrote:

On 11/12/2013 4:16 PM, Victor Stinner wrote:


It would also be nice to help developers looking for a sandbox for
their application. Please tell me if you know sandbox projects for
Python so I can redirect users of pysandbox to a safer solution. I
already know PyPy sandbox.


There are several websites running submitted Python code (and in some 
cases, many other languages).

ProjectEuler
CodeAcademy (I think they use someone else's code box)
CheckIO.org - python only
other coding challenge sites
I suspect they use sandboxed processes but have not seen anyone talk 
about what they are doing.




At edX, we use CodeJail to apply OS-level sandboxing to untrusted Python 
code: https://github.com/edx/codejail


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEPs shouldn't be considered docs

2013-10-11 Thread Ned Batchelder

I wanted to teach a co-worker about from __future__ import 
absolute_import today, so I thought I'd point them at the docs. The 
page for __future__ starts with a bunch of internal details that 
almost no one needs to know.  There's a table at the end that mentions 
the actual importable names, including absolute_import, but says 
nothing about then except to link to a PEP.


The absolute imports PEP has no simple description of what it does. Like 
many PEPs, it's mostly a summary of the debate around the design of the 
feature. The closest the PEP comes to describing the behavior of 
absolute_import is this parenthetical:


   For the second problem, it is proposed that all import statements be
   absolute by default (searching sys.path only) with special syntax
   (leading dots) for accessing package-relative imports.

And notice: that sentence describes it as a proposal.

I'd like to suggest that we not consider PEPs to be documentation. If a 
PEP has a good succinct description of how something works, then let's 
copy that text into the documentation at an appropriate place.  If a PEP 
doesn't have such a description, then all the more reason not to send 
readers there.


A link to the PEP is appropriate as a see also in the docs, but we 
shouldn't pretend that a PEP addresses the needs of people new to the 
feature.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Use cached builtins.

2013-10-02 Thread Ned Batchelder


On 10/2/13 1:31 PM, Antoine Pitrou wrote:

On Wed,  2 Oct 2013 18:16:48 +0200 (CEST)
serhiy.storchaka python-check...@python.org wrote:

http://hg.python.org/cpython/rev/d48ac94e365f
changeset:   85931:d48ac94e365f
user:Serhiy Storchaka storch...@gmail.com
date:Wed Oct 02 19:15:54 2013 +0300
summary:
   Use cached builtins.

What's the point? I don't think it's a good idea to uglify the code if
there isn't a clear benefit.


This type of micro-optimization seems especially unnecessary in a module 
intended for printing.


--Ned.


Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why not support user defined operator overloading ?

2013-09-29 Thread Ned Batchelder


On 9/29/13 11:06 AM, Xavier Morel wrote:

This is more of a python-ideas subject.

And one of the reasons likely is that it would require significantly
reworking the grammar to handle a kind of user-defined opname (similar
to name, but for operator tokens), with user-defined priority and
associativity, and the ability to import operators (or define how and
when operators become available compared to their definition)

That's a huge amount of complexity with little to gain.


It isn't just a matter of a more complex parser: where would the parser 
get the information about these new operators?  The obvious first answer 
is that they would be defined as part of classes, but that means the 
operator definition is in an imported module some place.  The importing 
module couldn't even be parsed until the class was imported.  This is a 
completely different compilation and execution model than Python has now.


Now you can compile a Python module without any access to the modules it 
will eventually import.  Importing is a run-time operation.  If you want 
user-defined classes to be able to define new syntax, then importing has 
to happen at compile time.  That's a completely different language.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why not support user defined operator overloading ?

2013-09-29 Thread Ned Batchelder


On 9/29/13 9:02 PM, 张佩佩 wrote:

On 2013/9/30 8:53 Greg Ewing wrote:

It does need to know the operator's precedence and
associativity, though, which means either declaring
it somewhere, or having some kind of fixed rule

I suggest all user defined operator are at lowest priority.


Peipei, this is an interesting discussion, but doesn't belong on 
Python-Dev.  The Python-Ideas mailing list is for exactly this kind of 
topic.


--Ned.


Regards
peipei
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]

2013-05-19 Thread Ned Batchelder


On 5/19/2013 7:22 PM, Mark Janssen wrote:

On Sun, May 19, 2013 at 1:13 PM, Tres Seaver tsea...@palladion.com wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 05/19/2013 10:48 AM, Guido van Rossum wrote:

Anyway, if you're doing arithmetic on enums you're doing it wrong.

Hmm, bitwise operations, even?

I think it's rather pointless to do bitwise operations on python
enums.  We're not that close to the machine.


It makes sense if the enums represent bit-oriented values that will be 
used close to the machine.  Python is used in many disciplines.


--Ned.


MarkJ
Tacoma, Washington
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Usage of += on strings in loops in stdlib

2013-02-12 Thread Ned Batchelder


On 2/12/2013 4:16 PM, Brett Cannon wrote:




On Tue, Feb 12, 2013 at 4:06 PM, Antoine Pitrou solip...@pitrou.net 
mailto:solip...@pitrou.net wrote:



Hi !

On Tue, 12 Feb 2013 23:03:04 +0200
Maciej Fijalkowski fij...@gmail.com mailto:fij...@gmail.com wrote:

 We recently encountered a performance issue in stdlib for pypy. It
 turned out that someone commited a performance fix that uses
+= for
 strings instead of .join() that was there before.

 Now this hurts pypy (we can mitigate it to some degree though) and
 possible Jython and IronPython too.

 How people feel about generally not having += on long strings in
 stdlib (since the refcount = 1 thing is a hack)?

I agree that += should not be used as an optimization (on strings) in
the stdlib code. The optimization is there so that uncareful code does
not degenerate, but deliberately relying on it is a bit devilish.
(optimisare diabolicum :-))


Ditto from me. If you're going so far as to want to optimize Python 
code then you probably are going to care enough to accelerate it in C, 
in which case you can leave the Python code idiomatic.


But the only reason .join() is a Python idiom in the first place is 
because it was the fast way to do what everyone initially coded as s 
+=    Just because we all learned a long time ago that joining was 
the fast way to build a string doesn't mean that .join() is the clean 
idiomatic way to do it.


--Ned.




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do more at compile time; less at runtime

2012-12-11 Thread Ned Batchelder


On 12/9/2012 5:22 PM, Mark Shannon wrote:
The current CPython bytecode interpreter is rather more complex than 
it needs to be. A number of bytecodes could be eliminated and a few 
more simplified by moving the work involved in handling compound 
statements (loops, try-blocks, etc) from the interpreter to the compiler.


As with all suggestions to optimize the bytecode generation, I'd like to 
re-iterate the need for a way to disable all optimization, for the sake 
of reasoning about the program.  For example, debugging, coverage 
measurement, etc.  This idea was misunderstood and defeated in 
http://bugs.python.org/issue2506, but I strongly believe it is important.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

2012-11-07 Thread Ned Batchelder



On 11/6/2012 5:12 PM, Guido van Rossum wrote:

On Tue, Nov 6, 2012 at 9:58 AM, Ned Batchelder n...@nedbatchelder.com wrote:

On 11/6/2012 11:26 AM, R. David Murray wrote:

On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka storch...@gmail.com
wrote:

Another counterintuitive (and possible wrong) example:

  {print('foo'): print('bar')}
 bar
 foo
 {None: None}

http://bugs.python.org/issue11205


This seems to me better left undefined, since there's hardly ever a need to
know the precise evaluation sequence between keys and values, and retaining
some amount of unspecified to allow for implementation flexibility is a
good thing.

Maybe. Do note that Python tries to be *different* than your average
C++ standard and actually prescribes evaluation orders in most cases.
The idea being that the kind of optimizations that C++ compilers get
to do by moving evaluation order around aren't worth it in Python
anyway, and it's better for the user if there are no arbitrary
differences in this area between Python implementations. Note that I
didn't say no surprises -- the post that started this thread shows
that surprises are still possible.



I think it's unfortunate that the current patch in the referenced bug ( 
http://bugs.python.org/issue11205 ) fixes the problem by adding one 
more bytecode to the compiled Python.  The other alternative seems to be 
changing the meaning of two opcodes. Neither of these are great 
alternatives.  I know we don't promise to maintain backward 
compatibility in the meanings of opcodes, but I'll bet actual code will 
be broken by changing the meaning of STORE_MAP and MAP_ADD.  Slowing 
down dict displays (just slightly) to get this right seems unfortunate also.


If the bug report is accurate, CPython and the reference manual have 
disagreed since Python 2.5, and many of us are now surprised to hear it, 
which means there can't have been much broken code.


I understand the point about C compilers having more opportunities to 
take advantage of undefined in the spec, but Python implementations 
are getting cleverer about how to implement Python semantics as well, 
perhaps we should have some faith in their future cleverness and give 
them even a little bit of leeway.  I don't imagine this little bit will 
actually be useful to them, but making this undefined will already help 
CPython by avoiding either an extra bytecode or a change in the opcodes.


There are plenty of places where different Python implementations 
differ, and even careful observers had different ideas about how keys 
and values were ordered in dict displays (I thought it was like a 
function call,  vs, I thought it was like an assignment). We've gone 
out of our way to maintain backward compatibility with the implemented 
behavior before (ordering of dict keys, for example).  The simplest 
change to make here is to update the reference and keep the implementation.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

2012-11-07 Thread Ned Batchelder



On 11/7/2012 12:08 PM, Serhiy Storchaka wrote:

On 07.11.12 17:12, Nick Coghlan wrote:

Since you've indicated the implementation is in the wrong here and you
also want to preserve opcode semantics, I think Skip's patch is
correct, but also needs to be applied to dict comprehensions (now we
have them). The extra bytecode is only ROT_TWO, which is one of the
cheapest we have kicking around :)


Not only to dict comprehensions, but also to item assignments.  It 
will be weird if a dict comprehension and a plain loop will be 
inconsistent.



Just to be clear: the reference guide says that the behavior *SHOULD BE* 
(but is not yet) this:


   Python 3.3.0
 {print(a):print(b)}
   a
   b
   {None: None}
 d = {}
 d[print(a)] = print(b)
   b
   a


Is this or is this not weird to you?

--Ned.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

2012-11-07 Thread Ned Batchelder


On 11/7/2012 5:11 PM, Terry Reedy wrote:

On 11/7/2012 4:39 PM, Ned Batchelder wrote:


Just to be clear: the reference guide says that the behavior *SHOULD BE*
(but is not yet) this:

Python 3.3.0
  {print(a):print(b)}
a
b
{None: None}
  d = {}
  d[print(a)] = print(b)
b
a
 

Is this or is this not weird to you?


Not weird. Expressions and assignment targets are each consistently 
evaluated left to right (except as *necessarily* alter by precedence), 
with expressions evaluated before targets.


Sorry, I should have been clearer: I was asking about weird not to say, 
This is weird and should be changed!, but to get clarification from 
Serhiy about his statement,  It will be weird if a dict comprehension 
and a plain loop will be inconsistent.  I honestly didn't know which 
behavior he considered inconsistent and therefore weird.


--Ned.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

2012-11-06 Thread Ned Batchelder


On 11/6/2012 11:26 AM, R. David Murray wrote:

On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka storch...@gmail.com 
wrote:

Another counterintuitive (and possible wrong) example:

 {print('foo'): print('bar')}
bar
foo
{None: None}

http://bugs.python.org/issue11205


This seems to me better left undefined, since there's hardly ever a need 
to know the precise evaluation sequence between keys and values, and 
retaining some amount of unspecified to allow for implementation 
flexibility is a good thing.


--Ned.



--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

2012-11-06 Thread Ned Batchelder



On 11/6/2012 1:19 PM, Devin Jeanpierre wrote:



On Nov 6, 2012 1:05 PM, Ned Batchelder n...@nedbatchelder.com 
mailto:n...@nedbatchelder.com wrote:


 On 11/6/2012 11:26 AM, R. David Murray wrote:

 On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka 
storch...@gmail.com mailto:storch...@gmail.com wrote:


 Another counterintuitive (and possible wrong) example:

  {print('foo'): print('bar')}
 bar
 foo
 {None: None}

 http://bugs.python.org/issue11205


 This seems to me better left undefined, since there's hardly ever a 
need to know the precise evaluation sequence between keys and values, 
and retaining some amount of unspecified to allow for implementation 
flexibility is a good thing.


Left undefined? The behavior was defined, but CPython didn't follow 
the defined behaviour.




I would change the reference manual to leave it undefined.  Clearly not 
many people have been bothered by the fact that CPython implemented it 
wrong.  If someone really needs to control whether the keys or values 
are evaluated first, they shouldn't use a dict literal.


--Ned.


--Devin (phone)



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Bug Day in October

2012-09-30 Thread Ned Batchelder


On 9/27/2012 6:30 PM, Éric Araujo wrote:

Hi all,

The Montreal-Python user group would like to host a bug day on October
27 (to be confirmed) at a partner university in Montreal.  It would be
cool to do a bug day on IRC like we used to (and in other physical
locations if people want to!) to get new contributors and close bugs.
What do you think?

Boston Python might get a physical location and join in.  Keep us posted!

--Ned.


Cheers
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST optimizer implemented in Python

2012-08-11 Thread Ned Batchelder


On 8/11/2012 2:30 PM, Victor Stinner wrote:

Hi,

I started to implement an AST optimizer in Python. It's easy to create
a new AST tree, so I'm surprised that I didn't find any existing
project.

https://bitbucket.org/haypo/misc/src/tip/python/ast_optimizer.py

To test its peephole optimizations (by checking manually its final
bytecode), I wrote a patch for Python to disable Python internal
peephole optimizer (on bytecode):
https://bitbucket.org/haypo/misc/src/tip/python/compile_disable_peephole.patch


I would very much like to see the ability to disable all optimizers.  As 
work continues on the various forms of optimization, please remember 
that sometimes programs are executed to reason about them, for example, 
when measuring test coverage, or when debugging. Just as gcc has a -O0 
option to disable all code optimizations so that you can more easily 
debug programs, it would be fabulous for CPython to have an option to 
disable all code optimizations.  This would include the existing 
peephole optimizer as well as any new optimizers such as Victor's AST 
optimizer.


Full disclosure: I previously raised this possibility in ticket, and it 
was not as popular as I had hoped: http://bugs.python.org/issue2506


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Playing with a new theme for the docs, iteration 2

2012-03-25 Thread Ned Batchelder


On 3/25/2012 2:34 AM, Georg Brandl wrote:

Here's another try, mainly with default browser font size, more contrast and
collapsible sidebar again:

http://www.python.org/~gbrandl/build/html2/
Georg, thanks so much for taking on this thankless task with grace and 
skill.  It can't be easy dealing with the death by a thousand tweaks, 
and I know I've contributed to the flurry.


Nowhere on the page is a simple link to the front page of python.org.  
Perhaps the traditional upper-left corner could get a bread-crumb before 
Python v3.3a1 documentation that simply links to python.org.  Maybe, 
use the word Python that is already there:  [Python] » [v3.3a1 
documentation].  People do arrive at doc pages via search engines, and 
connecting the docs up to the rest of the site would be a good thing.


Speaking of links to other pages, the doc front page, under Other 
resources lists Guido's Essays and New-style Classes second and third.  
These each point to extremely outdated material (Unifying types and 
classes in 2.2, and Unfortunately, new-style classes have not yet been 
integrated into Python's standard documention. ??).  Another, Other 
Doc Collections, points to an empty apache-style directory listing  
:-(.  These links should be removed if we don't want to keep those 
sections of the site up-to-date.  I know this is not strictly part of 
the redesign, but I just noticed it and thought I would throw it out there.


I agree about the outlined style for New notices, and the red for 
deprecation is extremely alarming! :)


I'll make one last plea for not justifying short paragraphs full of 
unbreakable elements, but I know I am in the minority.



I've also added a little questionable gimmick to the sidebar (when you collapse
it and expand it again, the content is shown at your current scroll location).
I especially like using dynamic elements on a page to adapt to a 
reader's needs. I have some other ideas that I'll try to cobble together.


--Ned.

Have fun!
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Playing with a new theme for the docs

2012-03-21 Thread Ned Batchelder


On 3/21/2012 6:16 AM, Oleg Broytman wrote:

On Wed, Mar 21, 2012 at 09:33:13AM +, Jonathan Hartley wrote:

On 21/03/2012 08:25, Dirkjan Ochtman wrote:

On Wed, Mar 21, 2012 at 07:00, Georg Brandlg.bra...@gmx.net   wrote:

OK, that seems to be the main point people make... let me see if I can
come up with a better compromise.

Would it be possible to limit the width of the page? On my 1920px
monitor, the lines get awfully long, making them harder to read.

I realise this is bikeshedding by now, but FWIW, please don't. If
people want shorter lines, they can narrow their browser, without
forcing that preference on the rest of us.

Seconded. My display is 1920x1200 but I use very large fonts and I'm
satisfied with line lengths.
The best thing to do is to set a max-width in ems, say 50em. This leaves 
the text at a reasonable width, but adapts naturally for people with 
larger fonts.


--Ned.


Oleg.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Playing with a new theme for the docs

2012-03-21 Thread Ned Batchelder


On 3/21/2012 9:44 AM, Serhiy Storchaka wrote:

21.03.12 03:58, Ned Batchelder написав(ла):

Books, magazines, and newspapers look good with full justification, web
pages do not. Can we switch to left-justified instead?


You can add line

p {text-align: left !important}

to your browser custom stylesheet.

If you are using Firefox or Chrome (Chromium), for them there are 
extensions (Stylish) that allow to apply the style to the particular 
site.


Any of the tweaks people are suggesting could be applied individually 
using this technique.  We could just as easily choose to make the site 
left-justified, and let the full-justification fans use custom 
stylesheets to get it.


The challenge for the maintainer of the docs site is to choose a good 
design that most people will see.  We're bound to disagree on what that 
design should be, and I suggest that probably none of us are designer 
enough to come up with the best one.  Perhaps we could find an 
interested designer to help?


--Ned.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Playing with a new theme for the docs

2012-03-21 Thread Ned Batchelder


On 3/21/2012 1:04 PM, Serhiy Storchaka wrote:
If I can get my five cents, I will tell about my impressions. I really 
liked the background of allocated blocks (such as notes and code 
snippets) has become less diverse (but still visible). The border 
around these blocks have become more accurate and more pleasant to 
emphasize blocks. It is very good that the sidebar is no longer 
confused look. And everything looks quite nice. But the font is a 
little bit small for my eyes (on the contrary current theme font a 
little bit big). This leads to too long (in characters) lines. Less 
obvious was the structure of the document (due to decrease the font 
size of the header and the removal of the dividing lines).


You can use Ctrl-+ to increase the size of the text, and modern browsers 
remember that for the next time you visit the site.


--Ned.

I would like to that the background color of .note tt has become a 
little lighter and quieter.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Playing with a new theme for the docs

2012-03-21 Thread Ned Batchelder


On 3/21/2012 2:46 PM, Guido van Rossum wrote:

On Wed, Mar 21, 2012 at 11:40 AM, Ned Batcheldern...@nedbatchelder.com  wrote:

You can use Ctrl-+ to increase the size of the text, and modern browsers
remember that for the next time you visit the site.

That doesn't mean the web designer shouldn't think at least twice
before specifying a smaller font than the browser default.

Yes, sorry, that was exactly my point earlier in this thread.  I was 
being a bit snarky with Serhiy.  Seems the standard here is for people 
to request their personal favorite tweaks, and then tell others that 
they can use browser customizations to get what they want.


Guido, you encouraged us to use science, but only after describing my 
science-based maximum line-length suggestion as coddling, then said we 
should let Georg get on with it, but only after reiterating your 
personal favorite tweak (which I happen to agree with).


There's no way a committee (which this thread effectively is) will come 
up with a good design.  Everyone will dislike something about it.  I 
think it would be interesting to use the power of the web to provide 
docs whose style could be adjusted a few ways to make people happy, but 
that is probably more than anyone is willing to volunteer for, I know I 
can't step up to do it.


Personally, I think two Python projects that have focused on docs and 
done a good job of it are Django and readthedocs.org.  Perhaps we could 
follow their lead?


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Playing with a new theme for the docs

2012-03-21 Thread Ned Batchelder


On 3/21/2012 3:06 PM, Fred Drake wrote:

On Wed, Mar 21, 2012 at 2:46 PM, Guido van Rossumgu...@python.org  wrote:

That doesn't mean the web designer shouldn't think at least twice
before specifying a smaller font than the browser default.

Yet 90% of designers (or more) insist on making text insanely small, commonly
specifying the size in pixles or (if we're lucky) points.

Not sure there's any lesson to be learned from this, aside from designers
really having it out for anyone who needs to read.
There are bad designers, or more to the point, designers who favor the 
overall look of the page at the expense of the utility of the page.  
That doesn't mean all designers are bad, or that design is bad.  Don't 
throw out the baby with the bathwater.


--Ned.


   -Fred


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Playing with a new theme for the docs

2012-03-21 Thread Ned Batchelder


On 3/21/2012 3:45 PM, Ethan Furman wrote:

Guido van Rossum wrote:
On Wed, Mar 21, 2012 at 7:18 AM, Ned Batchelder 
n...@nedbatchelder.com wrote:
The challenge for the maintainer of the docs site is to choose a 
good design

that most people will see.  We're bound to disagree on what that design
should be, and I suggest that probably none of us are designer 
enough to
come up with the best one.  Perhaps we could find an interested 
designer to

help?


I've come to the conclusion that good design is not so much a matter
of finding the best of anything (font, spacing rules, colors, icons,
artowork, etc.). Good design is highly subjective to fashion, and the
people who are recognized to be the best designers are more often than
not just those with a strong enough opinion to push their creative
ideas through. Then other designers, who are not quite as good but
still have a nose for the latest fashion, copy their ideas and for a
while anything that hasn't been redesigned looks old-fashioned.

(Before you say something about limitations of old technology, note
how often designers go back to older styles and manage to make them
look fashionable again.)

If you want something that attracts attention through controversy, get
one of those initial thought leaders. If you want something that looks
current today but which will probably be out of style next year, use
one of the style-following designers. If you want something that is
maximally useful, get a scientist with an ounce of style sense to do
your design... Oh hey, Georg *is* a scientist! And he's got more than
an ounce of style. So just let him do it and let's not try to
micromanage things. (I had to speak up about the low contrast because
Georg has young eyes and may not realize that this issue exists for
older Pythonistas.)


+1000

Deriding the entire discipline of design because some of its 
practitioners are hacks is like pointing at PHP kiddies as the reason 
why you don't need software architects.  Yes, we could make the 
mistake of over-designing it, and that would be a mistake.  The science 
you seek is something that designers are well-versed in.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Playing with a new theme for the docs

2012-03-21 Thread Ned Batchelder


On 3/21/2012 4:38 PM, Fred Drake wrote:

On Wed, Mar 21, 2012 at 3:13 PM, Ned Batcheldern...@nedbatchelder.com  wrote:

There are bad designers, or more to the point, designers who favor the
overall look of the page at the expense of the utility of the page.  That
doesn't mean all designers are bad, or that design is bad.  Don't throw
out the baby with the bathwater.

I get that.  I'm not bad-mouthing actual design, and there are definitely
good designers out there.

It's unfortunate they're so seriously outnumbered.

Yeah, just like software architects...  :-(

--Ned.


   -Fred


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Playing with a new theme for the docs

2012-03-20 Thread Ned Batchelder


On 3/20/2012 6:38 PM, Georg Brandl wrote:

Let me know what you think, or play around and send some improvements.
(The collapsible sidebar is not adapted to it yet, but will definitely
be integrated before I consider applying a new theme to the real docs.)
Not to add to the chorus of tweakers, but if I could change just one 
thing about the current theme, it would be to remove full justification 
of the text.  In text like ours with frequent long expressions, URLs, 
and the like, full justification is just an invitation to mangle the 
spacing of a paragraph.  The paragraphs are also quite short and often 
interrupted by samples, lists, headings, and so on, losing the design 
advantage of a clean right edge anyway.


Books, magazines, and newspapers look good with full justification, web 
pages do not.  Can we switch to left-justified instead?


--Ned.

Thanks,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RELEASED] Python 3.3.0 alpha 1

2012-03-05 Thread Ned Batchelder


On 3/5/2012 2:54 AM, Georg Brandl wrote:

On behalf of the Python development team, I'm happy to announce the
first alpha release of Python 3.3.0.

This is a preview release, and its use is not recommended in
production settings.

Python 3.3 includes a range of improvements of the 3.x series, as well 
as easier
porting between 2.x and 3.x.  Major new features in the 3.3 release 
series are:


* PEP 380, Syntax for Delegating to a Subgenerator (yield from)
* PEP 393, Flexible String Representation (doing away with the
  distinction between wide and narrow Unicode builds)
* PEP 409, Suppressing Exception Context
* PEP 3151, Reworking the OS and IO exception hierarchy
* The new packaging module, building upon the distribute and
  distutils2 projects and deprecating distutils
* The new lzma module with LZMA/XZ support
* PEP 3155, Qualified name for classes and functions
* PEP 414, explicit Unicode literals to help with porting
* The new faulthandler module that helps diagnosing crashes
* Wrappers for many more POSIX functions in the os and signal
  modules, as well as other useful functions such as sendfile()

For a more extensive list of changes in 3.3.0, see

http://docs.python.org/3.3/whatsnew/3.3.html

The 3.3 whatsnews page doesn't seem to mention PEP 414 or Unicode 
literals at all.


--Ned.

To download Python 3.3.0 visit:

http://www.python.org/download/releases/3.3.0/

Please consider trying Python 3.3.0a1 with your code and reporting any
bugs you may notice to:

http://bugs.python.org/


Enjoy!

--
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.3's contributors)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 414

2012-02-26 Thread Ned Batchelder


On 2/26/2012 6:14 AM, Nick Coghlan wrote:

As soon as you allow the use of from __future__ import
unicode_literals or a module level __metaclass__ = type, you can't
review diffs in isolation any more - whether the diff is correct or
not will depend on the presence or absence of module level tweak to
the language semantics.

Future imports work well for things like absolute imports, new
keywords, or statements becoming functions - if the future import is
missing when you expected it to be present (or vice-versa) will result
in a quick SyntaxError or ImportError that will point you directly to
the offending code. Unicode literals and implicitly creating new-style
classes are a different matter - for those, if the module level
modification takes place (or doesn't take place when you expected it
to be there), you get unexpected changes in behaviour instead of a
clear exception that refers directly to the source of the problem.
There are already __future__ imports that violate this principle:  from 
__future__ import division.  That doesn't mean I'm in favor of this new 
__future__, just keeping a wide angle on the viewfinder.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status regarding Old vs. Advanced String Formating

2012-02-24 Thread Ned Batchelder


On 2/24/2012 7:23 PM, Mark Lawrence wrote:
I think this is daft because all of the code has to be supported for 
the ten years that MVL has suggested.  I suggest a plan that says 
something like:-


Until Python 3.5 both methods of string formatting will be supported.
In Python 3.6 the the old formating style will be deprecated.
In Python 3.7 the old style is dead.

I'm fully aware that it isn't likely to be that easy, I'm simply 
trying to spark ideas from the core developers and users who are in a 
far better situation to judge this situation than I am.
I don't understand why we'd even consider getting rid of old-style 
formatting.  Python does a great job keeping things working into the 
future, and there are so many features in the language and library that 
are there to keep old code working in spite of newer ways to accomplish 
the same task.  Has Python *ever* removed a feature except in X.0 
releases?  Why are we even discussing this?


Two ways to format strings is no big deal, especially considering how 
heavily used these tools are.  And btw, if you include the 
almost-never-mentioned string.Template, there are at least three ways to 
do it, maybe more.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 clarification request: braces

2012-01-02 Thread Ned Batchelder


On 1/1/2012 11:44 PM, Nick Coghlan wrote:

I've been having an occasional argument with Benjamin regarding braces
in 4-line if statements:

   if (cond)
 statement;
   else
 statement;

vs.

   if (cond) {
 statement;
   } else {
 statement;
   }

He keeps leaving them out, I occasionally tell him they should always
be included (most recently this came up when we gave conflicting
advice to a patch contributor). He says what he's doing is OK, because
he doesn't consider the example in PEP 7 as explicitly disallowing it,
I think it's a recipe for future maintenance hassles when someone adds
a second statement to one of the clauses but doesn't add the braces.
(The only time I consider it reasonable to leave out the braces is for
one liner if statements, where there's no else clause at all)

Since Benjamin doesn't accept the current brace example in PEP 7 as
normative for the case above, I'm bringing it up here to seek
clarification.
I've always valued readability and consistency above brevity, and Python 
does too.  *Sometimes* using braces in C is a recipe for confusion, and 
only adds to the cognitive load in reading the code.  The examples 
elsewhere in this thread of mistakes and noisy diffs due to leaving out 
the braces are plenty of reason for me to always include braces.


The current code uses a mixture of styles, but that doesn't mean we need 
to allow any style in the future.  I'm in favor of PEP 7 being amended 
to either require or strongly favor the braces-always style.  Note: 
while we're reading the tea-leaves in PEP 7, it has an example of a 
single-line if clause with no braces.


Some people favor the braces-sometimes style because it leads to 
lighter code.  I think that's a misguided optimization.  Consistency 
is better than reducing the line count.


--Ned.

Cheers,
Nick.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 clarification request: braces

2012-01-02 Thread Ned Batchelder


On 1/2/2012 5:32 PM, Raymond Hettinger wrote:


Running  ``grep -B1 else Objects/*c`` shows that we've happily lived 
with a mixture of styles for a very long time.
ISTM, our committers have had good instincts about when braces add 
clarity and when they add clutter.
If Nick pushes through an always-use-braces mandate, A LOT of code 
will need to be changed.


I'm sure we can agree that 1) Nick isn't pushing through anything, 
this is a discussion about what to do, and 2) even if we agree to change 
PEP 7, no one would advocate having to go through all the C code to 
change it to a newly-agreed style.


--Ned.


Raymond





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hash collision security issue (now public)

2011-12-29 Thread Ned Batchelder


On 12/28/2011 9:09 PM, Raymond Hettinger wrote:

Also, randomizing the hash wreaks havoc on doctests, book examples
not matching actual dict reprs, and on efforts by users to optimize
the insertion order into dicts with frequent lookups.
I don't have a strong opinion about what to do about this vulnerability, 
but I know that none of these three reasons are a good reason to not 
change anything.  Dictionary key order has never been guaranteed, and 
changes from time to time.  Any code relying on it is broken to begin 
with. This is one of the reasons not to use doctests in the first place: 
comparing dicts textually has always been silly.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3 optimizations continued...

2011-09-01 Thread Ned Batchelder


On 8/30/2011 4:41 PM, stefan brunthaler wrote:

Ok, there there's something else you haven't told us. Are you saying
that the original (old) bytecode is still used (and hence written to
and read from .pyc files)?


Short answer: yes.
Long answer: I added an invocation counter to the code object and keep
interpreting in the usual Python interpreter until this counter
reaches a configurable threshold. When it reaches this threshold, I
create the new instruction format and interpret with this optimized
representation. All the macros look exactly the same in the source
code, they are just redefined to use the different instruction format.
I am at no point serializing this representation or the runtime
information gathered by me, as any subsequent invocation might have
different characteristics.
When the switchover to the new instruction format happens, what happens 
to sys.settrace() tracing?  Will it report the same sequence of line 
numbers?  For a small but important class of program executions, this is 
more important than speed.


--Ned.


Best,
--stefan

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Commit messages: please avoid temporal ambiguity

2011-05-09 Thread Ned Batchelder


On 5/9/2011 1:24 PM, Terry Reedy wrote:
A commit (push) partition time and behavior into before and after 
(with a short change period in between during which behavior is 
undefined).


Some commit messages have the form 'x does y'. Does 'does' mean before 
or after? Sometimes that is clear. 'x crashes' means before. 'x return 
correct value' means after. But some messages of this type are unclear 
to me as written.


Consider 'x raises exception'? The temporal reference is obvious to 
the committer but not necessary to everyone else. It could mean 'x 
used to segfault and now raises a catchable exception'. There was a 
fix like this (with a clear message) just today. It could also mean 'x 
used to raise but now return an answer. There have been many fixes 
like this.


Two minimal fixes are 'x raised exception' or 'make x raise exception'.


I've always favored X now properly raises an exception.

--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] sys.settrace: behavior doesn't match docs

2011-05-02 Thread Ned Batchelder


Indeed, the 2.0 code is very different, and got this case right.

I'm a little surprised no one is arguing that changing this code now 
could break some applications.  Maybe the fact no one noticed the docs 
were wrong proves that no one ever tried returning None from a local 
trace function.


--Ned.

On 4/30/2011 8:43 PM, Guido van Rossum wrote:

I think you need to go back farther in time. :-) In Python 2.0 the
call_trace function in ceval.c has a completely different signature
(but the docs are the same). I haven't checked all history but
somewhere between 2.0 and 2.3, SET_LINENO-less tracing was added, and
that's where the implementation must have gone wrong. So I think we
should fix the code.

--Guido

On Sat, Apr 30, 2011 at 3:49 PM, Ned Batcheldern...@nedbatchelder.com  wrote:

This week I learned something new about trace functions (how to write a C
trace function that survives a sys.settrace(sys.gettrace()) round-trip), and
while writing up what I learned, I was surprised to discover that trace
functions don't behave the way I thought, or the way the docs say they
behave.

The docs say:

The trace function is invoked (with event set to 'call') whenever a new
local scope is entered; it should return a reference to a local trace
function to be used that scope, or None if the scope shouldn’t be traced.

The local trace function should return a reference to itself (or to another
function for further tracing in that scope), or None to turn off tracing in
that scope.

It's that last part that's wrong: returning None from the trace function
only has an effect on the first call in a new frame.  Once the trace
function returns a function for a frame, returning None from subsequent
calls is ignored.  A local trace function can't turn off tracing in its
scope.

To demonstrate:

import sys

UPTO_LINE = 1

def t(frame, event, arg):
 num = frame.f_lineno
 print(line %d % num)
 if num  UPTO_LINE:
 return t

def try_it():
 print(twelve)
 print(thirteen)
 print(fourteen)
 print(fifteen)

UPTO_LINE = 1
sys.settrace(t)
try_it()

UPTO_LINE = 13
sys.settrace(t)
try_it()

Produces:

line 11
twelve
thirteen
fourteen
fifteen
line 11
line 12
twelve
line 13
thirteen
line 14
fourteen
line 15
fifteen
line 15

The first call to try_it() returns None immediately, preventing tracing for
the rest of the function.  The second call returns None at line 13, but the
rest of the function is traced anyway.  This behavior is the same in all
versions from 2.3 to 3.2, in fact, the 100 lines of code in sysmodule.c
responsible for Python tracing functions are completely unchanged through
those versions.  (A deeper mystery that I haven't looked into yet is why
Python 3.x intersperses all of these lines with line 18 interjections.)

I'm writing this email because I'm not sure whether this is a behavior bug
or a doc bug.  One of them is wrong, since they disagree.  The documented
behavior makes sense, and is what people have all along thought the trace
function did.  The actual behavior is a bit more complicated to explain, but
is what people have actually been experiencing.  FWIW, PyPy implements the
documented behavior.

Should we fix the code or the docs?  I'd be glad to supply a patch for
either.

--Ned.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/guido%40python.org






___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] sys.settrace: behavior doesn't match docs

2011-04-30 Thread Ned Batchelder

This week I learned something new about trace functions (how to write a 
C trace function that survives a sys.settrace(sys.gettrace()) 
round-trip), and while writing up what I learned, I was surprised to 
discover that trace functions don't behave the way I thought, or the way 
the docs say they behave.


The docs say:

   The trace function is invoked (with /event/ set to 'call') whenever
   a new local scope is entered; it should return a reference to a
   local trace function to be used that scope, or None if the scope
   shouldn't be traced.

   The local trace function should return a reference to itself (or to
   another function for further tracing in that scope), or None to turn
   off tracing in that scope.

It's that last part that's wrong: returning None from the trace function 
only has an effect on the first call in a new frame.  Once the trace 
function returns a function for a frame, returning None from subsequent 
calls is ignored.  A local trace function can't turn off tracing in 
its scope.


To demonstrate:

   import sys

   UPTO_LINE = 1

   def t(frame, event, arg):
num = frame.f_lineno
print(line %d % num)
if num  UPTO_LINE:
return t

   def try_it():
print(twelve)
print(thirteen)
print(fourteen)
print(fifteen)

   UPTO_LINE = 1
   sys.settrace(t)
   try_it()

   UPTO_LINE = 13
   sys.settrace(t)
   try_it()

Produces:

   line 11
   twelve
   thirteen
   fourteen
   fifteen
   line 11
   line 12
   twelve
   line 13
   thirteen
   line 14
   fourteen
   line 15
   fifteen
   line 15

The first call to try_it() returns None immediately, preventing tracing 
for the rest of the function.  The second call returns None at line 13, 
but the rest of the function is traced anyway.  This behavior is the 
same in all versions from 2.3 to 3.2, in fact, the 100 lines of code in 
sysmodule.c responsible for Python tracing functions are completely 
unchanged through those versions.  (A deeper mystery that I haven't 
looked into yet is why Python 3.x intersperses all of these lines with 
line 18 interjections.)


I'm writing this email because I'm not sure whether this is a behavior 
bug or a doc bug.  One of them is wrong, since they disagree.  The 
documented behavior makes sense, and is what people have all along 
thought the trace function did.  The actual behavior is a bit more 
complicated to explain, but is what people have actually been 
experiencing.  FWIW, PyPy implements the documented behavior.


Should we fix the code or the docs?  I'd be glad to supply a patch for 
either.


--Ned.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

2011-03-14 Thread Ned Batchelder


On 3/12/2011 12:39 AM, Eugene Toder wrote:

  You've got wishful thinking if you think a handful of tests can catch
  errors in code that sophisticated.

Why limit yourself with a handful of tests? Python is widespread,
there's*a lot*  of code in Python. Unlike with libraries, any code you
run tests the optimizer, so just run a lot of code. And, as I've said,
write a test generator.


As we're thinking about what the optimizer of the future should be, it 
would be great to have a way to turn it off completely.  This would 
allow the tests to run test code both with and without the optimizer, 
and to verify that the two results were the same.  Then we could 
automatically verify that the optimizer isn't changing semantics.


BTW: I also believe it would be very useful to make the 
turn-off-the-optimizer switch available for users so they can run their 
code unoptimized for times when they are more interested in program 
analysis than in execution speed.  See http://bugs.python.org/issue2506 
(closed 3 years ago) that I filed with this request.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-02 Thread Ned Batchelder


On 1/2/2011 8:17 AM, Victor Stinner wrote:

Le mercredi 29 décembre 2010 à 14:20 +0100, Martin v. Löwis a écrit :

Am 28.12.2010 18:08, schrieb Lukas Lueg:

Also, the
load_fast in lne 22 to reference x could be taken out of the loop as x
will always point to the same object

That's not true; a debugger may change the value of x.

That's why Python has the following option:

-O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x

I regulary recompile programs with gcc -O0 -g to debug them. It is very
difficult to debug (with gdb) a program compiled with gcc -O2: many
variables are stored in registers, and gdb doesn't support that
correctly.

Victor, you seem to be equating the gcc -O flag with the Python -O 
flag.  They are described similarly, but can't be used the same way.  In 
particular, there is no Python equivalent to gcc's -O0: there is no way 
to disable the Python peephole optimizer.


--Ned.

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-01 Thread Ned Batchelder


On 12/31/2010 12:51 PM, Cesare Di Mauro wrote:

2010/12/31 s...@pobox.com mailto:s...@pobox.com


 Another example. I can totally remove the variable i, just
using the
 stack, so a debugger (or, in general, having the tracing enabled)
 cannot even find something to change about it.

   Ethan -1

   Ethan Debugging is challenging enough as it is -- why would
you want to
   Ethan make it even more difficult?

snarky
I don't know.  Maybe he wants his program to run faster.
/snarky


:D

Aggressive optimizations can be enabled with explicit options, in 
order to leave normal debugger-prone code.
I wish the Python compiler would adopt a strategy of being able to 
disable optimizations.  I wrote a bug about a leaky abstraction 
optimization messing up coverage testing 2.5 years ago, and it was 
closed as won't fix: http://bugs.python.org/issue2506.  The debate there 
centered around, but that line isn't executed, because it's been 
optimized away.  It's common in sophisticated compilers (as in, any C 
compiler) to be able to choose whether you want optimizations for speed, 
or disabling optimizations for debugging and reasoning about the code.  
Python would benefit from the same choice.


--Ned.


If you use print statements for the bulk of your debugging (many
people do),
unrolling loops doesn't affect your debugging ability.

 Skip


It's a common practice. Also IDEs helps a lot, and advanced 
interactive shells too (such as DreamPie).


Cesare


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] os.path.normcase rationale?

2010-09-24 Thread Ned Batchelder


 On 9/24/2010 6:13 AM, Chris Withers wrote:

On 18/09/2010 23:36, Guido van Rossum wrote:

course, exists() and isdir() etc. do, and so does realpath(), but the
pure parsing functions don't.


Yes, but:

H:\echo foo  TeSt.txt
... import os.path
 os.path.realpath('test.txt')
'H:\\test.txt'
 os.path.normcase('TeSt.txt')
'test.txt'

Both feel unsatisfying to me :-S

How can I get 'TeSt.txt' from 'test.txt' (which feels like the 
contract normcase *should* have...)



http://stackoverflow.com/questions/3692261/in-python-how-can-i-get-the-correctly-cased-path-for-a-file

They can be used without a working
filesystem even. (E.g. you can import ntpath on a Unix box and happily
parse Windows paths.)


But what value does that add over just doing a .lower() on the path?

Chris
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

73 matches

Mail list logo