Re: [Python-ideas] (no subject)

[Antoine Pitrou]

On Tue, 5 Dec 2017 02:52:44 +1100
Steven D'Aprano  wrote:
> On Mon, Dec 04, 2017 at 01:52:19PM +0100, Antoine Pitrou wrote:
> > On Mon, 4 Dec 2017 23:16:11 +1100
> > Steven D'Aprano  wrote:  
> > > On Mon, Dec 04, 2017 at 12:06:38PM +0100, Antoine Pitrou wrote:
> > >   
> > > > There are definitely advantages.  Sorting calls __lt__ for each
> > > > comparison (that is, O(n log n) times) while __key__ would only be
> > > > called once per item at the start (that is, O(n) times).
> > > 
> > > Passing a key function doesn't magically turn a O(n log n) comparison 
> > > sort into a O(n) sort.  
> > 
> > Where did I say it did?  
> 
> See the text from you quoted above. You said there are "definitely 
> [performance] advantages" by using a key function. You then compare:
> 
> - calling __lt__ O(n log n) times, versus
> 
> - calling the key function O(n) times.
> 
> This is a classic "apples versus oranges" comparison. You compare 
> *actually sorting the list* with *not sorting the list* and conclude 
> that they key function provides a performance advantage.

At this point, I can only assume you are trolling by twisting my
words... even though you later quote the part which explicitly
clarifies that I was *not* saying what you claim I did.

Why you seem to think that is contributing anything to the discussion
rather than derailing it is beyond me.  In any case, don't expect
further responses from me.

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] (no subject)

[Gerald Britton]

>* On 25 Mar 2017, at 15:51, Gerald Britton > wrote:
> *> >* On 25 March 2017 at 11:24, Pavel Velikhov   >> wrote:
> *>* > No, the current solution is temporary because we just don’t have the
> *>* > manpower to
> *>* > implement the full thing: a real system that will rewrite parts of 
> PythonQL
> *>* > queries and
> *>* > ship them to underlying databases. We need a real query optimizer and 
> smart
> *>* > wrappers
> *>* > for this purpose. But we’ll build one of these for demo purposes soon
> *>* > (either a Spark
> *>* > wrapper or a PostgreSQL wrapper).
> *>* One thought, if you're lacking in manpower now, then proposing
> *>* inclusion into core Python means that the core dev team will be taking
> *>* on an additional chunk of code that is already under-resourced. That
> *>* rings alarm bells for me - how would you imagine the work needed to
> *>* merge PythonQL into the core Python grammar would be resourced?
> *>* I should say that in practice, I think that the solution is relatively
> *>* niche, and overlaps quite significantly with existing Python features,
> *>* so I don't really see a compelling case for inclusion. The parallel
> *>* with C# and LINQ is interesting here - LINQ is a pretty cool
> *>* technology, but I don't see it in widespread use in general-purpose C#
> *>* projects (disclaimer: I don't get to see much C# code, so my
> *>* experience is limited).
> *> >* I see lots of C# code, but (thankfully) not so much LINQ to SQL.  Yes, 
> it is a cool technology.  But I sometimes have a problem with the SQL it 
> generates.  Since I'm also a SQL developer, I'm sensitive to how queries are 
> constructed, for performance reasons, as well as how they look, for 
> readability and aesthetic reasons.
> *> >* LINQ queries can generate poorly-performing SQL, since LINQ is a 
> basically a translator, but not an AI.  As far as appearances go, LINQ 
> queries can look pretty gnarly, especially if they include sub queries or a 
> few joins.  That makes it hard for the SQL dev (me!) to read and understand 
> if there are performance problems (which there often are, in my experience)
> *>

We want to go beyond being a basic translator. Especially if the
common use-case will be integrating multiple databases. We can also
introduce decent-looking hints (maybe not always decent looking) to
generate better plans. Not sure about asethetics though...

>* So, I would tend to code the SQL separately and put it in a SQL view, 
>function or stored procedure.  I can still parse the results with LINQ (not 
>LINQ to SQL), which is fine.
> *> >* For similar reasons, I'm not a huge fan of ORMs either.  Probably my 
> bias towards designing the database first and building up queries to meet the 
> business goals before writing a line of Python, C#, or the language de jour.
> *

This sounds completely reasonable, but this means you’re tied to a
specific DBMS (especially if you’re using a lot of built-in functions
that are usually very specific to a database). PythonQL (when it has
enough functionality) should give you independence.


True though not always needed.  e.g. at present I'm working for a large
company with thousands of db servers in all the popular flavors.  The
probability of changing even one of them to a different vendor is
essentially zero.  The costs and dependencies far outweigh any hoped-for
advantage.  At the same time, I'm happy to optimize the SQL for different
target environments.  If I lack the specific expertise, I know where to go
to find it.  The Adapter pattern helps here.

It's actually more important for me to build queries that can be used in
multiple client languages. We're using Java, C++, C#, F#, VB, ...  and
Python, of course (and probably others that I don't know we use). I can
optimize the query once and not worry about the clients messing it up.

-- 
Gerald Britton, MCSE-DP, MVP
LinkedIn Profile: http://ca.linkedin.com/in/geraldbritton
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Chris Angelico]

On Mon, Dec 26, 2016 at 8:51 AM, Victor Stinner
 wrote:
> Le 24 déc. 2016 8:42 PM, "Neil Girdhar"  a écrit :
>> Usually, when an exception is hit that will (probably) crash the program,
>> no one cares about less than a microsecond of performance.
>
> Just one example. By design, hasattr(obj, name) raises an exception to
> return False.
>
> So it has the cost of building the exception + raise exc + catch it.

Printing an exception to the console can afford to be expensive,
though. So if the work can be pushed into __str__, it won't hurt
anything that try/excepts around it.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Victor Stinner]

Le 24 déc. 2016 8:42 PM, "Neil Girdhar"  a écrit :
> Usually, when an exception is hit that will (probably) crash the program,
no one cares about less than a microsecond of performance.

Just one example. By design, hasattr(obj, name) raises an exception to
return False.

So it has the cost of building the exception + raise exc + catch it.

Victor
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Thomas Nyberg]

On 12/24/2016 11:42 AM, Neil Girdhar wrote:

Usually, when an exception is hit that will (probably) crash the
program, no one cares about less than a microsecond of performance.


I would probably agree with you in the SyntaxError example, but not for 
the others. Programming with exceptions is totally standard in Python 
and they are often used in tight loops. See here:


https://docs.python.org/3/glossary.html#term-eafp
https://docs.python.org/3/glossary.html#term-lbyl

So keeping exceptions fast is definitely important.

Cheers,
Thomas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Neil Girdhar]

On Tuesday, November 29, 2016 at 4:08:19 AM UTC-5, Victor Stinner wrote:
>
> Hi, 
>
> Python is optimized for performance. Formatting an error message has a 
> cost on performances. 
>
>
Usually, when an exception is hit that will (probably) crash the program, 
no one cares about less than a microsecond of performance.
 

> I suggest you to teach your student to use the REPL and use a custom 
> exception handler: sys.excepthook: 
> https://docs.python.org/2/library/sys.html#sys.excepthook 
>
> Using a custom exception handler, you can run expensive functions, 
> like the feature: "suggest len when length is used". 
>
> The problem is then when students have to use a Python without the 
> custom exception handler. 
>
> Victor 
> ___ 
> Python-ideas mailing list 
> python...@python.org  
> https://mail.python.org/mailman/listinfo/python-ideas 
> Code of Conduct: http://python.org/psf/codeofconduct/ 
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Rob Cliffe]

On 29/11/2016 20:09, Terry Reedy wrote:

On 11/29/2016 11:32 AM, Rob Cliffe wrote:



On 29/11/2016 04:58, victor rajewski wrote:


Traceback (most recent call last):

 File "foo.py", line 2, in 

   l[10]=14

IndexError: list assignment index out of range


A better message might be:

You tried to use l[10] when l is only 4 elements long. You can add
items to l using l.append(value), or check your index value to make
sure that's really the position you wanted to access.




It would make sense to me to upgrade this particular error message to
IndexError: list assignment index 10 out of range 0 to 3 if it can
be done without too much difficulty or overhead.  (An empty list, and
perhaps one with only 1 element, would be special cases.)  Come to think
of it, is the word "assignment" needed?


It would help if the line were "l1[10] = 2 * l2[13] + 3".


You're right of course; "assignment" IS meaningful; I missed it.

>>> [][0]
Traceback (most recent call last):
  File "", line 1, in 
IndexError: list index out of range

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Stephen J. Turnbull]

Victor Stinner writes:

 > Using a custom exception handler, you can run expensive functions,
 > like the feature: "suggest len when length is used".

LGTM.

 > The problem is then when students have to use a Python without the
 > custom exception handler.

Put the exception handler in an importable module in the stdlib.

Usual caveats about high bar, etc, but this should solve the "have
Python, no handler" issue going forward.

Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[MRAB]

On 2016-11-29 19:45, Brendan Barnwell wrote:

On 2016-11-29 09:43, Brett Cannon wrote:

One way to make this cheap is to have a reasonable default message and
use attributes on the exceptions trigger the use of the default message.
Nearly a year ago I filed a bunch of issues for ideas on providing
attributes on exceptions where it made sense, e.g. an index attribute on
IndexError (http://bugs.python.org/issue18162). If we did this then for
classes like IndexError there constructor could be `IndexError(index=10,
start=0, end=3)` and then __str__() can lazily construct the string
representation using a default message, e.g. `"index {} is out of range
of{} to {}".format(index, start, end)`. Make the arguments keyword-only
and they become backwards-compatible and so the only overhead you pay
for these richer messages are keyword-based construction if you simply
never access the repr for the exception.


I absolutely think this is the way to go.  Having the relevant
information (the list that was too short, the index that was too big,
the key that wasn't there, etc.) is useful in many situations, and it's
much better to have that information in a programmatic form than just
squashed into an error message.  This then makes it relatively easy to
write wrappers that take bubbling-up exceptions and try to construct
more detailed messages for a less experienced audience.  Right now this
is difficult or impossible because the exception objects don't record
the information that would be needed for these expanded messages.

Couldn't that result in objects being held for longer, taking up memory, 
not being collected as promptly, and not releasing resources as quickly?


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Nathaniel Smith]

On Nov 29, 2016 9:43 AM, "Brett Cannon"  wrote:
>
>
>
> On Tue, 29 Nov 2016 at 02:39 Nathaniel Smith  wrote:
>>
>> On Tue, Nov 29, 2016 at 1:05 AM, Victor Stinner
>>  wrote:
>> > Hi,
>> >
>> > Python is optimized for performance. Formatting an error message has a
>> > cost on performances.
>>
>> Sure, but we have to look at this on a case-by-case basis. Is there
>> really important code out there that's generating NameErrors or
>> SyntaxErrors in an inner loop? That seems unlikely to me.
>>
>> Even IndexError I'm a bit skeptical about. I can believe that there's
>> code that intentionally generates and then catches IndexError, but
>> AttributeError in my experience is much more performance-sensitive
>> than IndexError, because every failed hasattr call allocates an
>> AttributeError and hasattr is commonly used for feature checks. Yet
>> AttributeError has a much more informative (= expensive) message than
>> IndexError:
>>
>> In [1]: object().a
>> AttributeError: 'object' object has no attribute 'a'
>>
>> In [2]: list()[0]
>> IndexError: list index out of range
>
>
> One way to make this cheap is to have a reasonable default message and
use attributes on the exceptions trigger the use of the default message.
Nearly a year ago I filed a bunch of issues for ideas on providing
attributes on exceptions where it made sense, e.g. an index attribute on
IndexError (http://bugs.python.org/issue18162). If we did this then for
classes like IndexError there constructor could be `IndexError(index=10,
start=0, end=3)` and then __str__() can lazily construct the string
representation using a default message, e.g. `"index {} is out of range
of{} to {}".format(index, start, end)`. Make the arguments keyword-only and
they become backwards-compatible and so the only overhead you pay for these
richer messages are keyword-based construction if you simply never access
the repr for the exception.

It seems like this might need some care, though, to make sure that these
extra attributes don't end up pinning objects in memory that shouldn't be?
Actually I always assumed that was why AttributeError's message was
constructed eagerly...

-n
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Terry Reedy]

On 11/29/2016 11:32 AM, Rob Cliffe wrote:



On 29/11/2016 04:58, victor rajewski wrote:


Traceback (most recent call last):

 File "foo.py", line 2, in 

   l[10]=14

IndexError: list assignment index out of range


A better message might be:

You tried to use l[10] when l is only 4 elements long. You can add
items to l using l.append(value), or check your index value to make
sure that's really the position you wanted to access.




It would make sense to me to upgrade this particular error message to
IndexError: list assignment index 10 out of range 0 to 3 if it can
be done without too much difficulty or overhead.  (An empty list, and
perhaps one with only 1 element, would be special cases.)  Come to think
of it, is the word "assignment" needed?


It would help if the line were "l1[10] = 2 * l2[13] + 3".




--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Brendan Barnwell]

On 2016-11-29 09:43, Brett Cannon wrote:

One way to make this cheap is to have a reasonable default message and
use attributes on the exceptions trigger the use of the default message.
Nearly a year ago I filed a bunch of issues for ideas on providing
attributes on exceptions where it made sense, e.g. an index attribute on
IndexError (http://bugs.python.org/issue18162). If we did this then for
classes like IndexError there constructor could be `IndexError(index=10,
start=0, end=3)` and then __str__() can lazily construct the string
representation using a default message, e.g. `"index {} is out of range
of{} to {}".format(index, start, end)`. Make the arguments keyword-only
and they become backwards-compatible and so the only overhead you pay
for these richer messages are keyword-based construction if you simply
never access the repr for the exception.


	I absolutely think this is the way to go.  Having the relevant 
information (the list that was too short, the index that was too big, 
the key that wasn't there, etc.) is useful in many situations, and it's 
much better to have that information in a programmatic form than just 
squashed into an error message.  This then makes it relatively easy to 
write wrappers that take bubbling-up exceptions and try to construct 
more detailed messages for a less experienced audience.  Right now this 
is difficult or impossible because the exception objects don't record 
the information that would be needed for these expanded messages.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Brett Cannon]

On Tue, 29 Nov 2016 at 10:28 Nick Timkovich  wrote:

> I would consider the speed of the "ultimate error handler" (i.e. whatever
> prints the traceback and kills the program) in the interpreter to be moot,
> so long as it takes a small fraction of a second. Optimizing Python's speed
> it crashes super-fast due to an *unhandled* NameError in your program seems
> folly.
>

So the performance worry isn't the traceback printer once the call stack
fully unwinds but the construction of the exception instance itself. E.g.
having to construct an expensive string for every instance of
AttributeError is more the worry than printing a traceback.


>
> Regarding more informative messages for (e.g.) IndexError, would those
> just apply to built-in types as they're the most universal, or should some
> additional introspection be done for similar ducks?
>
> If it's useful/there's interest, I could try to do some analysis of Python
> questions on SO and see what the most common errors are. I'd guess things
> like "'NoneType' object has no attribute ..." would probably be up there,
> but that's a whole can of worms as to why someone's trying to call a method
> on, index, etc. something they accidentally whacked (a = a.sort()).
>

I suspect if we decide to try and go with more informative messages with a
solution people are happy with then having an idea of where people get
tripped up regularly will be good to help focus the work.

And as for the NoneType issue specifically, there's an issue for that:
http://bugs.python.org/issue28702 .


>
> On Tue, Nov 29, 2016 at 11:42 AM, Chris Barker 
> wrote:
>
> On Tue, Nov 29, 2016 at 5:48 AM, Nick Coghlan  wrote:
>
> > SyntaxErrors in an inner loop? That seems unlikely to me.
>
>
> Syntax Errors are a special case, as by definition the code isn't being
> run yet (yes, there could be an eval in there...)
>
> So we could at least make those more informative without worrying about
> performance.
>
> Also -- would it be possible to tack on the more informative message at a
> higher level? Once the Exception bubbles up to the REPL, is there enough
> information available to make a more informative message?
>
> -CHB
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Nick Timkovich]

I would consider the speed of the "ultimate error handler" (i.e. whatever
prints the traceback and kills the program) in the interpreter to be moot,
so long as it takes a small fraction of a second. Optimizing Python's speed
it crashes super-fast due to an *unhandled* NameError in your program seems
folly.

Regarding more informative messages for (e.g.) IndexError, would those just
apply to built-in types as they're the most universal, or should some
additional introspection be done for similar ducks?

If it's useful/there's interest, I could try to do some analysis of Python
questions on SO and see what the most common errors are. I'd guess things
like "'NoneType' object has no attribute ..." would probably be up there,
but that's a whole can of worms as to why someone's trying to call a method
on, index, etc. something they accidentally whacked (a = a.sort()).

On Tue, Nov 29, 2016 at 11:42 AM, Chris Barker 
wrote:

> On Tue, Nov 29, 2016 at 5:48 AM, Nick Coghlan  wrote:
>
>> > SyntaxErrors in an inner loop? That seems unlikely to me.
>>
>
> Syntax Errors are a special case, as by definition the code isn't being
> run yet (yes, there could be an eval in there...)
>
> So we could at least make those more informative without worrying about
> performance.
>
> Also -- would it be possible to tack on the more informative message at a
> higher level? Once the Exception bubbles up to the REPL, is there enough
> information available to make a more informative message?
>
> -CHB
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Chris Barker]

On Tue, Nov 29, 2016 at 5:48 AM, Nick Coghlan  wrote:

> > SyntaxErrors in an inner loop? That seems unlikely to me.
>

Syntax Errors are a special case, as by definition the code isn't being run
yet (yes, there could be an eval in there...)

So we could at least make those more informative without worrying about
performance.

Also -- would it be possible to tack on the more informative message at a
higher level? Once the Exception bubbles up to the REPL, is there enough
information available to make a more informative message?

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Brett Cannon]

On Tue, 29 Nov 2016 at 02:39 Nathaniel Smith  wrote:

> On Tue, Nov 29, 2016 at 1:05 AM, Victor Stinner
>  wrote:
> > Hi,
> >
> > Python is optimized for performance. Formatting an error message has a
> > cost on performances.
>
> Sure, but we have to look at this on a case-by-case basis. Is there
> really important code out there that's generating NameErrors or
> SyntaxErrors in an inner loop? That seems unlikely to me.
>
> Even IndexError I'm a bit skeptical about. I can believe that there's
> code that intentionally generates and then catches IndexError, but
> AttributeError in my experience is much more performance-sensitive
> than IndexError, because every failed hasattr call allocates an
> AttributeError and hasattr is commonly used for feature checks. Yet
> AttributeError has a much more informative (= expensive) message than
> IndexError:
>
> In [1]: object().a
> AttributeError: 'object' object has no attribute 'a'
>
> In [2]: list()[0]
> IndexError: list index out of range
>

One way to make this cheap is to have a reasonable default message and use
attributes on the exceptions trigger the use of the default message. Nearly
a year ago I filed a bunch of issues for ideas on providing attributes on
exceptions where it made sense, e.g. an index attribute on IndexError (
http://bugs.python.org/issue18162). If we did this then for classes like
IndexError there constructor could be `IndexError(index=10, start=0,
end=3)` and then __str__() can lazily construct the string representation
using a default message, e.g. `"index {} is out of range of{} to
{}".format(index, start, end)`. Make the arguments keyword-only and they
become backwards-compatible and so the only overhead you pay for these
richer messages are keyword-based construction if you simply never access
the repr for the exception.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Rob Cliffe]

On 29/11/2016 04:58, victor rajewski wrote:


Traceback (most recent call last):

 File "foo.py", line 2, in 

   l[10]=14

IndexError: list assignment index out of range


A better message might be:

You tried to use l[10] when l is only 4 elements long. You can add 
items to l using l.append(value), or check your index value to make 
sure that's really the position you wanted to access.





It would make sense to me to upgrade this particular error message to
IndexError: list assignment index 10 out of range 0 to 3 if it can be 
done without too much difficulty or overhead.  (An empty list, and 
perhaps one with only 1 element, would be special cases.)  Come to think 
of it, is the word "assignment" needed?

Rob Cliffe
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Nick Coghlan]

On 29 November 2016 at 20:38, Nathaniel Smith  wrote:
> On Tue, Nov 29, 2016 at 1:05 AM, Victor Stinner
>  wrote:
>> Hi,
>>
>> Python is optimized for performance. Formatting an error message has a
>> cost on performances.
>
> Sure, but we have to look at this on a case-by-case basis. Is there
> really important code out there that's generating NameErrors or
> SyntaxErrors in an inner loop? That seems unlikely to me.

Right, we generally treat error message formatting code as being off
the critical performance path.

In many (most?) cases, the instances of uninformative error message
are just a symptom of the code in question being really *old*, such
that it predates the great many improvements made to the low level
error reporting machinery over the years.

That's not always true (e.g. parser errors are uninformative because
the parser doesn't keep track of the state needed to generate nicer
messages), but it seems to typically be true for runtime errors where
we don't even report the type or representation of a misbehaving
value.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Wes Turner]

On Tuesday, November 29, 2016, Wes Turner  wrote:

> The existing docs for errors and exceptions:
>
> - https://docs.python.org/2/library/exceptions.html
> - https://docs.python.org/3/library/exceptions.html
> - https://hg.python.org/cpython/file/tip/Doc/library/exceptions.rst
> - https://github.com/python/cpython/blob/master/Doc/library/exceptions.rst
>
> - https://docs.python.org/2/tutorial/errors.html
> - https://docs.python.org/3/tutorial/errors.html
> - https://hg.python.org/cpython/file/tip/Doc/tutorial/errors.rst
> - https://github.com/python/cpython/blob/master/Doc/tutorial/errors.rst
>
> - https://www.tutorialspoint.com/python/python_exceptions.htm
>
> - If the docs don't answer the question (and match to the search terms),
> they probably should.
>
> - [ ] DOC: something about why "except Exception: pass" is usually bad
> - [ ] DOC: something about SystemExit and atexit: https://docs.python.
> org/2/library/atexit.html
>
>
> You can get alot more traceback from pytest (w/ pytest-sugar) and/or nose
> (with nose-progressive).
>
> There is extra information in the stack at exception time; but, IIUC, it
> would take a number of subclasses with class-specific docs and/or class
> introspection to be as detailed as "you probably wanted .append there
> because this is a List and the length is n but the key was".
>
> Maybe a "learning mode" which automatically calls inspect.getdoc() on
> Exception would be useful (sys.excepthook)?
> Practically, I usually just open an extra IPython shell and run
> `list.append?` for docs or `list.append??` for (Python but not C!) source
> (inspect.getsource).
> IPython also prints the function signature with `?`
>

- [ ] How could I also print out type annotations w/ function signatures
and docstrings?/??/???
https://github.com/python/typeshed/blob/master/stdlib/2/typing.pyi


>
> The pdb++ debugger requires funcsigs in order to print function
> signatures. If pdb++ is installed, it preempts the standard pdb module; so
> `nosetests --pdb` and `pytest --pdb` launch pdb++ when an error or
> exception is raised.
>
> https://pypi.python.org/pypi/pdbpp/
>
> http://nose.readthedocs.io/en/latest/plugins/debug.html
>
> http://doc.pytest.org/en/latest/usage.html
>
> https://docs.python.org/2/library/inspect.html
>
> Exceptions could be better someday. Testing (and debugging) skills are
> always good to learn; coincidentally, there are many great tools for it.
>
> ... https://westurner.org/wiki/awesome-python-testing#debugging
>
> On Tuesday, November 29, 2016, Victor Stinner  > wrote:
>
>> Hi,
>>
>> Python is optimized for performance. Formatting an error message has a
>> cost on performances.
>>
>> I suggest you to teach your student to use the REPL and use a custom
>> exception handler: sys.excepthook:
>> https://docs.python.org/2/library/sys.html#sys.excepthook
>>
>> Using a custom exception handler, you can run expensive functions,
>> like the feature: "suggest len when length is used".
>>
>> The problem is then when students have to use a Python without the
>> custom exception handler.
>>
>> Victor
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Wes Turner]

On Tuesday, November 29, 2016, Nathaniel Smith  wrote:

> On Tue, Nov 29, 2016 at 1:05 AM, Victor Stinner
> > wrote:
> > Hi,
> >
> > Python is optimized for performance. Formatting an error message has a
> > cost on performances.
>
> Sure, but we have to look at this on a case-by-case basis. Is there
> really important code out there that's generating NameErrors or
> SyntaxErrors in an inner loop? That seems unlikely to me.
>
> Even IndexError I'm a bit skeptical about. I can believe that there's
> code that intentionally generates and then catches IndexError, but
> AttributeError in my experience is much more performance-sensitive
> than IndexError, because every failed hasattr call allocates an
> AttributeError and hasattr is commonly used for feature checks. Yet
> AttributeError has a much more informative (= expensive) message than
> IndexError:
>
> In [1]: object().a
> AttributeError: 'object' object has no attribute 'a'
>
> In [2]: list()[0]
> IndexError: list index out of range


https://docs.python.org/2/tutorial/datastructures.html#more-on-lists

https://docs.python.org/2/c-api/list.html#c.PyList_SetItem

https://github.com/python/cpython/search?utf8=✓=PyList_SetItem

- https://github.com/python/cpython/blob/master/Include/listobject.h
- https://github.com/python/cpython/blob/master/Objects/listobject.c#L208
- https://hg.python.org/cpython/file/tip/Objects/listobject.c#l208

https://docs.python.org/3/library/collections.abc.html#collections-abstract-base-classes

- list (typeshed: List)
- Sequence
- MutableSequence
-

It would be great if these continue to match:
https://www.google.com/search?q=IndexError%3A+list+index+out+of+range



>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> Python-ideas mailing list
> Python-ideas@python.org 
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Wes Turner]

The existing docs for errors and exceptions:

- https://docs.python.org/2/library/exceptions.html
- https://docs.python.org/3/library/exceptions.html
- https://hg.python.org/cpython/file/tip/Doc/library/exceptions.rst
- https://github.com/python/cpython/blob/master/Doc/library/exceptions.rst

- https://docs.python.org/2/tutorial/errors.html
- https://docs.python.org/3/tutorial/errors.html
- https://hg.python.org/cpython/file/tip/Doc/tutorial/errors.rst
- https://github.com/python/cpython/blob/master/Doc/tutorial/errors.rst

- https://www.tutorialspoint.com/python/python_exceptions.htm

- If the docs don't answer the question (and match to the search terms),
they probably should.

- [ ] DOC: something about why "except Exception: pass" is usually bad
- [ ] DOC: something about SystemExit and atexit:
https://docs.python.org/2/library/atexit.html


You can get alot more traceback from pytest (w/ pytest-sugar) and/or nose
(with nose-progressive).

There is extra information in the stack at exception time; but, IIUC, it
would take a number of subclasses with class-specific docs and/or class
introspection to be as detailed as "you probably wanted .append there
because this is a List and the length is n but the key was".

Maybe a "learning mode" which automatically calls inspect.getdoc() on
Exception would be useful (sys.excepthook)?
Practically, I usually just open an extra IPython shell and run
`list.append?` for docs or `list.append??` for (Python but not C!) source
(inspect.getsource).
IPython also prints the function signature with `?`

The pdb++ debugger requires funcsigs in order to print function signatures.
If pdb++ is installed, it preempts the standard pdb module; so `nosetests
--pdb` and `pytest --pdb` launch pdb++ when an error or exception is raised.

https://pypi.python.org/pypi/pdbpp/

http://nose.readthedocs.io/en/latest/plugins/debug.html

http://doc.pytest.org/en/latest/usage.html

https://docs.python.org/2/library/inspect.html

Exceptions could be better someday. Testing (and debugging) skills are
always good to learn; coincidentally, there are many great tools for it.

... https://westurner.org/wiki/awesome-python-testing#debugging

On Tuesday, November 29, 2016, Victor Stinner 
wrote:

> Hi,
>
> Python is optimized for performance. Formatting an error message has a
> cost on performances.
>
> I suggest you to teach your student to use the REPL and use a custom
> exception handler: sys.excepthook:
> https://docs.python.org/2/library/sys.html#sys.excepthook
>
> Using a custom exception handler, you can run expensive functions,
> like the feature: "suggest len when length is used".
>
> The problem is then when students have to use a Python without the
> custom exception handler.
>
> Victor
> ___
> Python-ideas mailing list
> Python-ideas@python.org 
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Victor Stinner]

Hi,

Python is optimized for performance. Formatting an error message has a
cost on performances.

I suggest you to teach your student to use the REPL and use a custom
exception handler: sys.excepthook:
https://docs.python.org/2/library/sys.html#sys.excepthook

Using a custom exception handler, you can run expensive functions,
like the feature: "suggest len when length is used".

The problem is then when students have to use a Python without the
custom exception handler.

Victor
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Mariatta Wijaya]

I'm +1 to the idea of improving error messages :)
(but maybe not to the exact new error messages proposed)

Raymond Hettinger touched on this topic during his Pycon Canada keynote, as
one of the positive contributions that you can do to cpython.

>

> Traceback (most recent call last):

>

>  File "foo.py", line 2, in 

>

>print(length(l))

>

> NameError: name 'length' is not defined

>

> A better message might be:

>

> Python doesn't recognise the function "length". Did you mean len?'

>

I recall he gave a similar example to this, where Python could suggest an
alternative in case of typo.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[victor rajewski]

Sorry, I forgot the subject line!

On Tue., 29 Nov. 2016, 3:58 pm victor rajewski,  wrote:

> I teach a computing subject to high school students using Python as our
> primary language. One thing that often causes confusion at the start is
> error messages/exceptions. I think it would lower entry point quite a bit
> by making error messages more readable to novices. Recent research found
> reduced frequency of overall errors, errors per student, and repeated
> errors when error messages were enhanced [1].
>
> This could be implemented by a command-line switch when running python, or
> a different python executable (along the lines of pythonw.exe vs
> python.exe) such as python-easy. A simpler error message might get rid of
> excess information which can confuse the novice (such info might be dumped
> to a file to allow more advanced users to debug), provide a brief
> description of what the error might mean, and/or how you might fix it. So a
> SyntaxError might tell the user that they've probably forgotten a :, and a
> NameError might say that the item has been defined yet, or they've made a
> typo. A couple of examples follow:
>
> Traceback (most recent call last):
>
>  File "foo.py", line 2, in 
>
>l[10]=14
>
> IndexError: list assignment index out of range
>
> A better message might be:
>
> You tried to use l[10] when l is only 4 elements long. You can add items
> to l using l.append(value), or check your index value to make sure that's
> really the position you wanted to access.
>
> Traceback (most recent call last):
>
>  File "foo.py", line 2, in 
>
>while name != "quit" and reponse != "quit":
>
> NameError: name 'reponse' is not defined
>
> A better message might be:
>
> You're trying to use the value of 'reponse', but that variable hasn't got
> a value yet. You can give it a value earlier in the code, or it could be a
> typo. You have a variable called 'response' - is that the one you meant?
>
> Traceback (most recent call last):
>
>  File "foo.py", line 2, in 
>
>print(length(l))
>
> NameError: name 'length' is not defined
>
> A better message might be:
>
> Python doesn't recognise the function "length". Did you mean len?'
>
>  File "foo.py", line 2
>
>for i in l
>
> ^
>
> SyntaxError: invalid syntax
>
> A better message might be:
>
> You have a for loop without a : (colon). Try adding a colon at the end of
> the line.
>
>
> Any thoughts?
>
> [1]: http://researchrepository.ucd.ie/handle/10197/7583
> --
>
> Victor Rajewski
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

-- 

Victor Rajewski
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Bernardo Sulzbach]

On 2016-11-29 02:58, victor rajewski wrote:

NameError: name 'reponse' is not defined

A better message might be:

You're trying to use the value of 'reponse', but that variable hasn't
got a value yet. You can give it a value earlier in the code, or it
could be a typo. You have a variable called 'response' - is that the one
you meant?



This is a really nice feature. GHC and Clang do it for several things 
and it usually helps (when a live highlighter isn't available).


Given that these are HS students, presumably seeing Python for the first 
time, shouldn't they be using PyCharm to get used to PEP 8 (no l[10]=14) 
and other best practices (e.g., such as using 'with' instead of 
'try/finally' when possible)? This would provide live syntactic (and 
some semantic) highlighting with short human-readable messages.


Overall, I already find Python's exceptions quite readable, and your 
suggestions propose some VERY long lines which would certainly wrap at a 
terminal. Maybe this should be an optional feature.


--
Bernardo Sulzbach
http://www.mafagafogigante.org/
mafagafogiga...@gmail.com
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] (no subject)

[victor rajewski]

I teach a computing subject to high school students using Python as our
primary language. One thing that often causes confusion at the start is
error messages/exceptions. I think it would lower entry point quite a bit
by making error messages more readable to novices. Recent research found
reduced frequency of overall errors, errors per student, and repeated
errors when error messages were enhanced [1].

This could be implemented by a command-line switch when running python, or
a different python executable (along the lines of pythonw.exe vs
python.exe) such as python-easy. A simpler error message might get rid of
excess information which can confuse the novice (such info might be dumped
to a file to allow more advanced users to debug), provide a brief
description of what the error might mean, and/or how you might fix it. So a
SyntaxError might tell the user that they've probably forgotten a :, and a
NameError might say that the item has been defined yet, or they've made a
typo. A couple of examples follow:

Traceback (most recent call last):

 File "foo.py", line 2, in 

   l[10]=14

IndexError: list assignment index out of range

A better message might be:

You tried to use l[10] when l is only 4 elements long. You can add items to
l using l.append(value), or check your index value to make sure that's
really the position you wanted to access.

Traceback (most recent call last):

 File "foo.py", line 2, in 

   while name != "quit" and reponse != "quit":

NameError: name 'reponse' is not defined

A better message might be:

You're trying to use the value of 'reponse', but that variable hasn't got a
value yet. You can give it a value earlier in the code, or it could be a
typo. You have a variable called 'response' - is that the one you meant?

Traceback (most recent call last):

 File "foo.py", line 2, in 

   print(length(l))

NameError: name 'length' is not defined

A better message might be:

Python doesn't recognise the function "length". Did you mean len?'

 File "foo.py", line 2

   for i in l

^

SyntaxError: invalid syntax

A better message might be:

You have a for loop without a : (colon). Try adding a colon at the end of
the line.


Any thoughts?

[1]: http://researchrepository.ucd.ie/handle/10197/7583
-- 

Victor Rajewski
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/