Re: What to use for finding as many syntax errors as possible.

[Antoon Pardon]

Op 10/10/2022 om 00:45 schreef Cameron Simpson:

On 09Oct2022 21:46, Antoon Pardon  wrote:
Is it that onerous to fix one thing and run it again? It was once 
when you

handed in punch cards and waited a day or on very busy machines.


Yes I find it onerous, especially since I have a pipeline with unit 
tests
and other tools that all have to redo their work each time a bug is 
corrected.


It is easy to get the syntax right before submitting to such a 
pipeline.  I usually run a linter on my code for serious commits, and 
I've got a `lint1` alias which basicly runs the short fast flavour of 
that which does a syntax check and the very fast less thorough lint phase.


If you have a linter that doesn't quit after the first syntax error, 
please provide a link. I already tried pylint and it also quits after 
the first syntax error.


--
Antoon Pardon
--
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Cameron Simpson]

On 10Oct2022 09:04, Antoon Pardon  wrote:
It is easy to get the syntax right before submitting to such a 
pipeline.  I usually run a linter on my code for serious commits, and 
I've got a `lint1` alias which basicly runs the short fast flavour of 
that which does a syntax check and the very fast less thorough lint 
phase.


If you have a linter that doesn't quit after the first syntax error, 
please provide a link. I already tried pylint and it also quits after 
the first syntax error.


I don't have such a linter. I did outline an approach for you to write 
one of your own by wrapping an existing parser program.


I have a personal "lint" script which runs a few linters. The first 
check is `py_compile` which quits at the first syntax error. The other 
linters are not even tried if that fails.


I do not know what your editing environment is; I'd have thought that 
some IDEs should make the first syntax error very obvious and easy to go 
to, and an obvious indication that the file as a whoe is syntacticly 
good/bad. If you have such, between them you could fairly easily resolve 
syntax errors rapidly, perhaps rapidly enough to make up for a 
stop-at-the-first-fail syntax check.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Karsten Hilbert]

Am Sun, Oct 09, 2022 at 09:58:14AM + schrieb Stefan Ram:

>   I often follow this rule. For me, it's about readability. Compare:
>
> if not open( disk ):
> error( "Can't open disk" )
> else:
> printf( "now imagine there's some larger block here" )
... ad infinitum 

Should this not be

if not open( disk ):
error( "Can't open disk" )
else:
do_lots_of_things_with(disk)

as for readability ?

Or even

if not open( disk ):
error( "Can't open disk" )
return what_needs_to_be_returned

do_lots_of_things_with(disk)

The latter version may need some code reorganization, though.

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Axy via Python-list]

On 09/10/2022 03:33, Jach Feng wrote:

Axy 在 2022年10月8日 星期�
�上午11:39:44 [UTC+8] 的信中寫道：

Hi there,

this is rather a philosophical question, but I assume I miss something.
I don't remember I ever used else clause for years I was with python and
my expectation was it executed only if the the main body was never run.
Ha-ha! I was caught by this mental trap.

So, seriously, why they needed else if the following pieces produce same
result? Does anyone know or remember their motivation?

Just curious.

Axy.

print('--- with else')


for i in [1,2,3]:
 print(i)
else:
 print(4)

for i in []:
 print(i)
else:
 print(5)

print('--- without else')

for i in [1,2,3]:
 print(i)
print(4)

for i in []:
 print(i)
print(5)

The else is always coming with the break, not the for.

However, the compiler does not complain.

  There are [for ...], [for...break...], and[for...break...else],


That's implied and contradicts Zen of Python, I think. If "else" came 
with "break" there had to be a strong indication of that, namely 
indentation, as it takes place for all other statements with their 
clauses. However, there's no such an explicit connection between "break" 
and "else". That's the point.


Well, sorry for this addition to the discussion which went weird way. I 
should had to be cautious mentioning particular coding style, that's a 
totally different subject, actually. Let's close it at last.



  but the [for...else] is insane.

Not in Python.

Axy.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Chris Angelico]

On Mon, 10 Oct 2022 at 20:46, Karsten Hilbert  wrote:
>
> Am Sun, Oct 09, 2022 at 09:58:14AM + schrieb Stefan Ram:
>
> >   I often follow this rule. For me, it's about readability. Compare:
> >
> > if not open( disk ):
> > error( "Can't open disk" )
> > else:
> > printf( "now imagine there's some larger block here" )
> ... ad infinitum 
>
> Should this not be
>
> if not open( disk ):
> error( "Can't open disk" )
> else:
> do_lots_of_things_with(disk)
>
> as for readability ?
>
> Or even
>
> if not open( disk ):
> error( "Can't open disk" )
> return what_needs_to_be_returned
>
> do_lots_of_things_with(disk)
>
> The latter version may need some code reorganization, though.

 I definitely prefer the fail-and-bail version (mainly because, as the
number of possible failure modes increases, the code complexity
increases linearly, whereas with if/else style it will increase
quadratically - you have to edit the entire block, and indent it, for
each one). But contrasting the original two? A complete wash. There's
no readability difference between them.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Chris Angelico]

On Mon, 10 Oct 2022 at 20:56, Axy via Python-list
 wrote:
>
> > The else is always coming with the break, not the for.
> However, the compiler does not complain.
> >   There are [for ...], [for...break...], and[for...break...else],
>
> That's implied and contradicts Zen of Python, I think. If "else" came
> with "break" there had to be a strong indication of that, namely
> indentation, as it takes place for all other statements with their
> clauses. However, there's no such an explicit connection between "break"
> and "else". That's the point.
>
> Well, sorry for this addition to the discussion which went weird way. I
> should had to be cautious mentioning particular coding style, that's a
> totally different subject, actually. Let's close it at last.
>
> >   but the [for...else] is insane.
> Not in Python.
>

Not sure what you mean, but a for-else without a break is quite
useless. What exactly ARE you arguing here?

The else is associated with the break to the exact extent that one is
essential to the other's value.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Axy via Python-list]

Not sure what you mean, but a for-else without a break is quite
useless. What exactly ARE you arguing here?

The else is associated with the break to the exact extent that one is
essential to the other's value.


I'm not arguing. That was just for the record, how things are done in 
Python. Basically, I simply asked a question and got a definite answer 
and clear understanding shortly, in a few replies. All the rest of this 
thread looks irrelevant to me, it's about coding style and probably 
should be continued under a different title, but I'm not interested to 
participate in it.


Axy.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Cheetah 3.3.0

[Oleg Broytman]

Hello!

I'm pleased to announce version 3.3.0, the 1st release
of branch 3.3 of CheetahTemplate3.


What's new in CheetahTemplate3
==

The contributors for this release are:
N Protokowicz, Enzo Conty, Andrea Mennucci, Saiprasad Kale, odidev,
Pierre Ossman. Many thanks!

Great move:

  - PyPI has wrongfully classified project ``Cheetah3`` as "critical".
This puts a burden to use 2FA to manage the project at PyPI. To
avoid the burden the project is renamed to ``CT3`` at PyPI.
There will be no updates for ``Cheetah3``.
Sorry for the inconvenience!

Minor features:

  - Use relative imports everywhere.

Tests:

  - Run pure-python ``NameMapper`` tests in a separate process.

  - Fixed a bug in tests with pure-python ``NameMapper``.

  - Add Python 3.10 to ``tox.ini``.

CI:

  - Migrated to GitHub Actions.

Due to the absent of Python 3.4 at GH Actions tests are not run and
wheels are not built. Installation from sources should work.

Due to GH Actions lacking old compilers for w32/w64 releases for old
Python versions (currently 2.7) are packaged without compiled
_namemapper.pyd extension. Cheetah can be used without compiled
_namemapper.pyd extension. A pure-python replacement should work;
``Cheetah`` imports it automatically if the compiled extension is
not available.

  - Stop testing at Travis CI.

  - Stop testing at AppVeyor.


What is CheetahTemplate3


Cheetah3 is a free and open source (MIT) Python template engine.
It's a fork of the original CheetahTemplate library.

Python 2.7 or 3.4+ is required.


Where is CheetahTemplate3
=

Site:
https://cheetahtemplate.org/

Download:
https://pypi.org/project/CT3/3.3.0

News and changes:
https://cheetahtemplate.org/news.html

StackOverflow:
https://stackoverflow.com/questions/tagged/cheetah

Mailing lists:
https://sourceforge.net/p/cheetahtemplate/mailman/

Development:
https://github.com/CheetahTemplate3

Developer Guide:
https://cheetahtemplate.org/dev_guide/


Example
===

Install::

$ pip install CT3 # (or even "ct3")

Below is a simple example of some Cheetah code, as you can see it's practically
Python. You can import, inherit and define methods just like in a regular Python
module, since that's what your Cheetah templates are compiled to :) ::

#from Cheetah.Template import Template
#extends Template

#set $people = [{'name' : 'Tom', 'mood' : 'Happy'}, {'name' : 'Dick',
'mood' : 'Sad'}, {'name' : 'Harry', 'mood' : 
'Hairy'}]

How are you feeling?

#for $person in $people

$person['name'] is $person['mood']

#end for


Oleg.
-- 
Oleg Broytmanhttps://phdru.name/p...@phdru.name
   Programmers don't die, they just GOSUB without RETURN.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Chris Angelico]

On Mon, 10 Oct 2022 at 21:57, Axy via Python-list
 wrote:
>
>
> > Not sure what you mean, but a for-else without a break is quite
> > useless. What exactly ARE you arguing here?
> >
> > The else is associated with the break to the exact extent that one is
> > essential to the other's value.
>
> I'm not arguing. That was just for the record, how things are done in
> Python. Basically, I simply asked a question and got a definite answer
> and clear understanding shortly, in a few replies. All the rest of this
> thread looks irrelevant to me, it's about coding style and probably
> should be continued under a different title, but I'm not interested to
> participate in it.

Here's where the "rest of this thread" started:

> Actually the reason I never used "else" was the violation of the rule of
> beauty "shortest block first".

You disparaged a feature on the basis of a style rule that few of us
had heard of or agree with. We all agree that coding style is
important; none of us would see block length as a reason to avoid
using an else clause on a for loop.

Your subsequent posts have left me confused as to what you're trying to convey.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Axy via Python-list]

On 10/10/2022 12:24, Chris Angelico wrote:

On Mon, 10 Oct 2022 at 21:57, Axy via Python-list
 wrote:



Not sure what you mean, but a for-else without a break is quite
useless. What exactly ARE you arguing here?

The else is associated with the break to the exact extent that one is
essential to the other's value.

I'm not arguing. That was just for the record, how things are done in
Python. Basically, I simply asked a question and got a definite answer
and clear understanding shortly, in a few replies. All the rest of this
thread looks irrelevant to me, it's about coding style and probably
should be continued under a different title, but I'm not interested to
participate in it.

Here's where the "rest of this thread" started:


Actually the reason I never used "else" was the violation of the rule of
beauty "shortest block first".

You disparaged a feature on the basis of a style rule that few of us
had heard of or agree with.

Oh, I'm really sorry. My apologies.

  We all agree that coding style is
important; none of us would see block length as a reason to avoid
using an else clause on a for loop.


As I understand from the above there must be a committee that delegates 
a speaker? Where to read rules? How to participate? There's something 
beyond this list I'm not aware of yet?


Axy.
--
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Chris Angelico]

On Mon, 10 Oct 2022 at 22:37, Axy via Python-list
 wrote:
>
>
> On 10/10/2022 12:24, Chris Angelico wrote:
> > On Mon, 10 Oct 2022 at 21:57, Axy via Python-list
> >  wrote:
> >>
> >>> Not sure what you mean, but a for-else without a break is quite
> >>> useless. What exactly ARE you arguing here?
> >>>
> >>> The else is associated with the break to the exact extent that one is
> >>> essential to the other's value.
> >> I'm not arguing. That was just for the record, how things are done in
> >> Python. Basically, I simply asked a question and got a definite answer
> >> and clear understanding shortly, in a few replies. All the rest of this
> >> thread looks irrelevant to me, it's about coding style and probably
> >> should be continued under a different title, but I'm not interested to
> >> participate in it.
> > Here's where the "rest of this thread" started:
> >
> >> Actually the reason I never used "else" was the violation of the rule of
> >> beauty "shortest block first".
> > You disparaged a feature on the basis of a style rule that few of us
> > had heard of or agree with.
> Oh, I'm really sorry. My apologies.
> >   We all agree that coding style is
> > important; none of us would see block length as a reason to avoid
> > using an else clause on a for loop.
>
> As I understand from the above there must be a committee that delegates
> a speaker? Where to read rules? How to participate? There's something
> beyond this list I'm not aware of yet?

We in this thread. Look at the past replies.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Grant Edwards]

On 2022-10-10, Chris Angelico  wrote:
> On Mon, 10 Oct 2022 at 11:52, MRAB  wrote:
>>
>> On 2022-10-10 00:40, dn wrote:
>> > On Sun, 9 Oct 2022 at 15:39, Axy via Python-list
>> >  wrote:
>> >
>> >> "shortest block first"
>> >
>> > Have never heard this advice before. Kind-of rankled with me, as it did
>> > for others.

I've followed that advice for several decades. I find it much easier
to read code that's organized that way -- particularly when the
difference in block sizes is large (e.g. the first block is one line,
and the second is a a hundred).

--
Grant

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Weatherby,Gerard]

try:
open(disk)
except:
error(“Can’t open disk”)
lots of things

From: Python-list  on 
behalf of Karsten Hilbert 
Date: Monday, October 10, 2022 at 5:46 AM
To: python-list@python.org 
Subject: Re: for -- else: what was the motivation?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Am Sun, Oct 09, 2022 at 09:58:14AM + schrieb Stefan Ram:

>   I often follow this rule. For me, it's about readability. Compare:
>
> if not open( disk ):
> error( "Can't open disk" )
> else:
> printf( "now imagine there's some larger block here" )
... ad infinitum 

Should this not be

if not open( disk ):
error( "Can't open disk" )
else:
do_lots_of_things_with(disk)

as for readability ?

Or even

if not open( disk ):
error( "Can't open disk" )
return what_needs_to_be_returned

do_lots_of_things_with(disk)

The latter version may need some code reorganization, though.

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nWwvwKC2eetL3YQnTS4jDih5YRZ_ziu4gPuU73R7LDX8-Kq0bXR-h4E_0kJopk-ud2oeAq6NwvykTB40o82ris6pM1aC$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Calvin Spealman]

On Sat, Oct 8, 2022 at 5:35 PM rbowman  wrote:

> On 10/7/22 21:32, Axy wrote:
> > So, seriously, why they needed else if the following pieces produce same
> > result? Does anyone know or remember their motivation?
>
> In real scenarios there would be more logic in the for block that would
> meet a condition and break out of the loop. If the condition is never
> met, the else block runs. To steal from w3schools:
>
>
> fruits = ["apple", "peach", "cherry"]
> for x in fruits:
>print(x)
>if x == "banana":
>  break
> else:
>print("Yes we got no bananas")
>

I wonder if for/else could have been less confusing if it was referred to
as for-break-else and if the else clause was only valid syntax if the for
loop actually contained a break statement in the first place.

-- 

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.speal...@redhat.com  M: +1.336.210.5107
[image: https://red.ht/sig] 
TRIED. TESTED. TRUSTED. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Weatherby,Gerard]

Core developer Raymond Hettinger explains the history starting at 15:40 
https://www.youtube.com/watch?v=OSGv2VnC0go

(which I found on stackoverflow 
https://stackoverflow.com/questions/9979970/why-does-python-use-else-after-for-and-while-loops
 )

TL:DR
The “else” is a historical artificial from the way developers thought during 
the transition from unstructured (i.e. “GOTO”) programming to structured 
programming. Since we all do structured now, it seems odd.


From: Python-list  on 
behalf of Calvin Spealman 
Date: Monday, October 10, 2022 at 10:38 AM
To: python-list@python.org 
Subject: Re: for -- else: what was the motivation?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On Sat, Oct 8, 2022 at 5:35 PM rbowman  wrote:

> On 10/7/22 21:32, Axy wrote:
> > So, seriously, why they needed else if the following pieces produce same
> > result? Does anyone know or remember their motivation?
>
> In real scenarios there would be more logic in the for block that would
> meet a condition and break out of the loop. If the condition is never
> met, the else block runs. To steal from w3schools:
>
>
> fruits = ["apple", "peach", "cherry"]
> for x in fruits:
>print(x)
>if x == "banana":
>  break
> else:
>print("Yes we got no bananas")
>

I wonder if for/else could have been less confusing if it was referred to
as for-break-else and if the else clause was only valid syntax if the for
loop actually contained a break statement in the first place.

--

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.speal...@redhat.com  M: +1.336.210.5107
[image: 
https://urldefense.com/v3/__https://red.ht/sig__;!!Cn_UX_p3!lSA-VzlSq_UQvMUO2npcFCxacA2jOlt2ZD6LCiwdfMGpYexlKgYpIgDCxlHPs473PNyH7cww2ufihhXbt2Z5$
  ] 

TRIED. TESTED. TRUSTED. 

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!lSA-VzlSq_UQvMUO2npcFCxacA2jOlt2ZD6LCiwdfMGpYexlKgYpIgDCxlHPs473PNyH7cww2ufihnTgBbx-$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Axy via Python-list]

On 10/10/2022 15:52, Weatherby,Gerard wrote:

Core developer Raymond Hettinger explains the history starting at 15:40 
https://www.youtube.com/watch?v=OSGv2VnC0go

(which I found on stackoverflow 
https://stackoverflow.com/questions/9979970/why-does-python-use-else-after-for-and-while-loops
 )

TL:DR
The “else” is a historical artificial from the way developers thought during 
the transition from unstructured (i.e. “GOTO”) programming to structured 
programming. Since we all do structured now, it seems odd.


From: Python-list  on behalf of 
Calvin Spealman 
Date: Monday, October 10, 2022 at 10:38 AM
To: python-list@python.org 
Subject: Re: for -- else: what was the motivation?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On Sat, Oct 8, 2022 at 5:35 PM rbowman  wrote:


On 10/7/22 21:32, Axy wrote:

So, seriously, why they needed else if the following pieces produce same
result? Does anyone know or remember their motivation?

In real scenarios there would be more logic in the for block that would
meet a condition and break out of the loop. If the condition is never
met, the else block runs. To steal from w3schools:


fruits = ["apple", "peach", "cherry"]
for x in fruits:
print(x)
if x == "banana":
  break
else:
print("Yes we got no bananas")


I wonder if for/else could have been less confusing if it was referred to
as for-break-else and if the else clause was only valid syntax if the for
loop actually contained a break statement in the first place.


Sounds reasonable. It would be something alike UnboundLocalError when a 
local variable referenced before assignment. If they won't remove "else" 
completely in far future, that checking really worths implementing now.


Excellent stackoverflow link, thanks!

Axy.
--
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Axy via Python-list]

On 10/10/2022 15:52, Weatherby,Gerard wrote:
I wonder if for/else could have been less confusing if it was 
referred to
as for-break-else and if the else clause was only valid syntax if the 
for

loop actually contained a break statement in the first place.


Sounds reasonable. It would be something alike UnboundLocalError when 
a local variable referenced before assignment. If they won't remove 
"else" completely in far future, that checking really worths 
implementing now.


Actually, I think a warning would be sufficient, as in the following 
quick prototype. If someone can implement this quickly in CPython, that 
would be great (last time I hacked it, it was 2.4)


Axy.

import ast

tree = ast.parse('''
# sample code
a = 0
for i in 'asd':
    a += i
    while x:
    pass
    else:
    print('wow')
    break
    print(i)
else:
    print(0)
''', mode='exec')

def check_ast(node):
    if isinstance(node, (ast.For, ast.AsyncFor, ast.While)):
    if node.orelse and have_no_break(node.body):
    print(f'Warning: the loop at line {node.lineno} has no 
"break" statement,'
  f' "else" clause at line {node.orelse[0].lineno} 
won\'t run')

    else:
    for child in ast.iter_child_nodes(node):
    check_ast(child)

def have_no_break(loop_body):
    for node in loop_body:
    if isinstance(node, (ast.For, ast.AsyncFor, ast.While)):
    # nested loop
    check_ast(node)
    elif isinstance(node, ast.Break):
    return False
    elif isinstance(node, list):
    for child in ast.iter_child_nodes(node):
    if have_no_break(child) == False:
    return False
    return True


for node in tree.body:
    check_ast(node)
--
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Weatherby,Gerard]

pylint, at least, provides a warning:

fe.py:4:0: W0120: Else clause on loop without a break statement 
(useless-else-on-loop)


sum = 0
for i in range(5):
sum += i
else:
print("Always executes")
print(sum)


From: Python-list  on 
behalf of Axy via Python-list 
Date: Monday, October 10, 2022 at 1:10 PM
To: python-list@python.org 
Subject: Re: for -- else: what was the motivation?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

> On 10/10/2022 15:52, Weatherby,Gerard wrote:
>> I wonder if for/else could have been less confusing if it was
>> referred to
>> as for-break-else and if the else clause was only valid syntax if the
>> for
>> loop actually contained a break statement in the first place.
>
> Sounds reasonable. It would be something alike UnboundLocalError when
> a local variable referenced before assignment. If they won't remove
> "else" completely in far future, that checking really worths
> implementing now.

Actually, I think a warning would be sufficient, as in the following
quick prototype. If someone can implement this quickly in CPython, that
would be great (last time I hacked it, it was 2.4)

Axy.

import ast

tree = ast.parse('''
# sample code
a = 0
for i in 'asd':
 a += i
 while x:
 pass
 else:
 print('wow')
 break
 print(i)
else:
 print(0)
''', mode='exec')

def check_ast(node):
 if isinstance(node, (ast.For, ast.AsyncFor, ast.While)):
 if node.orelse and have_no_break(node.body):
 print(f'Warning: the loop at line {node.lineno} has no
"break" statement,'
   f' "else" clause at line {node.orelse[0].lineno}
won\'t run')
 else:
 for child in ast.iter_child_nodes(node):
 check_ast(child)

def have_no_break(loop_body):
 for node in loop_body:
 if isinstance(node, (ast.For, ast.AsyncFor, ast.While)):
 # nested loop
 check_ast(node)
 elif isinstance(node, ast.Break):
 return False
 elif isinstance(node, list):
 for child in ast.iter_child_nodes(node):
 if have_no_break(child) == False:
 return False
 return True


for node in tree.body:
 check_ast(node)
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mlK4jRkfDC_akw-fIqWaMVf707GQsiyvj_sRHTsFnuG4ak5mKWwSavtz4njlBNIu1H0VHrR9gyjuQpxGqZ1dacU1Xw$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Axy via Python-list]

On 10/10/2022 19:25, Weatherby,Gerard wrote:


pylint, at least, provides a warning:

fe.py:4:0: W0120: Else clause on loop without a break statement 
(useless-else-on-loop)



I'm using flake8, it does not, alas.

Axy.
--
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Peter J. Holzer]

On 2022-10-10 12:40:44 +1300, dn wrote:
> On 10/10/2022 05.56, Peter J. Holzer wrote:
> > On 2022-10-09 12:18:09 -0400, Avi Gross wrote:
> > > Some would argue for a rule related to efficiency of execution. When you
> > > have multiple blocks as in an if-else or case statement with multiple
> > > choices, that you order the most common cases first. Those shorten
> > > execution more often than the rarer cases especially the ones that should
> > > never happen.
> > 
> > Those of us who started programming on 8 bit homecomputers of course
> > have efficiency always at the back of their heads, but I find this
> 
> ... for mainframes just as much as micro-computers!

I knew this would be coming :-).

> Regarding execution-efficiencies, I'm sure @Avi knows better than I: It
> seems likely that Python, as an interpreted language, will create 'blocks'
> of its virtual-machine code in the same order as they appear in the
> Python-source. However, aren't there optimising compilers which do something
> intelligent with the equivalent clauses/suites in other languages?

They can certainly move the code around. So something like

if a:
long block a
elif b:
long block b
else:
long block c

could be compiled into (pseudo-python):

if a:
goto label_a
elif b:
goto label_b
long block c
goto end
label_a:
long block a
goto end
label_b:
long block b
end:
pass

If they can prove that it makes no difference they can also change the
order of checking conditions:

def f(a: int) -> str:
if a < 0:
return "negative"
elif a > 100:
return "big"
else:
return "meh"

could be compiled into the equivalent of:

def f(a: int) -> str:
if a > 100:
return "big"
elif a < 0:
return "negative"
else:
return "meh"

since an int cannot be simultaneously less then 0 and larger than 100.

But if the order does matter, as in

def f(a: bool, b: bool) -> int:
if a:
return 1
elif b:
return 2
else:
return 3

it can't do any transformations which would change the result.

(note that this is a point where it's really important what a language
does or does not guarantee)

> 
> Regardless, is a Jump-instruction which transfers else-control to a block
> five machine-instructions 'down', any less efficient than a jump which spans
> 50-instructions?

That depends on the CPU. Some CPUs have different jump instructions for
different sizes (for example, for a typical risc cpu the instruction and
the offset have to fit in a 32 bit word. So you would be limited to
direct jumps of plus or minus a few hundred megabytes. For longer jumps
you need to load the address into a register and jump indirectly. CISC
CPUs with variable length instructions may have finer distinctions.) A
very short jump may continue within the same cache line or maybe even on
an instruction which has already been decoded. A longer jump may have to
load the next instruction from RAM. Or a jump to the beginning of a
cache line may be faster than one to the middle. And then of course
there is branch prediction ...

So lots of variability there.


> > There is of course the opposite view that you should just get all of the
> > confounding factors out of the way first, so that the default is also
> > the common case. I also do that sometimes, but then I don't hide this in
> > in an else: clause but do something like this:
> > 
> > for item in whatever:
> >  if not_this_one(item):
> >  continue
> >  if neither_this_one(item):
> >  continue
> >  if cant_continue(item):
> >  break
> >  if oopsie():
> >  raise SomeError()
> > 
> >  do_something_with(item)
> >  and_some_more(item)
> >  we_are_done(item)
> > 
> > which shows visually what the main purpose of the loop (or function or
> > other block) is.
> 
> Nicely stated!
> 
> NB I've seen sufficient of @Peter's posts to know that this was never (even
> implied to be) intended as a snippet for all occasions!
> 
> 
> It also illustrates why such is less readable: because we have to scan four
> if-statements before we can 'see' the purpose of the loop. My 'itch' would
> be to extract this code 'out' to a function - that way the name will convey
> the somewhat-obscured purpose of the loop.
> 
> 
> Alternately, reduce the 'distractions':-
> 
> try:
> for item in whatever:
> inspect_the_data( item )
> do_something_with(item)
> and_some_more(item)
> we_are_done(item)
> except SomeError:
> ...
> except CustomBreakException:
> ... ?pass?# same effect as break
> 
> by 'hiding' in:
> 
> def inspect_the_data( item ):
> if not_this_one(item):
> continue

This doesn't work. You can't continue (or break) a loop in a calling
function. (I think it works (or 

Re: for -- else: what was the motivation?

[Peter J. Holzer]

On 2022-10-09 22:38:28 -0400, avi.e.gr...@gmail.com wrote:
> [This is an answer for Peter and can easily be skipped by those who know or
> have no wish to.]
> 
> Strictly speaking Peter, the word "pipe" may not mean quite something in
> Python but other concepts like chaining may be better.
> 
> The original use of the word I am used to was at the UNIX shell level where
> an in-core data structure called a pipe was used

To paraphrase Mark Twain, the reports of Unix' death have been greatly
exaggerated.

Unix (or at least its bastard offsprings Linux and OSX) is alive and
well and still has pipes. I use them every day.


> If you have an object with some method you can call on it (or in some cases
> a normal function call) that returns  another (or the same) object, then you
> can write code like:
> 
> This.that.other

Yeah, I thought you might be referring to that. I've just never seen the
term "pipeline" for that construct (I think "method chaining" is
reasonably common).

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Peter J. Holzer]

On 2022-10-10 09:23:27 +1100, Chris Angelico wrote:
> On Mon, 10 Oct 2022 at 06:50, Antoon Pardon  wrote:
> > I just want a parser that doesn't give up on encoutering the first syntax
> > error. Maybe do some semantic checking like checking the number of 
> > parameters.
> 
> That doesn't make sense though.

I think you disagree with most compiler authors here.

> It's one thing to keep going after finding a non-syntactic error, but
> an error of syntax *by definition* makes parsing the rest of the file
> dubious.

Dubious but still useful.

> What would it even *mean* to not give up?

Read the blog post on Lezer for some ideas:
https://marijnhaverbeke.nl/blog/lezer.html

This is in the context of an editor. But the same problem applies to
compilers. It's not very important if a compile run only takes a second
or so but even then it might be helpful to see several error messages
and not only one at a time. It becomes much more important as compile
times get longer (as an extreme[1] example, when I worked on a largeish
cobol program in the 1980s, compiling the thing took about half an hour.
I really wanted to fix *everything* before starting the compiler again.)

Marijn isn't the only person who revisited this problem recently[2].
I've read a few other blog posts and papers on that topic at about the
same time.

hp

[1] Yes, there are programs where a full compile takes much longer than
that. But you can usually get away with recompiling only a small
part, so you don't have to wait that long during normal development.
That cobol compiler couldn't do that.

[2] "Recently" means "in the last 10 years or so".

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Chris Angelico]

On Tue, 11 Oct 2022 at 06:34, Peter J. Holzer  wrote:
>
> On 2022-10-10 09:23:27 +1100, Chris Angelico wrote:
> > On Mon, 10 Oct 2022 at 06:50, Antoon Pardon  wrote:
> > > I just want a parser that doesn't give up on encoutering the first syntax
> > > error. Maybe do some semantic checking like checking the number of 
> > > parameters.
> >
> > That doesn't make sense though.
>
> I think you disagree with most compiler authors here.
>
> > It's one thing to keep going after finding a non-syntactic error, but
> > an error of syntax *by definition* makes parsing the rest of the file
> > dubious.
>
> Dubious but still useful.

There's a huge difference between non-fatal errors and syntactic
errors. The OP wants the parser to magically skip over a fundamental
syntactic error and still parse everything else correctly. That's
never going to work perfectly, and the OP is surprised at this.

> > What would it even *mean* to not give up?
>
> Read the blog post on Lezer for some ideas:
> https://marijnhaverbeke.nl/blog/lezer.html
>
> This is in the context of an editor.

Incidentally, that's actually where I would expect to see that kind of
feature show up the most - syntax highlighters will often be designed
to "carry on, somehow" after a syntax error, even though it often
won't make any sense (just look at what happens to your code
highlighting when you omit a quote character). It still won't always
be any use, but you do see *some* attempt at it.

But if the OP would be satisfied with that, I rather doubt that this
thread would even have happened. Unless, of course, the OP still lives
in the dark ages when no text editor available had any suitable
features for code highlighting.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: for -- else: what was the motivation?

[avi.e.gross]

I just want to get clear on what may be a side issue

The suggestion is that a loop with an ELSE clause only makes sense if the
user can BREAK out without finishing, right?

My first question is whether a warning or even error makes sense if there is
no BREAK statement anywhere in the loop but there is another way to break
out such as a RETURN statement or perhaps from an error induced that is not
caught and then propagates outward?

Or is there some mechanism that would still run the ELSE clause in such
cases?

The second is to note some break statements are potentially never executed
such as: if False: break

There are actually scenarios where you can remain in a loop to the end and
not do anything rather than break, but of course in that case, the ELSE
clause running makes little sense to only run if you finish normally unless
it can use a shared Boolean variable you used to skip the rest of the loop
and use that to decide. That can be done easily enough without the ELSE, I
would think.



-Original Message-
From: Python-list  On
Behalf Of Weatherby,Gerard
Sent: Monday, October 10, 2022 2:25 PM
To: Axy ; python-list@python.org
Subject: Re: for -- else: what was the motivation?

pylint, at least, provides a warning:

fe.py:4:0: W0120: Else clause on loop without a break statement
(useless-else-on-loop)


sum = 0
for i in range(5):
sum += i
else:
print("Always executes")
print(sum)


From: Python-list  on
behalf of Axy via Python-list 
Date: Monday, October 10, 2022 at 1:10 PM
To: python-list@python.org 
Subject: Re: for -- else: what was the motivation?
*** Attention: This is an external email. Use caution responding, opening
attachments or clicking on links. ***

> On 10/10/2022 15:52, Weatherby,Gerard wrote:
>> I wonder if for/else could have been less confusing if it was 
>> referred to as for-break-else and if the else clause was only valid 
>> syntax if the for loop actually contained a break statement in the 
>> first place.
>
> Sounds reasonable. It would be something alike UnboundLocalError when 
> a local variable referenced before assignment. If they won't remove 
> "else" completely in far future, that checking really worths 
> implementing now.

Actually, I think a warning would be sufficient, as in the following quick
prototype. If someone can implement this quickly in CPython, that would be
great (last time I hacked it, it was 2.4)

Axy.

import ast

tree = ast.parse('''
# sample code
a = 0
for i in 'asd':
 a += i
 while x:
 pass
 else:
 print('wow')
 break
 print(i)
else:
 print(0)
''', mode='exec')

def check_ast(node):
 if isinstance(node, (ast.For, ast.AsyncFor, ast.While)):
 if node.orelse and have_no_break(node.body):
 print(f'Warning: the loop at line {node.lineno} has no "break"
statement,'
   f' "else" clause at line {node.orelse[0].lineno} won\'t
run')
 else:
 for child in ast.iter_child_nodes(node):
 check_ast(child)

def have_no_break(loop_body):
 for node in loop_body:
 if isinstance(node, (ast.For, ast.AsyncFor, ast.While)):
 # nested loop
 check_ast(node)
 elif isinstance(node, ast.Break):
 return False
 elif isinstance(node, list):
 for child in ast.iter_child_nodes(node):
 if have_no_break(child) == False:
 return False
 return True


for node in tree.body:
 check_ast(node)
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-
list__;!!Cn_UX_p3!mlK4jRkfDC_akw-fIqWaMVf707GQsiyvj_sRHTsFnuG4ak5mKWwSavtz4n
jlBNIu1H0VHrR9gyjuQpxGqZ1dacU1Xw$
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Jon Ribbens via Python-list]

On 2022-10-10, Calvin Spealman  wrote:
> On Sat, Oct 8, 2022 at 5:35 PM rbowman  wrote:
>> On 10/7/22 21:32, Axy wrote:
>> > So, seriously, why they needed else if the following pieces produce same
>> > result? Does anyone know or remember their motivation?
>>
>> In real scenarios there would be more logic in the for block that would
>> meet a condition and break out of the loop. If the condition is never
>> met, the else block runs. To steal from w3schools:
>>
>>
>> fruits = ["apple", "peach", "cherry"]
>> for x in fruits:
>>print(x)
>>if x == "banana":
>>  break
>> else:
>>print("Yes we got no bananas")
>>
>
> I wonder if for/else could have been less confusing if it was referred to
> as for-break-else and if the else clause was only valid syntax if the for
> loop actually contained a break statement in the first place.

Watch out, I suggested that here some years ago and it was derided
as being an "arrogant and foolish" idea.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Robert Latest via Python-list]

Chris Angelico wrote:
> Yes, I'm aware that code readability becomes irrelevant for
> short-duration projects. Beside the point. I'm wondering how important
> it really is to have the shortest block first.

I usually put the most expected / frequent / not negated block first if the
whole if/else statement is not "too long". Sometimes whatever you want to do
becomes pointless if a certain conditions is not met, in which case I do an
early break or return and have no else block at all.

> Given that for-else is an excellent, if rarely-used, construct

I knew it existed but coming from C I never thought to exploit it. I know I
wrote loops like this:

found = None
while not found:
found = search(something)
if found:
break
if not found:
complain()

Need to look into using "else" in these cases.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Jach Feng]

Axy 在 2022年10月10日 星期一下午5:55:29 [UTC+8] 的信中寫道：
> On 09/10/2022 03:33, Jach Feng wrote: 
> > The else is always coming with the break, not the for. 
> However, the compiler does not complain. 
Sure, the compiler will not complain even in a IOCCC contest:-)

> > but the [for...else] is insane. 
> Not in Python. 
The confusion is always in human mind.

---Jach
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Robert Latest via Python-list]

Grant Edwards wrote:
> I've followed that advice for several decades. I find it much easier
> to read code that's organized that way -- particularly when the
> difference in block sizes is large (e.g. the first block is one line,
> and the second is a a hundred).

If any conditionally executed blocks is a hundred lines, I believe your code
needs refactoring. I know mine does. Either the long block should go into an
extra function, or you do a "fail and bail" (just learned that phrase).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Michael F. Stemper]

On 09/10/2022 10.49, Avi Gross wrote:

Anton

There likely are such programs out there but are there universal agreements
on how to figure out when a new safe zone of code starts where error
testing can begin?

For example a file full of function definitions might find an error in
function 1 and try to find the end of that function and resume checking the
next function.  But what if a function defines local functions within it?
What if the mistake in one line of code could still allow checking the next
line rather than skipping it all?

My guess is that finding 100 errors might turn out to be misleading. If you
fix just the first, many others would go away. If you spell a variable name
wrong when declaring it, a dozen uses of the right name may cause errors.
Should you fix the first or change all later ones?


How does one declare a variable in python? Sometimes it'd be nice to
be able to have declarations and any undeclared variable be flagged.

When I was writing F77 for a living, I'd (temporarily) put:
  IMPLICIT CHARACTER*3
at the beginning of a program or subroutine that I was modifying,
in order to have any typos flagged.

I'd love it if there was something similar that I could do in python.

--
Michael F. Stemper
87.3% of all statistics are made up by the person giving them.
--
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Robert Latest via Python-list]

 wrote:
> Cameron,
>
> Your suggestion makes me shudder!

Me, too

> Removing all earlier lines of code is often guaranteed to generate errors as
> variables you are using are not declared or initiated, modules are not
> imported and so on.

all of which aren't syntax errors, so the method should still work. Ugly as
hell though. I can't think of a reason to want to find multiple syntax errors
in a file.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Chris Angelico]

On Tue, 11 Oct 2022 at 08:55, Robert Latest via Python-list
 wrote:
>
> Chris Angelico wrote:
> > Yes, I'm aware that code readability becomes irrelevant for
> > short-duration projects. Beside the point. I'm wondering how important
> > it really is to have the shortest block first.
>
> I usually put the most expected / frequent / not negated block first if the
> whole if/else statement is not "too long". Sometimes whatever you want to do
> becomes pointless if a certain conditions is not met, in which case I do an
> early break or return and have no else block at all.
>
> > Given that for-else is an excellent, if rarely-used, construct
>
> I knew it existed but coming from C I never thought to exploit it. I know I
> wrote loops like this:
>
> found = None
> while not found:
> found = search(something)
> if found:
> break
> if not found:
> complain()
>
> Need to look into using "else" in these cases.

Yep, that's exactly what for-else is great at!

while True:
if search(something): break
else:
complain()

(although rather than a "while True", you're probably iterating over
things to search for, in some way)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Robert Latest via Python-list]

Axy wrote:
>> Also not really a justification for "shortest block first". Wanting
>> some elaboration on that. What's the value in it?
>
> Well, the value is productivity. No need to save puzzles "what this 
> hanging else belongs to?"

If you find yourself asking that question, the if-block is probably too long to
begin with.

> Code small things first and return early, same 
> as taking a test: do easy and quick things first and boring and 
> difficult ones later.

Yes, but in that case you have a very long indented "else" block, and at the
point where the unindent happens you are scratching your head again like
before. Better to immediately return or break and not to use any "else" block
at all.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Robert Latest via Python-list]

Michael F. Stemper wrote:
> How does one declare a variable in python? Sometimes it'd be nice to
> be able to have declarations and any undeclared variable be flagged.

To my knowledge, the closest to that is using __slots__ in class definitions.
Many a time have I assigned to misspelled class members until I discovered
__slots__.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Robert Latest via Python-list]

Antoon Pardon wrote:
> I would like a tool that tries to find as many syntax errors as possible 
> in a python file.

I'm puzzled as to when such a tool would be needed. How many syntax errors can
you realistically put into a single Python file before compiling it for the
first time?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Michael F. Stemper]

On 09/10/2022 15.02, Peter J. Holzer wrote:

On 2022-10-09 15:32:13 -0400, Avi Gross wrote:

and of course no pipelines.


Since you've now used that term repeatedly: What is a pipeline in
Python?


Could it be what's discussed starting on page 35 of this presentation?


--
Michael F. Stemper
Life's too important to take seriously.
--
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Cameron Simpson]

On 11Oct2022 08:02, Chris Angelico  wrote:

There's a huge difference between non-fatal errors and syntactic
errors. The OP wants the parser to magically skip over a fundamental
syntactic error and still parse everything else correctly. That's
never going to work perfectly, and the OP is surprised at this.


The OP is not surprised by this, and explicitly expressed awareness that 
resuming a parse had potential for "misparsing" further code.


I remain of the opinion that one could resume a parse at the next 
unindented line and get reasonable results a lot of the time.


In fact, I expect that one could resume tokenising at almost any line 
which didn't seem to be inside a string and often get reasonable 
results.


I grew up with C and Pascal compilers which would _happily_ produce many 
complaints, usually accurate, and all manner of syntactic errors. They 
didn't stop at the first syntax error.


All you need in principle is a parser which goes "report syntax error 
here, continue assuming ". For Python that might mean 
"pretend a missing final colon" or "close open brackets" etc, depending 
on the context. If you make conservative implied corrections you can get 
a reasonable continued parse, enough to find further syntax errors.


I remember the Pascal compiler in particular had a really good "you 
missed a semicolon _back there_" mode which was almost always correct, a 
nice boon when correcting mistakes.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Help, PyCharm fails to recognize my tab setting...See attached picture of the code.

[Kevin M. Wilson via Python-list]

C:\Users\kevin\PycharmProjects\Myfuturevalue\venv\Scripts\python.exe 
C:\Users\kevin\PycharmProjects\Myfuturevalue\FutureValueCal.py   File 
"C:\Users\kevin\PycharmProjects\Myfuturevalue\FutureValueCal.py", line 31    
elif (years > 50.0) or (years < 1.0) :    ^IndentationError: expected an 
indented block after 'if' statement on line 29
Process finished with exit code 1


Good sense makes one slow to anger, and it is his glory tooverlook an offense.

Proverbs 19:11

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Cameron Simpson]

On 09/10/2022 10.49, Avi Gross wrote:
My guess is that finding 100 errors might turn out to be misleading. 
If you

fix just the first, many others would go away. If you spell a variable name
wrong when declaring it, a dozen uses of the right name may cause errors.
Should you fix the first or change all later ones?


Just to this, these are semantic errors, not syntax errors. Linters do 
an ok job of spotting these. Antoon is after _syntax errors_.


On 10Oct2022 08:21, Michael F. Stemper  wrote:

How does one declare a variable in python? Sometimes it'd be nice to
be able to have declarations and any undeclared variable be flagged.


Linters do pretty well at this. They can trace names and their use 
compared to their first definition/assignment (often - there are of 
course some constructs which are correct but unclear to a static 
analysis - certainly one of my linters occasionally says "possible 
undefine use" to me because there may be a path to use before set). This 
is particularly handy for typos, which often make for "use before set" 
or "set and not used".



I'd love it if there was something similar that I could do in python.


Have you used any lint programmes? My "lint" script runs pyflakes and 
pylint.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Chris Angelico]

On Tue, 11 Oct 2022 at 09:18, Cameron Simpson  wrote:
>
> On 11Oct2022 08:02, Chris Angelico  wrote:
> >There's a huge difference between non-fatal errors and syntactic
> >errors. The OP wants the parser to magically skip over a fundamental
> >syntactic error and still parse everything else correctly. That's
> >never going to work perfectly, and the OP is surprised at this.
>
> The OP is not surprised by this, and explicitly expressed awareness that
> resuming a parse had potential for "misparsing" further code.
>
> I remain of the opinion that one could resume a parse at the next
> unindented line and get reasonable results a lot of the time.

The next line at the same indentation level as the line with the
error, or the next flush-left line? Either way, there's a weird and
arbitrary gap before you start parsing again, and you still have no
indication of what could make sense. Consider:

if condition # no colon
code
else:
code

To actually "restart" parsing, you have to make a guess of some sort.
Maybe you can figure out what the user meant to do, and parse
accordingly; but if that's the case, keep going immediately, don't
wait for an unindented line. If you want for a blank line followed by
an unindented line, that might help with a notion of "next logical
unit of code", but it's very much dependent on the coding style, and
if you have a codebase that's so full of syntax errors that you
actually want to see more than one, you probably don't have a codebase
with pristine and beautiful code layout.

> In fact, I expect that one could resume tokenising at almost any line
> which didn't seem to be inside a string and often get reasonable
> results.

"Seem to be"? On what basis?

> I grew up with C and Pascal compilers which would _happily_ produce many
> complaints, usually accurate, and all manner of syntactic errors. They
> didn't stop at the first syntax error.

Yes, because they work with a much simpler grammar. But even then,
most syntactic errors (again, this is not to be confused with semantic
errors - if you say "char *x = 1.234;" then there's no parsing
ambiguity but it's not going to compile) cause a fair degree of
nonsense afterwards.

The waters are a bit muddied by some things being called "syntax
errors" when they're actually nothing at all to do with the parser.
For instance:

>>> def f():
... await q
...
  File "", line 2
SyntaxError: 'await' outside async function

This is not what I'm talking about; there's no parsing ambiguity here,
and therefore no difficulty whatsoever in carrying on with the
parsing. You could ast.parse() this code without an error. But
resuming after a parsing error is fundamentally difficult, impossible
without guesswork.

> All you need in principle is a parser which goes "report syntax error
> here, continue assuming ". For Python that might mean
> "pretend a missing final colon" or "close open brackets" etc, depending
> on the context. If you make conservative implied corrections you can get
> a reasonable continued parse, enough to find further syntax errors.

And, more likely, you'll generate a lot of nonsense. Take something like this:

items = [
item[1],
item2],
item[3],
]

As a human, you can easily see what the problem is. Try teaching a
parser how to handle this. Most likely, you'll generate a spurious
error - maybe the indentation, maybe the intended end of the list -
but there's really only one error here. Reporting multiple errors
isn't actually going to be at all helpful.

> I remember the Pascal compiler in particular had a really good "you
> missed a semicolon _back there_" mode which was almost always correct, a
> nice boon when correcting mistakes.
>

Ahh yes. Design a language with strict syntactic requirements, and
it's not too hard to find where the programmer has omitted them. Thing
is Python just doesn't HAVE those semicolons. Let's say that a
variant Python required you to put a U+251C ├ at the start of every
statement, and U+2524 ┤ at the end of the statement. A whole lot of
classes of error would be extremely easy to notice and correct, and
thus you could resume parsing; but that isn't benefiting the
programmer any. When you don't have that kind of information
duplication, it's a lot harder to figure out how to cheat the fix and
go back to parsing.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Thomas Passin]

On 10/10/2022 9:21 AM, Michael F. Stemper wrote:

On 09/10/2022 10.49, Avi Gross wrote:

Anton

There likely are such programs out there but are there universal 
agreements

on how to figure out when a new safe zone of code starts where error
testing can begin?

For example a file full of function definitions might find an error in
function 1 and try to find the end of that function and resume 
checking the

next function.  But what if a function defines local functions within it?
What if the mistake in one line of code could still allow checking the 
next

line rather than skipping it all?

My guess is that finding 100 errors might turn out to be misleading. 
If you
fix just the first, many others would go away. If you spell a variable 
name

wrong when declaring it, a dozen uses of the right name may cause errors.
Should you fix the first or change all later ones?


How does one declare a variable in python? Sometimes it'd be nice to
be able to have declarations and any undeclared variable be flagged.

When I was writing F77 for a living, I'd (temporarily) put:
   IMPLICIT CHARACTER*3
at the beginning of a program or subroutine that I was modifying,
in order to have any typos flagged.

I'd love it if there was something similar that I could do in python.


The Leo editor (https://github.com/leo-editor/leo-editor) will notify 
you of undeclared variables (and some syntax errors) each time you save 
your (Python) file.


--
https://mail.python.org/mailman/listinfo/python-list

RE: What to use for finding as many syntax errors as possible.

[avi.e.gross]

Michael,

A reasonable question. Python lets you initialize variables but has no
explicit declarations. Languages differ and I juggle attributes of many in
my mind and am reacting to the original question NOT about whether and how
Python should report many possible errors all at once but how ANY language
can be expected to do this well. Many others do have a variable declaration
phase or an optional declaration or perhaps just a need to declare a
function prototype so it can be used by others even if the formal function
creation will happen later in the code.

But what I meant in a Python context was something like this:

Wronk = who cares # this should fail
...
If (Wronk > 5): ...
...
Wronger = Wronk + 1
...
X = minimum(Wronk, Wronger, 12)

The first line does not parse well so you have an error. But in any case as
the line makes no sense, Wronk is not initialized to anything. Later code
may use it  in various ways and some of those may be seen as errors for an
assortment of reasons, then at one point the code does provide a value for
Wronk and suddenly code beyond that has no seeming errors. The above
examples are not meant to be real but just give a taste that programs with
holes in them for any reason may not be consistent. The only relatively
guaranteed test for sanity has to start at the top and encounter no errors
or missing parts based on an anything such as I/O errors. 

And I suggest there are some things sort of declared in python such as:

Import numpy as np

Yes, that brings in code from a module if it works and initializes a
variable called np to sort of point at the module or it's namespace or
whatever, depending on the language. It is an assignment but also a way to
let the program know things. If the above is:

Import grumpy as np

Then what happens if the code tries to find a file named "grumpy" somewhere
and cannot locate it and this is considered a syntax error rather than a
run-time error for whatever reason? Can you continue when all kinds of
functionality is missing and code asking to make a np.array([1,2,3]) clearly
fails?

Many of us here are talking past each other.

Yes, it would be nice to get lots of info and arguably we may eventually
have machine-learning or AI programs a bit more like SPAM detectors that
look for patterns commonly found and try to fix your program from common
errors or at least do a temporary patch so they can continue searching for
more errors. This could result in the best case in guessing right every
time. If you allowed it to actually fix your code, it might be like people
who let their spelling be corrected and do not proofread properly and send
out something embarrassing or just plain wrong!

And it will compile or be interpreted without complaint albeit not do
exactly what it is supposed to!




-Original Message-
From: Python-list  On
Behalf Of Michael F. Stemper
Sent: Monday, October 10, 2022 9:22 AM
To: python-list@python.org
Subject: Re: What to use for finding as many syntax errors as possible.

On 09/10/2022 10.49, Avi Gross wrote:
> Anton
> 
> There likely are such programs out there but are there universal 
> agreements on how to figure out when a new safe zone of code starts 
> where error testing can begin?
> 
> For example a file full of function definitions might find an error in 
> function 1 and try to find the end of that function and resume 
> checking the next function.  But what if a function defines local
functions within it?
> What if the mistake in one line of code could still allow checking the 
> next line rather than skipping it all?
> 
> My guess is that finding 100 errors might turn out to be misleading. 
> If you fix just the first, many others would go away. If you spell a 
> variable name wrong when declaring it, a dozen uses of the right name may
cause errors.
> Should you fix the first or change all later ones?

How does one declare a variable in python? Sometimes it'd be nice to be able
to have declarations and any undeclared variable be flagged.

When I was writing F77 for a living, I'd (temporarily) put:
   IMPLICIT CHARACTER*3
at the beginning of a program or subroutine that I was modifying, in order
to have any typos flagged.

I'd love it if there was something similar that I could do in python.

--
Michael F. Stemper
87.3% of all statistics are made up by the person giving them.
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Chris Angelico]

On Tue, 11 Oct 2022 at 13:10,  wrote:
> If the above is:
>
> Import grumpy as np
>
> Then what happens if the code tries to find a file named "grumpy" somewhere
> and cannot locate it and this is considered a syntax error rather than a
> run-time error for whatever reason? Can you continue when all kinds of
> functionality is missing and code asking to make a np.array([1,2,3]) clearly
> fails?

That's not a syntax error. Syntax is VERY specific. It is an error in
Python to attempt to add 1 to "one", it is an error to attempt to look
up the upper() method on None, it is an error to try to use a local
variable you haven't assigned to yet, and it is an error to open a
file that doesn't exist. But not one of these is a *syntax* error.

Syntax errors are detected at the parsing stage, before any code gets
run.  The vast majority of syntax errors are grammar errors, where the
code doesn't align with the parseable text of a Python program.
(Non-grammatical parsing errors include using a "nonlocal" statement
with a name that isn't found in any surrounding scope, using "await"
in a non-async function, and attempting to import braces from the
future.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: What to use for finding as many syntax errors as possible.

[avi.e.gross]

Cameron, or OP if you prefer,

I think by now you have seen a suggestion that languages make choices and
highly structured ones can be easier to "recover" from errors and try to
continue than some with way more complex possibilities that look rather
unstructured.

What is the error in code like this?

A,b,c,d = 1,2,

Or is it an error at all?

Many languages have no concept of doing anything like the above and some
tolerate a trailing comma and some set anything not found to some form of
NULL or uninitialized and some ...

If you look at human language, some are fairly simple and some are way too
organized. But in a way it can make sense. Languages with gender will often
ask you to change the spelling and often how you pronounce things not only
based on whether a noun is male/female or even neuter but also insist you
change the form of verbs or adjectives and so on that in effect give
multiple signals that all have to line up to make a valid and understandable
sentence. Heck, in conversations, people can often leave out parts of  a
sentence such as whether you are talking about "I" or "you" or "she" or "we"
because the rest of the words in the sentence redundantly force only one
choice to be possible. 

So some such annoying grammars (in my opinion) are error
detection/correction codes in disguise. In days before microphones and
speakers, it was common to not hear people well, like on a stage a hundred
feet away with other ambient noises. Missing a word or two might still allow
you to get the point as other parts of the sentence did such redundancies.
Many languages have similar strictures letting you know multiple times if
something is singular or plural. And I think another reason was what I call
stranger detection. People who learn some vocabulary might still not speak
correctly and be identifiable as strangers, as in spies.

Do we need this in the modern age? Who knows! But it makes me prefer some
languages over others albeit other reasons may ...

With the internet today, we are used to expecting error correction to come
for free. Do you really need one of every 8 bits to be a parity bit, which
only catches may half of the errors, when the internals of your computer are
relatively error free and even the outside is protected by things like
various protocols used in making and examining packets and demanding some be
sent again if some checksum does not match? Tons of checking is built in so
at your level you rarely think about it. If you get a message, it usually is
either 99.% accurate, or you do not have it shown to you at all. I am
not talking about SPAM but about errors of transmission.

So my analogies are that if you want a very highly structured language that
can recover somewhat from errors, Python may not be it.

And over the years as features are added or modified, the structure tends to
get more complex. And R is not alone. Many surviving languages continue to
evolve and borrow from each other and any program that you run today that
could partially recover and produce pages of possible errors, may blow up
when new features are introduced.

And with UNICODE, the number of possible "errors" in what is placed in code
for languages like Julia that allow them in most places ...


-Original Message-
From: Python-list  On
Behalf Of Cameron Simpson
Sent: Monday, October 10, 2022 6:17 PM
To: python-list@python.org
Subject: Re: What to use for finding as many syntax errors as possible.

On 11Oct2022 08:02, Chris Angelico  wrote:
>There's a huge difference between non-fatal errors and syntactic 
>errors. The OP wants the parser to magically skip over a fundamental 
>syntactic error and still parse everything else correctly. That's never 
>going to work perfectly, and the OP is surprised at this.

The OP is not surprised by this, and explicitly expressed awareness that
resuming a parse had potential for "misparsing" further code.

I remain of the opinion that one could resume a parse at the next unindented
line and get reasonable results a lot of the time.

In fact, I expect that one could resume tokenising at almost any line which
didn't seem to be inside a string and often get reasonable results.

I grew up with C and Pascal compilers which would _happily_ produce many
complaints, usually accurate, and all manner of syntactic errors. They
didn't stop at the first syntax error.

All you need in principle is a parser which goes "report syntax error here,
continue assuming ". For Python that might mean "pretend a
missing final colon" or "close open brackets" etc, depending on the context.
If you make conservative implied corrections you can get a reasonable
continued parse, enough to find further syntax errors.

I remember the Pascal compiler in particular had a really good "you missed a
semicolon _back there_" mode which was almost always correct, a nice boon
when correcting mistakes.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.o

RE: What to use for finding as many syntax errors as possible.

[avi.e.gross]

I stand corrected Chris, and others, as I pay the sin tax.

Yes, there are many kinds of errors that logically fall into different
categories or phases of evaluation of a program and some can be determined
by a more static analysis almost on a line by line (or "statement" or
"expression", ...)  basis and others need to sort of simulate some things
and look back and forth to detect possible incompatibilities and yet others
can only be detected at run time and likely way more categories depending on
the language.

But when I run the Python interpreter on code, aren't many such phases done
interleaved and at once as various segments of code are parsed and examined
and perhaps compiled into block code and eventually executed? 

So is the OP asking for something other than a Python Interpreter that
normally halts after some kind of error? Tools like a linter may indeed fit
that mold. 

This may limit some of the objections of when an error makes it hard for the
parser to find some recovery point to continue from as no code is being run
and no harmful side effects happen by continuing just an analysis. 

Time to go read some books about modern ways to evaluate a language based on
more mathematical rules including more precisely what is syntax versus ...

Suggestions?

-Original Message-
From: Python-list  On
Behalf Of Chris Angelico
Sent: Monday, October 10, 2022 10:42 PM
To: python-list@python.org
Subject: Re: What to use for finding as many syntax errors as possible.

On Tue, 11 Oct 2022 at 13:10,  wrote:
> If the above is:
>
> Import grumpy as np
>
> Then what happens if the code tries to find a file named "grumpy" 
> somewhere and cannot locate it and this is considered a syntax error 
> rather than a run-time error for whatever reason? Can you continue 
> when all kinds of functionality is missing and code asking to make a 
> np.array([1,2,3]) clearly fails?

That's not a syntax error. Syntax is VERY specific. It is an error in Python
to attempt to add 1 to "one", it is an error to attempt to look up the
upper() method on None, it is an error to try to use a local variable you
haven't assigned to yet, and it is an error to open a file that doesn't
exist. But not one of these is a *syntax* error.

Syntax errors are detected at the parsing stage, before any code gets run.
The vast majority of syntax errors are grammar errors, where the code
doesn't align with the parseable text of a Python program.
(Non-grammatical parsing errors include using a "nonlocal" statement with a
name that isn't found in any surrounding scope, using "await"
in a non-async function, and attempting to import braces from the
future.)

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Chris Angelico]

On Tue, 11 Oct 2022 at 14:13,  wrote:
> With the internet today, we are used to expecting error correction to come
> for free. Do you really need one of every 8 bits to be a parity bit, which
> only catches may half of the errors...

Fortunately, we have WAY better schemes than simple parity, which was
only really a thing in the modem days. (Though I would say that
there's still a pretty clear distinction between a good message where
everything has correct parity, and line noise where half of them
don't.) Hamming codes can correct one-bit errors (and detect two-bit
errors) at a price of log2(size)+1 bits of space. Here's a great
rundown:

https://www.youtube.com/watch?v=X8jsijhllIA

There are other schemes too, but Hamming codes are beautifully elegant
and easy to understand.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Chris Angelico]

On Tue, 11 Oct 2022 at 14:26,  wrote:
>
> I stand corrected Chris, and others, as I pay the sin tax.
>
> Yes, there are many kinds of errors that logically fall into different
> categories or phases of evaluation of a program and some can be determined
> by a more static analysis almost on a line by line (or "statement" or
> "expression", ...)  basis and others need to sort of simulate some things
> and look back and forth to detect possible incompatibilities and yet others
> can only be detected at run time and likely way more categories depending on
> the language.
>
> But when I run the Python interpreter on code, aren't many such phases done
> interleaved and at once as various segments of code are parsed and examined
> and perhaps compiled into block code and eventually executed?

Hmm, depends what you mean. Broadly speaking, here's how it goes:

0) Early pre-parse steps that don't really matter to most programs,
like checking character set. We'll ignore these.
1) Tokenize the text of the program into a sequence of
potentially-meaningful units.
2) Parse those tokens into some sort of meaningful "sentence".
3) Compile the syntax tree into actual code.
4) Run that code.

Example:
>>> code = """def f():
... print("Hello, world", 1>=2)
... print(Ellipsis, ...)
... return True
... """
>>>

In step 1, all that happens is that a stream of characters (or bytes,
depending on your point of view) gets broken up into units.

>>> for t in tokenize.tokenize(iter(code.encode().split(b"\n")).__next__):
... print(tokenize.tok_name[t.exact_type], t.string)

It's pretty spammy, but you can see how the compiler sees the text.
Note that, at this stage, there's no real difference between the NAME
"def" and the NAME "print" - there are no language keywords yet.
Basically, all you're doing is figuring out punctuation and stuff.

Step 2 is what we'd normally consider "parsing". (It may well happen
concurrently and interleaved with tokenizing, and I'm giving a
simplified and conceptualized pipeline here, but this is broadly what
Python does.) This compares the stream of tokens to the grammar of a
Python program and attempts to figure out what it means. At this
point, the linear stream turns into a recursive syntax tree, but it's
still very abstract.

>>> import ast
>>> ast.dump(ast.parse(code))
"Module(body=[FunctionDef(name='f', args=arguments(posonlyargs=[],
args=[], kwonlyargs=[], kw_defaults=[], defaults=[]),
body=[Expr(value=Call(func=Name(id='print', ctx=Load()),
args=[Constant(value='Hello, world'), Compare(left=Constant(value=1),
ops=[GtE()], comparators=[Constant(value=2)])], keywords=[])),
Expr(value=Call(func=Name(id='print', ctx=Load()),
args=[Name(id='Ellipsis', ctx=Load()), Constant(value=Ellipsis)],
keywords=[])), Return(value=Constant(value=True))],
decorator_list=[])], type_ignores=[])"

(Side point: I would rather like to be able to
pprint.pprint(ast.parse(code)) but that isn't a thing, at least not
currently.)

This is where the vast majority of SyntaxErrors come from. Your code
is a sequence of tokens, but those tokens don't mean anything. It
doesn't make sense to say "print(def f[return)]" even though that'd
tokenize just fine. The trouble with the notion of "keeping going
after finding an error" is that, when you find an error, there are
almost always multiple possible ways that this COULD have been
interpreted differently. It's as likely to give nonsense results as
actually useful ones.

(Note that, in contrast to the tokenization stage, this version
distinguishes between the different types of word. The "def" has
resulted in a FunctionDef node, the "print" is a Name lookup, and both
"..." and "True" have now become Constant nodes - previously, "..."
was a special Ellipsis token, but "True" was just a NAME.)

Step 3: the abstract syntax tree gets parsed into actual runnable
code. This is where that small handful of other SyntaxErrors come
from. With these errors, you absolutely _could_ carry on and report
multiple; but it's not very likely that there'll actually *be* more
than one of them in a file. Here's some perfectly valid AST parsing:

>>> ast.dump(ast.parse("from __future__ import the_past"))
"Module(body=[ImportFrom(module='__future__',
names=[alias(name='the_past')], level=0)], type_ignores=[])"
>>> ast.dump(ast.parse("from __future__ import braces"))
"Module(body=[ImportFrom(module='__future__',
names=[alias(name='braces')], level=0)], type_ignores=[])"
>>> ast.dump(ast.parse("def f():\n\tdef g():\n\t\tnonlocal x\n"))
"Module(body=[FunctionDef(name='f', args=arguments(posonlyargs=[],
args=[], kwonlyargs=[], kw_defaults=[], defaults=[]),
body=[FunctionDef(name='g', args=arguments(posonlyargs=[], args=[],
kwonlyargs=[], kw_defaults=[], defaults=[]),
body=[Nonlocal(names=['x'])], decorator_list=[])],
decorator_list=[])], type_ignores=[])"

If you were to try to actually compile those to bytecode, they would fail:

>>> compile(ast.parse("from __future__ import braces"), "-", "exec")

What to use for finding as many syntax errors as possible.

[avi.e.gross]

I think we are in agreement here, Chris. My point is that the error
detection and correction is now done at levels where there is not much need
to use earlier and inefficient methods like parity bits set aside. We use
protocols like TCP and IP and layers above them and above those to maintain
the integrity of packets and sessions and forms of encryption allowing
things like authentication. There is tons of overhead, even when some is
fairly efficient, but we hardly notice it unless things go wrong.

So written language sent (as in this email/post) does not need lots of
redundancy and all the extra effort is, IMNSHO opinion, largely wasted. If I
see a bear, I do not wish to check their genitals or DNA to determine their
irrelevant gender before asking someone to run from it. If I happen to know
the gender, as in a zoo, gender only matters for things like breeding
purposes. I do not want to memorize terms in languages that have not only
words like lion and lioness or duck and drake and goose and gander, but for
EVERYTHING in some sense so I can say the equivalent of ANIMAL-male and
ANIMAL-female with unique words. Life would be so much simpler if I could
say your dog was nice and not be corrected that it was a bitch and I used
the wrong word endings. If I really wanted to say it was a female dog, well
I could just add a qualified. Most of the time, who cares?

The same applies to so much grammatical nonsense which is also usually
riddled with endless exceptions to the many rules. Make the languages simple
with little redundancy and thus far easier to learn.

I can say similar things about some programming languages that either have
way too many rules or too few of the right ones.

There are tradeoffs and if you want a powerful language it will likely not
be easy to control. If you want a very regulated language, you may find it
not very useful as many things are hard to do ad others not possible. I know
that strongly typed languages often have to allow some method of cheating
such as unions of data types, or using a parent class as the sort of
object-type to allow disparate objects to live together. Python is far from
the most complex but as noted, it is not trivial to evaluate even the syntax
past errors.

But I admit it is fun and a challenge to learn both kinds and I spent much
of my time doing so. I like the flexibility of seeing different approaches
and holding contradictions in my mind while accepting both and yet neither!
LOL!


-Original Message-
From: Python-list  On
Behalf Of Chris Angelico
Sent: Monday, October 10, 2022 11:24 PM
To: python-list@python.org
Subject: Re: What to use for finding as many syntax errors as possible.

On Tue, 11 Oct 2022 at 14:13,  wrote:
> With the internet today, we are used to expecting error correction to 
> come for free. Do you really need one of every 8 bits to be a parity 
> bit, which only catches may half of the errors...

Fortunately, we have WAY better schemes than simple parity, which was only
really a thing in the modem days. (Though I would say that there's still a
pretty clear distinction between a good message where everything has correct
parity, and line noise where half of them
don't.) Hamming codes can correct one-bit errors (and detect two-bit
errors) at a price of log2(size)+1 bits of space. Here's a great
rundown:

https://www.youtube.com/watch?v=X8jsijhllIA

There are other schemes too, but Hamming codes are beautifully elegant and
easy to understand.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list