Re: [Python-Dev] Is 4.0 a major breaking changes release?

[Greg Ewing]

On Sat, Feb 3, 2018 at 10:46 PM, Ned Deily > wrote:

 I've seen comments about making somewhat major changes in 4.0 -

now I'm

 concerned that I should pause my porting effort until that is

released.  Is

 python 4 going to be another python 3?


Guido has repeatedly promised that there will never be another
upheaval as big as the 2-to-3 one, and that the change from
Python 3.9 to 4.0 won't be anything special. Hopefully we can
trust him on that.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is 4.0 a major breaking changes release?

[Steve Holden]

On Sat, Feb 3, 2018 at 10:46 PM, Ned Deily  wrote:

> On Feb 3, 2018, at 17:40, Alex Walters  wrote:
> > I am still working on porting code from 2.x to 3.x.  As of late on the
> lists
> > I've seen comments about making somewhat major changes in 4.0 - now I'm
> > concerned that I should pause my porting effort until that is released.
> Is
> > python 4 going to be another python 3?
>
> At this point, Python 4 is just a convenient tag for really big changes.
> There are no concrete plans or schedule for such a major undertaking.  Port
> away to Python 3.x!
>
> ​"Py3K?" they said, "oh, no, we're just noodling around with a few ideas
..."​ :-)

To be honest, and historically fair to Guido, he did warn for a long time
that we should expect breaking changes in an eventual wart-removal release.

It seemed to me the biggest disappointment was the team not having the
resources to devote to a mooted but never really achieved reorganisation of
the stdlib.

​Rectifying that omission would, I hope, be included as a priority in any
Python 4 design. Since people rely on the stdlib hugely, automated
translation of at least 98% of existing stdlib imports should be a goal.
But that's just  me.
​
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Steve Holden]

As a Bear of Relatively Little Brain, I've grown up understanding, and
teaching, that mutable things aren't to be used as dict keys. I'm aware
that immutability isn't strictly the required condition, but it for most
people, that's the primary reason for using frozen sets and tuples, for
example, and immutability serves as a practical and comprehensible first
approximation. So I'm at a loss to understand why I am being offered a
feature that (especially during maintenance by a different developer) might
be prone to bizarre errors caused by a change in hash.

I realise that this won't happen very often, but the difficulty of the
debug task should surely merit at least some warning for us bears - you
know, the ones that take your work and use it to do mundane things with.

On a slightly tangential note, us bears are very glad that such questions
are taken seriously and discussed in such depth. Thank you all.

Steve Holden

On Sat, Feb 3, 2018 at 6:44 AM, Guido van Rossum  wrote:

> It appears Eric and I are the only ones in favor of keeping the current
> behavior. But I still am not convinced by all the worries about "attractive
> nuisances" and all the other bad names this feature has been called. We
> don't know that any of the doomsday scenarios will happen. In my
> experience, usually once something is rolled out the set of issues that are
> *actually* raised is entirely different from the things its designers were
> worried about.
>
> Please don't commit a change to roll this back without checking in with
> me; I have some misgivings about the problem being raised here that I still
> need to think through more carefully. In the meantime, please try to use
> dataclasses with 3.7b1!
>
> On Fri, Feb 2, 2018 at 10:25 PM, Nick Coghlan  wrote:
>
>>
>>
>> On 3 Feb. 2018 1:09 am, "Eric V. Smith"  wrote:
>>
>>
>> The problem with dropping hash=True is: how would you write __hash__
>> yourself? It seems like a bug magnet if you're adding fields to the class
>> and forget to update __hash__, especially in the presence of per-field
>> hash=False and eq=False settings. And you'd need to make sure it matches
>> the generated __eq__ (if 2 objects are equal, they need to have the same
>> hash value).
>>
>>
>> I think anyone that does this needs to think *very* carefully about how
>> they do it, and offering both "hash=True" and "frozen=True" is an
>> attractive nuisance that means people will write "hash=True" when what they
>> wanted was "frozen=True".
>>
>> In particular, having to work out how write a maintainable "__hash__"
>> will encourage folks to separate out the hashed fields as a separate frozen
>> subrecord or base class.
>>
>> If this proves to be an intolerable burden then the short hand spelling
>> could be added back in 3.8, but once we ship it we're going to be stuck
>> with explaining the interactions indefinitely.
>>
>> Cheers,
>> Nick.
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is 4.0 a major breaking changes release?

[Terry Reedy]

On 2/3/2018 5:50 PM, Zachary Ware wrote:

On Sat, Feb 3, 2018 at 4:40 PM, Alex Walters  wrote:

I am still working on porting code from 2.x to 3.x.  As of late on the lists
I've seen comments about making somewhat major changes in 4.0 - now I'm
concerned that I should pause my porting effort until that is released.  Is
python 4 going to be another python 3?


Emphatically no.  Anyone suggesting big breaking changes in 4.0 is
exercising wishful thinking :)

There may be some cleanup in 4.0, but only removing things that have
been deprecated for a long time in 3.x but hadn't been removed to
maintain compatibility with 2.7.


So do your porting with deprecation warnings on.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is 4.0 a major breaking changes release?

[Dan Stromberg]

On Sat, Feb 3, 2018 at 2:40 PM, Alex Walters  wrote:
> I am still working on porting code from 2.x to 3.x.  As of late on the lists
> I've seen comments about making somewhat major changes in 4.0 - now I'm
> concerned that I should pause my porting effort until that is released.  Is
> python 4 going to be another python 3?

https://www.curiousefficiency.org/posts/2014/08/python-4000.html

Python 0.x to 1.x was a small change.
Python 1.x to 2.x was a small change.

I doubt there'll be anything as important as str -> (bytes, unicode)
to merit a breaking change.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is 4.0 a major breaking changes release?

[Paul Moore]

On 3 February 2018 at 22:40, Alex Walters  wrote:
> I am still working on porting code from 2.x to 3.x.  As of late on the lists
> I've seen comments about making somewhat major changes in 4.0 - now I'm
> concerned that I should pause my porting effort until that is released.  Is
> python 4 going to be another python 3?

No. Guido has gone on record as saying this won't happen.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is 4.0 a major breaking changes release?

[Zachary Ware]

On Sat, Feb 3, 2018 at 4:40 PM, Alex Walters  wrote:
> I am still working on porting code from 2.x to 3.x.  As of late on the lists
> I've seen comments about making somewhat major changes in 4.0 - now I'm
> concerned that I should pause my porting effort until that is released.  Is
> python 4 going to be another python 3?

Emphatically no.  Anyone suggesting big breaking changes in 4.0 is
exercising wishful thinking :)

There may be some cleanup in 4.0, but only removing things that have
been deprecated for a long time in 3.x but hadn't been removed to
maintain compatibility with 2.7.

-- 
Zach
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is 4.0 a major breaking changes release?

[Ned Deily]

On Feb 3, 2018, at 17:40, Alex Walters  wrote:
> I am still working on porting code from 2.x to 3.x.  As of late on the lists
> I've seen comments about making somewhat major changes in 4.0 - now I'm
> concerned that I should pause my porting effort until that is released.  Is
> python 4 going to be another python 3?

At this point, Python 4 is just a convenient tag for really big changes.  There 
are no concrete plans or schedule for such a major undertaking.  Port away to 
Python 3.x!

--
  Ned Deily
  n...@python.org -- []

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Is 4.0 a major breaking changes release?

[Alex Walters]

I am still working on porting code from 2.x to 3.x.  As of late on the lists
I've seen comments about making somewhat major changes in 4.0 - now I'm
concerned that I should pause my porting effort until that is released.  Is
python 4 going to be another python 3?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Ethan Furman]

On 02/02/2018 10:44 PM, Guido van Rossum wrote:


It appears Eric and I are the only ones in favor of keeping the current 
behavior. But I still am not convinced by all
the worries about "attractive nuisances" and all the other bad names this 
feature has been called. We don't know that
any of the doomsday scenarios will happen. In my experience, usually once 
something is rolled out the set of issues that
are *actually* raised is entirely different from the things its designers were 
worried about.


This may all be true, but consider how many times we have asked, "How does attrs 
handle that?"

It would be wise to also ask, "What pitfalls have attrs discovered, and what would 
they do different if they could?"

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "threading.Lock().locked()" is not documented

[Benjamin Peterson]

On Sat, Feb 3, 2018, at 11:25, Gregory P. Smith wrote:
> On Wed, Jan 31, 2018 at 4:46 PM Jesus Cea  wrote:
> 
> > https://docs.python.org/3.6/library/threading.html doesn't document
> > "threading.Lock().locked()", and it is something quite useful.
> >
> > In fact, it is used in "threading.py" itself. For instance, lines 109,
> > 985, 1289.
> >
> > Is there any reason to not document it?.
> >
> 
> No good reason.  It is a public method and part of the public API
> regardless of being documented or not.

While I agree about .locked's present publicness, use of locked predicates in 
concurrent programs is usually serious code smell. So, the documentation should 
come with a warning note.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Gregory P. Smith]

On Fri, Feb 2, 2018 at 10:25 PM Nick Coghlan  wrote:

>
>
> On 3 Feb. 2018 1:09 am, "Eric V. Smith"  wrote:
>
>
> The problem with dropping hash=True is: how would you write __hash__
> yourself? It seems like a bug magnet if you're adding fields to the class
> and forget to update __hash__, especially in the presence of per-field
> hash=False and eq=False settings. And you'd need to make sure it matches
> the generated __eq__ (if 2 objects are equal, they need to have the same
> hash value).
>
>
> I think anyone that does this needs to think *very* carefully about how
> they do it, and offering both "hash=True" and "frozen=True" is an
> attractive nuisance that means people will write "hash=True" when what they
> wanted was "frozen=True".
>

> In particular, having to work out how write a maintainable "__hash__" will
> encourage folks to separate out the hashed fields as a separate frozen
> subrecord or base class.
>
> If this proves to be an intolerable burden then the short hand spelling
> could be added back in 3.8, but once we ship it we're going to be stuck
> with explaining the interactions indefinitely.
>

+1 Nick put words to my chief concerns.

It is easy for an author see hash=True in existing code somewhere (cargo
culting) and assume it does what they want, or quickly glance at the the
API and see "hash=True" without actually taking the time to understand the
implications of that to see that the parameter named "frozen" is the one
they are supposed to want that _safely_ makes their dataclass properly
hashable, not the more attractive parameter named "hash" that enables
dangerous behavior.

Forcing people who need a __hash__ method to write it explicitly sounds
like a good thing to me.  I am not at all worried about someone forgetting
to add a new field to an implementation of the __hash__ method when adding
a new field, the fields and __hash__ method are all defined in the same
place in the code.

I expect someone with a common need for always having a __hash__ method
will produce a library on top of dataclasses that implements something like
our current hash=True behavior.  If that kind of thing turns out to be
widely used, we can reintroduce the feature in dataclasses in 3.8 or later,
informed by what we see practical uses actually doing.

In my practical experience, people writing Python code do not need to learn
and understand the intricacies of what it means to have a __hash__ method,
be hashable, or "frozen".  We intentionally warn people against writing
dunder methods other than __init__ in their code as they are often power
features with less obvious semantics than it may seem at first glance
making such code harder to maintain.

Even calling the parameter "hash=" and saying it adds a __hash__ method as
the PEP currently does seems to launder the danger, washing away the
"dunder smell" that adding a special considerations __hash__ method carries.

The PEP (and presumably forthcoming dataclasses module documentation) says
"This is a specialized use case and should be considered carefully" which I
agree with.  But any time we suggest that in an API, how about having the
API name make it clear that this is special and not to be done lightly?  I
guess i'm arguing against using "hash=" as the arg name in favor of
"danger_there_be_vorpal_rabbits_hash_me_maybe=" or something more usefully
similar if we're going to have it.

-gps
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "threading.Lock().locked()" is not documented

[Gregory P. Smith]

On Wed, Jan 31, 2018 at 4:46 PM Jesus Cea  wrote:

> https://docs.python.org/3.6/library/threading.html doesn't document
> "threading.Lock().locked()", and it is something quite useful.
>
> In fact, it is used in "threading.py" itself. For instance, lines 109,
> 985, 1289.
>
> Is there any reason to not document it?.
>

No good reason.  It is a public method and part of the public API
regardless of being documented or not.  Always feel free to make a
documentation update PR.

The method does have a docstring.
  https://github.com/python/cpython/blob/master/Modules/_threadmodule.c#L193

-gps


>
> (I didn't investigate other objects in the module).
>
> --
> Jesús Cea Avión _/_/  _/_/_/_/_/_/
> j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
> Twitter: @jcea_/_/_/_/  _/_/_/_/_/
> jabber / xmpp:j...@jabber.org  _/_/  _/_/_/_/  _/_/  _/_/
> "Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
> "My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
> "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate crypt module and revert PR 3854

[Guido van Rossum]

You can deprecate the crypt module, update the docs to explain it's
outdated and point to other 3rd party solutions. A few years from now we
may be able to delete it. (With an intermediate step of issuing a
non-silent deprecation warning.) Until then just leave it be. Possibly
that's what your desired outcome is anyway?

On Sat, Feb 3, 2018 at 4:07 AM, Christian Heimes 
wrote:

> On 2018-02-02 17:18, Guido van Rossum wrote:
> > I'm all for nudging people in the direction of xcrypt. I assume we can't
> > just switch the C-level crypt with xcrypt and leave the Python API
> > unchanged?
> >
> > However until a usable solution exist (either in the stdlib or as 3rd
> > party) I don't think we should deprecate anything (deprecating things
> > before the replacement is ready is stressful for everyone involved).
> >
> > I'm also not sure I agree with removing support for old hashes. By all
> > means put in the docs that they are unsafe. But if someone has a
> > database full of old hashes it would be nice to be able to at least
> > read/verify it, right?
> >
> > Was a release already made with blowfish, extended DES and NT-Hash? (And
> > what's so bad with blowfish? It's mentioned in the heading of the xcrypt
> > project too.)
>
> I answered some of your questions in other replies and will answer the
> remaining concerns on Monday.
>
> You suggested a 3rd party module. I have cloned the crypt module with
> Serhiy's improvements and turned it into a standalone module with a
> ctypes interface, https://github.com/tiran/legacycrypt . I'll release
> the package as soon as I find time to polish the documentation and give
> Serhiy his will deserved credit for his work.
>
> Christian
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

[asrp]

Oh and I forgot to mention that since everything is mostly done at run time, 
there could just be different modules for (slightly) different languages. I 
don't know if that'd be problematic for maintenance though.
asrp

> Sent: Saturday, February 03, 2018 at 3:17 PM
> From: asrp 
> To: "Christian Tismer" 
> Cc: python-dev@python.org
> Subject: Re: [Python-Dev] A minimal Python interpreter written in Python for 
> experimenting with language changes
>
> Hi,
> 
> (My other reply to Alex and Steven doesn't seems to appear, at least not on 
> pipermail so I hope these don't come in the wrong order.)
> 
> > In the readme to python_terp you say:
> > 
> > """
> > python_terp is intended to make language modification to Python easier
> > to preview changes more quickly and is not intended for full CPython
> > compatibility. However, a large subset of Python is already included. In
> > particular, enough to run the first stage of its parser.
> > """
> > 
> > This needs clarification.
> > What do you mean by subset? A real subset or also things that
> > are different and will stay different?
> > To what extent are you planning to stay compatible, and where
> > do you plan to deviate?
> > 
> > The reason that I'm asking is that by compatible I mean the
> > compatibility of PyPy. If you can reach that, and be it just
> > by a subset, then it makes sense to speak of Python.
> > 
> 
> Good questions! There are two part to what I meant by that passage.
> 
> One is a reflection of the current state of the interpreter and its 
> capabilities. I put an example instead listing all differences because its 
> easier to update but admittedly much less precise.
> 
> The other concerns the intention for the future. My mind is not completely 
> made but here's what I currently think it should/could be.
> 
> First, fidelity would be determined by how the interpreter is used (if its 
> used). More (potentially) drastic changes in the beginning than later on. 
> Hopefully the deviations from Python (if any) for this reason does not grow 
> too much from this. But since its for testing language changes (to Python), I 
> don't think it would stray too much, if any.
> 
> Second, I'd like to keep the whole thing small (of low complexity) as much as 
> possible so in some cases, the less featureful choice will be kept. For 
> example, in an earlier version, I had more functions parameters handling 
> (defaults, *args, **kwargs, etc) but remove it because just setting up the 
> initial local scope from the parameters passed and function definition took 
> more than 100 lines. (Maybe I just went about this the wrong way...)
> 
> However, except for easier or simpler implementation of the interpreter or 
> some of the initial AST nodes, I don't intend to make any additions not in 
> the Python language. One example of this is simple_for and single_if 
> statements which I explain a bit in this old post:
> 
> http://blog.asrpo.com/bootstrap_chicken_or_egg
> 
> Although those are mainly intended to bootstrap the other statements like 
> if_stmt and for_stmt and not used again later.
> 
> (Well, there might be some difference for class method lookup (late-binding 
> instead of early-binding) that's needed for edit-and-continue. But the 
> intention would be to list all of those differences.)
> 
> Feel free to ask again if I missed some aspect of your questions here.
> 
> asrp
> 
> > Sent: Saturday, February 03, 2018 at 12:29 PM
> > From: "Christian Tismer" 
> > To: "asrp asrp" , python-dev@python.org
> > Subject: Re: [Python-Dev] A minimal Python interpreter written in Python 
> > for experimenting with language changes
> >
> > Hi user with no real name yet,
> > 
> > On 03.02.18 01:01, asrp asrp wrote:
> > > Hello,
> > > 
> > > I don't know if this is the right place to post this. Please redirect as 
> > > needed.
> > > 
> > > I've made a small Python interpreter in Python with runtime AST node 
> > > semantics and edit-and-continue. I thought it could make prototyping 
> > > language changes more easily and visualize usage before writing them in C.
> > > 
> > > Its here: https://github.com/asrp/python_terp
> > > 
> > > So, for example, redefining the for_stmt function in the right scope 
> > > changes the behaviour of future for loops at runtime.
> > > 
> > > Although from discussion I've read in PEPs, actual implementation always 
> > > look like a non-issue (which seems like magic to me) so maybe no-one here 
> > > actually needs this.
> > > 
> > > (I really needed edit-and-continue for one of my projects but of course, 
> > > running it in this extra interpreter is much too slow.)
> > > 
> > > asrp
> > 
> > In the readme to python_terp you say:
> > 
> > """
> > python_terp is intended to make language modification to Python easier
> > to preview changes more quickly and is not intended for full CPython
> > compatibility. However, a large subset of Python is 

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

[asrp]

> Are you aware of pypy?
> 

Yes, but I do not know it well. It seems to be more focused on JIT compiling 
with many of its functions written for special cases and I did not find a way 
to make experimental changes easily.

I even tried (briefly) to have the main loop of this interpreter run in RPython 
but didn't find enough information on which subset of Python does RPython 
handle.

> Can you give an example of how you would do that? I don't mean the 
> mechanism used, I mean how would a developer implement a new syntactic 
> feature. Suppose I wanted to add a new clause to for...else, let's say:
>
> for ... :
>block
> otherwise:
># runs only if the for-loop was empty
> 
> How would do I do that?
> 

Here's one way to do that. Sorry for the lack of doc for the moment.

To alter the language's grammar, you'd create a string with the new grammar, 
parse it and set the result to pyterp.parser. Then, depending on how the 
grammar was changed (which affects what AST is created), you'd change the 
corresponding function to handle the new semantics.

I didn't quite understand what you mean by "was empty". However, I noticed that 
"else" blocks after for are ignored in the current implement. Here's an example 
session adding "else".

If you tell me a bit more about the intended behaviour of "otherwise", I'd be 
happy to do an example with that clause.

$ ipython -i test/python_repl.py 
p>> simport simple_ast
p>> for i in [1, 2]:
... print i
... 
1
2
p>> for i in [1, 2, 3]:
... print i
... else:
... print 100
... 
1
2
3
p>> ^D

In the above for loop, the else block is ignored! I press Control-D to exist 
the interpreter.

In [1]: grammar = python_grammar.full_definition + python_grammar.extra

In [2]: grammar += r"""
   ...: for_stmt = "for" {exprlist} "in" {testlist} ":" {suite} 
{(SAME_INDENT "else" ":" {suite}) | void=pass_stmt}
   ...: """

In [3]: pyterp.parser = python.Interpreter(i3.parse("grammar", grammar))

In [4]: pyterp.repl()

Now edit for_stmt in simple_ast.py

 except StopIteration:
+evaluate(else_block)
 return

(If else_block was not already an (ignored) parameter, we'd have to change the 
`def for_stmt...` line too.) And run in the REPL we just restarted.

p>> simple_ast.reload_module(simple_ast)
p>> for i in [1, 2, 3]:
... print i
... else:
... print 100
... 
1
2
3
100
p>> for i in [1, 2, 3]:
... print i
... break
... else:
... print 100
... 
1
p>> 



Some notes:

1. I'm using the host Python interpreter to change the grammar here but that's 
not strictly necessary if we expose pyterp itself in the global scope.
2. I'm editing simple_ast.py and reloading because its less changes but 
redefining a new function and setting simple_ast.for_stmt (in other words, 
monkey-patching the change) should also work.

If we wanted an "otherwise" clause on top of an "else" clause, we probably want 
to make some provision for the runtime to distinguish between the two so I'd 
edit the grammar with something like

for_stmt = "for" {exprlist} "in" {testlist} ":" {suite} {((SAME_INDENT 
"else" ":" {suite}) | void=pass_stmt)=else_block ((SAME_INDENT "otherwise" ":" 
{suite}) | void=pass_stmt)=otherwise_block}

(In this case, "otherwise" blocks have to come after the "else" block if both 
are present.)

asrp
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

[asrp]

Hi,

(My other reply to Alex and Steven doesn't seems to appear, at least not on 
pipermail so I hope these don't come in the wrong order.)

> In the readme to python_terp you say:
> 
> """
> python_terp is intended to make language modification to Python easier
> to preview changes more quickly and is not intended for full CPython
> compatibility. However, a large subset of Python is already included. In
> particular, enough to run the first stage of its parser.
> """
> 
> This needs clarification.
> What do you mean by subset? A real subset or also things that
> are different and will stay different?
> To what extent are you planning to stay compatible, and where
> do you plan to deviate?
> 
> The reason that I'm asking is that by compatible I mean the
> compatibility of PyPy. If you can reach that, and be it just
> by a subset, then it makes sense to speak of Python.
> 

Good questions! There are two part to what I meant by that passage.

One is a reflection of the current state of the interpreter and its 
capabilities. I put an example instead listing all differences because its 
easier to update but admittedly much less precise.

The other concerns the intention for the future. My mind is not completely made 
but here's what I currently think it should/could be.

First, fidelity would be determined by how the interpreter is used (if its 
used). More (potentially) drastic changes in the beginning than later on. 
Hopefully the deviations from Python (if any) for this reason does not grow too 
much from this. But since its for testing language changes (to Python), I don't 
think it would stray too much, if any.

Second, I'd like to keep the whole thing small (of low complexity) as much as 
possible so in some cases, the less featureful choice will be kept. For 
example, in an earlier version, I had more functions parameters handling 
(defaults, *args, **kwargs, etc) but remove it because just setting up the 
initial local scope from the parameters passed and function definition took 
more than 100 lines. (Maybe I just went about this the wrong way...)

However, except for easier or simpler implementation of the interpreter or some 
of the initial AST nodes, I don't intend to make any additions not in the 
Python language. One example of this is simple_for and single_if statements 
which I explain a bit in this old post:

http://blog.asrpo.com/bootstrap_chicken_or_egg

Although those are mainly intended to bootstrap the other statements like 
if_stmt and for_stmt and not used again later.

(Well, there might be some difference for class method lookup (late-binding 
instead of early-binding) that's needed for edit-and-continue. But the 
intention would be to list all of those differences.)

Feel free to ask again if I missed some aspect of your questions here.

asrp

> Sent: Saturday, February 03, 2018 at 12:29 PM
> From: "Christian Tismer" 
> To: "asrp asrp" , python-dev@python.org
> Subject: Re: [Python-Dev] A minimal Python interpreter written in Python for 
> experimenting with language changes
>
> Hi user with no real name yet,
> 
> On 03.02.18 01:01, asrp asrp wrote:
> > Hello,
> > 
> > I don't know if this is the right place to post this. Please redirect as 
> > needed.
> > 
> > I've made a small Python interpreter in Python with runtime AST node 
> > semantics and edit-and-continue. I thought it could make prototyping 
> > language changes more easily and visualize usage before writing them in C.
> > 
> > Its here: https://github.com/asrp/python_terp
> > 
> > So, for example, redefining the for_stmt function in the right scope 
> > changes the behaviour of future for loops at runtime.
> > 
> > Although from discussion I've read in PEPs, actual implementation always 
> > look like a non-issue (which seems like magic to me) so maybe no-one here 
> > actually needs this.
> > 
> > (I really needed edit-and-continue for one of my projects but of course, 
> > running it in this extra interpreter is much too slow.)
> > 
> > asrp
> 
> In the readme to python_terp you say:
> 
> """
> python_terp is intended to make language modification to Python easier
> to preview changes more quickly and is not intended for full CPython
> compatibility. However, a large subset of Python is already included. In
> particular, enough to run the first stage of its parser.
> """
> 
> This needs clarification.
> What do you mean by subset? A real subset or also things that
> are different and will stay different?
> To what extent are you planning to stay compatible, and where
> do you plan to deviate?
> 
> The reason that I'm asking is that by compatible I mean the
> compatibility of PyPy. If you can reach that, and be it just
> by a subset, then it makes sense to speak of Python.
> 
> Cheers - Chris
> 
> -- 
> Christian Tismer-Sperling:^)   tis...@stackless.com
> Software Consulting  : http://www.stackless.com/
> Karl-Liebknecht-Str. 121 : 

Re: [Python-Dev] Deprecate crypt module and revert PR 3854

[Christian Heimes]

On 2018-02-02 17:18, Guido van Rossum wrote:
> I'm all for nudging people in the direction of xcrypt. I assume we can't
> just switch the C-level crypt with xcrypt and leave the Python API
> unchanged?
> 
> However until a usable solution exist (either in the stdlib or as 3rd
> party) I don't think we should deprecate anything (deprecating things
> before the replacement is ready is stressful for everyone involved).
> 
> I'm also not sure I agree with removing support for old hashes. By all
> means put in the docs that they are unsafe. But if someone has a
> database full of old hashes it would be nice to be able to at least
> read/verify it, right?
> 
> Was a release already made with blowfish, extended DES and NT-Hash? (And
> what's so bad with blowfish? It's mentioned in the heading of the xcrypt
> project too.)

I answered some of your questions in other replies and will answer the
remaining concerns on Monday.

You suggested a 3rd party module. I have cloned the crypt module with
Serhiy's improvements and turned it into a standalone module with a
ctypes interface, https://github.com/tiran/legacycrypt . I'll release
the package as soon as I find time to polish the documentation and give
Serhiy his will deserved credit for his work.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

[Christian Tismer]

Hi user with no real name yet,

On 03.02.18 01:01, asrp asrp wrote:
> Hello,
> 
> I don't know if this is the right place to post this. Please redirect as 
> needed.
> 
> I've made a small Python interpreter in Python with runtime AST node 
> semantics and edit-and-continue. I thought it could make prototyping language 
> changes more easily and visualize usage before writing them in C.
> 
> Its here: https://github.com/asrp/python_terp
> 
> So, for example, redefining the for_stmt function in the right scope changes 
> the behaviour of future for loops at runtime.
> 
> Although from discussion I've read in PEPs, actual implementation always look 
> like a non-issue (which seems like magic to me) so maybe no-one here actually 
> needs this.
> 
> (I really needed edit-and-continue for one of my projects but of course, 
> running it in this extra interpreter is much too slow.)
> 
> asrp

In the readme to python_terp you say:

"""
python_terp is intended to make language modification to Python easier
to preview changes more quickly and is not intended for full CPython
compatibility. However, a large subset of Python is already included. In
particular, enough to run the first stage of its parser.
"""

This needs clarification.
What do you mean by subset? A real subset or also things that
are different and will stay different?
To what extent are you planning to stay compatible, and where
do you plan to deviate?

The reason that I'm asking is that by compatible I mean the
compatibility of PyPy. If you can reach that, and be it just
by a subset, then it makes sense to speak of Python.

Cheers - Chris

-- 
Christian Tismer-Sperling:^)   tis...@stackless.com
Software Consulting  : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate crypt module and revert PR 3854

[Christian Heimes]

On 2018-02-02 21:21, Nathaniel Smith wrote:
> On Feb 2, 2018 7:24 AM, "Christian Heimes"  > wrote:
> 
> Shortly after the PR has landed, I was made aware that glibc has
> deprecated crypt(3) API [2] and favor of an external library called
> libxcrypt [3] from OpenWall Linux. I have patched Python 3.7 [4] to
> support libxcrypt.
> 
> In light of deprecation of crypt(3) glibc function and bad quality of
> hashing algorithms, I'd like to raise the motion to revert 3854 and
> deprecate the crypt module. The whole module should be rather moved into
> 3rd party library that wraps xcrypt.
> 
> 
> If the crypt module can just as well use libxcrypt, and that's how the
> distros are going to build it, then what's the advantage of deprecating
> it? Won't it just continue to work indefinitely?

libxcrypt is now an external library that must be installed separately.
It bloats CPython's dependency tree and the size of Python container
images for a legacy feature.

For Python 4.0 I'd like to drop some of the old, dead batteries and
include include useful batteries.

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate crypt module and revert PR 3854

[Christian Heimes]

On 2018-02-02 21:31, Antoine Pitrou wrote:
> On Fri, 2 Feb 2018 16:23:20 +0100
> Christian Heimes  wrote:
>> Hi,
>>
>> in PR 3854 [1] Serhiy added blowfish, extended DES and NT-Hash to
>> Python's crypt mdodule. I vetoed against addition of the APIs because
>> all these hashing algorithms are not state of the art. Their quality
>> ranges from old to horribly, horriblye broken beyond any repair.
>>
>> Shortly after the PR has landed, I was made aware that glibc has
>> deprecated crypt(3) API [2] and favor of an external library called
>> libxcrypt [3] from OpenWall Linux. I have patched Python 3.7 [4] to
>> support libxcrypt.
>>
>> In light of deprecation of crypt(3) glibc function and bad quality of
>> hashing algorithms, I'd like to raise the motion to revert 3854 and
>> deprecate the crypt module.
> 
> Those are two separate proposals.
> 
> On the topic of reverting PR #3854, I don't see the point.  Is Blowfish
> more fragile than the other algorithms?  If not, it sounds ok to add it.

Blowfish password hashing algorithms (also known as bcrypt) hasn't been
state of the art of a long time. Here is an old blog post from 2012 that
explains some of the issues with Blowfish:

http://www.unlimitednovelty.com/2012/03/dont-use-bcrypt.html

> On the topic of deprecating the crypt module, that doesn't sound like a
> good idea right now.  People may need to generate crypt()-compatible
> output for various reasons, such as being able to automate system
> administration tasks.

IMO legacy support is not a good argument to keep the crypt module. The
passlib [1] package provides an excellent selection of legacy and modern
password derivation and hashing algorithms. As an admin you cannot rely
on the crypt module. libc's crypt() function usually provides a subset.
For example most Linux libc do not provide blowfish based hashing.

Christian

[1] https://pythonhosted.org/passlib/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

[Christian Tismer]

On 03.02.18 08:12, Steven D'Aprano wrote:
> On Sat, Feb 03, 2018 at 12:50:11AM -0500, Terry Reedy wrote:
>> On 2/2/2018 7:01 PM, asrp asrp wrote:
>>> I don't know if this is the right place to post this. Please redirect as 
>>> needed.
>>
>> This list is for development *of* cpython.  Development *with* python in 
>> general belongs on python-list.
> 
> This list is for development of Python the language, not just CPython 
> the interpreter. It seems to me that announcing a new Python 
> interpreter, especially one designed for the purpose of allowing rapid 
> experimentation with the language, is on topic for this list.
> 
> 

Well spoken!

-- 
Christian Tismer-Sperling:^)   tis...@stackless.com
Software Consulting  : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate crypt module and revert PR 3854

[Christian Heimes]

On 2018-02-02 18:05, Serhiy Storchaka wrote:
> 02.02.18 18:18, Guido van Rossum пише:
>> I'm all for nudging people in the direction of xcrypt. I assume we
>> can't just switch the C-level crypt with xcrypt and leave the Python
>> API unchanged?
>>
>> However until a usable solution exist (either in the stdlib or as 3rd
>> party) I don't think we should deprecate anything (deprecating things
>> before the replacement is ready is stressful for everyone involved).
>>
>> I'm also not sure I agree with removing support for old hashes. By all
>> means put in the docs that they are unsafe. But if someone has a
>> database full of old hashes it would be nice to be able to at least
>> read/verify it, right?
>>
>> Was a release already made with blowfish, extended DES and NT-Hash?
>> (And what's so bad with blowfish? It's mentioned in the heading of the
>> xcrypt project too.)
> 
> To clarify, extended DES and NT-Hash were not added. They were removed
> from my PR after Christians request. Only the Blowfish method was added,
> and it is so strong as SHA-2 methods. It is the only method supported on
> OpenBSD.

Ah, I misinterpreted the subject of the PR. The closed PR still mentions
extended DES and NT-Hash. I'm sorry and blame my travel fatigue. The
email was written at the airport after I had a conversion with somebody
about new Python 3.7 features.

> This PR is not a single enhancement made in the crypt module recently. I
> also extended tests and added support for configuring SHA-2 methods.
> There is an open PR (not merged before 3.7b1 unfortunately) for using
> crypt_r() instead of crypt(): https://bugs.python.org/issue28503.

In general I'm all for more tests and improvements of existing modules.
However in this case Python 3.7 is sending wrong signals. For example
additional of blowfish was prominently features on the largest German
newsletter for IT. Both blowfish and SSHA (salted sha) are legacy
password hashing algorithms. Glibc has moved them out of the main
library for a good reason. (*)

> If deprecate the crypt module, should modules pwd, grp and spwd be
> deprecated too? The crypt module is needed for checking password hashes
> provided by spwd.

The pwd and grp module are fine. The modules use proper libc APIs that
are internally backed by NSS (libc's Name Service Switch, not Mozilla's
Network Security Service). APIs such getpwnam are defined and
standardized since POSIX.1-2001. The pwd and grp automatically work with
any configured user and group provider, even LDAP, IdM or Active Directory.

Fun fact: Golang programs are usually statically compiled and don't even
use libc. However Go's os/user package requires CGO and libc because it
has to interface with libc and NSS to acquire user and group information.

The spwd module is a different story. It's a direct interface to
/etc/shadown using Linux-only APIs. The shadow DB API requires root
permission. I think it even circumvents system security policies and
identity provider.

tl;dr
pwd + grp == good, required
crypt + spwd == bad

Regards,
Christian

(*) Most Linux distros never had blowfish in libc anyway.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com