Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Matthew Einhorn]

On Thu, Aug 30, 2018 at 8:31 PM James Lu  wrote:

> Why shouldn't Python be better at implementing Domain Specific Languages?
>
> [snip]
>
> It would be nice if there was a DSL for describing neural networks (Keras).
> The current syntax looks like this:
>
> model.add(Dense(units=64, activation='relu', input_dim=100))
> model.add(Dense(units=10, activation='softmax'))
>
>
How about something like this?

with model:
with Dense() as dense:
dense.units = 64
dense.activation = 'relu'
dense.input_dim = 100

with Dense() as dense:
dense.units = 10
dense.activation = 'softmax'

The `with` creates a context to which subsequent layers are added when
created within the context.

But it does suffer from the fact that the `with` object's (or the
underlying stack that would implement
this in the model/layers) scope is not local to the function, so if within
the `with` context you call a
function that creates a layer, the layer will be added to the caller's
context, which would be surprising.

I was working on a similar approach for a python GUI, and `with` seemed
like a very nice candidate,
but ran into this issue, which didn't seem to have a clean fix.

I also thought about using an decorator that pre-processes the AST for a
GUI description, which for
your example would look like something like:

with model:
with Dense():
units = 64
activation = 'relu'
input_dim = 100

with Dense():
units = 10
activation = 'softmax'

But here the issues are (a) the similarity with local variables may be
confusing (b) you'd need to
either make all `with` statements special, or annotate the `with`
statements that are going to be
processed by the compiler (e.g. by prefixing the object with a dash). It
seemed messy
enough that I'm still pondering this.

Matt
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Abdur-Rahmaan Janhangeer]

the implementation is very easy just a program and some ifs

if you need more complex, just some reading on tokens etc
but the first choice is enough

if in py competitions people are solving regex like questions and word
extractions without knowing they are solving compiler theory problems,
implementing DSL is very easy

Abdur-Rahmaan Janhangeer
https://github.com/Abdur-rahmaanJ
Mauritius
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Extend the warnings module with Deprecation utility methods and classes

[Ilya Kulakov]

Very nice, thank you.

It would be a good start to collect usage patterns like this as seen in the 
wild for design consideration and test cases.

> On Aug 30, 2018, at 10:43 PM, Joshua Harlow  wrote:
> 
> And mirrored at https://github.com/openstack/debtcollector 
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Steven D'Aprano]

On Fri, Aug 31, 2018 at 11:39:16AM -0400, James Lu wrote:

> We should all take a look at Ruby Blocks and think about how Python 
> could benefit from something similar.

You are not the first person to suggest Ruby-like anonymous blocks or 
multi-statement lambdas.

https://duckduckgo.com/?q=python-ideas+ruby+blocks

Its not enough to just think about the benefits. We also need to think 
about the costs, the disadvantages, the possible syntax, and how well it 
fits into the existing language.

If anyone has some new and interesting ideas, that would be fantastic. 
But if we're just going to rehash the same rejected ideas again, please 
don't. Python is 20+ years old and the idea of multi-statement anonymous 
functions has been discussed since before Ruby even existed.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Steven D'Aprano]

On Fri, Aug 31, 2018 at 11:14:35AM +0400, Abdur-Rahmaan Janhangeer wrote:

> let me take an example :
> 
> a DSL to calculate the cost of houses
[...]
> --- file ---
> house num 1,000
> house price 250,000
> calculate sum
> 
> --- output ---
> $ 250 000 000

I don't think the problem is people coming up with DSLs for their 
problems, but implementing those DSLs. Writing an informal specification 
is easy; writing the implementation is trickier.

For the above, you need an interpreter for the DSL, otherwise it is 
just a static data file that generates no output at all.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[James Lu]

We should all take a look at Ruby Blocks and think about how Python could 
benefit from something similar.

> On Aug 31, 2018, at 3:14 AM, Abdur-Rahmaan Janhangeer  
> wrote:
> 
> i believe a DSL is simple enough for an enthusiastic py programmer to write 
> if you really wanted one
> 
> just write the tasks you need to accomplish, the data needed, the constrcuts 
> needed (if needed), the feel/look of it on your editor
> 
> plan first, come up with a good mock, then implement it. implementation is 
> easy, ideas are hard. good ideas offload the efforts on the implementation 
> side, they can also save you future troubles
> 
> let me take an example :
> 
> a DSL to calculate the cost of houses
> 
> aim : calculate cost of houses
> 
> input :
> num of houses
> price of house
> 
> output :
> price of houses
> 
> technical tasks :
> show to screen
> 
> it might go on like that
> 
> --- file ---
> 
> house num 1,000
> house price 250,000
> calculate sum
> 
> --- output ---
> 
> $ 250 000 000
> 
> in the above example, assumptions were made and functions crammed but you 
> have a dsl. real-life dsls are not far from the specs of this one but differ 
> in the tools used
> 
> Abdur-Rahmaan Janhangeer
> https://github.com/Abdur-rahmaanJ
> Mauritius
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fix some special cases in Fractions?

[Oscar Benjamin]

On Thu, 30 Aug 2018 at 17:36, Stephan Houben  wrote:
>
> I would also like to point out that the current behavior of Fraction
> is consistent with other parts of the numeric system, e.g.
> 1/1 produces 1.0 (rather than 1)
> math.sqrt(4) produces 2.0 (rather than 2)
> 1j-1j produces 0j (rather than 0.0 or 0)
>
> So in general the type of the output is determined by what
> the operation would in general return for that input type,
> as opposed to a more specific type which only applies
> to the specific input value.

An exception is integer exponentiation:

>>> 1 ** 1
1
>>> 1 ** -1
1.0

Given two rationals q1 and q2 usually q1 ** q2 will not be a rational
number. Integer exponentiation results in an integer for half of all
possible integer pairs.

To do the same with Fraction(a, b) ** Fraction(c, d) would require
verifying that both a and b have exact integer dth roots which is more
complicated than simply checking the sign of an integer exponent. The
extra complexity would slow things down a bit but then again the
fractions module is there for precisely those people who are happy to
have substantial slow-down for the sake of exactness.

>From a backwards compatibility perspective the old behaviour would
still be available in a way that works already: float(q1) **
float(q2).

Also it would also be straight-forward to implement this given the
integer maths (iroot etc) functions that were discussed in a recent
thread on this list:
https://mail.python.org/pipermail/python-ideas/2018-July/051917.html

However: Why would you do this operation if you wanted an exact
result? I have at some point wanted (for ints or Fractions) a function
root(q, n) that gives an exact root or an error. This proposal would
mean that q1 ** q2 would be exact occasionally and would return a
float the rest of the time though. If you care about
exactness/accuracy in the code you write then it is not okay to be
unsure whether your variable is float or Fraction. You need to know
because it makes a big difference to how you calculate things (it
isn't generally possible to write optimal polymorphic code over
exact/inexact arithmetic).

The proposed behaviour would look nicer in an interactive session but
might not be any more useful in situations where you *really* care
about exactness.

--
Oscar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Abdur-Rahmaan Janhangeer]

as for elm, you have to look twice not to see the python of it

Abdur-Rahmaan Janhangeer
https://github.com/Abdur-rahmaanJ
Mauritius
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Jeff Allen]

The word "domain" appears in this sense on the first page of Aho and 
Ullman and ANTLR (which I know you've used) describes itself as a tool 
for building domain-specific languages. Both pre-date Ruby I'm fairly sure.


James Lu, quoting Jonathan Fine, used the term "internal DSL" and 
although that's new to me, people seem to be interpreting it in the 
sense that Gradle is a Groovy DSL (note caps), a build tool a lot of 
software developers will be familiar with. In that case, what you write 
really is Groovy, but the execution environment has been pre-conditioned 
with objects and libraries that (almost) make a new language. When you 
understand what's going on (not sure I always do), it becomes possible 
to mix Gradle statements and Groovy freely. The most immediate benefit 
is that all the apparatus of expressions and types/methods is already 
present. So "internal" is the key word.


The point about punctuation is spot-on, I think: Groovy is relatively 
free of (makes optional) some punctuation, including the parentheses 
that make calls easily identifiable in Python. So quite possibly 
starting from Python is limiting if what you want is an*internal* DSL 
with a grammar you choose: the object system is fantastic-plastic, but 
the grammar is not. DSLs embedded in Python are common, of course 
(f-strings, regexes, SQL), and DSLs can generate Python from fragments 
with almost no constraints on their own grammar. iPython strikes me as 
possibly a Python internal DSL, or Django, but what they've done does 
not take us far from pure Python.


Jeff Allen

On 31/08/2018 05:07, Guido van Rossum wrote:


Hm. YAML is indeed a great, readable alternative to JSON or XML. But 
the term DSL implies (to me) more than just nested key-value pairs. 
(Though who knows maybe that's all Keras needs, and then it's a poor 
argument for having a DSL.)


Then again maybe I'm confusing DSL (which appears to be a Rubyism) 
with "little language": http://wiki.c2.com/?LittleLanguage


--
--Guido van Rossum (python.org/~guido )



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Jonathan Fine]

James Lu started this thread by quoting me. Thank you, James, for the
compliment. And I feel somewhat obliged to contribute here, are at
removed I started the thread.

In the message James quoted, I also said


But most strengths, in another situation, can be a weakness. Language
design is often a compromise between conciseness and readability, ease
of use and performance, good for beginners and good for experts, and
many other factors. Such as innovation and stability.

Guido's decisions, as BDFL, have shaped Python and its community into
what it is now. It is one set of compromises. Other languages, such as
Ruby, have made different compromises. Now that the BDFL is on
vacation, the challenge is to maintain the essence of Python while
continuing to innovate.


It's important, of course, for the developers of a DSL to understand
the domain. I'm starting to learn http://elm-lang.org/. It describes
itself as

> A delightful language for reliable webapps.
> Generate JavaScript with great performance and no runtime exceptions.

The Elm developers have learnt a great deal from Python, and I think
that we in turn can learn from them. Particularly about catching
coding errors early, with good feedback. But that's a different
thread.

So I'd say to focus on improving the API to an existing library is a
good way to develop our understanding of DSLs more generally. James
provided a Keras example

model.add(Dense(units=64, activation='relu', input_dim=100))
model.add(Dense(units=10, activation='softmax'))

What might be better here, if allowed, is

model.extend([
Dense(units=64, activation='relu', input_dim=100),
Dense(units=10, activation='softmax'),
])

Another approach would be to provide a fluent interface.

https://martinfowler.com/bliki/FluentInterface.html
https://en.wikipedia.org/wiki/Fluent_interface

Done this want, we might get something like jQuery

   (
model
.dense(units=64, activation='relu', input_dim=100)
.dense(units=10, activation='softmax')
   )

JSON and XML and YAML have already been mentioned. Here's another,
XML-ish approach.

A combined list-dictionary is quite common. It's used widely in XML
(and SGML before it). So how to create such. A few years ago I
experimented with an API such as

 A(a=1, b=2)[
 X(1, 2, 3),
  Y[ ],
  ]

As I recall, someone told me that https://kivy.org does something
similar. Kivi and Elm, are systems I'd like to learn. Ease of use is
important in language and library design. We can learn from the
success of others, as well as from our own successes and failures
(smile).

-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Paul Moore]

On Fri, 31 Aug 2018 at 06:36, Steven D'Aprano  wrote:

> Please no, Ruby has that, and the meaning of expressions depends on
> whether you put whitespace around operators or not.
>
> Given:
>
> def a(x=4)
> x+2
> end
> b = 1
>
> the result of "a+b" depends on the spaces around the plus sign:
>
> irb(main):005:0> a + b
> => 7
> irb(main):006:0> a +b
> => 3

Ewww. And people complain about *python's* use of significant whitespace???!?!!

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fix some special cases in Fractions?

[Neil Girdhar]

On Thu, Aug 30, 2018 at 9:03 AM Steven D'Aprano  wrote:

> On Wed, Aug 29, 2018 at 09:39:05PM -0700, Neil Girdhar wrote:
>
> > Would there be any problem with changing:
> >
> > In [4]: Fraction(1, 1) ** Fraction(2, 3)
> > Out[4]: 1.0
> >
> > In [5]: Fraction(-1, 1) ** Fraction(2, 3)
> > Out[5]: (-0.4998+0.8660254037844387j)
> >
> > In [6]: Fraction(0, 1) ** Fraction(2, 3)
> > Out[6]: 0.0
> >
> > I'd like these to be Fraction(1), Fraction(1), and Fraction(0).
>
> If anyone else has mentioned the backward-compatibility issue by now, I
> haven't see it. I believe that would make it a fairly big problem.
>
>
I don't think it's that big a problem since Fraction < numbers.Real


> I expect that there is code out in the wild which (for good or ill) now
> expects 1**Fraction(2, 3) to return 1.0, rather than Fraction(1), and
> similarly for the other examples. Changing that could break people's
> code.
>
> Even if we had consensus that this was a good idea, or at least
> consensus from the maths-folk who care about this sort of thing (I'd
> like to know what Uncle Timmy and Mark think of this idea), it would
> still probably need a "__future__" import to activate it, or a
> deprecation period. Or some other annoyance.
>
> Possibly the simpler approach would be to add a subclass that does what
> you want. UnitRootFraction or something.
>
> Then the only argument will be whether such a subclass ought to go into
> the fractions module, or your own personal toolkit :-)
>
> I must admit though, I'm a bit curious as to what you are doing that
> having 1**Fraction(2,3) return 1.0 is an annoyance, but having
> 27**Fraction(2,3) return 8.998 instead of Fraction(9) isn't.
>
> You're right, it would have been better in my example (linked above) to
support all of those things.

>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/aZIHpPhe0mw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] On evaluating features

[Stephen J. Turnbull]

Executive summary: Although I'm obviously not particularly in favor of
this feature, my opinion is no more valuable than anyone else's.  The
point of this post is to show how features have been advocated
successfully in the past.

James Lu writes:

 > > By comparison,
 > >
 > >   x, y += a, b
 > >
 > > is neither more expressive, nor easier to read, nor significantly
 > > harder to type, than
 > >
 > >   x += a
 > >   y += b
 >
 > I agree. I contend that x, y += func(...) is more readable than the three-
 > statement alternative with a namespace pollution.

I'm sure it is.  But what's the context?

That is, just saying you think it's more readable isn't enough.  What
has succeeded in the past (in approximate order of effectiveness) has
been

1.  A reasonably comprehensive survey of the Python stdlib to find use
cases for the "ugly" construct, and then providing a few "side by
side" comparisons "before and after" the new syntax, as well as
statistics for the frequency of the "ugly" construct, and how
often translation to the new feature improved the code.

2.  A similar exercise for some other large, reasonably well-known
code base by a respected project.

3.  A few examples from some code base.

Here's an example of the kind of analysis I mean.  I went looking for
examples of swaps in the Python sources by grepping for "sort".  I
found several in Tools/demo/sortvisu.py, but strangely enough,
although it uses sequence assignment for the swap, it does this:

def swap(self, i, j):
if i == j: return
self.countswap()
item = self.items[i]
other = self.items[j]
self.items[i], self.items[j] = other, item
item.swapwith(other)

The reason is that this is not a sorting application as such, but
rather a demo of tkinter.  So the sort is instrumented in several
ways, and in particular the items being swapped know "where they are"
in the visualization (the item.swapwith(other) call).  Thus the
temporary variables item and other are not spurious.

I'm not sure if I think

def swap(self, i, j):
if i == j: return
self.countswap()
item, other = self.items[i], self.items[j]
self.items[i], self.items[j] = other, item
item.swapwith(other)

is less readable, but I don't think it's more readable.  And with the
instrumentation removed, I'm not sure whether I prefer

def swap(self, i, j):
if i == j: return
self.items[i], self.items[j] = self.items[j], self.items[i]

or

def swap(self, i, j):
if i == j: return
item, other = self.items[i], self.items[j]
self.items[i], self.items[j] = other, item

It's pretty marginal.  Finally, I think

if i != j:
self.items[i], self.items[j] = self.items[j], self.items[i]

inlined instead of

self.swap(i, j)

perceptibly harms readability, although it would be somewhat faster
due to eliminating the function call, and if speed were at a premium I
would prefer the inline swap to the introduction of any temporary
variables.[1]

My guess is that a lot of realistic cases where you apparently *could*
use augmented assignment of sequences to avoid introducing one or more
temporary variables are going to have some of the same features that
the variables are actually useful beyond the augmented assignment, or
the expressions on one side of the assignment or both are sufficiently
complex that introducing the temporary variable with a descriptive
name improves readability.  But the proof of that will be in doing the
work.

Steve


Footnotes: 
[1]  And if you're reading this, Victor, of course I'd benchmark
before using the less readable version in production. :-)

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Abdur-Rahmaan Janhangeer]

i believe a DSL is simple enough for an enthusiastic py programmer to write
if you really wanted one

just write the tasks you need to accomplish, the data needed, the
constrcuts needed (if needed), the feel/look of it on your editor

plan first, come up with a good mock, then implement it. implementation is
easy, ideas are hard. good ideas offload the efforts on the implementation
side, they can also save you future troubles

let me take an example :

a DSL to calculate the cost of houses

aim : calculate cost of houses

input :
num of houses
price of house

output :
price of houses

technical tasks :
show to screen

it might go on like that

--- file ---

house num 1,000
house price 250,000
calculate sum

--- output ---

$ 250 000 000

in the above example, assumptions were made and functions crammed but you
have a dsl. real-life dsls are not far from the specs of this one but
differ in the tools used

Abdur-Rahmaan Janhangeer
https://github.com/Abdur-rahmaanJ
Mauritius
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why shouldn't Python be better at implementing Domain Specific Languages?

[Michael Selik]

On Thu, Aug 30, 2018, 9:23 PM David Mertz  wrote:

> On Fri, Aug 31, 2018, 12:08 AM Guido van Rossum  wrote:
>
>> Hm. YAML is indeed a great, readable alternative to JSON or XML. But the
>> term DSL implies (to me) more than just nested key-value pairs. (Though who
>> knows maybe that's all Keras needs, and then it's a poor argument for
>> having a DSL.)
>>
>
> Keras is deliberately very declarative in defining models. So indeed
> sequences and mappings and scalars is everything it needs. Maybe I'll
> actually implement the idea I sketch out in a small independent library.
>

Defining Keras models reminds me of the ugly ASP code I'd sometimes write
to create HTML in an object-oriented fashion. Writing plain HTML was
usually more pleasant and readable.

I suggested XML with that memory in mind. It's cumbersome for many tasks,
but Keras models in particular might be a good fit. Or YAML.

>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/