Re: [Readable-discuss] Readable Lisp 2.0, brainstorming

[luke wallace]

I went to great pains to explain that what symbols mean would have to be
explained by the symbol/function creator in a comment inside the
symbol/function definition. The editor would pull the explanation from that
comment. The comment would have some identifier, such as "Usage:" or
"Tooltip:" to prefix the comment, so that the editor knew it was a special
comment. Each function/symbol would have to be manually commented in this
way before the editor knew about it - after all, we can't expect an editor
to speak to humans in an understandable way about what a function does
without telling it so - because if it could we probably wouldn't need human
programmers any more.

I've been doing a lot of research on this and I hit a jackpot of
information. What I'm basically describing and edging towards is called a
projectional editor. Jetbrains MPS is probably the most advanced
projectional editor out there, which does not need a parser, because it
directly manipulates the AST (abstract syntax tree). It displays the code
in a number of views - the 2 minute video is worth more than a thousand
words: https://www.youtube.com/watch?v=XolJx4GfMmg

Jetbrains MPS appears to be an extremely complex piece of software, the
interface, while functional, is about as complicated as an airplane
cockpit, and I'm just not a fan of that kind of thing. Programming should
be simple and elegant like lisp, even within a projectional editor. I
suspect there's a better way to make a projectional editor by building the
abstract syntax tree using s-expressions, which would make it easier and
simpler to implement DSL's while keeping the end-result pretty, but this is
just a hunch, more research is required..
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Readable-discuss mailing list
Readable-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/readable-discuss

Re: [Readable-discuss] Readable Lisp 2.0, brainstorming

[David A. Wheeler]

On Mon, 20 Feb 2017 15:36:22 -0600, luke wallace  
wrote:
> Here's a similar but slightly different prototype
> 
> https://i.imgur.com/vApN1Yt.png

Quick nitpick  - please use these terms:
() - parentheses
[] - (square) brackets
{} - (curly) braces

I realize that some people call *all* of the paired characters "brackets",
but since we have to talk about paired characters often around here, it's
helpful to be very precise about these terms :-).

> This prototype uses a flatter version of tables, and much more 'magic'
> would have to be programmed into the editor
> cells shaded grey are prefix-comments, in this case, for the function
> 'define'.

There are bit minuses to this.  This assumes that the editor
"knows" what terms like 'define' and '<=' and 'if' mean.
I imagine there are some uses for it, primarily if you use Lisp
exactly the same way as you'd use a fixed-syntax language like Java.

But fixed-syntax languages are going to be much better at being themselves
than a Lisp ever could.  If you need a fixed-syntax language, there are
over a hundred of them.  A key advantage to Lisp is that you can
embed many different languages & can trivially create your own.
Mini-languages abound! There's no particular reason that "<=" or "define"
would have the same meaning each time even within one line,
and there's no way an editor could determine the meaning in general
(it would have to recursively expand macros across all files,
and even then, there's no mechanism for capturing this information in general).

That said, *displaying* boxes is a good idea.  I think it's been done before,
but it's fair to point out that modern systems make this *much* easier and
more practical.  And you don't need *any* magic to display the boxes.

So, when will you be implementing a prototype :-) ?

--- David A. Wheeler

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Readable-discuss mailing list
Readable-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/readable-discuss

Re: [Readable-discuss] Readable Lisp 2.0, brainstorming

[luke wallace]

Here's a similar but slightly different prototype

https://i.imgur.com/vApN1Yt.png

This prototype uses a flatter version of tables, and much more 'magic'
would have to be programmed into the editor

cells shaded grey are prefix-comments, in this case, for the function
'define'.
prefix-comments would have to be set up for each function for a given
language/variant, but would enable a form-filling experience resulting in
lisp code - and most near computer illiterates wouldn't mind coding if it
was mostly filling in forms.

cells with bold text are contextual-comments - the usage comment is
contextual to the 'factorial' function, and the 'then' and 'else' comments
are contextual to the 'if' function. these are set up for each function for
a given language/variant, just like prefix-comments.

The 'define' cell spans the whole table - therefore everything under it is
wrapped in its parens
The next part relies on a bit of magic, 'factorial' starts an open paren,
but does not close unless the 'Parameter(s)' box is left empty. If the
parameter(s) block is left empty, factorial will be a function without
arguments and close paren. The editor would have to handle these conditions
for the 'define' symbol (or any symbol used to create functions).

The 'usage' prefix-comment and its corresponding contextual-comment are
used for the tooltip in the editor wherever the function is used (including
in the definition code of this recursive function), and also as a readable
snippet when someone looks at the function definition, to understand what
it does. This might help libraries of functions be more understandable in
the long run, as they would be self documenting not just in how to use
them, but where and why to use them.

The 'code' prefix-comment uses special logic to open/close parens. The
first piece of actual code, the 'if' opens a parens and does not close
until a new symbol in the same column exists. Since 'then' and 'else' are
not symbols, but are comments, it doesn't close until the end of the table
(before the 'define' ends though).

'n <= 1' opens, then closes because '1' is on the same column, and is
actual code
'1' opens, then closes because 'n*' is on the same column, and is actual
code
'n* opens, then 'factorial' opens, then 'n-1' opens, from left to right,
and they all close at the end of the table, but before the 'if'.

The goal of doing it this way is to prevent excessive nesting of cells -
and use implied nesting through the logic of columns.

On Mon, Feb 20, 2017 at 1:03 PM, David A. Wheeler 
wrote:

> On Mon, 20 Feb 2017 11:47:35 -0600, luke wallace <
> lukewallace1...@gmail.com> wrote:
> > Here is my new idea for an ideal lisp authoring format, which relies on
> an
> > html/css/js editor using pretty components around source lisp code.
> Source
> > lisp code is still saved as a text file, but is, upon loading into this
> > theoretical editor, "pretty formatted" as shown in the image below.
>
> Very interesting.  Even if it's not used for editing, I can imagine
> something
> like this as the output of a pretty-printing process.
>
> I wonder about *editing* while doing this, though.
> Perhaps entering "(" would automatically add a closing ")",
> show the box (if not preceded by whitespace), and move the user inside.
>
> > Inside this editor, creating parentheses in the right context would
> instead
> > create nested table cells as shown in the image. Instead of working with
> > parentheses, newbies might find it easier to look at tables like this...
>
> Newbies don't stay newbies.  I'd like to think that any "readable Lisp"
> would be just as useful to experts as to newbies.  Experts have to read
> a lot of code... making their lives easier is a good idea too :-).
>
> > Using more html/css/js magic, it would be possible in such an editor to
> do
> > things like display a tooltip when a user hovered over recognized symbols
> > such as <= in the image. The tooltip could display an explanatory message
> > like "less than or equal to". While experienced programmers may not need
> > the <= symbol explained to them, by explaining every symbol, it should
> > become even more readable in the editor.
>
> One challenge is that "<=" doesn't have a fixed meaning; it might mean the
> underlying Lisp implementation, or the meaning of "<=" as redefined in
> some other language you're defining there.
>
> > In this prototype image, the parentheses are shown as square table cells,
> > but they could be rounded, given line width, background colors, or change
> > colors on mouse-hover, keyboard focus, or when using a debugger to step
> > through code line by line (indicating what portion of the table is being
> > evaluated) etc. The possibilities are endless if we step out of the
> > shackles of needing to see source code as the language dictates it.
>
> Sure.  You need to be cafeful, though; bugs are easy to hide if the
> developer
> can't easily "see" what the code actually *is*.  I think 

Re: [Readable-discuss] Readable Lisp 2.0, brainstorming

[luke wallace]

Here is my new idea for an ideal lisp authoring format, which relies on an
html/css/js editor using pretty components around source lisp code. Source
lisp code is still saved as a text file, but is, upon loading into this
theoretical editor, "pretty formatted" as shown in the image below.

Inside this editor, creating parentheses in the right context would instead
create nested table cells as shown in the image. Instead of working with
parentheses, newbies might find it easier to look at tables like this,
which show the nested structure more visually. The example image uses the
factorial function from the readable website homepage. There is an extra
column with text in bold which are comments. Comments in this manner can be
put anywhere, and will always be bold so they are easily distinguished.
This allows newbies to document their code, or an editor to document their
code for them, for functions like "if" that the theoretical editor knows
about.

Also, the "define" function is the parent function for this block, so it
has its text centered - an easy visual way to distinguish the parent
besides just the outer border.

Comments could be disabled by a menu option in the editor.
Specifically, these special comments which are generated by the editor
could be hidden at the users request, while keeping different types of
comments from the source file visible in some way.

So far, this just shows infix notation (taken from readable lisp) and a new
way to represent parentheses and comments/dynamic comments.

https://i.imgur.com/Q9lpfVa.png

Using more html/css/js magic, it would be possible in such an editor to do
things like display a tooltip when a user hovered over recognized symbols
such as <= in the image. The tooltip could display an explanatory message
like "less than or equal to". While experienced programmers may not need
the <= symbol explained to them, by explaining every symbol, it should
become even more readable in the editor.

In this prototype image, the parentheses are shown as square table cells,
but they could be rounded, given line width, background colors, or change
colors on mouse-hover, keyboard focus, or when using a debugger to step
through code line by line (indicating what portion of the table is being
evaluated) etc. The possibilities are endless if we step out of the
shackles of needing to see source code as the language dictates it.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Readable-discuss mailing list
Readable-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/readable-discuss

Re: [Readable-discuss] Readable Lisp 2.0, brainstorming

[David A. Wheeler]

On Thu, 1 Sep 2016 19:37:43 -0400, luke wallace  
wrote:
> My idea is that by converting lisp into a simplified abstract syntax tree
> form, and then performing formatting options on that abstract syntax tree
> (such as prefixing parts of it with syntactic sugar, or compacting some
> levels of the tree, or reformatting the order of operators and operands, we
> are able to create human-readable formatting for conventional lisp code
> 
> Below are two theoretical examples:
> 
> http://i.imgur.com/JNOXQ4M.png

The pictures are fine representations of abstract syntax trees (ASTs).
You need to handle "lists of lists" (like let's first argument);
that can be tricky but is doable.
If you mean for people to view them directly (say auto-generated as you edit),
I'd be a little worried about screenspace - that's always limited compared
to the amount of code being managed.  If you only showed "the part you're
working on" that might not be bad.

Constantly updating a graphical AST representation
of "what I'm working on" might be snappy; if you just keep regenerating from the
text you completely eliminate the problem of selecting things, and you can
highlight "where the cursor is".

If you mean for people to *edit* those directly,
it can be tricky work out how to select the "right spot" for insertion, etc.
I think trying to edit that would be more work than useful.
It's been done for stuff like mathematics:
  http://icm.mcs.kent.edu/reports/2006/ICM-200601-0001.pdf
However, that presumes that you can know what a given operation is
given its name, and how many parameters that operation would take,
which is not true in Lisp.

I'm guessing that you really mean to show those final representations, though, 
e.g.,
sqrt(a+b*(c-d)) or whatever.  If you use a Lisp as a subset of Python
that can work out fine but Python is a better Python.
A key power of Lisp is that you can define your own languages (e.g., macros),
use templating (e.g., ","), and other powerful techniques that exploit 
homomorphism.
The earlier AST is trivially homomorphic.  However, the "final" representation
shown does *not* appear to be homomorphic.  That's fatal.

Trivial example: the final version added a "then" and "else" after an "if".
But there's no reason to believe that the "if" is actually
the built-in Lisp "if". I could easily define a macro that rewrites "if" into
something else (and indeed, that's a common thing to do in packages).
For example, maybe "if" means "interface" at that point.
Similarly, "defun" (Common Lisp) or "define" (Scheme) might mean the
specific names predefined in the language.. or not.

If you just want a specialized language inside a larger Lisp program,
feel free to create it.  Many have.  But if you want to write Lisp programs in
a "more readable" way, your notation has to be homomorphic & handle
arbitrary s-expressions.  There's simply no way to know what the symbols
"mean" while you're editing them.

I'm *not* saying it's impossible.  It's doable.  Sweet-expressions are one 
solution,
and the wisp developers have created another.
But you have to consider the general case.  If you limit yourself to
symbols with fixed meanings, there are other languages that already do the job.
If you're going to use a Lisp, it should be because Lisp provides some
advantage to you.

--- David A. Wheeler

--
___
Readable-discuss mailing list
Readable-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/readable-discuss

Re: [Readable-discuss] Readable Lisp 2.0, brainstorming

[David A. Wheeler]

On Thu, 1 Sep 2016 19:37:43 -0400, luke wallace  
wrote:
> Let's try to use wishful thinking in order to reinvent how to make lisp
> more accessible and useful to complete newbies.

Thanks for sharing your thoughts!  I'd prefer this discussion be public, if 
possible,
so that everyone can share in the results.

> The editor would do all the box drawing for
> you, and be saved as lisp in source files (the editor would just display it
> in readable format once opened).

As I understand it, you want to have a bidirectional translation within the
editor, so that what is *saved* is traditional Lisp code, but what is *viewed*
is some more readable format.

Interestingly enough, bidirectional translation is already *partly* possible 
with
the readable notations right now. We already have programs that translate each 
way.
Presuming you choose the top level (sweet-expressions), you could convert to 
sweet,
edit, then translate back on save.  It's not ideal though (see below).

However, there are several issues with bidirectional translation (translations 
on read and save):
1. You *really* have to trust the bi-directional translation tools.
Even the slightest defect can cause problems, because "what you run" isn't 
"what you see".
2. You have to be careful on handling comments.
The current sweetening routine drops comments outside the top level.
This lossiness is not ideal, but for its intended purpose it's okay... but 
for bidirectional
work it's not a good thing.
3. You either lose the formatting on both ends, or you spend a *lot* of time 
trying to
"fix up" formatting.  If someone formats it in a certain way in the 
traditional Lisp code,
it's tricky to retain that, and the other way as well.
4. If you work with other people who use different formatting, you can end up
with a lot of pseudo-changes, which would greatly interfere with version 
control systems
(it'd look like you changed a lot of lines, when it was just reformatted).
This is easy to solve: Pick a specific formatter and its options, and 
*always* run that
before saving.

It's certainly *possible* to do bidirectional translations.  But it's a lot of 
work.
Most GUI programming tools, for example, store a representation of the GUI 
display
and only do 1-way translations from the GUI to the underlying representation.
You're certainly welcome to try some things out - I'm a little skeptical that 
it's worth
the work, but the proof is always in the doing.

--- David A. Wheeler

--
___
Readable-discuss mailing list
Readable-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/readable-discuss