Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Thomas Adam]

On Thu, Oct 27, 2016 at 12:58:14AM +0100, Dominik Vogt wrote:
> Ah.  Technically, the branch is probably as good as it will get.
> Any changed to the "stdint.h" branch can easily be backported, but
> that may not be necessary.
> 
> However, if we agree on this becoming the "long time stable"
> version we'll need something on the web page that explains:
> 
>  * What this release is and what future work it will receive
>(maintenance but no new features).
>  * Where information about version 2 and version 3 is found.
>  * What the plans for version 3 are.
> 
> We'll also need similar information for the mailing lists.  When
> we publish the release it must be clear that "that's it" and
> development of version 2 is finally over (unless steps up to take
> over the project).

Yes.

> Also, a first "very unstable" development release should already
> be in place so that people who are interested in the new work can
> download and try it and don't have to use git.

You can always point people here:

https://github.com/fvwmorg/fvwm/archive/master.zip

(Or any repo name) -- which will download HEAD of master.  No git needed.
Interestingly though, I just don't think that's as big a problem as you think
it is.

> Just thinking of it, once the version split is done, it must be
> possible to install version 2 and version 3 in parallel.  It is
> really possible at the moment?  A couple of days ago I've noticed
> that each "make install" just overwrites the files in
> $prefix/share.  Does the --program-suffix configure option work?
> Maybe we should make "--program-suffix=-3" the default for now?
> And we need to tell people how to install both versions in
> parallel.

It's not possible, no.  Plus, have you seen configure.ac lately?  It's a
bloody mess.  I don't think you'll have to worry about paralell
installs---if/when/maybe fvwm3 starts, we'll fix it there in a different way.

Kindly,
Thomas

Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Thomas Adam]

On Thu, Oct 27, 2016 at 12:41:03AM +0100, Dominik Vogt wrote:
> On Thu, Oct 27, 2016 at 12:15:36AM +0100, Thomas Adam wrote:
> > On Wed, Oct 26, 2016 at 11:45:53PM +0100, Dominik Vogt wrote:
> > > Maybe, maybe not.  This work only changes the inner structure of
> > > the parser yet and has few user visible changes (some rarely
> > > triggered quoting issues).
> > 
> > Any chance we could at least get that done though?  It seems daft not to.
> 
> I'm not exactly sure what you're refering to with "that".

I'm referring to relesing fvwm2-stable and fvwm-2.6.7 -- that's still
outstanding, and from what I understand fvwm2-stable can be declared.
fvwm-2.6.7 is ready when the default-config is merged.

-- Thomas Adam

Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Thomas Adam]

On Wed, Oct 26, 2016 at 11:45:53PM +0100, Dominik Vogt wrote:
> Maybe, maybe not.  This work only changes the inner structure of
> the parser yet and has few user visible changes (some rarely
> triggered quoting issues).

Any chance we could at least get that done though?  It seems daft not to.

[...]

-- Thomas Adam

Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Dominik Vogt]

On Wed, Oct 26, 2016 at 01:45:46PM +0100, Thomas Adam wrote:
> On Wed, Oct 26, 2016 at 02:21:57PM +0200, Dominik Vogt wrote:
> > First of all I hope I'll be able to find the crash caused by a free of a 
> > string that is still
> > used, then experiment with it to see whether a step by step replacement of 
> > the old
> > parser is feasible while keeping the old parser around for a while.  
> > Disentangling the
> > builtin functions from the parser without changing their syntax is an
> > important step towards a new parser with a new syntax.  As long as all the
> > functions do their own parsing, replacing the syntax is a huge amount of
> > work.
> 
> But what you're describing is the biggest piece of work which would almost
> define fvwm3.  Thus far, there's still an on-going discussion about how we're
> going to do that.  Heck, we haven't even declared fvwm2-stable or fvwm-2.6.7
> yet.  Aren't you jumping the gun a little?

Maybe, maybe not.  This work only changes the inner structure of
the parser yet and has few user visible changes (some rarely
triggered quoting issues).

> I'd argue---quite strongly---that any work towards a new parser should be done
> with breaking everything completely.  Sure, you can still keep some of the
> surrounding things, but trying to maintain backwards-compatibility isn't
> necessary given the measures we're putting in place for fvwm2's future.

It's not about staying compatible but about changing the parser in
steps with a useable intermediary result that can be tested.
Donig the parser from zero would mean to have nothing that works
even remotely for months.

> But at the moment, we haven't even discussed what that will look like, or what
> requirements we might have.  I started to do that recently when I put out a
> proposal for a new syntax to see how that fared.

Yeah, I've seen that.  The current work should be independent of
the syntax we're aiming at though.  I would imagine changing the
parser in several distinct steps:

  1. Restructuring the code so that different parsers can be plugged
 in at run time on a per builtin basis.
 [This is what the current  work is about.]

  2. Replace the current builtins parsing code with an abstract
 syntax description and provide a parsing core that can parse
 the input and provide the output the builtins need.
 [This is the really big chunk of work.]

  3. Replace the internal parser language (quoting, argument
 passing, variable expansion, conditional syntax etc.).

  4. Rewrite the builtins' syntax descriptions to a new style that
 will hopefully be easier to read, write and parse.

Note that once step 1 is complete, the following steps can be
written and tested in a separate parser plugin, so the code stays
useable even while the new parser is still under construction.
You just switch to the unfinished parser for certain commands.
Once the result is stable, the duplicate code can be eliminated
(or kept around for a while to give people more time to adapt).

Ciao

Dominik ^_^  ^_^

-- 

Dominik Vogt

Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Thomas Adam]

On Wed, Oct 26, 2016 at 02:21:57PM +0200, Dominik Vogt wrote:
> First of all I hope I'll be able to find the crash caused by a free of a 
> string that is still
> used, then experiment with it to see whether a step by step replacement of 
> the old
> parser is feasible while keeping the old parser around for a while.  
> Disentangling the
> builtin functions from the parser without changing their syntax is an
> important step towards a new parser with a new syntax.  As long as all the
> functions do their own parsing, replacing the syntax is a huge amount of
> work.

Yes, I remember that bug when we first started this, but it was a pain to
track down.  Hopefully you'll have more luck.  :)

But what you're describing is the biggest piece of work which would almost
define fvwm3.  Thus far, there's still an on-going discussion about how we're
going to do that.  Heck, we haven't even declared fvwm2-stable or fvwm-2.6.7
yet.  Aren't you jumping the gun a little?

I'd argue---quite strongly---that any work towards a new parser should be done
with breaking everything completely.  Sure, you can still keep some of the
surrounding things, but trying to maintain backwards-compatibility isn't
necessary given the measures we're putting in place for fvwm2's future.

But at the moment, we haven't even discussed what that will look like, or what
requirements we might have.  I started to do that recently when I put out a
proposal for a new syntax to see how that fared.

Oh, and the point of a separate repo still stands, in my eyes.  You might
think it moot, or even an unnecessary point, but I feel it's a very important
one.  It reinforces (psychologically) that it's separate from fvwm2.  It
reinforces that fvwm3 is discrete, and it reinforces the idea that it will
become divergent from fvwm2 quite quickly.  It's not as if we're backporting
features or fixes between the two.

Copy---if it makes you feel any easier---the entirety of the files from fvwm's
master branch to new fvwm3 repository.  Do your work there, the set up is
easy.  Having a separate fvwm3 repository also allows to integrate anything we
wish to the main fvwm website as well (when that becomes appropriate).

Kindly,
Thomas

Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Dominik Vogt]

> Gesendet: Mittwoch, 26. Oktober 2016 um 12:06 Uhr
> Von: "Thomas Adam" 
> An: "Dominik Vogt" 
> Cc: fvwm-workers@fvwm.org
> Betreff: Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing 
> in __execu...
>
> On Wed, Oct 26, 2016 at 12:04:46PM +0200, Dominik Vogt wrote:
> > > On Tue, Oct 25, 2016 at 03:41:45PM -0700, GitHub wrote:
> > > >   Branch: refs/heads/dv/new-parser-2
> > ...
> > > 
> > > I ported all of this over before here:
> > > 
> > > https://github.com/fvwmorg/fvwm/tree/new-parser
> > > 
> > > You should just rebase that against master, and use that if it's any use?
> > 
> > This new branch is the result of rebasing the new-parser branch against 
> > master.  There
> > were some conflicts with the USE_DECOR removal patch, and rebase had some 
> > trouble
> > with the mvwm subdirectory, so I've rewritten the history to eliminate this 
> > directory
> > (and merged some commits for simplicity).
> 
> Cool -- but what do you intend to do with this work?

First of all I hope I'll be able to find the crash caused by a free of a string 
that is still
used, then experiment with it to see whether a step by step replacement of the 
old
parser is feasible while keeping the old parser around for a while.  
Disentangling the
builtin functions from the parser without changing their syntax is an important 
step
towards a new parser with a new syntax.  As long as all the functions do their 
own
parsing, replacing the syntax is a huge amount of work.

Ciao

Dominik ^_^  ^_^

-- 

Dominik Vogt

Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Thomas Adam]

On Wed, Oct 26, 2016 at 12:04:46PM +0200, Dominik Vogt wrote:
> > On Tue, Oct 25, 2016 at 03:41:45PM -0700, GitHub wrote:
> > >   Branch: refs/heads/dv/new-parser-2
> ...
> > 
> > I ported all of this over before here:
> > 
> > https://github.com/fvwmorg/fvwm/tree/new-parser
> > 
> > You should just rebase that against master, and use that if it's any use?
> 
> This new branch is the result of rebasing the new-parser branch against 
> master.  There
> were some conflicts with the USE_DECOR removal patch, and rebase had some 
> trouble
> with the mvwm subdirectory, so I've rewritten the history to eliminate this 
> directory
> (and merged some commits for simplicity).

Cool -- but what do you intend to do with this work?

Thomas

Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Dominik Vogt]

> On Tue, Oct 25, 2016 at 03:41:45PM -0700, GitHub wrote:
> >   Branch: refs/heads/dv/new-parser-2
...
> 
> I ported all of this over before here:
> 
> https://github.com/fvwmorg/fvwm/tree/new-parser
> 
> You should just rebase that against master, and use that if it's any use?

This new branch is the result of rebasing the new-parser branch against master. 
 There
were some conflicts with the USE_DECOR removal patch, and rebase had some 
trouble
with the mvwm subdirectory, so I've rewritten the history to eliminate this 
directory
(and merged some commits for simplicity).

Ciao

Dominik ^_^  ^_^

-- 

Dominik Vogt

Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Thomas Adam]

On Tue, Oct 25, 2016 at 03:41:45PM -0700, GitHub wrote:
>   Branch: refs/heads/dv/new-parser-2
>   Home:   https://github.com/fvwmorg/fvwm
>   Commit: 1d38f7a2727f28f1817e1c951bc07267002d2006
>   
> https://github.com/fvwmorg/fvwm/commit/1d38f7a2727f28f1817e1c951bc07267002d2006
>   Author: Dominik Vogt 
>   Date:   2016-10-25 (Tue, 25 Oct 2016)
> 
>   Changed paths:
> M fvwm/Makefile.am
> M fvwm/add_window.c
> M fvwm/builtins.c
> A fvwm/cmdparser.h
> A fvwm/cmdparser_old.c
> A fvwm/cmdparser_old.h
> M fvwm/colorset.c
> M fvwm/conditional.c
> M fvwm/events.c
> M fvwm/ewmh.c
> M fvwm/ewmh_events.c
> M fvwm/expand.c
> M fvwm/expand.h
> M fvwm/functions.c
> M fvwm/functions.h
> M fvwm/fvwm.c
> M fvwm/fvwm.h
> M fvwm/menucmd.c
> M fvwm/menus.c
> M fvwm/module_interface.c
> M fvwm/move_resize.c
> M fvwm/read.c
> M fvwm/repeat.c
> M fvwm/schedule.c
> M fvwm/update.c
> M fvwm/virtual.c
> M fvwm/windowlist.c
> 
>   Log Message:
>   ---
>   !!! Start work on replacing the parsing in __execute_command_line ...

I ported all of this over before here:

https://github.com/fvwmorg/fvwm/tree/new-parser

You should just rebase that against master, and use that if it's any use?

-- Thomas Adam

Re: [fvwmorg/fvwm] 1d38f7: !!! Start work on replacing the parsing in __execu...

[Dominik Vogt]

I've rebased the new-parser branch to the current master.  I've
not yet tested it, but it compiles and has no mention of "mvwm" in
its history anymore, so coming rebases should be much easier.

@Thomas:  Can you remember what the patches to FScreen.h in ~2 and
~6 are about?

On Tue, Oct 25, 2016 at 03:41:45PM -0700, GitHub wrote:
>   Branch: refs/heads/dv/new-parser-2
>   Home:   https://github.com/fvwmorg/fvwm
>   Commit: 1d38f7a2727f28f1817e1c951bc07267002d2006
>   
> https://github.com/fvwmorg/fvwm/commit/1d38f7a2727f28f1817e1c951bc07267002d2006
>   Author: Dominik Vogt 
>   Date:   2016-10-25 (Tue, 25 Oct 2016)
> 
>   Changed paths:
> M fvwm/Makefile.am
> M fvwm/add_window.c
> M fvwm/builtins.c
> A fvwm/cmdparser.h
> A fvwm/cmdparser_old.c
> A fvwm/cmdparser_old.h
> M fvwm/colorset.c
> M fvwm/conditional.c
> M fvwm/events.c
> M fvwm/ewmh.c
> M fvwm/ewmh_events.c
> M fvwm/expand.c
> M fvwm/expand.h
> M fvwm/functions.c
> M fvwm/functions.h
> M fvwm/fvwm.c
> M fvwm/fvwm.h
> M fvwm/menucmd.c
> M fvwm/menus.c
> M fvwm/module_interface.c
> M fvwm/move_resize.c
> M fvwm/read.c
> M fvwm/repeat.c
> M fvwm/schedule.c
> M fvwm/update.c
> M fvwm/virtual.c
> M fvwm/windowlist.c
> 
>   Log Message:
>   ---
>   !!! Start work on replacing the parsing in __execute_command_line ...
> 
> ... with hook functions that are implemented elsewhere.  This aims
> at allowing to writing and testing multiple parsers in parallel
> and at moving all parsing and expansion stuff to a different
> place.
> 
> 
>   Commit: 71b8dd6064553ad646bfadebfb501a95f7d5da9c
>   
> https://github.com/fvwmorg/fvwm/commit/71b8dd6064553ad646bfadebfb501a95f7d5da9c
>   Author: Dominik Vogt 
>   Date:   2016-10-25 (Tue, 25 Oct 2016)
> 
>   Changed paths:
> M fvwm/cmdparser.h
> M fvwm/cmdparser_old.c
> M fvwm/functions.c
> 
>   Log Message:
>   ---
>   add debug output and fix positional arguments
> 
> 
>   Commit: 4deefa9510ef6da3ca9d2e4d1a90aa59785e4a74
>   
> https://github.com/fvwmorg/fvwm/commit/4deefa9510ef6da3ca9d2e4d1a90aa59785e4a74
>   Author: Dominik Vogt 
>   Date:   2016-10-25 (Tue, 25 Oct 2016)
> 
>   Changed paths:
> M fvwm/cmdparser_old.c
> M fvwm/functions.c
> 
>   Log Message:
>   ---
>   add debug output and fix command line prefixes
> 
> 
>   Commit: 7f357808de11b36d40f86c1d5731597036df56f3
>   
> https://github.com/fvwmorg/fvwm/commit/7f357808de11b36d40f86c1d5731597036df56f3
>   Author: Dominik Vogt 
>   Date:   2016-10-25 (Tue, 25 Oct 2016)
> 
>   Changed paths:
> M fvwm/Makefile.am
> M fvwm/cmdparser.h
> A fvwm/cmdparser_hooks.h
> M fvwm/cmdparser_old.c
> M fvwm/conditional.h
> M fvwm/functable.c
> M fvwm/functable.h
> A fvwm/functable_complex.c
> A fvwm/functable_complex.h
> M fvwm/functions.c
> M fvwm/functions.h
> M fvwm/fvwm.h
> M fvwm/misc.c
> M fvwm/repeat.c
> M fvwm/screen.h
> M libs/FScreen.h
> 
>   Log Message:
>   ---
>   Convert special logic in __execute_command_line for functions ...
> 
> ... that may move a window to a separate function flag.
> 
> Move find_builtin_function() to functable.[ch].
> 
> Moved some functions to functable.[ch] and functable_complex.[ch].
> 
> This separation is needed for the parsing hooks.  It was very difficult to get
> the #icludes right.  The DesktopsInfo structure was moved to libs/FScreen.h
> so that the library does not depend on core header files anymore.
> 
> Move WindowConditionMask from mvwm.h to conditional.h.
> 
> Finish 1st draft of hook implementation (__execute_command_line)
> 
> 
>   Commit: 5f747ebc65614dd91906bb8bfbcf7f73452fa7ec
>   
> https://github.com/fvwmorg/fvwm/commit/5f747ebc65614dd91906bb8bfbcf7f73452fa7ec
>   Author: Dominik Vogt 
>   Date:   2016-10-25 (Tue, 25 Oct 2016)
> 
>   Changed paths:
> M fvwm/cmdparser.h
> M fvwm/cmdparser_hooks.h
> M fvwm/cmdparser_old.c
> M fvwm/expand.c
> M fvwm/functions.c
> 
>   Log Message:
>   ---
>   Split pos_args into a string with all positional args and an array.
> 
> 
>   Commit: 6c1872dd38731d9a5a8f42bcfa5a2b36f5b73149
>   
> https://github.com/fvwmorg/fvwm/commit/6c1872dd38731d9a5a8f42bcfa5a2b36f5b73149
>   Author: Dominik Vogt 
>   Date:   2016-10-25 (Tue, 25 Oct 2016)
> 
>   Changed paths:
> M fvwm/builtins.c
> M fvwm/events.c
> 
>   Log Message:
>   ---
>   Update execute_function calls for libstroke.
> 
> 
>   Commit: fd9bf7044023134b056ca85be61ae45a08345c47
>   
> https://github.com/fvwmorg/fvwm/commit/fd9bf7044023134b056ca85be61ae45a08345c47
>   Author: Dominik Vogt 
>   Date:   2016-10-25 (Tue, 25 Oct 2016)
> 
>   Changed paths:
> M fvwm/cmdparser_old.c
> M