On Sun, Aug 17, 2014 at 11:26:22AM +0100, Thomas Adam wrote:
> On Sat, Aug 16, 2014 at 01:48:39PM +0100, Dominik Vogt wrote:
> > First half of the documentation attached. I've spent all morning
> > writing and need a break now. Whats missing is a list of the
> > actual syntaxes the commands and the modules use.
>
> Wow. This is perfect. I've started analysing this in a lot of detail
> and am in the process of diagramming this since I think that's useful.
Be aware for a few mistakes and oversimplifications though. It's
all coming back from my memory now. In times past I've spent a lot
of time on the parsing, but couldn't fix some of the bugs without
breaking existing scripts. There are some nasty interactions
related to the order of expansion, tokenization and quoting.
> Would you like me to write up the syntaxes the commands and modules use?
That would be grand. I started that yesterday, but we have 184
commands at the moment, and parsing of many is quite complicated.
So, when I noticed that I was becoming more interested in ticking
commands off the list than documenting their syntax properly, I
stopped.
> I've already started, but don't want it to overlap with yourself if
> you're already doing it.
I see. I've attached the new version of the document with a
couple of command syntaxes at the end. If we spend time on this
now, it would be good if we write down the syntax in EBNF right
away and try to identify types of information that is common to
multiple commands. The does not have to be done in all details
yet, for example, a CONDIFION_FLAG rule for the conditional flags
would be enough for now; it can be broken down into the individual
flags later.
Note that the grouping of conditional commands in the attached
file is wrong - I assumed that the syntax would be more or less
identical for them, but noticed that this is not the case.
Ciao
Dominik ^_^ ^_^
--
Dominik Vogt
Overview of fvwm2 parsing as of 16-Aug-2014
===========================================
Sources of input to parse
-------------------------
* Configuration files and files read with the Read command
- May be preprocessed by FvwmCpp or FvwmM4
* Input from the PipeRead command
* Input from modules to fvwm through the module pipes
- Fixed or generated strings from modules (e.g. FvwmPager,
FvwmButtons etc.)
- User input, e.g. from FvwmConsole or FvwmCommand.
* Module input from fvwm through the module pipes is parsed by
the receiving module.
- Binary input in predefined packets is only partially parsed
(packet type). The format and meaning of the packets is a
property of the module protocol.
- The payload of some string packets (e.g. generated by the
SendToModule command) is a free form string that is
interpreted by the receiving module. The module does any
necessary parsing. Example:
SendToModule FvwmButtons changebutton 0 title foo, activetitle bar
* Commands generated internally by fvwm as the result of an
external event, for example, an Ewmh message. This is used
quite often; instead of calling the function to maximize a
window directly, one may generate a command string that is
passed to execute_function(). Examples are
virtual.c:CMD_EdgeResistance() or
ewmh_events.c:ewmh_DesktopGeometry().
Various properties of parser input and the communication channels
-----------------------------------------------------------------
* Maximum length of of lines read from a file is hard coded ro
1024 bytes in read.c:run_command_stream(). This is applicable
to configuration files and the Read and Piperead commands.
* The maximum depth of nested files opened through the Read
command is hard coded to 40 (read.c).
* Input from files or the PipeRead command may have an unlimited
number of lines.
* The maximum length of packets that can be sent in one chunk
from fvwm to the modules or back is hard coded to 256 including
the packet header (FvwmPacketMaxSize in Module.h). This is an
artifact of module communication through pipes. 256 bytes is
the size that all systems guarantee to be sent through the pipe
in an atomic block.
Anything larger than that may be split into multiple write
calls, leading to mutilated messages in either direction.
Packets from fvwm to the modules are simply truncated to 256
bytes (module_interface.c:make_vpacket()). Packets from the
modules to fvwm are possibly sent in multiple pieces (write
call in Module.c:SendText()). Fvwm can theoretically handle
any size of module input (module_list.c:module_receive()).
* Input from modules to fvwm is pre-parsed in
module_interface.c:module_input_execute(), i.e. fvwm checks
whether the command name of the packet is "popup" and does some
special processing if that is the case (this command name is
not to be confused with the command from the text to be
executed by fvwm; its a fixed field in the packet). This is
done before any regular processing of the input.
* Input from the modules is not executed right away but stored in
a queue and executed when fvwm can process it (this is
important because otherwise module input might interfere with
function execution) (module_interface.c:ExecuteCommandQueue()).
Fvwm's central parsing function: execute_function()
---------------------------------------------------
* All input to the parser is passed through
functions.c:__execute_function(). This function is slightly
mis-named. A better name would be _execute_command_line().
* __execute_function is called from two places:
- execute_function() is just a wrapper exported rom
functions.c. There are two more wrappers,
execute_function_override_wcontext() and
execute_function_override_window() which change the execution
context of the command (see below) and then call
execute_function().
- __run_complex_function_items() calls __execute_function() to
process the individual commands of a complex function. In
other words: The items of a complex function are passed twice
through the parser. First when the complex function is
defined, and a second time when they are executed (they are
treated differently in both passes, see further down).
* The call to __execute_function() is passed various pieces of
information:
voisd __execute_function(
cond_rc_t *cond_rc,
const exec_context_t *exc,
char *action,
FUNC_FLAGS_TYPE exec_flags,
char *args[],
Bool has_ref_window_moved)
- cond_rc is a pointer to memory where the return code of
command line execution is to be stored. This can be used in
scripting through complex functions
(__run_complex_function_items()) and by the internal caller
of execute_function. It is also one of the parameters of
the CMD_... functions, and it's important that cond_rc is
passed around and not set to a NULL pointer except by the
first call triggering command line processing.
- exc is the execution context of the command line
(execcontext.h). This is a vital data structure for command
line processing and execution. It contains information
about the originator of the command and the assiciated
application (type (module, event, scheduler, ...), the
actual X event, the module, the window). This data is used
during command execution, e.g. to identify the window on
which a command is executed or to determine the pointer
position; some commands work differently when input comes
from the mouse or the keyboard. It is also used during
variable expansion (e.g. to expand window related extended
parameters like $[w.id]).
It is important to properly set the execution context
whenever a command line is generated. Currently, some of
the Ewmh code fails to do so for historical reasons which I
won't explain here. This needs to be fixed eventually.
- action is the command line to parse and execute.
- exec_flags are flags that affect command parsing and
execution that may become necessary in some contexts not
related to the originator of the command. The flags are
defined in the structure execute_flags_t in functions.h.
~ FUNC_NEEDS_WINDOW commands that need a context window are
not executed if it's missing.
~ FUNC_DONT_REPEAT the command line is not repeatable (see
CMD_Repeat).
~ FUNC_ADD_TO signals that the command is the addtofunc or
"+" command.
~ FUNC_DECOR signals that the command is a decor related
command.
~ FUNC_ALLOW_UNMANAGED allows running a command with an
overrideredirect window.
~ FUNC_IS_UNMANAGED signals that the context window is an
overrideredirect window.
~ FUNC_DONT_EXPAND_COMMAND suppresses expansion of variables
on the command line.
~ FUNC_DONT_DEFER suppresses deferring execution of commands
that need a target window (normally these are put into the
queue and processed later).
- args is the array of the positional arguments during complex
function execution. For calls from execute_function() it's
a NULL pointer.
- has_ref_window_moved is a separate flag (hack) that triggers
some special treatment of the execution of the command Move,
Resize and AnimatedMove (but forgets the ResizeMove and
ResizeMoveMaximize commands). This should ratehr be stored
in exec_flags.
Command line parsing
--------------------
The command independent parsing of any command line is done by the
__execute_function() call. The parsing procedure is as follows.
Remember that "action" is the command line to be parsed. The
pointer and the memory it points to are updated during the parsing
process.
Step 1
Handle NULL pointer, whitespace and comments
(a) Stop parsing and execute nothing if action is a NULL
pointer.
*DONE*
(b) Strip all whitespace from the beginning of the command
line.
(c) Stop parsing and execute nothing if action is a NULL
pointer or an empty string. (Note that some of the
functions in libs/Parse.c never return an empty string but
rather a NULL pointer. Parsing in the individual commands
often relies on that.)
*DONE*
(d) If the first character is '#', is a comment. Stop parsing
and execute nothing.
*DONE*
Step 2
(a) Increment the nested function depth.
(b) If the function depth is too high, print an error message
and stop.
*DONE*
(c) Determine the context window.
Step 3
Handle prefixes
(a) If action begins with '-', set the FUNC_DONT_EXPAND_COMMAND
flag and skip the '-' character.
(b) Peek the first token of action.
(c) If the token is "silent", set the global
Scr.flags.are_functions_silent if not already set, strip
the token from action and go back to (b).
(d) If the token is "keeprc", set the internal do_keep_rc flag,
strip the token from action and go back to (b). (A dummy
return code variable is used in the command execution.)
Step 4
Finish if there is no remaining action to execute.
Step 5
Parse the command token
(a) Get (but not strip) the command token from the action.
(b) Expand variables in the token (expand.c:expand_vars()).
(c) If the token does not start with a '*':
~ strip all characters from and including the first
whitespace from the command token.
[Bug: Complex function names cannot have embedded
whitespace because of this, see comment in the code.]
[Note: If a command line begins with
"*foo"
(including the double quotes), the double quotes are
removed by GetNextToken, and the remaining token does
begin with '*'.]
~ find the internal command matching this token from the
builtin function table. Note that any token _beginning_
with "+" or " +" is treated as the "+" command.
[What the heck is this good for?]
Step 6
If a we're currently adding to a decor, and the command token
designates a builtin function that does not have the FUNC_DECOR
flag set, generate a warning that the command cannot be added to
the decor.
Step 7
If the FUNC_DONT_EXPAND_COMMAND flag is not set, expand
variables in the action, including the part from which the
command token was extracted above. (The ismod flag of the call
to expand_vars() flag is set if the action begins with '*').
[Note: The command token was parsed before expansion in step 5.
Now the whole line including the token is expanded before
further processing. This may or may not cause subtle bugs with
quoting and expansion.]
Step 8
If the expanded action begins with '*' treat it as a module
configuration line.
*GOTO step 10*
[BUG: Contrary to step 5 (c), a line beginning with
"*foo"
is not recognized as a module configuration line because it
begins with double quotes, not '*'.]
Setp 9
Execute the action
(a) Prepare the execution context.
(b) If it's a builtin function other than "Function", strip the
first token from the expanded action and defer or execute
the CMD_<command>() function as necessary with the proper
execution context.
*GOTO step 10*
(c) If it's the builtin command "Function", strip the first
token from the expanded action.
[Note: this may again be a different substring than in step
5 (c) or step 8.]
(d) Call
execute_complex_function with the remaining expanded action.
(e) If no complex function with that name can be found and it
was not the builtin command "Function", assume that the
builtin command was "Module" and try to execute CMD_Module
with the remaining expanded action as its arguments.
Step 10
Cleanup
(a) Clear the Scr.flags.are_functions_silent flag if set in
step 3 (c).
(b) Store the number of pending breaks from the functions
return code structure in the original cond_rc (which may be
a different one in case the command was prefixed with
keeprc).
(c) Decrement the nested function depth.
Tokenization
------------
The token parsing code is in libs/Parse.c:DoPeekToken() and
CopyToken() (called by the former). DoPeekToken takes the input
string, a pointer to memory where it stores the pointer to the
resulting token, a pointer to memory to store the output string
(buffer with hardcoded length MAX_TOKEN_LENGTH = 1023 bytes
(Parse.h)), and may be provided a string of additional characters
to be treated as spaces, a string of input token delimiter
characters, and a pointer where the character that ended the token
is stored (output delimiter). By default, the set of space
characters contains the character class determined by isspace(),
and the set of input delimiter characters is empty
Step 1 (DoPeekToken)
(a) Strip all space characters (see above) from the beginning of
the input string.
(b) Call CopyToken() with the remaining string.
Step 2 (CopyToken)
(a) Set the src pointer to the beginning of the input string.
The dest pointer is passed in as a function argument.
(b) If src is at the end of the string,
*GOTO stp 3*
(c) If *src is a space character or an input delimiter,
*GOTO stp 3*
(d) If *src is a quote character (either a double quote
chararcter, a single quote or a backtick),
*GOTO step 2A*
(e) Otherwise,
*GOTO step 2B*
Step 2A
(a) c := *src
(b) Skip over the src quote character (src++)
(c) If *src is c or the end of string,
*GOTO (f)*
(d) If *src is a backslash and the next character is not the end
of the input string (null byte), skip over the backslash in
the input (src++).
(e) If there's still room in the output string, copy *src to
*dest and increment both pointers, otherwise just skip over
the input character (src++).
*GOTO (c)*
(f) If *src is c, skip over it (src++).
*GOTO step 2 (b)*
Step 2B
(a) If *src is a backslash and the next character is not the end
of the input string (null byte), skip over the backslash in
the input (src++).
(b) If there's still room in the output string, copy *src to
*dest and increment both pointers, otherwise just skip over
the input character (src++).
*GOTO step 2 (b)*
Step 3
(a) Set the output delimiter to the character pointer to by src
(i.e. the first character after the token).
(b) Terminate the destination string with a null byte.
(d) If src points to a string of zero or more spaces followed by
an input delimiter, store the delimiter as the output
delimiter and return a pointer to the character after that.
(e) Otherwise, if src is not at the end of the string, return
the character after src.
(f) Otherwise src points to the end of the string; return src.
Step 4 (DoPeekToken)
(a) If the remaining string has the length zero, set the token
pointer to NULL.
(b) Return a pointer to the beginning of the next token in the
input string.
*DONE*
Variable expansion
------------------
Variable expansion is done expand.c:expand_vars(). The function
takes an input string, the execution context, cond_rc and an array
of positional parameters and returns the string with the variables
expanded. It also takes the flags addto (never set) and ismod
(set in step 7 of the parsing algorithm if the command begins with
'*').
[BUG: The addto flag is actually never set when the function is
called but instead used as a bogus local variable.]
Step 1
(a) l := length of input
(b) If the input begins with '*', set the addto flag.
(c) Calculate the length of the expanded string and allocate
memory for it.
Step 2 (Scan for variables)
(a) Begin at the start of the input and output strings.
(b) Copy all character up to but excluding the next '$'
character to the output string (stop at the end of string).
(c) If we're at the end of the string, stop.
*DONE*
(d) Otherwise we're now at a '$' character. If the ismod flag
set and the next character is a letter, copy both to the
output string.
*GOTO (b)*
[Note: In module configuration lines, variable in the form
"$<letter>" are not expanded. There are probably several
bugs because of this logic.]
Step 3 (Expand a variable)
(a) If the next character is a '$', copy one '$' to the output,
skip over two '$' in the input and
*GOTO step 2 (b)*
(b) If the next character is '[',
*GOTO step 4*
(c) If the "$<character>" sequence does not designate a valid
one character variable, copy it to the output and
*GOTO step 2 (b)*
(d) Otherwise, skip over the '$' and the next character in the
input and copy the value of the one character variable to
the output.
*GOTO step 2 (b)*
Step 4 (Expand an extended variable)
(a) If addto is set, copy the "$[" to the output and
*GOTO step 2 (b)*
(b) Determine the name of the extended variable. The name
starts after the initial '[' character and ends before the
final ']' character. The final ']' character is the first
occurence of the character ']' after the '$', where the
number of '[' minus the number of ']' is zero (i.e., square
brackets can be nested). If the end of string is
encountered, just copy everything to the output and
*GOTO step 2 (b)*
(c) Otherwise, if the variable name contains at least one '$',
call expand vars with the name to expand nested variable
references. The result of this expansion is taken as the
new variable name.
(d) If an extended variable with that name exists, copy its
value into the output, skip the "$[...]" in the input and
*GOTO step 2 (b)*
(e) Otherwise copy the "$[...]" to the output and
*GOTO step 2 (b)*
The second level of parsing
---------------------------
When __execute_function has finished its work it either tries to
execute a module, a builtin command or a complex function (or
nothing at all and just returns). Module execution is treated
like a call of CMD_Module(), so there are two different ways how
parsing continues. A third context of parsing is, when a module
receives a packet from fvwm.
* Parsing of builtin commands is done individually by the
CMD_<builtin>() functions.
* Parsing and of a complex function call and its items is handled
by the function functions.c:execute_complex_function().
* Parsing of module configuration lines is done by the modules
with help from libs/Modules.c. There is also an optional step
of module variable expansion that is implemented in
Modules.c:module_expand_action(). It replaces some module
variables in the string that is going to be sent to fvwm,
e.g. "$width" or "$x". FvwmButtons, FvwmIconMan and
FvwmTaskBar and use this mechanism.
Parsing of complex function calls
---------------------------------
Implemented in functions.c:execute_complex_function().
Step 1
(a) Peek the first token of the action.
(b) Look it up in the list of complex functions.
(c) If no such function exists, return an error.
*DONE*
(d) Split the action into tokens using GetNextToken. Store the
original action (without the function name) followed by the
first ten tokens in the positional arguments array.
(e) Set up an execution context for the function items.
(f) Call __run_complex_function_items().
Step 2 (__run_complex_function_items())
(a) For each complec function iten, call __execute_function()
with the FUNC_DONT_DEFER flag and the list of positional
arguments. This causes the function item command text to be
passed through the parser again.
*DONE*
Parsing needs of the builtin commands
-------------------------------------
* AddButtonStyle
!!!
* AddTitleStyle
!!!
* AddToDecor
!!!
* AddToFunc
!!!
* AddToMenu
!!!
* All
!!!
* Any, Current, Next, None, PointerWindow, Prev [(CONDITION)] COMMAND
CONDITION: WINDOWNAMEPATTERN | [!]KEYWORD [, [!]KEYWORD] ...
WINDOWNAMEPATTERN: TOKEN
KEYWORD: !!!
COMMAND: RESTOFLINE
* AnimatedMove
!!!
* Asterisk
!!!
* Beep
!!!
* BorderStyle
!!!
* Break
!!!
* BugOpts
!!!
* BusyCursor
!!!
* ButtonState
!!!
* ButtonStyle
!!!
* ChangeDecor
!!!
* ChangeMenuStyle MENUSTYLENAME MENUNAME
MENUSTYLENAME: TOKEN
MENUNAME: TOKEN
* CleanupColorsets VOID
* ClickTime INT:0+
(stores the negative value during startup>
* Close, Delete, Destroy VOID
(work on context window)
* ColorLimit
(obsolete, has been removed)
* ColormapFocus "FollowsMouse" | "FollowsFocus"
* Colorset INT:o- OPTION [, OPTION] ...
OPTION: KEYWORD [VALUE] ...
KEYWORD: "fg" | "Fore" | "Foreground" | "bg" | "Back" | "Background" |
"hi" | "Hilite" | "Hilight | "sh" | "Shade" | "Shadow" | "fgsh" |
"Pixmap" | "TiledPixmap" | "AspectPixmap" | "Transparent" |
"RootTransparent" | "Shape" | "TiledShape" | "AspectShape" |
"NoShape" | "[V|B|D|S|C|R|Y]Gradient" | "Tint" | "fgTint" | "bgTint" |
"Alpha" | "fgAlpha" | "Dither" | "NoDither" | "IconTint" |
"IconAlpha" | "Plain"
VALUE: INT | COLOURNAME | IMAGEFILENAME | PERCENTAGE:0-100
(memory usage depends on highest colorset number)
* CopyMenuStyle MENUSTYLENAME MENUSTYLENAME
MENUSTYLENAME: TOKEN
* CursorMove
!!!
* CursorStyle
!!!
* DefaultColors
!!!
* DefaultColorset
!!!
* DefaultFont
!!!
* DefaultIcon
!!!
* DefaultLayers
!!!
* Deschedule
!!!
* Desk
!!!
* DesktopName
!!!
* DesktopSize
!!!
* DestroyDecor
!!!
* DestroyFunc
!!!
* DestroyMenu
!!!
* DestroyMenuStyle MENUSTYLENAME
MENUSTYLENAME: TOKEN
* DestroyModuleConfig
!!!
* DestroyStyle
!!!
* DestroyWindowStyle
!!!
* Direction
!!!
* Echo
!!!
* EchoFuncDefinition
!!!
* EdgeCommand
!!!
* EdgeLeaveCommand
!!!
* EdgeResistance
!!!
* EdgeScroll
!!!
* EdgeThickness
!!!
* Emulate
!!!
* EscapeFunc
!!!
* EwmhBaseStruts
!!!
* EwmhNumberOfDesktops
!!!
* Exec
!!!
* ExecUseShell
!!!
* FakeClick
!!!
* FakeKeypress
!!!
* FlipFocus
!!!
* Focus
!!!
* FocusStyle
!!!
* Function
!!!
* GlobalOpts
!!!
* GnomeButton
!!!
* GnomeShowDesks
!!!
* GotoDesk
!!!
* GotoDeskAndPage
!!!
* GotoPage
!!!
* HideGeometryWindow
!!!
* HilightColor
!!!
* HilightColorset
!!!
* IconFont
!!!
* IconPath
!!!
* Iconify
!!!
* IgnoreModifiers
!!!
* ImagePath
!!!
* InfoStoreAdd
!!!
* InfoStoreRemove
!!!
* KeepRc
!!!
* Key
!!!
* KillModule
!!!
* Layer
!!!
* LocalePath
!!!
* Lower, Raise, RaiseLower
!!!
* Maximize
!!!
* Menu
!!!
* MenuStyle
!!!
* Module
!!!
* ModuleListenOnly
!!!
* ModulePath
!!!
* ModuleSynchronous
!!!
* ModuleTimeout
!!!
* Mouse
!!!
* Move
!!!
* MoveThreshold
!!!
* MoveToDesk
!!!
* MoveToPage
!!!
* MoveToScreen
!!!
* NoWindow
!!!
* Nop
!!!
* OpaqueMoveSize
!!!
* Pick
!!!
* PipeRead
!!!
* PixmapPath
!!!
* PlaceAgain
!!!
* Plus
!!!
* PointerKey
!!!
* Popup
!!!
* PrintInfo
!!!
* Quit
!!!
* QuitScreen
!!!
* QuitSession
!!!
* Read
!!!
* Recapture
!!!
* RecaptureWindow
!!!
* Refresh
!!!
* RefreshWindow
!!!
* Repeat
!!!
* Resize
!!!
* ResizeMaximize
!!!
* ResizeMove
!!!
* ResizeMoveMaximize
!!!
* RestackTransients
!!!
* Restart
!!!
* SaveQuitSession
!!!
* SaveSession
!!!
* ScanForWindow
!!!
* Schedule
!!!
* Scroll
!!!
* SendToModule
!!!
* SetAnimation
!!!
* SetEnv
!!!
* Silent
!!!
* SnapAttraction
!!!
* SnapGrid
!!!
* State
!!!
* Stick
!!!
* StickAcrossDesks
!!!
* StickAcrossPages
!!!
* Stroke
!!!
* StrokeFunc
!!!
* Style
!!!
* TearMenuOff
!!!
* Test
!!!
* TestRc
!!!
* ThisWindow
!!!
* Title
!!!
* TitleStyle
!!!
* UnsetEnv
!!!
* UpdateDecor
!!!
* UpdateStyles
!!!
* Wait
!!!
* WarpToWindow
!!!
* WindowFont
!!!
* WindowId
!!!
* WindowList
!!!
* WindowShade
!!!
* WindowShadeAnimate
!!!
* WindowStyle
!!!
* WindowsDesk
!!!
* XSync
!!!
* XSynchronize
!!!
* Xinerama
!!!
* XineramaPrimaryScreen
!!!
* XineramaSls
!!!
* XineramaSlsScreens
!!!
* XineramaSlsSize
!!!
* XorPixmap
!!!
* XorValue
!!!
