Hi everyone!
As I promissed, I worked this weekend to finish the documentation for the
literate programming patches.
Please take a look and tell me what you think.
I also did one more patch to the ShowLatexLog function. This patch is to enable
literate programers
to view the "Build Program" log file. I did not added a new entry in the File
menu, instead I just check
if both log files are present and if that is the case, I load the newest one.
Simple and Small.
This patch was done against version 1.0.0
Regards
Edmar
--
/*----------------------------------------------------------------------*/
/* Edmar Wienskoski Jr. - [EMAIL PROTECTED] */
/* - http://www.cs.rice.edu/~wiensk */
/*----------------------------------------------------------------------*/
____
| [] | ______()_||_
---+----+--- ------------ ------------ ------------ | [] |
| | | | | | | | | | ___| |
|_|______|_| |__________| |__________| |__________| |______________\
"o-o o-o""o-o o-o""o-o o-o""o-o o-o""o-o O-O-O o-o "
#This file was created by <edmar> Mon Feb 22 16:21:02 1999
#LyX 1.0 (C) 1995-1998 Matthias Ettrich and the LyX Team
\lyxformat 2.15
\textclass literate-article
\begin_preamble
\end_preamble
\language default
\inputencoding default
\fontscheme default
\graphics default
\paperfontsize default
\spacing single
\papersize Default
\paperpackage a4
\use_geometry 0
\use_amsmath 0
\paperorientation portrait
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\defskip medskip
\quotes_language english
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default
\layout Title
LyX and Literate Programming
\layout Author
Edmar Wienskoski Jr.
\newline
[EMAIL PROTECTED]
\layout Date
\latex latex
\backslash
today
\layout Standard
\begin_inset LatexCommand \tableofcontents
\end_inset
\layout Section
Introduction
\layout Standard
The main purpose of this documentation is to show how to use LyX for literate
programming.
I assume that you are familiar with this programming technique, and now
what
\begin_inset Quotes eld
\end_inset
tangling
\begin_inset Quotes erd
\end_inset
and
\begin_inset Quotes eld
\end_inset
weaving
\begin_inset Quotes erd
\end_inset
means.
If that is not the case, please follow the web links provided in the following
sections.
There is a lot of good documentation out there covering from development
history to latest tools tips.
\layout Standard
I also assume that you are familiar with LyX itself to a point that you
are comfortable changing your lyxrc file, and X resources file.
If that is not the case please refer to other LyX documentation to cover
your specific needs.
\layout Subsection
Literate Programming
\layout Standard
>From the Literate Programming FAQ :
\layout Quotation
Literate programming is the combination of documentation and source together
in a fashion suited for reading by human beings.
In fact, literate programs should be enjoyable reading, even inviting!
(Sorry Bob, I couldn't resist!) In general, literate programs combine source
and documentation in a single file.
Literate programming tools then parse the file to produce either readable
documentation or compilable source.
The WEB style of literate programming was created by D.E.
Knuth during the development of his TeX typesetting software.
\layout Standard
Another excerpt says:
\layout Quotation
\emph on
How is literate programming different from verbose commenting?
\layout Quotation
There are three distinguishing characteristics.
In order of importance, they are:
\begin_deeper
\layout Itemize
flexible order of elaboration
\layout Itemize
automatic support for browsing
\layout Itemize
typeset documentation, especially diagrams and mathematics
\end_deeper
\layout Standard
Now that I tickled your curiosity, take a look in the references.
\layout Subsection
References
\layout Standard
The complete Literate Programming FAQ can be found at:
\layout Quote
http://shelob.ce.ttu.edu/daves/faq.html
\layout Standard
The FAQ lists 23 (twenty three !) different literate programming tools.
Some are specialized or
\begin_inset Quotes eld
\end_inset
tailored
\begin_inset Quotes erd
\end_inset
for particular programming languages, other have general scope.
I particularly selected
\noun on
Noweb
\noun default
for my own use for several reasons:
\layout Itemize
It can generate the documentation either in latex or html.
\layout Itemize
It is a open architecture, i.e., it is easy to plug in new filters and do
special processing that you may need.
\layout Itemize
There is a good selection of filters available already (the html is one
of them).
\layout Itemize
It is free.
\layout Standard
The Noweb web page can be found at:
\layout Quote
http://www.cs.virginia.edu/~nr/noweb
\layout Standard
Starting from there you can reach many other interesting links, even literate
program examples.
\layout Section
LyX and Literate Programming
\layout Standard
I did some changes to LyX to enable it interact with Noweb.
Those changes were rather simple, and I tried to do them in a most
\begin_inset Quotes eld
\end_inset
Noweb independent
\begin_inset Quotes erd
\end_inset
way, i.e., I expect that you can use LyX with some other literate programming
tool of your choice by just re-configuring your lyxrc file.
\layout Subsection
Generating documents and code (weaving and tangling)
\layout Subsubsection
Selecting the document class
\layout Standard
If you have installed Noweb and LyX successfully, whenever you open a new
document or try to change the document class of an existing one, you will
find three new document class available to choose from:
\layout Itemize
Article (Noweb)
\layout Itemize
Book (Noweb)
\layout Itemize
Report (Noweb)
\layout Standard
You must choose one of the three to create your literate documents from.
\layout Standard
Note however, that literate documents are not limited to these three classes.
It can be done with any other style like letter or slides or combined with
other class variations like Article (AMS).
I limited to these three because I didn't want to clog the list of options
with all possible variations.
If you have special needs that cannot be covered by one of the three classes
above, let me know and I can arrange to insert a new entry, or teach you
how to do it.
\begin_float footnote
\layout Standard
It is very simple, it involves a creation of a file with four lines, and
running again the auto configuration.
\end_float
Moreover, if you use other literate tool than Noweb you may need to create
new document classes for it.
\layout Subsubsection
Typing code in
\layout Standard
LyX enables you to write code with a layout named
\noun on
Scrap
\noun default
.
\begin_float footnote
\layout Standard
The equivalent Noweb term is
\begin_inset Quotes eld
\end_inset
Chunk
\begin_inset Quotes erd
\end_inset
.
For historical reasons, I got used to the term
\begin_inset Quotes eld
\end_inset
scrap
\begin_inset Quotes erd
\end_inset
introduced by other literate tool named Nuweb, which I used for many years
before rendering myself to Noweb.
\end_float
Noweb delimits scraps like this
\layout LyX-Code
<<My scrap>>=
\layout LyX-Code
\protected_separator
code
\layout LyX-Code
\protected_separator
more code
\layout LyX-Code
\protected_separator
even more code
\layout LyX-Code
\protected_separator
@
\layout Standard
The problem is that whatever is written in between the << and the
\family typewriter
@
\family default
must be taken literally, i.e., LyX should be prevented to do any special
interpretation of what is written there.
For that reason, I defined a new layout named Scrap, that works like a
LaTeX paragraph but has a free spacing capability.
In other words, it is a combination of the LaTeX and the LyX-Code layouts.
\layout Standard
The down side of this
\begin_inset Quotes eld
\end_inset
semi-hack
\begin_inset Quotes erd
\end_inset
is that consecutive paragraphs of code will be spaced with one empty line
in the source code and also in the printed documentation.
Of course I don't want this, the work around is to enter each line of code
within a single scrap, with a newline (ctrl-return).
The example above will look like this:
\begin_float footnote
\layout Standard
If you have a printed version of this document you will not see any diference
from the previous example and this one.
\end_float
\layout LyX-Code
<<My scrap>>=
\newline
\protected_separator
code
\newline
\protected_separator
more code
\newline
\protected_separator
even more code
\newline
\protected_separator
@
\layout Standard
This layout works fine, the only real inconvenience is that you have to
type ctrl-return instead of a plain return.
\begin_float footnote
\layout Standard
It is in my list of
\begin_inset Quotes eld
\end_inset
improvements
\begin_inset Quotes erd
\end_inset
to fix that.
\end_float
IMHO it is a small price to pay for the benefits LyX brings to you for
typing, compiling, and debugging your literate program.
\layout Subsubsection
Generating the documentation
\layout Standard
At this point you already have a new document file with a proper document
class, and with some code and text on it.
How do I print it ? The answer is simple, you select
\begin_inset Quotes eld
\end_inset
Update dvi
\begin_inset Quotes erd
\end_inset
or
\begin_inset Quotes eld
\end_inset
Update PostScript
\begin_inset Quotes erd
\end_inset
or
\begin_inset Quotes eld
\end_inset
View dvi
\begin_inset Quotes erd
\end_inset
, etc.
Just like you would do for a plain document.
No special procedure is required.
\layout Standard
To situate you, I will explain now what happens inside LyX:
\layout Enumerate
When the "Update dvi" menu option is chosen, a latex file is generated.
\begin_deeper
\layout Standard
If the document is of any literate class the generated file will be named
with an extension name defined by
\family typewriter
\backslash
literate_extension
\family default
, otherwise the file will have the usual
\family typewriter
.tex
\family default
extension.
\end_deeper
\layout Enumerate
Note that the distinction so far is in the name of the file, no special
processing is required by LyX.
Remember that you typed the code using the Scrap layout.
That by itself takes care of the business.
\layout Enumerate
If the document is of any literate class LyX will execute:
\begin_deeper
\layout LyX-Code
\begin_inset Quotes eld
\end_inset
\backslash
literate_command
\begin_inset Quotes erd
\end_inset
<
\emph on
filename
\emph default
>
\emph on
filename
\emph default
.out
\layout LyX-Code
\begin_inset Quotes eld
\end_inset
\backslash
literate_error_filter
\begin_inset Quotes erd
\end_inset
<
\emph on
filename
\emph default
.out >
\emph on
filename
\emph default
.log
\layout Standard
Otherwise it will just skip this step.
\end_deeper
\layout Enumerate
Finally, LaTeX is invoked and the regular post processing continues as in
a plain document.
\layout Standard
The purpose of the
\family typewriter
\backslash
literate_command
\family default
is to transform the newly created
\family typewriter
.nw
\family default
file into a
\family typewriter
.tex
\family default
file.
\layout Standard
The purpose of the
\family typewriter
\backslash
literate_error_filter
\family default
command is to help LyX display error boxes in case that the
\family typewriter
\backslash
literate_command
\family default
reports any error.
This filter can be a C program, a script, or any thing you want.
Its job is to identify error messages from your literate tool, in our case
Noweb, and convert them to a common format that LyX understands.
The last chapter in this document has a C program that can be used to parse
Noweb error messages.
This scheme introduces on more command for the user to configure, but has
the advantage to be
\begin_inset Quotes eld
\end_inset
literate tool
\begin_inset Quotes erd
\end_inset
independent.
Moreover, if you don't bother to set it, you can still use the system,
you just loose the ability to have a nice display and interaction with
the error messages.
\layout Subsubsection
Generating the code
\layout Standard
When the "Build Program" File menu option is chosen or the corresponding
button in the toolbar is pressed, a latex file with extension
\family typewriter
\backslash
literate_extension
\family default
is generated just like step 1 above.
Then LyX invokes
\family typewriter
\backslash
build_command
\family default
to generate the code and
\family typewriter
\backslash
build_error_filter
\family default
to process the compilation error messages, as in step 3 above.
\layout Standard
Note that no special processing is required from LyX.
\layout Standard
Let me remind that I am assuming that you are familiar with Noweb.
In this case you know what to do to get a executable file starting from
a noweb file (
\family typewriter
.nw
\family default
extension).
It is likely that you are going to create a makefile for that purpose.
For that reason, after LyX creates the noweb file, it invokes the command
defined by
\family typewriter
\backslash
build_program
\family default
, which default is
\begin_inset Quotes eld
\end_inset
make
\begin_inset Quotes erd
\end_inset
.
In case you want to process your literate file with a script, or some other
program, just insert in your lyxrc file a entry with:
\layout LyX-Code
\backslash
build_command
\begin_inset Quotes eld
\end_inset
my_script my_arguments
\begin_inset Quotes erd
\end_inset
\layout Standard
The
\family typewriter
\backslash
build_error_filter
\family default
differs from the
\family typewriter
\backslash
literate_error_filter
\family default
only in that the former will identify error messages from your compiler.
Again, the last chapter has a C program that can be used to parse gcc,
or xlc error messages.
If your compiler is other than those you may want to change this program
or create a new one just for your case.
Once again, the same comments from the previous section are valid here:
this is optional, you can skip it if you can afford to loose a the LyX
error handling feature.
\layout Standard
In summary, the "Build Program" internal function sequence is pretty much
like the "Update dvi" one but involving different commands.
See schematic:
\layout LyX-Code
"Update dvi"
\protected_separator
\protected_separator
-> generate .tex
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
-> invoke -> show latex errors
\layout LyX-Code
on plain doc
\protected_separator
\protected_separator
\protected_separator
file
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\backslash
latex
\layout LyX-Code
\protected_separator
\protected_separator
\layout LyX-Code
"Update dvi"
\protected_separator
\protected_separator
\protected_separator
-> generate same -> invoke
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
-> invoke -> show latex errors
\layout LyX-Code
on literate doc
\protected_separator
\protected_separator
file with .nw
\protected_separator
\protected_separator
\backslash
literate_command
\backslash
\protected_separator
\backslash
latex
\layout LyX-Code
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
extension
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\backslash
_______ -> show literate errors
\layout LyX-Code
\protected_separator
\protected_separator
\layout LyX-Code
"Build Program"
\protected_separator
-> generate same -> invoke
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
-> show compilation error
\layout LyX-Code
on literate doc
\protected_separator
\protected_separator
file with .nw
\protected_separator
\protected_separator
\backslash
build_command
\layout LyX-Code
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
extension
\protected_separator
\protected_separator
\protected_separator
\protected_separator
( make )
\layout Subsection
Configuring LyX
\layout Standard
There are five new controls that you can put in your lyxrc file:
\layout Description
\family typewriter
\backslash
literate_command
\family default
This specify which literate tool you are going to use.
During normal LyX configuration, the auto configuration script you try
to find the noweb package for you.
If the search is successful, the default value for this control is set
to:
\begin_deeper
\layout LyX-Code
noweave -delay -index
\layout Standard
Otherwise the default value is set to
\begin_inset Quotes eld
\end_inset
none
\begin_inset Quotes erd
\end_inset
.
\end_deeper
\layout Description
\family typewriter
\backslash
literate_extension
\family default
This specify the filename extension to be used for literate documents.
Some literate tools requires specific file extension names.
The default is set during configuration as either
\begin_inset Quotes eld
\end_inset
.nw
\begin_inset Quotes erd
\end_inset
or
\begin_inset Quotes eld
\end_inset
none
\begin_inset Quotes erd
\end_inset
, depending on whether the auto configuration found the noweb package or
not.
\layout Description
\family typewriter
\backslash
build_command
\family default
Defaults to
\begin_inset Quotes eld
\end_inset
make
\begin_inset Quotes erd
\end_inset
.
You need to create a makefile to do the tangling, compilation, etc.
\begin_deeper
\layout Standard
Another way is to maintain the makefile or building script itself as part
of the document.
That way every project can be compiled with the same command line:
\layout LyX-Code
notangle -Rbuild-script
\emph on
filename
\emph default
.nw | sh
\layout Standard
To use this scheme, create a script named
\family typewriter
project-build
\family default
that executes the command above:
\layout LyX-Code
#!/bin/sh
\layout LyX-Code
notangle -Rbuild-script $1 | sh
\layout Standard
Finally set LyX to execute the
\family typewriter
project-build
\family default
script:
\layout LyX-Code
\backslash
build_command project-build
\layout Standard
Every time you start a new project, define a scrap with name
\family typewriter
build-script
\family default
that do the compilation for that project.
Whenever you select
\begin_inset Quotes eld
\end_inset
Build Program
\begin_inset Quotes erd
\end_inset
, LyX will extract that scrap (
\family typewriter
notangle -Rbuild-script...
\family default
) and execute it (
\family typewriter
...|sh
\family default
).
\end_deeper
\layout Description
\family typewriter
\backslash
literate_error_filter
\family default
Defaults to
\begin_inset Quotes eld
\end_inset
cat
\begin_inset Quotes erd
\end_inset
.
If you compile the program given in the next chapter you can set:
\begin_deeper
\layout LyX-Code
\backslash
literate_error_filter
\begin_inset Quotes eld
\end_inset
listerrors n
\begin_inset Quotes erd
\end_inset
\layout Standard
to process Noweb(noweave) error messages
\end_deeper
\layout Description
\family typewriter
\backslash
build_error_filter
\family default
Defaults to
\begin_inset Quotes eld
\end_inset
cat
\begin_inset Quotes erd
\end_inset
.
If you compile the program given in the next chapter you can set:
\begin_deeper
\layout LyX-Code
\backslash
build_error_filter
\begin_inset Quotes eld
\end_inset
listerrors b
\begin_inset Quotes erd
\end_inset
\layout Standard
to process both Noweb(notangle) error messages and gcc error messages.
\end_deeper
\layout Subsection
Debug extensions
\layout Standard
I also implemented a new function in the LyX server: the
\begin_inset Quotes eld
\end_inset
server-goto-file-row" function, to be used with ddd/gdb or other debugger.
\layout Standard
When debugging code with ddd/gdb, it is possible to invoke a text editor
at the current execution position with a single key stroke.
The default ddd configuration for that is shift-ctrl-V.
It happens that you can define the editor command line invocation in ddd
by accessing the edit->preferences->helpers window and changing the "Edit
Sources" entry.
\layout Standard
I take advantage of the new created LyX server function and this ddd feature,
and set
\begin_inset Quotes eld
\end_inset
Edit Sources
\begin_inset Quotes erd
\end_inset
to:
\layout Quote
echo "LYXCMD:monitor:server-goto-file-row:@FILE@ @LINE@" >~/.lyxpipe.in
\layout Standard
With this, whenever you are using ddd and find a point in the program that
you want to edit, you just press shift-ctrl-V (in the ddd window), and
ddd you forward this information to LyX through the LyX server and then
the LyX window will show the same file with the cursor at the same position
ddd was pointing to.
No more guessing nor long scrolling to locate a point in the program back
from debugging !
\layout Standard
Note however that you must enable the LyX server to get this feature working
(it is disabled by default).
To enable it, insert in your lyxrc file:
\layout Quote
\backslash
serverpipe "/home/<your home directory>/.lyxpipe"
\layout Standard
Read the LyX server documentation for further information.
\layout Subsection
Toolbar extensions
\layout Standard
I created six new buttons that can be added to your LyX toolbar.
Five of these buttons are short cuts to layout styles: Standard, Section,
LaTeX, LyX-Code, and Scrap.
I created those because they are the ones I use most and I was tired of
scrolling the layout menu looking for the one I wanted.
The last one is a short cut to the
\begin_inset Quotes eld
\end_inset
Build Program
\begin_inset Quotes erd
\end_inset
File menu entry.
\layout Standard
LyX has a range of buttons that are available for tool bar customization.
In my toolbar I like to combine the six short cuts above with two more:
One for
\begin_inset Quotes eld
\end_inset
Update dvi
\begin_inset Quotes erd
\end_inset
and the other for
\begin_inset Quotes eld
\end_inset
View dvi
\begin_inset Quotes erd
\end_inset
File menu entries.
Here is how it looks like:
\layout LyX-Code
\backslash
begin_toolbar
\layout LyX-Code
\backslash
layouts
\layout LyX-Code
\backslash
add layout Standard
\layout LyX-Code
\backslash
add layout Section
\layout LyX-Code
\backslash
add layout LaTeX
\layout LyX-Code
\backslash
add layout LyX-Code
\layout LyX-Code
\backslash
add layout Scrap
\layout LyX-Code
\backslash
separator
\layout LyX-Code
\backslash
add buffer-view
\layout LyX-Code
\backslash
add buffer-typeset
\layout LyX-Code
\backslash
add build-program
\layout LyX-Code
\backslash
separator
\layout LyX-Code
.
\layout LyX-Code
.
\layout LyX-Code
.
\layout LyX-Code
\backslash
end_toolbar
\layout Subsection
Colors customization
\layout Standard
There are a number of colors in LyX that can be customized.
For each one there are two ways to do it: Either by inserting a line in
your X resource files, or by adding a flag in the LyX command line.
\layout Standard
One of things that bother people is the LaTeX font color.
The default color is red, since the scraps uses LaTeX font, and there is
a lot of scraps in literate documents, you may get tired to see everything
in red.
Example:
\layout Itemize
In the X resource file:
\begin_deeper
\layout Quote
lyx*LatexColor: gold
\end_deeper
\layout Itemize
In the command line:
\begin_deeper
\layout Quote
lyx -LatexColor gold
\end_deeper
\layout Standard
The next thing is the visible presence of the newline character in the screen.
You can choose the color of this particular character and make it blend
in the background.
I recommend you choosing a color that are close to the background but not
equal, such that you still can see it is there, but it is not bothering
you anymore.
Example:
\layout Itemize
In the X resource file:
\begin_deeper
\layout Quote
lyx*BackgroundColor: gray20
\layout Quote
lyx*NewLineColor: gray35
\end_deeper
\layout Itemize
In the command line:
\begin_deeper
\layout Quote
lyx -BackgroundColor gray20 -NewLineColor gray35
\end_deeper
\layout Standard
If you don't like a bright clear background burning your eyes all day long,
you can reverse background and foreground colors with this:
\layout Itemize
In the X resource file:
\begin_deeper
\layout Quote
lyx*Reverse: 1
\end_deeper
\layout Itemize
In the command line:
\begin_deeper
\layout Quote
lyx -Reverse 1
\end_deeper
\layout Standard
The problem with all those color alterations is that you may need to change
other colors more to insure that math, inset, etc, are
\begin_inset Quotes eld
\end_inset
normal
\begin_inset Quotes erd
\end_inset
in respect to the rest.
The following is the selection I use in my own X resource file.
It is just a suggestion.
I encourage you to try and find the color set that work best for you.
\layout LyX-Code
lyx*Reverse:
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
1
\layout LyX-Code
lyx*MathColor:
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
green
\layout LyX-Code
lyx*LatexColor:
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
gold
\layout LyX-Code
lyx*OnOffLineColor:
\protected_separator
\protected_separator
\protected_separator
\protected_separator
magenta
\layout LyX-Code
lyx*InsetColor:
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
yellow
\layout LyX-Code
lyx*LightedColor:
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
gray50
\layout LyX-Code
lyx*NewLineColor:
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
gray35
\layout LyX-Code
lyx*BackgroundColor:
\protected_separator
\protected_separator
\protected_separator
gray20
\layout Section
Example Program
\layout Standard
We mentioned in the previous section the existence of a program that filters
the compilation errors and put them in a format suitable for LyX reading
it.
Although it is likely that future releases of LyX will incorporate this
code, the filtering mechanism itself is essential to allow fully utilization
of the literate programming tools (remember that those tools can be used
with any kind of language and any kind of compiler).
\layout Standard
As new formats are incorporated into LyX, more and more users will be able
to rely into the default LyX configuration (which is the Unix program
\emph on
cat
\emph default
).
The need for a external program will then be restricted for not so common
languages / tools, and the program in this section will serve merely as
a example.
\layout Subsection
Introduction
\layout Standard
In this section we present a filter that recognizes compilation error messages
from noweb, gnu C, and the IBM C compiler (xlc).
\layout Standard
The filter is required to read from standard input, parse the input for
error messages and copy the error messages to the standard output.
During the output process, the program must give the error messages in
a format that LyX can interpret.
Today this format is the format that LaTeX gives the error messages.
Of course, nothing prevents that future LyX releases be able to read other
formats as well (like gcc error messages for example).
The idea is that this mechanism is necessary to fully explore the literate
programming tools capabilities.
\layout Subsection
Algorithm
\layout Scrap
<<Function bodies>>=
\newline
int
\newline
main (int argc, char **argv)
\newline
{
\newline
\protected_separator
if (argc == 2) {
\newline
\protected_separator
\protected_separator
\protected_separator
switch (argv[1][0]) {
\newline
\protected_separator
\protected_separator
\protected_separator
case 'n':
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
<<Scan input for Noweb error messages>>
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
break;
\newline
\protected_separator
\protected_separator
\protected_separator
case 'x':
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
<<Scan input for Xlc error messages>>
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
break;
\newline
\protected_separator
\protected_separator
\protected_separator
case 'a':
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
<<AIX system using both Noweb and Xlc>>
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
break;
\newline
\protected_separator
\protected_separator
\protected_separator
case 's':
\newline
\protected_separator
\protected_separator
\protected_separator
case 'b':
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
<<Solaris and Linux systems using both Noweb and Gcc>>
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
break;
\newline
\protected_separator
\protected_separator
\protected_separator
case 'g':
\newline
\protected_separator
\protected_separator
\protected_separator
default:
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
<<Scan input for Gcc error messages>>
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
break;
\newline
\protected_separator
\protected_separator
\protected_separator
}
\newline
\protected_separator
} else {
\newline
\protected_separator
\protected_separator
\protected_separator
<<Scan input for Gcc error messages>>
\newline
\protected_separator
}
\newline
}
\newline
@
\layout Scrap
<<Function prototypes>>=
\newline
int main (int argc, char **argv);
\newline
@
\layout Subsection
Data Structures
\layout Standard
We resort to some global variables to allow access from several different
routines.
Those are the buffer and related pointers used during the parse of the
input.
\layout Scrap
<<Global variables>>=
\newline
char
\protected_separator
\protected_separator
\protected_separator
buffer[200][200];
\newline
int
\protected_separator
\protected_separator
\protected_separator
\protected_separator
last_buf_line;
\newline
int
\protected_separator
\protected_separator
\protected_separator
\protected_separator
last_err_line;
\newline
int
\protected_separator
\protected_separator
\protected_separator
\protected_separator
err_line;
\newline
@
\layout Subsection
The output format
\layout Standard
The output format mimics the LaTeX error messages format.
This function prints a number of lines residing in the global variable
\family typewriter
buffer
\family default
, a program name and line number.
There is no special requirement on the input strings, they can be anything.
\layout Scrap
<<Function bodies>>=
\newline
void
\newline
output_error (int buf_size, int error_line, char *tool)
\newline
{
\newline
\protected_separator
int
\protected_separator
\protected_separator
\protected_separator
\protected_separator
i;
\newline
\protected_separator
\newline
\protected_separator
fprintf(stdout, "! Build Error: ==> %s ==>
\backslash
n", tool);
\newline
\protected_separator
for (i=0; i<buf_size; i++)
\newline
\protected_separator
\protected_separator
\protected_separator
fprintf(stdout, "%s", buffer[i]);
\newline
\protected_separator
fprintf(stdout, " ...
\backslash
n
\backslash
nl.%d ...
\backslash
n
\backslash
n", error_line);
\newline
}
\newline
@
\layout Scrap
<<Function prototypes>>=
\newline
void output_error (int buf_size, int error_line, char *tool);
\newline
@
\layout Subsection
Functions Implementation
\layout Standard
Both noweave and notangle routines, always output one single line for each
error found, thus to scan the buffer for Noweb error messages is enough
to exam one input line at a time.
Note that the noweb software does not provide a line error number, so all
errors boxes related to Noweb messages will be displayed at the beginning
of the file.
\layout Scrap
<<Scan input for Noweb error messages>>=
\newline
{
\newline
\protected_separator
last_buf_line = 0;
\newline
\protected_separator
while (fgets(buffer[0], 200, stdin)) {
\newline
\protected_separator
\protected_separator
\protected_separator
if (noweb_try(0))
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
output_error(1, 0, "noweb");
\newline
\protected_separator
}
\newline
}
\newline
@
\layout Standard
The examination itself is very stupid.
Unfortunately Noweb doesn't have any characteristic that would help to
identify one of its error messages.
The solution I found was to collect all possible output messages in an
array of strings, and turn the examination into a linear search in this
array.
\layout Scrap
<<Global variables>>=
\newline
char *noweb_msgs[] = {
\newline
\protected_separator
"couldn't open file",
\newline
\protected_separator
"couldn't open temporary file",
\newline
\protected_separator
"error writing temporary file",
\newline
\protected_separator
"ill-formed option",
\newline
\protected_separator
"unknown option",
\newline
\protected_separator
"Bad format sequence",
\newline
\protected_separator
"Can't open output file",
\newline
\protected_separator
"Can't open temporary file",
\newline
\protected_separator
"Capacity exceeded:",
\newline
\protected_separator
"Ignoring unknown option -",
\newline
\protected_separator
"This can't happen:",
\newline
\protected_separator
"non-numeric line number in"
\newline
};
\newline
@
\layout Standard
A Noweb error message can be any string that contains a matching pair of
< <
\protected_separator
\protected_separator
\protected_separator
> >, or any of the above strings.
\layout Scrap
<<Function bodies>>=
\newline
int
\newline
noweb_try (int buf_line)
\newline
{
\newline
\protected_separator
char
\protected_separator
\protected_separator
\protected_separator
*s, *b;
\newline
\protected_separator
int
\protected_separator
\protected_separator
\protected_separator
\protected_separator
i;
\newline
\protected_separator
\newline
\protected_separator
b = buffer[buf_line];
\newline
\protected_separator
s = strstr(b, "<<");
\newline
\protected_separator
if (s != NULL) {
\newline
\protected_separator
\protected_separator
\protected_separator
s = strstr(s+2, ">>");
\newline
\protected_separator
\protected_separator
\protected_separator
if (s != NULL)
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
return 1;
\newline
\protected_separator
} else {
\newline
\protected_separator
\protected_separator
\protected_separator
for (i=0; i<12; i++) {
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
s = strstr (b, noweb_msgs[i]);
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
if (s != NULL)
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
break;
\newline
\protected_separator
\protected_separator
\protected_separator
}
\newline
\protected_separator
\protected_separator
\protected_separator
if (s != NULL)
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
return 1;
\newline
\protected_separator
}
\newline
\protected_separator
return 0;
\newline
}
\newline
@
\layout Scrap
<<Function prototypes>>=
\newline
int noweb_try (int buf_line);
\newline
@
\layout Standard
The Xlc compiler always output one single line for each error found, thus
to scan the buffer for Xlc error messages it is enough to exam one input
line at a time.
\layout Scrap
<<Scan input for Xlc error messages>>=
\protected_separator
\newline
{
\newline
\protected_separator
last_buf_line = 0;
\newline
\protected_separator
while (fgets(buffer[last_buf_line], 200, stdin)) {
\newline
\protected_separator
\protected_separator
\protected_separator
if (xlc_try(0))
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
output_error(1, err_line, "xlc");
\newline
\protected_separator
}
\newline
}
\newline
@
\layout Standard
A Xlc error message is easy to identify.
Every error message starts with a quoted string with no spaces, a comma,
a space, the word
\begin_inset Quotes eld
\end_inset
line
\begin_inset Quotes erd
\end_inset
, a space, and some variable text.
The following routine tests if a given buffer line matches this criteria:
\layout Scrap
<<Function bodies>>=
\newline
int
\newline
xlc_try (int buf_line)
\newline
{
\newline
\protected_separator
char
\protected_separator
\protected_separator
\protected_separator
*s, *t;
\newline
\protected_separator
\newline
\protected_separator
t = buffer[buf_line];
\newline
\protected_separator
s = t+1;
\newline
\protected_separator
while (*s != '"' && *s != ' ' && *s != '
\backslash
0')
\newline
\protected_separator
\protected_separator
\protected_separator
s++;
\newline
\protected_separator
if (*t != '"' || *s != '"' || strncmp(s+1, ", line ", 7) != 0)
\newline
\protected_separator
\protected_separator
\protected_separator
return 0;
\newline
\protected_separator
s += 8;
\newline
\protected_separator
err_line = atoi(s);
\newline
\protected_separator
return 1;
\newline
}
\newline
@
\layout Scrap
<<Function prototypes>>=
\newline
int xlc_try (int buf_line);
\newline
@
\layout Standard
The Gcc compiler error messages are more complicated to scan.
Each error can span more than one line in the buffer.
The good news is that every buffer line on each error has the same pattern,
and share the same line number.
Thus the strategy will be to accumulate lines in the buffer while the reported
line number is still the same.
At the time they differ, all the accumulated lines, except the last one,
will belong to one single error message, which now can be output-ed to
LyX.
\layout Standard
Every gcc error message contains a string with no space followed by a
\begin_inset Quotes eld
\end_inset
:
\begin_inset Quotes eld
\end_inset
.
It the next character is a space, then this line is a header of a error
message and the next line will detail the line number of the source code
where the error was found.
Otherwise, the next thing is a integer number followed by another
\begin_inset Quotes eld
\end_inset
:
\begin_inset Quotes eld
\end_inset
.
\layout Scrap
<<Scan input for Gcc error messages>>=
\newline
{
\newline
\protected_separator
char
\protected_separator
\protected_separator
\protected_separator
*s, *t;
\newline
\protected_separator
\newline
\protected_separator
last_buf_line = 0;
\newline
\protected_separator
while (fgets(buffer[last_buf_line], 200, stdin)) {
\newline
\protected_separator
\protected_separator
\protected_separator
/****** Skip lines until I find an error */
\newline
\protected_separator
\protected_separator
\protected_separator
s = strpbrk(buffer[last_buf_line], " :");
\newline
\protected_separator
\protected_separator
\protected_separator
if (s == NULL || *s == ' ')
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
continue; /* No gcc error found here */
\newline
\protected_separator
\protected_separator
\protected_separator
do {
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
<<Gcc error message criteria is to find a "...:999:" or a "...: ">>
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
/****** OK It is an error message, get line number */
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
err_line = atoi(s+1);
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
if (last_err_line == 0 || last_err_line == err_line) {
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
last_err_line = err_line;
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
continue; /* It's either a header or a continuation, don't output yet */
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
}
\newline
\protected_separator
\protected_separator
\protected_separator
/****** Completed the scan of one error message, output it to LyX */
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
discharge_buffer(1);
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
break;
\newline
\protected_separator
\protected_separator
\protected_separator
} while (fgets(buffer[last_buf_line], 200, stdin));
\newline
\protected_separator
}
\newline
\protected_separator
\protected_separator
/****** EOF completes the scan of whatever was being scanned */
\newline
\protected_separator
discharge_buffer(0);
\newline
}
\newline
@
\layout Scrap
<<Gcc error message criteria is to find a "...:999:" or a "...: ">>=
\newline
/****** Search first ":" in the error number */
\newline
s = strpbrk(buffer[last_buf_line], " :");
\newline
last_buf_line++;
\newline
if (s == NULL || *s == ' ')
\newline
\protected_separator
<<No gcc error found here, but it might terminate the scanning of a previous
one>>
\newline
/****** Search second ":" in the error number */
\newline
t = strpbrk(s+1, " :");
\newline
if (t == NULL || *t == ' ')
\newline
\protected_separator
<<No gcc error found here, but it might terminate the scanning of a previous
one>>
\newline
/****** Verify if is all digits between ":" */
\newline
if (t != s+1+strspn(s+1, "0123456789"))
\newline
\protected_separator
<<No gcc error found here, but it might terminate the scanning of a previous
one>>
\newline
@
\layout Scrap
<<No gcc error found here, but it might terminate the scanning of a previous
one>>=
\newline
{
\newline
\protected_separator
\protected_separator
err_line = 0;
\newline
\protected_separator
\protected_separator
discharge_buffer(1);
\newline
\protected_separator
\protected_separator
continue;
\newline
}
\newline
@
\layout Standard
As we mentioned, when the scan of one Gcc error message is completed everything
in the buffer except the last line is one single error message.
But if the scan terminates with a EOF or through finding one line that
does not match the Gcc error message criteria, them there is no
\begin_inset Quotes eld
\end_inset
last line
\begin_inset Quotes erd
\end_inset
in the buffer to be concerned with.
In those cases we empty the buffer completely.
\layout Scrap
<<Function bodies>>=
\newline
void
\newline
discharge_buffer (int save_last)
\newline
{
\newline
\protected_separator
if (last_err_line != 0) {
\newline
\protected_separator
\protected_separator
\protected_separator
if (save_last != 0) {
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
output_error(last_buf_line-1, last_err_line, "gcc");
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
strcpy (buffer[0], buffer[last_buf_line-1]);
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
last_err_line = err_line;
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
last_buf_line = 1;
\newline
\protected_separator
\protected_separator
\protected_separator
} else {
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
output_error (last_buf_line, last_err_line, "gcc");
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
last_err_line = 0;
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
last_buf_line = 0;
\newline
\protected_separator
\protected_separator
\protected_separator
}
\newline
\protected_separator
}
\newline
}
\newline
@
\layout Scrap
<<Function prototypes>>=
\newline
void discharge_buffer (int save_last);
\newline
@
\layout Standard
To combine the scan of Noweb error messages and Xlc error messages is very
simple.
We just try each one for every input line:
\layout Scrap
<<AIX system using both Noweb and Xlc>>=
\newline
{
\newline
\protected_separator
last_buf_line = 0;
\newline
\protected_separator
while (fgets(buffer[0], 200, stdin)) {
\newline
\protected_separator
\protected_separator
\protected_separator
if (noweb_try(0))
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
output_error(1, 0, "noweb");
\newline
\protected_separator
\protected_separator
\protected_separator
else if (xlc_try(0))
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
output_error(1, err_line, "xlc");
\newline
\protected_separator
}
\newline
}
\newline
@
\layout Standard
To combine the scan of Noweb error messages and Gcc error messages is simple
if we realize that there is not possible to find a Noweb error message
in the middle of a Gcc error message.
So we just repeat the Gcc procedure and test for Noweb error messages in
the beginning of the scan:
\layout Scrap
<<Solaris and Linux systems using both Noweb and Gcc>>=
\newline
{
\newline
\protected_separator
char
\protected_separator
\protected_separator
\protected_separator
*s, *t;
\newline
\protected_separator
\newline
\protected_separator
last_buf_line = 0;
\newline
\protected_separator
while (fgets(buffer[last_buf_line], 200, stdin)) {
\newline
\protected_separator
\protected_separator
\protected_separator
/****** Skip lines until I find an error */
\newline
\protected_separator
\protected_separator
\protected_separator
if (last_buf_line == 0 && noweb_try(0)) {
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
output_error(1, 0, "noweb");
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
continue;
\newline
\protected_separator
\protected_separator
\protected_separator
}
\newline
\protected_separator
\protected_separator
\protected_separator
s = strpbrk(buffer[last_buf_line], " :");
\newline
\protected_separator
\protected_separator
\protected_separator
if (s == NULL || *s == ' ')
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
continue; /* No gcc error found here */
\newline
\protected_separator
\protected_separator
\protected_separator
do {
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
<<Gcc error message criteria is to find a "...:999:" or a "...: ">>
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
/****** OK It is an error, get line number */
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
err_line = atoi(s+1);
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
if (last_err_line == 0 || last_err_line == err_line) {
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
last_err_line = err_line;
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
continue; /* It's either a header or a continuation, don't output yet */
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
}
\newline
\protected_separator
\protected_separator
\protected_separator
/****** Completed the scan of one error message, output it to LyX */
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
discharge_buffer(1);
\newline
\protected_separator
\protected_separator
\protected_separator
\protected_separator
\protected_separator
break;
\newline
\protected_separator
\protected_separator
\protected_separator
} while (fgets(buffer[last_buf_line], 200, stdin));
\newline
\protected_separator
}
\newline
\protected_separator
\protected_separator
/****** EOF completes the scan of whatever was being scanned */
\newline
\protected_separator
discharge_buffer(0);
\newline
}
\newline
@
\layout Subsection
Wrapping the code into a file
\layout Scrap
<<listerrors.c>>=
\newline
#include <stdio.h>
\newline
#include <strings.h>
\protected_separator
\protected_separator
\protected_separator
\newline
\protected_separator
\newline
<<Global variables>>
\newline
<<Function prototypes>>
\newline
<<Function bodies>>
\newline
@
\layout Standard
To build this program, we want to add the
\begin_inset Quotes eld
\end_inset
-L
\begin_inset Quotes erd
\end_inset
option in the tangle command to force gdb to load the file
\family typewriter
Literate.nw
\family default
instead of
\family typewriter
listerrors.c
\family default
.
In accordance with this, we pass the
\begin_inset Quotes eld
\end_inset
-g
\begin_inset Quotes erd
\end_inset
option to gcc.
\layout Scrap
<<build-script>>=
\newline
#!/usr/local/bin/bash
\newline
notangle -L -Rlisterrors.c Literate.nw > listerrors.c
\newline
gcc -g -o listerrors listerrors.c
\newline
@
\layout Standard
This project can tangled and compiled from LyX if you set
\family typewriter
\backslash
build_command
\family default
to call a generic script that always extracts a scrap named
\family typewriter
build-script
\family default
and execute it.
Here is a example of such generic script:
\layout LyX-Code
#!/bin/sh
\newline
notangle -Rbuild-script $1 | sh
\the_end
wiensk-990222-viewlog.patch.gz
