Author: kjs
Date: Sat Mar 24 05:40:29 2007
New Revision: 17709
Modified:
trunk/compilers/pirc/doc/design.pod
Log:
compilers/pirc:
* add stuff to doc/design.pod
Modified: trunk/compilers/pirc/doc/design.pod
==============================================================================
--- trunk/compilers/pirc/doc/design.pod (original)
+++ trunk/compilers/pirc/doc/design.pod Sat Mar 24 05:40:29 2007
@@ -11,7 +11,9 @@
PIRC currently consists of only a PIR parser, together with a lexer.
-=head1 OVERVIEW OF THE LEXER
+=head1 THE LEXER
+
+=head2 OVERVIEW
The lexer is defined in C<pirlexer.c>. The header file lists all tokens that
may be returned by the lexer.
@@ -41,7 +43,55 @@
and then returns the T_ENDM token, which can then be matched by the parser.
-=head1 OVERVIEW OF THE PARSER
+=head2 IMPLEMENTATION
+
+The lexer is represented by a structure called C<lexer_state>. It is
predeclared
+in the header file, and defined in the source file. This is to prevent direct
+access to its members.
+
+The structure looks like this:
+
+ typedef struct lexer_state {
+ struct file_buffer *curfile;
+ char *token_chars;
+ char *charptr;
+
+ } lexer_state;
+
+It has three pointers: the first one is a pointer to a file_buffer structure,
which
+will be discussed shortly. The second is a pointer to the current token's
characters.
+This is a buffer in which the characters for the current token are stored.
+The field C<charptr> acts like an index to add and delete characters from
C<token_chars>.
+
+As mentioned, the lexer reads the source file and returns tokens. The source
file is
+represented by another structure, called C<file_buffer>. It looks like this:
+
+ typedef struct file_buffer {
+ char *filename;
+ char *buffer;
+ char *curchar;
+ unsigned filesize;
+ unsigned int line;
+ unsigned short linepos;
+ char lastchar;
+ struct file_buffer *prevbuffer;
+
+ } file_buffer;
+
+The first field C<filename> holds the name of the file that is being scanned.
+The second field C<buffer> is a pointer to the contents of the file. The field
+C<curchar> points into this buffer to the current position, and can be
considered
+the I<cursor>.
+Then, C<filesize> contains the length of the file; C<line> keeps track of what
line
+is being processed; linepos keeps track of the number of characters since the
last
+newline character (used for error messages). The field C<lastchar> stores the
+last character that is returned. This is used to check whether we're at the
beginning
+of a newline (which is necessary to parse heredoc delimiters).
+
+
+=head1 THE PARSER
+
+=head2 OVERVIEW
The parser is defined in C<pirparser.c>. The header file only predeclares the
C<parser_state> structure, but its definition is written in the C file, to hide
@@ -55,7 +105,25 @@
The parser does not know anything about the spelling of tokens, although it can
request these through C<find_keyword()>.
-B<more to come later>
+
+=head2 IMPLEMENTATION
+
+The parser is represented by a structure called C<parser_state>. Its layout is
+shown below:
+
+ typedef struct parser_state {
+ struct lexer_state *lexer;
+ token curtoken;
+ char *heredoc_ids[10];
+ int heredoc_index;
+ unsigned parse_errors;
+
+ } parser_state;
+
+It consists of the following fields: a pointer to a lexer_state object; the
current
+token as returned by the lexer; a list to store heredoc identifiers, which is
used
+when parsing heredocs as subroutine arguments; an index for this list; and
finally a
+counter to keep track of parse errors.
=head1 AUTHOR
