I'm basically hoping Terence would answer this. And since this is my first
direct question i must say the following:
Terence, when i came across Antlr i was just blown away by what you had
created.
Before that i was about to first write my own simplistic LL1 or SLR. Then i
thought i might as well extend some of the work already done. So i looked at
existing compilers such as Yacc and Cup. Then I decided to look at antlr. I had
heard about it but never examined it.
There it was: antlWorks, EBNF, and LL* parsing (i don't even know what that
means but it sure is a lot more complicated than LL1 and LL2.)
I'm also surprised that you still actively develop this and monitor all these
forums.
You should just be kicking back and letting us bid for work on this fine tool
because it would be a privilege to have contributed to this whole thing.
With this out of the way, I need to understand how this whole token
insertion/replacement works.
I'm hoping that i could formulate my concern in this post or in a few
subsequent replies to this.
To start, i'll just copy your comments from within the code in this post.
Based on what i understand, insertions tokens get priority over the replacement
one. The last replacement overwrites any previous one. The insertions should
later to treated with the oldest (time of insertion) to the newest.
So far so good. Now we get to the point where we need to insert them in the
stream (toString method).
Here, conceptually, i need to know if we're using a static stream (the image is
frozen before any rewrite) or the image is dynamic. This is similar to the
rowset cursors in databases.
To illustrate, let's assume "t" stands or an original token. The stream would
have t0 - t5 tokens.
Insertions are represented as "i?" where "?" stands for its index. such as i2
and i3 are insertions at 2 and 3 respectively.
Replacements (and deletions) are shown as "r?-?" where "??" stands for start
and end of the replacements (index and lastIndex). For example r2-4 means
replace from 2 to 4.
Our rewrite list at index 2 has the following: i2.1, i2.2 which are to be
inserted in the order: 2.1 then 2.2
And we also have one single replacement at the same index 2: r2-4 (replace from
2-4)
So which one would it be?
scenario a) static treatment of the stream:
before: t0 t1 t2 t3 t4 t5
after inserts: t0 t1 i2.1 i2.2 t2 t3 t4 t5
after replace: t0 t1 i2.1 i2.2 r2-4 t5
scenario b) dynamic
before: t0 t1 t2 t3 t4 t5
after inserts: t0 t1 i2.1 i2.2 t2 t3 t4 t5
after replace: t0 t1 r2-4 t3 t4 t5
if (b) then what if we also had an i3.1:
scenario c) dynamic with all inserts first then another pass for replacements
before: t0 t1 t2 t3 t4 t5
after inserts: t0 t1 i2.1 i3.1 i2.2 t2 t3 t4 t5 (i think even this can have
different sceanrios)
after replace: t0 t1 r2-4 t2 t3 t4 t5
scenario d) dynamic but index takes priority
before: t0 t1 t2 t3 t4 t5
after inserts (index2): t0 t1 i2.1 i2.2 t2 t3 t4 t5
after replace (index2): t0 t1 r2-4 i3.1 t3 t4 t5
after insert (index3): t0 t1 r2-4 i3.1 t3 t4 t5
thanks
jeff
copied from TokenRewriteStream:
* When there are multiple instructions at the same index, the instructions
* must be ordered to ensure proper behavior. For example, a delete at
* index i must kill any replace operation at i. Insert-before operations
* must come before any replace / delete instructions. If there are
* multiple insert instructions for a single index, they are done in
* reverse insertion order so that "insert foo" then "insert bar" yields
* "foobar" in front rather than "barfoo". This is convenient because
* I can insert new InsertOp instructions at the index returned by
* the binary search. A ReplaceOp kills any previous replace op. Since
* delete is the same as replace with null text, i can check for
* ReplaceOp and cover DeleteOp at same time. :)
_______________________________________________
antlr-dev mailing list
[email protected]
http://www.antlr.org:8080/mailman/listinfo/antlr-dev
