Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-31 Thread Yihui Xie
Hi Romain, I've been thinking for quite a long time on how to keep comments when parsing R code and finally got a trick with inspiration from one of my friends, i.e. to mask the comments in special assignments to cheat R parser: # keep.comment: whether to keep the comments or not #

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-31 Thread Romain Francois
Hi, Thank you for this (inspired) trick. I am currently in the process of extracting out the parser from R (ie the gram.y file) and making a custom parser using the same grammar but structuring the output in a different manner, more suitable for what the syntax highlighter will need. You

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-31 Thread hadley wickham
At the moment, I am concentrating efforts deep down in the parser code, but there are other challenges: - once the expressions are parsed, we will need something that investigates to find evidence about function calls, to get an idea of where the function is defined (by the user, in a

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-31 Thread Romain Francois
hadley wickham wrote: At the moment, I am concentrating efforts deep down in the parser code, but there are other challenges: - once the expressions are parsed, we will need something that investigates to find evidence about function calls, to get an idea of where the function is defined (by the

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-23 Thread Romain Francois
Duncan Murdoch wrote: On 22/03/2009 4:50 PM, Romain Francois wrote: Romain Francois wrote: Peter Dalgaard wrote: Duncan Murdoch wrote: On 3/20/2009 2:56 PM, romain.franc...@dbmail.com wrote: It happens in the token function in gram.c: Â Â Â c = SkipSpace(); Â Â Â if (c == '#') c =

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-23 Thread Duncan Murdoch
On 23/03/2009 3:10 AM, Romain Francois wrote: Duncan Murdoch wrote: However, it might make sense for you to have your own parser, based on the grammar in R's parser, but handling white space differently. Certainly it would make sense to do that before making changes to the base R one.

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-22 Thread Romain Francois
Romain Francois wrote: Peter Dalgaard wrote: Duncan Murdoch wrote: On 3/20/2009 2:56 PM, romain.franc...@dbmail.com wrote: It happens in the token function in gram.c: Â Â Â c = SkipSpace(); Â Â Â if (c == '#') c = SkipComment(); and then SkipComment goes like that: static int

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-22 Thread Duncan Murdoch
On 22/03/2009 4:50 PM, Romain Francois wrote: Romain Francois wrote: Peter Dalgaard wrote: Duncan Murdoch wrote: On 3/20/2009 2:56 PM, romain.franc...@dbmail.com wrote: It happens in the token function in gram.c: Â Â Â c = SkipSpace(); Â Â Â if (c == '#') c = SkipComment(); and then

[Rd] Why does the lexical analyzer drop comments ?

2009-03-20 Thread romain . francois
It happens in the token function in gram.c:     c = SkipSpace();     if (c == '#') c = SkipComment(); and then SkipComment goes like that: static int SkipComment(void) {     int c;     while ((c = xxgetc()) != '\n' c != R_EOF) ;     if (c == R_EOF) EndOfFile = 2;    

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-20 Thread Duncan Murdoch
On 3/20/2009 2:56 PM, romain.franc...@dbmail.com wrote: It happens in the token function in gram.c: Â Â Â c = SkipSpace(); Â Â Â if (c == '#') c = SkipComment(); and then SkipComment goes like that: static int SkipComment(void) { Â Â Â int c; Â Â Â while ((c = xxgetc()) != '\n' c !=

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-20 Thread Romain Francois
Peter Dalgaard wrote: Duncan Murdoch wrote: On 3/20/2009 2:56 PM, romain.franc...@dbmail.com wrote: It happens in the token function in gram.c: Â Â Â c = SkipSpace(); Â Â Â if (c == '#') c = SkipComment(); and then SkipComment goes like that: static int SkipComment(void) { Â Â Â int c; Â Â

Re: [Rd] Why does the lexical analyzer drop comments ?

2009-03-20 Thread Peter Dalgaard
Duncan Murdoch wrote: On 3/20/2009 2:56 PM, romain.franc...@dbmail.com wrote: It happens in the token function in gram.c: Â Â Â c = SkipSpace(); Â Â Â if (c == '#') c = SkipComment(); and then SkipComment goes like that: static int SkipComment(void) { Â Â Â int c; Â Â Â while ((c =