Re: Question about lexing
@15, no. The only thing that's in the string that I use std::strtold/std::atof on is the decimal number itself. No extraneous symbols are in the string.
URL: https://forum.audiogames.net/post/588756/#p588756
--
Audiogames-reflector mailing list
Audiogames
Re: Question about lexing
Do you have another token for the subtraction operation that's matching the - first? That could be interfering with it.
URL: https://forum.audiogames.net/post/588701/#p588701
--
Audiogames-reflector mailing list
Audiogames-reflector@sabahattin-gucukoglu.com
Re: Question about lexing
Do you have another token for the negative operation that's matching the - first? That could be interfering with it.
URL: https://forum.audiogames.net/post/588701/#p588701
--
Audiogames-reflector mailing list
Audiogames-reflector@sabahattin-gucukoglu.com
Re: Question about lexing
Okay, so, I got this working. Sort of, anyway. I'm running into an issue with std::atof/std::strtold. The test code we have to use has this _expression_ in it:x=-5.63e002;However, both on Windows and on Ubuntu (in WSL) this yields 563 (discarding x, the equals
Re: Question about lexing
Okay, so, I got this working. Sort of, anyway. I'm running into an issue with std::atof/std::strtold. The test code we have to use has this _expression_ in it:x=-5.63e002;However, both on Windows and on Ubuntu (in WSL) this yields 563 (discarding x
Re: Question about lexing
@11A DFA is a finite state machine where the nodes are the states and the edges are the transitions. In practice, what happens with the generators is they write out a table of DFA state transitions, but we're coding it ourselves, so we can collapse any "
Re: Question about lexing
I odnt know whether i want to study CS after reading this forum threat. Thanks Ethin lol.Seriously though, this is kinda fascinating. I hope they will teahcu s that as well.
URL: https://forum.audiogames.net/post/586352/#p586352
--
Audiogames-reflector
Re: Question about lexing
@10, yeah, he didn't really teach it well. He never actually covered this assignment in class, nor did he cover how to actually do this in code. He's showed us the grammar and how it expands into tokens, and how lexing works with a DFA, but never showed us how
Re: Question about lexing
@9This is a pretty standard college assignment, it's just been something like 7 years since I did it. If you want to do the FSM approach, you need a class with roughly the following interface:class Reader {
char next();
char peak();
};Then you split your
Re: Question about lexing
@8, I'm not even sure how I'd write the really big state machines of doom by hand. I prefer PEGs to be honest; they make writing the grammar a lot easier. I don't know why my professor wants me to write the lexer and parser by hand; presumably it makes it easier
Re: Question about lexing
I probably can't help much on the C++ specifics. I avoid C++ unicode handling like the plague, so much so that I would literally choose another language over C++ if it was the primary part of the project I'm working on. They made a lot of fundamentally bad
Re: Question about lexing
@5, I tried using a set of regular expressions using std::regex:[_a-zA-Z][_a-zA-Z0-9]+ for identifiers;[1-9]\d* for integers; and[-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)? for realsProblem is the last one. My professor gives the following regular expressions for each
Re: Question about lexing
@5, I tried using a set of regular expressions using std::regex:[_a-zA-Z][_a-zA-Z0-9]+ for identifiers;[1-9]\d* for integers; and[-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)? for realsProblem is the last one. My professor gives the following regular expressions for each
Re: Question about lexing
@5, I tried using a set of regular expressions using std::regex:[_a-zA-Z][_a-zA-Z0-9]+ for identifiers;[1-9]\d* for integers; and[-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)? for realsProblem is the last one. My professor gives the following regular expressions for each
Re: Question about lexing
@5, I tried using a set of regular expressions using std::regex:[_a-zA-Z][_a-zA-Z0-9]+ for identifiers;[1-9]\d* for integers; and[-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)? for realsProblem is the last one. My professor gives the following regular expressions for each
Re: Question about lexing
Ah this takes me back! I think 4 gave a pretty good summary.Also, I hope the professor has shown at least how to get started. You can't be expected to work on something like this without knowing a little bit about basic structure.But if you want to read a little
Re: Question about lexing
There's nothing particularly wrong with using a regular _expression_ anchored to the beginning of an in-memory string, i.e.:^\d+.?\d*For a lame floating point tokenizer. The key is that ^ keeps it from zipping ahead: it'll only match what comes next.In practice
Re: Question about lexing
@1,It has been some time since I've written a lexer/tokeniser, but I'll give a brief overview of the structure I was taught to use at uni. Hopefully you can use it to work from.For a left to right lexer, you basicly want to convert from a stream of characters
Re: Question about lexing
@2The lexer is just the component that given the imput stream breaks it into tokens which then are feed to the parser, which will generally construct an abstract sintax tree from it.In short, the parser doesn't care about the actual lexing logic, it just receives
Re: Question about lexing
@2The lexer is just the component that given the imput stream breaks it into tokens which then are feed to the parser, which will generally construct an abstract sintax tree from it.In short, the parser doesn't care about the actual lexing logic, it just receives
Re: Question about lexing
I would personally ask the professor about how he expects you to complete it. If he gave you reference regex that is incorrect, then maybe you are supposed to just fix the regex. If you are asking us these questions then the assignment doesn't seem to be very well
Question about lexing
So, I'm taking theoretical computer science at the moment and we're covering DFAs/NFAs, lexers, parsers and compilers. My professor wants us to develop a lexer to recognize the following code:program abc
integer a,b,c
real x
begin
b=6;
a=b+-4;
x=-5.63e002;
end
endAs
Question about lexing
So, I'm taking theoretical computer science at the moment and we're covering DFAs/NFAs, lexers, parsers and compilers. My professor wants us to develop a lexer to recognize the following code:program abc
integer a,b,c
real x
begin
b=6;
a=b+-4;
x=-5.63e002;
end
endAs
23 matches
Mail list logo
