Hello everyone. I was wondering if there is some sort of tests for the Tokenizer along with the correct output of tokens as well as a way of representing tokens. What I have in mind is running the tokenizer on some HTML input and printing the tokens in the same way the correct output is written. I will then be comparing the result I have with the correct one provided character by character. :)
Thanks in advance. Mohammad
