: That would work, but in that case, the buffering isn't needed. If you
: wanted to use BufferedTokenStream for some reason it would be:
Yeah, you're right ... in the back of my mind i was thinking of more
obscure cases i couldn't articulate and tried using Trey's case as an
example.
Instead: imaging you wanted a TokenFilter that could do balanced paren
matching whil parsing source code ... you want to loop over hte input
stream untill you find something yo uare looking for, and on the way some
things you encounter can be put directly in the output strem, while other
tokens you hang on to because your not ready to toss them on and you're
not sure how many more tokens you need to read before you will be ready.
: +1 for the general idea of flushing state (but I'm not sure how hard
: implementing the exact semantics w.r.t. "before the output stream is
: exhausted").
wouldn't something like this work...
public final Token next() throws IOException {
boolean finshed = false;
while (true) {
if (!outQueue.isEmpty()) return outQueue.removeFirst();
Token t = inQueue.isEmpty() ? input.next() : inQueue.removeFirst();
if (t==null) {
if (finished)
return null;
done();
finished = true;
continue;
}
Token out = process(t);
if (out!=null) return out;
// loop back to top in case process() put something on the output queue
}
}
...allthough, now that i think about it, we probably don't need a done
method .. and sufficiently complex filter that needs to know when the
input stream is totally exhausted so it can flush it's state could allways
just use the peek method in it's process method before returning whatever
it is it's going to return. ie, any class that could be implimented be...
public class Foo extends BufferingTokenFilter{
public Token process(Token t) {
return doSomethingComplicated(t);
}
public void done() {
flushInternalState();
}
}
...could ust as easily be implimented as...
public class Foo extends BufferingTokenFilter{
public Token process(Token t) {
Token r = doSomethingComplicated(t);
if (null == inQueue.peek(1))
flushInternalState()
return r;
}
}
...right?
-Hoss
