On 28/01/13 16:50, Rob Vesse wrote:
It uses a marker object placed onto the BlockingQueue to indicate end of
input so the indicator is in-stream.
(BTW I think this will also ways cause a ClassCastException
@SuppressWarnings("unchecked")
private final T endMarker = (T)new Object();
)
Ugh, this is ugly as hell, though necessary in the case of
RiotParserPuller. Personally I don't think this is necessary in the
StreamRDF case since you have a finish() method that is called so once
you've received that call and set your finished flag you can use it and a
determination of the buffer being empty to detect the end of the input.
You sound like you are trying to avoid end-of-stream markers. Any
reason why?
I coded it this way intentionally to solve a specific problem I
encountered and to protect users from getting stuck in a deadlock.
It is possible for you to invoke the parser such that the parser fails
before ever calling start() on the stream e.g. by calling
RDFDataMgr.parse() with incorrect/insufficient arguments to describe the
input format. This leaves the user blocked in hasNext() or next()
forever. A semaphore would not solve that problem and would lead to the
same deadlock situation that I introduced the started flag to workaround.
However I think a semaphore could perhaps be used instead of the existing
started flag in conjunction with tryAcquire() to wait a fixed amount of
time before throwing the IllegalStateException rather than relying on the
current boolean flag.
If there is one queue with end of stream (and "error encountered" marker?)
(I have changed it to use AtomicBoolean because, in theory, a plain
filed can be cached into the thread and the object member slot only read
occasionally if at all - Stephen's taught me that :-)
If I make the polling improvements I suspect that a marker will remain
unnecessary. I will work on refactoring the code and consolidating where
possible tomorrow.
Any reason not to use an in-flow end of item marker?
My worry (and experience!) with polling is that it works in many cases
but there can be more unusual, though hardly pathological, cases where
waking up to poll is wasting a thread. Choosing the polling times is
always a compromise of efficiency and latency.
We want new item or finished going true to noticed quickly.
Waiting on two different things is what drives the need to poll. So can
we wait on one thing.
Have I missed something you've encountered?
(Stephen - thoughts?)
Andy
