Quoting Eddie Epstein <[email protected]>:
Primarily concerned with the index repository. First of all, I don't
understand how multiple annotators could be iterating over an index that
other annotators are modifying at the same time.
This is the magic of the concurrent collection classes in the java SDK
:) The iterators are not guaranteed to "see" concurrent modifications,
but by definition that does not matter if the annotators in question
are suitable to run in parallel.
Secondly, Marshall has
spent much time making an index robust to inadvertent user changes. For
example an annotator modifies some "key" feature values for an indexed FS,
or [even impossible to know] modifies feature values that another annotator
has used as keys in a custom index. These scenarios are now detected and
automatically fixed under the covers, but both require looking across all
indexes in all views at the same time. Allowing multiple annotators to
operate R/W on a common index repository would preclude both automatic
fixup and detection.
As mentioned automatic "index protection" is one of the things I
hadn't yet included since it was added very recently - from what I
understand it's always been an issue but hasn't caused noticeable
problems. It should not be a problem to support this in a similar way
with the new implementation.
I don't think this is any more of an problem if things are being
parallelized. If one annotator depends on an ordering by a particular
feature, it logically implies that it wouldn't be logically
independent of any other annotators which modify that feature. So it
wouldn't be a candidate for running in parallel with them in any case.
In any case, the fact that it might be possible to use a function
incorrectly in general doesn't sound like a good reason to not support it.
The same could be said of pretty much anything (again the existence of the
low level API is one good example of this!)
True. It could be a choice to turn off index protection in order to run in
multi-threaded mode.
Eddie
Regards,
Nick
