Hi Philip, I downloaded Knowtator and played around with it a bit. It looks pretty slick (although I think it could be improved by following some of the more standard UI conventions, like dialogs with "ok" and "cancel" buttons, for example; though I guess that's Protege, not Knowtator).
I got the XML export to work as well, and it looks like this can easily be transformed to a UIMA ingestible format. So if as a first step, we wanted to create annotations in Knowtator and export them to UIMA, that could be done via the Knowtator XML export and XSLT. However, we would be missing the other direction: take a UIMA type system and create a Protege "type system" or whatever that is called. Any thoughts on that? Also note that we have a contribution by Joern Kottmann in the sandbox called "CAS editor". This is Eclipse based tooling that also allows you to manually create annotations. It is still under development, but maybe there could be some cross-fertilization. [I know it's a bit early to talk about legal issues, but please note that the Mozilla license is incompatible with the Apache license. If any of your code were to move to Apache, it would need to be relicensed under ASL 2.0.] Re funding: we are optimistic that there will be more UIMA innovation awards by IBM this year. Watch this space for the announcement. I would think that a Knowtator/UIMA integration could be a candidate. A human annotation tool that works well with UIMA would be an important addition to our ecosystem, so I'm glad this discussion is happening. BTW, are people familiar with the annotation standards work that is going on at ISO (http://www.tc37sc4.org/)? This bears watching as well, as it might evolve into the annotation standard that everybody's been waiting for. At least that seems to be the plan of the folks working on it ;-). --Thilo Philip Ogren wrote: > I'm glad I happened to browse the archive today! I just joined the list > today because I have noticed a couple of bugs that I want to post > somewhere. So, I developed and maintain Knowtator and am also steeped in > UIMA technology - I have been using it for just over a year and a half > now. I would love to make Knowtator tightly integrated with UIMA. The > frame-based representation that Knowtator uses via Protege (Frames > edition) is *very* similar to the type system framework in UIMA. The > Protege frames is actually much more expressive and I would be surprised > if the representational capability does not completely subsume UIMA's > type system representational capability. Knowtator can not make use out > of some of the more advanced representational constructs such as slot > inheritance (i.e. superslots and subslots, e.g. has-characteristic might > subsume has-color) or multiple inheritance - but I think it handles > anything that can be represented in a UIMA type system. I think it would > be a really nice fit. > > A one-off solution as described previously is going to be extremely > frustrating. Things that you might want to do with an annotation tool > include: > > - calculate inter-annotator agreement in a wide variety of ways > - consolidate/adjudicate disagreement between annotators > - mundane data management tasks > - keep track of who annotated what > - merge sets of annotations together > - stand-alone annotation on a laptop (a perk for annotators if it is a > part-time job that they can do during hours of their own choosing) > - work on / visualize subsets of annotations > - there are a bajillion user-interface considerations > > I'm sure there are many other things I haven't thought of off the top of > my head. We have been working really hard on Knowtator the last few > months and have tried to make it much easier to get started with and > more user friendly. If you have looked at it previously and got > discouraged, then I encourage you to take a look at the latest version > and the updated documentation. It is still a clunky with respect to > importing and exporting annotation data (which we are currently working > on to make easier) - but having a UIMA solution would make this problem > go away for this community. We have created some one-off scripts to go >> from one to the other that we could possibly make available with a > little effort if there is interest. > > If you have ideas about how this effort could be funded I would be > grateful for suggestions. We are considering applying for an Eclipse > Innovation Award as an appropriate venue but we don't really know what > the odds are of getting it funded for this work. Or, if you have > interest in working on this yourself, I would be thrilled to provide > expertise. > > Thanks, > Philip > > > > > > > > ----include---- > One manual annotation tool that is open source is Knowtator (which is > licensed under MPL 1.1). As I understand it, Knowtator is intended for > manual annotation entities and relationships in text. It is a layer on > top of the Prot�g� open source ontology editor. I'm not really familiar > enough with Knowtator to explicitly recommend it. Considering its > stated goals and the framework that it was developed on, it seems like > it might be particularly well suited to enabling manual annotations for > relatively elaborate type systems that have a lot of structure and many > common relation annotation types. The flip side is that it may be > overkill for the (more common) task of marking up instances of a flat > list of named-entity types. In any event, my point here is just that > anyone who is thinking of building a mapping from an open source manual > annotation tool to UIMA may want to consider Knowtator, especially if > they are interested in a lot of expressive power.
