On 8/26/2014 11:41 AM, Jens Grivolla wrote: > On Tue, Aug 26, 2014 at 4:33 AM, Marshall Schor <[email protected]> wrote: > >> On 8/25/2014 6:54 PM, Jens Grivolla wrote: >>> Is the JSON serialization documented somewhere? >> Yes, there's a chapter in the reference book. You can build that >> (uima-docbook-references), until it's released. >> >> There are also lots of Javadocs in the main implementing class: >> XmiCasSerializer. (It's in this class because it shares a lot of the >> machinery >> with Xmi serialization). > > Ok, I'll take another deeper look at this. > > >>> It would be very useful if the serialization allowed one to easily pull >> out >>> a partial CAS with just a subset of the views (by only including some >>> subtrees of the JSON structure), and merge views into it. >> Another optional part of the serialization is a list of views, together >> with an >> array of numbers each one of which represents a serialized Feature >> Structure >> that is indexed in that view. >> > Yes, I saw that. But the FSs themselves are all together. > > Feature Structures can be classified into "Annotations" and other types >> (not a >> subtype of Annotation). >> >> Annotations are special - they have an implied reference to a particular >> subject >> of analysis. So they are restricted to being indexed in the view that is >> associated with that subject-of-analysis. >> >> Other types (not subtypes of Annotation (or more precisely, >> AnnotationBase)) do >> not have this restriction, and can be indexed in multiple views. >> > Thanks for the refresher. That means that the view-specific parts (Sofa and > Annotations) could be grouped with the view they belong to and there could > just be an additional section for FSs that are not bound to a view. > > That would make it much easier to just pull out a subset of views (e.g. > when storing the CAS in MongoDB), as well as write individual views back, > without retrieving all the FSs one doesn't need. The main problem I see is > with the FS ids, which probably need to be unique across the entire CAS. > Thus creating new annotations in a partial CAS that doesn't know about the > other views could create conflicts. > > My usecase is e.g. passing a document through video analysis (a batch > process not directly integrated as a UIMA AE), which would generate a lot > of output that gets integrated in the CAS. I then need to pass just the > view that includes the OCR output to a text analysis pipeline, and would > like to avoid retrieving all the other views from the database and passing > a huge CAS through my pipeline when I really only need a small part of the > information.
Hmmm, It seems your use case is a requirement on how things are stored in your Database. All FSs include a "sofaId" field, identifying their sofa. So it seems to me that if all the FSs are stored in the Database, you need a query which retrieves just those that match a particular sofaId, and also perhaps match just a particular set of types. Is this correct? If, on the other hand, you have a use case where you want UIMA to serialize out a CAS in JSON format while filtering so only some types associated with some Views are output, that would take some work :-) . There is already a "filter" (also available in Xmi Serialization) that lets you filter the things that are serialized by both Type and Features within a Type. But, currently, there is no filter that specifies a View. If this was added, there are some edge cases to figure out. Assume you ask for only View 1. If a FS was indexed in View 1, but had references to FSs that were indexed in another view, those could be included, or not. (The basic algorithm serializes FSs that are indexed, and other FSs that are reachable from those.) And, if there are FSs that are indexed in View 2, that have a reference to FSs that are associated (let's say they're subtypes of Annotation", which causes an association) with View 1, should those be serialized, even though we're not serializing the FSs that perhaps was the only way it could be found? [ I'm thinking that the referenced FS might not be indexed in View 1...]. And, all this has to work also with "Delta CAS" serialization, where another filter is used to only send "new" FSs created after a mark was set. Anyway, I appreciate any input / thoughts on how view filtering ought to work. -Marshall > > -- Jens >
