Hi Reshu, The collectionProcessingComplete() method in UIMA-AS has a limitation: a Collection Processing Complete request sent to the UIMA-AS Analysis Service is cascaded down to all delegates; however, if a particular delegate is scaled-out, only one of the instances of the delegate will get this call.
Since DUCC is using UIMA-AS to scale out the Job processes, it has no way to deliver a CPC to all instances. The applications we have been running on DUCC have used the Work Item CAS as a signal to CAS consumers to do CPC level processing. That is discussed in the first reference above, in the paragraph "Flushing Cached Data". Eddie On Wed, Mar 26, 2014 at 9:48 AM, reshu.agarwal <[email protected]>wrote: > On 03/26/2014 06:43 PM, Eddie Epstein wrote: > >> Are you using standard UIMA interface code to Solr? If so, which Cas >> Consumer? >> >> Taking at quick look at the source code for SolrCASConsumer, the batch and >> collection process complete methods appear to do nothing. >> >> Thanks, >> Eddie >> >> >> On Wed, Mar 26, 2014 at 6:08 AM, reshu.agarwal <[email protected]> >> wrote: >> >> On 03/21/2014 11:42 AM, reshu.agarwal wrote: >>> >>> Hence we can not attempt batch processing in cas consumer and it >>>> increases our process timing. Is there any other option for that or is >>>> it a >>>> bug in DUCC? >>>> >>>> Please reply on this problem as if I am sending document in solr one by >>> one by cas consumer without using batch process and committing solr. It >>> is >>> not optimum way to use this. Why ducc is not calling collection Process >>> Complete method of Cas Consumer? And If I want to do that then What is >>> the >>> way to do this? >>> >>> I am not able to find any thing about this in DUCC book. >>> >>> Thanks in Advanced. >>> >>> -- >>> Thanks, >>> Reshu Agarwal >>> >>> >>> Hi Eddie, > > I am not using standard UIMA interface code to Solr. I create my own Cas > Consumer. I will take a look on that too. But the problem is not for > particularly to use solr, I can use any source to store my output. I want > to do batch processing and want to use collectionProcessComplete. Why DUCC > is not calling it? I check it with UIMA AS also and my cas consumer is > working fine with it and also performing batch processing. > > -- > Thanks, > Reshu Agarwal > >
