I debugged using the all_in_one setup and found that I was returning null from getProgress(), which was not creating any workitems. So I setup the function and Cases started moving.
Thanks for all of your help, Neal. From: Neal Lewis <[email protected]> To: [email protected] Date: 11/15/2013 05:48 AM Subject: Re: DUCC not leaving Initializing State Thanks Eddie, I missed the all_in_one in the documentation, so I'll try it out and get back to you guys. -Neal On 11/14/2013 06:02 PM, Eddie Epstein wrote: > Good job. FYI, it is strongly recommended to use all_in_one as described > in the documentation on developing a new sample application. Your > aggregate most likely does not include the built-in DUCC flow controller > which implements SendToLast. > > After all child CASes created for a work itemhave been generated, if > the CC needs a signal to clean up, then use SendToLast. The DucCasCC > does need that signal to close the output zip package. > > Eddie > > > > > On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <[email protected]> wrote: > >> So, I found one GLARING problem that I missed completely: When testing >> my CR and CM setup, I looped through the CR's getNext(), adding to the >> JCas that I just created. But the CR's getNext() didn't loop internally, >> it only added IDs one at a time. So, instead of the CR creating a CAS with >> N Workitems and halting, it was create N CASes with 1 workitem. I've >> fixed that. >> >> So I've made an Aggregate AE of my CM and a simple AE, and ran them with >> my CR in a uimaFit SimplePipeline. The outputs are as I expect for now. >> >> >> But I do have a question. The Workitems have an option of SendToLast, >> which in the RawText Example is set to "true". The DuccCasCC in the >> sample pulls WorkItem FSes for output stuff. The CR adds the Workitems, >> but the CM doesn't reattach them to the new CASes before returning next(). >> >> >> Should the Workitems stay in the CAS throughout processing (ie., should >> the CMs add them to the new CASes) ? OR is SendToLast enough? >> >> Thanks! >> >> Neal >> >> >> >> From: Neal R Lewis/Almaden/IBM@IBMUS >> To: [email protected] >> Date: 11/14/2013 01:20 PM >> Subject: Re: DUCC not leaving Initializing State >> >> >> >> The minimal wrapper creates a Cas using uimaFit and loop over the CR, >> then to run over the iterator from cm.processAndOutputNewCASes, then >> output each cas in the loop like below: >> >> CollectionReader cr = CollectionReaderFactory.createReader >> ("path.to.my.CasReader"); >> AnalysisEngine cm = AnalysisEngineFactory.createEngine( >> "path.to.my.CasMultiplier"); >> >> JCas jcas = JCasFactory.createJCas(" >> desc.DuccJobFlowControlTS); >> >> while (cr.hasNext()){ >> cr.getNext(jcas.getCas()); >> } >> >> CasIterator casIterator = >> cm.processAndOutputNewCASes(jcas.getCas()); >> while (casIterator.hasNext()) { >> CAS outCas = casIterator.next(); >> System.out.println(outCas.getDocumentText()); >> } >> >> I'll need to debug a full standalone PEAR or Pipeline. But at this point I >> >> was able to at least get the CASes out. >> >> Below is the output of the JP file before the stall and after loading JVM >> stuff. It stays like this until shutdown. As far as I can tell the CM >> Loads and is waiting. >> >> >> + JVM >> LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >> +------------------------------------------------------------------ >> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor - Initializing >> ExecutorService 'pooling_ducc.jd.queue.91_1' >> 03:53:44.794 - 1: >> org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName: >> INFO: top_level_input_queue_service_1 Service Starting - Listening for >> Messages >> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor - Initializing >> ExecutorService 'pooling_ducc.jd.queue.91_1' >> 03:53:44.797 - 30: >> org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run(): INFO: >> >> Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id: 30 >> 03:53:44.799 - 1: >> >> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification: >> >> INFO: Uima EE Client Blocking - Awaiting Top Level Controller >> Initialization Notification >> 03:53:44.887 - 30: >> >> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70): >> >> INFO: Initializing Cas Multiplier Parameters >> 03:53:44.887 - 30: >> >> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71): >> >> INFO: init HOST ADDRESS: >> https://thepit.element.almaden.ibm.com/cgi-bin/qm2 >> 03:53:44.887 - 30: >> >> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72): >> >> INFO: init LENIENT: false >> 03:53:44.887 - 30: >> >> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73): >> >> INFO: init PARAM_ENCODING: UTF-8 >> 03:53:44.887 - 30: >> >> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74): >> >> INFO: init PARAM_LANGUAGE: en >> 03:53:44.933 - 30: >> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO: Outputting >> >> CASes in XmiCas format, zip compressed at level=7 >> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages From >> Queue:ducc.jd.queue.91 >> 03:53:45.97 - 30: >> >> org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize: >> >> INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready To >> Process. ******** >> 03:53:45.97 - 1: >> >> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners: >> >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint: >> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002 Broker: >> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false >> 03:53:45.290 - 1: >> >> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners: >> >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint: >> queue://ducc.jd.queue.91 Selector: Command=2001 Broker: >> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false >> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ... >> Component started: managedService >> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A Starting >> Camel. Use ctrl + c to terminate the JVM. >> >> >> >> >> >> >> From: Eddie Epstein <[email protected]> >> To: [email protected] >> Date: 11/14/2013 10:37 AM >> Subject: Re: DUCC not leaving Initializing State >> >> >> >> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <[email protected]> wrote: >> >>> I have modeled a CasReader and CasMultiplier based on the RawTextExample >>> in the duccbook, and have successfully ran the CR and CM in eclipse with >> a >>> minimal wrapper. >> >> Did you debug the job using one of the --all_in_one varieties, or some >> other >> minimal wrapper? >> >> >> I am using the same CC from the Sample (DuccCasCC) >>> I am experiencing a problem where my DUCC job goes through >>> WaitingForDriver, WaitingForResources, then Initializing, then doesn't >>> change state. After several minutes I just cancel the job. >>> >> Did you look in the JP logfile? >> >> Eddie >> >> >>
