Yi-Wen, With respect to the ducc.log, I still saw that the DUCC Job was Running as far as the JD was concerned (as evidenced by the JD's log file). There may be some latency in reflecting this fact back to the Orchestrator daemon from the JD. Note that the Orchestrator writes the ducc.log file. The ducc.log not showing Running could be the subject of a Jira, but is not of concern with respect to getting your Job to actually run.
I'm not sure I understand your statements regarding All-In-One testing. You ran your Job successfully without any error using All-In-One, or that had some issue too? Did you run All-In-One locally or remotely? Lou. On Tue, Nov 17, 2015 at 5:18 PM, Yi-Wen Liu <[email protected]> wrote: > Hi, > > The ducc.log didn't show running: > 2015.11.16 23:29:26 Mon Received > 2015.11.16 23:29:26 Mon WaitingForDriver > 2015.11.16 23:29:54 Mon WaitingForResources > 2015.11.16 23:30:04 Mon Assigned > 2015.11.16 23:30:14 Mon Initializing > 2015.11.16 23:34:48 Mon Completing > 2015.11.16 23:35:18 Mon Completed > 2015.11.16 23:35:18 Mon completion type: CanceledByDriver > 2015.11.16 23:35:18 Mon rationale: unspecified > > And I didn't find error in all_in_one local: > > [1] Loading model: > > [1] ... > > [1] Loading model: > > [1] . > > [1] Loading configuration. > > [1] Loading feature templates. > > [1] Loading lexica. > > [1] Loading model: > > [1] ................................ > > [1] Loading model: > > [1] ............................. > > [1] 17 Nov 2015 13:49:33 INFO Chunker - Chunker model file: > org/apache/ctakes/chunker/models/chunker-model.zip > > [1] 17 Nov 2015 13:49:35 INFO JdbcConnectionResourceImpl - Connection > established to: > jdbc:hsqldb:res:org/apache/ctakes/dictionary/lookup/umls2011ab/umls > > [1] 17 Nov 2015 13:49:35 INFO JdbcConnectionResourceImpl - Connection > established to: > jdbc:hsqldb:res:org/apache/ctakes/dictionary/lookup/rxnorm-hsqldb/umls > > [1] 17 Nov 2015 13:49:35 INFO JdbcConnectionResourceImpl - Connection > established to: > jdbc:hsqldb:res:org/apache/ctakes/dictionary/lookup/orange_book_hsqldb/umls > > [1] 17 Nov 2015 13:49:35 INFO UmlsDictionaryLookupAnnotator - Parsing > descriptor: > > /Users/ducc/src/apache-ctakes-3.2.2/resources/org/apache/ctakes/dictionary/lookup/LookupDesc_Db.xml > > [1] 17 Nov 2015 13:49:35 INFO FirstTokenPermLookupInitializerImpl - > Exclusion tagset loaded: [dt, to, rp, ls, pos, md, vbd, vbg, vb, ex, vbp, > vbn, pdt, vbz, wp, wrb, in, wps, pp$, prp$, wdt, prp, pp, cc, cd] > > [1] 17 Nov 2015 13:49:35 INFO FirstTokenPermLookupInitializerImpl - > Exclusion tagset loaded: [to, dt, rp, ex, vbp, ls, vbn, pdt, wp, vbz, wrb, > in, pos, wps, md, wdt, pp$, vbd, vb, vbg, pp, cc, cd] > > [1] 17 Nov 2015 13:49:35 INFO UmlsDictionaryLookupAnnotator - Using > ctakes.umlsaddr: https://uts-ws.nlm.nih.gov/restful/isValidUMLSUser: > yiwenliu > > [1] 17 Nov 2015 13:49:39 INFO LvgCmdApiResourceImpl - Loading NLM Norm and > Lvg with config file = > > /Users/ducc/src/apache-ctakes-3.2.2/resources/org/apache/ctakes/lvg/data/config/lvg.properties > > [1] 17 Nov 2015 13:49:39 INFO LvgCmdApiResourceImpl - config file > absolute path = > > /Users/ducc/src/apache-ctakes-3.2.2/resources/org/apache/ctakes/lvg/data/config/lvg.properties > > [1] 17 Nov 2015 13:49:39 INFO LvgCmdApiResourceImpl - cwd = > /Users/ducc/src > > [1] 17 Nov 2015 13:49:39 INFO LvgCmdApiResourceImpl - cd > /Users/ducc/src/apache-ctakes-3.2.2/resources/org/apache/ctakes/lvg/ > > [1] 17 Nov 2015 13:49:40 INFO LvgCmdApiResourceImpl - cd /Users/ducc/src > > It looks like it suddenly canceled by driver and no more output on screen. > Another log file shows connection refused so I checked driver logs, and > found the three errors in first email. > > Is that error org.xml.sax.SAXParseException; Trying to serialize non-XML > 1.0 character caused by invalid space and characters in Xml, or it's more > like syntax error of CR Xml? > > Thanks, > Yi-Wen > > On Tue, Nov 17, 2015 at 1:36 PM, Lou DeGenaro <[email protected]> > wrote: > > > Hi Yi-Wen, > > > > Does your CR function correctly outside of DUCC? Have you tired > debugging > > using DUCC's All-In-One feature as suggested previously by Eddie? > > > > Lou. > > > > On Tue, Nov 17, 2015 at 3:54 PM, Yi-Wen Liu <[email protected]> wrote: > > > > > Hi, > > > > > > I tried to remove redundant white spaces but it doesn't work, and I > > didn't > > > add any other special characters in CR. > > > > > > Is that error caused by invalid space and characters in Xml, or it's > more > > > like syntax error of CR Xml? > > > > > > > > > > > > Thanks, > > > Yi-Wen > > > > > > > On Nov 17, 2015, at 5:37 AM, Lou DeGenaro <[email protected]> > > > wrote: > > > > > > > > Yi-Wen, > > > > > > > > The DUCC Job first visits the Initialization state. This is when the > > Job > > > > Driver (JD) initializes your UIMA CollecitonReader (CR). The CR that > > you > > > > specified in your DUCC Job submission is constructed and its > getTotal() > > > > method is invoked to determine the number of work items. But I think > > > your > > > > Job made it past the Initialization state and into the Running state. > > > > Looking at the log snippits that you posted, I see: > > > > > > > > 16 Nov 2015 23:34:38,353 INFO JobDriver - T[18] advanceJdState > > > > current=Initializing request=Active result=Active > > > > > > > > This means the JD has exited its Initializing state and is now Active > > and > > > > is fetching work items. Further evidence is the entry: > > > > > > > > 16 Nov 2015 23:34:39,066 ERROR CasManager - T[17] getMetaCas > > > > > > > > What do the contents of the file ducc.log in your Job's log directory > > > > show? You should see "Initializing" followed by "Running". > > > > > > > > It looks like the root cause of your problem is that your CR's > > getNext() > > > is > > > > trying to produce work items for the JD but is failing: > > > > > > > > org.xml.sax.SAXParseException; Trying to serialize non-XML 1.0 > > > > character: , 0x0 > > > > at > > > org.apache.uima.util.XMLSerializer$CharacterValidatingContentHand > > > > ler.checkForInvalidXmlChars(XMLSerializer.java:254) > > > > > > > > I recommend determining why your CR is broken, fixing, and > > re-submitting > > > > your Job to DUCC. > > > > > > > > Lou. > > > > > > > >> On Tue, Nov 17, 2015 at 2:46 AM, Yi-Wen Liu <[email protected]> > wrote: > > > >> > > > >> Hello, > > > >> > > > >> I am still stuck in the following error of DUCC and job always got > > > canceled > > > >> by driver: > > > >> > > > >> 16 Nov 2015 23:34:38,353 INFO JobDriver - T[18] advanceJdState > > > >> current=Initializing request=Active result=Active > > > >> 16 Nov 2015 23:34:39,066 ERROR ProxyLogger - T[17] loggify > > > >> org.xml.sax.SAXParseException; Trying to serialize non-XML 1.0 > > > >> character: , 0x0 > > > >> at > > > >> > > > > > > org.apache.uima.util.XMLSerializer$CharacterValidatingContentHandler.checkForInvalidXmlChars(XMLSerializer.java:254) > > > >> > > > >> 16 Nov 2015 23:34:39,066 ERROR CasManager - T[17] getMetaCas > > > >> org.apache.uima.ducc.container.common.classloader.ProxyException > > > >> at > > > >> > > > > > > org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.getMetaCas(ProxyJobDriverCollectionReader.java:127) > > > >> at > > > >> > > > > > > org.apache.uima.ducc.container.jd.cas.CasManager.getMetaCas(CasManager.java:65) > > > >> at > > > >> > > > > > > org.apache.uima.ducc.container.jd.fsm.wi.ActionGet.engage(ActionGet.java:118) > > > >> at > > > >> > org.apache.uima.ducc.container.common.fsm.Fsm.transition(Fsm.java:172) > > > >> > > > >> 16 Nov 2015 23:34:39,068 ERROR ActionGet - T[17] engage > > > >> org.apache.uima.ducc.container.jd.JobDriverException > > > >> at > > > >> > > > > > > org.apache.uima.ducc.container.jd.cas.CasManager.getMetaCas(CasManager.java:74) > > > >> at > > > >> > > > > > > org.apache.uima.ducc.container.jd.fsm.wi.ActionGet.engage(ActionGet.java:118) > > > >> > > > >> > > > >> However, when I looked into output directory, there were output > (much > > > >> omitted from below): > > > >> > > > >> <?xml version="1.0" encoding="UTF-8"?><CAS version="2"> > > > >> <uima.cas.Sofa _indexed="0" _id="1" sofaNum="1" > > sofaID="_InitialView" > > > >> mimeType="text" sofaString=" Dr. > > > >> Nutritious Medical Nutrition Therapy for > > > >> Hyperlipidemia Referral from: Julie Tester, RD, > LD, > > > >> CNSD Phone contact: (555) 555-1212 Height: 144 cm > > > >> Current Weight: 45 kg Date of current weight: 02-29-2001 Admit > > > Weight: > > > >> 53 kg BMI: 18 kg/m2 Diet: General Daily Calorie > > > needs > > > >> (kcals): 1500 calories, assessed as HB + 20% for > > > activity. Daily > > > >> Protein needs: 40 grams, assessed as 1.0 g/kg. Pt has been > > on > > > a > > > >> 3-day calorie count and has had an average intake of 1100 calories. > > She > > > >> was instructed to drink 2-3 cans of liquid supplement to help > promote > > > >> weight gain. She agrees with the plan and has my number for further > > > >> assessment. May want a Resting Metabolic Rate as well. She takes an > > > aspirin > > > >> a day for knee pain. "/> > > > >> <org.apache.ctakes.typesystem.type.structured.DocumentID > > _indexed="1" > > > >> _id="8" documentID="testpatient_plaintext_1.txt"/> > > > >> <uima.tcas.DocumentAnnotation _indexed="1" _id="10" _ref_sofa="1" > > > >> begin="0" end="735" language="x-unspecified"/> > > > >> <org.apache.ctakes.typesystem.type.textspan.Segment _indexed="1" > > > >> _id="15" _ref_sofa="1" begin="0" end="735" id="SIMPLE_SEGMENT"/> > > > >> <org.apache.ctakes.typesystem.type.textspan.Sentence _indexed="1" > > > >> _id="21" _ref_sofa="1" begin="2" end="16" sentenceNumber="0"/> > > > >> .. > > > >> .. > > > >> .. > > > >> <org.apache.ctakes.typesystem.type.refsem.UmlsConcept _id="2975" > > > >> codingScheme="SNOMED" code="103733002" oid="103733002#SNOMED" > > > score="0.0" > > > >> disambiguated="false" cui="C0087111" tui="T061"/> > > > >> <uima.cas.FSArray _id="3005" size="4"> > > > >> <i>2975</i> > > > >> <i>2985</i> > > > >> <i>2965</i> > > > >> <i>2995</i> > > > >> </uima.cas.FSArray> > > > >> <org.apache.ctakes.typesystem.type.refsem.UmlsConcept _id="2924" > > > >> codingScheme="SNOMED" code="243121000" oid="243121000#SNOMED" > > > score="0.0" > > > >> disambiguated="false" cui="C0418981" tui="T061"/> > > > >> <uima.cas.FSArray _id="2934" size="1"> > > > >> <i>2924</i> > > > >> </uima.cas.FSArray> > > > >> <org.apache.ctakes.typesystem.type.refsem.UmlsConcept _id="2870" > > > >> codingScheme="RXNORM" code="1191" oid="1191#RXNORM" score="0.0" > > > >> disambiguated="false" cui="C0004057" tui="T121"/> > > > >> <org.apache.ctakes.typesystem.type.refsem.UmlsConcept _id="2880" > > > >> codingScheme="RXNORM" code="1191" oid="1191#RXNORM" score="0.0" > > > >> disambiguated="false" cui="C0004057" tui="T109"/> > > > >> <uima.cas.FSArray _id="2890" size="2"> > > > >> <i>2880</i> > > > >> <i>2870</i> > > > >> </uima.cas.FSArray> > > > >> </CAS> > > > >> > > > >> > > > >> I am not sure what's going on here, the job wasn't in running stage > > but > > > has > > > >> output... > > > >> It always got canceled by driver and showed run fails after > > > initializing, > > > >> so I didn't expect there would have any output file. > > > >> And I also doubt whether the content in output file is > correct....any > > > >> suggestion is appreciated! > > > >> > > > >> Thanks, > > > >> Yi-Wen > > > >> > > > > > >
