svn commit: r1532764 - /uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex
Author: eae
Date: Wed Oct 16 13:48:04 2013
New Revision: 1532764
URL: http://svn.apache.org/r1532764
Log:
UIMA-2682 More progress on DUCC application development
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex?rev=1532764r1=1532763r2=1532764view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex
Wed Oct 16 13:48:04 2013
@@ -129,7 +129,7 @@ which will connect back to the debug con
A DUCC job is a UIMA application comprised of user code broken into a
Collection
Reader running in the Job Driver and an Agreggate Analysis Engine (analysis
pipeline) running in one
or more Job Processes, with every Job Process running multiple instances of
the pipeline, each in a different
-thread. The major components of this UIMA application are as follows:
+thread. The major components of the basic Job Process application are as
follows:
\begin{itemize}
\item User Collection reader - segments the input collection in to Work Items
@@ -139,9 +139,46 @@ thread. The major components of this UIM
\item DUCC built-in Flow Controller - routes Work Item CASes to the CM and
optionally to the CC or AE \ CC.
\end{itemize}
-It is best to develop and debug the interactions between these components as
one,
-single-threaded UIMA aggregate. DUCC provides an easy way to accomplish this,
using
-the all\_in\_one specification parameter.
+\subsection{DUCC built-in Flow Controller}
+This flow controller provides separate flows for Work Item CASes and for CASes
produced by the CM and/or AE.
+Its behavior is controlled by the existence of a CM component, and then
further specified by the
+org.apache.uima.ducc.Workitem feature structure in the Work Item CAS.
+
+When no CM is defined the Work Item CAS is simply delivered to the AE, and
then to the CC if defined.
+Any CASes created by the AE will be routed to the CC.
+
+With a defined CM, the Work Item CAS is delivered only to the CM, and then
returned from the JP when processing
+of all child CASes created by the CM and AE has completed. Work Item CAS flow
can be further refined by the CR by
+creating a org.apache.uima.ducc.Workitem feature structure and setting the
setSendToLast feature to true,
+or by setting the setSendToAll feature to true.
+
+\subsection{Workitem Feature Structure}
+This feature structure is defined in DuccJobFlowControlTS.xml, located in
uima-ducc-common.jar.
+In addition to Work Item CAS flow control features, the WorkItem feature
structure includes features that are useful
+for a DUCC job application. Here is the complete list of features:
+
+\begin{description}
+ \item[sendToLast] (Boolean) - indicates the Work Item CAS be sent to the CC
+ \item[sendToAll] (Boolean) - indicates Work Item CAS be sent to the AE and CC
+ \item[inputspec] (String) - reference to Work Item input data
+ \item[outputspec] (String) - reference to Work Item output data
+ \item[encoding] (String) - useful for reading Work Item input data
+ \item[language] (String) - used by the CM for setting document text language
+ \item[bytelength] (Integer) - size of Work Item
+ \item[blockindex] (Integer) - used if a Work Item is one of multiple pieces
of an input resource
+ \item[blocksize] (Integer) - used to indicate block size for splitting an
input resource
+ \item[lastBlock] (Boolean) - indicates this is the last block of an input
resource
+\end{description}
+
+\subsection{Deployment Descriptor (DD) Jobs}
+Job Processes with arbitrary aggregate hierarchy, flow control and threading
can be fully specified
+via a complete UIMA AS Deployment Descriptor. DUCC will modify the input queue
to use DUCC's private
+broker and input queue name to correspond to the DUCC job ID.
+
+\subsection{Debugging}
+It is best to develop and debug the interactions between job application
components as one,
+single-threaded UIMA aggregate. DUCC provides an easy way to accomplish this,
for both basic
+and DD job models, using the all\_in\_one specification parameter.
\begin{description}
\item[all\_in\_one=local] When set to local, all Job components are run in
the same
svn commit: r1532796 - in /uima/site/trunk/uima-website: docs/downloads/gscl2013/slides_5.pdf docs/gscl13.html xdocs/gscl13.xml
Author: pkluegl Date: Wed Oct 16 15:27:42 2013 New Revision: 1532796 URL: http://svn.apache.org/r1532796 Log: no jira - added another set of slides (paper 5) Added: uima/site/trunk/uima-website/docs/downloads/gscl2013/slides_5.pdf (with props) Modified: uima/site/trunk/uima-website/docs/gscl13.html uima/site/trunk/uima-website/xdocs/gscl13.xml Added: uima/site/trunk/uima-website/docs/downloads/gscl2013/slides_5.pdf URL: http://svn.apache.org/viewvc/uima/site/trunk/uima-website/docs/downloads/gscl2013/slides_5.pdf?rev=1532796view=auto == Binary file - no diff available. Propchange: uima/site/trunk/uima-website/docs/downloads/gscl2013/slides_5.pdf -- svn:mime-type = application/octet-stream Modified: uima/site/trunk/uima-website/docs/gscl13.html URL: http://svn.apache.org/viewvc/uima/site/trunk/uima-website/docs/gscl13.html?rev=1532796r1=1532795r2=1532796view=diff == --- uima/site/trunk/uima-website/docs/gscl13.html (original) +++ uima/site/trunk/uima-website/docs/gscl13.html Wed Oct 16 15:27:42 2013 @@ -250,7 +250,7 @@ hosts two UIMA-related events: li p bSentiment Analysis and Visualization using UIMA and Solr/bbr / -iCarlos RodrÃguez-Penagos, David GarcÃa Narbona, Guillem Massó Sanabre, Jens Grivolla, Joan Codina/i - (a href=http://ceur-ws.org/Vol-1038/paper_5.pdf;paper/a) +iCarlos RodrÃguez-Penagos, David GarcÃa Narbona, Guillem Massó Sanabre, Jens Grivolla, Joan Codina/i - (a href=http://ceur-ws.org/Vol-1038/paper_5.pdf;paper/a, slides: a href=downloads/gscl2013/slides_5.pdfpdf/a) /p /li li Modified: uima/site/trunk/uima-website/xdocs/gscl13.xml URL: http://svn.apache.org/viewvc/uima/site/trunk/uima-website/xdocs/gscl13.xml?rev=1532796r1=1532795r2=1532796view=diff == --- uima/site/trunk/uima-website/xdocs/gscl13.xml (original) +++ uima/site/trunk/uima-website/xdocs/gscl13.xml Wed Oct 16 15:27:42 2013 @@ -88,7 +88,7 @@ hosts two UIMA-related events: li p bSentiment Analysis and Visualization using UIMA and Solr/bbr/ -iCarlos RodrÃguez-Penagos, David GarcÃa Narbona, Guillem Massó Sanabre, Jens Grivolla, Joan Codina/i - (a href=http://ceur-ws.org/Vol-1038/paper_5.pdf;paper/a) +iCarlos RodrÃguez-Penagos, David GarcÃa Narbona, Guillem Massó Sanabre, Jens Grivolla, Joan Codina/i - (a href=http://ceur-ws.org/Vol-1038/paper_5.pdf;paper/a, slides: a href=downloads/gscl2013/slides_5.pdfpdf/a) /p /li li
svn commit: r1532804 - /uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
Author: degenaro
Date: Wed Oct 16 15:53:16 2013
New Revision: 1532804
URL: http://svn.apache.org/r1532804
Log:
UIMA-3351 DUCC orchestrator (WS) employment of enums for ShareType and LogType
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java?rev=1532804r1=1532803r2=1532804view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
Wed Oct 16 15:53:16 2013
@@ -108,6 +108,9 @@ public class DuccHandler extends DuccAbs
private static IAuthenticationManager iAuthenticationManager =
AuthenticationManager.getInstance();
+ private enum ShareType { JD, MR, SPC, SPU, UIMA };
+ private enum LogType { POP, UIMA };
+
private String duccVersion
= duccContext+/version;
private String duccLoginLink=
duccContext+/login-link;
@@ -334,28 +337,25 @@ public class DuccHandler extends DuccAbs
}
*/
- private String buildLogFileName(IDuccWorkJob job, IDuccProcess process,
String type) {
+ private String buildLogFileName(IDuccWorkJob job, IDuccProcess process,
ShareType type) {
String retVal = ;
- if(type == UIMA) {
- String fType = type;
- retVal =
job.getDuccId().getFriendly()+-+fType+-+process.getNodeIdentity().getName()+-+process.getPID()+.log;
- }
- if(type == MR) {
- String fType = POP;
- retVal =
job.getDuccId().getFriendly()+-+fType+-+process.getNodeIdentity().getName()+-+process.getPID()+.log;
- }
- else if(type == SPU) {
- String fType = UIMA;
- retVal =
job.getDuccId().getFriendly()+-+fType+-+process.getNodeIdentity().getName()+-+process.getPID()+.log;
- }
- else if(type == SPC) {
- String fType = POP;
- retVal =
job.getDuccId().getFriendly()+-+fType+-+process.getNodeIdentity().getName()+-+process.getPID()+.log;
- }
- else if(type == JD) {
+ switch(type) {
+ case UIMA:
+ retVal =
job.getDuccId().getFriendly()+-+LogType.UIMA.name()+-+process.getNodeIdentity().getName()+-+process.getPID()+.log;
+ break;
+ case MR:
+ retVal =
job.getDuccId().getFriendly()+-+LogType.POP.name()+-+process.getNodeIdentity().getName()+-+process.getPID()+.log;
+ break;
+ case SPU:
+ retVal =
job.getDuccId().getFriendly()+-+LogType.UIMA.name()+-+process.getNodeIdentity().getName()+-+process.getPID()+.log;
+ break;
+ case SPC:
+ retVal =
job.getDuccId().getFriendly()+-+LogType.POP.name()+-+process.getNodeIdentity().getName()+-+process.getPID()+.log;
+ break;
+ case JD:
retVal = jd.out.log;
+ break;
}
-
return retVal;
}
@@ -405,7 +405,7 @@ public class DuccHandler extends DuccAbs
String pname_idJob = idJob;
String pname_idPro = idPro;
- private void buildJobProcessListEntry(StringBuffer sb, DuccWorkJob job,
IDuccProcess process, String type, int counter) {
+ private void buildJobProcessListEntry(StringBuffer sb, DuccWorkJob job,
IDuccProcess process, ShareType type, int counter) {
String location = buildJobProcessListEntry;
String logsjobdir =
job.getUserLogsDir()+job.getDuccId().getFriendly()+File.separator;
String logfile = buildLogFileName(job, process, type);
@@ -1071,13 +1071,13 @@ public class DuccHandler extends DuccAbs
while(iterator.hasNext()) {
DuccId processId = iterator.next();
IDuccProcess process =
job.getDriver().getProcessMap().get(processId);
- buildJobProcessListEntry(sb, job, process,
JD, ++counter);
+ buildJobProcessListEntry(sb, job, process,
ShareType.JD, ++counter);
}
iterator =
svn commit: r1532810 - /uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
Author: degenaro
Date: Wed Oct 16 16:15:47 2013
New Revision: 1532810
URL: http://svn.apache.org/r1532810
Log:
UIMA-3351 DUCC orchestrator (WS) employment of enums for ShareType and LogType
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java?rev=1532810r1=1532809r2=1532810view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
Wed Oct 16 16:15:47 2013
@@ -423,17 +423,19 @@ public class DuccHandler extends DuccAbs
long id = process.getDuccId().getFriendly();
System.out.println(id);
*/
- if(type.equals(SPC)) {
+ switch(type) {
+ case SPC:
sb.append(job.getDuccId().getFriendly()+.+process.getDuccId().getFriendly());
- }
- else if(type.equals(SPU)) {
+ break;
+ case SPU:
sb.append(job.getDuccId().getFriendly()+.+process.getDuccId().getFriendly());
- }
- else if(type.equals(MR)) {
+ break;
+ case MR:
sb.append(job.getDuccId().getFriendly()+.+process.getDuccId().getFriendly());
- }
- else {
+ break;
+ default:
sb.append(process.getDuccId().getFriendly());
+ break;
}
sb.append(/td);
// Log
@@ -485,7 +487,8 @@ public class DuccHandler extends DuccAbs
sb.append(/td);
break;
default:
- if(type.equals(MR)) {
+ switch(type) {
+ case MR:
sb.append(td);
int code = process.getProcessExitCode();
if(LinuxSignals.isSignal(code)) {
@@ -500,9 +503,9 @@ public class DuccHandler extends DuccAbs
else {
sb.append(ProgramExitCode+=+code);
}
- sb.append(/td);
- }
- else {
+ sb.append(/td);
+ break;
+ default:
ProcessDeallocationType deallocationType =
process.getProcessDeallocationType();
String toolTip =
ProcessDeallocationType.getToolTip(deallocationType);
if(toolTip == null) {
@@ -519,6 +522,7 @@ public class DuccHandler extends DuccAbs
break;
}
sb.append(/td);
+ break;
}
break;
}
@@ -534,83 +538,84 @@ public class DuccHandler extends DuccAbs
}
sb.append(/td);
// Time:init
- if(type.equals(MR)) {
- //
- }
- else {
- StringBuffer loadme = new StringBuffer();
- String initTime = 00;
- String itd0 = td align=\right\;
- String itd1 = /td;
- String isp0 = span;
- String isp1 = /span;
- try {
- TimeWindow t = (TimeWindow)
process.getTimeWindowInit();
- if(t != null) {
- long now = System.currentTimeMillis();
- String tS = t.getStart(+now);
- String tE = t.getEnd(+now);
- initTime = getDuration(jobid,tE,tS);
- if(t.isEstimated()) {
- isp0 = span
title=\estimated\ class=\health_green\;
- }
- else {
- isp0 = span
class=\health_black\;
+ switch(type) {
+ case MR:
+ break;
+
svn commit: r1532852 - in /uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common: DuccProcess.java IDuccProcess.java
Author: degenaro
Date: Wed Oct 16 18:34:16 2013
New Revision: 1532852
URL: http://svn.apache.org/r1532852
Log:
UIMA-3351 DUCC orchestrator (OR) should not count Time Run for JP's that are
not used
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/DuccProcess.java
uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/IDuccProcess.java
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/DuccProcess.java
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/DuccProcess.java?rev=1532852r1=1532851r2=1532852view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/DuccProcess.java
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/DuccProcess.java
Wed Oct 16 18:34:16 2013
@@ -264,6 +264,27 @@ public class DuccProcess implements IDuc
return retVal;
}
+
+ public boolean isAssignedWork() {
+ boolean retVal = true;
+ IDuccProcessWorkItems pwi = getProcessWorkItems();
+ if(pwi == null) {
+ retVal = false;
+ }
+ else {
+ if((pwi.getCountDispatch() == 0)
+(pwi.getCountDone() == 0 )
+(pwi.getCountError() == 0)
+(pwi.getCountLost() == 0)
+(pwi.getCountPreempt() == 0)
+(pwi.getCountRetry() == 0)
+ ) {
+ retVal = false;
+ }
+ }
+ return retVal;
+ }
+
public boolean isActive() {
boolean retVal = false;
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/IDuccProcess.java
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/IDuccProcess.java?rev=1532852r1=1532851r2=1532852view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/IDuccProcess.java
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-transport/src/main/java/org/apache/uima/ducc/transport/event/common/IDuccProcess.java
Wed Oct 16 18:34:16 2013
@@ -75,6 +75,8 @@ public interface IDuccProcess extends Se
public boolean isPreempted();
public boolean isComplete();
+ public boolean isAssignedWork();
+
public boolean isDeallocated();
public boolean isDefunct();
svn commit: r1532854 - /uima/sandbox/uima-ducc/trunk/uima-ducc-orchestrator/src/main/java/org/apache/uima/ducc/orchestrator/ProcessAccounting.java
Author: degenaro
Date: Wed Oct 16 18:35:23 2013
New Revision: 1532854
URL: http://svn.apache.org/r1532854
Log:
UIMA-3351 DUCC orchestrator (OR) should not count Time Run for JP's that are
not used
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-orchestrator/src/main/java/org/apache/uima/ducc/orchestrator/ProcessAccounting.java
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-orchestrator/src/main/java/org/apache/uima/ducc/orchestrator/ProcessAccounting.java
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-orchestrator/src/main/java/org/apache/uima/ducc/orchestrator/ProcessAccounting.java?rev=1532854r1=1532853r2=1532854view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-orchestrator/src/main/java/org/apache/uima/ducc/orchestrator/ProcessAccounting.java
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-orchestrator/src/main/java/org/apache/uima/ducc/orchestrator/ProcessAccounting.java
Wed Oct 16 18:35:23 2013
@@ -34,7 +34,6 @@ import org.apache.uima.ducc.transport.ev
import org.apache.uima.ducc.transport.event.common.DuccWorkMap;
import org.apache.uima.ducc.transport.event.common.IDuccProcess;
import org.apache.uima.ducc.transport.event.common.IDuccProcessMap;
-import org.apache.uima.ducc.transport.event.common.IDuccProcessWorkItems;
import org.apache.uima.ducc.transport.event.common.IDuccState.JobState;
import org.apache.uima.ducc.transport.event.common.IDuccWork;
import org.apache.uima.ducc.transport.event.common.IDuccWorkExecutable;
@@ -517,32 +516,16 @@ public class ProcessAccounting {
// uima-3351
private void adjustRunTime(IDuccProcess process) {
- IDuccProcessWorkItems pwi = process.getProcessWorkItems();
- if(pwi == null) {
- setNoRunTime(process);
+ if(!process.isAssignedWork()) {
+ ITimeWindow twr = process.getTimeWindowRun();
+ if(twr == null) {
+ twr = new TimeWindow();
+ process.setTimeWindowRun(twr);
+ }
+ long time = 0;
+ twr.setStartLong(time);
+ twr.setEndLong(time);
}
- else {
- if((pwi.getCountDispatch() == 0)
-(pwi.getCountDone() == 0 )
-(pwi.getCountError() == 0)
-(pwi.getCountLost() == 0)
-(pwi.getCountPreempt() == 0)
-(pwi.getCountRetry() == 0)
- ) {
- setNoRunTime(process);
- }
- }
- }
-
- private void setNoRunTime(IDuccProcess process) {
- ITimeWindow twr = process.getTimeWindowRun();
- if(twr == null) {
- twr = new TimeWindow();
- process.setTimeWindowRun(twr);
- }
- long time = 0;
- twr.setStartLong(time);
- twr.setEndLong(time);
}
// /uima-3351
svn commit: r1532855 - /uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
Author: degenaro
Date: Wed Oct 16 18:36:16 2013
New Revision: 1532855
URL: http://svn.apache.org/r1532855
Log:
UIMA-3351 DUCC orchestrator (OR) should not count Time Run for JP's that are
not used
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java?rev=1532855r1=1532854r2=1532855view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-web/src/main/java/org/apache/uima/ducc/ws/server/DuccHandler.java
Wed Oct 16 18:36:16 2013
@@ -623,26 +623,48 @@ public class DuccHandler extends DuccAbs
String rtd1 = /td;
String rsp0 = span;
String rsp1 = /span;
- try {
- TimeWindow t = (TimeWindow) process.getTimeWindowRun();
- if(t != null) {
- long now = System.currentTimeMillis();
- String tS = t.getStart(+now);
- String tE = t.getEnd(+now);
- runTime = getDuration(jobid,tE,tS);
- if(t.isEstimated()) {
- rsp0 = span title=\estimated\
class=\health_green\;
- }
- else {
- rsp0 = span class=\health_black\;
- }
+ // UIMA-3351
+ boolean useTimeRun = true;
+ switch(type) {
+ case SPC:
+ break;
+ case SPU:
+ break;
+ case MR:
+ break;
+ case JD:
+ break;
+ case UIMA:
+ if(!process.isAssignedWork()) {
+ useTimeRun = false;
}
+ break;
+ default:
+ break;
}
- catch(Exception e) {
- duccLogger.trace(location, jobid, no worries, e);
- }
- catch(Throwable t) {
- duccLogger.trace(location, jobid, no worries, t);
+ // /UIMA-3351
+ if(useTimeRun) {
+ try {
+ TimeWindow t = (TimeWindow)
process.getTimeWindowRun();
+ if(t != null) {
+ long now = System.currentTimeMillis();
+ String tS = t.getStart(+now);
+ String tE = t.getEnd(+now);
+ runTime = getDuration(jobid,tE,tS);
+ if(t.isEstimated()) {
+ rsp0 = span
title=\estimated\ class=\health_green\;
+ }
+ else {
+ rsp0 = span
class=\health_black\;
+ }
+ }
+ }
+ catch(Exception e) {
+ duccLogger.trace(location, jobid, no worries,
e);
+ }
+ catch(Throwable t) {
+ duccLogger.trace(location, jobid, no worries,
t);
+ }
}
sb.append(rtd0);
sb.append(rsp0);
svn commit: r1532869 - /uima/sandbox/uima-ducc/trunk/uima-ducc-examples/src/main/java/org/apache/uima/ducc/test/service/UIMA_Service.java
Author: cwiklik
Date: Wed Oct 16 19:20:54 2013
New Revision: 1532869
URL: http://svn.apache.org/r1532869
Log:
UIMA-2674 align with uima-as version
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-examples/src/main/java/org/apache/uima/ducc/test/service/UIMA_Service.java
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-examples/src/main/java/org/apache/uima/ducc/test/service/UIMA_Service.java
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-examples/src/main/java/org/apache/uima/ducc/test/service/UIMA_Service.java?rev=1532869r1=1532868r2=1532869view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-examples/src/main/java/org/apache/uima/ducc/test/service/UIMA_Service.java
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-examples/src/main/java/org/apache/uima/ducc/test/service/UIMA_Service.java
Wed Oct 16 19:20:54 2013
@@ -164,7 +164,7 @@ public class UIMA_Service implements App
* @throws Exception
*/
public SpringContainerDeployer deploy(String[] springContextFiles) throws
Exception {
-SpringContainerDeployer springDeployer = new SpringContainerDeployer();
+SpringContainerDeployer springDeployer = new SpringContainerDeployer(this);
// now try to deploy the array of spring context files
springDeployer.deploy(springContextFiles);
// Poll the deployer for the initialization status. Wait for either
successful
svn commit: r1532926 - in /uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook: duccbook.tex part3/ducc-applications.tex
Author: eae
Date: Wed Oct 16 22:22:33 2013
New Revision: 1532926
URL: http://svn.apache.org/r1532926
Log:
UIMA-2682 More progress on DUCC application development
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex?rev=1532926r1=1532925r2=1532926view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/duccbook.tex
Wed Oct 16 22:22:33 2013
@@ -22,6 +22,9 @@
\usepackage{caption}
\captionsetup[table]{skip=18pt}
+%list margins
+\usepackage{enumitem}
+
\title{\Huge \textbf{Distributed UIMA Cluster Computing}}
\author{The DUCC Team}
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex?rev=1532926r1=1532925r2=1532926view=diff
==
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part3/ducc-applications.tex
Wed Oct 16 22:22:33 2013
@@ -5,47 +5,43 @@
A DUCC job consists of two process types, a Job Driver process and one or more
Job Processes. These processes are connected together via UIMA-AS.
The Job Driver uses the UIMA-AS client API to send Work Item CASes
-to the job's input queue. Job Processes instantiate the analytic pipeline are
deployed
-as UIMA-AS services comsuming CASes from the job input queue.
+to the job's input queue. Job Processes containing the analytic pipeline are
deployed
+as UIMA-AS services comsume CASes from the job input queue.
-The Job Driver process wraps the job's Collection Reader whose
+The Job Driver process wraps the job's Collection Reader (CR). The CR
function is to define the collection of Work Items to be processed.
-The CollectionReader returns a small CAS for each Work Item containing a
-reference to the Work Item data. The Job Driver then submits each CAS to the
job's
-input queue so that it can be delivered to an available pipeline in a Job
Process.
+The Collection Reader returns a small CAS for each Work Item containing a
+reference to the Work Item data which is sent to the job's
+input queue so that it can be delivered to the next available pipeline in a
Job Process.
-A typical job's analytic pipeline consists of an Aggregate Analysis Engine
comprised by
+A basic job's analytic pipeline consists of an Aggregate Analysis Engine
comprised by
the user specified CAS Multiplier (CM), Analysis Engine (AE) and CAS
Consumer (CC) components, along with a built-in DUCC Flow Controller.
The Work Item CAS is typically sent only to the CM and returned by
the Job Process when all child CASes produced by the CM have completed
processing; optionally the CR can configure Work Item CAS flow to go to the CC
-or to the AE \ CC to flush context if necessary.
+or to the AE \ CC to complete all processing for that Work Item.
\begin{description}
\item[Note:] Although the Job Driver will receive back the Work
Item CAS,
- there is no provision for any user code to analyze results there.
Therefore a
+ there is no provision for any user code to receive the CAS.
Therefore a
Job Process typically adds no results to a Work Item CAS.
\end{description}
\subsection{Basic Job Process Threading Model}
In addition to the pipeline definition of explicitly named CM, AE and CC
components, the job
specification also includes the number of pipeline threads to run in each
- Job Process. Each pipeline thread receives Work Items independently.
+ Job Process (using the job specification parameter: process\_thread\_count).
+ Each pipeline thread receives Work Items independently.
DUCC creates an aggreate descriptor for the pipeline, and then creates a
Deployment Descriptor for the Job Process which deploys the specified number
- of synchronous pipelines (using the job specification parameter:
process\_thread\_count)
-
- UIMA configuration parameters in the CR, CM, AE or CC components can be
overriden using
- job specification parameters: driver\_descriptor\_CR\_overrides,
process\_descriptor\_CM\_overrides,
- process\_descriptor\_AE\_overrides and process\_descriptor\_CC\_overrides,
respectively.
+ of synchronous
