Author: challngr
Date: Wed Jun 12 19:18:00 2013
New Revision: 1492381
URL: http://svn.apache.org/r1492381
Log:
UIMA-2682 Duccbook updates.
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/ducc-api.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/job-logs.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/services.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/job-details.tex
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/services.tex
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex?rev=1492381&r1=1492380&r2=1492381&view=diff
==============================================================================
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/cli/ducc-services.tex
Wed Jun 12 19:18:00 2013
@@ -66,7 +66,7 @@
Prints the usage text to the console.
\end{description}
- \subsection{ducc\_services $--$register Options}
+ \subsection{ducc\_services --register Options}
\label{subsec:cli.ducc-services.register}
The {\em register} function submits a service specification to DUCC.
DUCC stores this
information until it is {\em unregistered}. Once registered, a service
may be
@@ -89,7 +89,7 @@
\end{description}
- \subsection{ducc\_services $--$start Options}
+ \subsection{ducc\_services --start Options}
The start function instructs DUCC to allocate resources for a service and
to start it in those
resources. The service remains running until explicitly stopped. DUCC will
attempt to keep the
@@ -129,7 +129,7 @@ ducc_services --start UIMA-AS:Service23:
\end{verbatim}
\end{description}
- \subsection{ducc\_services $--$stop Options}
+ \subsection{ducc\_services --stop Options}
The stop function instructs DUCC to stop some number of service instances.
If no specific number
is specified, all instances are stopped. This is used only for registered
services. Use
\hyperref[sec:cli.service-cancel]{{\em ducc\_service\_cancel}} command to
stop submitted services.
@@ -170,7 +170,7 @@ ducc_services --stop UIMA-AS:Service23:t
\end{description}
- \subsection{ducc\_services $--$modify Options}
+ \subsection{ducc\_services --modify Options}
The modify function dynamically updates some of the attributes of a
registered service.
\begin{description}
@@ -217,7 +217,7 @@ ducc_services --stop UIMA-AS:Service23:t
\end{verbatim}
\end{description}
- \subsection{ducc\_services $--$query Options}
+ \subsection{ducc\_services --query Options}
The query function returns details about all known services of all types
and classes, including
the DUCC ids of the service instances (for submitted and registered
services), the DUCC ids of
the jobs using each service, and a summary of each service's queue and
performance statistics,
@@ -225,7 +225,7 @@ ducc_services --stop UIMA-AS:Service23:t
All information returned by {\em ducc\_services $--$query} is also
available via the
\hyperref[ws:services-page]{Services Page} of the Web Server as well as
the DUCC Service API (see
- the \href{apidocs/index.html}{JavaDoc}).
+ the JavaDoc).
\begin{description}
\item[$--$query {[service-id or endpoint]}] This indicates that a service
is to be stopped. The
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/ducc-api.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/ducc-api.tex?rev=1492381&r1=1492380&r2=1492381&view=diff
==============================================================================
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/ducc-api.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/ducc-api.tex
Wed Jun 12 19:18:00 2013
@@ -1,8 +1,8 @@
\section{Overview Of The DUCC API}
- The DUCC API provides a simple programatic interface to DUCC for submission
and
- cancelation of work. The DUCC CLI is implemented using the API and
provides a
- model for how to use the API.
+ The DUCC API provides a simple programmatic (Java) interface to DUCC for
submission and
+ cancellation of work. (Note that the DUCC CLI is implemented using the API
and provides a
+ model for how to use the API.)
All the API objects are instantiated using the same arguments as the CLI.
The API
provides three variants for supplying arguments:
@@ -12,9 +12,9 @@
\item A Java Properties file for example {\tt DuccJobSubmit(Properties
args)}.
\end{enumerate}
- Upon instantiation of an API object, the {\tt boolean execute()} method is
called. This
+ After instantiation of an API object, the {\tt boolean execute()} method is
called. This
method transmits the arguments to DUCC. If DUCC receives and accepts the
args, the method
- return ``true'', otherwise it returns ``false. Methods are provided to
retrieve relevent
+ return ``true'', otherwise it returns ``false. Methods are provided to
retrieve relevant
information when the {\tt execute()} returns such as IDs, messages, etc.
In the case of jobs and managed reservations, if the specification
requested debug,
@@ -25,10 +25,10 @@
the constructor. The callback object provides a means to direct messages
to the
API user. If the callback is not provided, messages are written to
standard output.
- The API is thread-safe, so developers may manage multiple, simulataneous
requests to
+ The API is thread-safe, so developers may manage multiple, simultaneous
requests to
DUCC.
- Below is the ``main()'' method of DuccJobSubmit, demonstating the use of
the API:
+ Below is the ``main()'' method of DuccJobSubmit, demonstrating the use of
the API:
\begin{verbatim}
public static void main(String[] args) {
try {
@@ -53,21 +53,8 @@
\section{Compiling and Running With the DUCC API}
- Three jar files are required in the CLASSPATH to compile programs using the
DUCC API. These
- jars are reside in \ducchome/lib:
- \begin{enumerate}
- \item uima-ducc-cli.jar
- \item uima-ducc-common.jar
- \item uima-ducc-transport.jar
- \end{enumerate}
-
- The jar file {\tt uima-ducc-cli.jar} contains a manifest with the full
runtime CLASSPATH
- needed for the API. Therefore it is the only jar file needed for
execution. For
- example:
-\begin{verbatim}
- export CLASSPATH=${DUCC_HOME}/lib/uima-ducc-cli.jar:${CLASSPATH}
- java <my ducc code>
-\end{verbatim}
+ A single DUCC jar file is required for both compilation and execution of
the DUCC API,
+ {\tt uima-ducc-cli.jar}. This jar is found in \duccruntime/lib.
\section{Java API}
\ifpdf
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/job-logs.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/job-logs.tex?rev=1492381&r1=1492380&r2=1492381&view=diff
==============================================================================
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/job-logs.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/job-logs.tex
Wed Jun 12 19:18:00 2013
@@ -6,7 +6,7 @@
\chapter{Job Logs}
\label{chap:job-logs}
-The DUCC logs are managed by Apache log4j. The DUCC log4j configuration file
is
+\paragraph{Overview}The DUCC logs are managed by Apache log4j. The DUCC log4j
configuration file is found in
\duccruntime/log4j.xml. It is not in the scope of this document to describe
log4j or its
configuration mechanism. Details on log4j can be found at
\url{http://logging.apache.org/log4j}.
@@ -14,61 +14,64 @@ The "user logs" are the Job Driver (JD)
of a job. The JD log is divided between two physical files:
\begin{enumerate}
- \item Stdout and default UIMA logging output written by the UIMA collection
reader.
-
- \item The diagnostic logs written the the DUCC JD wrapper around the job's
collection reader.
+ \item Stdout and default UIMA logging output written by the UIMA collection
reader.
+ \item The diagnostic logs written by the DUCC JD wrapper around the job's
collection reader.
\end{enumerate}
-A number of other useful files are written to the log directory:
-\begin{enumerate}
-
- \item A properties file containing the full job specification for the job.
This includes all the
- parameters specified by the user as well as the default parameters. This
file is called
- {\tt job-specification.properties.}
-
-
- \item The UIMA pipeline descriptor constructed by DUCC that describes the
process that is
- dispatched to each Job Process (JP). The name of this file is of the form
+\paragraph{Contents of the Log Directory} A number of other useful files are
written to the log directory:
+\begin{enumerate}
+ \item A properties file containing the full job specification for the job.
This includes all the
+ parameters specified by the user as well as the default parameters. This
file is called
+ {\tt job-specification.properties.}
+ \item The UIMA pipeline descriptor constructed by DUCC that describes the
process that is
+ dispatched to each Job Process (JP). The name of this file is of the
form
\begin{verbatim}
JOBID-uima-ae-descriptor-PROCESS.xml
\end{verbatim}
+ where
+ \begin{description}
+ \item[JOBID] This is the numerical id of the job as assigned by DUCC.
+ \item[PROCESS] This is the process id of the Job Driver (JD) process.
+ \end{description}
- where
-
- \begin{description}
- \item[JOBID] This is the numerical id of the job as assigned by DUCC.
- \item[PROCESS] This is the process id of the Job Driver (JD) process.
- \end{description}
-
- \item The UIMA-AS service descriptor that defines the process that
defines the job as as UIMAAS
- service. The name of this file is of the form
+ \item The UIMA-AS service descriptor that defines the process that defines
the job as as UIMAAS
+ service. The name of this file is of the form
\begin{verbatim}
JOBID-uima-as-dd-PROCESS.xml
\end{verbatim}
-
- where
- \begin{description}
- \item[JOBID] This is the numerical id of the job as assigned by
DUCC.
- \item[PROCESS] This is the process id of the Job Driver (JD)
process.
- \end{description}
-
- \item A colllection of gzipped ``json'' files containing the performance
breakdown of the job.
- These can be read and formatted using
\hyperref[sec:cli.ducc-perf-stats]{ducc\_perf\_stats.}
- \end{enumerate}
+
+ where
+ \begin{description}
+ \item[JOBID] This is the numerical id of the job as assigned by DUCC.
+ \item[PROCESS] This is the process id of the Job Driver (JD) process.
+ \end{description}
+
+ \item A colllection of gzipped ``json'' files containing the performance
breakdown of the job.
+ These can be read and formatted using
\hyperref[sec:cli.ducc-perf-stats]{ducc\_perf\_stats.}
+\end{enumerate}
-The Job Process logs are written to the configured log directory and are of
the following form:
+\paragraph{Job Process Logs}
+The Job Process logs are written to the configured log directory. There is
one job process log
+for every job processes started for the job. The log names are of the
following form:
\begin{verbatim}
JOBID-TYPE-NODE-PROCESS.log
\end{verbatim}
where
\begin{description}
- \item[JOBID] This is the numerical id of the job as assigned by DUCC.
- \item[TYPE] This is either the string "UIMA" for JP logs, or "JD" for JD
logs.
- \item[NODE] This is the name of the machine where the process ran.
- \item[PROCESS] This is the Unix process id of the process on the indicated
node.
+\item[JOBID] This is the numerical id of the job as assigned by DUCC.
+\item[TYPE] This is either the string "UIMA" for JP logs, or "JD" for JD logs.
+\item[NODE] This is the name of the machine where the process ran.
+\item[PROCESS] This is the Unix process id of the process on the indicated
node.
\end{description}
+\paragraph{Job Driver Logs}
+There are several Job Driver logs.
+ 988-JD-agent86-1-58087.log
+ jd.out.log
+ jd.err.log
+
+\paragraph{Sample Log Directory}
This shows the contents a sample log directory for a small job that consisted
of two processes.
\begin{verbatim}
@@ -78,37 +81,52 @@ This shows the contents a sample log dir
100-UIMA-bluej290-2-32766.log
100-UIMA-bluej291-63-13594.log
jd.out.log
- job-performance-summary.ser
job-specification.properties
+ job-performance-summary.json.gz
+ job-processes-data.json.gz
+ work-item-status.json.gz
+
\end{verbatim}
In this example,
-\begin{itemize}
- \item[] The file {\tt 100-JD-bluej290-1-29383.log} is the diagnostic JD
log, where the JD executed on node
+\begin{description}
+ \item[100-JD-bluej290-1-29383.log] \hfill \\
+ is the diagnostic JD log, where the JD executed on node
bluej290-1 in process 29383.
- \item[] The file {\tt 100-uima-ae-descriptor-29383.xml} is the UIMA
pipeline descriptor describing the
+ \item[100-uima-ae-descriptor-29383.xml] \hfill \\
+ is the UIMA pipeline descriptor describing the
service process that is launched in each JP, where the JD process is
29383.
- \item[] The file {\tt 100-uima-as-dd-29383.xml} is the UIMA-AS service
descriptor where the client is
+ \item[100-uima-as-dd-29383.xml] \hfill \\
+ is the UIMA-AS service descriptor where the client is
the JD process running in process 29383.
- \item[] The file {\tt 100-UIMA-bluej290-2-32766.log} is a JP log for job
100, that ran on node
+ \item[100-UIMA-bluej290-2-32766.log] \hfill \\
+ is a JP log for job 100, that ran on node
bluej290-2, in process 32766.
- \item[] The file {\tt 100-UIMA-bluej291-63-13594.log} is a JP log for job
100, that ran on node
+ \item[100-UIMA-bluej291-63-13594.log] \hfill \\
+ is a JP log for job 100, that ran on node
bluej291-63, in process 13594
- \item[] The file {\tt jd.out.log} is the user's JD log, containing the
user's collection reader output.
-
- \item[] These files are the performance summary files:
-\begin{verbatim}
-job-performance-summary.json.gz
-job-processes-data.json.gz
-job-specification.properties
-work-item-status.json.gz
-\end{verbatim}
+ \item[jd.out.log] \hfill \\
+ is the user's JD log, containing the user's collection reader output.
-\end{itemize}
+ \item[job-performance-summary.json.gz] \hfill \\
+ This contains the raw statistics describing
+ the operation of each analytic. It corresponds to
\hyperref[sec:performance]{Performance}
+ tab of the Job Details page in the Web Server.
+
+ \item[job-process.json.gz] \hfill \\
+ This contains the raw statistics describing
+ the performance of each individual job process. It corresponds
\hyperref[sec:ws-processes]{Processes}
+ tab of the Job Details page in the Web Server.
+
+ \item[work-item-status.json.gz] \hfill \\
+ This contains the raw statistics describing
+ the operation of each individual work-item. It corresponds to
\hyperref[sec:ws-work-items]{Work Items}
+ tab of the Job Details page in the Web Server.
+ \end{description}
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/services.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/services.tex?rev=1492381&r1=1492380&r2=1492381&view=diff
==============================================================================
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/services.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/services.tex
Wed Jun 12 19:18:00 2013
@@ -5,8 +5,8 @@
\item A service is one or more long-running processes that await
requests from
UIMA pipeline components and return something in response. These
processes
are usually managed by DUCC but need not be.
- \item A sservice is accompanied by a small program called a
``pinger'' that
- the DUCC Service Manager uses to guage the availability and health
of the
+ \item A service is accompanied by a small program called a
``pinger'' that
+ the DUCC Service Manager uses to gauge the availability and health
of the
service. This process must always be supplied to DUCC; however,
DUCC will
supply a default pinger for UIMA-AS services.
\end{itemize}
@@ -18,13 +18,13 @@
\begin{itemize}
\item Insure services are available for jobs before allowing the
jobs to start. This fail-fast
- prevents unncessary allocation of resources (with potential
eviction of healthy processes)
- for jobs that can't run, as well as quick feedback to users that
something is amis.
+ prevents unnecessary allocation of resources (with potential
eviction of healthy processes)
+ for jobs that can't run, as well as quick feedback to users that
something is amiss.
- \item Manage the startup and management of services: allocate
resources, spawn the
+ \item Manage the start-up and management of services: allocate
resources, spawn the
processes, insure the processes stay alive, handle errors, etc.
- \item Report on the state and availablity of services.
+ \item Report on the state and availability of services.
\end{itemize}
@@ -55,9 +55,9 @@
The {\em service-type} must be either UIMA-AS or CUSTOM.
The {\em unique id and contact information} is any string needed to
insure the service is
- uniquely named. This string is passsed to the service pinger and must
contain sufficient
+ uniquely named. This string is passed to the service pinger and must
contain sufficient
information for the pinger to contact the service. For UIMA-AS
services, service endpoint is
- inferred by the CLI by inspection of the UIMA XML descriptor. The
UIMA-AS
+ inferred by the CLI by inspection of the UIMA-AS service's DD XML
descriptor. The UIMA-AS
service endpoint is always of the form:
\begin{verbatim}
UIMA-AS:queue-name:broker-url
@@ -73,21 +73,21 @@
\section{Service Classes.}
\label{sec:service.classes}
- Services may be started externally to DUCC, explicitly through
- DUCC as a job, or as registered services. It is also possible to
register a ``ping-only''
- service that has no process managed by DUCC, consisting only of a
pinger. To distiinguish
- among the various behaviours and management mechanisms for service we
define a number
- of {\em service classes}.
+ Services may be started externally to DUCC, explicitly through DUCC as
with
+ \hyperref[sec:cli.service-submit]{ducc\_service\_submit}, or as
registered services. It is
+ also possible to register a ``ping-only'' service that has no process
managed by DUCC,
+ consisting only of a pinger. To distinguish among the various
behaviours and management
+ mechanisms for service we define a number of {\em service classes}.
\subsection{Implicit Services.}
\label{sec:services.implicit}
- An external service is started externally to DUCC and discovered by DUCC
only when it is
+ An implicit service is started externally to DUCC and discovered by DUCC
only when it is
referenced by a job's {\em service\_dependency} parameter. Such a
service is called
an {\em implicit} service.
- On submission of a job with a dependency on a, the SM sets up a "ping"
thread based on the
+ On submission of a job with a dependency on a service, the SM sets up a
"ping" thread based on the
service endpoint of the dependency to discover if the service exists at
the endpoint. If so,
- the SM adds the service to its list of known services and marks the job
"ready to schedule".
+ the SM updates its list of known services and marks the job "ready to
schedule".
When jobs referencing implicit services exit, a timer is set and DUCC
continues to monitor the
service against the possibility that subsequent jobs will need it. Once
the last job using the
@@ -97,20 +97,28 @@
\subsubsection{Implicit UIMA-AS services}
If a job \hyperref[sec:service.endpoints]{references} a UIMA-AS service
that is not known to the
DUCC Service Manager, the Service Manager will start
- its default internal pinger to monitoring the ActiveMq queue and service
response. The
- service is monitored throughout the lifetime of the job. If the service
should stop
+ its default internal pinger. The default pinger uses JMX to monitor the
ActiveMQ queue statistics,
+ and also issues UIMA-AS ``get-meta'' requests to the service to insure
it is responsive. The
+ service is monitored throughout the lifetime of the job.
+
+ If the service should stop
responding, its state is updated to "not-responding" but the job is
allowed to continue as
- DUCC cannot tell if the job is using the service.
+ DUCC cannot tell if the job is using the service. New jobs are not
granted resources if
+ their pre-requisite services are not active, as determined by the ping
process.
\subsubsection{Implicit CUSTOM services, or ``ping-only'' Service}
If a job \hyperref[sec:service.endpoints]{references} a CUSTOM service,
the service must be
registered and have a CUSTOM \hyperref[sec:service.pingers]{pinger}
associated with it. Such a
- service is refered to as a ``ping-only'' service. DUCC will start the
pinger and monitor the
- service as expected.
+ service is referred to as a ``ping-only'' service. DUCC will start the
pinger and monitor the
+ service as expected.
+
+ A CUSTOM pinger is written to an interface defined by the DUCC API. The
pinger may perform
+ any action needed to determine if the CUSTOM service is active and
responsive and may return
+ a string for display in the web server, summarizing service statistics.
\subsection{Submitted Services.} A
\hyperref[sec:cli.service-submit]{submitted service} is a
- service that is submitted to DUCC as a with the ducc\_service\_submit
CLI. Both UIMA-AS and
+ service that is submitted to DUCC with the ducc\_service\_submit CLI.
Both UIMA-AS and
CUSTOM services may be submitted for execution by DUCC. Because DUCC is
managing this service
it can provide more support than for implicit services. However, DUCC
does not persist the
service definition. If a submitted service exits involuntarily
(crashes), DUCC will make
@@ -118,36 +126,41 @@
will stop the service.
A submitted service is intended for short-term services, and for easy
debugging of services.
- To persist a service over system restarts and other failures, services
should be registered.
+ To persist a service over system restarts and other failures, services
must be registered.
\subsection{Registered Services.}
\hyperref[sec:cli.ducc-services]{Registered services} are
- fully managed by DUCC. A service is registered with DUCC using the DUCC
services CLI. Service
- registrations are persisted by DUCC and last over DUCC or system
restarts.
+ fully managed by DUCC. A service is registered with DUCC using the
+ \hyperref[sec:cli.ducc-services]{ducc\_services} CLI. Service
registrations are persisted by
+ DUCC and last over DUCC and cluster restarts.
There are several variants on Registered Services:
\begin{description}
\item[Autostarted Services] An autostarted service is a registered
service that is flagged to be
automatically started when the DUCC system is started. When DUCC is
started, the SM checks the
- service registry for all service that are marked for automatic
startup. If flagged for autostart,
- the DUCC Service Manager submits the registered numuber of instances
- on behalf of the registering user. If an instance should die, DUCC
automaticlly restarts
+ service registry for all service that are marked for automatic
start-up. If flagged for autostart,
+ the DUCC Service Manager submits the registered number of instances
+ on behalf of the registering user. If an instance should die, DUCC
automatically restarts
the instance. Short of massive failures, DUCC will insure the
service is always running
- and immediately ready for use.
+ and immediately ready for use with no manual intervention.
\item[On-Demand Services] An on-demand service is a registered service
that is started only
when referenced by the service-dependency of another job or service.
If the service is
already started (e.g. by reference from some other job), the
dependent job/service is
marked ready to schedule as indicated above. If not, the service
registry is checked and
- if a matching service is found, DUCC startes it. When the service
has completed
+ if a matching service is found, DUCC starts it. When the service has
completed
initialization a pinger is started and all jobs waiting on it are
then started.
When the last job or service that references the on-demand service
exits, a timer is
established to keep the service alive for a while (in anticipation
that it will be needed
- again soon.) When the keep-alive timer exipires, and there are no
more dependent
+ again soon.) When the keep-alive timer expires, and there are no
more dependent
jobs or services, the on-demand service is automatically stopped to
free up its resources for
other work.
+ Note that if an incoming job references a service that is not
registered and which is not
+ responsive to the default pinger, that job is marked ``Services
Unavailable'' and canceled
+ by the system.
+
\item[Ping-Only Services]
\phantomsection\label{subsub:services.ping-only}
\hyperref[sec:services.implicit]{Ping-only services} consist of only
@@ -162,8 +175,8 @@
A service pinger is a small program that queries a service on behalf of
the
DUCC Service Manager to:
\begin{itemize}
- \item Report on the availiability of the service, and
- \item Report on the healh of the service.
+ \item Report on the availability of the service, and
+ \item Report on the health of the service.
\end{itemize}
Service pingers are always written in Java and must implement an
abstract class,
@@ -173,14 +186,15 @@
When a service is deployed by
DUCC, the Service Manager spawns a DUCC process that instantiates the
pinger for
the service. On a regular basis, the Service Manager sends a request to
the pinger
- to query the service health.
+ to query the service health. The pinger is expected to respond within a
reasonable
+ period of time and if it fails to do so, the service is marked
unresponsive.
\subsection{Declaring a Pinger in A Service}
If your service is a UIMA-AS service, there is no need to create or
declare a pinger as DUCC
provides a default pinger. If a CUSTOM pinger is required, it must be
declared in the service
descriptor, and the service must be registered. See
- \hyperref[sec:cli.ducc-services]{ducc\_services} for details on service
registration spcifying
+ \hyperref[sec:cli.ducc-services]{ducc\_services} for details on service
registration specifying
ping directives.
\subsection{Implementing a Pinger}
@@ -226,7 +240,7 @@ public abstract class AServicePing
\item[boolean alive] Set this to ``true'' if the service is
responsive. If a pinger responds
``false'' (or does not respond), the Service Manager will assume
the service is unavailable
and will not allow jobs dependent on this service to start.
(Dependent jobs that are already
- started are allowed to continue, but are annoated in the web
server, such that developers
+ started are allowed to continue, but are annotated in the web
server, such that developers
will know the job may not be functioning because of the service.)
\item[boolean healthy] The pinger may perform analysis on the
service to determine whether
the service is ``healthy'' or not. This is strictly subjective
and is used by the
@@ -244,7 +258,7 @@ public abstract class AServicePing
\end{description}
A sample CUSTOM pinger is shown below. The pinger assumes a simple
- service port that, on connection, returns an integer. If the connect
and read of the integer succeds,
+ service port that, on connection, returns an integer. If the connect
and read of the integer succeeds,
the ping is marked successful. This assumes the service endpoint is of
the form
\begin{verbatim}
CUSTOM:hostname:port
@@ -305,13 +319,13 @@ public class CustomPing
This section provides the information needed to use the pinger API and
build a
custom pinger.
- \paragraph{Establish compile classpath} One DUCC jar is required in the
classpath to build your pinger:
+ \paragraph{Establish compile CLASSPATH} One DUCC jar is required in the
CLASSPATH to build your pinger:
\begin{verbatim}
- uima-ducc-common.jar
+ $DUCC_HOME/lib/uima-ducc-common.jar
\end{verbatim}
This provides the definition for the {\em AServicePing} and {\em
ServiceStatistics} classes.
- \paragraph{Create a resistration}Next, create a service registration for
the pinger. While
+ \paragraph{Create a registration}Next, create a service registration for
the pinger. While
debugging, set the directive
\begin{verbatim}
service_ping_dolog = true
@@ -323,7 +337,7 @@ public class CustomPing
Once the pinger is debugged you may want to turn logging off:
\begin{verbatim}
-service_ping_dolog = false
+ service_ping_dolog = false
\end{verbatim}
A sample service registration may look something like the following:
@@ -340,8 +354,8 @@ service_ping_dolog = false
\end{verbatim}
\paragraph{Register and start the pinger} Start up your custom service
so the pinger has something to connect to, and then start
- the pinger. It may be easier to debug the pinger if you start the
service outside of DUCC. Once
- the pinger is working it is straighforward to integrate it into the
pinger's service registration.
+ the pinger. It may be easier to debug the pinger if you initially start
the service outside of DUCC. Once
+ the pinger is working it is straightforward to integrate it into the
pinger's service registration.
\begin{verbatim}
/home/ducc/ducc_runtime/bin/ducc_services --register myping.svc
/home/ducc/ducc_runtime/bin/ducc_services --start CUSTOM:localhost:7175
@@ -354,3 +368,11 @@ service_ping_dolog = false
ducc_services --stop <serviceid>
\end{verbatim}
where $<$serviceid$>$ is the id returned when you registered the pinger.
+
+ \paragraph{If all else fails ...}
+ If your pinger does not work and you cannot determine the reason, it may
be useful to check
+ the Service Manager's log for clues. The Service Manager's log is found
in
+\begin{verbatim}
+ $DUCC_HOME/logs/sm.log
+\end{verbatim}
+
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/job-details.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/job-details.tex?rev=1492381&r1=1492380&r2=1492381&view=diff
==============================================================================
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/job-details.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/job-details.tex
Wed Jun 12 19:18:00 2013
@@ -203,6 +203,7 @@
\end{description}
\subsection{Work Items}
+ \label{sec:ws-work-items}
This tab provides details for each individual work item. Columns include:
\begin{description}
@@ -233,6 +234,7 @@
\subsection{Performance}
+ \label{sec:performance}
This tab shows performance summaries of all the pipeline components. The
statistics
are aggregated over all instances of each component in each process of the
job.
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/services.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/services.tex?rev=1492381&r1=1492380&r2=1492381&view=diff
==============================================================================
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/services.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part2/webserver/services.tex
Wed Jun 12 19:18:00 2013
@@ -1,6 +1,6 @@
\section{Services Page}
-
+ \label{ws:services-page}
This page shows details of all services.
The Services page contains the following columns:
