Re: problem if use remoteAnalysisEngine in deployment script

2013-07-17 Thread Eddie Epstein
On Tue, Jul 16, 2013 at 5:23 AM, reshu.agarwal wrote: > Hi, > > I tried deploying the services with the other example which uses the > flowcontrollerand the services starts but with these errors: > > Deployment xml: > > > > > >brokerURL="tcp://localhost:**61616" prefetch="0"/

Re: problem if use remoteAnalysisEngine in deployment script

2013-07-18 Thread Eddie Epstein
On Thu, Jul 18, 2013 at 12:22 AM, reshu.agarwal wrote: > On 07/17/2013 07:30 PM, Eddie Epstein wrote: > >> Component Descriptor Editor >> > Hi, > > It is not a flow controller its an aggregate descriptor which is using the > flow controller I just put it into flow

Re: UIMA AS Asynchronous = true not behaving as expected.

2013-08-07 Thread Eddie Epstein
What client program are you using to drive the uima-as service? Anything using sendAndReceiveCAS will only send one at a time. Please use the runRemoteAsyncAE.sh program and specify the number of outstanding CASes with -p. With no specification the number is 2. Eddie On Wed, Aug 7, 2013 at 12:3

Re: UIMA AS Asynchronous = true not behaving as expected.

2013-08-07 Thread Eddie Epstein
Scaling via async=false and the number of instances = 6 DOES speed the > pipeline up by 6x, so I think the client is working correctly. It seems to > just be an issue when the async=true. I checked in the JMX console with > async=true and slow annotator = 6 instances and inside JMX UIMA reports >

Re: XmiCasSerializer error in UIMA-AS

2013-10-18 Thread Eddie Epstein
Is this a solid error that is easily reproduced? The error is occurring when UIMA-AS is returning the CAS from the service. You could add XMI serialization to file at the end of AE processing, for the good and failing cases. If so lucky to have that serialization fail too, could try inserting the

Re: "Run as AS aggregate" and pre-fetching

2013-10-21 Thread Eddie Epstein
nt descriptor > "Run as AS aggregate"? > > I found an email from a year ago online where Eddie Epstein wrote: > "UIMA-AS will put every asynchronous component in a separate thread.Using > the ComponentDescriptorEditor on a UIMA-AS deployment > descriptor, marking an

Re: XmiCasSerializer error in UIMA-AS

2013-10-22 Thread Eddie Epstein
a rules for modifying the annotations in my CAS. In > both cases, serialization to file succeeded, with the second file > containing the expected results of rule application. > > Unfortunately, the UIMA-AS serialization still produces the error in my > original mail. > > Pr

Re: XmiCasSerializer error in UIMA-AS

2013-10-28 Thread Eddie Epstein
10:21 AM, Eddie Epstein wrote: > Can you share the before and after XmiCas files for a case where UIMA-AS > service serialization failed? No confidential info please! > > Eddie > > > > On Tue, Oct 22, 2013 at 2:27 AM, Prokopis Prokopidis wrote: > >> No, this is

Re: XmiCasSerializer error in UIMA-AS

2013-10-29 Thread Eddie Epstein
annotations. Eddie On Tue, Oct 29, 2013 at 9:39 AM, Peter Klügl wrote: > On 28.10.2013 13:47, Eddie Epstein wrote: > > Assuming you would rather not share the CAS files, what I'd be looking > for > > is the xmi:id values changing between the before and after files. There >

Re: XmiCasSerializer error in UIMA-AS

2013-10-30 Thread Eddie Epstein
t; previously existing annotations. > > Are there any known modifications that cause problems? > > Peter > > > Eddie > > > > > > > > > > > > On Tue, Oct 29, 2013 at 9:39 AM, Peter Klügl >wrote: > > > >> On 28.10.2013 13

Re: DUCC not leaving Initializing State

2013-11-14 Thread Eddie Epstein
On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis wrote: > I have modeled a CasReader and CasMultiplier based on the RawTextExample > in the duccbook, and have successfully ran the CR and CM in eclipse with a > minimal wrapper. Did you debug the job using one of the --all_in_one varieties, or some

Re: DUCC not leaving Initializing State

2013-11-14 Thread Eddie Epstein
ng Listener on Endpoint: > queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002 Broker: > tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false > 03:53:45.290 - 1: > > org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:

Re: DUCC Work Items and Send To Last

2013-11-17 Thread Eddie Epstein
On Sun, Nov 17, 2013 at 1:03 PM, Neal R Lewis wrote: > Let's assume a pipeline like that in the Sample that contains a CR as the > Job Driver, and the JP contains a CM, AE, and CC. > > If I understand correctly, the CR's getNext() returns a single Workitem > CAS - a CAS tagged with the Workitem

Re: uimaFIT: managing component configurations

2013-11-28 Thread Eddie Epstein
Just in case you haven't considered it, version 2.4.2 has a new option for overriding configuration parameters outside of component descriptions, called external overrides. It is implemented via an additional settings file. Component descriptors remain static, like code, which would work well for P

Re: big offsets efficiency, and multiple offsets

2013-12-05 Thread Eddie Epstein
On 05/12/13 10:04, Jens Grivolla wrote: > I agree that it might make more sense to model our needs more directly >> instead of trying to squeeze it into the schema we normally use for text >> processing. But at the same time I would of course like to avoid having >> to reimplement many of the thi

Re: uima-as 2.3.1 - java.io.IOException: Frame size of 147 MB larger than max allowed 100 MB

2014-01-27 Thread Eddie Epstein
On Thu, Jan 23, 2014 at 9:28 AM, Thomas Ginter wrote: > It is likely then that your expansion is happening after the remote > service is called or else is not yet big enough to be over the 100MB limit. > Also note that by default UIMA-AS [Java] services use a delta-CAS interface. Only changes to

Re: Installing pears on DUCC

2014-02-11 Thread Eddie Epstein
The Pear installer is part of the standard UIMA SDK, not currently included in DUCC. Definitely something that should be clarified in DUCC. Thanks, Eddie On Tue, Feb 11, 2014 at 10:51 AM, Bai Shen wrote: > I've successfully set up DUCC in single user mode and run the example job > through it.

Re: Installing pears on DUCC

2014-02-11 Thread Eddie Epstein
You should not need UIMA-AS SDK installed. Eddie On Tue, Feb 11, 2014 at 12:14 PM, Bai Shen wrote: > So I need to install UIMA SDK in addition to DUCC? What about UIMA-AS? > > > On Tue, Feb 11, 2014 at 11:21 AM, Eddie Epstein > wrote: > > > The Pear installer is p

Re: Installing pears on DUCC

2014-02-11 Thread Eddie Epstein
pears and then deployed them. Then I > was able to send a CAS to the queue and have it processed. > > I'm still trying to understand how all of the pieces interact and what all > changes DUCC brings. > > Thanks. > > > On Tue, Feb 11, 2014 at 2:53 PM, Eddie Epste

Re: Installing pears on DUCC

2014-02-11 Thread Eddie Epstein
more machines and don't see how to use DUCC, please describe here. On Tue, Feb 11, 2014 at 3:28 PM, Bai Shen wrote: > Okay, I'll go ahead and redo my setup using UIMA-AS 2.4.2. > > How do I get DUCC to control my UIMA-AS setup? > > > On Tue, Feb 11, 2014 at 3:2

Re: Error deploying pear on AS 2.4.2

2014-02-12 Thread Eddie Epstein
A pear is a packed UIMA analysis engine, or AE. UIMA-AS deploys services that contain AEs. The command deployAsyncService requires a UIMA-AS Deployment Descriptor. On Wed, Feb 12, 2014 at 10:52 AM, Bai Shen wrote: > I'm running the following command to deploy my pear to UIMA-AS 2.4.2. > > deplo

Re: Error deploying pear on AS 2.4.2

2014-02-13 Thread Eddie Epstein
r, > but I can't find any mention about deploying it. > > Is there a specific section that I should be looking at? > > Thanks. > > > On Wed, Feb 12, 2014 at 3:09 PM, Eddie Epstein > wrote: > > > A pear is a packed UIMA analysis engine, or AE. UIMA-AS dep

Re: dlclose Assertion fails for C++-Analysis Engine

2014-02-18 Thread Eddie Epstein
UIMA-C++ uses several libraries including APR, ICU and xercesc. Does OpenCV drag in any of these at different version levels? Eddie On Tue, Feb 18, 2014 at 6:34 AM, Peter Retzlaff wrote: > Hello, > > we are working on a UIMA project with several C++-Analysis Engines > (UIMA-AS 2.4 + UIMACPP 2.4

Re: dlclose Assertion fails for C++-Analysis Engine

2014-02-18 Thread Eddie Epstein
e same libs. As far as I can tell, this is not the case. > > Peter > > On 02/18/2014 02:22 PM, Eddie Epstein wrote: > > UIMA-C++ uses several libraries including APR, ICU and xercesc. Does > OpenCV > > drag in any of these at different version levels? > > &g

Re: Accessing uima as pipeline from a REST interface

2014-02-26 Thread Eddie Epstein
Hi Mihaela, The UimaAsynchronousEngine is designed to support multiple threads accessing a service. The engine API object has a CAS pool size parameter and initial CAS size parameters to support this. >From the RunRemoteAsyncAE.java sample code: // Add the Cas Pool Size and initial FS heap si

Re: status Lost=1 in DUCC

2014-03-20 Thread Eddie Epstein
The job driver running the user's CR should not sent documents, only references to documents, or preferably references to a set of documents. Please see http://uima.apache.org/d/uima-ducc-1.0.0/duccbook.html#x1-1290008.2 Eddie On Thu, Mar 20, 2014 at 7:14 AM, reshu.agarwal wrote: > > Hi, > > I

Re: status Lost=1 in DUCC

2014-03-21 Thread Eddie Epstein
Hi Reshu, Attachments are not delivered to this mailing list. Given that your application CR is following the guidelines, please answer Lou's questions. Eddie On Fri, Mar 21, 2014 at 12:13 AM, reshu.agarwal wrote: > On 03/21/2014 01:39 AM, Eddie Epstein wrote: > >>

Re: Ducc Problems

2014-03-21 Thread Eddie Epstein
A short discussion about Cas Consumers is at http://uima.apache.org/d/uima-ducc-1.0.0/duccbook.html#x1-138.3 Individual Work Items may be treated as collections. The Work Item CAS can be used to trigger collection process complete for these collections. See http://uima.apache.org/d/uima-ducc-1

Re: status Lost=1 in DUCC

2014-03-21 Thread Eddie Epstein
ly. It could be that the DUCC job driver is miss-handling this type of exception. Thanks, Eddie On Fri, Mar 21, 2014 at 7:59 AM, reshu.agarwal wrote: > On 03/21/2014 05:06 PM, Eddie Epstein wrote: > >> Hi Reshu, >> >> Attachments are not delivered to this mailing lis

Re: Ducc Problems

2014-03-26 Thread Eddie Epstein
Are you using standard UIMA interface code to Solr? If so, which Cas Consumer? Taking at quick look at the source code for SolrCASConsumer, the batch and collection process complete methods appear to do nothing. Thanks, Eddie On Wed, Mar 26, 2014 at 6:08 AM, reshu.agarwal wrote: > On 03/21/201

Re: Ducc Problems

2014-03-26 Thread Eddie Epstein
the paragraph "Flushing Cached Data". Eddie On Wed, Mar 26, 2014 at 9:48 AM, reshu.agarwal wrote: > On 03/26/2014 06:43 PM, Eddie Epstein wrote: > >> Are you using standard UIMA interface code to Solr? If so, which Cas >> Consumer? >> >> Tak

Re: Problems with Annotators using a non-default View

2014-03-26 Thread Eddie Epstein
Would be nice to see the big picture, how the JMS services are integrated into the full pipeline. Sounds like you have two, separate, "remote" JMS annotators, which are being called from an aggregate on a "local" client. And guessing that there is sofamapping on the aggregate, for the two remote de

Re: Problems with Annotators using a non-default View

2014-03-27 Thread Eddie Epstein
This sounds like what I described, C=aggregate, A and B are remote delegates. Anyway, unless sofamapping is used, B needs to use getView("SomeView").

Re: Ducc Problems

2014-03-27 Thread Eddie Epstein
commit) in the process method after every 50 documents. Regards Eddie On Thu, Mar 27, 2014 at 1:35 AM, reshu.agarwal wrote: > On 03/26/2014 11:34 PM, Eddie Epstein wrote: > >> Hi Reshu, >> >> The collectionProcessingComplete() method in UIMA-AS has a limitation: a >>

Re: Ducc Problems

2014-03-28 Thread Eddie Epstein
which are doing incremental commits. The solution is to schedule the job in a non-preemptable class. On Fri, Mar 28, 2014 at 1:22 AM, reshu.agarwal wrote: > On 03/28/2014 01:28 AM, Eddie Epstein wrote: > >> Hi Reshu, >> >> The Job model in DUCC is for the Collection Reader to send

Re: Cas Timeout Exception in DUCC

2014-03-28 Thread Eddie Epstein
There is a job specification parameter: --process_per_item_time_max Maximum elapsed time (in minutes) for processing one CAS. Try setting that to something big enough. Eddie On Fri, Mar 28, 2014 at 6:18 AM, reshu.agarwal wrote: > On 03/28/2014 03:23 PM, reshu.agarwal wrote: > >>

Re: Cas Timeout Exception in DUCC

2014-03-31 Thread Eddie Epstein
Reshu, Please look in the logfile of the job process. Maybe 10 minutes is still not enough? Eddie On Mon, Mar 31, 2014 at 2:42 AM, reshu.agarwal wrote: > On 03/28/2014 05:36 PM, Eddie Epstein wrote: > >> There is a job specification parameter: >> --process

Re: problem in calling DUCC Service with ducc_submit

2014-04-01 Thread Eddie Epstein
Declaring a service dependency does not affect application code paths. The job still needs to connect to the service in the normal way. DUCC uses services dependency for several reasons: to automatically start services when needed by a job; to not give resources to a job or service for which a dep

Re: What if head node fails in DUCC

2014-04-01 Thread Eddie Epstein
Correct. Most DUCC daemons running on the head node are restartable. We expect to complete this work so that in the case of head node failure a new head node can automatically be started. Currently DUCC can be configured such that no active user work is affected if a head node goes down. However,

Re: What if head node fails in DUCC

2014-04-01 Thread Eddie Epstein
on the head node are unaffected. Eddie On Tue, Apr 1, 2014 at 8:26 AM, reshu.agarwal wrote: > On 04/01/2014 05:28 PM, Eddie Epstein wrote: > >> Correct. Most DUCC daemons running on the head node are restartable. We >> expect to complete this work so that in the case of h

Re: problem in calling DUCC Service with ducc_submit

2014-04-04 Thread Eddie Epstein
the AE in a DUCC job would have such a delegate. Eddie On Thu, Apr 3, 2014 at 1:00 AM, reshu.agarwal wrote: > On 04/01/2014 05:21 PM, Eddie Epstein wrote: > >> The >> job still needs to connect to the service in the normal way. >> >> DUCC uses services depe

Re: Cas Timeout Exception in DUCC

2014-04-04 Thread Eddie Epstein
Reshu, Can you zip up the logfile directory for one of these failing jobs and make it available for download? Eddie On Thu, Apr 3, 2014 at 1:24 AM, reshu.agarwal wrote: > On 03/31/2014 04:37 PM, Eddie Epstein wrote: > >> Reshu, >> >> Please look in the logfile of t

Re: Complex architectures with multiple CASes - how to?

2014-04-09 Thread Eddie Epstein
One approach is for the cas multiplier to put the question View in each search CAS, using the CasCopier, and create a second view for search result processing. Down stream annotators would then be multi-view, getting question analysis results from one view and doing search analysis in the other. E

Re: Sofa-unaware AEs that create new views in an AAE

2014-04-17 Thread Eddie Epstein
Hi Peter, The logic is that since a sofa aware component may have one or more input and/or output views, such a component needs to use getView to specify which to use. For sofa aware delegates, sofa mapping enables the delegate to hard wire input and/or output View names in annotator code (or ann

Re: Sofa-unaware AEs that create new views in an AAE

2014-04-18 Thread Eddie Epstein
On Thu, Apr 17, 2014 at 9:17 AM, Peter Klügl wrote: > Am 17.04.2014 15:01, schrieb Eddie Epstein: > > Hi Peter, > > > > The logic is that since a sofa aware component may have one or > > more input and/or output views, such a component needs to use > > getView to

Re: Sofa-unaware AEs that create new views in an AAE

2014-04-22 Thread Eddie Epstein
method. Is this a better model for you? Eddie On Tue, Apr 22, 2014 at 6:47 AM, Peter Klügl wrote: > Am 18.04.2014 15:23, schrieb Eddie Epstein: > > On Thu, Apr 17, 2014 at 9:17 AM, Peter Klügl >wrote: > > > >> Am 17.04.2014 15:01, schrieb Eddie Epstein: > >>

Re: Sofa-unaware AEs that create new views in an AAE

2014-04-22 Thread Eddie Epstein
I think so too. Have created https://issues.apache.org/jira/browse/UIMA-3765to cover this work. Eddie On Tue, Apr 22, 2014 at 12:26 PM, Peter Klügl wrote: > Am 22.04.2014 18:20, schrieb Peter Klügl: > > Am 22.04.2014 18:10, schrieb Eddie Epstein: > >> The current design

Re: Parallel Flow Controller?

2014-05-16 Thread Eddie Epstein
On Thu, May 15, 2014 at 8:51 AM, Petr Baudis wrote: > So my FixedParallelFlowController is a major simplification of that, > the tradeoff is that the AEs run in parallel must be grouped in their > own aggregate AE (again, very little overhead of that - though I'm > not sure how it could affect e.

Re: Is there a way to tell UIMA component to only extract some kind of entities when run opennlp.pear?

2014-06-11 Thread Eddie Epstein
Hi Jeffery, According the info at http://uima.apache.org/d/uimaj-2.6.0/tutorials_and_users_guides.html#ugr.tug.aae.result_specification_setting "The default Result Specification is taken from the Engine's output Capability Specification." So it should be possible to deploy the UIMA-AS service

Re: Building UIMA-CPP on (K)Ubuntu 14.04

2014-07-22 Thread Eddie Epstein
Looking at a build on RHEL, jni.h was resolved with: --with-jdk="/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/include -I/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/include/linux" which follows the instructions in README.4src I had built ICU [and all other dependencies] and installed it in a p

Re: Building UIMA-CPP on (K)Ubuntu 14.04

2014-07-22 Thread Eddie Epstein
regards, > > Egbert > > > Op dinsdag 22 juli 2014 09:30:41 schreef Eddie Epstein: > > Looking at a build on RHEL, jni.h was resolved with: > > --with-jdk="/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/include > > -I/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_

Re: Passing additional parameters through to CPE components

2014-07-24 Thread Eddie Epstein
A CPE descriptor can override configuration parameters defined in any integrated components. Documentation a little bit below http://uima.apache.org/d/uimaj-2.6.0/references.html#ugr.ref.xml.cpe_descriptor.descriptor.cas_processors.individual 3.6.1.2. Element This element provides a way to overri

Re: Passing additional parameters through to CPE components

2014-07-24 Thread Eddie Epstein
rily known at design time. However, > UIMA isn’t passing it through to UimaContext presumably because it isn’t > defined in the MyCollectionReader.xml. > > Hope that helps clear it up. > > > On 24 Jul 2014, at 14:51, Eddie Epstein wrote: > > > A CPE descriptor can override con

Re: UIMA AS NullPointerException in CasDefinition constructor

2014-07-28 Thread Eddie Epstein
Hi Egbert, The README file for UIMA-AS shows an application example with Deploy_MeetingDetectorTAE.xml.Does that run OK for you? Assuming yes, can you give more details about the scenario, perhaps the explicit commands used? The descriptors used? Eddie On Mon, Jul 28, 2014 at 11:46 AM, Egbert

Re: Building UIMA-CPP on (K)Ubuntu 14.04

2014-07-29 Thread Eddie Epstein
full > / relative path in the annotator descriptors instead of just a library > name. Where should > I report these feature requests? > > Thanks again! > > Egbert > > > On Tuesday, July 22, 2014 05:04:36 PM Eddie Epstein wrote: > > Good to hear the build worked.

Re: UIMA AS NullPointerException in CasDefinition constructor

2014-08-04 Thread Eddie Epstein
m Working Examples (or actually, not-working > > examples) that I've constructed to pinpoint the problem. > > > > Commenting out line 62 (the call to setCollectionReader) 'fixes' the > > problem. However, as you can see, the implementation o

Re: Uima AS out of memory

2014-08-20 Thread Eddie Epstein
When using deployAsyncService.sh to start a UIMA AS service, the default Java heap size is Xmx800M. To override this, export an environment parameter UIMA_JVM_OPTS with JVM arguments. For example: $ export UIMA_JVM_OPTS="-Xmx6G -Xms2G" $ deployAsyncService.sh myDeploymentDescriptor.xml On

Re: Could UIMA AS client send custom key value parameters to annotator?

2014-10-01 Thread Eddie Epstein
There is no mechanism for a uima-as client to modify the result specification of a remote service. Since type/feature control cannot indicate many other behavioral characteristics, like speed vs accuracy tradeoffs, the suggested approach for dynamic control is to use dedicated feature structures in

Re: UIMA DUCC - Multi-machine Installation

2014-10-30 Thread Eddie Epstein
The $DUCC_RUNTIME tree needs to be on a shared filesystem accessible from all machines. For single user mode ducc_ling could be referenced from there as well. But for multiuser setup, ducc_ling needs setuid and should be installed on the root drive. Eddie On Thu, Oct 30, 2014 at 10:08 AM, James B

Re: UIMA DUCC - Multi-machine Installation

2014-10-31 Thread Eddie Epstein
> have successfully setup and ran my UIMA analysis engine on single user > mode. I also followed DUCCBOOK to setup ducc_ling but I am sure how to get > it worked on a cluster of machines. > > Thanks, > Tam > > On Thu, Oct 30, 2014 at 11:08 PM, Eddie Epstein > wrote:

Re: UIMA DUCC - Multi-machine Installation

2014-11-05 Thread Eddie Epstein
t; Regards, > Tam > > On Fri, Oct 31, 2014 at 9:37 PM, Eddie Epstein > wrote: > > > Hi Tam, > > > > In the install documentation, > > http://uima.apache.org/d/uima-ducc-1.0.0/installation.html, > > the section "Multi-User Installation and Verific

Re: UIMA DUCC - Multi-machine Installation

2014-11-06 Thread Eddie Epstein
ean 500 work items per machine? > > Regards, > Tam > > On Thu, Nov 6, 2014 at 1:20 AM, Eddie Epstein wrote: > > > Hi, > > > > There is a default limit of 500 work items dispatched at the same time. > How > > many dispatched are shown for the job? > >

Re: DUCC stuck at WaitingForResources on an Amazon Linux

2014-11-13 Thread Eddie Epstein
Simon, The DUCC resource manager logs into rm.log. Did you look there for reasons the resources are not being allocated? Eddie On Wed, Nov 12, 2014 at 4:07 PM, Simon Hafner wrote: > 4 shares total, 2 in use. > > 2014-11-12 5:06 GMT-06:00 Lou DeGenaro : > > Try looking at your DUCC's web server

Re: DUCC stuck at WaitingForResources on an Amazon Linux

2014-11-15 Thread Eddie Epstein
On Fri, Nov 14, 2014 at 8:11 PM, Simon Hafner wrote: > So to run effectively, I would need more memory, because the job wants > two shares? ... Yes. With a larger node it works. What would be a > reasonable memory size for a ducc node? > > Really depends on the application code. Quoting from the

Re: DUCC doesn't use all available machines

2014-11-17 Thread Eddie Epstein
DuccRawTextSpec.job specifies that each job process (JP) run 8 analytic pipeline threads. So for this job with 100 work items, no more than 13 JPs would ever be started. After successful initialization of the first JP, DUCC begins scaling up the number of JPs using doubling. During JP scale up the

Re: DUCC-Un-managed Reservation??

2014-11-18 Thread Eddie Epstein
On Tue, Nov 18, 2014 at 1:05 AM, reshu.agarwal wrote: > > Hi, > > I am bit confused. Why we need un-managed reservation? Suppose we give 5GB > Memory size to this reservation. Can this RAM be consumed by any process if > required? > Basically yes. See more info about "Rogue Process" in the duccb

Re: DUCC web server interfacing

2014-11-20 Thread Eddie Epstein
The preferred approach is to run the analytics as a DUCC service, and have an application driver that feeds the service instances with incoming data. This service would be a scalable UIMA-AS service, which could have as many instances as are needed to keep up with the load. The driver would use the

Re: DUCC web server interfacing

2014-11-20 Thread Eddie Epstein
Ooops, in this case the web server would be feeding the service directly. On Thu, Nov 20, 2014 at 9:04 PM, Eddie Epstein wrote: > The preferred approach is to run the analytics as a DUCC service, and have > an application driver that feeds the service instances with incoming data.

Re: DUCC web server interfacing

2014-11-21 Thread Eddie Epstein
On Thu, Nov 20, 2014 at 10:01 PM, D. Heinze wrote: > Eddie... thanks. Yes, that sounds like I would not have the advantage of > DUCC managing the UIMA pipeline. > Depends on the definition of "managing". DUCC manages the lifecycle of analytic pipelines running as job processes and as services.

Re: DUCC org.apache.uima.util.InvalidXMLException and no logs

2014-11-27 Thread Eddie Epstein
Those are the only two log files? Should be a ducc.log (probably with no more info than on the console), and either one or both of the job driver logfiles: jd.out.log and jobid-JD-jdnode-jdpid.log. If for some reason the job driver failed to start, check the job driver agent log (the agent managing

Re: Ducc: Rename failed

2014-11-28 Thread Eddie Epstein
DuccCasCC component has presumably created /home/ducc/analysis/txt.processed/5911.txt_0_processed.zip_temp and written to it? Did you run this sample job in something other than cluster mode? On Fri, Nov 28, 2014 at 10:23 AM, Simon Hafner wrote: > When running DUCC in cluster mode, I get "Re

Re: Ducc: Rename failed

2014-11-28 Thread Eddie Epstein
-28 10:45 GMT-06:00 Eddie Epstein : > > DuccCasCC component has presumably created > > /home/ducc/analysis/txt.processed/5911.txt_0_processed.zip_temp and > written > > to it? > I don't know, the _temp file doesn't exist anymore. > > > Did you run this s

Re: DUCC doesn't use all available machines

2014-11-28 Thread Eddie Epstein
on - the BaseCap - > > so a max of 16 will be scheduled for it, subject to fair-share > constraint. > > > > 17 Nov 2014 15:07:38,880 INFO RM.RmJob - */getPrjCap/* 208927 bobuser > O 2 > > T 343171 NTh 128 TI 143171 TR 6748.601431980907 R 1.8967e-02 QR 5043 P > 6509 >

Re: DUCC doesn't use all available machines

2014-11-30 Thread Eddie Epstein
On Sat, Nov 29, 2014 at 4:46 PM, Simon Hafner wrote: > I've thrown some numbers at it (doubling each) and it's running at > comfortable 125 procs. However, at about 6.1k of 6.5k items, the procs > drop down to 30. > 125 processes at 8 threads each = 1000 active pipelines. How CPU cores are these

Re: DUCC doesn't use all available machines

2014-11-30 Thread Eddie Epstein
On Sun, Nov 30, 2014 at 11:48 AM, Simon Hafner wrote: > 2014-11-30 7:25 GMT-06:00 Eddie Epstein : > > On Sat, Nov 29, 2014 at 4:46 PM, Simon Hafner > wrote: > > > >> I've thrown some numbers at it (doubling each) and it's running at > >> comfort

Re: Serializing Specific View to XMI

2014-12-04 Thread Eddie Epstein
I think that is not supported directly. One could use the CasCopier to copy the view(s) of interest to a new, empty CAS and serialize to xmi file from that. Eddie On Wed, Dec 3, 2014 at 9:04 AM, Jakob Sahlström wrote: > Hi, > > I'm dealing with a CAS with multiple views, namely a Gold View and

Re: DUCC- Agent1 is on Physical and Agent2 is on virtual=Slow the job process timing

2014-12-19 Thread Eddie Epstein
Hi Reshu, On Fri, Dec 19, 2014 at 12:26 AM, reshu.agarwal wrote: > > Hi, > > Is there any problem if one Agent node is on Physical(Master) and one > agent node is on virtual? > > I am running a job which is having avg processing timing of 20 min when I > have configured a single machine DUCC (ph

Re: Ruta parallel execution

2014-12-19 Thread Eddie Epstein
Hi Silvestre, An aggregate deployed with UIMA-AS can be used to run delegate annotators in parallel, with a few restrictions. - the aggregate must be deployed as async=true - the parallel delegates must each be running in remote processes - the delegates must not modify preexisting FS As Jens

Re: Ducc Problems

2015-03-03 Thread Eddie Epstein
troy() >>>> methods >>>> in UIMA-AS are not called. >>>> There should be some evidence in JP logs at the very end. Look for >>>> something like this: >>>> >>>> Process Received a Message. Is Process target for message:tr

Re: UIMA CPE appears not to utilise more than a single thread

2015-04-13 Thread Eddie Epstein
The CPE runs pipeline threads in parallel, not necessarily CAS processors. In a CPE descriptor, generally all non-CasConsumer components make up the pipeline. Change the following line to indicate how many pipeline threads to run, and make sure the casPoolSize is number of threads +2. Eddie On

Re: Error handling in flow control

2015-04-24 Thread Eddie Epstein
Can you give more details on the overall pipeline deployment? The initial description mentions a CPE and it mentions services. The CPE was created before flow controllers or CasMutipliers existed and has no support of them. Services could be Vinci services for the CPE or UIMA-AS services or ??? On

Re: Error handling in flow control

2015-04-26 Thread Eddie Epstein
e me > consider implementing a custom multithreaded collection processor but I > wanted to avoid this. > > Hope this clarifies what I am trying to do. Cheers :) > > > On 24 Apr 2015, at 16:50 , Eddie Epstein wrote: > > > > Can you give more details on the overall pipel

Re: UIMA-AS and ActiveMQ ports

2015-04-27 Thread Eddie Epstein
UIMA-AS has example deployment descriptors using placeholders for the broker: ${defaultBrokerURL} If these placeholders are used and the user doesn't specify a value for the Java property "defaultBrokerURL" then some code in UIMA-AS will use a default value of tcp://localhost:61616. That is the onl

Re: DUCC- process_dd

2015-04-30 Thread Eddie Epstein
The simplest way of vertically scaling a Job process is to specify the analysis pipeline using core UIMA descriptors and then using --process_thread_count to specify how many copies of the pipeline to deploy, each in a different thread. No use of UIMA-AS at all. Please check out the "Raw Text Proce

Re: DUCC- process_dd

2015-05-01 Thread Eddie Epstein
process_dd. But > How?? > > Thanks in advanced. > > Reshu. > > > On 05/01/2015 03:28 AM, Eddie Epstein wrote: > >> The simplest way of vertically scaling a Job process is to specify the >> analysis pipeline using core UIMA descriptors and then using >> --pro

Re: Multi-threaded UIMA ParallelStep

2015-05-20 Thread Eddie Epstein
Parallel-step currently only works with remote delegates. The other approach, using CasMultipliers, allows an arbitrarily amount of parallel processing in-process. A CM would create a separate CAS for each delegate intended to run in parallel, and use a feature structure to hold a unique identifier

Re: Multi-threaded UIMA ParallelStep

2015-05-20 Thread Eddie Epstein
eline instances in separate threads? UIMA-AS would do this by specifying N instances of a synchronous top-level aggregate. Eddie On Wed, May 20, 2015 at 8:49 AM, Petr Baudis wrote: > Hi! > > On Wed, May 20, 2015 at 07:56:33AM -0400, Eddie Epstein wrote: > > Parallel-step curre

Re: UIMAj3 ideas

2015-07-10 Thread Eddie Epstein
Hi Petr, Good comments which will likely generate lots of responses. For now please see comments on scaleout below. On Thu, Jul 9, 2015 at 6:52 PM, Petr Baudis wrote: > * UIMAfit is not part of core UIMA and UIMA-AS is not part of core > UIMA. It seems to me that UIMA-AS is doing things

Re: DUCC multi-node installation. Beginner's questions.

2015-07-22 Thread Eddie Epstein
Hi Sergii, The ducc_runtime tree needs to be installed on a shared filesystem that all DUCC nodes have mounted in the same location. Just install the ducc runtime once from the DUCC head node. All other DUCC nodes simply need to have the mounted filesystem and common user accounts with identical u

Re: DUCC multi-node installation. Beginner's questions.

2015-07-23 Thread Eddie Epstein
l the > other > nodes. > Maybe it will save someone like me a couple of hours. > > Thanks again and cheers, > Sergii > > On Wed, Jul 22, 2015 at 3:02 PM, Eddie Epstein > wrote: > > > Hi Sergii, > > > > The ducc_runtime tree needs to be installed on a

Re: Error when trying to drop CAS with FlowController

2015-09-06 Thread Eddie Epstein
Hi Torsten, The documentation says ... public FinalStep(boolean aForceCasToBeDropped) Creates a new FinalStep, and may indicate that a CAS should be dropped. This can only be used for CASes that are produced internally to the aggregate. It is an error to attempt to drop a CAS that was p

Re: Error when trying to drop CAS with FlowController

2015-09-06 Thread Eddie Epstein
not be dropped? > > Cheers, > > -- Richard > > On 06.09.2015, at 15:58, Eddie Epstein wrote: > > > Hi Torsten, > > > > The documentation says ... > > > > public FinalStep(boolean aForceCasToBeDropped) > > > > Creates a new FinalStep, and may

Re: CAS merger/multiplier N:M mapping

2015-09-06 Thread Eddie Epstein
Hi Petr On Sun, Sep 6, 2015 at 10:11 AM, Petr Baudis wrote: > Hi! > > I'm currently struggling to perform a complex flow transformation with > UIMA. I have multiple (N) CASes with some fulltext search results. > I chop these search results to sentences and would like to pick the top > M sen

Re: Error when trying to drop CAS with FlowController

2015-09-06 Thread Eddie Epstein
returned by processAndOutputNewCASes does not contain the input CAS? > > Cheers, > > -- Richard > > On 06.09.2015, at 16:21, Eddie Epstein wrote: > > > Hi Richard, > > > > FinalStep() in a CasMultiplier aggregate means to stop further flow > > in

Re: Error when trying to drop CAS with FlowController

2015-09-06 Thread Eddie Epstein
e do explicitly not want certain CASes to continue the processing path. > > -- Richard > > On 06.09.2015, at 17:04, Eddie Epstein wrote: > > > Richard, > > > > In general the input CAS must continue down some processing path. > > Where is it stored and what trigg

Re: Error when trying to drop CAS with FlowController

2015-09-06 Thread Eddie Epstein
;t get why. > > Cheers, > > -- Richard > > On 06.09.2015, at 17:14, Eddie Epstein wrote: > > > How about the filter adds a FeatureStructure indicating that the CAS > should > > be dropped. > > Then when the INNER_AAE returns the CAS, the flow controller in the >

Re: Error when trying to drop CAS with FlowController

2015-09-07 Thread Eddie Epstein
ion with a special FeatureStructure, but > this has the disadvantage that the consumer needs to be aware of that. > It would be easier if some CASes could simply be dropped. > I guess this could even be useful for flat workflows. > > -Torsten > > > Am 06/09/15 17:31 schrie

Re: Error when trying to drop CAS with FlowController

2015-09-07 Thread Eddie Epstein
Can the filter in the INNER_AAE modify such CASes, perhaps by deleting data, that would result in the existing consumer effectively ignoring them? On Mon, Sep 7, 2015 at 11:08 AM, Zesch, Torsten wrote: > >The consumer does not have to be modified if the flow controller > >drops CASes marked to b

Re: CAS merger/multiplier N:M mapping

2015-09-07 Thread Eddie Epstein
Petr, > > > (I'm somewhat tempted to cut my losses short (much too late) and > > > abandon UIMA flow control altogether, using only simple pipelines and > > > having custom glue code to connect these together, as it seems like > > > getting the flow to work in interesting cases is a huge time s

Re: Error when trying to drop CAS with FlowController

2015-09-07 Thread Eddie Epstein
the document text, but > as far as I know the document text cannot be changed once it is set. > > Am 07/09/15 17:14 schrieb "Eddie Epstein" unter : > > >Can the filter in the INNER_AAE modify such CASes, perhaps > >by deleting data, that would result in the existing

  1   2   3   >