We've had success deploying annotators on cluster nodes (using UIMA-AS deployment descriptors) registered to a UIMA-AS broker running on the head node. If the cluster use shared data folders, you only need to put the code in one place for it to 'appear' on all nodes.
Then we run a collection reader and CAS consumer on the head node, with the amount of scale-out specified on the command line of runRemoteAsyncAE.sh, something like this: $UIMA_HOME/bin/runRemoteAsyncAE.sh -c (path.to)XmiCollectionReader.xml tcp://localhost:6 1616 (name of deployed service) -p (number of nodes) -o output_foldername With enough scale-out, the limiting factor becomes the speed of the CR and CC on the head node. This is the briefest explanation I can give, not sure it's a 'best practice' but it works. :) On 4/27/2012 3:35 PM, John David Osborne wrote:
Hello, Is there any best practice documentation out there for running UIMA/UIMA-AS on a cluster? I have only run single machine instances of UIMA (mostly through Eclipse) and have not investigated the ability to perform multiple simultaneous analyses in order to process large document collections. It's not clear to me how UIMA would operate in a cluster environment, do people really do message passing using JMI? I'm guessing this is the case as I seeing references to MPICH, SGE or other things I am more used to. I've looked through some of the documentation (including all the Overview & SDK setup) but am not finding anything helpful. I've also tried googling but I am not getting much except this: http://comments.gmane.org/gmane.comp.apache.uima.general/2131 which makes me think it is possible. Currently with my level of confusion I think it may be best to have multiple instances of UIMA on a cluster and just submit jobs processing discrete document sets to our SGE cluster and ignore whatever scaling features are actually present in UIMA since the document processing I plan to do is data parallel. -John
-- Eric Riebling Senior Systems Programmer http://ericriebling.com CMU Language Technologies Institute
