Hi Frank,
It's great to know your workflows run well on EC2. If you can
document and share what you have done, I think it will be definitely
valuable to other users in Kepler community.
Best wishes
Sincerely yours
Jianwu Wang
jianwu at sdsc.edu
http://users.sdsc.edu/~jianwu/
Assistant Project Scientist
Scientific Workflow Automation Technologies (SWAT) Laboratory
San Diego Supercomputer Center
University of California, San Diego
San Diego, CA, U.S.A.
On 3/23/2011 3:47 PM, Frank White wrote:
> Hello Lugman:
> I can speak to at least some of what you want to accomplish. My
> colleague and I have taken Jianwu Wang's and Jing Tao's fine work on
> the Kepler Workflow Run engine (see
> https://kepler-project.org/developers/interest-groups/distributed/technical-documentation/the-overview-of-the-workflow-run-engine-component)
>
> and built up an AMI that we regularly run on Amazon EC2. The AMI is
> built from base Amazon Fedora core 14 image, onto which we have
> installed Kepler 2.1, Tomcat6 and Axis2 - all of which are required to
> run the Kepler Web Service. This enables us to make SOAP/REST calls to
> execute Kepler .kar files that have been uploaded to a Kepler
> repository. There were a few difficulties along the way, but with
> Jing's and Jianwu's help, we were able to make it work quite reliably.
> On top of that, we have implemented several actors that call other
> Amazon services, actors that call just plain Java classes, and actors
> that call Java classes that wrap some fairly complex C++ libraries.
>
> Overall, we were able to find ample examples of how to do at least
> some of what you describe. We would be willing to share both the
> 'recipe' that we documented to build up our Kepler AMI, as well as the
> AMI itself as long as you understand that we are a small start-up, and
> have very limited resources to support it.
>
> Cheers,
>
> Frank White
> Optensity, Inc.
>
>
> Message: 1
> Date: Tue, 22 Mar 2011 15:51:17 -0700
> From: Jianwu Wang<jianwu at sdsc.edu>
> To: Luqman Hodgkinson<luqman at berkeley.edu>
> Cc:kepler-users at kepler-project.org
> Subject: Re: [kepler-users] dataflow using Kepler on Amazon EC2
> Message-ID:<4D8927E5.5030006 at sdsc.edu>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> Hi Luqman,
>
> Your target is still not clear to me. Please break it into sub
> tasks so that we can help more efficiently. Or you can try Kepler first
> before getting more specific questions to ask.
>
> About Kepler workflow execution on EC2, I did some experiments on
> it and don't think it is hard to execute Kepler workflows on EC2.
>
> Best wishes
>
> Sincerely yours
>
> Jianwu Wang
> jianwu at sdsc.edu
> http://users.sdsc.edu/~jianwu/
>
> Assistant Project Scientist
> Scientific Workflow Automation Technologies (SWAT) Laboratory
> San Diego Supercomputer Center
> University of California, San Diego
> San Diego, CA, U.S.A.
>
>
> On 3/21/2011 5:10 PM, Luqman Hodgkinson wrote:
>
>> >
>> >
>> >
>> > Dear Kepler developers,
>> > I have a collection of Java classes linked by a custom dataflow
>> architecture. All classes are in a single project but some of these
>> classes call executables written in languages other than Java. I am
>> investigating the possibility of transitioning to Kepler. Essentially
>> my desires are to link these Java classes in a DAG representing the
>> dataflow and to execute the dataflow in Amazon EC2. The data flowing
>> along the edges are arbitrary custom Java classes. Additionally it is
>> important to cache intermediate results. The data is acquired from a
>> few web services: iRefIndex, IntAct, UniProt, and Gene Ontology.
>> There are complex software dependencies so after setting up the
>> dataflow I would like to save the entire system as an abstract
>> machine image (AMI). How difficult would this transition be, and
>> would it be worth the effort? I would appreciate your comments and
>> advice.
>> > Sincerely, with best wishes,
>> > Luqman Hodgkinson,
>> > Ph.D. student, UC-Berkeley
>> > _______________________________________________
>> > Kepler-users mailing list
>> > Kepler-users at kepler-project.org
>> > http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
> ------------------------------
>
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at kepler-project.org
> http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
>
>
> End of Kepler-users Digest, Vol 70, Issue 10
> ********************************************
>
