Hello,
On Mon, Jan 12, 2009 at 2:57 PM, A. Jorge Garcia <[email protected]> wrote:
> OK, I suppose noone is using dSage then?
>
> Does anyone have a current email address for Yi Qiang? I used the one
> on his dSage website, but he's not replying so I'm thinking that's not
> his current email address....
DSage is primarily designed for handling embarrassingly parallel
algorithms where there doesn't need to be a whole lot of communication
between the nodes. It doesn't use MPI or anything like that which
works at a much lower level.
For most nontrivial applications, you'll want to subclass
DistributedFunction. A simple example would be the following:
class DistributedFunctionTest(DistributedFunction):
"""
This is a very simple DistributedFunction.
Only for educational purposes.
"""
def __init__(self, dsage, n, name='DistributedFunctionTest'):
DistributedFunction.__init__(self, dsage)
self.n = n
self.name = name
self.result = 0
self.results = []
self.code = """DSAGE_RESULT=%s"""
self.outstanding_jobs = [Job(code=self.code % i, username='yqiang')
for i in range(1, n+1)]
self.start()
def process_result(self, job):
self.done = len(self.waiting_jobs) == 0
self.result += (job.result)
This distributed function will compute the sum of the first n numbers.
This is a bit of a silly example as each of the workers doesn't do
really any work, but it illustrates how these things work.
The idea is that you create a bunch of Job objects which contain the
code you want to run on the workers (the machines that actually
perform the computation.) When the job is finished, the
process_result() method is called with the corresponding job object as
input. Anything stored in the DSAGE_RESULT variable in the Job's code
will be accessible as job.result. It is the responsibility of
process_result() to set the done flag.
Here is an example of how this is used:
First, we get our DSage object.
sage: d = dsage.start_all()
This is a shortcut for starting the DSage server, starting two
workers, and making a client connection to the server which will be
stored in the object d. The object d is how one typically interacts
with a DSage server.
Next, we create an instance of our DistributedFunctionTest:
sage: test = sage.dsage.dist_functions.all.DistributedFunctionTest(d, 5)
We check to see if it is done.
sage: test.done
False
sage: len(test.waiting_jobs)
5
After waiting a bit, we see
sage: len(test.waiting_jobs)
2
Finally, you'll see something like
sage: test.done
True
Once test.done is True, then you can get the result which we have
stored in test.result:
sage: test.result
15
If one were doing this on a cluster, you'd start workers on each of
the machines in the cluster and then have each of those workers
connect to a single server.
If you have a more specific question about using DSage, I'd be happy
to answer it.
--Mike
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sage-edu" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/sage-edu?hl=en
-~----------~----~----~----~------~----~------~--~---
