> I see this is ASL,
What is ASL ???
> so that seems workable, but I'd be interested in
> hearing more about how it integrates w/ what we are doing?
A Ga works by evoluting a population of individuals toward a desired goal.
Generally, a GA needs thousands of iterations with hundreds of individuals to
find a satisfiying solution. The fitness function, that calculates the
closeness of a given individual to the desired goal, is called for each
individual of the population for each iteration. In our case, and especially
for very large dataset the fitness function can be implemented with the
Map-Reduce strategy to exploit distributed computing.
The main program, will launch the GA using WatchMaker, each time the GA needs
to calculate the fitness of a given indiviudal it calls a specific class given
by us, this class should configure a Hadoop Job and launch its execution (Iam
not sure about how to it yet, but Iam already working on that part).
My secondary goal is to implement a ready to use generic fitness function for
WatchMaker that calls internally Hadoop, those allowing us to use the GA for
different problems such as multiclass classification (I will do it if I don't
run out of time), classifcation...
The implemented fitness function is a good mesure for binary classification, it
takes a set of rules and calculates their quality over large datasets using
Hadoop for the parallelism. So it can be used outside the GA.
Abdel Hakim Deneche
_____________________________________________________________________________
Envoyez avec Yahoo! Mail. Capacité de stockage illimitée pour vos emails.
http://mail.yahoo.fr
