On Mar 26, 2008, at 7:38 AM, deneche abdelhakim wrote:
I see this is ASL,
What is ASL ???
Sorry, Apache Software License. It means we can package the library,
if needed, with our code. Gotta keep the lawyers happy!
so that seems workable, but I'd be interested in
hearing more about how it integrates w/ what we are doing?
A Ga works by evoluting a population of individuals toward a desired
goal. Generally, a GA needs thousands of iterations with hundreds of
individuals to find a satisfiying solution. The fitness function,
that calculates the closeness of a given individual to the desired
goal, is called for each individual of the population for each
iteration. In our case, and especially for very large dataset the
fitness function can be implemented with the Map-Reduce strategy to
exploit distributed computing.
The main program, will launch the GA using WatchMaker, each time the
GA needs to calculate the fitness of a given indiviudal it calls a
specific class given by us, this class should configure a Hadoop Job
and launch its execution (Iam not sure about how to it yet, but Iam
already working on that part).
My secondary goal is to implement a ready to use generic fitness
function for WatchMaker that calls internally Hadoop, those allowing
us to use the GA for different problems such as multiclass
classification (I will do it if I don't run out of time),
classifcation...
The implemented fitness function is a good mesure for binary
classification, it takes a set of rules and calculates their quality
over large datasets using Hadoop for the parallelism. So it can be
used outside the GA.
Sounds good. I look forward to the proposal.
-Grant
