Hi Noor,
>>one big question, is that possible with Jess?
Short Answer: I think so.
DISCLAIMER:: All of the following is just my random ideas of how to proceed
with your problem. I am flying totally blind here not knowing the
particulars of what you are currently implementing or the tools you are
using. The examples that I've made up are totally to motivate our
discussion -- I make no claim as to their absolute correctness.
I think the key to this (and anyone out there who knows better please help
me if I'm wrong) is whether or not you have source code access to the Broker
implementation such that you can modify it. In the literature that I have
seen, there are often "high level" and "low level" filters or front
controller components to the Broker. This will contain the logic to choose
which resource (node) to send the job (request). If *you* as you say are
going to actually write the Broker, then there should be no problem!
Also I think the Host daemons that report on resource status to the Broker
will have to either transmit their status as Jess facts or have the Broker
convert them to facts once the update arrives at the Broker.
We can model the Jess end of it to give you some ideas.
Let the actual Broker be the component with Jess deciding which resource
should receive the job request. Then, let the working memory of the
Broker/Jess controller contain Jess facts that represent all of the grid
resources available to the client making the job request. Here are some
prototype templates to give you an idea:
(detemplate resource
"Models a grid computing resource"
(slot resource-id (type STRING))
(slot is-available (type SYMBOL))
(slot URL (type STRING))
(slot ram-available (type LONG))
(multislot CPUs))
(deftemplate cpu
"Models a resource CPU"
(slot cpu-id (type STRING))
(slot resource-id (type STRING))
(slot clock-speed (type FLOAT))
(slot percent-allocated)(type INT))
We also need some type of fact to represent the actual job request so that
Jess can make the "mapping" of the request parameters to the available
resource parameters. Again, I'm just guessing at this, but perhaps you can
wrap the actual request object in a Jess fact like so...
(deftemplate job
"Wraps an incoming job request as a Jess fact"
(slot id (type STRING))
(slot job-OBJECT (type OBJECT))
(slot submit-time (type LONG))
(slot maximum-process-time)(type INT))
Now, you need some way of computing the estimated time to complete the job
based on the job parameters and the available resources subject to the
constraint the job be completed in no more than maximum-process-time.
Again, what the algorithm or method would be that you would use I can only
guess. Some ideas would be:
(1) If the job is to run a particular program on some data, then you could
try to compute the estimated time based on the cyclomatic complexity of the
program. There are examples in the literature of this.
(2) You could model the estimated time, T, to complete a job, J, as a linear
(or non linear) function of the resource parameters in your grid.
For example lets say that you knew that all job requests were to process n
amount of records. Let r = RAM, and c = CPU speed and T = estimated time.
Then, given a table of benchmarks on your resources ...
+---------------------+
| Resource R |
+----------+-----------+---------+----------+
| n | r (GB) | c (Ghz)| T (sec) |
+----------+-----------+---------+----------+
| 50000 | 1.0 | 1.66 | 2.33 |
+----------+-----------+---------+----------+
| 150000 | 2.0 | 2.33 | 1.07 |
+----------+-----------+---------+----------+
| ... | ... | ... | ... |
+----------+-----------+---------+----------+
... you could develop a regression equation :
T = x1*r + x2*c + x3*n where the coefficients x1, x2, and x3 are to be
determined.
Then for any job of size n you would be able to estimate the completion time
for resource R. Just write a method in the Broker that computes this for
you. Perhaps you might be able to analyze existing server logs to get this
benchmark data.
(3) Create some algorithm of your choice that computes the estimated time
using another technique.
As for the controller rules, using the previous regression example, a
made-up example might look like:
;; A Jess function to compute our estimated completion time
(deffunction compute-estimated-time (?r ?c ?n)
;; I'll make up some coefficients for the regression...
(return (+ (* 0.4 ?r) (* 1.2 ?c) (* 0.55 ?n))))
Now we have a rule that uses this function to do the allocation...
(defrule allocate-resource-to-job
; If there is a job...
?job <- (job (id ?id)(job-OBJECT ?jobOBJ)
(submit-time ?st)(maximum-process-time ?mt))
; ...and there is an available CPU of
; certain clock speed and availability...
?cpu <- (cpu (cpu-id ?cpuid)(resource-id ?rd)
(clock-speed ?cs)(percent-allocated ?pa)
; ...and there is a resource that hosts
; this CPU with available RAM
?resource <- (resource (resource-id ?rd)
(is-available ?isAvailble&TRUE)
(URL ?url)(ram-available ?ra&(>= ?ra 1.5))
(CPUs $? ?cpu ?$))
; ...then get the "size" of the job
; and compute if the job can be completed by
; this resource in the allotted time.
(test
(<= (compute-estimated-time ?ra ?cs (?jobOBJ getSize)) ?mt)
=>
;; code to modify resource availability
;; here if resource is maxed-out.
;; If so, then allocate the job to this CPU on this host
(allocate-job ?jobOBJ)
(retract ?job))
I have probably fantasized too much at this point, so I'll stop and let you
tell me how far off I am.
Cheers,
Jason
On Jan 10, 2008 8:25 AM, Mohd. Noor <[EMAIL PROTECTED]> wrote:
Hi Jason and All
You are absolutely right!
I think, I will be using Gridbus broker from Rajkumar Buyya( Aus) to write
the broker. It is there already.
What I am going to focus is on the matching between numerous resources and
client, thus I think I can made it using a rule base. The working memory
could be the current state of the resource (supply by hosts) and the client
will provide the rule. The things is both attribute let say from resource
will be number of CPUs and from user could be a time of expected results.
This is impossible to be mapped between time and number of cpu since its not
identical. Thus I have to transform one rule; let say from working memory to
the one that common such, as time- both client and hosts being map via
"time" now.
I thinks they have one term in expert system that related with this type of
implementation - it is transforming
How it is related to me. Is that RHS rule has to be translated?
one big question, is that possible with Jess?
Cheers
mnoor
-----------------------------------------------------------
Jason Morris
Morris Technical Solutions LLC
http://www.morris-technical-solutions.com
