Inderpreet Chopra wrote:
Thanks Martin.
Please see my comments inline.
On Wed, Jan 14, 2009 at 2:49 AM, Martin Feller <[email protected]
<mailto:[email protected]>> wrote:
Inderpreet Chopra wrote:
Hi all
I am working on the fault-tolerance and the security aspects of
Grid. I want to work on these two aspects in globus .
For this I want to know about the internal structure of Globus
and its components.
Do you mean Globus or the job management component in Globus?
I am consider about job management component only. Thanks for providing
the Execution management links, that clears many of my doubts.
Is Globus is using some Async or sync queues for taking request
for executing the jobs?
I'm not entirely sure what you mean by that. A client does not hold
a connection
open until a job finished processing completely, only until a job
resource has
been created on the server-side, which is used so that a client can
refer
to his job. Once the job resource has been created the processing starts
on the server-side, without client-interaction.
What here i want to ask is that, client requests the GRAM for the job
execution. If there are few clients, then GRAM will handle the requests
from all the clients. But what will happen if thousands of client try to
approach the grid system for job execution.
So i want to ask is that, is GRAM maintains some queues in which it
pushes the requests if large number of them comes at one time.
There is a queue (not in ws-gram, but in the underlying Java WS Core),
that takes incoming requests for all services. Server threads pick requests
from the queue and the desired service is called, and finally the response
is sent back to the client.
If the rate of incoming requests is higher than the rate of requests being
processed, the queue fills up, and at a certain point there will be
connection timeouts. My guess is that most servers work that way and that
the problem of 'Denial of Service' is not yet solved.
What if some node fails in between means is there some way to
recover the pending task being stuck in the erronous node?
What do you mean by "some node"? A client, the globus-server, or a
machine of
a cluster which was picked by the local resource manager (like PBS,
Condor) to
execute the job?
Here by node i means:
- GRAM itself fails after taking the request from the client. Then as
far as i read, the grid will be dead.
I'd rather say this GRAM instance is no longer available in the grid,
and not the grid is dead.
-The scheduling node fails, but it assigns jobs before going into the
dead state. So my question is that, Is GRAM communicate with the
execution nodes ( nodes that are actually assigned the execution of job)
directly or through the scheduler to get the current status of the job
state. If it communicate through the scheduler, then there is problem.
GRAM gets information about the jobs via the local resource manager (scheduler),
it does not communicate with the execution nodes directly. More precisely, a
program (the Scheduler Event Generator (SEG)) scans the logfile of the local
resource manager and forwards the information about jobs to GRAM
(GRAM4/ws-gram).
For GRAM2/Pre-WS-GRAM there is also periodical polling using the job status
query commands of the local resource manager.
Why is communication via the local resource manager necessarily a problem?
Martin
Can anyone please guide me in getting answer to my questions and
also some documents describing globus internal architecture.
The following webpage gives an overview over key concepts gram (job
management):
http://www.globus.org/toolkit/docs/4.2/4.2.1/execution/key/#executionKey
A starting point for more information about gram is this:
http://www.globus.org/toolkit/docs/4.2/4.2.1/execution/
Martin
Hey i am all new with Globus, so might be all these will be stupid
questions for you, but i want to clear all doubts before actually
starting working. Some leftovers that are still making me awake are:
- If according to Job Lifetime limit, the job does not finish, GRAM will
cancel the job. Is there any means with which we can resubmit the job
automatically? I guess we need to write some custom GRAM like component.
- What is actual way of processing single job. I guess it should be
distributed to different nodes rather than, the complete job executed on
single node. So if that is case, how GRAM manages the response from
different nodes and combine to reply to client?
Also any suggestions related to my area of intrest i.e fault
tolerance and security in globus. What is still pending that i
can take and work upon.
Regards,
Inderpreet
Inderpreet
Research Scholar
TU, Patiala
