Hi John:
5.) Krishna, if I am reading between the lines correctly it would almost
seem that you are talking about some sort of P2P overlay topology for
the grid. If this correct I think that it is a fantastic idea. I have
Yes, developing a P2P overlay like topology to connect different nodes
of Achemi is one of our top priorities. It will be great have your
contribution.
Thanks
Raj
been involved in a couple of P2P apps and I would be glad to lend a hand
with implementation for Alchemi. This would allow for clustering of
resources. It would also allow for pushing through firewalls and all
manner of network nastiness that can happen. But, like I alluded to
above, I'll be busy until probably mid-May getting settled in with my
new employer. After that I would be happy to contribute to the project.
Have a great day,
John
On 2/24/06, *Krishna* <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
wrote:
Hi John,
What Tibor said is basically correct. I have added my comments inline as
well.
Krishna.
>
> 1. In Alchemi's case the controller of an AppDomain is the
Manager.
> What happens if a Manager goes down or there is a network
> interuption of some sort? Is the Executor in its appdomain
aware
> enough that it knows that it should reclaim its resources and
> unload itself due to whatever failure? The way in which I
handle
> this is the controller actually resides in the AppDomain so you
> can equate that to the Executor. If the Manager keeps pushing
> work to the executor then the Executors lease lifetime
would not
> time out and it would not unload.
>
> */[Tibor Biro] In Alchemi the AppDomain is created on the Executor.
> One AppDomain is created per application. The AppDomain is kept alive
> until the Manager tells the Executor that the application
finished. If
> multiple threads are running at the same time on the same
Executor for
> the same application then the same AppDomain is used. Currently
we are
> having problems if the thread inside the sandbox hangs for whatever
> reason. If the Executor just dies then the Manager re-schedules the
> thread to another Executor but if the thread hangs then nothing
happens./*
>
[Krishna: ] This is correct. I tried sometime back to get some logging
out of the secondary appdomains, but havenot been able to do so. This
was a while back, and I havent been able to concentrate on Alchemi for
any significant period of time in the past 2 months, due to my focus
diverted towards our broker software. (http://www.gridbus.org/broker)
(http://sourceforge.net/projects/gridbusbroker ).
> 2. In Alchemi, and correct me if I'm wrong here, the Executor can
> be viewed as the controller of work units, whatever those may
> be, as you alluded to above. The executor can execute many
> different types of work units with in a specific app domain. So
> in effect you have an appdomain per manager work unit on each
> node. This is a bit different than what I am doing. I 'group'
> like work units, i.e. reports, into one appdomain and service
> any requests for those work units from that appdomain. A
'group'
> in Alchemi's case is a particular instance of a Manager
> enlisting a particular node to do work on its behalf. Is this a
> correct assumption? The approach I am using would still work in
> this instance. The executor would have a lease lifetime
> associated with them and would unload themselves *after* that
> lease expires. This would allievate the Manager from having to
> control the lifetime of the appdomain on the co-operating
nodes.
>
> */[Tibor Biro] This behavior was just changed so I'll describe the
> version in CVS. Each Executor creates an AppDomain for each
> application, identified by the AppID. If a work unit comes from the
> same app ID and there is an existing AppDomain for it then that
one is
> used instead of creating a new one. So all work units from the same
> app ID are grouped in the same AppDomain. By default the Executor
only
> accepts one work unit at a time but it can be configured to accept
> multiple work units (this is the feature I am working on now) in
which
> case it is possible to have multiple work units running at once
in the
> same AppDomain./*
>
> */ I think it would be useful to have the Executor's AppDomain "time
> out" somehow. Currently the Manager has to terminate the AppDomain so
> if the Manager fails to do that the AppDomain is never unloaded.
> Another thing that sometimes happens is that an AppDomain cannot be
> unloaded, probably because of the hanging threads./*
>
[Krishna :] Tibor, I am not sure what behaviour has changed (perhaps
you
are talking about multiple threads on an executor). The behaviour Tibor
has outlined, (with the exception of multiple threads), is what was
intended from the beginning. So, an AppDomain is expected to exist (or
is created otherwise) for each GApplication. All the GThreads that are
part of the same GApplication run in the same Appdomain on the Executor.
The Manager ofcourse doesnot directly control AppDomains on the
Executor. It tells the Executor when the GApp is finished, so
appropriate clean up can be performed.
> 3. Caching of work dlls. Currently Alchemi pushes the required
dlls
> down to a slave system whenever it instigates work on a node.
> Those same dlls are purged from the system once work is
complete
> and the manager no longer needs the node to do work? If this is
> the case have you thought about some sort of caching scheme so
> that the managers and nodes would not have re-push the dlls if
> they have executed this type of work before?
>
> */[Tibor Biro] Caching the DLLs would be useful. We'll need the dlls
> to be signed though so we can properly identify them./*
>
[Krishna: ] This may be an improper way of putting it but: there is
some
kind of caching going on now. I mean, the dll files which are part of
the Application manifest, get copied to the Executor only once. So, in
that sense, all Gthreads which use those dlls would be able to do so,
without copying the set of dlls specified in the manifest multiple
times. So this can be considered as some caching, if you like, but if
additional capabilities are needed, we should surely look into providing
them.
> */ /*
>
> 4. Security. Right now Alchemi is pretty wide open when it
comes to
> security. This is actually a very large issue with a lot of
> things to take into consideration. This would be easier to
put a
> box around if we could say exactly what each executor would be
> doing and how it would be getting its data to operate on but
> each executor could do a multitude of things. Maybe if we say
> from an executor what a work unit would need would could lock
> out other things. We could then, from a cooperating node, say
> that we only want to allow access to particular resources and
> even give the opportunity to trust particular managers fully. I
> don't know, I am just throwing out ideas here.
>
> */[Tibor Biro] I agree that security is currently an issue. I see
this
> controlled mainly from the Executor at this point. In the future a
> centralized control would be nice to have. In the end what I would
> like to see is a way to specify the rights based on several
parameters
> such as where the dll is coming from, who the user is and whether the
> dll is signed or not. I am thinking about something like the .NET
Code
> Access Security Policies, maybe the built-in CAS securities can
be used./*
>
> Okay, that should be enough questions to help me get my mind wrapped
> around it a little tighter and see how I might contribute what I have
> or modify what I have to help out.
>
[Krishna: ] Yes, I agree. I think .Net CAS would be the way to go. In
fact, the appDomain creation code in the Executor was created with the
intention that it would be used in combination with CAS. There are even
some code comments in there to that effect (I think). Of course, this
needs to be expanded, so that additional policies, for example per user
/ per manager / per dll / per machine could be added / managed, on the
Executor, and perhaps even the Manager. I would like to have the
Executor bit done first though, so things are bascially designed from
the ground-up for a de-centralised security system, and then extended so
it can be used in a centralised way (via the Manager), if needed for
convenience. In effect, the Executor security settings should override
those set on the Manager (or rather the tighter security of the two
should be applied). This will also fall in line nicely with the concept
of autonomous resources being part of the Grid, so that if the Executor
and Manager are owned / operated by different individuals /
organisations the owner/administrator will still have full control of
his/her part of the "Grid".
I also just wanted to remind everyone that Alchemi was (and in my
opinion should continue to be) developed with the idea of scaling to a
large-ish Grid system with hetergenous, autonomous, and perhaps even
geo-graphically distributed resources, which can also work in a
cluster-like environment. I agree that the .Net remoting architecture,
lends itself more to a LAN environment, than a WAN-like one, however,
the idea is to make it so that it is not restricted to cluster / LAN
environment.
> Thanks,
>
> John
>
--
Life should NOT be a journey to the grave with the intention of arriving
safely in an attractive and well preserved body, but rather to skid in
sideways, paddle in one hand, beer in the other, body thoroughly used
up, totally worn out and screaming "WOO HOO what a ride!"
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
alchemi-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/alchemi-users
