I see the a couple of differences in the approach that we are each using. Let me ask some clarifing questions so I can better answer your questions.
- In Alchemi's case the controller of an AppDomain is the Manager. What happens if a Manager goes down or there is a network interuption of some sort? Is the Executor in its appdomain aware enough that it knows that it should reclaim its resources and unload itself due to whatever failure? The way in which I handle this is the controller actually resides in the AppDomain so you can equate that to the Executor. If the Manager keeps pushing work to the executor then the Executors lease lifetime would not time out and it would not unload.
- In Alchemi, and correct me if I'm wrong here, the Executor can be viewed as the controller of work units, whatever those may be, as you alluded to above. The executor can execute many different types of work units with in a specific app domain. So in effect you have an appdomain per manager work unit on each node. This is a bit different than what I am doing. I 'group' like work units, i.e. reports, into one appdomain and service any requests for those work units from that appdomain. A 'group' in Alchemi's case is a particular instance of a Manager enlisting a particular node to do work on its behalf. Is this a correct assumption? The approach I am using would still work in this instance. The executor would have a lease lifetime associated with them and would unload themselves *after* that lease expires. This would allievate the Manager from having to control the lifetime of the appdomain on the co-operating nodes.
- Caching of work dlls. Currently Alchemi pushes the required dlls down to a slave system whenever it instigates work on a node. Those same dlls are purged from the system once work is complete and the manager no longer needs the node to do work? If this is the case have you thought about some sort of caching scheme so that the managers and nodes would not have re-push the dlls if they have executed this type of work before?
- Security. Right now Alchemi is pretty wide open when it comes to security. This is actually a very large issue with a lot of things to take into consideration. This would be easier to put a box around if we could say exactly what each executor would be doing and how it would be getting its data to operate on but each executor could do a multitude of things. Maybe if we say from an executor what a work unit would need would could lock out other things. We could then, from a cooperating node, say that we only want to allow access to particular resources and even give the opportunity to trust particular managers fully. I don't know, I am just throwing out ideas here.
Thanks,
John
On 2/23/06, Krishna <[EMAIL PROTECTED]> wrote:
Hi John,
It sounds interesting, and looks like parts of it can be used in
Alchemi's management of appdomains for running user's applications, and
of course...we would definitely want you to contribute ideas / code /
anything you can, towards Alchemi :)
Currently we have a simple way of unloading an application domain: it is
done when the manager informs the executor that the "GApplication" is
complete : meaning that no further jobs would be sent to the Executor as
part of THAT GApplication. Of course, for another GApplication, a new
app domain would be loaded.
So do you think this model would be too restrictive / insufficient? Is
lease-time a better way of managing app domains on the Executor? If so,
how would the manager determine the lease time for an app domain: please
bear in mind, that a user's GApplication can actually have completely
independent tasks i.e GThreads or GJobs, which could even do completely
different things, reference different dlls, run for variying lengths of
time, all of which will not be known by the manager beforehand.
Ideas anyone?
Cheers
Krishna...
John Sheppard wrote:
> Krishna,
> That is good to hear. I have a fairly robust 'plugin' framework
> that does something similiar to what Alchemi is doing. I actually
> manage lease-lifetime on the appdomains via the marshal-by-ref
> controller that gets loaded into each app domain that then loads up
> the plugin(s)/work units. The appdomain has fairly restricted access
> to the system. This allows for a 'plugin' or 'package' to hang around
> until its lease times out. If another work package comes in during
> the time that the lease is alive it will go ahead a service the
> request. If the lease expires it will unload the app domain there by
> reclaiming resources. The way in which I am currently using it is for
> reporting using Active Reports. It was not designed to be remotable
> but we worked around that issue by encapsulating the report controller
> as a remoting service that controls the plugins/reports by loading
> them in their own app domain and just passing back serialized active
> report content that is then loaded up by whatever client called it.
> The manager also handles the caching of the plugins in their own
> folder and monitors the original non-cached location for updates so
> that if a report is replaced it will stop using the old version and
> let it lease expire while at the same time caching and loading the new
> version which allows us to add and update plugins on the fly without
> having to restart the system. Would some similiar mechanism be of
> interest to Alchemi? If so I would be glad to contribute the
> knowledge that I have gained in my current endeavors.
>
> John
>
> On 2/21/06, *Krishna* < [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
> wrote:
>
> Hi John,
>
> Alchemi already does exactly that!
> Run each Grid application in a seperate Application domain (on the
> executor) created dynamically at runtime;
> It has always done that, right from the beginning (It is not just
> .Net2.0, even .Net 1.1 gives you good control over AppDomains.). This
> way we already do have a simple sandbox on the executor, however
> there is a need to enforce proper permissions on the sandbox.
> Currently
> the appdomain created on each Executor gives the app full permissions.
> This is actually where improvements need to be made.
>
> Cheers
> Krishna.
>
> Jonathan Mitchem wrote:
>
> >John,
> >
> >Do you have any references, books or web, that discuss .NET 2.0's
> >ability to use app domains in that manner?
> >
> >Jonathan
> >
> >On 2/21/06, John Sheppard < [EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
> >
> >
> >>Krishna,
> >> Another approach you might want to look into is to run each
> grid app in
> >>it's own appdomain and limit the abilities of that app
> domain. You can lock
> >>it down to a specific directory if the executor service is
> dropping data
> >>files to given directories for a grid app. You could lock out
> app domains
> >>from using the System.Net namespaces there by limiting phone
> home abilities
> >>and you can make the executor more resilent by allowing the
> offending app
> >>domain to die upon an unhandled exception rather than bringing
> down the
> >>executor. You could also take advantage of caching grid apps on
> the client
> >>machine where you wouldn't have to push or sip that app if you
> already have
> >>the dlls on your system. .NET 2.0 does a very good job of
> allowing you set
> >>up Sandboxing environs using app domains.
> >>
> >>John
> >>
> >>
> >>On 2/21/06, Krishna < [EMAIL PROTECTED]
> <mailto: [EMAIL PROTECTED]>> wrote:
> >>
> >>
> >>>Hi Jonathan,
> >>>
> >>>I guess a simple way to prevent "grid-viruses", would be to use
> the .NEt
> >>>CAS (Code access security)
> >>>feature. We will need to implement some code in Alchemi to run
> user code
> >>>under reduced priveleges inside a sand-box kind of environment on
> >>>an Executor.
> >>>
> >>>Cheers
> >>>Krishna.
> >>>
> >>>Jonathan Mitchem wrote:
> >>>
> >>>
> >>>
> >>>>I've been thinking about security recently, and started
> questioning
> >>>>the security of a distributed system such as Alchemi.
> >>>>
> >>>>Is there anything that actually "constrains" the grid
> environment on a
> >>>>machine so that a user doesn't allow some sort of distributed
> malware
> >>>>to damage their machine?
> >>>>
> >>>>For instance, an application that reads the files on the machine
> >>>>hosting the Executor, searches for certain files or filetypes
> (like,
> >>>>password and private key files), and then sends them to a
> specified
> >>>>address. And maybe even proceeds to break their encryption.
> >>>>
> >>>>Or, an application that creates several threads so that every
> machine
> >>>>has a copy of the required DLLs, which subsequently proceeds
> to remove
> >>>>critical system files from every machine.
> >>>>
> >>>>Is there anything to prevent such sort of usage? And if not
> (since
> >>>>I'm presuming there isn't), how would we go about preventing such
> >>>>damage?
> >>>>
> >>>>
> >>>>Jonathan
> >>>>
> >>>>
> >>>>-------------------------------------------------------
> >>>>This SF.net email is sponsored by: Splunk Inc. Do you grep
> through log
> >>>>
> >>>>
> >>files
> >>
> >>
> >>>>for problems? Stop! Download the new AJAX search engine that
> makes
> >>>>searching your log files as easy as surfing
> the web. DOWNLOAD SPLUNK!
> >>>>
> >>>>
> >>> http://sel.as-us.falkag.net/sel?cmd=k&kid3432&bid#0486&dat1642
> <http://sel.as-us.falkag.net/sel?cmd=k&kid%103432&bid#0486&dat%121642 >
> >>>
> >>>
> >>>>_______________________________________________
> >>>>Alchemi-developers mailing list
> >>>> [email protected]
> <mailto:[email protected] >
> >>>>
> >>>>
> >>>>
> >>https://lists.sourceforge.net/lists/listinfo/alchemi-developers
> >>
> >>
> >>>>
> >>>>
> >>>>
> >>>
> >>>-------------------------------------------------------
> >>>This SF.net email is sponsored by: Splunk Inc. Do you grep
> through log
> >>>
> >>>
> >>files
> >>
> >>
> >>>for problems? Stop! Download the new AJAX search engine that
> makes
> >>>searching your log files as easy as surfing the web. DOWNLOAD
> SPLUNK!
> >>>
> >>>
> >>>
> >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
> < http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642>
> >>
> >>
> >>>_______________________________________________
> >>>Alchemi-developers mailing list
> >>>[email protected]
> <mailto: [email protected] >
> >>>
> >>>
> >>>
> >> https://lists.sourceforge.net/lists/listinfo/alchemi-developers
> >>
> >>
> >>
> >>--
> >>Life should NOT be a journey to the grave with the intention of
> arriving
> >>safely in an attractive and well preserved body, but rather to
> skid in
> >>sideways, paddle in one hand, beer in the other, body thoroughly
> used up,
> >>totally worn out and screaming "WOO HOO what a ride!"
> >>
> >>
> >
> >
> >-------------------------------------------------------
> >This SF.net email is sponsored by: Splunk Inc. Do you grep
> through log files
> >for problems? Stop! Download the new AJAX search engine that makes
> >searching your log files as easy as surfing the web. DOWNLOAD
> SPLUNK!
> > http://sel.as-us.falkag.net/sel?cmd=k&kid3432&bid#0486&dat1642
> < http://sel.as-us.falkag.net/sel?cmd=k&kid%103432&bid#0486&dat%121642 >
> >_______________________________________________
> >Alchemi-developers mailing list
> > [email protected]
> <mailto:[email protected]>
> > https://lists.sourceforge.net/lists/listinfo/alchemi-developers
> >
> >
> >
> alche
>
>
>
>
> --
> Life should NOT be a journey to the grave with the intention of
> arriving safely in an attractive and well preserved body, but rather
> to skid in sideways, paddle in one hand, beer in the other, body
> thoroughly used up, totally worn out and screaming "WOO HOO what a ride!"
--
Life should NOT be a journey to the grave with the intention of arriving safely in an attractive and well preserved body, but rather to skid in sideways, paddle in one hand, beer in the other, body thoroughly used up, totally worn out and screaming "WOO HOO what a ride!"
