On Mon, May 12, 2008 at 8:42 PM, Bart Smaalders <[EMAIL PROTECTED]> wrote:
>
> Shawn Walker wrote:
>
> > On Mon, May 12, 2008 at 7:08 PM, Jordan Brown (Sun)
> > <[EMAIL PROTECTED]> wrote:
> >
> > > Would this process involve listening on a network or local port? In
> other
> > > words, is the depot server started a general purpose server, or is it in
> > > some fashion tied tightly to the requester (like talking via a pipe
> instead
> > > of a TCP connection)?
> > >
> >
> > For my prototype hack, it would have to be a tcp connection since
> > that's how the depot server communicates.
> >
> >
> > > Opening up a listening socket, even a local one, is security-impolite.
> > >
> >
> > This is really just a prototype, and yes, I agree it is security-impolite.
> >
> >
> > > It also sounds kind of inefficient, moving all those bytes through an
> > > "extra" process and some kind of IPC.
> > >
> >
> > Yes, but it does allow re-use of all of the existing depot code.
> >
> > I was discussing the "no on-disk package format" with Stephen and Bart
> > at Community One.
> >
> > I'm just trying to get a better feel for what some of the ideas have been.
> >
> >
>
> One of the approaches I've been musing about is to abstract the depot
> <> client communications more completely.... the goals would be as
> follows:
>
> 0) facilitate mirroring by allowing catalogs, manifests and files to come
> from different sources. By signing the catalogs and manifests, content
> can come from anywhere.
>
> 1) make downloaded files usable between pkg invocations; when your wireless
> craps out 450 MB into a large update, it's kind of frustrating that you
> have
> to start from the beginning. Also, installing multiple zones means a
> local
> cache is a good idea...
>
> 2) enable the use of alternate transports, such as multicast, local mirror,
> BitTorrent, DVDs, USB sticks, etc.
As part of the cherrypy depot work I've been doing, it has occurred to
me that one depot server, with many repositories is an interesting
option. As a result, I've been wanting to create depot and repository
objects and have our existing pkg.depotd just instantiate one of them.
We've already done this partially with depotcontroller.py in my
opinion.
The "napkin sketch in my head" looks something like this:
pkg.depot
pkg.depot.http (http://, https://)
pkg.depot.ftp (ftp://)
pkg.depot.smb (smb://))
pkg.depot.ssh (ssh://)
pkg.repository
pkg.repository.http
pkg.repository.file
pkg.repository.smb
pkg.repository.ssh
class depot(object):
def __init__(uri):
# uri is how the depot should be accessible (e.g. http://depot:10000/)
self.repos = {}
def add_repository(self, uri, mountpoint):
# use mountpoint as name and check for duplicate
# if no duplicate found then:
# parse uri to determine type of pkg.repo object, then
self.repos[mountpoint] = pkg.repository.type(uri)
def start():
for repo in self.repos.values():
repo.start() # Allow repository to initialise, prepare itself, etc.
def stop():
for repo in self.repos.values():
repo.stop() # Allow repository to cleanup, etc.
...
class http(depot):
def search(self, mountpoint, args):
repo = self.repos[mountpoint]
results = repo.search(args)
# return results to client
...
def versions(self, mountpoint, args):
...
def catalog(self, mountpoint, args):
...
def filelist(self, mountpoint, args):
...
def manifest(self, mountpoint, args):
...
def add(self, mountpoint, args):
...
def file(self, mountpoint, args):
...
def abandon(self, mountpoint, args):
...
def close(self, mountpoint, args):
...
def open(self, mountpoint, args):
...
...
class repository(object):
def __init__(uri):
# uri is where to get the data from (e.g. http://depot:10000/depot1/)
...
class http(repository):
def start():
# do initialisation
def stop():
# do cleanup
def search(args):
...
def versions(args):
...
def catalog(args):
...
def filelist(args):
...
def manifest(args):
...
def add(args):
...
def file(args):
...
def abandon(args):
...
def close(args):
...
def open(args):
...
Basic idea is that the depot server would:
depot = pkg.depot.httpd('http://localhost:10001/')
depot.add_repository('file:///var/depot1/', 'depot1')
depot.start()
The client would:
# parse uri to determine repository type then
repo = pkg.repository.type(uri)
repo.start()
repo.operation(args)
repo.stop()
Beg pardon if you find crumbles on my "napkin", I'm pretty tired now :-)
> 3) improve the asynchronous nature of pkg install by queuing download/
> uncompression and verification of received content.
Definitely. I think using the above "napkin sketch" we could make that
a reality.
--
Shawn Walker
"To err is human -- and to blame it on a computer is even more so." -
Robert Orben
_______________________________________________
pkg-discuss mailing list
pkg-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
