On Fri, 1 Aug 2003, Ka-Ping Yee wrote:

> On Fri, 1 Aug 2003, zachary rosen wrote:
> > A quick note - the decentralized system that is being proposed is NOT peer
> > to peer.  At the top, at the aggregator, it functions just the same as the
> > centralized solution: One database, searchable and acessable by all - ie
> > napster.
> >
> > The difference is how the metada gets to the central DB.  Either it is
> > aggregated from nodes, or required to be centrally submitted to bypass the
> > technical hurdles of aggregation.
>
> I believe we are all agreed up to this point.

Woo! :)

> > I am under the impression that feasability or technical-hard-ness of
> > building the metadata collection functionality to be decentralized rather
> > than requiring it to be centrally submitted does not nearly outweigh the
> > problems with admining, maintaining, and hosting the central solution.
>
> We have to maintain the central host anyway, because we've already
> agreed that we need a main search site.  The question is whether we
> ought to introduce *additional* administrative tasks for people
> running other DeanSpace sites in the media network.

I completely disagree.  I don't see how having node admins help manage
their local repositories increases the amount (*additional*)
administrative work.  We want to have all nodes have media hosting
capabilities (I believe).  This means there will have to be admin work
done to set up a media hosting module no matter what.  The only real
difference between centralized and decentralized in terms of admin work
required then becomes the mundane maintenance tasks: pruning and
organization.  If the nodes are empowered to maintain their own local
repository then doing this work is offloaded from the potentially very
large bottlneck of having it all maintained / pruned in a central site by
one set of admins (DMT) to the many capable node admins.  Am I missing
something?


> > When i get home i will be rifling through some books for quotes to support
> > my claim that Reed's Law and End-to-End principals support the
> > decentraralized design over the centralized design.
>
> Since we seem to have different perspectives on what these terms mean,
> i'd rather talk directly about the reasoning, as in "This design is
> better for the following reasons..." rather than "This design is better
> because there's a principle that says so."

Of course ;p

> Now, i'm not against fault-tolerance.  Yes, of course it would be good
> to have a resilient system where some of the hosts can fail and things
> keep working.  We can certainly talk about ways to do that after the
> data has been centrally submitted (e.g. more traditional kinds of
> redundancy, such as mirror sites for the search engine).

With the decentralized solution fault-tolerance is vastly easier to
manage: it is built into the system - every node that does media stuff
directly offloads hosting tasks from the central server.  Furthermore,
when problems arise the decentralized solution handles them far more
gracefully.  Even if the central aggregator goes down the network is still
mostly functional - the only thing that breaks is central search.  If the
central site goes down then the entire system is basically sunk.

It seems to me that the technical hurdles of designing reduncancy into the
central solution to deal with these very real problems would be harder and
require more engineering effort than creating the network decentralized in
the first place.  Furthermore the decentralzied solution can handle these
problems enumerably better.

-Zack

Reply via email to