On Fri, 1 Aug 2003, Ka-Ping Yee wrote: > On Fri, 1 Aug 2003, zachary rosen wrote: > > A quick note - the decentralized system that is being proposed is NOT peer > > to peer. At the top, at the aggregator, it functions just the same as the > > centralized solution: One database, searchable and acessable by all - ie > > napster. > > > > The difference is how the metada gets to the central DB. Either it is > > aggregated from nodes, or required to be centrally submitted to bypass the > > technical hurdles of aggregation. > > I believe we are all agreed up to this point.
Woo! :) > > I am under the impression that feasability or technical-hard-ness of > > building the metadata collection functionality to be decentralized rather > > than requiring it to be centrally submitted does not nearly outweigh the > > problems with admining, maintaining, and hosting the central solution. > > We have to maintain the central host anyway, because we've already > agreed that we need a main search site. The question is whether we > ought to introduce *additional* administrative tasks for people > running other DeanSpace sites in the media network. I completely disagree. I don't see how having node admins help manage their local repositories increases the amount (*additional*) administrative work. We want to have all nodes have media hosting capabilities (I believe). This means there will have to be admin work done to set up a media hosting module no matter what. The only real difference between centralized and decentralized in terms of admin work required then becomes the mundane maintenance tasks: pruning and organization. If the nodes are empowered to maintain their own local repository then doing this work is offloaded from the potentially very large bottlneck of having it all maintained / pruned in a central site by one set of admins (DMT) to the many capable node admins. Am I missing something? > > When i get home i will be rifling through some books for quotes to support > > my claim that Reed's Law and End-to-End principals support the > > decentraralized design over the centralized design. > > Since we seem to have different perspectives on what these terms mean, > i'd rather talk directly about the reasoning, as in "This design is > better for the following reasons..." rather than "This design is better > because there's a principle that says so." Of course ;p > Now, i'm not against fault-tolerance. Yes, of course it would be good > to have a resilient system where some of the hosts can fail and things > keep working. We can certainly talk about ways to do that after the > data has been centrally submitted (e.g. more traditional kinds of > redundancy, such as mirror sites for the search engine). With the decentralized solution fault-tolerance is vastly easier to manage: it is built into the system - every node that does media stuff directly offloads hosting tasks from the central server. Furthermore, when problems arise the decentralized solution handles them far more gracefully. Even if the central aggregator goes down the network is still mostly functional - the only thing that breaks is central search. If the central site goes down then the entire system is basically sunk. It seems to me that the technical hurdles of designing reduncancy into the central solution to deal with these very real problems would be harder and require more engineering effort than creating the network decentralized in the first place. Furthermore the decentralzied solution can handle these problems enumerably better. -Zack
