1. I have stored millions of nodes in ZK, ZK works well. there is no problem to store a large number of nodes into ZK. Currently there is only one row in -ROOT-, namely there is only one region in .META. There are many files in a region. before ZK is bottleneck, NameNode is bottleneck. assume that one region will consume 1K memory in ZK, one million regions only consume 1G memory.
2. putting .META. into ZK have nothing to do with SPOF of master. wherever we put .META. table, we must handle SPOF of master. 3.the region server that host .META. may be a bottleneck. we can improve the read performance of ZK by adding more observers. On Mon, Mar 21, 2011 at 9:15 PM, <[email protected]> wrote: > I don't think it's a good idea. > 1. Why we need -ROOT-? Because we are afraid .META. is larger than a > region's size. So .META. can be larger and ZK is not suitable for too > many data. > 2. Even we put .META. into ZK, we still need a server to manage the data > (now it's Master Server). And the server is still a single point. > So the problem is how can we do when Master Server got problem? Rather > than why we don't put .META. in ZK? > > On 2011年03月18日 23:58, Ted Dunning wrote: > > Excuse me? > > > > How does that affect the issue of snapshotting a table? > > > > And how can replication NOT involve meta-data? > > > > On Thu, Mar 17, 2011 at 10:55 PM, jiangwen w<[email protected]> > wrote: > > > >> replication does not involve meta data. > >> > >> On Fri, Mar 18, 2011 at 12:32 PM, Ryan Rawson<[email protected]> > wrote: > >> > >>> Is it possible to search a list of z nodes? That is what we do now with > >>> meta > >>> in hbase. > >>> > >>> I used to be a fan, but I think self hosting all important meta data is > >> the > >>> best approach. It makes lots of things easier, like replication, > >> snapshots, > >>> etc. > >>> On Mar 17, 2011 9:27 PM, "jiangwen w"<[email protected]> wrote: > >>>> how do you think about moving meta table to ZK, so no meta table are > >>> needed. > >>>> if we do so, we need enhance ZK in the following way: > >>>> 1. let children of ZNode in order. > >>>> > >>>> if we do so, we can benifit: > >>>> > >>>> 1. no need to treat meta table as a special way. this will simplify > the > >>> code > >>>> a lot > >>>> 2. ZK is highly available, so we don't worry the availablility of the > >>> meta > >>>> data. > >>>> 3. currently if the region server where meta table is on failed, the > >>> whole > >>>> cluster may pause. > >>>> if we move meta table to ZK, there is no such problem. > >>>> 4. meta table may be a hotspot, but in ZK reading is scalable by > adding > >>> more > >>>> observers. > >>>> > >>>> > >>>> Sincerely > > TREND MICRO EMAIL NOTICE > The information contained in this email and any attachments is confidential > and may be subject to copyright or other intellectual property protection. > If you are not the intended recipient, you are not authorized to use or > disclose this information, and we request that you notify us by reply mail > or telephone and delete the original message from your mail system.
