Hello all, I read the distributed log code closely. I found that the DL namespace is a flat namespace. There will be a potential issue if there are a lot of streams created under a same namespace. I am very curious what are the thoughts behind that. Here are some questions:
- How many streams that a namespace can support? It seems to be bound with the limitation on the number of children that a zookeeper znode can have. What's the maximum number of logs do you guys have? - Why not choose a tree representation? Then it might be easier to organize streams. For example, if I want to use multiple dl streams as partitions, I can just easily organize them together under same znode. - Also if it is a tree-like namespace, it might be easier to implement a filesystem over the streams. Each file can be backed by one dl stream. In that way, I can also use DL as long term storage. Any thoughts? Appreciate your comments. -- - jderrick
