Hi, Here are some initial ideas, welcome to join and discuss:
【Architecture】 shared-nothing 【RPC framework】 thrift 【Design goal】 - Having recommended configuration, which can deal with most common scenarios - When Facing extreme scenarios, we have solutions to guarantee the system the undead 【System information (ip and port of all nodes in the cluster)】 - How to store: Store full copy in the config file in each node, update in real-time. - Update protocol: Raft, each log contains the whole information. 【Metadata and schema management】 Metadata (The data allocation info) : partition -> nodes Schema: The schema tree. For example: root.sg1.d1.s1, double, rle, snappy - Partition: storage group - Allocation: consistent hashing - Replication factor: 2N+1 (3 or 5 may be enough) - Update Protocol: Raft 【Data management】 Partition: storage group + time range Allocation: Allocation manager (we can apply different allocation strategy) 【Config file】 ## If it is cluster Initialization, it needs to be the all nodes. Otherwise (scale out) it is any one or more living nodes in the cluster. hosts: (ip:port)+ 【Script】 - start-cluster.sh : Run in each node when a cluster initialization - stop-cluster.sh : Run in arbitrary living node in the cluster - start-node.sh : Run in the newly added node - remove-node.sh ip port : Run in arbitrary living node in the cluster Best, Jialin Qiao Xiangdong Huang <[email protected]> 于2019年12月4日周三 下午8:23写道： > Hi, > > I notice Tian Jiang is working on `cluster_new` branch and the branch > keeps updating frequently. > > Distributed version is not an easy work and is likely not a single guy can > finish well. > I'd like to begin to read the codes to understand Tian Jiang's idea. > I find a very old issue [1] which is for the distributed version. > > So, let's discuss about the design and current codes in that issue. @江天 > <[email protected]> > > And welcome other contributors to joining. > > [1] https://issues.apache.org/jira/browse/IOTDB-68 > < > https://issues.apache.org/jira/browse/IOTDB-68?jql=project%20%3D%20IOTDB%20AND%20text%20~%20distribution > > > > Best, > ----------------------------------- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > -- ————————————————— Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
