popozy opened a new pull request #697: URL: https://github.com/apache/servicecomb-service-center/pull/697
### Pull request message Implement ms datasource register service in etcd mode. ### History pr: + datasource framework: https://github.com/apache/servicecomb-service-center/pull/695 ### Target Refactoring service-center to make it support different backend storage(Etcd, Mongo, etc.) and kv design ### Background At this point, register-information from services in mesh could be stored in etcd cluster(embeded or isolated-cluster) with a certain kv design format(see the design example [here](https://github.com/apache/servicecomb-service-center/blob/master/examples/etcd_data_struct.yaml)). ### Problem Howere, the service-comb can not make it if new kv design format from another database want to be introduced to the project. For example, kv format will be designed like [this](https://github.com/apache/servicecomb-service-center/blob/master/examples/mongodb_data_struct.yaml) when we take mongo into consideration. ### Enhancement So, I'd like to **abstract kv constructing operation from api service layer to modules named datasource**, where the module acts like shim between backend databse and api bussiness layer. They act as a plugin located in server/service/dep, server/service/auth, server/service/ms. **The storageShim module will construct request metada from service-resigter-api to different kv according to different backend storage cluster.** ### Changes after refactoring Take the service information register as a example. #### Before refactoring ```go // api layer func CreateServicePri(ctx context, request CreateServiceRequest) { // parameter validation // tracing // construct kv in etcd-cluster format serviceKey := &pb.MicroServiceKey{ Tenant: domainProject, Environment: service.Environment, AppId: service.AppId, ServiceName: service.ServiceName, Alias: service.Alias, Version: service.Version, } index := apt.GenerateServiceIndexKey(serviceKey) key := apt.GenerateServiceKey(domainProject, service.ServiceId) keyBytes := util.StringToBytesWithNoCopy(key) indexBytes := util.StringToBytesWithNoCopy(index) aliasBytes := util.StringToBytesWithNoCopy(apt.GenerateServiceAliasKey(serviceKey)) opts := []registry.PluginOp{ registry.OpPut(registry.WithKey(keyBytes), registry.WithValue(data)), registry.OpPut(registry.WithKey(indexBytes), registry.WithStrValue(service.ServiceId)), } uniqueCmpOpts := []registry.CompareOp{ registry.OpCmp(registry.CmpVer(indexBytes), registry.CmpEqual, 0), registry.OpCmp(registry.CmpVer(keyBytes), registry.CmpEqual, 0), } failOpts := []registry.PluginOp{ registry.OpGet(registry.WithKey(indexBytes)), } // call etcd client to store metadata resp, err := backend.Registry().TxnWithCmp(ctx, opts, uniqueCmpOpts, failOpts) // resp check return resp, err } ``` #### After refactor ``` go // scserver start up main.go func main.go() { // init a instance err := ms.Init(ms.Options{ Endpoint: "", PluginImplName: ms.ImplName(archaius.GetString("servicecomb.ms.name", "etcd")), }) // err check } // api layer func CreateServicePri(ctx context, request CreateServiceRequest) { // parameter validation // quota check // call etcd client to store metadata resp, err := ms.MicroService().RegisterService(ctx, request) // resp check return resp, err } ``` **Attention**: datasource act as a kv wrapper according to the interface implementor, will not do parameter and response validation ``` go // datasource module // ms etcd implement // install ms plugin func init() { ms.Install("etcd", func(opts ms.Options) (ms.DataSource, error) { return etcd.NewDataSource(), nil }) } type DataSource struct{} func NewDataSource() *DataSource { // TODO: construct a reasonable DataSource instance log.Warnf("microservice mgt data source enable etcd mode") inst := &DataSource{} if err := inst.initialize(); err != nil { return inst } return inst } // RegisterService implement func RegisterService(ctx context, request CreateServiceRequest) (*registry.PluginResponse, error) { // construct kv in etcd-cluster format opts, uniqueCmpOpts, failOpts := constructEtcdFormat(ctx, request) resp, err := backend.Registry().TxnWithCmp(ctx, opts, uniqueCmpOpts, failOpts) return resp, err } // ms mongo implement func RegisterService(ctx context, request CreateServiceRequest) (*registry.PluginResponse, error) { // construct kv in etcd-cluster format key, val := constructMongoFormat(ctx, request) resp, err := backend.Registry().TxnWithCmp(ctx, key, val) return resp, err } ``` UT for example can be refered [here](https://github.com/apache/servicecomb-service-center/blob/5e390e7e6ff880dd282dd631deb726e8a3a99d64/server/service/microservice_test.go#L46) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
