On Wed, Jun 17, 2015 at 8:35 PM, Kaushal M <[email protected]> wrote:
> At the Gluster Summit, everyone agreed that GlusterD should be the > first component to be targeted for GlusterFS-4.0. We had good > discussions on what GlusterD lacks currently and what is required for > GlusterD-2.0. KP and I had promised to send an update to mailing list > summarizing the discussions, and this is it. > > Along with the summary, we'll also be discussing our plans for Manila > integration and how we are planning to do it with GlusterD-2.0. > > ## Gluster Summit Summary > In the summit, KP and I presented a talk titled *GLUSTERD NEXT GEN > OR: HOW WE CAN STOP BEING A ROADBLOCK AND INSTEAD BECOME THE ROAD TO > SCALABILITY*. The slides can be viewed at [1][1]. There is no video > recording of it unfortunately. > > The summary of the presentation is below. > > ### Problems > GlusterD, as it is currently, is not scalable which will prevent > GlusterFS as whole from scaling. The scaling issues can be classified > into, > - Node scalability > - Maintenance scalability > - Integration scalability > - Contribution scalability > > #### Node scalability > This is the general scalability we all think about, scale in terms of > node/machine/clients. GlusterD scalability is help back in this > because of the store, transaction and synchronization mechanisms used > in GlusterD. > > #### Maintenance scalability > Maintenance scalability is to with the problems we as GlusterD > maintainers faced. This is mainly related to the huge, monolithic code > base of the current GlusterD, which makes splitting of maintenance and > ownership tasks hard. > > #### Integration scalability > Integration scalability can split into internal and external integration. > Internal integration is the integration dealing with new features > being added to GlusterFS. Every new feature being added needs to touch > GlusterD or CLI in some way. This has generally been done with > copy/paste coding which has added to the maintenence overhead. > External integration is the integration of Gluster with other projects > or other projects with Gluster. This is hurt due to the > non-availability of a proper API for GlusterD operations. All > interaction with GlusterD currently happens only via the CLI. And the > output we provide is generally inconsistent to be programatically > parsed. > > #### Contribution scalability > Getting new contributors for GlusterD is hard. New contributors are > put off because GlusterD is hard to understand, and because there > isn't enough documentation. > > > So GlusterD-2.0 will need to > - be scalable to 1000s of nodes > - have lower maintenance costs > - enable better external and internal integrations > - make it easier for newer contributors > > ### Design characteristics for GlusterD-2.0 > For GlusterD-2.0 to satisfy the above listed requirements and solve > the problems listed before, it should have the following > characteristics. > One other design characteristic I can think of is to have GlusterD-2.0 be able to make best use of Nodes based on the Node topology, not sure if this is covered by any other feature below ? IOW, admin should be able to provide a hierarchy of how nodes are provisioned (rack, site, zone, region, geo etc) and glusterd should use this info while provisioning bricks for volume creation > #### Centralized store > This should help with our numbers scalability issues. GlusterD-2.0 > will be built around a centralized store. This means, instead of the > GlusterD volume and peer information being persisted on all nodes, it > will be stored only on a subset of the nodes in a trusted storage > pool. > > We are looking at solutions like etcd and consul, both of which > provide a distributed key/value store (and some more useful features > on top), to provide the centralized store. The transaction mechanism > for Gluster operations will be built around this centralized store. > > Moving to an external store provider and a transaction framework built > around it will reduce a lot of the complexity in GlusterD. > > #### Plugins > This mainly for the maintainability and internal integration aspects. > GlusterD-2.0 will have a plug-gable, modular design. We expect all the > commands of GlusterD to be implemented as plugins. Certain other parts > of GlusterD, including things like volgen, volume-set, rest api, etc. > This will allow new features to be integrated in to GlusterD easily. > The code for these plugins is expected to live with the feature, and > not in GlusterD. > > Doing a plugin design requires the defining of well defined plugin > interfaces to not just plug into GlusterD-2.0, but also interact with > GlusterD well. I'll be posting a document on this to the mailing list > soon. > > The GlusterD maintainers/team will be doing the implementation of a > few core commands required including volume create, start, stop, > delete, expand, shrink and set/reset options. > > #### Rest API > To provide better a better method for external projects to hook on to. > GlusterD-2.0 will provide a HTTP REST API, with JSON output and proper > API versioning. We will be looking for inspiration from other storage > solutions to define the APIs presented to the users and admins. > > #### Logging > To help with debuggability, GlusterD-2.0 will provide better logs that > should allow easier tracking of operations across machines. We are > currently looking at the oVirt logging mechanism, which tags all logs > of an operation with a unique id, for inspiration. > > #### Documentation > We hope to do a much better job of documenting the technical details > of GlusterD-2.0, starting with design documents. We still don't have > > > ## Manila and GlusterD-2.0 > > Manila is the File System as a Service component of OpenStack. Manila > has two GlusterFS providers neither of which currently support the > full Manila API. One uses a sub-directory over NFS approach and the > other uses a volume based approach. Both of them require specific > features from GlusterFS to be completely functional by the next > OpenStack release, which is expected for October 2015. Csaba listed > these requirements at [2][2]. This lists two possible approaches, > 1. Implement directory level operations (required by the sub-directory > approach) > 2. Implement intelligent volume creation (IVC) (required for the > volume based approach) > (Please read the linked mail-thread for more context) > > I have 2 things to say here : 1) Manila uses GlusterFS SSL auth mode, so is there any discussion happening around adding support in GlusterD-2.0 for managing SSL certificates ? For eg: Other (eg Netapp) storage arrays do provide basic digital cert mgmt support for server and client auth. I feel it would be good to have glusterd provide support to generate, install, manage self-signed and/or CA signed digital certs. This will not just help Manila usecase, but also help GlusterFS provide a complete solution for digital cert management which will aid the SSL auth support feature of GlusterFS In fact, this can be done in a modular way where in the default implm will be that of GlusterD cert module, but optionally one can use openstack Barbican service too as a cert mgmt service 2) In the manila community there was some intense discussion on the definition of multi-tenancy when applied to storage and network level multi-tenancy was voted as being very important in Manila cloud usecase. Do we have any thoughts on how GlusterD-2.0 / GlusterFS-4.0 can look at providing tenant separation at network layer ? thanx, deepak
_______________________________________________ Gluster-devel mailing list [email protected] http://www.gluster.org/mailman/listinfo/gluster-devel
