Dear Sean, Great compilation, this will help for sure!! Thank you!!
Best Regards, Sheel Rana On Fri, May 6, 2016 at 10:38 PM, Sean McGinnis <[email protected]> wrote: > At the Design Summit in Austin, the Cinder team met over three days > to go over a variety of topics. This is a general summary of the > notes captured from each session. > > We were also able to record most sessions. Please see the > openstack-cinder YouTube channel for all its minute and tedious > glory: > > https://www.youtube.com/channel/UCJ8Koy4gsISMy0qW3CWZmaQ > > Replication Next Steps > ====================== > Replication v2.1 was added in Mitaka. This was a first step in supporting > a simplified use case. A few drivers were able to implement support for > this in Mitaka, with a few already in the queue for support in Newton. > > There is a desire to add the ability to replicate smaller groups of volumes > and control them individually for failover, failback, etc. Eventually we > would also want to expose this functionality to non-admin users. This will > allow tenants to group their volumes by application workload or other user > specific constraint and give them control over managing that workload. > > It was agreed that it is too soon to expose this at this point. We would > first like to get broader vendor support for the current replication > capabilities before we add anything more. We also want to improve the admin > experience with handling full site failover. As it is today, there is a lot > of manual work that the admin would need to do to be able to fully recover > from a failover. There are ways we can make this experience better. So > before > we add additional things on top of replication, we want to make sure what > we have is solid and at least slightly polished. > > Personally, I would like to see some work done with Nova or some third > party > entity like Smaug or other projects to be able to coordinate activities on > the compute and storage sides in order to fail over an environment > completely > from a primary to secondary location. > > Related to the group replication (tiramisu) work was the idea of generic > volume groups. Some sort of grouping mechanism would be required to tie in > to that. We have a grouping today with consistency groups, but that has its > own set of semantics and expectations that doesn't always fully mesh with > what users would want for group replication. > > There have also been others looking at using consistency groups to enable > vendor specific functionality not quite inline with the intent of what > CGs are meant for. > > We plan on creating a new concept of a group that has a set of possible > types. > One of these types will be consistency, with the goal that internally we > can > shift things around to convert our current CG concept to be a group of type > consistency while still keeping the API interface that users are used to > for > working with them. > > But beyond that we will be able to add things like a "replication" type > that > will allow users to group volumes, that may or not be able to be snapped in > a IO order consistent manner, but that can be acted on as a group to be > replicated. We can also expand this group type to other concepts moving > forward to meet other use cases without needing to introduce a wholly new > concept. The mechanisms for managing groups will already be in place and a > new > type will be able to be added using existing plumbing. > > Etherpad: > https://etherpad.openstack.org/p/cinder-newton-replication > > Active/Active High Availability > =============================== > Work continues on HA. Gorka gave an overview of the work completed so far > and > the work left to do. We are still on the plan proposed at the Tokyo Summit, > just a lot of work to get it all implemented. The biggest variations are > around > the host name used for the "clustered" service nodes and the idea that we > will > not attempt to do any sort of automatic cleanup for in-progress work that > gets > orphaned due to a node failure. > > Etherpad: > https://etherpad.openstack.org/p/cinder-newton-activeactiveha > > Mitaka Recap > ============ > Two sessions were devoted to going over what had changed in Mitaka. There > were > a lot of things introduced that developers and code reviewers now need to > be > aware of, so we wanted to spend some time educating everyone on these > things. > > Conditional DB Updates > ---------------------- > To try to eliminate races (partly related to the HA work) we will now use > conditional updates. This will eliminate the gap between checking a value > in > setting it, making it one atomic DB update. Better performance than locking > around operations. > > Microversions > ------------- > API microversions was implemented in Mitaka. The new /v3 endpoint should be > used. Any change in the API should now be implemented as a micrversion > bump. > Devref in Cinder with details of how to use this and more detail as to when > a microversion is needed and when it is not. > > Rolling Upgrades > ---------------- > Devref added for rolling upgrades and versioned objects. Discussed need to > make > incremental DB changes rather than all in one release. First release add > new > colume - write to both, read from original. Second release - write to both, > read from new column. Third release - original column can now be deleted. > > Recommended service upgrade order: cinder-api, cinder-scheduler, > cinder-volume, > cinder-backup. After all services upgraded, send SIGHUP to each to release > version pins. > > Multinode grenade tests need to be set up to get regular testing on rolling > upgrades to ensure we don't let anything through that will cause problems. > > Some additional patches are in progress to fix a few internal things like > unversioned dicts that are passed around. This will help us make sure we > don't > change one of those structures in an incompatible way. > > Etherpads: > https://etherpad.openstack.org/p/cinder-newton-mitakarecap > https://etherpad.openstack.org/p/cinder-newton-rollingupgrades > > Scalable Backup > =============== > The backup service was decoupled from the c-vol service in Mitaka. This > allows > us to move backup to other nodes to offload that work. We can also scale > out > to allow more backup operations to work in parallel. > > Also some discussion of whether KVMs change block tracking could be used to > further optimize this process. > > Some other issues were identified with backup. It is currently a sequential > process, so backups can take a long time. Some work is being done to break > out > the backup chunks into separate processes to allow concurrent processing. > > The idea of having different backup types was brought up. This will require > more discussion. The idea is to have a way to configure different volumes > to > be able to be backed up to different backup target types (i.e., one to > Swift > backend, one to Ceph backend, one to GCS, etc.). > > There are some implications for the HA work being done and how to do clean > up. > Those issues will need to be fully identified and worked through. > > Etherpad: > https://etherpad.openstack.org/p/cinder-newton-scalablebackup > > Testing Process > =============== > We discussed our various available testing and how we can get better test > coverage. We also want to optimize the testing process so everyone isn't > stuck > running long running tests just to validate small changes. > > For unit tests, tests that take more than 3-5 seconds to run will be moved > out > to our integration tests. Most of these are doing more than just "unit" > testing > so makes more sense for them to be separated out. > > There's some desire to gate on coverage, but it was discussed how that can > be > a difficult thing to enforce. There was a lot of concern that a hard policy > around coverage would lead to a lot of useless unit tests that don't > really add > valuable testing, just adds coverage over a given path to get the numbers > up. > > It may be nice to have something like our non-voting pylint jobs to help > detect when our test coverage decreases, but not sure if the additional > infra > load to run this would be worth it. > > We have a functional test job that is currently non-voting. It has been > broken > recently and due to it being non-voting it was missed for some time. This > needs > to be fixed and the job changed to be voting. Since the majority of these > tests > were moved from the unit tests, they should be fine to make voting once > passing > again. > > Having in-tree tempest tests were also discussed. We've used tempest for > testing things not in line with the mission of tempest. We also have tests > in > there that may not really be relevant or needed for other projects getting > them by running all tempest tests. For the ones that are really Cinder > specific > we should migrate them from being in tempest to being in tree with Cinder. > That > also gives us full control over what is included. > > We also discussed plans for how to test API microversions and adding pylint > testing to os-brick. > > Etherpad: > https://etherpad.openstack.org/p/cinder-newton-testingprocess > > CinderClient and OpenStackClient > ================================ > A group of folks are working through a list of commands to make sure osc > has > functional parity with cinderclient. We need to have the same level of > support > in osc before we can even consider deprecating the cinder client CLI. > > The idea was raised to create a wiki page giving a "lookup" of cinder > client > commands to openstackclient commands to help users migrate over to the new > client. Will need to decided how to support this given long term plans for > the > wiki, but having the lookup posted somewhere should help. > > We do have an extra challenge in that we added a cinderclient extension for > working with os-brick as a stand alone storage management tool. We will > work > with the osc team to see what the options are for supporting something like > this and decide what the best end user experience would be for this. It > may be > that we don't deprecate python-cinderclient CLI just for this use case. > > Etherpad: > https://etherpad.openstack.org/p/cinder-newton-cinderclienttoosc > > Unconference > ============ > Made midcycle plans based on survey results. (Has since changed since we > ran in > to complications with hotal availability) > > Talked breifly about non-Cinder volume replication and how to recover > during > DR. Discussed whether all backends support managing existing volumes. How > to > discover available volumes. What to do with snapshots on manage. > > Etherpad: > https://etherpad.openstack.org/p/cinder-newton-unconference > > Nova Cross Project > ================== > All about multiattach. Matt Riedman wrote up a nice recap of this > discussion > already: > > http://lists.openstack.org/pipermail/openstack-dev/2016-May/094018.html > > Contributors Meetup > =================== > API capability and feature discovery. > What to do for backends that only support allocation in units greater than > 1G. > Next steps for user message reporting. > Release priorities: > - User messaging > - Active/Active HA > - Improved functional test coverage > - Better support for cheesecake (repl v2.1) > Extending attached volume. > Discussed qemu-img convert error a couple drivers were seeing. > FICON protocol support. > Generic group discussion. > Force detach. > Replication questions. > QoS support for IOPs per GB. > Per tenant QoS. > Cinder callbacks to Nova for task completion. > Config options inheritance (defined in Default, apply to subsections) > Deadlines (http://releases.openstack.org/newton/schedule.html) > Status of image caching. > Driver interface compliance checks and interface documentation. > > Etherpad: > https://etherpad.openstack.org/p/cinder-newton-contributorsmeetup > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
