> Hi Eoghan, > > Thanks for the note below. However, one thing the overview below does not > cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged. Many > folks feel that this technology is a viable solution for the problem space > discussed below.
Great question Brad! As it happens we've been working closely with Paul Dix (lead developer of InfluxDB) to ensure that this metrics store would be usable as a backend driver. That conversation actually kicked off at the Juno summit in Atlanta, but it really got off the ground at our mid-cycle meet-up in Paris on in early July. I wrote a rough strawman version of an InfluxDB driver in advance of the mid-cycle to frame the discussion, and Paul Dix traveled to the meet-up so we could have the discussion face-to-face. The conclusion was that InfluxDB would indeed potentially be a great fit, modulo some requirements that we identified during the detailed discussions: * shard-space-based retention & backgrounded deletion * capability to merge individual timeseries for cross-aggregation * backfill-aware downsampling The InfluxDB folks have committed to implementing those features in over July and August, and have made concrete progress on that score. I hope that provides enough detail to answer to your question? Cheers, Eoghan > Thanks, > > Brad > > > Brad Topol, Ph.D. > IBM Distinguished Engineer > OpenStack > (919) 543-0646 > Internet: bto...@us.ibm.com > Assistant: Kendra Witherspoon (919) 254-0680 > > > > From: Eoghan Glynn <egl...@redhat.com> > To: "OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org>, > Date: 08/06/2014 11:17 AM > Subject: [openstack-dev] [tc][ceilometer] Some background on the gnocchi > project > > > > > > Folks, > > It's come to our attention that some key individuals are not > fully up-to-date on gnocchi activities, so it being a good and > healthy thing to ensure we're as communicative as possible about > our roadmap, I've provided a high-level overview here of our > thinking. This is intended as a precursor to further discussion > with the TC. > > Cheers, > Eoghan > > > What gnocchi is: > =============== > > Gnocchi is a separate, but related, project spun up on stackforge > by Julien Danjou, with the objective of providing efficient > storage and retrieval of timeseries-oriented data and resource > representations. > > The goal is to experiment with a potential approach to addressing > an architectural misstep made in the very earliest days of > ceilometer, specifically the decision to store snapshots of some > resource metadata alongside each metric datapoint. The core idea > is to move to storing datapoints shorn of metadata, and instead > allow the resource-state timeline to be reconstructed more cheaply > from much less frequently occurring events (e.g. instance resizes > or migrations). > > > What gnocchi isn't: > ================== > > Gnocchi is not a large-scale under-the-radar rewrite of a core > OpenStack component along the lines of keystone-lite. > > The change is concentrated on the final data-storage phase of > the ceilometer pipeline, so will have little initial impact on the > data-acquiring agents, or on transformation phase. > > We've been totally open at the Atlanta summit and other forums > about this approach being a multi-cycle effort. > > > Why we decided to do it this way: > ================================ > > The intent behind spinning up a separate project on stackforge > was to allow the work progress at arms-length from ceilometer, > allowing normalcy to be maintained on the core project and a > rapid rate of innovation on gnocchi. > > Note that that the developers primarily contributing to gnocchi > represent a cross-section of the core team, and there's a regular > feedback loop in the form of a recurring agenda item at the > weekly team meeting to avoid the effort becoming silo'd. > > > But isn't re-architecting frowned upon? > ====================================== > > Well, the architecture of other OpenStack projects have also > under-gone change as the community understanding of the > implications of prior design decisions has evolved. > > Take for example the move towards nova no-db-compute & the > unified-object-model in order to address issues in the nova > architecture that made progress towards rolling upgrades > unneccessarily difficult. > > The point, in my understanding, is not to avoid doing the > course-correction where it's deemed necessary. Rather, the > principle is more that these corrections happen in an open > and planned way. > > > The path forward: > ================ > > A subset of the ceilometer community will continue to work on > gnocchi in parallel with the ceilometer core over the remainder > of the Juno cycle and into the Kilo timeframe. The goal is to > have an initial implementation of gnocchi ready for tech preview > by the end of Juno, and to have the integration/migration/ > co-existence questions addressed in Kilo. > > Moving the ceilometer core to using gnocchi will be contingent > on it demonstrating the required performance characteristics and > providing the semantics needed to support a v3 ceilometer API > that's fit-for-purpose. > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev