Re: [openstack-dev] [Bilean][CloudKitty][Telemetry] Opendiscussionaround Bilean and existing OpenStack components

2016-07-08 Thread Stéphane Albert
Hi,

On Fri, Jul 08, 2016 at 10:39:57AM +0800, 吕冬兵 wrote:
> Hi Stéphane,
> 
> I got what you mean, I’m sure that CloudKitty can rating by event, but
> I have some other puzzles. Let me generally describe the arch of
> Bilean first
> 
> [events] <—> [rating] <—> [billing]
> 
> When event coming from notification, bilean engine will rating the
> resource according policy and rule. A rule reference to a kind of
> resource(instance, volumes e.g), and a policy consists of several
> rules, different user can use different billing policy). After that
> engine will update user’s rate. Bilean doesn’t have period task to do
> billing work, but triggered by events and period task to do health
> check for users (daily task now). When user’s balance is almost used
> up, bilean engine will notify user.
We don't have the same semantics in CloudKitty but it's pretty similar
in the way rating rules processing is done, except that we don't work on
an user basis but tenant/project basis. We want to change this
limitation and enable the cloud admin to apply rating rules on whatever
level he wants. We'll be happy to have you implement this in CloudKitty.
About your health check, we've got an API on top of the storage and a
way to generate report, you can easily get the current total for a
specific user (1 API call). You can implement this with a simple cron
querying the API with the user_id, and checking if the number is past
your set threshold. It's not directly in the code because it's not our
main goal. Plus it's easily done, the amount of code to create a tool
doing this is minimal:

- Load CloudKitty's SDK
- Request total for an user
- Check threshold
- Custom alarming code

> So you see, Bilean is a closed-loop billing solution, what I mean
> trigger-based billing is not just rating by events, the main thing is
> billing. Does CloudKitty do billing too? And how? If not or has
> different solution about that, force to combine them will make it
> neither fish nor fowl.
We don't do billing per se, but we implement a way to crunch OpenStack
data to get sensible numbers (cost). Billing is out of the scope since
it requires knowledge of the local laws, such as VAT, accounting, etc.
There is a lot of billing solutions out there, and most companies
already have one. It looks like you want to do alarming based on
resource consumption, having a cost if enough.

As far as I understand your use case, I don't see why you need to
reimplement components. There is event support in Ceilometer, if you
need to access events directly you can subscribe to them. CloudKitty is
responsible of rating, Gnocchi of metrics storage. The only tricky part
is the alarming one which if I understand correctly is just a matter of
a few lines of Python code. Or even better an Aodh alarm based on
Gnocchi metrics which is the result of CloudKitty's computation.

If I missed or misunderstood something, feel free to correct me.

I would like to find a way to collaborate on this use case without
scattering efforts on similar projects.

Cheers,
Stéphane

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Bilean][CloudKitty][Telemetry] Open discussionaround Bilean and existing OpenStack components

2016-07-07 Thread Stéphane Albert
On Thu, Jul 07, 2016 at 09:50:02AM +0800, 吕冬兵 wrote:
> Hi,
> 
> I'm sorry to see this discussion so late:). Thanks for attention.
> 
> I don't oppose to contribute to add trigger-based solution to
> CloudKitty, I just want to know if it's possible to support
> trigger-based for CloudKitty based on now arch, pls generally describe
> how. And another thing I want to make sure is that if it's good to mix
> two different solution in one component.
Hi,

At the moment we have an arch that looks like this

[collector] <-> [orchestrator] <-> [rating, storage, etc]

The orchestrator is responsible of polling data from different backends
using collectors.
We are currently working on a refactor of the internal architecture, we
plan on having rating engines as drivers to ease transitions and
possibility to fully integrate with other engines. And we'll add the "on
demand collect" which is basically a smarter rating engine.
We can easily do something like this:

   [events]
  ^
  |
  v
[collector] <-> [orchestrator] <-> [rating, storage, etc]

Which is just adding notification handling to the orchestrator. Events
will then be processed by the rating engine.

Everything can be enabled/disabled easily using configuration so that
users can choose what type they need (poll/event). I don't see what can
be the blocking point here.

The new rating engine is on the agenda of next Monday meeting, we can
add a point to talk about collaboration on event based rating.

I'm available if you need any information.

Cheers,
Stéphane

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Bilean][CloudKitty][Telemetry] Open discussion around Bilean and existing OpenStack components

2016-07-01 Thread Stéphane Albert
Hi,

I would like to continue the discussion that started in the [review][1]
for the Big-Tent integration of the project Bilean.

In the [review][1] the Bilean team stated that a new project was needed
to overcome limitations of existing components.
In this thread, I would like to have an open discussion about what
features are lacking in the available components and what needs to be
done to integrate the Bilean use case with the current components.

I'm not opposed to changes and new features in CloudKitty, and I'm
pretty sure that trigger-based billing can be integrated in CloudKitty's
codebase.

>From my perspective, CloudKitty team is a small team and having two
teams working on rating/billing is just scattering contributions and is
detrimental to both projects. It brings confusion to users minds about
what components should be used.

We can add this topic to our meeting agenda, so we can have a talk on
IRC.

I'm hoping we'll find a solution that benefits existing projects and
enables you to implement your trigger-based billing solution.

Cheers,
Stéphane

[1]: https://review.openstack.org/#/c/334350/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [cloudkitty] PTL candidacy

2016-03-18 Thread Stéphane Albert

Hi everyone,

I'm announcing my candidacy for PTL of the CloudKitty team during the 
Newton

cycle.

Some of the features and improvements of the project I envisioned are 
not yet

ready and contributed and there is more to come. But due to lack of
contributor's time we'll need more time to make them a reality.
That's why I'd like to apply for a second cycle to help the project get 
these

features and keep the project going forward.
My main focus during this cycle will be to improve project communication 
and

community development.

As we are about to get released as part of Mitaka, I hope it will help 
the

CloudKitty project get a better focus from other people and increase
contributions.

During the Newton cycle I'd like to keep the project focused on these 
specific

points:

- Improve code QA and global test coverage, Mikata was a good step 
forward but

  we can go further.

- Improve internal data modeling and global user experience with new 
APIs.


- Improve performances and scaling.

Review link: https://review.openstack.org/#/c/294327/

Cheers

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [cloudkitty] Meeting canceled for week 52 and 53

2015-12-21 Thread Stéphane Albert
Hi,

Most of the guys are currently in vacations, we are canceling meetings
for the next 2 weeks. We'll be back at the beginning of January.

Happy holidays

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [cloudkitty] Feature freeze for release 0.5

2015-12-16 Thread Stéphane Albert
Hi,

As you might already know we are using the independent release which
means that we can release whenever we want.

Like we discussed during the Tokyo summit our goal is to release a
couple versions before the Mitaka release which should correspond to the
1.0 version.

Every version will implement a new feature, or improve some part of the
code, deprecating the old one.

If we want to implement all the features we envisioned we'll need to
release 0.5 version ASAP to focus on next features. Most of the code
regarding 0.5 are pending review.

The last goals are to integrate bugfixes and Gnocchi support before
releasing first rc. I got positive feedbacks from people testing the
gnocchi integration. Code only needs minor fixes and code review, people
should start reviewing code as it's mostly finished and only a few minor
parts are subject to changes.

Some gerrits rights problems are being fixed so that we can release a
0.5 branch and start focusing on next version while RC is being tested.
You can still send patch to review, but their integration will be
postponed until we have a 0.5 branch.
Any blueprint that has not yet been approved will not be part of the 0.5
release.

Just a quick reminder that 0.6 will feature new collector and resources
models. Which will help people needing multiple collector instances for
different backends (for example multiple ceilometer backends). A
compatibility layer will be added to support legacy code and data
format.

We're only a few steps away from releasing gnocchi support in
CloudKitty. Let's focus and make it a success :)

Cheers

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [cloudkitty] IRC meetings frequency and dates

2015-12-16 Thread Stéphane Albert
Hi,

We're having more and more contributors to the CloudKitty project, which
is really nice. But sadly we hardly see new faces at the meetings.
This mail is just to make a quick check to find out what's the reason
behind that.

Is this because the meeting hours are hard to attend? I know that most
new contributors are located in Asia and 2PM UTC is pretty late for
them. Or maybe Monday is not the best fit for this.

Due to the small number of people involved in the meetings we might
change the frequency of them.

If you want to attend the meetings but can't for some reason, answer
this mail and tell us why. We'll find a way to make it convenient for
more people.

Cheers

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [cloudkitty] Changing release model type

2015-12-16 Thread Stéphane Albert
Hi,

Just a quick note to let you know that we are changing CloudKitty
release type. We used independent one which forbids us of having stable
branches and being part of the OpenStack release. We already discussed
it during the OpenStack summit in Tokyo and it was pretty clear that we
needed to change that.
The new release type will be cycle-with-intermediary which lets us have
multiple intermediary releases but tag a specific one as the end of
cycle release, which means we'll have a Mitaka one.

There is a pending patch in review
https://review.openstack.org/#/c/257868/.

Thanks Thierry for the tip and the reminder :)

Cheers

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [cloudkitty] Remove inactive contributors from cloudkitty-core

2015-12-03 Thread Stéphane Albert
Hi,

We didn't get your opinion Guillaume, are you approving it?

Cheers

- Forwarded message from Stéphane Albert <sheeprine...@nullplace.com> -

Date: Tue, 24 Nov 2015 11:51:42 +0100
From: Stéphane Albert <sheeprine...@nullplace.com>
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [cloudkitty] Remove inactive contributors from 
cloudkitty-core
User-Agent: Mutt/1.5.21 (2010-09-15)

Hi,

Some people in the cloudkitty-core group are not active anymore. They
haven't produced a single contribution during the last cycle. Here is
the list of the two people:

- Adrian Turjak
- François Magimel

I think we should remove them from the group. We've got new faces in the
project, let's get some space for them ready :).

Are you guys OK with this?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

- End forwarded message -

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [gnocchi][rating] Issues regarding gnocchi support in CloudKitty

2015-11-26 Thread Stéphane Albert
Hi Julien,

You'll find attached to this mail two dump files.

gnocchi_resource.txt is an example of resource requests and responses
from gnocchi.

gnocchi_measure.txt an example of a timeframe request.
Metric (vcpus) measure
==
Data stored by ceilometer with default configuration (devstack).

GET 
http://10.8.8.168:8041/v1/metric/9c26bbea-6041-4067-9384-f6aa9b4ce120/measures
← 200 application/json 1.71kB 295ms
Host: 10.8.8.168:8041
Connection:   keep-alive
X-Auth-Token: 90cf2d940e464ae0aef733d5f124aa43
Accept-Encoding:  gzip, deflate
Accept:   application/json, */*
User-Agent:   keystoneauth1
No request content
Date:Thu, 26 Nov 2015 14:58:59 GMT
Server:  Apache/2.4.7 (Ubuntu)
content-length:  1752
Keep-Alive:  timeout=5, max=100
Connection:  Keep-Alive
Content-Type:application/json; charset=UTF-8
JSON
[
[
"2015-11-23T00:00:00+00:00",
86400.0,
1.0
],
[
"2015-11-24T00:00:00+00:00",
86400.0,
1.0
],
[
"2015-11-25T00:00:00+00:00",
86400.0,
1.0
],
[
"2015-11-26T00:00:00+00:00",
86400.0,
1.0
],
[
"2015-11-25T15:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-25T16:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-25T17:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-25T18:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-25T19:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-25T20:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-25T21:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-25T22:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-25T23:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T00:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T01:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T02:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T03:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T04:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T05:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T06:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T07:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T08:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T09:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T10:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T11:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T12:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T13:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T14:00:00+00:00",
3600.0,
1.0
],
[
"2015-11-26T03:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T04:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T05:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T06:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T07:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T08:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T09:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T10:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T11:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T12:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T13:00:00+00:00",
300.0,
1.0
],
[
"2015-11-26T14:00:00+00:00",
300.0,
1.0
]
]

Request for a 1h timeframe
==

GET 
http://10.8.8.168:8041/v1/metric/9c26bbea-6041-4067-9384-f6aa9b4ce120/measures?start=2015-11-23T10%3A00%3A0.0%2B00%3A00=2015-11-23T11%3A00%3A0.0%2B00%3A00=max
← 200 application/json 2B 64ms
Host: 10.8.8.168:8041
Connection:   keep-alive
X-Auth-Token: 948f7cf696d94c41908e819237112876
Accept-Encoding:  gzip, deflate
Accept:   application/json, */*
User-Agent:   python-keystoneclient
No request content
Date:Thu, 26 Nov 2015 14:17:39 GMT
Server:  Apache/2.4.7 (Ubuntu)
content-length:  2
Keep-Alive:  timeout=5, max=86
Connection:  Keep-Alive
Content-Type:application/json; charset=UTF-8
JSON

   [m:Auto]
[]

We don't get any data, but there is data with a bigger 

Re: [openstack-dev] [gnocchi][rating] Issues regarding gnocchi support in CloudKitty

2015-11-26 Thread Stéphane Albert
On Thu, Nov 26, 2015 at 05:04:02PM +0100, Julien Danjou wrote:
> On Thu, Nov 26 2015, Stéphane Albert wrote:
> 
> Here a first reply about resource handling.
> 
> > Resource data
> > =
> > Duplicated resource informations from ceilometer, only revision timestamps
> > changing.
> 
> So I guess that's a bug in Gnocchi, we should not create new revision if
> there is 0 change. Would you mind opening a bug?
I'll open a bug with a full history dump, sometime it's due to the host
changing (flapping) from name to hash, this might be due to ceilometer.
But most of the time there is no change.
> 
> > Search for active instances during a timeframe
> > ==
> 
> […]
> 
> > Here the revision is outside of the requested timeframe.
> 
> I don't get that comment. You didn't ask for any specific revision, you
> asked for a start/end timestamps. So that looks correct to me. You get
> the instances that was active between 10:33 and 17:33.
I didn't ask for a specific revision because I can't ;)
I get the list of instances active from 10:33 to 17:33, but with the
latest metadata which can be 2 or 3 days from the timeframe requested.
When you are doing rating based on metadata it's problematic.
> 
> > Same request with a filter on the revision
> > ==
> 
> […]
> 
> > Empty response because the filter is not matching with the latest resource
> > revision.
> 
> Yes, that's normal too.
> 
> Revision are about resources modification.
> You don't need to search based on revision if you want to retrieve
> active resources during a timeframe. Just started_at/ended_at.
See above.

> 
> > Workaround
> > ==
> > Search for every resource of type 'instances' active during the timeframe. 
> > The
> > generic request is just to reduce the amount of data transfered as its
> > useless.
> 
> I don't see what you are working around. You get _exactly_ the same
> result that you got with "Search for active instances during a
> timeframe" so what's the problem in the first place?
I do this to get the id of active instances. So I can then query gnocchi
for the correct revision filtering on the id.
> 
> > Request the correct revision from the resource_id we found before.
> 
> I don't understand what you call a "correct revision".
The correct revision is the revision matching the timeframe (10:33 to
17:33). So I can get the metadata that were applying to the instance in
this timeframe.
> 
> If what you want is the list of active resource during a timeframe and
> their revision within that timeframe, you can just do:
> 
> POST http://10.8.8.168:8041/v1/search/resource/instance?history=true
> {
> "and": [
> {
> "or": [
> {
> "=": {
> "ended_at": null
> }
> },
> {
> ">=": {
> "ended_at": "2015-11-23T10:33:26.388112+00:00"
> }
> }
> ]
> },
> {
> "or": [
> {
> "=": {
> "ended_at": null
> }
> },
> {
> "<=": {
> "ended_at": "2015-11-23T17:33:26.388112+00:00"
> }
> }
> ]
> },
> {
> "<=": {
> "started_at": "2015-11-23T17:33:26.388112+00:00"
> }
> },
> {
> "<=": {
> "revision_start": "2015-11-23T17:33:26.388112+00:00"
> }
> }
> ]
> }
If there is more than one revision, then I'll get multiple revisions for
a resource. This imply that you request all the revisions and then
filter them client side. It's not the most efficient way to proceed as
it could be directly done in the DB query.

Thanks for taking time to answer Julien.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [gnocchi][rating] Issues regarding gnocchi support in CloudKitty

2015-11-26 Thread Stéphane Albert
On Thu, Nov 26, 2015 at 05:31:38PM +0100, Julien Danjou wrote:
> On Thu, Nov 26 2015, Stéphane Albert wrote:
> 
> > I'll open a bug with a full history dump, sometime it's due to the host
> > changing (flapping) from name to hash, this might be due to
> > ceilometer.
> 
> This would be a Ceilometer (or above) bug indeed. I think I saw
> something recently. It's possible the host differs between polling and
> notifications; feel free to also open a bug on the Ceilometer side.
I'll see if there's a bug declared in ceilometer's bug tracker and if
not report a new one.
> 
> >> > Search for active instances during a timeframe
> >> > ==
> >> 
> >> […]
> >> 
> >> > Here the revision is outside of the requested timeframe.
> >> 
> >> I don't get that comment. You didn't ask for any specific revision, you
> >> asked for a start/end timestamps. So that looks correct to me. You get
> >> the instances that was active between 10:33 and 17:33.
> > I didn't ask for a specific revision because I can't ;)
> 
> Why can't you? You can request revisions based on their timeframes.
By "I can't" I meant I can't without asking for history. I just want to
have the same result as the standard query (the latest revision) but
with a maximum revision timestamp.
> Or is it because the revision date is based no the upload timestamp
> rather than the actual date of the revision?
> 
> This is something we could fix in the API and Ceilometer I imagine,
> giving the ability to provide a timestamp for the change.
> 
> > If there is more than one revision, then I'll get multiple revisions for
> > a resource. This imply that you request all the revisions and then
> > filter them client side. It's not the most efficient way to proceed as
> > it could be directly done in the DB query.
> 
> But but… there might be multiple revision between two timeframe. If you
> want to rate an instance that was up during 10:33 and 17:33, and that
> resource has been e.g. resized 3 times at 12:00, 13:00 and 15:00, you'll
> get 3 revisions from Gnocchi. Isn't that what you want?
Yes and no... In CloudKitty the collection period is our epsilon. So we
won't process multiple values for it. That's the way CloudKitty is
working at the moment, it might change in the future. We get the latest
metas and most of the time get max value from the aggregated measures
(vcpus, mem, disk, etc). It's just that at the moment ceilometer/gnocchi
is returning way too much revisions for a timeframe (and they don't have
relevant data most of the time). If we query all active instances for a
tenant we can quickly get huge load of data.

We'll have support for ceilometer events soon, so we'll start doing diff
on gnocchi resources to detect changes and apply rate on these. But at
the moment it's out of our scope.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [gnocchi][rating] Issues regarding gnocchi support in CloudKitty

2015-11-25 Thread Stéphane Albert
Hi,

We've started implementing gnocchi support in CloudKitty. But we're
facing some issues. Some problems where client sided and are now fixed,
but for some others it might doing wrong requests or not using gnocchi's
full potential.

Here are some problems we're actually facing:

We can't directly query gnocchi resource indexer for a specific resource
revision. Let me explain it, if you want to search for every instances
active for a specific timeframe you'll get a result but the revision
will be the latest, even if it's after the timeframe you requested. If
you add resource revision to the request you'll get an empty request if
it doesn't match the latest version. We can't add history to the request
as we'll have more than one response when we only need latest revision
for the timeframe. One workaround is to first do a request to query
every active instance and then do a query for every instance with
history filtering on UUID and revision and limiting to one result. It's
suboptimal as it requires many requests to get the correct information.


The other problem we are having is when requesting measures. Let me
first tell you how CloudKitty is collecting data to set the context.
We've got a collection period (default 3600 seconds), every period we
query the backend for a specific timeframe (start = last collection,
stop = last + period). If we're missing data we restart from where we
stopped, which can be pretty far in the past.
We're using gnocchi as ceilometer's storage and by default it's pretty
hard to exploit data stored. If we request data too far in the past that
is using a higher granularity than our collection period then we got an
empty dataset from gnocchi. It's pretty ambiguous as we don't know if
it's because there is no data, or because our timeframe is too short.
That means we need to query for the archive policy and find what should
be the minimum granularity for the period. In some way it's normal to no
get data because gnocchi is unable to provide accurate data for this
period, but at least get information that we are requesting too
precisely. Or send us the data, since we've got the timestamp and
the granularity, we can easily detect that the result is not accurate
and decide what we should do.

Sorry for the wall of text, and thanks to people that took time to read
it.

BTW, we're available in #cloudkitty channel. You can reach me on most
openstack channels too, I'm in UTC+1 timezone.

Hoping we can find a solution and ship gnocchi support in the next
CloudKitty version, which is pretty much ready.

Cheers

Stéphane Albert   -  05 82 95 65 36
Objectif Libre  www.objectif-libre.com
Infrastructure et Formations Linux

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [cloudkitty] Remove inactive contributors from cloudkitty-core

2015-11-24 Thread Stéphane Albert
Hi,

Some people in the cloudkitty-core group are not active anymore. They
haven't produced a single contribution during the last cycle. Here is
the list of the two people:

- Adrian Turjak
- François Magimel

I think we should remove them from the group. We've got new faces in the
project, let's get some space for them ready :).

Are you guys OK with this?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [rating] CloudKitty 23rd meeting agenda

2015-11-23 Thread Stéphane Albert
Hi,

I'm back from Japan, we'll start having regular meetings again.

Here's today's agenda:

* New contributions
* Gnocchi support
* Pending reviews
* Road to version 0.5
* Next version goals

During this meeting we need to iron out remaining bugs to prepare
release 0.5. As soon as this version is released we'll move forward and
transition to the Mitaka cycle.
I want to define every version goals and migration/update possibilities.
We'll deprecate old code and transition to new and robust one.

Feel free to join us at 2PM UTC today.

Cheers

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [cloudkitty] IRC meetings canceled for two weeks

2015-10-19 Thread Stéphane Albert
Hi,

The summit is coming and most people are already in Japan. This makes
the organisation of meetings pretty difficult.
We'll cancel this week's meeting and next weeks too as most of the
contributors will be in Tokyo talking CloudKitty's future.

We'll resume normal schedule after the summit.

Cheers

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [cloudkitty] Now that we are in the Big Tent what's the future?

2015-10-08 Thread Stéphane Albert
Hi everyone,

As you might have guessed based on the title CloudKitty is now part of
the Big Tent. It's not new as the decision was taken one week ago, but
some people might not be aware of this.

Thanks to our Big Tent integration we are lucky enough to get two design
summit slots.

The first slot is Wednesday, October 28 at 4:40pm. It's a working
session in the Tachibana room, which has a capacity of 8 people. Our
goal is to plan the future support of gnocchi in CloudKitty. Since some
internal parts are subject to change in the next cycle we want to be
sure it will fit perfectly with gnocchi to minimize future efforts. If
you are working on gnocchi or willing to help us on that, join us.

The second slot is Thursday, October 29 at 5:20pm. It's a fishbowl
session in the Kotobuki room, which is way bigger and can accommodate 55
people. We will review the pending blueprints, discuss of future changes
and the scope of this cycle. We would love to have people with new
ideas, use cases or wanting to help us on this project.
As we can have way more people in the room, feel free to join us and
discuss with us the future of the project.

We'll plan agendas for the design summit during the next IRC meeting. If
you want to add specific points join us next Monday (12th) at 14:00 UTC.

Cheers

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [cloudkitty] No IRC meeting today

2015-10-05 Thread Stéphane Albert
Hi,

Most of the people involved in the project won't be able to attend
today's meeting. Meeting is canceled.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev