[openstack-dev] Concerns about Gnocchi's performance

2015-01-15 Thread 张灿
Hello,

I’m testing gnocchi to work with ceilometer these days and I have some concerns 
about its performance.

I noticed in the code, every time a measure gets updated, gnocchi first reads 
all the previous contents of that measure, updates the measure in memory and 
writes the updated contents as well as original contents back. IMHO, the 
overhead here could be very large. As an improvement, I propose we use some 
form of `seek` that only modifies the updated parts.

So I’d like to hear about your opinions, and if there’s already a solution, 
please point it to me.



Regards,
Can Zhang


signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Concerns about Gnocchi's performance

2015-01-15 Thread Julien Danjou
On Thu, Jan 15 2015, 张灿 wrote:

Hi,

 I noticed in the code, every time a measure gets updated, gnocchi first reads
 all the previous contents of that measure, updates the measure in memory and
 writes the updated contents as well as original contents back. IMHO, the
 overhead here could be very large. As an improvement, I propose we use some
 form of `seek` that only modifies the updated parts.

Yep, that's the main downside of the use of the Carbonara based backend
in Gnocchi. But honestly, since it's download/upload are parallel and
the file size is quite small, sending a metric should be pretty fast in
most circumstances. Though that's sure it's gonna be faster with
backends where you just have to send the actual metrics (OpenTSDB or
InfluxDB likely).

 So I’d like to hear about your opinions, and if there’s already a
 solution, please point it to me.

I'd love to improve the backends you describe with some sort of `seek'
solution, unfortunately I don't think it'll be possible to do so using
Swift. Though it might be possible to do so using Ceph and the file
based backends, so maybe it'd be worth looking at that. Any help
appreciated!

-- 
Julien Danjou
# Free Software hacker
# http://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Concerns about Gnocchi's performance

2015-01-15 Thread 张灿

 On Jan 15, 2015, at 18:46, Julien Danjou jul...@danjou.info wrote:
 
 Yep, that's the main downside of the use of the Carbonara based backend
 in Gnocchi. But honestly, since it's download/upload are parallel and
 the file size is quite small, sending a metric should be pretty fast in
 most circumstances. Though that's sure it's gonna be faster with
 backends where you just have to send the actual metrics (OpenTSDB or
 InfluxDB likely).
 

As a further question, how do you estimate the storage impact of a vm 
instance’s metrics? I’m not very sure about the `back_window` and `definition` 
part, could you clarify that for me?




Regards,
Can Zhang


signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Concerns about Gnocchi's performance

2015-01-15 Thread Julien Danjou
On Thu, Jan 15 2015, 张灿 wrote:

 As a further question, how do you estimate the storage impact of a vm
 instance’s metrics? I’m not very sure about the `back_window` and `definition`
 part, could you clarify that for me?

That defines the number of measure (data points) that are stored for
each metric. The size of an actual metric depends on these parameters.
So depending on the granularity and the number of points, the size of a
metric can go from a few kilobytes to a few megabytes.

-- 
Julien Danjou
-- Free Software hacker
-- http://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev