Re: [openstack-dev] [all] The future of the integrated release

[Nadya Privalova]

Hello all,

As a Ceilometer's core, I'd like to add my 0.02$.

During previous discussions it was mentioned several projects which were
started or continue to be developed after Ceilometer became integrated. The
main question I'm thinking of is why it was impossible to contribute into
existing integrated project? Is it because of Ceilometer's architecture,
the team or there are some other (maybe political) reasons? I think it's a
very sad situation when we have 3-4 Ceilometer-like projects from different
companies instead of the only one that satisfies everybody. (We don't see
it in other projects. Though, maybe there are several Novas os Neutrons on
StackForge and I don't know about it...)
Of course, sometimes it's much easier to start the project from scratch.
But there should be strong reasons for doing this if we are talking about
integrated project.
IMHO the idea, the role is the most important thing when we are talking
about integrated project. And if Ceilometer's role is really needed (and I
think it is) then we should improve existing implementation, "merge" all
needs into the one project and the result will be still Ceilometer.

Thanks,
Nadya


On Fri, Aug 15, 2014 at 12:41 AM, Joe Gordon  wrote:

>
>
>
> On Wed, Aug 13, 2014 at 12:24 PM, Doug Hellmann 
> wrote:
>
>>
>> On Aug 13, 2014, at 3:05 PM, Eoghan Glynn  wrote:
>>
>> >
>> >>> At the end of the day, that's probably going to mean saying No to more
>> >>> things. Everytime I turn around everyone wants the TC to say No to
>> >>> things, just not to their particular thing. :) Which is human nature.
>> >>> But I think if we don't start saying No to more things we're going to
>> >>> end up with a pile of mud that no one is happy with.
>> >>
>> >> That we're being so abstract about all of this is frustrating. I get
>> >> that no-one wants to start a flamewar, but can someone be concrete
>> about
>> >> what they feel we should say 'no' to but are likely to say 'yes' to?
>> >>
>> >>
>> >> I'll bite, but please note this is a strawman.
>> >>
>> >> No:
>> >> * Accepting any more projects into incubation until we are comfortable
>> with
>> >> the state of things again
>> >> * Marconi
>> >> * Ceilometer
>> >
>> > Well -1 to that, obviously, from me.
>> >
>> > Ceilometer is on track to fully execute on the gap analysis coverage
>> > plan agreed with the TC at the outset of this cycle, and has an active
>> > plan in progress to address architectural debt.
>>
>> Yes, there seems to be an attitude among several people in the community
>> that the Ceilometer team denies that there are issues and refuses to work
>> on them. Neither of those things is the case from our perspective.
>>
>
> Totally agree.
>
>
>>
>> Can you be more specific about the shortcomings you see in the project
>> that aren’t being addressed?
>>
>
>
> Once again, this is just a strawman.
>
> I'm just not sure OpenStack has 'blessed' the best solution out there.
>
>
> https://wiki.openstack.org/wiki/Ceilometer/Graduation#Why_we_think_we.27re_ready
>
> "
>
>- Successfully passed the challenge of being adopted by 3 related
>projects which have agreed to join or use ceilometer:
>   - Synaps
>   - Healthnmon
>   - StackTach
>   
> 
>   "
>
>
> Stacktach seems to still be under active development (
> http://git.openstack.org/cgit/stackforge/stacktach/log/), is used by
> rackspace in production and from everything I hear is more mature then
> ceilometer.
>
>
>>
>> >
>> >> Divert all cross project efforts from the following projects so we can
>> focus
>> >> our cross project resources. Once we are in a bitter place we can
>> expand our
>> >> cross project resources to cover these again. This doesn't mean
>> removing
>> >> anything.
>> >> * Sahara
>> >> * Trove
>> >> * Tripleo
>> >
>> > You write as if cross-project efforts are both of fixed size and
>> > amenable to centralized command & control.
>> >
>> > Neither of which is actually the case, IMO.
>> >
>> > Additional cross-project resources can be ponied up by the large
>> > contributor companies, and existing cross-project resources are not
>> > necessarily divertable on command.
>>
>
> Sure additional cross-project resources can and need to be ponied up, but
> I am doubtful that will be enough.
>
>
>>
>> What “cross-project efforts” are we talking about? The liaison program in
>> Oslo has been a qualified success so far. Would it make sense to extend
>> that to other programs and say that each project needs at least one
>> designated QA, Infra, Doc, etc. contact?
>>
>> Doug
>>
>> >
>> >> Yes:
>> >> * All integrated projects that are not listed above
>> >
>> > And what of the other pending graduation request?
>> >
>> > Cheers,
>> > Eoghan
>>
>> >
>> > ___
>> > OpenStack-dev mailing list
>> > OpenStack-dev@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Complex resource_metadata could fail to store in MongoDB

[Nadya Privalova]

IMHO it's ok and even very natural to expect "escaped" query from users.
e.g, we store the following structure

{metadata:
{ Zoo:
   {Foo.Boo: ''value"}}}

and query should be "metadata.Zoo.Foo\.Boo" .
In this case it's not necessary to know deep of tree.

Thanks,
Nadya



On Fri, Aug 29, 2014 at 3:21 PM, Igor Degtiarov 
wrote:

> Hi, folks.
>
> I was interested in the problem with storing of samples, that contain
> complex resource_metadata, in MongoDB database [1].
>
> If data is a dict that has a  key(s) with dots (i.e. .), dollar signs
> (i.e. $), or null characters,
> it wouldn't be stored. It is happened because these characters are
> restricted to use in fields name in MongoDB [2], but so far there is no any
> verification of the metadata in ceilometers mongodb driver, as a result we
> will lose data.
>
> Solution of this problem seemed to be rather simple, before storing data
> we check keys in resourse_metadata, if it is a dict, and "quote" keys with
> restricted characters in a similar way, as it was done in a change request
> of redesign separators in columns in HBase [2]. After that store metering
> data.
>
> But other unexpected difficulties appear on the step of getting data. To
> get stored data we constructs a meta query, and structure of that query was
> chosen identical to initial query in MongoDB. So dots is used as a
> separator for three nods of stored data.
>
> Ex. If it is needed to check value in field "Foo"
>
> {metadata:
> { Zoo:
>{Foo: ''value"}}}
>
> query would be: "metadata.Zoo.Foo"
>
> We don't know how deep is dict in metadata, so it is impossible to propose
> any correct parsing of query, to "quote" field names contain dots.
>
> I see two way for improvements. First is rather complex and based of
> redesign structure of the metadata query in ceilometer. Don't know if it is
> ever possible.
>
> And second is based on removing from the samples "bad" resource_metadata.
> In this case we also lose metadata,  but save other metering data. Of
> course queries for not saved metadata will return nothing, so it is not
> complete solution, but some kind of the hook.
>
> What do you think about that?
> Any thoughts and propositions are kindly welcome.
>
> [1] https://bugs.launchpad.net/mos/+bug/1360240
> [2] http://docs.mongodb.org/manual/reference/limits/
> [3] https://review.openstack.org/#/c/106376/
>
> -- Igor Degtiarov
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Adding Nejc Saje to ceilometer-core

[Nadya Privalova]

I'm in :)
+1

On Thu, Sep 11, 2014 at 4:58 PM, gordon chung  wrote:

> > Nejc has been doing a great work and has been very helpful during the
>
> > Juno cycle and his help is very valuable.
>
> > I'd like to propose that we add Nejc Saje to the ceilometer-core group.
>
> can we minus because he makes me look bad? /sarcasm
>
> +1 for core.
>
> cheers,
> *gord*
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Adding Dina Belova to ceilometer-core

[Nadya Privalova]

May I be the first  :)? Big +1 from me. Thanks Dina!

On Thu, Sep 11, 2014 at 5:24 PM, Julien Danjou  wrote:

> Hi,
>
> Dina has been doing a great work and has been very helpful during the
> Juno cycle and her help is very valuable. She's been doing a lot of
> reviews and has been very active in our community.
>
> I'd like to propose that we add Dina Belova to the ceilometer-core
> group, as I'm convinced it'll help the project.
>
> Please, dear ceilometer-core members, reply with your votes!
>
> --
> Julien Danjou
> // Free Software hacker
> // http://julien.danjou.info
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] storage driver testing

[Nadya Privalova]

Hello Sandy,

I'm very interested in performance results for Ceilometer. Now we have
successfully installed Ceilometer in the HA-lab with 200 computes and 3
controllers. Now it works pretty good with MySQL. Our next steps are:

1. Configure alarms
2. Try to use Rally for OpenStack performance with MySQL and MongoDB (
https://wiki.openstack.org/wiki/Rally)

We are open to any suggestions.

Thanks,
Nadya



On Wed, Nov 27, 2013 at 9:42 PM, Sandy Walsh wrote:

> Hey!
>
> We've ballparked that we need to store a million events per day. To that
> end, we're flip-flopping between sql and no-sql solutions, hybrid solutions
> that include elastic search and other schemes. Seems every road we go down
> has some limitations. So, we've started working on test suite for load
> testing the ceilometer storage drivers. The intent is to have a common
> place to record our findings and compare with the efforts of others.
>
> There's an etherpad where we're tracking our results [1] and a test suite
> that we're building out [2]. The test suite works against a fork of
> ceilometer where we can keep our experimental storage driver tweaks [3].
>
> The test suite hits the storage drivers directly, bypassing the api, but
> still uses the ceilometer models. We've added support for dumping the
> results to statsd/graphite for charting of performance results in real-time.
>
> If you're interested in large scale deployments of ceilometer, we would
> welcome any assistance.
>
> Thanks!
> -Sandy
>
> [1] https://etherpad.openstack.org/p/ceilometer-data-store-scale-testing
> [2] https://github.com/rackerlabs/ceilometer-load-tests
> [3] https://github.com/rackerlabs/instrumented-ceilometer
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Nova] Tokens in memcache become unauthorized

[Nadya Privalova]

Hi guys,

I've faced with nova+memcache issue on the cluster in HA-mode.

Issue is related to nova in case of using REST (that includes Horizon):
it's impossible to use auth-token several times because it became
unauthorized in cache.

Logs:

Nov 13 06:58:49 controller-1461 nova keystoneclient.middleware.auth_token
DEBUG Token validation failure.
Traceback (most recent call last):
  File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 684, in _validate_u
ser_token
cached = self._cache_get(token_id)
  File
"/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 898, in _cache_get
raise InvalidUserToken('Token authorization failed')
InvalidUserToken: Token authorization failed
Nov 13 06:58:49 controller-1461 nova keystoneclient.middleware.auth_token
DEBUG Marking token 211b590c4ba94d62a3981fbf91e934dc as unauthorized in
memcache

Issue was fixed after the following was added to [keystone_authtoken]
section in nova.conf to all controller's nodes:
memcache_security_strategy=ENCRYPT
memcache_secret_key=any_key

But from docs I see that these configs are not required. The issue does not
appear with any other services but all of them have
memcache_security_strategy empty. So is it a nova-bug or for HA-mode I
should configure this params? May the issue be caused by the fact that
memcaches are not syncked on controllers?

Thanks,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] [Rally] Does Ceilometer affect instance creation?

[Nadya Privalova]

Hi guys,

I decided to forward this message to dev list too. On the previous week I
was investigating Ceilometer performance. And this letter is a brief
description of my results.

Lab description:
3 controllers
187 computes
HA: Galera for MySQL
memcached is on, RabbitMQ in HA mode

Ceilometer processes are running as follows:
1 controller: ceilometer-api, ceilometer-agent-central,
ceilometer-collector, ceilometer-agent-compute
2 and 3 controller: ceilometer-collector
all computes: ceilometer-agent-compute

My idea was just to see how Ceilometer works in real-life circumstances. I
decided to use Rally to see how much Ceilometer's work affects the speed of
instance booting and removing time. Rally is a new instrument for me and I
hope Rally's guys will fix me if I'm wrong in config description.

Scenario:
1. Test boot-time without Ceilometer. Just an empty Lab
2. Run manually 100 instances which will be always polled by Ceilometer in
next scenarios
3. Start Ceilometer with different "interval" in pipeline.conf in _all_
computes and controllers.
4. Use the following Rally config:
"NovaServers.boot_and_delete_server": [
{"args": {"flavor_id": 2, "image_id":
"430c55c4-d9f6-4fa9-9888-5ca1f9531530"},
 "execution": "continuous",
 "config": {"times": 400, "active_users": 100, "tenants": 10,
"users_per_tenant": 20}}
  ]
The config means that the method boot_and_delete_server (create one
instance and delete it at once) will be called 400 times. Max "level of
concurrency" is 100 i.e. 100 active users (as max) may create instance at
the same time. Amount of users = tenants*users_per_tenant = 200.

Rally measures how long method 'boot_and_delete_server' worked. And here is
a list of results:

+--+-+-++
|  max   |  avg  |  min   |
scenario   |
+--+-+-++
| 288.868812084 | 109.638720818 | 35.1642079353 |   empty lab   |
+-+--+-++
| 321.71191287   | 173.705494534 | 72.4606912136 |  20 sec polling  |
+-+--+-++
| 389.527067184 | 217.689281176 | 71.4955689907 | 10 sec polling  |
+-+--+-++
| 445.328452826 | 243.213246702 | 117.31561017   |  5 sec polling   |
+-+--+-++

The second interesting moment is that Ceilometer affects user creation very
much. Unfortunately, Rally doesn't measure this metric and I cannot provide
concrete results.

Guys, if you have any questions or comments you are welcome! I think that
2x difference between avg time in "empty lab" and "5 sec polling" scenario
is not a bad result. But 100 instances that were being monitored during the
test is not a real load for the lab. What do you think? Should I repeat the
test with 1000 instances?

Thanks,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] [Rally] Does Ceilometer affect instance creation?

[Nadya Privalova]

Julien,

Yes, I use the same SQL for Nova and Ceilometer. Thanks for pointing this
out. My bad, I didn't take it into account. So if we want to use Ceilometer
+ MySQL in production (in theory :) ) we need to use separate controllers
with Ceilometer's MySQL only. And each controller may run it's own
collector which will write data into "local" MySQL. Am I right that only
one instance of central-agent may be started (WIP
https://wiki.openstack.org/wiki/Ceilometer/blueprints/tasks-distribution)?
Please Julien correct me if I'm wrong. And maybe Ceilometer has
recommendations for production deployment and I just missed it?


On Tue, Dec 10, 2013 at 4:25 PM, Boris Pavlovic wrote:

> Nadya, Julien,
>
>
> We are working around profiling system based on logs. This will allows us
> to detect bottlenecks.
> We should make a couple of small patches in each project to support
> profiling.
>
> As we are going to be well integrated with OpenStack infrastructure we are
> going to use Ceilometer as a log collector.
> And we already made patch for this in Ceilometer:
> https://review.openstack.org/#/c/60262/
> So it will be nice to get it reviewed/merged.
>
>
> Thanks.
>
>
> Best regards,
> Boris Pavlovic
>
>
>
>
>
>
> On Tue, Dec 10, 2013 at 3:19 PM, Julien Danjou  wrote:
>
>> On Tue, Dec 10 2013, Nadya Privalova wrote:
>>
>> Hi Nadya,
>>
>> > Guys, if you have any questions or comments you are welcome! I think
>> that
>> > 2x difference between avg time in "empty lab" and "5 sec polling"
>> scenario
>> > is not a bad result. But 100 instances that were being monitored during
>> the
>> > test is not a real load for the lab. What do you think? Should I repeat
>> the
>> > test with 1000 instances?
>>
>> You didn't mention where you were storing the metrics. If you store them
>> in the same MySQL DB that's used by Nova for example, it's likely that's
>> the problem is the load Ceilometer puts on the MySQL cluster slows Nova
>> down.
>>
>> I don't think it's worth running the test with more instances for now.
>> The results are clear, the next action should be to see why things are
>> slowed down that much. It shouldn't happen.
>>
>> --
>> Julien Danjou
>> # Free Software hacker # independent consultant
>> # http://julien.danjou.info
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer][Glance][Oslo]Notification issue with Glance

[Nadya Privalova]

Hello, guys!

We faced with Glance notifications issue during Tempest tests for
Ceilometer. We tried to send notification by ourselves (during
investigation we've found that Glance uses almost the same code):

from oslo.config import cfg
from oslo import messaging


CONF.rabbit_host = 'localhost'
CONF.rabbit_port = 5672
CONF.rabbit_use_ssl = False
CONF.rabbit_userid = 'guest'
CONF.rabbit_password = 'guest'
CONF.rabbit_virtual_host = '/'
CONF.rabbit_notification_exchange = 'glance'
CONF.rabbit_notification_topic = 'notifications'
CONF.rabbit_durable_queues = False
CONF.notification_driver = 'rabbit'

tr = messaging.get_transport(CONF, 'rabbit://')
n = messaging.Notifier(tr, 'messaging', 'image.localhost')
n.info({},'image.hello_world', 'Hello, World!')

And no messages were detected in Rabbit. And no ERRORs.
Please advice, maybe this issue is known?

Thanks for any help,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] [Rally] Does Ceilometer affect instance creation?

[Nadya Privalova]

Doug,

Sorry for confusing you with 'local' term. I meant that collector is up on
the node which is one of the Galera-nodes. Data will be replicated and all
the Galera nodes will be synced.

Nadya


On Thu, Dec 12, 2013 at 2:01 AM, Doug Hellmann
wrote:

>
>
>
> On Tue, Dec 10, 2013 at 9:47 AM, Nadya Privalova 
> wrote:
>
>> Julien,
>>
>> Yes, I use the same SQL for Nova and Ceilometer. Thanks for pointing this
>> out. My bad, I didn't take it into account. So if we want to use Ceilometer
>> + MySQL in production (in theory :) ) we need to use separate controllers
>> with Ceilometer's MySQL only. And each controller may run it's own
>> collector which will write data into "local" MySQL. Am I right that only
>> one instance of central-agent may be started (WIP
>> https://wiki.openstack.org/wiki/Ceilometer/blueprints/tasks-distribution)?
>> Please Julien correct me if I'm wrong. And maybe Ceilometer has
>> recommendations for production deployment and I just missed it?
>>
>
> You will want all of the ceilometer collectors writing to the same
> database, rather than having a local database for each one. Otherwise when
> you query the ceilometer API you won't see all of the results.
>
> Doug
>
>
>
>>
>>
>> On Tue, Dec 10, 2013 at 4:25 PM, Boris Pavlovic 
>> wrote:
>>
>>> Nadya, Julien,
>>>
>>>
>>> We are working around profiling system based on logs. This will allows
>>> us to detect bottlenecks.
>>> We should make a couple of small patches in each project to support
>>> profiling.
>>>
>>> As we are going to be well integrated with OpenStack infrastructure we
>>> are going to use Ceilometer as a log collector.
>>> And we already made patch for this in Ceilometer:
>>> https://review.openstack.org/#/c/60262/
>>> So it will be nice to get it reviewed/merged.
>>>
>>>
>>> Thanks.
>>>
>>>
>>> Best regards,
>>> Boris Pavlovic
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Dec 10, 2013 at 3:19 PM, Julien Danjou wrote:
>>>
>>>> On Tue, Dec 10 2013, Nadya Privalova wrote:
>>>>
>>>> Hi Nadya,
>>>>
>>>> > Guys, if you have any questions or comments you are welcome! I think
>>>> that
>>>> > 2x difference between avg time in "empty lab" and "5 sec polling"
>>>> scenario
>>>> > is not a bad result. But 100 instances that were being monitored
>>>> during the
>>>> > test is not a real load for the lab. What do you think? Should I
>>>> repeat the
>>>> > test with 1000 instances?
>>>>
>>>> You didn't mention where you were storing the metrics. If you store them
>>>> in the same MySQL DB that's used by Nova for example, it's likely that's
>>>> the problem is the load Ceilometer puts on the MySQL cluster slows Nova
>>>> down.
>>>>
>>>> I don't think it's worth running the test with more instances for now.
>>>> The results are clear, the next action should be to see why things are
>>>> slowed down that much. It shouldn't happen.
>>>>
>>>> --
>>>> Julien Danjou
>>>> # Free Software hacker # independent consultant
>>>> # http://julien.danjou.info
>>>>
>>>> ___
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev@lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa][ceilometer] Who can be a asked to help review tempest tests?

[Nadya Privalova]

Hi David,

I'm working on tempest tests for Ceilometer too.
I think this thread is a good place to make a reminder about our strategy
how to avoid duplications in change requests.

1. We have something like a test plan
https://etherpad.openstack.org/p/ceilometer-test-plan
2. Ceilometer team's decided that it is easier to keep all changes in one
bp instead of creation a new one for each change. The main bp is here
https://blueprints.launchpad.net/tempest/+spec/add-basic-ceilometer-tests

Please if you are about to start contributing to Ceilometer's tempest
please add info to any of these resources.

Thanks,
Nadya


On Tue, Dec 17, 2013 at 4:11 AM, David Kranz  wrote:

>  On 12/16/2013 05:50 PM, Doug Hellmann wrote:
>
>  It might be good, to start out, if we all look at them. That way we can
> all learn a bit about tempest, too. If you add "ceilometer-core" as a
> reviewer gerrit will expand the group name.
>
>  Doug
>
> Thanks, Doug. That makes sense.  The only reason I asked this was because
> I didn't know it was possible to ask all of you like that!
>
>  -David
>
>
>
> On Mon, Dec 16, 2013 at 5:33 PM, David Kranz  wrote:
>
>> Ceilometer team, we are reviewing tempest tests and hope to see more. The
>> tempest review team is hoping to identify some ceilometer devs who could
>> help answer questions or provide a review if needed for ceilometer patches.
>> Since ceilometer is new we are not all familiar with many of the details.
>> Can any one on the ceilometer team volunteer?
>>
>>  -David
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> ___
> OpenStack-dev mailing 
> listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer][Oslo] Consuming Notifications in Batches

[Nadya Privalova]

Hi John,

As for me your ideas look very interesting. As I understood notification
messages will be kept in MQ for some time (during batch-basket is being
filled), right? I'm concerned about the additional load that will be on MQ
(Rabbit).

Thanks,
Nadya


On Fri, Dec 20, 2013 at 3:31 AM, Herndon, John Luke wrote:

> Hi Folks,
>
> The Rackspace-HP team has been putting a lot of effort into performance
> testing event collection in the ceilometer storage drivers[0]. Based on
> some results of this testing, we would like to support batch consumption
> of notifications, as it will greatly improve insertion performance. Batch
> consumption in this case means waiting for a certain number of
> notifications to arrive before sending to the storage
> driver.
>
> I¹d like to get feedback from the community about this feature, and how we
> are planning to implement it. Here is what I’m currently thinking:
>
> 1) This seems to fit well into oslo.messaging - batching may be a feature
> that other projects will find useful. After reviewing the changes that
> sileht has been working on in oslo.messaging, I think the right way to
> start off is to create a new executor that builds up a batch of
> notifications, and sends the batch to the dispatcher. We’d also add a
> timeout, so if a certain amount of time passes and the batch isn’t filled
> up, the notifications will be dispatched anyway. I’ve started a
> blueprint for this change and am filling in the details as I go along [1].
>
> 2) In ceilometer, initialize the notification listener with the batch
> executor instead of the eventlet executor (this should probably be
> configurable)[2]. We can then send the entire batch of notifications to
> the storage driver to be processed as events, while maintaining the
> current method for converting notifications into samples.
>
> 3) Error handling becomes more difficult. The executor needs to know if
> any of the notifications should be requeued. I think the right way to
> solve this is to return a list of notifications to requeue from the
> handler. Any better ideas?
>
> Is this the right approach to take? I¹m not an oslo.messaging expert, so
> if there is a proper way to implement this change, I¹m all ears!
>
> Thanks, happy holidays!
> -john
>
> 0: https://etherpad.openstack.org/p/ceilometer-data-store-scale-testing
> 1:
> https://blueprints.launchpad.net/oslo.messaging/+spec/bulk-consume-messages
> 2: https://blueprints.launchpad.net/ceilometer/+spec/use-bulk-notification
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [QA][Tempest][Ceilometer] Pollster's testing strategy

[Nadya Privalova]

Hi guys!

For QA and Tempest guys brief description of Ceilometer's pollstering.
Ceilometer has several agents that once in 'interval' asks Nova, Glance and
other services about their metrics. We need to test this functionality,
'Interval' is defined in pipeline.yaml file and is 10 minutes by default.

I'd like to discuss the strategy of pollster's testing in tempest. Now we
need to wait 10 min to test the correctness of pollsters' work, as I
understand it is not appropriate.
I see the following solutions here:
1. Add smth like 'if tempest then interval = 5 sec'. This change should go
to gating AFAIU
2. Add additional functionality in Ceilometer: run_all_pollsters_on_demand.
I think this may be useful not only in tempest.

All your comments will be highly appreciated,

Thanks,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][Tempest][Ceilometer] Pollster's testing strategy

[Nadya Privalova]

Thanks, Julien!

Will create blueprint on Monday and will start implementation.


On Fri, Dec 20, 2013 at 7:51 PM, Julien Danjou  wrote:

> On Fri, Dec 20 2013, Nadya Privalova wrote:
>
> Hi Nadya,
>
> > For QA and Tempest guys brief description of Ceilometer's pollstering.
> > Ceilometer has several agents that once in 'interval' asks Nova, Glance
> and
> > other services about their metrics. We need to test this functionality,
> > 'Interval' is defined in pipeline.yaml file and is 10 minutes by default.
> >
> > I'd like to discuss the strategy of pollster's testing in tempest. Now we
> > need to wait 10 min to test the correctness of pollsters' work, as I
> > understand it is not appropriate.
> > I see the following solutions here:
> > 1. Add smth like 'if tempest then interval = 5 sec'. This change should
> go
> > to gating AFAIU
> > 2. Add additional functionality in Ceilometer:
> run_all_pollsters_on_demand.
> > I think this may be useful not only in tempest.
>
> I think having 2. is a good idea. It could be used in any environment
> for debugging or having almost-real-time feedback.
>
> --
> Julien Danjou
> # Free Software hacker # independent consultant
> # http://julien.danjou.info
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer][Tempest]Tracking of blueprints and changes

[Nadya Privalova]

Hello guys!

I hope all of you are enjoying the holidays! But I'd like to raise a
Tempest question. Again. I hope this email will not be lost after vacations
:)
After the summit we decided to track all tests that are being created for
Ceilometer in Tempest here:
https://blueprints.launchpad.net/tempest/+spec/add-basic-ceilometer-tests.
Besides, we've created an etherpad page with a test plan
https://etherpad.openstack.org/p/ceilometer-test-plan.

But it turned out that it works very bad. Now we have at least 3 change
requests that have common functionality implemented. So we definitely need
more reliable mechanism for tracking any changes.
That's why I suggest to create a separate blueprint for each functionality.
E.g. "Ceilometer client for Tempest", "Notification testing" with several
bps that depend on it ("Swift notifications", "Glance notifications", "Nova
notifications") and so on. In future we may vary the detail level of
blueprints but now we need very detailed ones because different people have
started to work on e.g. notifications.
So there are the following action items:
1. Resolve all conflicts in changes that are on review now (see my comment
to https://review.openstack.org/#/c/39237/ patch set 21 for more details)
2. Create set of blueprints from the testplan we have
3. Add Tempest discussions to Ceilometer weekly meeting agenda (done)

I may take care of all the items above. I need only "ok" from PTLs and
Cores.
Anyway, we've started working on 1st item, because it is urgent. The second
one may be postponed due to holidays.

And one more important thing. Code review for Ceilometer tests in Tempest
is too slow. Some of change requests are created almost a half a year ago!
Ceilometer guys, please be informed. I think all of us are interested in
good tests.

Thank you for attention,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer][Tempest]Tracking of blueprints and changes

[Nadya Privalova]

Hello all,

Thank you for your suggestions about tracking bps. I'll create a
spreadsheet.

David,

1. https://review.openstack.org/#/c/55276/ is ready for review
2. We decided to move base.py to a separate change request:
https://review.openstack.org/#/c/64304/ to make it merged ASAP. But now I
see that reviewers want to see an example of usage. We didn't wanted to add
any examples there. There are several changes that are abandoned now but
will be restored soon with dependency on 64304 (
https://review.openstack.org/#/c/43481/,
https://review.openstack.org/#/c/46745/). What do you think is it OK to
keep 64304 as a separate one?
3. https://review.openstack.org/#/c/39237/ is In Progress because it should
be depended on 2.
4. Not sure why https://review.openstack.org/#/c/64299/ is a separate one.
It is a part of 1 now. I think it should be abandoned.

And about "ceilometer", "metering", "telemetry". Originally 'metering' was
used, but official was changed to 'telemetry'. I'm not the person who
resolved such questions but I guess that 'telemetry' is the final solution.
Anyway, I will discuss it with Julien.

Thanks,
Nadya


On Mon, Dec 30, 2013 at 11:53 PM, David Kranz  wrote:

>  On 12/27/2013 05:27 AM, Nadya Privalova wrote:
>
>   Hello guys!
>
>  I hope all of you are enjoying the holidays! But I'd like to raise a
> Tempest question. Again. I hope this email will not be lost after vacations
> :)
>  After the summit we decided to track all tests that are being created for
> Ceilometer in Tempest here:
> https://blueprints.launchpad.net/tempest/+spec/add-basic-ceilometer-tests.
> Besides, we've created an etherpad page with a test plan
> https://etherpad.openstack.org/p/ceilometer-test-plan.
>
>  But it turned out that it works very bad. Now we have at least 3 change
> requests that have common functionality implemented. So we definitely need
> more reliable mechanism for tracking any changes.
> That's why I suggest to create a separate blueprint for each
> functionality. E.g. "Ceilometer client for Tempest", "Notification testing"
> with several bps that depend on it ("Swift notifications", "Glance
> notifications", "Nova notifications") and so on. In future we may vary the
> detail level of blueprints but now we need very detailed ones because
> different people have started to work on e.g. notifications.
>  So there are the following action items:
>  1. Resolve all conflicts in changes that are on review now (see my
> comment to https://review.openstack.org/#/c/39237/ patch set 21 for more
> details)
>  2. Create set of blueprints from the testplan we have
>  3. Add Tempest discussions to Ceilometer weekly meeting agenda (done)
>
>  I may take care of all the items above. I need only "ok" from PTLs and
> Cores.
>  Anyway, we've started working on 1st item, because it is urgent. The
> second one may be postponed due to holidays.
>
>  And one more important thing. Code review for Ceilometer tests in Tempest
> is too slow. Some of change requests are created almost a half a year ago!
> Ceilometer guys, please be informed. I think all of us are interested in
> good tests.
>
>  Thank you for attention,
>  Nadya
>
>
>  So the tempest patches for ceilometer are still not a coherent set. Can
> you please mark anything that is not ready for review as Work In Progress,
> or abandon until there is really something to review?
>
> Also, having looked at a few of these, I am confused about the usage of
> "ceilometer", "metering", "telemetry". Is there an explanation for the
> context in which each of these terms is to be used?
>
>  -David
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] Aggregation discussion

[Nadya Privalova]

Hi team,

I've decided to move discussion about aggregation in mailing list.
Here is a description about my idea and I really need your comments.

*Idea:*
The goal is to improve performance when user gets statistics for meter. Now
we have fixed list of statistics (min, max and so on). During request a
user may specify the following params:
1. query
2. group_by
3. period

The idea of bp is to pre-calculate some kind of requests and store them to
a separate table in database.
The pre-calculated statistics is called aggregates. Aggregates may be
merged among each others and with any Statistics' objects.
Note, that aggregates will be transparent for users. No changes in api is
required during get_statistics.

Example:
Let's assume we have 6 Samples about 'image' meter. All of them belong to
one day (e.g. 1st May) but have happened in different times:
11.50, 12.25, 12.50, 13.25, 13.50 and 14.25.  User would like to get
statistics about this meter from start = 11.30 till end = 14.30. So we need
to process all samples.
But we may process these samples earlier and already have pre-calculated
results for full hour 12.00 and 13.00. In this case we may get  Sample
11.50 and 14.25 from "meters" table and merge statistics for them with
already calculated Statistic result from "aggregates" table.
This example "saved" only 2 reads from DB. But if we consider metrics from
pollsters with interval = 5 sec (720 Samples per hour) we will save 719
reads with aggregate usage.

*Limitations: *
Of course we cannot aggregate data for all periods, group_by's and queries.
But we may allow user to configure what queries and group_by's he or she is
interested in. For instance, it may be useful for UI where we show graph
with statistics for each hour.  I think that period should not be
configurable, period may be only hour and day.

Example of entries in db:
 image_9223372035472681807  column=H:avg,
timestamp=1389255460712, value=1.0   (I will not copy all columns. The list
of columns is [column=H:min, column=H:max, column=H:sum and so on])
Example of filtered_aggregates in db (filter is "image by project"):
image_project_8c62fb0cd16c41498245095761b1a263_9223372035472681807
column=H:avg, timestamp=1389255460712, value=1.0


More details here: https://etherpad.openstack.org/p/ceilometer-aggregation
Draft implementation for HBase is here:
https://review.openstack.org/#/c/65681/1

Thanks for your attention,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Aggregation discussion

[Nadya Privalova]

Jay,

Thanks for comments!
The question you raised was discussed several times within Ceilometer team
but as I understand there is no official resolution yet.
I agree with you that statistics' collection is the main Ceilometer's goal.
But at the same time there should be a way to visualize the result of
Ceilometer's work.
My opinion is that the current set of data queries (get samples, get
statistics) is the minimum set and it's ok to keep it as a part of
Ceilometer's functionality. We need it at least for UI.

So, my proposal is to make this existing queries faster. Looks like our
vision are the same :)

Pre-calculate when? :) During processing of samples, or during some
> periodic job?


It should be a periodic job, right.

The term "aggregate" really just means a generic grouping or
> summarization. If you are looking for a term that represents the
> rules/heuristics for maintaining rolling calculations, perhaps the term
> "report" is better?
>

Hmm, I think that 'aggregate' is ok. In "my terminology" an aggregate is a
ready set of statistics for concrete meter, period and query. Anyway, will
think about it.

Thanks,
Nadya


On Mon, Jan 13, 2014 at 2:13 AM, Jay Pipes  wrote:

> On Fri, 2014-01-10 at 17:10 +0400, Nadya Privalova wrote:
> > Idea:
> > The goal is to improve performance when user gets statistics for
> > meter. Now we have fixed list of statistics (min, max and so on).
> > During request a user may specify the following params:
> > 1. query
> > 2. group_by
> > 3. period
> >
> > The idea of bp is to pre-calculate some kind of requests and store
> > them to a separate table in database.
>
> Pre-calculate when? :) During processing of samples, or during some
> periodic job?
>
> > The pre-calculated statistics is called aggregates.
>
> The term "aggregate" really just means a generic grouping or
> summarization. If you are looking for a term that represents the
> rules/heuristics for maintaining rolling calculations, perhaps the term
> "report" is better?
>
> > Aggregates may be merged among each others and with any Statistics'
> > objects.
> > Note, that aggregates will be transparent for users. No changes in api
> > is required during get_statistics.
> >
> > Example:
> > Let's assume we have 6 Samples about 'image' meter. All of them belong
> > to one day (e.g. 1st May) but have happened in different times:
> > 11.50, 12.25, 12.50, 13.25, 13.50 and 14.25.  User would like to get
> > statistics about this meter from start = 11.30 till end = 14.30. So we
> > need to process all samples.
> > But we may process these samples earlier and already have
> > pre-calculated results for full hour 12.00 and 13.00. In this case we
> > may get  Sample 11.50 and 14.25 from "meters" table and merge
> > statistics for them with already calculated Statistic result from
> > "aggregates" table.
> > This example "saved" only 2 reads from DB. But if we consider metrics
> > from pollsters with interval = 5 sec (720 Samples per hour) we will
> > save 719 reads with aggregate usage.
>
> Hmm. So, aggregation and grouping are the domain of data warehousing and
> OLAP Servers. I don't believe that putting this functionality directly
> in to Ceilometer is a good idea. I believe it would be better to
> delegate this kind of functionality to well-known and used tools like
> Pentaho [1], which can use a variety of different backend storage
> systems.
>
> Bottom line, I believe Ceilometer should focus strictly on the
> collection of samples/alarms, the pre-processing of those things, and
> the storage of those things, however I do not believe that Ceilometer
> should become an OLAP analytics tool when existing ones already fill
> that need.
>
> Best,
> -jay
>
> [1] http://www.pentaho.com/5.0 &
> http://en.wikipedia.org/wiki/Mondrian_OLAP_server
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel-dev] [OSTF][Ceilometer] ceilometer meters and samples delete

[Nadya Privalova]

Hi guys,

I would ask in another way.
Ceilometer has a mechanism to add a sample through POST. So it looks not
consistent not to allow user to delete a sample.
IMHO, insertion and deletion through REST looks a little bit hacky: user
always has an ability to fake data collected from OpenStack services. But
maybe I don't see any valuable usecases.
Anyway, it seems reasonable to have both add_sample and delete_sample in
API or not to have neither.

Thanks,
Nadya


On Fri, Jan 17, 2014 at 3:50 PM, Julien Danjou  wrote:

> On Fri, Jan 17 2014, Dmitry Iakunchikov wrote:
>
> > Ceilometer's tests creates some data which could not be deleted.
> > For example: after creation new instance, ceilometer creates new meters
> for
> > this instance and there  is no possibility to delete them. Even after
> > instance deletion they would not be deleted.
> >
> > Should we take attention to this? And find some way to delete all data,
> for
> > example database wipe or patch to ceilometer client...
> > Or just to skip this?
>
> Why would you delete this?
> There's a time-to-live mechanism in Ceilometer to delete old samples.
>
> --
> Julien Danjou
> // Free Software hacker / independent consultant
> // http://julien.danjou.info
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer]Collector's performance

[Nadya Privalova]

Hi team!

Last week we were working on notification problem in ceilometer during
tempest tests creation. Tests for notification passed successfully on
Postgres but failed on MySQL. This made us start investigations and this
email contains some results.

As it turned out, tempest as it is is something like performance-testing
for Ceilometer. It contains 2057 tests. Almost in all test OpenStack
resources are being created and deleted: images, instances, volumes. E.g.
during instance creation nova sends 9 notifications. And all the tests are
running in parallel for about 40 minutes.
>From ceilometer-collector logs we may found very useful message:

2014-03-10 09:42:41.356

22845 DEBUG ceilometer.dispatcher.database
[req-16ea95c5-6454-407a-9c64-94d5ef900c9e - - - - -] metering data
storage.objects.outgoing.bytes for b7a490322e65422cb1129b13b49020e6 @
2014-03-10T09:34:31.090107:

So collector starts to process_metering_data in dispatcher only in 9:42 but
nova sent it in 9:34. To look at whole picture please take look at picture
[1]. It illustrates time difference based on this message in logs.
Besides, I decided to take a look on difference between the RPC-publisher
sends the message and the collector receives the message. To create this
plot I've parsed the lines like below from anotifications log:

2014-03-10 09:25:49.333

22833 DEBUG ceilometer.openstack.common.rpc.amqp [-] UNIQUE_ID is
683dd3f130534b9fbb5606aef862b83d.


After that I found the corresponding id in collector log:

2014-03-10 09:25:49.352

22845 DEBUG ceilometer.openstack.common.rpc.amqp [-] received
{u'_context_domain': None, u'_context_request_id':
u'req-0a5fafe6-e097-4f90-a68a-a91da1cff22c',

u'args': {u'data': [...,
 u'message_id': u'f7ad63fc-a835-11e3-8223-bc764e205385',
u'counter_type': u'gauge'}]}, u'_context_read_only': False,
u'_unique_id': u'683dd3f130534b9fbb5606aef862b83d',

u'_context_user_identity': u'- - - - -', u'_context_instance_uuid':
None, u'_context_show_deleted': False, u'_context_tenant': None,
u'_context_auth_token': '',

} _safe_log
/opt/stack/new/ceilometer/ceilometer/openstack/common/rpc/common.py:280

So in the example above we see time-difference only in 20 milliseconds. But
it grows very quickly :( To see it please take a look on picture [2].

To summarize pictures:
1. Picture 1: Axis Y: amount of seconds between nova creates notification
and the collector retrieves the message. Axis X: timestamp
2. Picture 2: Axis Y: amount of seconds between the publisher publishes the
message and the collector retrieves the message. Axis X: timestamp

These pictures are almost the same and it makes me think that collector
cannot manage with big amount of messages. What do you think about it? Do
you agree or you need more evidences, e.g. amount of messages in rabbit or
amth else?
Let's discuss that in [Ceilometer] topic first, I will create a new thread
about testing strategy in tempest later. Because in this circumstances we
forced to refuse from created notification tests and cannot reduce time for
polling because it will make everything even worst.

[1]: http://postimg.org/image/r4501bdyb/
[2]: http://postimg.org/image/yy5a1ste1/

Thanks for your attention,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer]Collector's performance

[Nadya Privalova]

Ildiko,

Thanks for question, I forgot to write about it. The results for mysql, the
link to logs
http://logs.openstack.org/36/64136/20/check/check-tempest-dsvm-full/e361520/.
But I guess postgress stuff looks the same because it failed during last
test run (https://review.openstack.org/#/c/64136/). Will check tomorrow
anyway.

Nadya


On Tue, Mar 11, 2014 at 10:01 PM, Ildikó Váncsa
wrote:

>  Hi Nadya,
>
>
>
> You mentioned multiple DB backends in your mail. Which one did you use to
> perform these tests or did you get the same/similar performance results in
> case of both?
>
>
>
> Best Regards,
>
> Ildiko
>
>
>
> *From:* Nadya Privalova [mailto:nprival...@mirantis.com]
> *Sent:* Tuesday, March 11, 2014 6:05 PM
> *To:* OpenStack Development Mailing List
> *Subject:* [openstack-dev] [Ceilometer]Collector's performance
>
>
>
> Hi team!
>
> Last week we were working on notification problem in ceilometer during
> tempest tests creation. Tests for notification passed successfully on
> Postgres but failed on MySQL. This made us start investigations and this
> email contains some results.
>
> As it turned out, tempest as it is is something like performance-testing
> for Ceilometer. It contains 2057 tests. Almost in all test OpenStack
> resources are being created and deleted: images, instances, volumes. E.g.
> during instance creation nova sends 9 notifications. And all the tests are
> running in parallel for about 40 minutes.
>
> From ceilometer-collector logs we may found very useful message:
>
> 2014-03-10 09:42:41.356 
> <http://logs.openstack.org/36/64136/20/check/check-tempest-dsvm-full/e361520/logs/screen-ceilometer-collector.txt.gz#_2014-03-10_09_42_41_356>
>  22845 DEBUG ceilometer.dispatcher.database 
> [req-16ea95c5-6454-407a-9c64-94d5ef900c9e - - - - -] metering data 
> storage.objects.outgoing.bytes for b7a490322e65422cb1129b13b49020e6 @ 
> 2014-03-10T09:34:31.090107:
>
> So collector starts to process_metering_data in dispatcher only in 9:42
> but nova sent it in 9:34. To look at whole picture please take look at
> picture [1]. It illustrates time difference based on this message in logs.
>
> Besides, I decided to take a look on difference between the RPC-publisher
> sends the message and the collector receives the message. To create this
> plot I've parsed the lines like below from anotifications log:
>
>
>
> 2014-03-10 09:25:49.333 
> <http://logs.openstack.org/36/64136/20/check/check-tempest-dsvm-full/e361520/logs/screen-ceilometer-anotification.txt.gz#_2014-03-10_09_25_49_333>
>  22833 DEBUG ceilometer.openstack.common.rpc.amqp [-] UNIQUE_ID is 
> 683dd3f130534b9fbb5606aef862b83d.
>
>
>
>
>
>  After that I found the corresponding id in collector log:
>
> 2014-03-10 09:25:49.352 
> <http://logs.openstack.org/36/64136/20/check/check-tempest-dsvm-full/e361520/logs/screen-ceilometer-collector.txt.gz#_2014-03-10_09_25_49_352>
>  22845 DEBUG ceilometer.openstack.common.rpc.amqp [-] received 
> {u'_context_domain': None, u'_context_request_id': 
> u'req-0a5fafe6-e097-4f90-a68a-a91da1cff22c',
>
>
>
> u'args': {u'data': [...,
>  u'message_id': u'f7ad63fc-a835-11e3-8223-bc764e205385', u'counter_type': 
> u'gauge'}]}, u'_context_read_only': False, u'_unique_id': 
> u'683dd3f130534b9fbb5606aef862b83d',
>
>
>
> u'_context_user_identity': u'- - - - -', u'_context_instance_uuid': None, 
> u'_context_show_deleted': False, u'_context_tenant': None, 
> u'_context_auth_token': '',
>
>
>
> } _safe_log 
> /opt/stack/new/ceilometer/ceilometer/openstack/common/rpc/common.py:280
>
> So in the example above we see time-difference only in 20 milliseconds.
> But it grows very quickly :( To see it please take a look on picture [2].
>
> To summarize pictures:
>
> 1. Picture 1: Axis Y: amount of seconds between nova creates notification
> and the collector retrieves the message. Axis X: timestamp
>
> 2. Picture 2: Axis Y: amount of seconds between the publisher publishes
> the message and the collector retrieves the message. Axis X: timestamp
>
> These pictures are almost the same and it makes me think that collector
> cannot manage with big amount of messages. What do you think about it? Do
> you agree or you need more evidences, e.g. amount of messages in rabbit or
> amth else?
>
> Let's discuss that in [Ceilometer] topic first, I will create a new thread
> about testing strategy in tempest later. Because in this circumstances we
> forced to refuse from created notification te

[openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

[Nadya Privalova]

Hi folks,

I'd like to discuss Ceilometer's tempest situation with you.
Now we have several patch sets on review that test core functionality of
Ceilometer: notificaton and pollstering (topic
https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/add-basic-ceilometer-tests,n,z).
But there is a problem: Ceilometer performance is very poor on mysql and
postgresql because of the bug
https://bugs.launchpad.net/ceilometer/+bug/1291054. Mongo behaves much
better even in single thread and I hope that it's performance will be
enough to successfully run Ceilometer tempest tests.
Let me explain in several words why tempest tests is mostly performance
tests for Ceilometer. The thing is that Ceilometer service is running
during all other nova, cinder and so on tests run. All the tests create
instances, volumes and each creation produces a lot of notifications. Each
notification is the entry to database. So Ceilometer cannot process such a
big amount of notifications quickly. Ceilometer tests have 'telemetry'
prefix and it means that they will be started in the last turn. And it
makes situation even worst.
So my proposal:
1. create a non-voting job with Mongo-backend
2. make sure that tests pass on Mongo
3. merge tests to tempest but skip that on postgres and mysql till
bug/1291054 is resolved
4. make the new job 'voting'

The problem is only in Mongo installation. I have a cr
https://review.openstack.org/#/c/81001/ that will allow us to install Mongo
from deb. From the other hand there is
https://review.openstack.org/#/c/74889/ that enables UCA. I'm collaborating
with infra-team to make the decision ASAP because AFAIU we need tempest
tests in Icehouse (for more discussion you are welcome to thread
[openstack-dev] Updating libvirt in gate jobs).

If you have any thoughts on this please share.

Thanks for attention,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

[Nadya Privalova]

Ok, so we don't want to switch to UCA, let's consider this variant.
What options do we have to make possible to run Ceilometer jobs with Mongo
backend?
I see only  https://review.openstack.org/#/c/81001/ or making Ceilometer
able to work with old Mongo. But the last variant looks inappropriate at
least in Icehouse.
What am I missing here? Maybe there is smth else we can do?


On Tue, Mar 18, 2014 at 9:28 PM, Tim Bell  wrote:

>
>
> If UCA is required, what would be the upgrade path for a currently running
> OpenStack Havana site to Icehouse with this requirement ?
>
>
>
> Would it be an online upgrade (i.e. what order to upgrade the different
> components in order to keep things running at all times) ?
>
>
>
> Tim
>
>
>
> *From:* Chmouel Boudjnah [mailto:chmo...@enovance.com]
> *Sent:* 18 March 2014 17:58
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra]
> Ceilometer tempest testing in gate
>
>
>
>
>
> On Tue, Mar 18, 2014 at 5:21 PM, Sean Dague  wrote:
>
>  So I'm still -1 at the point in making UCA our default run environment
> until it's provably functional for a period of time. Because working
> around upstream distro breaks is no fun.
>
>
>
> I agree, if UCA is not very stable ATM, this os going to cause us more
> pain, but what would be the plan of action? a non-voting gate for
> ceilometer as a start ? (if that's possible).
>
> Chmouel
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

[Nadya Privalova]

Hi all,
First of all, thanks for your suggestions!

To summarize the discussions here:
1. We are not going to install Mongo (because "is's wrong" ?)
2. Idea about spawning several collectors is suspicious (btw there is a
patch that run several collectors: https://review.openstack.org/#/c/79962/.)

Let's try to get back to original problem. All these solutions were
suggested to solve the problem of high load on Ceilometer. AFAIK, Tempest's
goal is to test projects` interactions, not performance testing. The
perfect tempest's behaviour would be "start ceilometer only for Ceilometer
tests". From one hand it will allow not to load db during other tests, from
the other hand "projects` interactions" will be tested because during
Ceilometer test we create volums, images and instances. But I'm afraid that
this scenario is not possible technically.
There is one more idea. Make Ceilometer able to monitor not all messages
but filtered set of messages. But anyway this is a new feature and cannot
be added right now.

Tempest guys, if you have any thoughts about first suggestion "start
ceilometer only for Ceilometer tests" please share.

Thanks,
Nadya




On Thu, Mar 20, 2014 at 3:23 AM, Sean Dague  wrote:

> On 03/19/2014 06:09 PM, Doug Hellmann wrote:
> > The ceilometer collector is meant to scale horizontally. Have you tried
> > configuring the test environment to run more than one copy, to process
> > the notifications more quickly?
>
> The ceilometer collector is already one of the top running processes on
> the box -
>
> http://logs.openstack.org/82/81282/2/check/check-tempest-dsvm-full/693dc3b/logs/dstat.txt.gz
>
>
> Often consuming > 1/2 a core (25% == 1 core in that run, as can been
> seen when qemu boots and pegs one).
>
> So while we could spin up more collectors, I think it's unreasonable
> that the majority of our cpu has to be handed over to the metric
> collector to make it function responsively. I thought the design point
> was that this was low impact.
>
> -Sean
>
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

[Nadya Privalova]

Sean, thank for analysis.
JFYI, I did some initial profiling, it's described here
https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg19030.html.


On Thu, Mar 20, 2014 at 2:15 PM, Sean Dague  wrote:

> On 03/20/2014 05:49 AM, Nadya Privalova wrote:
> > Hi all,
> > First of all, thanks for your suggestions!
> >
> > To summarize the discussions here:
> > 1. We are not going to install Mongo (because "is's wrong" ?)
>
> We are not going to install Mongo "not from base distribution", because
> we don't do that for things that aren't python. Our assumption is
> dependent services come from the base OS.
>
> That being said, being an integrated project means you have to be able
> to function, sanely, on an sqla backend, as that will always be part of
> your gate.
>
> > 2. Idea about spawning several collectors is suspicious (btw there is a
> > patch that run several collectors:
> > https://review.openstack.org/#/c/79962/ .)
>
> Correct, given that the collector is already one of the most expensive
> processes in a devstack run.
>
> > Let's try to get back to original problem. All these solutions were
> > suggested to solve the problem of high load on Ceilometer. AFAIK,
> > Tempest's goal is to test projects` interactions, not performance
> > testing. The perfect tempest's behaviour would be "start ceilometer only
> > for Ceilometer tests". From one hand it will allow not to load db during
> > other tests, from the other hand "projects` interactions" will be tested
> > because during Ceilometer test we create volums, images and instances.
> > But I'm afraid that this scenario is not possible technically.
> > There is one more idea. Make Ceilometer able to monitor not all messages
> > but filtered set of messages. But anyway this is a new feature and
> > cannot be added right now.
> >
> > Tempest guys, if you have any thoughts about first suggestion "start
> > ceilometer only for Ceilometer tests" please share.
>
> The point of the gate is that it's integrated and testing the
> interaction between projects. Ceilometer can be tested on it's own in
> ceilometer unit tests, or by creating ceilometer functional tests that
> only run on the ceilometer jobs.
>
> While I agree that Tempest's job is not to test performance, we do have
> to give some basic sanity checking here that the software is running in
> some performance profile that we believe is base usable.
>
> Based on the latest dstat results, I think that's a dubious assessment.
> The answer on the collector side has to be something other than
> horizontal scaling. Because we're talking about the collector being the
> 3rd highest utilized process on the box right now (we should write a
> dstat plugin to give us cumulative data, just haven't gotten there yet).
>
> So right now, I think performance analysis for ceilometer on sqla is
> important, really important. Not just horizontal scaling, but actual
> performance profiling.
>
> -Sean
>
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

[Nadya Privalova]

Tim, yep. If you use one db for Ceilometer and Nova then nova's performance
may be affected. I've seen this issue.
Will start profiling ASAP.


On Thu, Mar 20, 2014 at 3:59 PM, Tim Bell  wrote:

>
> +1 for performance analysis to understand what needs to be optimised.
> Metering should be light-weight.
>
> For those of us running in production, we don't have an option to turn
> ceilometer off some of the time. That we are not able to run through the
> gate tests hints that there are optimisations that are needed.
>
> For example, turning on ceilometer caused a 16x increase in our Nova API
> call rate, see
> http://openstack-in-production.blogspot.ch/2014/03/cern-cloud-architecture-update-for.htmlfor
>  details.
>
> Tim
>
> > -Original Message-
> > From: Sean Dague [mailto:s...@dague.net]
> > Sent: 20 March 2014 11:16
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer
> tempest testing in gate
> >
> ...
> >
> > While I agree that Tempest's job is not to test performance, we do have
> to give some basic sanity checking here that the software is running in some
> > performance profile that we believe is base usable.
> >
> > Based on the latest dstat results, I think that's a dubious assessment.
> > The answer on the collector side has to be something other than
> horizontal scaling. Because we're talking about the collector being the 3rd
> highest
> > utilized process on the box right now (we should write a dstat plugin to
> give us cumulative data, just haven't gotten there yet).
> >
> > So right now, I think performance analysis for ceilometer on sqla is
> important, really important. Not just horizontal scaling, but actual
> > performance profiling.
> >
> >   -Sean
> >
> > --
> > Sean Dague
> > Samsung Research America
> > s...@dague.net / sean.da...@samsung.com
> > http://dague.net
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Dependency freeze exception for happybase (I would like version 0.8)

[Nadya Privalova]

Hi folks,

Running rests against 0.8. Will update you ASAP

Thanks,
Nadya


On Fri, Mar 28, 2014 at 1:39 PM, Thierry Carrez wrote:

> Thomas Goirand wrote:
> > I'd like to ask everyone's opinion here. Is it ok to do a freeze
> > exception in this case? If yes (please, everyone, agree! :) ), then
> > would >=0.8 or >=0.4,!=0.6,!=0.7 be better?
>
> At this point I think it's safest to go with ">=0.4,!=0.6,!=0.7", *if*
> Ceilometer folks confirm that 0.8 is fine by them. That way distros that
> are stuck with 0.5 are not otherwise adversely affected.
>
> --
> Thierry Carrez (ttx)
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Dependency freeze exception for happybase (I would like version 0.8)

[Nadya Privalova]

Today I've tested 0.6 and 0.8. They are acceptable for us. But 0.4 is not.
So I'd like to support Thomas's suggestion about freeze exception for
happybase.

Thanks, Nadya


On Fri, Mar 28, 2014 at 1:56 PM, Nadya Privalova wrote:

> Hi folks,
>
> Running rests against 0.8. Will update you ASAP
>
> Thanks,
> Nadya
>
>
> On Fri, Mar 28, 2014 at 1:39 PM, Thierry Carrez wrote:
>
>> Thomas Goirand wrote:
>> > I'd like to ask everyone's opinion here. Is it ok to do a freeze
>> > exception in this case? If yes (please, everyone, agree! :) ), then
>> > would >=0.8 or >=0.4,!=0.6,!=0.7 be better?
>>
>> At this point I think it's safest to go with ">=0.4,!=0.6,!=0.7", *if*
>> Ceilometer folks confirm that 0.8 is fine by them. That way distros that
>> are stuck with 0.5 are not otherwise adversely affected.
>>
>> --
>> Thierry Carrez (ttx)
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Dependency freeze exception for happybase (I would like version 0.8)

[Nadya Privalova]

Sean, please see my comment in bug regarding 0.7. Unfortunately I have
nothing to suggest here. If distributive contains only 0.7 we cannot work
with HBase through happybase. And I don't see any solutions.
But anyway if you change requirements for happybase (I guess we need to get
0.7 back, no?) could you please remove 0.4? 0.4 doesn't allow to work with
pools but we need to.

Thanks,
Nadya


On Fri, Mar 28, 2014 at 9:07 PM, Sean Dague  wrote:

> On 03/28/2014 12:50 PM, Thierry Carrez wrote:
> > Thierry Carrez wrote:
> >> Julien Danjou wrote:
> >>> On Thu, Mar 27 2014, Thomas Goirand wrote:
> >>>
>  -happybase>=0.4,<=0.6
>  +happybase>=0.8
> >>>
> >>> Good for me, and Ceilometer is the only one using happybase as far as I
> >>> know so that shouldn't be a problem.
> >>
> >> OK so I would be fine with "happybase>=0.4,!=0.6,!=0.7" as it allows 0.8
> >> to be run without adversely impacting downstreams who don't have it
> >> available.
> >
> > Actually that should be "happybase>=0.4,!=0.7" since 0.6 was allowed
> before.
>
> The review has comments about 0.7 being the version in Debian and Ubuntu
> right now https://review.openstack.org/#/c/82438/.
>
> Which is problematic, as that's a version that's specifically being
> called out by the ceilometer team as not useable. Do we know why, and if
> it can be worked around?
>
> -Sean
>
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Dependency freeze exception for happybase (I would like version 0.8)

[Nadya Privalova]

Sean,
The problem with 0.4 is a very new one. We had critical bug that was fixed
by ConnectionPool usage. This feature is available since 0.5. So we had two
options: 1. Go to rc1 with critical bug (not supported HBase in fact) 2.
Fix it but have incorrect dependency on 0.4. And we've chosen option 2.

I don't ask to fix dependency on 0.4 if the final decision is "not to
change requirements". I absolutely agree with you that it's too late. I
think it doesn't worth it. 0.4 is too old and unlikely to be used (anyway I
agree that it's still a problem). But if requirements is changed my plea is
to remove 0.4.

Jim,
Yes, it would help. But now I cannot imagine how we can put HBase on
gating. It is not in official repos. Anyway, I'm thinking about it and hope
that solution will be find soon.

Thanks,
Nadya




On Sat, Mar 29, 2014 at 3:31 AM, James E. Blair wrote:

> Sean Dague  writes:
>
> > So how did Ceilometer get into this situation? Because the ceilometer
> > requirements are happybase>=0.4,<=0.6
>
> Is this a case where testing minimums might have helped?
>
> -Jim
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-Infra][Ceilometer][MagnetoDB] HBase database in devstack

[Nadya Privalova]

Hi,
Yep, it would be great to have HBase installed on gating for Ceilometer.
Now we use self-written mocked HBase to test functionality. But HBase
backend is becoming more complex and it's really hard to add 'new features'
in mocked HBase. Hortonworks is the main and the largest contributor in
Hadoop eco-system so I think that their repos are very stable. But actually
both variants are acceptable for me.
I'd like to note that now Ceilometer doesn't work with Cassandra. But there
are several blueprints about it
https://blueprints.launchpad.net/ceilometer/+spec/cassandra-driver and
Magneto-related one
https://blueprints.launchpad.net/ceilometer/+spec/support-magnetodb . So
Cassandra will be important for Ceilometer on gating too.
Besides, I'd like to note that all these NoSQL solutions are very likely to
be used in production in future (if compare with SQL I mean). And I think
that all of us are interested in testing things that will be used in real
life.

Looking forward infra and devstack teams` inputs.

Thanks,
Nadya


On Tue, Apr 8, 2014 at 7:18 PM, Ilya Sviridov wrote:

> Hello infra and devstack,
>
>
> I would like to start thread about adding of nosql databases support to
> devstack for development and gating purposes.
>
> Currently there is necessity of HBase and Cassandra in MagnetoDB project
> for running tempest tests.
>
> We have implemented Cassandra as part of MagnetoDB devstack integration (
> https://github.com/stackforge/magnetodb/tree/master/contrib/devstack) and
> started working on HBase now (
> https://blueprints.launchpad.net/magnetodb/+spec/devstack-add-hbase).
>
> From other side, HBase and Cassandra are supported as database backends in
> Ceilometer and it can be useful for development and gating to have it in
> devstack.
>
> So, it looks like common task for both projects and eventually will be
> integrated to devstack, so I'm suggesting to start that discussion in order
> push ahead with it.
>
> Cassandra and HBase are both Java applications, so come with JDK as
> dependency. It is proved we can use OpenJDK available in debian repos.
>
> The database itself are distributed in two ways:
>
> - as debian packages build and hosted by software vendors
>  HBase deb http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x HDP
> main
>  Cassandra deb http://debian.datastax.com/community  stable main
> - as tar.gz hosted on Apache Download Mirrors
>  HBase  http://www.apache.org/dyn/closer.cgi/hbase/
>  Cassandra http://www.apache.org/dyn/closer.cgi/cassandra/
>
> The distributions provided by Apache Foundation looks more reliable, but I
> heard, that third party sources can be not stable enough to introduce them
> as dependencies in devstack gating.
>
> I have registered BP in devstack project about adding HBase
> https://blueprints.launchpad.net/devstack/+spec/add-hbase-to-devstack and
> we have started working on it.
>
> Please share your thoughts about it to help make it real.
> Thank you.
>
>
> Have a nice day,
> Ilya Sviridov
> isviridov @ FreeNode
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [savanna] swift integration - optional?

[Nadya Privalova]

Hi Jon,

The main reason why this button is on UI is to let user know that Savanna
supports swift.
If a cluster doesn't have swift at all user may disable swift explicitly,
but it's the truth that if he or she doesn't disable swift nothing will be
broken.

Regards,
Nadya



On Wed, Sep 11, 2013 at 6:30 PM, Jon Maron  wrote:

> Hi,
>
>   I noticed that the swift integration is optionally enabled via a
> configuration property?  Is there a reason for not making it available as a
> base, feature of the cluster (i.e. simply allowing access to swift should
> it be required)?  What would be a scenario in which it would be beneficial
> to explicitly disable it?
>
> -- Jon
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Savanna][EDP] Issues from internal UI demo

[Nadya Privalova]

Hello folks,
Today I've showed a demo with EDP UI. Here a feedback from our
Mirantis-team.  Items without NOTEs should be done, I believe. For some of
them bugs are already filed:

1. We need URL hints for DataSouce (as it done for JobBinary)
2. DataSource UI should depend on DataSource type (
https://wiki.openstack.org/w/images/6/62/DataSourceCreation.jpg)
3. Usability issue. User should create DataSource each time. Each time he
or she should set auth info for DataSource. It will be better to split
DataSource on two objects: DataSourceAuth and DataSourceeURL. *NOTE: that's
just a proposal*
4. JobBinary TODOs:
4.1. Remove  swift-external option (we will not support it in current
phase)
4.2 *NOTE: that's just a proposal: *Hide swift-internal option if Swift
is not available
4.3. *NOTE: that's just a proposal: *Set available storage for binaries
in savanna.conf. Becuase it's  unlikely to store binaries in db-internal if
swift is available
4.4 People wants to write scripts in Horizon (h :/)
5. Job contains now:
name
description
type
job_origin_id
input_type
output_type
job_configs

Type is determined by JobOrigin. input_type and output_type are determined
by JobExecution's DataSources. So, remove Job. Changes in objects after
that:
in JobExecution:
job_id ===> job_origin_id

in JobOrigin:
type appears

6. Config for job. Configs will be key-value. We will get user a hint about
configs that user may use but  user may set it's own. That's because we are
not sure where to get all possible job.xml configs.

Tomorrow I'm planning to resolve 5th.

Your feedback will be very appreciated
Thanks,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] [qa] Ceilometer ERRORS in normal runs

[Nadya Privalova]

Hi guys,

I can share my experience with devstack+ceilometer. There is certainly a
problem with MongoDB, because Ceilometer requires more fresh Mongo than
devstack provides. But I didn't experienced problems with SQL.
And just a quick question about testing: are there any plans to test
Ceilometer with different db-backends in devstack? Or do you suggest that
it is not devstack's responsibility?

Thanks,
Nadya


On Tue, Oct 22, 2013 at 6:55 PM, David Kranz  wrote:

> On 10/22/2013 10:19 AM, Sean Dague wrote:
>
>> On 10/21/2013 10:27 AM, Neal, Phil wrote:
>>
>>> Sean, we currently have a BP out there to investigate basic tempest
>>> integration and I think this might fall under the same umbrella.
>>> Unfortunately I've not been able to free up my development time
>>> for it, but I've assigned it out to someone who can take a look and
>>> report back.
>>>
>>> https://blueprints.launchpad.**net/tempest/+spec/basic-**
>>> tempest-integration-for-**ceilometer
>>>
>>
>> This is kind of worse than tempest integration issues. As far as I can
>> tell ceilometer via devstack is basically non functional at all. And sort
>> of worse than non functional, it's spewing errors, a lot.
>>
>> This really ought to be a top ceilometer item to address, otherwise we
>> should probably turn off celiometer in devstack by default, because it's
>> really not working at the moment.
>>
>> -Sean
>>
>>  Here are the two errors showing up persistently that are not
> whitelisted. Such log errors are now being shown in the console log right
> after the tempest tests finish.
>
> https://bugs.launchpad.net/**ceilometer/+bug/1243251
> 2013-10-21 21:11:00.229 | 2013-10-21 21:05:20.046 5624 ERROR
> ceilometer.collector.**dispatcher.database [-] Failed to record metering
> data: QueuePool limit of size 5 overflow 10 reached, connection timed out,
> timeout 30
>
>
> https://bugs.launchpad.net/**ceilometer/+bug/1243249
> 2013-10-21 20:22:27.600 | Log File: ceilometer-alarm-evaluator
> 2013-10-21 20:22:27.600 | 2013-10-21 20:14:33.038 22760 ERROR
> ceilometer.alarm.service [-] alarm evaluation cycle failed
>
> See also 
> https://bugs.launchpad.net/**ceilometer/+bug/1237671
>
>  -David
>
>
>
> __**_
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.**org 
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer][qa]Tempest tests for Ceilometer

[Nadya Privalova]

Hello, guys!

I hope everybody has eventually got home after the summit and feeling ok :)
So it's time to proceed thinking about integration, unit and performance
testing in Ceilometer. First of all I'd like to appreciate your help in
composing etherpad
https://etherpad.openstack.org/p/icehouse-summit-ceilometer-integration-tests.
If you didn't participate in design session about integration tests
but
have thoughts about it please add your comments.

Here is a list of ceilometer-regarding cr in tempest (just a reminder):

   1. https://review.openstack.org/#/c/39237/
   2. https://review.openstack.org/#/c/55276/

And even more but they are abandoned due to reviewers' inactivity (take a
look in whiteboard):
https://blueprints.launchpad.net/tempest/+spec/add-basic-ceilometer-tests .
Is there any reasons why cr were not reviewed?

I guess the first step to be done is test plan. I've created a doc
https://etherpad.openstack.org/p/ceilometer-test-plan and plan to start
working on it. If you have any thoughts about the plan - you are welcome!

Thanks,
Nadya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer][qa]Tempest tests for Ceilometer

[Nadya Privalova]

Eoghan,

I've updated the agenda. Actually, I'm ready to start working on tasks'
coordination (division) but need some time to get acquainted with
Ceilometer infra (gating, devstack problems). Anyway, we will discuss it on
irc. So, Zhi Kun Liu, please join us :)

Julien,

Thanks for looking into crs. As I see 39237 made a good progress today

Thanks,
Nadya

On Wed, Nov 13, 2013 at 3:01 PM, Julien Danjou  wrote:

> On Tue, Nov 12 2013, Nadya Privalova wrote:
>
> Hi Nadya,
>
> > Here is a list of ceilometer-regarding cr in tempest (just a reminder):
> >
> >1. https://review.openstack.org/#/c/39237/
> >2. https://review.openstack.org/#/c/55276/
>
> 39237 seems on a good road to go in at this stage.
>
> > And even more but they are abandoned due to reviewers' inactivity (take a
> > look in whiteboard):
> >
> https://blueprints.launchpad.net/tempest/+spec/add-basic-ceilometer-tests.
> > Is there any reasons why cr were not reviewed?
>
> Many patches expired because it was impossible for them to get a +1 from
> Jenkins, due to a bug in devstack. Now that it has been fixed, it's
> possible to have a patch such as 39237 to pass.
>
> > I guess the first step to be done is test plan. I've created a doc
> > https://etherpad.openstack.org/p/ceilometer-test-plan and plan to start
> > working on it. If you have any thoughts about the plan - you are welcome!
>
> I'd suggest to use the blueprint whiteboard at:
>
>
> https://blueprints.launchpad.net/tempest/+spec/add-basic-ceilometer-tests
>
> I'm afraid that otherwise this Etherpad will be lost. :-(
>
> --
> Julien Danjou
> # Free Software hacker # independent consultant
> # http://julien.danjou.info
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel-dev] [OSTF][Ceilometer] ceilometerclient proxy issue

[Nadya Privalova]

Hi guys,

I've created the bp
https://blueprints.launchpad.net/python-ceilometerclient/+spec/proxy-support.
Will start working on this asap.

Thanks,
Nadya


On Fri, Nov 15, 2013 at 3:23 PM, Mike Scherbakov
wrote:

> Hi Ceilometer team,
> do you have any plans for http proxy support (see concerns below)?
>
> In short, in the Fuel deployment we see a need for HTTP Proxy support in
> Ceilometer client as Ceilometer API is not directly available. We access
> Nova via proxy server, and need the same for Ceilometer.
>
> Thanks,
>
> On Fri, Nov 15, 2013 at 3:07 PM, Vladimir Kuklin wrote:
>
>> I do insist on python-ceilometer patching. This should not take long as
>> it requires only patching of core methods, that create HTTP requests.
>>
>>
>> On Fri, Nov 15, 2013 at 1:45 PM, Roman Sokolkov wrote:
>>
>>>  Hi,
>>>
>>> OSTF uses http proxy for access to openstack networks. But
>>> ceilometerclient doesn't support http_proxy bcause it based on httlib
>>> python module.
>>>
>>> We've resolved this issue in non-HA mode by endpoint replacement (points
>>> on 10.20.. network that accessible from master node).
>>>
>>> But this trick not work with HA mode, because API services listen only
>>> on internal address (192.168) and there is no network access from master
>>> node to the API.
>>>
>>> Savana and Murano clients also affected by this issue. They resolve
>>> problem by 0.0.0.0 bindings. May be some other hacks..
>>>
>>> We need your opinion on it. Possible options:
>>>
>>>- bind ceilometer on 0.0.0.0 (not preffered, i think)
>>>- patch ceilometerclient with http_proxy support
>>>- some network workaround (iptables, routing)
>>>- other
>>>
>>> Thanks, Roman S.
>>>
>>> --
>>> Mailing list: https://launchpad.net/~fuel-dev
>>> Post to : fuel-...@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>> Yours Faithfully,
>> Vladimir Kuklin,
>> Senior Deployment Engineer,
>> Mirantis, Inc.
>> +7 (495) 640-49-04
>> +7 (926) 702-39-68
>> Skype kuklinvv
>> 45bk3, Vorontsovskaya Str.
>> Moscow, Russia,
>> www.mirantis.com 
>> www.mirantis.ru
>> vkuk...@mirantis.com 
>>
>> --
>> Mailing list: https://launchpad.net/~fuel-dev
>> Post to : fuel-...@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~fuel-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Mike Scherbakov
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [savanna-dashboard] version 0.3 - updated UI mockups for EDP workflow

[Nadya Privalova]

Hi all,

I've created a temporary page for UIs mockups. Please take a look:
https://wiki.openstack.org/wiki/Savanna/UIMockups/JobCreationProposal

Chad, it's just pictures demonstrate how we see dependencies in UI. It's
not a final decision.
Guys, feel free to comment this. I think it's time to start discussions.

Regards,
Nadya


On Mon, Sep 9, 2013 at 10:19 PM, Chad Roberts  wrote:

> Updated UI mockups for savanna dashboard EDP.
> https://wiki.openstack.org/wiki/Savanna/UIMockups/JobCreation
>
> Regards,
> Chad
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev