Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API
I know customers are using the 'pushing sample API', but don’t know whether they're applying transformer to those data or not. -Lianhao Lu > -Original Message- > From: Julien Danjou [mailto:jul...@danjou.info] > Sent: Wednesday, October 05, 2016 12:35 AM > To: gordon chung > Cc: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [telemetry] Deprecating the Ceilometer > API > > On Tue, Oct 04 2016, gordon chung wrote: > > > so one thing we probably do need to keep is the ability push samples > > (and events?). i know previously people were actually using this > feature. > > That's debatable. > > Pushing events is IMHO out of scope of Ceilometer – it's related to > oslo.messaging notification at this point. So if we want to allow that via > an API endpoint, we should build one in OpenStack over > oslo.messaging. Good idea or not, I don't know. > > Pushing samples is probably doable with some kind of small API, but I > am not sure it's worth anything. You could still directly push the data to > Gnocchi and save some you some load. Unless you have transformers > to apply, but… really? > > -- > Julien Danjou > /* Free Software hacker >https://julien.danjou.info */ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack-ansible] git repo in infra_repo container not working
Hi guys,
I'm encounter new problems with the OSA SHA master
eb3aec7827e78d81469ff4489c28963ee602117c (I use this version because previously
the master version cf79d4f6 has problems with the nova transport_url settings
which blocks nova-api to be launched), the problem is that the git repo in
infra_repo containers not working, which blocks me from going on.
TASK [os_nova : Get package from git] **
FAILED - RETRYING: TASK: os_nova : Get package from git (4 retries left).
... ...
FAILED - RETRYING: TASK: os_nova : Get package from git (1 retries left).
fatal: [infra2_nova_console_container-2e227e79]: FAILED! => {"changed": false,
"cmd": "/usr/bin/git ls-remote
http://172.29.236.15:8181/openstackgit/spice-html5 -h refs/heads/54cc41299be
a8cd681ed0262735e0fd821cd774a", "failed": true, "msg": "fatal: repository
'http://172.29.236.15:8181/openstackgit/spice-html5/' not found", "rc": 128,
"stderr": "fatal: repository 'http:
//172.29.236.15:8181/openstackgit/spice-html5/' not found\n", "stdout": "",
"stdout_lines": []}
cmd: /usr/bin/git ls-remote http://172.29.236.15:8181/openstackgit/spice-html5
-h refs/heads/54cc41299bea8cd681ed0262735e0fd821cd774a
msg: fatal: repository 'http://172.29.236.15:8181/openstackgit/spice-html5/'
not found
stderr: fatal: repository 'http://172.29.236.15:8181/openstackgit/spice-html5/'
not found
The 172.29.236.15 is my LB ip(which use haproxy). I then "curl -L
http://172.29.236.15:8181/openstackgit/spice-html5/; and it worked ok.
Then I ssh into one of the infra-repo containers behind the LB, and find the
foloowings:
- "curl http://127.0.0.1:8181/penstackgit/spice-html5/; works, it
display the file content under directory
/var/www/repo/openstackgit/spice-html5/.
- "git ls-remote /var/www/repo/openstackgit/spice-html5/" works.
- "git ls-remote http://127.0.0.1:8181/penstackgit/spice-html5/;
doesn't. It says "fatal: repository
'http://127.0.0.1:8181/penstackgit/spice-html5/' not found"
Seems the git repo over http is not working. I checked other repos under
/var/www/repo/openstackgit/, e.g. ceilometer, neutron, nova, they all have the
same problems.
Any suggestions?
-Lianhao Lu
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry] stepping down as PTL
On Mar 12, 2016 00:32, gordon chung wrote: > hi folks, > > as the PTL nominations are open, i just want to note that i won't be > running again as the Telemetry PTL for Newton cycle. > > i've thoroughly enjoyed my time as PTL which afforded me the > opportunity to work with and learn from some great individuals who > share the same passion to collaborate openly. i have the utmost > confidence that the projects will continue to grow and become more > refined under the next leadership. > > personal thanks to everyone in Aodh, Ceilometer and Gnocchi > communities as well as the projects that provided valuable feedback by > collaborating with us. i thank you for sharing in the decision making > so i could spread the blame around. i also thank you for reading > through terribly written messages that make you question whether the > shift keys on all my keyboards are broken. > > cheers, > Thank you for the great leadership in the past cycles. Your openness and great work of leading the telemetry community will always be remembered. Really appreciate your help and guidance! -Lianhao __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][ceilometer] unable to see compute.node.cpu.* metrics in ceilometer
On Mar 8, 2016 02:15, Kapil wrote: > Hi > > > I enabled ComputeDriverCPUMonitor in nova.conf on one of the compute > nodes, restarted nova-compute, ceilometer-agent-compute on the compute > node and ceilometer-collector, ceilometer-api, > ceilometer-agent-central, ceilometer-agent-notification on the > controller node. > > However, I cannot see the cpu meters or samples in ceilometer. Which version of nova/ceilometer are you using? First you need to make sure that nova have generated the related notifications. Please be noted that there is a configuration change in recent nova. See [1] for how to configure nova to use UBS. > > Any suggestions to what may be the issue ? > [1] https://01.org/sites/default/files/utilization_based_scheduing_in_openstack_compute_nova-revision002.pdf -Lianhao __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer] Unable to get IPMI meter readings
On Mar 5, 2016 04:10, Kapil wrote: > Yes, I had to look through the source code of the ipmi pollster class > to figure out why the error was being raised. Apparently, I don't have > Intel node manager installed, so power plugin was not being loaded. > I had to write my own plugin to get that data using ipmi-dcmi command > which is not specific to Intel I guess. > Is there any plan to add dcmi support to ceilometer ? > Currently no, but welcome to contribute dcmi support. -Lianhao Lu __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer] Unable to get IPMI meter readings
Hi Kapil, Currenlyt, the ipmi pollsters can only get the ipmi data from system bus due to the security concerns. So you have the make sure the ceilometer-agent-ipmi is running on the same machine you want get the hardware.ipmi.node.power metric from. Also you should make sure your machine have NodeManager features and enabled that in your bios settings, otherwise the the hardware.ipmi.node.power pollster won't be loaded because it will checks whether your machine support that during load time. -Lianhao Lu > -Original Message- > From: Kapil [mailto:kapil6...@gmail.com] > Sent: Friday, March 04, 2016 2:34 AM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] [ceilometer] Unable to get IPMI meter > readings > > So, we upgraded our openstack install from Juno to Kilo 2015.1.1 > > Not sure if this fixed some stuff, but I can now get samples for > hardware.ipmi.(fan|temperature). However, I want to get > hardware.ipmi.node.power samples and I get the following error in > the ceilometer log- > > ERROR ceilometer.agent.base [-] Skip loading extension for > hardware.ipmi.node.power > > > I edited pipeline.yaml as follows- > sources: > - name: meter_ipmi > interval: 10 > resources: > - "ipmi://" > meters: > - "hardware.ipmi.node.power" > sinks: > - ipmi_sink > sinks: > - name: ipmi_sink > transformers: > publishers: > - notifier://?per_meter_topic=1 > > > I also checked "rabbitmqctl list_queues | grep metering" and all the > queues are empty. > > > Do I need to change anything in ceilometer.conf or on the controller > nodes ? Currently, I am working only with the compute node and only > running ceilometer queries from controller node. > > > Thanks > > > Regards, > Kapil Agarwal > > On Thu, Feb 25, 2016 at 12:20 PM, gordon chung <g...@live.ca> wrote: > > > at quick glance, it seems like data is being generated[1]. if you > check > your queues (rabbitmqctl list_queues for rabbit), do you see > any items > sitting on notification.sample queue or metering.sample > queue? do you > receive other meters fine? maybe you can query db directly to > verify > it's not a permission issue. > > [1] see: 2016-02-25 13:36:58.909 21226 DEBUG > ceilometer.pipeline [-] > Pipeline meter_sink: Transform sample >at 0x7f6b3630ae50> from 0 transformer _publish_samples > /usr/lib/python2.7/dist-packages/ceilometer/pipeline.py:296 > > On 25/02/2016 8:43 AM, Kapil wrote: > > Below is the output of ceilometer-agent-ipmi in debug mode > > > > http://paste.openstack.org/show/488180/ > > ᐧ > > > > Regards, > > Kapil Agarwal > > > > On Wed, Feb 24, 2016 at 8:18 PM, Lu, Lianhao > <lianhao...@intel.com > > > <mailto:lianhao...@intel.com>> wrote: > > > > On Feb 25, 2016 06:18, Kapil wrote: > > > Hi > > > > > > > > > I discussed this problem with gordc on the telemetry IRC > channel > > but I > > > am still facing issues. > > > > > > I am running the ceilometer-agent-ipmi on the compute > nodes, I > > changed > > > pipeline.yaml of the compute node to include the ipmi > meters and > > > resource as "ipmi://localhost". > > > > > > - name: meter_ipmi > > > interval: 60 > > > resources: > > > - ipmi://localhost meters: - "hardware.ipmi.node*" > - > > > "hardware.ipmi*" - "hardware.degree*" sinks: - > meter_sink I > > > have ipmitool installed on the compute nodes and > restarted the > > > ceilometer services on compute and controller nodes. > Yet, I am not > > > receiving any ipmi meters when I run "ceilometer meter- > list". I also > > > tried passing the hypervisor IP address and the ipmi > address I get > > > when I run "ipmitool lan print" to resources but to no > avail. > > > > > > > > > Please help in this regard. > > > > > > > > > Thanks > &
Re: [openstack-dev] [ceilometer] Unable to get IPMI meter readings
On Feb 25, 2016 06:18, Kapil wrote: > Hi > > > I discussed this problem with gordc on the telemetry IRC channel but I > am still facing issues. > > I am running the ceilometer-agent-ipmi on the compute nodes, I changed > pipeline.yaml of the compute node to include the ipmi meters and > resource as "ipmi://localhost". > > - name: meter_ipmi > interval: 60 > resources: > - ipmi://localhost meters: - "hardware.ipmi.node*" - > "hardware.ipmi*" - "hardware.degree*" sinks: - meter_sink I > have ipmitool installed on the compute nodes and restarted the > ceilometer services on compute and controller nodes. Yet, I am not > receiving any ipmi meters when I run "ceilometer meter-list". I also > tried passing the hypervisor IP address and the ipmi address I get > when I run "ipmitool lan print" to resources but to no avail. > > > Please help in this regard. > > > Thanks > Kapil Agarwal Hi Kapil, Would you please turn on debug/verbose configurations and paste the log of ceilometer-agent-ipmi on http://paste.openstack.org ? -Lianhao Lu __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [ceilometer][gnocchi] 'bad' resource_id
Hi stackers, In ceilometer, some metrics(e.g. network.incoming.bytes for VM net interface, hardware.network.incoming.bytes for host net interface, compute.node.cpu.percentage for nova compute node host cpu utilization, etc.) don't have their resource_id in UUID format(which is required by gnocchi). Instead, they have something like . as their resource_id, in some cases even the part won't be in uuid format. Gnocchi will treat these kind of resource_id as bad id, and build a new UUID format resource_id for them. Since users are mostly using resource_id to identify their resources, changing user passed in resource_id would require the users extra effort to identify the resources in gnocchi and link them with the resources they original passed in. It seems there're several options to handle this kind of 'bad' resource_id problem. I'm writing this email to ask for your opinios. 1. Create new types of resource in gnocchi, and put original resource_id information as new resource attributes for that specific type. This might require adding different new code in gnocchi for different types of metrics with 'bad' resource_id, but it might give user fine grain control and aware they're dealing with special types of resources with 'bad' resource_id. 2. Added a new resource attribute original_resource_id in the generic resource type, and inhence will be inherited by all resource types in goncchi. This won't require adding new code for resources with 'bad' id, but might require adding a new db index on original_resource_id for resource search purpose. Any comments or suggestions? Best Regards, -Lianhao Lu __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer][gnocchi] 'bad' resource_id
On Dec 16, 2015 14:13, Chris Dent wrote: > On Wed, 16 Dec 2015, Lu, Lianhao wrote: > >> In ceilometer, some metrics(e.g. network.incoming.bytes for VM net >> interface, hardware.network.incoming.bytes for host net interface, >> compute.node.cpu.percentage for nova compute node host cpu >> utilization, >> etc.) don't have their resource_id in UUID format(which is required >> by gnocchi). Instead, they have something like . as >> their resource_id, in some cases even the part won't be in uuid >> format. Gnocchi will treat these kind of resource_id as bad id, and >> build a new UUID format resource_id for them. Since users are mostly >> using resource_id to identify their resources, changing user passed >> in resource_id would require the users extra effort to identify the >> resources in gnocchi and link them with the resources they original >> passed in. > > Just for the sake of completeness can you describe the use cases where > the resource_id translation that gnocchi does does not help the use > case. The one way translation is used in the body of search queries as > well as in any URL which contains a resource_id. > > I'm sure there are use cases where it breaks down, but I've not heard > them enumerated explicitly. > I'm not saying the translation will break down anything. It's just that in the case of using ceilometer/gnocchi together, when ceilometer samples are stored into gnocchi, the users need to do extra steps to figure out which resource to query to find its related metrics in the bad resource_id case. By simply looking at the http://docs.openstack.org/admin-guide-cloud/telemetry-measurements.html , the users can not easily identify the resource and its related metrics in gnocchi. The users need to be able to do a resource search based on resource attributes, such as original_resource_id, because in ceilometer/gnocchi cases, they don't get a chance to see the new resource_id gnocchi generated unless they search. Say we have configured nova to send out compute node metrics notification which will be turns into compute.node.cpu.percentage samples by ceilometer and stored into gnocchi, the original resource_id would be constructed as _ of the nova compute node machine. But when admin want to search that resource in gnocchi, he either search for a specific new type of resource with some conditions or search for a generic resource with condition of original_resource_id="_", otherwise he doesn't have ways to find the resource which is identified by the original resource_id. -Lianhao __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Why ceilometer do not offer run_tests.sh script?
On Apr 29, 2015 09:49, Luo Gangyi wrote: Hi guys, When I try to run unit tests of ceilometer, I find there is no run_tests.sh script offers. And when I use tox directly, I got a message ' 'Could not find mongod command'. Please use setup-test-env-mongodb.sh instead. See tox.ini for details. So another question is why unit tests needs mongo? It's used for the scenario tests on different db backend. Will be moved into functional test though. https://review.openstack.org/#/c/160827/ Can someone give me some hint? -Lianhao Lu __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer][FFE] Floating IP traffic statistics meters
Hi Megh, Thanks for contributing to Ceilometer. I've left some comment on your patch. Also, ceilometer follows the official blueprint/spec process as described https://wiki.openstack.org/wiki/Blueprints. -Lianhao -Original Message- From: Megh Bhatt [mailto:me...@juniper.net] Sent: Saturday, March 28, 2015 8:56 AM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [ceilometer][FFE] Floating IP traffic statistics meters Hello, Apologies for the double post, forgot to include FFE in the subject: I'd like to request an exemption for the following to go into the Kilo release. This work is crucial for: Cloud operators need to be able to bill customers based on floating IP traffic statistics. Why this needs an FFE? It's officially new feature adding 4 new meters Status of the work: In summary the patch only introduces 4 new meters - ip.floating.transmit.packets, ip.floating.transmit.bytes, ip.floating.receive.packets, ip.floating.receive.bytes and adds 2 new functions to the neutron_client - a) Function to get list of all floating IPs and 2) Get information about a specific port. - The patch necessary for this is already submitted for the review - https://review.openstack.org/#/c/166491/ - The document impact patch has already been reviewed and is waiting for the ceilometer commit to go through - https://review.openstack.org/#/c/166489/ Thanks Megh __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: OpenStack-dev- requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer] unable to collect compute.node.cpu.* data
Hi Frank, Could you try ‘celometer sample-list’ to see if the compute.node.cpu samples are there? -Lianhao From: Du Jun [mailto:dj199...@gmail.com] Sent: Wednesday, November 05, 2014 3:44 AM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [ceilometer] unable to collect compute.node.cpu.* data Hi all, I attempt to collect compute.node.cpu as the following link mentions: http://docs.openstack.org/developer/ceilometer/measurements.html#compute-nova I set: compute_monitors = ComputeDriverCPUMonitor in /etc/nova/nova.conf and restart nova-compute, nova-scheduler, ceilometer-agent-notification, ceilometer-api, ceilometer-collector. From ceilometer-agent-notification's log, I can see agent transform and publish data samples compute.node.cpu.* What's more, from ceilometer database, I can see all the meters compute.node.cpu.* mysql select * from meter; ++-++---+ | id | name| type | unit | ++-++---+ | 39 | compute.node.cpu.frequency | gauge | MHz | | 41 | compute.node.cpu.idle.percent | gauge | % | | 38 | compute.node.cpu.idle.time | cumulative | ns| | 45 | compute.node.cpu.iowait.percent | gauge | % | | 42 | compute.node.cpu.iowait.time| cumulative | ns| | 36 | compute.node.cpu.kernel.percent | gauge | % | | 44 | compute.node.cpu.kernel.time| cumulative | ns| | 37 | compute.node.cpu.percent| gauge | % | | 43 | compute.node.cpu.user.percent | gauge | % | | 40 | compute.node.cpu.user.time | cumulative | ns| However, when I type ceilometer meter-list It shows nothing about compute.node.cpu.*, so I wonder what's wrong with my steps. -- Regards, Frank ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Adding Dina Belova to ceilometer-core
Definitely +1 from me Lianhao -Original Message- From: Julien Danjou [mailto:jul...@danjou.info] Sent: Thursday, September 11, 2014 9:25 PM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [Ceilometer] Adding Dina Belova to ceilometer-core Hi, Dina has been doing a great work and has been very helpful during the Juno cycle and her help is very valuable. She's been doing a lot of reviews and has been very active in our community. I'd like to propose that we add Dina Belova to the ceilometer-core group, as I'm convinced it'll help the project. Please, dear ceilometer-core members, reply with your votes! -- Julien Danjou // Free Software hacker // http://julien.danjou.info ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [ceilometer] Do we have IRC meeting this week?
Looks like many are in Paris midcycle meet-up. Do we have the weekly IRC meeting today? -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [oslo][messaging] Question of oslo.messaging notificaiton listener
Hi oslo.messaging gurus, When we're debugging a ceilometer bug #1320420, we find that for the oslo messaging notification listener, if we have multiple endpoints registered through oslo.messaging.get_notification_listener(), and one of the endpoints raise an exception, that would stop all other endpoints processing that notification. Is it designed so purposely or is it a bug? Best Regards, -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [infra]gate-***-requirements failed across all projects
Hi guys, Looks like gate-***-requirements test failed across all the projects https://review.openstack.org/#/q/status:open+branch:master+topic:openstack/requirements,n,z due to an overlap check failure? I saw a patch https://review.openstack.org/#/c/97893/ to revert the check but that patch seems not get populated to the test farm. Best Regards, -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] nominating Ildikó Váncsa and Nadya Privalova to ceilometer-core
+1 for both. Best Regards, Lianhao -Original Message- From: Eoghan Glynn [mailto:egl...@redhat.com] Sent: Monday, March 10, 2014 5:15 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [ceilometer] nominating Ildikó Váncsa and Nadya Privalova to ceilometer-core Folks, Time for some new blood on the ceilometer core team. I'd like to nominate Ildikó Váncsa and Nadya Privalova as ceilometer cores in recognition of their contributions([1], [2]) over the Icehouse cycle, and looking forward to their continued participation for Juno. Both showed up with design summit proposals in HK, and have since demonstrated staying power as project contributors, proving themselves as eagle-eyed reviewers. Contribution highlights: * Ildikó co-authored the complex query API extension with Balazs Gibizer and showed a lot of tenacity in pushing this extensive blueprint through gerrit over multiple milestones. * Nadya has shown much needed love to the previously neglected HBase driver bringing it much closer to feature parity with the other supported DBs, and has also driven the introduction of ceilometer coverage in Tempest. This nomination doubles as my +1 for both ildikov nprivalova. Ceilometer cores, please respond with your yeas or nays on list. Cheers, Eoghan [1] http://bit.ly/ildikov-icehouse-reviews http://bit.ly/ildikov-icehouse-commits [2] http://bit.ly/nprivalova-icehouse-reviews http://bit.ly/nprivalova-icehouse-commits ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer] Discussion of the resource loader support patch
Gordon Chung wrote on 2014-01-21: If the resources defined in the pipeline doesn't match any resource file loader, it will be treated as directly passing to the pollsters. E.g. resources: - fileloader:///foo/bar - snmp://2.2.2.2 The endpoint 'snmp://2.2.2.2' will be passed to the pollsters along with the those read from the file /foo/bar. i don't have any particular opposition to the code. i have two questions, the first is related to validation. if we load a list of resources from pipeline.yaml and another 'loader' source, what happens when the same resource(s) are listed in both pipeine.yaml and 'loader' source. do we just let it continue with duplicates or do we try to filter? No, we'll remove the duplication, so the final resources passed to the pollster don't have this kind of duplication. also, another question would be, what other type of 'loaders' are there aside from fileloader? i don't really have any ideas outside of fileloader so it'd be interesting to know of other use-cases. Maybe a DB loader to load the resources from a central management DB, maybe a HTTP loader to query for resources definition using restful APIs, etc. Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [ceilometer] Discussion of the resource loader support patch
Hi CM guys, jd__ wanted to hold off the patch https://review.openstack.org/58747 because he thinks it's not generic enough and want to have a further discussion about the resource loader support. So I put it here my original thought and design about the patch as a start point. The initial intension is to allow loading new resources(endpoints) without modifying the pipeline configuration file or restarting the central agent. If the admin sets the resource loader in the resources filed in the pipeline file, e.g. resources: - fileloader:///foo/bar The central agent's PollingTask would use the corresponding resource loader to load new resources, every polling interval time before getting the samples from the pollsters for those endpoints. In the above configuration example, the file resource loader would read the resources definition from the file /foo/bar and pass those to the pollsters. The resource loader implementation can have its own internal cache, like the file resource loader, so it doesn't mean it has to open and read the whole file every polling time unless the corresponding file is updated. If the resources defined in the pipeline doesn't match any resource file loader, it will be treated as directly passing to the pollsters. E.g. resources: - fileloader:///foo/bar - snmp://2.2.2.2 The endpoint 'snmp://2.2.2.2' will be passed to the pollsters along with the those read from the file /foo/bar. Any comment? p.s. Another patch https://review.openstack.org/#/c/58489/ for the same bp was approved before, but it failed in the gate test due to another previously merged patch. I've resolved that issue and would like to see reviews on this too. Thanks! Best Regards, -Lianhao Lu ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][infra] nova py27 unit test failures in libvirt
Hi guys, This afternoon I suddenly find that there are quite a lot of nova py27 unit test failures on Jenkins, like http://logs.openstack.org/15/62815/5/gate/gate-nova-python27/82d5d52/console.html. It seems to me that the registerCloseCallback method is not available any more in virConnect class. I'm not sure whether this is caused by a new version of libvirt python binding? Any comments? -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][scheduler][metrics] Additional metrics
Abbass MAROUNI wrote on 2013-11-21: Hello, I'm in the process of writing a new scheduling algorithm for openstack nova. I have a set of compute nodes that I'm going to filter and weigh according to some metrics collected from these compute nodes. I saw nova.compute.resource_tracker and metrics (ram, disk and cpu) that it collects from compute nodes and updates the rows corresponding to compute nodes in the database. I'm planning to write some modules that will collect the new metrics but I'm wondering if I need to modify the database schema by adding more columns in the 'compute_nodes' table for my new metrics. Will this require some modification to the compute model ? Then how can I use these metrics during the scheduling process, do I fetch each compute node row from the database ? Is there any easier way around this problem ? Best Regards, There are currently some effort on this: https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling https://blueprints.launchpad.net/nova/+spec/extensible-resource-tracking - Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer
Doug Hellmann wrote on 2013-11-19: On Mon, Nov 18, 2013 at 12:25 PM, Devananda van der Veen devananda@gmail.com wrote: Hi Lianhao Lu, I briefly summarized my recollection of that session in this blueprint: https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent I've responded to your questions inline as well. On Sun, Nov 17, 2013 at 10:24 PM, Lu, Lianhao lianhao...@intel.com wrote: Hi stackers, During the summit session Expose hardware sensor (IPMI) data https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors, it was proposed to deploy a ceilometer agent next to the ironic conductor to the get the ipmi data. Here I'd like to ask some questions to figure out what's the current missing pieces in ironic and ceilometer for that proposal. 1. Just double check, ironic won't provide API to get IPMI data, right? Correct. This was generally felt to be unnecessary. 2. If deploying a ceilometer agent next to the ironic conductor, how does the agent talk to the conductor? Through rpc? My understanding is that ironic-conductor will emit messages to the ceilimeter agent, and the communication is one-way. These could be triggered by a periodic task, or by some other event within Ironic, such as a change in the power state of a node. Cool, so this eliminates the need for a separate agent. The ceilometer work can be done in the collector, to receive the new messages. Does this means we lose the ability to specify the different polling interval for different monitoring data, like we have in ceilometer pipeline? 3. Does the current ironic conductor have rpc_method to support getting generic ipmi data, i.e. let the rpc_method caller specifying arbitrary netfn/command to get any type of ipmi data? No, and as far as I understand, it doesn't need one. It would only need that if we were going to poll for the data, but if ironic is emitting notifications then we don't have to do that. 4. I believe the ironic conductor uses some kind of node_id to associate the bmc with its credentials, right? If so, how can the ceilometer agent get those node_ids to ask the ironic conductor to poll the ipmi data? And how can the ceilometer agent extract meaningful information from that node_id to set those fields in the ceilometer Sample(e.g. recource_id, project_id, user_id, etc.) to identify which physical node the ipmi data is coming from? This question perhaps requires a longer answer. Ironic references physical machines (nodes) internally with an integer node_id and externally with a standard uuid. When a Nova instance is created, it will be associated to a node, that node will have a reference to the nova instance_uuid which is exposed in our API, and can be passed to Ceilometer's agent. I believe that nova instance_uuid will enable ceilometer to detect the project, user, etc. If ironic has those values (at least the project), and can include them in the notification payload, that will make processing the incoming notifications significantly less expensive. Should Ironic emit messages regarding nodes which are not provisioned? Physical machines that don't have a tenant instance on them are not associated to any project, user, tenant, quota, etc, so I suspect that we shouldn't notify about them. It would be like tracking the unused disks in a SAN. I don't think it would be useful, but if someone else does then it seems OK to include them. I think it'd better for Ironic to emit those data in case some users want to collect them, at least Ironic should have a configuration setting to emit those kind of data. -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ironic][Ceilometer] get IPMI data for ceilometer
Hi stackers, During the summit session Expose hardware sensor (IPMI) data https://etherpad.openstack.org/p/icehouse-summit-ceilometer-hardware-sensors, it was proposed to deploy a ceilometer agent next to the ironic conductor to the get the ipmi data. Here I'd like to ask some questions to figure out what's the current missing pieces in ironic and ceilometer for that proposal. 1. Just double check, ironic won't provide API to get IPMI data, right? 2. If deploying a ceilometer agent next to the ironic conductor, how does the agent talk to the conductor? Through rpc? 3. Does the current ironic conductor have rpc_method to support getting generic ipmi data, i.e. let the rpc_method caller specifying arbitrary netfn/command to get any type of ipmi data? 4. I believe the ironic conductor uses some kind of node_id to associate the bmc with its credentials, right? If so, how can the ceilometer agent get those node_ids to ask the ironic conductor to poll the ipmi data? And how can the ceilometer agent extract meaningful information from that node_id to set those fields in the ceilometer Sample(e.g. recource_id, project_id, user_id, etc.) to identify which physical node the ipmi data is coming from? Best Regards, -Lianhao Lu ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ceilometer] umbrella blueprints of central agent improvement
Hi ceilometer stackers, I've created the umbrella blueprints for central agent improvement https://blueprints.launchpad.net/ceilometer/+spec/central-agent-improvement. Please add the dependencies if I missed something. Besides, I'm appreciated if someone could comment on the spec of the bp https://blueprints.launchpad.net/ceilometer/+spec/support-resources-pipeline-item and to get it approved. -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] compute agent cannot start
Which version of six do you have? I think you at least need six 1.4.0 -Lianhao -Original Message- From: Shixiong Shang [mailto:sparkofwisdom.cl...@gmail.com] Sent: Friday, November 15, 2013 4:47 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Ceilometer] compute agent cannot start Hi, Guys: I am trying to run ceilometer agent on compute node, and it gave me the following traceback. I believe I hit this bug (https://bugs.launchpad.net/nova/+bug/1244055;). However, I would like to know whether there is any workaround? sudo python /usr/local/bin/ceilometer-agent-compute Traceback (most recent call last): File /usr/local/bin/ceilometer-agent-compute, line 6, in module from ceilometer.compute.manager import agent_compute File /usr/local/lib/python2.7/dist-packages/ceilometer/compute/manager.py, line 22, in module from ceilometer import agent File /usr/local/lib/python2.7/dist-packages/ceilometer/agent.py, line 24, in module from ceilometer import pipeline File /usr/local/lib/python2.7/dist-packages/ceilometer/pipeline.py, line 28, in module from ceilometer import publisher File /usr/local/lib/python2.7/dist-packages/ceilometer/publisher/__init__.py, line 40, in module @six.add_metaclass(abc.ABCMeta) AttributeError: 'module' object has no attribute 'add_metaclass' Thanks! Shixiong ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] compute agent cannot start
How do you replace, just manual copying? I think you should pip install -U six.
Best Regards,
Lianhao
-Original Message-
From: Shixiong Shang [mailto:sparkofwisdom.cl...@gmail.com]
Sent: Friday, November 15, 2013 10:35 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Ceilometer] compute agent cannot start
Hi, Lianhao:
I downloaded six package, 1.4.1 from pypi.python.org and replaced all
six.py files with the latest version, including the one under
/usr/lib/python2.7/dist-packages directory.
more /usr/lib/python2.7/dist-packages/six.py
import operator
import sys
import types
__author__ = Benjamin Peterson benja...@python.org
__version__ = 1.4.1
However, ceilometer-agent-compute still complained of VersionConflict.
18:30:04.376 11553 ERROR stevedore.extension [-] Could not load 'libvirt':
(six 1.3.0 (/usr/lib/python2.7/dist-packages),
Requirement.parse('six=1.4.1'))
2013-11-14 18:30:04.376 11553 ERROR stevedore.extension [-] (six 1.3.0
(/usr/lib/python2.7/dist-packages),
Requirement.parse('six=1.4.1'))
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension Traceback (most
recent call last):
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension File
/usr/lib/python2.7/dist-packages/stevedore/extension.py, line 89, in
_load_plugins
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension invoke_kwds,
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension File
/usr/lib/python2.7/dist-packages/stevedore/named.py, line 57, in
_load_one_plugin
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension ep,
invoke_on_load, invoke_args, invoke_kwds,
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension File
/usr/lib/python2.7/dist-packages/stevedore/extension.py, line 101, in
_load_one_plugin
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension plugin = ep.load()
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension File
build/bdist.linux-x86_64/egg/pkg_resources.py, line 2107, in load
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension if require:
self.require(env, installer)
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension File
build/bdist.linux-x86_64/egg/pkg_resources.py, line 2120, in require
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension
working_set.resolve(self.dist.requires(self.extras),env,installer)))
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension File
build/bdist.linux-x86_64/egg/pkg_resources.py, line 580, in resolve
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension raise
VersionConflict(dist,req) # XXX put more info here
2013-11-14 18:30:04.376 11553 TRACE stevedore.extension VersionConflict: (six
1.3.0 (/usr/lib/python2.7/dist-packages),
Requirement.parse('six=1.4.1'))
Thanks!
Shixiong
On Thu, Nov 14, 2013 at 9:02 PM, Lu, Lianhao lianhao...@intel.com wrote:
Which version of six do you have? I think you at least need six 1.4.0
-Lianhao
-Original Message-
From: Shixiong Shang [mailto:sparkofwisdom.cl...@gmail.com]
Sent: Friday, November 15, 2013 4:47 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Ceilometer] compute agent cannot start
Hi, Guys:
I am trying to run ceilometer agent on compute node, and it gave me
the following traceback. I believe I hit this bug
(https://bugs.launchpad.net/nova/+bug/1244055;). However, I would
like to know whether there is any workaround?
sudo python /usr/local/bin/ceilometer-agent-compute
Traceback (most recent call last):
File /usr/local/bin/ceilometer-agent-compute, line 6, in module
from ceilometer.compute.manager import agent_compute
File
/usr/local/lib/python2.7/dist-packages/ceilometer/compute/manager.py, line
22, in module
from ceilometer import agent
File /usr/local/lib/python2.7/dist-packages/ceilometer/agent.py,
line 24, in module
from ceilometer import pipeline
File
/usr/local/lib/python2.7/dist-packages/ceilometer/pipeline.py, line 28, in
module
from ceilometer import publisher
File
/usr/local/lib/python2.7/dist-packages/ceilometer/publisher/__init__.py,
line 40, in module
@six.add_metaclass(abc.ABCMeta)
AttributeError: 'module' object has no attribute 'add_metaclass'
Thanks!
Shixiong
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org
[openstack-dev] Full schedule for HongKong summit?
Hi all, Looks like we have http://icehousedesignsummit.sched.org/ for the summit session schedules and http://openstacksummitnovember2013.sched.org/ for presentations/panels schedules. Do we have a schedule combined both of the two? -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] what's in scope of Ceilometer
Gordon Chung wrote on 2013-08-29: so we're in the process of selling Ceilometer to product teams so that they'll adopt it and we'll get more funding :). one item that comes up from product teams is 'what will Ceilometer be able to do and where does the product takeover and add value?' the first question is, Ceilometer currently does metering/alarming/maybe a few other things... will it go beyond that? specifically: capacity planning, optimization, dashboard(i assume this falls under horizon/ceilometer plugin work), analytics. they're pretty broad items so i would think they would probably end up being separate projects? another question is what metrics will we capture. some of the product teams we have collect metrics on datacenter memory/cpu utilization, cluster cpu/memory/vm, and a bunch of other clustered stuff. i'm a nova-idiot, but is this info possible to retrieve? is the consensus that Ceilometer will collect anything and everything the other projects allow for? We're currently implementing a plugin-able framework in nova to collect metrics from nova compute nodes and send them into message bus, see https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling for the patches of that, and also the corresponding ceilometer's notification listener https://review.openstack.org/42838 for that. Besides, the ceilometer hardware agent(https://blueprints.launchpad.net/ceilometer/+spec/monitoring-physical-devices)is the place where to poll for data from any other physical hosts. -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] rpc.call to nova-conductor NOT return
Ok, we found the reason. We forgot the eventlet.monkeypatch() to patch those system modules to be greenthread-friendly. -Lianhao Lu, Lianhao wrote on 2013-08-07: Hi guys, Sorry to bug, but I have met a problem about the rpc call. I wrote some python code to rpc.call to nova-condcutor, but that rpc.call seems never return. (I'm using rabbitmq as the rpc backend) My code snippet is something like: conductor_api = conductor.API() ctxt = context.get_admin_context() services = conductor_api.service_get_all(ctxt) It blocks at conductor_api.service_get_all. I did some debug, and found that the main thread blocked at the line https://github.com/openstack/nova/blob/master/nova/openstack/common/rpc/amqp.py#L486 waiting to get the reply messages of the previous rpc.call to nova-conductor out of the data queue self._dataqueue (it's of type eventlet.queue.LightQueue). Also I've confirmed that nova-conductor has received the rpc.call and served it. And those reply messages from nova-conductor actually had been added into that data queue by the line https://github.com/openstack/nova/blob/master/nova/openstack/common/rpc/amqp.py#L193 . However, the main thread seemed never be able to wake up again. Here is a simplified version of my code which can reproduce the same problem: 1. Have a nova-conductor service running # nova-conductor --config-file nova config file 2. Get my simple python code # wget https://raw.github.com/lianhao/novadbtest/master/t.py 3. Run the python code # python t.py --config-file same nova config file as the nova-conductor Can anyone give me some clue where I'm doing it wrong? Thanks a lot! Best Regards, -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Best Regards, Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] question about rpc call dispatch in nova
Hi guys, When I'm doing some debugging work in nova, I've got some problems which I can't explain and needs some advices/comment from the guru in the community. With the rabbitmq as my rpc backend, I've started 2 conductor service, nova-condutor and my-condutor(which are very similar, except some different rpc and DB name configuration), and a python script 'my-test' using the nova.conductor.api to do the rpc call to conductor, using the following configurations: Nova-conductor: rabbit_host=localhost, control_exchange=nova, topic=conductor My-conductor: rabbit_host=localhost, control_exchange=myex, topic=conductor My-test: rabbit_host=localhost, control_exchange=myex, topic=conductor 1. If I started nova-conductor first, then started my-conductor after it, I found that the rpc calls issued by my-test were always serviced by nova-conductor. I don't know why this would happened. Because according to my understanding, only my-conductor can serve the rpc calls issued by my-test, because both of them used the same exchange. Any explanation for this? 2. If I started my-conductor first, then started nova-conductor after it, the AMQP messages for the rpc calls issued by my-test were received by my-conductor which is expected. But the rpc call was never returned. I did some debug and found that the greenthread to execute the method of nova.amqp.ProxyCallback._process_data() to serve the rpc call was successfully created, but never had a chance to run, until I 'Ctrl-C' the my-condcutor. Any clue for this? Thanks -Lianhao ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Nomination for Mehdi Abaakouk
+1 -Lianhao Angus Salkeld wrote on 2013-07-31: On 31/07/13 10:56 +0200, Julien Danjou wrote: Hi, I'd like to propose to add Mehdi Abaakouk (sileht) to ceilometer-core. He has been a valuable contributor for the last months, doing a lot of work in the alarming blueprints, and useful code reviews. +1 -- Julien Danjou -- Free Software hacker - freelance consultant -- http://julien.danjou.info ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
