[openstack-dev] [nova][libvirt] VIR_MIGRATE_NON_SHARED_INC works more like full copy in block migration ?
Hi devs, Do you test the difference between within and without VIR_MIGRATE_NON_SHARED_INC ? When I add VIR_MIGRATE_NON_SHARED_INC in block_migration_flags in nova, nova block migration behaves more like a full copy of disk. The disk in destination nodes is large(10GB+) and the process of live migration is slow. However, when I remove VIR_MIGRATE_NON_SHARED_INC in block_migration_flags in nova, nova block migration behaves more like a incremental copy of disk. The disk in destination nodes is small(10MB-) and the process of live migration is very fast. So I was confused about what VIR_MIGRATE_NON_SHARED_INC really means. Can someone give me some hints? -- Luo gangyiluogan...@cmss.chinamobile.com -- -- ??: "Mark McLoughlin";; : 2016??1??5??(??) 5:12 ??: "openstack-dev" ; : [openstack-dev] [nova][libvirt] Deprecating the live_migration_flagand block_migration_flag config options Hi commit 8ecf93e[1] got me thinking - the live_migration_flag config option unnecessarily allows operators choose arbitrary behavior of the migrateToURI() libvirt call, to the extent that we allow the operator to configure a behavior that can result in data loss[1]. I see that danpb recently said something similar: https://review.openstack.org/171098 "Honestly, I wish we'd just kill off 'live_migration_flag' and 'block_migration_flag' as config options. We really should not be exposing low level libvirt API flags as admin tunable settings. Nova should really be in charge of picking the correct set of flags for the current libvirt version, and the operation it needs to perform. We might need to add other more sensible config options in their place [..]" I've just proposed a series of patches, which boils down to the following steps: 1) Modify the approach taken in commit 8ecf93e so that instead of just warning about unsafe use of NON_SHARED_INC, we fix up the config option to a safe value. https://review.openstack.org/263431 2) Hard-code the P2P flag for live and block migrations as appropriate for the libvirt driver being used. For the qemu driver, We should always use VIR_MIGRATE_PEER2PEER both live and block migrations. Without this option, you get: Live Migration failure: Requested operation is not valid: direct migration is not supported by the connection driver OTOH, the Xen driver does not support P2P, and only supports "unmanaged direct connection". https://review.openstack.org/263432 3) Require the use of the UNDEFINE_SOURCE flag, and the non-use of the PERSIST_DEST flag. Nova itself persists the domain configuration on the destination host, but it assumes the libvirt migration call removes it from the source host. So it makes no sense to allow operators configure these flags. https://review.openstack.org/263433 4) Add a new config option for tunneled versus native: [libvirt] live_migration_tunneled = true This enables the use of the VIR_MIGRATE_TUNNELLED flag. We have historically defaulted to tunneled mode because it requires the least configuration and is currently the only way to have a secure migration channel. danpb's quote above continues with: "perhaps a "live_migration_secure_channel" to indicate that migration must use encryption, which would imply use of TUNNELLED flag" So we need to discuss whether the config option should express the choice of tunneled vs native, or whether it should express another choice which implies tunneled vs native. https://review.openstack.org/263434 5) Add a new config option for additional migration flags: [libvirt] live_migration_extra_flags = VIR_MIGRATE_COMPRESSED This allows operators to continue to experiment with libvirt behaviors in safe ways without each use case having to be accounted for. https://review.openstack.org/263435 We would disallow setting the following flags via this option: VIR_MIGRATE_LIVE VIR_MIGRATE_PEER2PEER VIR_MIGRATE_TUNNELLED VIR_MIGRATE_PERSIST_DEST VIR_MIGRATE_UNDEFINE_SOURCE VIR_MIGRATE_NON_SHARED_INC VIR_MIGRATE_NON_SHARED_DISK which would allow the following currently available flags to be set: VIR_MIGRATE_PAUSED VIR_MIGRATE_CHANGE_PROTECTION VIR_MIGRATE_UNSAFE VIR_MIGRATE_OFFLINE VIR_MIGRATE_COMPRESSED VIR_MIGRATE_ABORT_ON_ERROR VIR_MIGRATE_AUTO_CONVERGE VIR_MIGRATE_RDMA_PIN_ALL 6) Deprecate the existing live_migration_flag and block_migration_flag config options. Operators would be expected to migrate to using the live_migration_tunneled or live_migration_extra_flags config options.
Re: [openstack-dev] [Ceilometer][Gnocchi] inconsistentinstanceattributes cause infinite update
Yes, it will be better if we can synchronise them. But to achieve this, we have to do some modification in nova side which seems not easy :( -- Luo Gangyi luogan...@cmss.chinamobile.com -- Original -- From: "gord chung";<g...@live.ca>; Date: Mon, Dec 21, 2015 10:17 PM To: "openstack-dev"<openstack-dev@lists.openstack.org>; Subject: Re: [openstack-dev] [Ceilometer][Gnocchi] inconsistentinstanceattributes cause infinite update shall we try to synchronise the two datapoints more than? i think that'd be the better route since pollster and notification values are attempting to meter the same thing. On 21/12/2015 8:33 AM, Luo Gangyi wrote: > gord, I have looked your link, seems related. > But still, image_ref is a problem. -- gord __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][Gnocchi] inconsistent instanceattributes cause infinite update
@gord, I have looked your link, seems related. But still, image_ref is a problem. -- Luo Gangyi luogan...@cmss.chinamobile.com -- Original -- From: "gord chung";<g...@live.ca>; Date: Mon, Dec 21, 2015 09:12 PM To: "openstack-dev"<openstack-dev@lists.openstack.org>; Subject: Re: [openstack-dev] [Ceilometer][Gnocchi] inconsistent instanceattributes cause infinite update we actually looked at this problem here: https://review.openstack.org/#/c/235702/ i think we should add support of new attribute instead. On 20/12/2015 10:52 AM, Luo Gangyi wrote: Hi devs, I found a problem which may cause infinite update of instance's attributes in gnocchi. Let's see the resource definition of instance. - resource_type: instance metrics: - 'instance' - 'memory' - 'memory.usage' - 'memory.resident' - 'vcpus' - 'cpu' - 'cpu_util' - 'disk.root.size' ... attributes: host: resource_metadata.host image_ref: resource_metadata.image_ref_url ... Here is the problem, although they have same attributes, they are *not* same. Some of them came from nova's notifications and the others are came from ceilometer-compute-agent. 1) Those came from notifications, their attributes looks like image_ref :http://10.133.12.125:9292/images/ host: compute.lgy-openstack-kilo.novalocal 2) Those came from ceilometer-compute-agent, image_ref : http://10.133.12.125:8774/4994e42421a04beda56fff7d817e810e/images/8d6a9cd9-48ae-4a41-bd13-262a46c93d72 host:ea8f8e465d9caff06e80a0fda6f30d02725e0b55dc0fd940954cb55c Such difference will cause alternately and infinitely update of a instance's attributes if we enable nova audit. So I suggest we seperate these meters which came from notifications to another resource type like "instance_from_notification". Any other idea? -- Luo Gangyi luogan...@chinamobile.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- gord__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ceilometer][Gnocchi] inconsistent instance attributes cause infinite update
Hi devs, I found a problem which may cause infinite update of instance's attributes in gnocchi. Let's see the resource definition of instance. - resource_type: instance metrics: - 'instance' - 'memory' - 'memory.usage' - 'memory.resident' - 'vcpus' - 'cpu' - 'cpu_util' - 'disk.root.size' ... attributes: host: resource_metadata.host image_ref: resource_metadata.image_ref_url ... Here is the problem, although they have same attributes, they are *not* same. Some of them came from nova's notifications and the others are came from ceilometer-compute-agent. 1) Those came from notifications, their attributes looks like image_ref :http://10.133.12.125:9292/images/ host: compute.lgy-openstack-kilo.novalocal 2) Those came from ceilometer-compute-agent, image_ref : http://10.133.12.125:8774/4994e42421a04beda56fff7d817e810e/images/8d6a9cd9-48ae-4a41-bd13-262a46c93d72 host:ea8f8e465d9caff06e80a0fda6f30d02725e0b55dc0fd940954cb55c Such difference will cause alternately and infinitely update of a instance's attributes if we enable nova audit. So I suggest we seperate these meters which came from notifications to another resource type like "instance_from_notification". Any other idea? ------ Luo Gangyi luogan...@chinamobile.com__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][Gnocchi]Gnocchi cannot deal with combined resource-id ?
> I was talking about that: > > https://git.openstack.org/cgit/openstack/ceilometer/tree/etc/ceilometer/gnocchi_resources.yaml#n67 I check it again, it seems gnocchin_resouces.yaml was updated on Sep 1, 2015, and my Ceilometer code is not updated. Thanks Julien. ---------- Luo Gangyi luogan...@cmss.chinamobile.com -- Original -- From: "Julien Danjou";<jul...@danjou.info>; Date: Sat, Sep 12, 2015 10:54 PM To: "Luo Gangyi"<lgy...@foxmail.com>; Cc: "OpenStack Development Mailing L"<openstack-dev@lists.openstack.org>; Subject: Re: ?? [openstack-dev] [Ceilometer][Gnocchi]Gnocchi cannot deal with combined resource-id ? On Sat, Sep 12 2015, Luo Gangyi wrote: > I checked it again, no "ignored" is marked, seems the bug of devstack ;( I was talking about that: https://git.openstack.org/cgit/openstack/ceilometer/tree/etc/ceilometer/gnocchi_resources.yaml#n67 > And it's OK that gnocchi is not perfect now, but I still have some worries > about how gnocchi deal with or going to deal with instance--tapxxx > condition. > I see 'network.incoming.bytes' belongs to resouce type 'instance'. > But no attributes of instance can save the infomation of tap name. > Although I can search > all metric ids from resouce id(instance uuid), how do I distinguish them > from different taps of an instance? Where do you see network.incoming.bytes as being linked to an instance? Reading gnocchi_resources.yaml I don't see that. -- Julien Danjou -- Free Software hacker -- http://julien.danjou.info__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] ?????? [Ceilometer][Gnocchi] Gnocchi cannot deal with combined resource-id ?
Thanks Julien :) > But I don't think Ceilometer has this metric posted to Gnocchi yet, the > code is a bit young and not finished on the Ceilometer side. If you > check gnocchi_resources.yaml, it's still marked "ignored" for now. I checked it again, no "ignored" is marked, seems the bug of devstack ;( And it's OK that gnocchi is not perfect now, but I still have some worries about how gnocchi deal with or going to deal with instance--tapxxx condition. I see 'network.incoming.bytes' belongs to resouce type 'instance'. But no attributes of instance can save the infomation of tap name. Although I can search all metric ids from resouce id(instance uuid), how do I distinguish them from different taps of an instance? -- Luo Gangyi luogan...@cmss.chinamobile.com -- -- ??: "Julien Danjou";<jul...@danjou.info>; ????: 2015??9??12??(??) 0:01 ??: "Luo Gangyi"<lgy...@foxmail.com>; : "OpenStack Development Mailing L"<openstack-dev@lists.openstack.org>; : Re: [openstack-dev] [Ceilometer][Gnocchi] Gnocchi cannot deal with combined resource-id ? On Fri, Sep 11 2015, Luo Gangyi wrote: Hi Gangyi, > I am using master branch and newest code for testing. Cool. > For the purpose for learning the structure of gnocchi, I changed the > default UUID type of mysql from binary to char, so I can easily link > the resource-id(I mean in database), metric id and directory name of > storing measures. Bah, don't do that ?C and rather use PostgreSQL, it's the recommended backend. :) > When I did that, I found all the metrics where their resource id is > combined(here, I mean in Ceilometer, such as instance-xxx-tap) > have no measures stored. > Log in Ceilometer collector records this: > " > 2015-09-11 07:55:55.097 10636 ERROR ceilometer.dispatcher.gnocchi [-] > Resource > instance-0001-4641f59e-994c-4255-b0ec-43a276d1c19c-tap8aadb7ad-d7 > creation failed with status: 400: > > 400 Bad Request > > > 400 Bad Request > The server could not comply with the request since it is either malformed > or otherwise incorrect. > Invalid input: required key not provided @ data['display_name'] > > > > " > So I wander whether gnocchi cannot deal with such combined resource-id > metrics or whether it is because I change the UUID type or whatever. Yes, it can, but the problem is more likely in the Ceilometer collector dispatcher code that is sending the data. From the error you have, it seems it tries to create an instance but it has no value for display_name, so it is denied by Gnocchi. If this is from a standard devstack installation, I'd suggest to open a bug on Ceilometer. > And another question is how to query measures for those metrics whose > resource id is combined. They are resource on their own so if you know their id you can just access the metrics at /v1/resources///metric//measures. > For example, I want to query the network traffic of an vm, I know the > instance uuid -, I know metric name > 'network.incoming.byte.rate' but I do not know the exact resouce_id > and metric id. What procedure should I do ? You need to know the ID of the resource and then ask for its metric on the REST interface. If you don't know the ID of the resource, you can search for it by instance id. But I don't think Ceilometer has this metric posted to Gnocchi yet, the code is a bit young and not finished on the Ceilometer side. If you check gnocchi_resources.yaml, it's still marked "ignored" for now. -- Julien Danjou ;; Free Software hacker ;; http://julien.danjou.info__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ceilometer][Gnocchi] Gnocchi cannot deal with combined resource-id ?
Hi devs, I am trying Ceilometer with gnocchi. I find that gnocchi cannot deal with combined resource-id such as instance-xx-tapxx or instance--vda. I'm not sure whether it is my configuration problem or just bug. And if such combined resource-id can be processed correctly, what about its metadata(or called attributes)? In current design, gnocchi seems treat instance-aaa, instance-aaa-tap111, instance-aaa-tap222 as equal although they have parent-child relationship and share many attributes. Is anyone else have the same problem and concern? -- Luo gangyiluogan...@chinamobile.com__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][Gnocchi] Gnocchi cannot deal with combined resource-id ?
Hi, Julien I am using master branch and newest code for testing. For the purpose for learning the structure of gnocchi, I changed the default UUID type of mysql from binary to char, so I can easily link the resource-id(I mean in database), metric id and directory name of storing measures. When I did that, I found all the metrics where their resource id is combined(here, I mean in Ceilometer, such as instance-xxx-tap) have no measures stored. Log in Ceilometer collector records this: " 2015-09-11 07:55:55.097 10636 ERROR ceilometer.dispatcher.gnocchi [-] Resource instance-0001-4641f59e-994c-4255-b0ec-43a276d1c19c-tap8aadb7ad-d7 creation failed with status: 400: 400 Bad Request 400 Bad Request The server could not comply with the request since it is either malformed or otherwise incorrect. Invalid input: required key not provided @ data['display_name'] " So I wander whether gnocchi cannot deal with such combined resource-id metrics or whether it is because I change the UUID type or whatever. And another question is how to query measures for those metrics whose resource id is combined. For example, I want to query the network traffic of an vm, I know the instance uuid -, I know metric name 'network.incoming.byte.rate' but I do not know the exact resouce_id and metric id. What procedure should I do ? ------ Luo Gangyi luogan...@cmss.chinamobile.com -- Original -- From: "Julien Danjou";<jul...@danjou.info>; Date: Fri, Sep 11, 2015 06:31 PM To: "Luo Gangyi"<lgy...@foxmail.com>; Cc: "OpenStack Development Mailing L"<openstack-dev@lists.openstack.org>; Subject: Re: [openstack-dev] [Ceilometer][Gnocchi] Gnocchi cannot deal with combined resource-id ? On Fri, Sep 11 2015, Luo Gangyi wrote: Hi Luo, > I find that gnocchi cannot deal with combined resource-id such as > instance-xx-tapxx or instance--vda. I'm not sure whether it is my > configuration problem or just bug. Which version are you testing? The master branch has no support for resource ID that are not UUID. > And if such combined resource-id can be processed correctly, what about its > metadata(or called attributes)? In current design, gnocchi seems treat > instance-aaa, instance-aaa-tap111, instance-aaa-tap222 as equal although they > have parent-child relationship and share many attributes. We just merged support for those resources. We do not store any attribute other than the name and parent instance AFAICS. What do you miss as an attribute exactly? -- Julien Danjou # Free Software hacker # http://julien.danjou.info__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][Gnocchi] Gnocchi cannot deal withcombined resource-id ?
Hi, Chris > With that change in place you can still use the 'instance-xxx-tap' ID > in the part of /v1/metric/resource// URLs and in > search queries. But we cannot know the tap name through nova api or neutron api. In general, there exists some conditions that we cannot know the exact resource id in advance. And using hash of resource id also makes fuzzy search impossible. What we know are some relationships between resources, like a vm has several taps. The resource instance-111 and resource instance-111-tap111 has inborn connections. So I suggest that we should add some attribute to describe such relation. ---------- Luo Gangyi luogan...@cmss.chinamobile.com__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Let's set instance.root_gb=0 forvolume-backed instances
I agree! A positive root_gb of volume-backed instances which has different volume size is confusing! Last week I submit a patch to correct disk filter(https://review.openstack.org/#/c/194059/). But if we set instance.root_gb=0 for volume-backed instances, such problems could be solved fundamentally. -- Luo Gangyi -- Original -- From: Feodor Tersin;fter...@hotmail.com; Date: Fri, Jun 26, 2015 09:42 PM To: openstack-dev@lists.openstack.orgopenstack-dev@lists.openstack.org; Subject: [openstack-dev] [Nova] Let's set instance.root_gb=0 forvolume-backed instances Some solutions have been proposed to solve these problems [2]. One problem was even discussed here [3]. However all these solutions are aimed at correcting specific algorithms to allow them to take into account a type of an instance. But this does not affect other algorithms, which also use root_gb as local space size [4]. I.e. other similar problems are retained. I propose to fix all of this by setting 0 for root_gb, min_size, size [5], since 0 is a natural value of these attributes. Any objections? [1] https://bugs.launchpad.net/nova/+bug/1334974 https://bugs.launchpad.net/nova/+bug/1459491 https://bugs.launchpad.net/nova/+bug/1466305 https://bugs.launchpad.net/nova/+bug/1457517 [2] https://review.openstack.org/#/c/136284/ https://review.openstack.org/#/c/184982/ https://review.openstack.org/#/c/186247/ https://review.openstack.org/#/c/192971/ [3] http://lists.openstack.org/pipermail/openstack-dev/2015-March/058138.html [4] https://github.com/openstack/nova/blob/master/nova/notifications.py#L407 https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L250 [5] https://bugs.launchpad.net/nova/+bug/1469179__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer] polling agent configuration speculation
Hi, Chris, In current, ceilometer load pollsters by agent namespace, So do you mean you want load pollsters one by one through their name(maybe defined in pipeline.yaml)? If loading all pollsters in one time do not cost much, I think your change is bit unnecessary. But if it does cost much, your change is meaningful. BTW, I like the idea Separate polling and publishing/transforming into separate workers/processes. -- Luo gangyiluogan...@cmss.chinamobile.com -- Original -- From: Chris Dent;chd...@redhat.com; Date: Mon, Jun 8, 2015 09:04 PM To: openstack-operatorsopenstack-operat...@lists.openstack.org; OpenStack-devOpenStack-dev@lists.openstack.org; Subject: [openstack-dev] [ceilometer] polling agent configuration speculation (Posting to the mailing list rather than writing a spec or making code because I think it is important to get some input and feedback before going off on something wild. Below I'm talking about speculative plans and seeking feedback, not reporting decisions about the future. Some of this discussion is intentionally naive about how things are because that's not really relevant, what's relevant is how things should be or could be. tl;dr: I want to make the configuration of the pollsters more explicit and not conflate and overlap the entry_points.txt and pipeline.yaml in confusing and inefficient ways. * entry_points.txt should define what measurements are possible, not what measurements are loaded * something new should define what measurements are loaded and polled (and their intervals) (sources in pipeline.yaml speak) * pipeline.yaml should define transformations and publishers Would people like something like this?) The longer version: Several of the outcomes of the Liberty Design Summit were related to making changes to the agents which gather or hear measurements and events. Some of these changes have pending specs: * Ceilometer Collection Agents Split https://review.openstack.org/#/c/186964/ Splitting the collection agents into their own repo to allow use and evolution separate from the rest of Ceilometer. * Adding Meta-Data Caching Spec https://review.openstack.org/#/c/185084/ Adding metadata caching to the compute agent so the Nova-API is less assaulted than it currently is. * Declarative notification handling https://review.openstack.org/#/c/178399/ Be able to hear and transform a notification to an event without having to write code. Reviewing these and other specs and doing some review of the code points out that we have an opportunity to make some architectural and user interface improvements (while still maintain existing functionality). For example: The current ceilometer polling agent has an interesting start up process: 1 It determines which namespaces it is operating in ('compute', 'central', 'ipmi'). 2 Using entry_points defined in setup.cfg it initializes all the polling extensions and all the discovery extensions (independent of sources defined in pipeline.yaml) 3 Every source in pipeline.yaml is given a list of pollsters that match the meters defined by the source, creating long running tasks to do the polling. 4 Each task does resource discovery and partitioning coordination. 5 measurements/samples are gathered and are transformed and published according the sink rules in pipeline.yaml A couple things about this seem less than ideal: * 2 means we load redundant stuff unless we edit entry_points.txt. We do not want to encourage this sort of behavior. entry_points is not configuration[1]. We should configure elsewhere to declare I care about things X (including the option of all things) and then load the tools to do so, on demand. * Two things are happening in the same context in step 5 and that seems quite limiting with regard to opportunities for effective maintenance and optimizing. My intuition (which often needs to sanity checked, thus my posting here) tells me there are some things we could change: * Separate polling and publishing/transforming into separate workers/processes. * Extract the definition of sources to be polled from pipeline.yaml to its own file and use that to be the authority of which extensions are loaded for polling and discovery. What do people think? [1] This is really the core of my concern and the main part I want to see change. -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__ OpenStack Development Mailing List (not for usage questions) Unsubscribe:
[openstack-dev] [nova][cinder][libvirt] Serious problem of block migration of a hybrid instance (instance based on local disk and cinder volume attached)
Hi devs, I find a serious problem when I try to live migrate a hybrid instance (instance based on local disk and cinder volume attached) . In current nova, such operation is allowed. But I don't think libvirt can deal with such condition correctly. When block migration of a hybird intance is triggered, nova will initiate a new connection to the volume in destination compute node and then call libvirt to do the block migration. However, libvirt doesn't distinguish local disk and a network volume, it may copy both local disk and cinder volume from source to destination. It is dangerous to write the same volume simutaneously on source and destination!! and apparently there is no need to do this! I believe we should do something to correct this, maybe patch libvirt? or just forbid user do such operation? -- Luo Gangyi__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] ?????? [heat][ceilometer] autoscaling
I don't understand what you mean. Firstly, ceilometer doesn't return meters or samples to heat. In fact, heat configures an alarm in ceilometer and the action of this alarm is to send a REST to heat. When heat gets this REST, it triggers autoscalling. Besides, you can use #ceilometer alarm-list to see what alarm heat configures, then you could run #ceilometer query-sample to see the meter and sample. Hope it helps. -- Luo gangyiluogan...@chinamobile.com -- -- ??: ICHIBA Sara;ichi.s...@gmail.com; : 2015??5??6??(??) 8:25 ??: openstack-devopenstack-dev@lists.openstack.org; : [openstack-dev] [heat][ceilometer] autoscaling hey there, Please I wanna know if their is anyway I can have cpu, ram and network meters for each VM returned by ceilometer to heat for autoscaling tasks? In advance, thank you for your response, Sara__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ceilometer][Gnocchi] How to try ceilometer with gnocchi ?
Hi guys, I want to try using ceilometer with gnocchi, but I didn't any docs about how to configure it. I have check the master branch of ceilometer and didn't see how ceilometer interact with gnocchi neither (I think there should be something like a gnocchi-dispatcher?) Any hints? -- Luo gangyiluogan...@chinamobile.com__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ceilometer] Why ceilometer do not offer run_tests.sh script?
Hi guys, When I try to run unit tests of ceilometer, I find there is no run_tests.sh script offers. And when I use tox directly, I got a message ' 'Could not find mongod command'. So another question is why unit tests needs mongo? Can someone give me some hint? Thanks a lot. -- Luo gangyiluogan...@chinamobile.com__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Why ceilometer do notoffer run_tests.sh script?
Thanks for your reply. Yes, we can do as you said. But still a bit weird using database in unit tests :) -- Luo gangyiluogan...@chinamobile.com -- Original -- From: Lu, Lianhao;lianhao...@intel.com; Date: Wed, Apr 29, 2015 10:18 AM To: OpenStack Development Mailing List (not for usage questions)openstack-dev@lists.openstack.org; Subject: Re: [openstack-dev] [Ceilometer] Why ceilometer do notoffer run_tests.sh script? On Apr 29, 2015 09:49, Luo Gangyi wrote: Hi guys, When I try to run unit tests of ceilometer, I find there is no run_tests.sh script offers. And when I use tox directly, I got a message ' 'Could not find mongod command'. Please use setup-test-env-mongodb.sh instead. See tox.ini for details. So another question is why unit tests needs mongo? It's used for the scenario tests on different db backend. Will be moved into functional test though. https://review.openstack.org/#/c/160827/ Can someone give me some hint? -Lianhao Lu __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] ?????? [ceilometer] time based auto scaling
Agree with ZhiQiang. Maybe we could achieve this by heat itself or other project like Mistral, but it seems more natural to achieve this through ceilometer alarm system. -- Luo gangyiluogan...@chinamobile.com -- -- ??: ZhiQiang Fan;aji.zq...@gmail.com; : 2015??4??29??(??) 12:23 ??: OpenStack Development Mailing Listopenstack-dev@lists.openstack.org; : [openstack-dev] [ceilometer] time based auto scaling Hi devs, I'm thinking to add new type of alarm for time based auto scaling, but not sure if there is a better way to achieve it outside ceilometer scope Currently we can auto scaling based on vm load, but it will take several minutes to do it. For the worst case, when the vm load is heavy, ceilometer needs 10 minutes (by default) to collect the performance data, and alarm need 1 minutes to evaluate it, maybe there is evaluation_periods which set to higher that 1 to avoid performance peak. So even though we can collect data by 1 minutes, but evaluation periods may be set to 3, so the worst case is after vm load perfomance in high level for 4 minutes, then the alarm is triggered, then heat will expand vm count, nova will take dozens seconds or more to create, finally the service on the in the heat server group will performance bad or unavailable for several minutes, which is not acceptable for some critical applications. But if we can know some high load which related with time, for example, 08:00am will be a peak, and after 22:00pm will definitely drop to idel level, then heat can increase some vms at 07:30am, then decrease some vms at 22:00 (or decrease by load as normal routine) However, current ceilometer only provide time constraint alarm, which will only evaluate but not guarantee it will be triggered. And heat cannot provide something like timer, but just can wait for the signal. So I propose to add a new type of alarm, which will definitely send a signal to action when it is evaluated (with or without repeat, it will work well with time constraint) Any suggestion?__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Add config option for real deletes insteadof soft-deletes
Hi, Artom I checked my cluster (20 compute nodes fully operated for 5 month, with 258 VMs and 112 users), the datebase size of nova only 1.5MB. So, is it necessary to do the cleanup? -- Luo gangyiluogan...@chinamobile.com -- Original -- From: Artom Lifshitz;alifs...@redhat.com; Date: Wed, Apr 22, 2015 05:42 AM To: openstack-devopenstack-dev@lists.openstack.org; Subject: [openstack-dev] [Nova] Add config option for real deletes insteadof soft-deletes Hello, I'd like to gauge acceptance of introducing a feature that would give operators a config option to perform real database deletes instead of soft deletes. There's definitely a need for *something* that cleans up the database. There have been a few attempts at a DB purge engine [1][2][3][4][5], and archiving to shadow tables has been merged [6] (though that currently has some issues [7]). DB archiving notwithstanding, the general response to operators when they mention the database becoming too big seems to be DIY cleanup. I would like to propose a different approach: add a config option that turns soft-deletes into real deletes, and start telling operators if you turn this on, it's DIY backups. Would something like that be acceptable and feasible? I'm ready to put in the work to implement this, however searching the mailing list indicates that it would be somewhere between non trivial and impossible [8]. Before I start, I would like some confidence that it's closer to the former than the latter :) Cheers! [1] https://blueprints.launchpad.net/nova/+spec/db-purge-engine [2] https://blueprints.launchpad.net/nova/+spec/db-purge2 [3] https://blueprints.launchpad.net/nova/+spec/remove-db-archiving [4] https://blueprints.launchpad.net/nova/+spec/database-purge [5] https://blueprints.launchpad.net/nova/+spec/db-archiving [6] https://review.openstack.org/#/c/18493/ [7] https://review.openstack.org/#/c/109201/ [8] http://lists.openstack.org/pipermail/openstack-operators/2014-November/005591.html __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova] Mislead of Nova Documents in configuring hosts aggregates
Hi, guys, I found a misleading configuration in openstack official documents. When I follow the guide (http://docs.openstack.org/trunk/config-reference/content/section_compute-scheduler.html) to configure nova host aggregates, I got following DEBUG message in nova-schuduler log: 2015-04-16 09:30:47.395 1143 DEBUG nova.scheduler.filters.compute_capabilities_filter [req-66f091c4-c26f-482f-b432-cbd0f2456625 None] (node-132.domain.bcec, node-132.domain.bcec) ram:128519 disk:6240256 io_ops:0 instances:0 fails instance_type extra_specs requirements host_passes /usr/lib/python2.6/site-packages/nova/scheduler/filters/compute_capabilities_filter.py:73 2015-04-16 09:30:47.395 1143 INFO nova.filters [req-66f091c4-c26f-482f-b432-cbd0f2456625 None] Filter ComputeCapabilitiesFilter returned 0 hosts Then I looked the source code, I found this problem was caused by the conflict of ComputeCapabilitiesFilter and AggregateInstanceExtraSpecsFilter if we did not use scope in extra_specs. In official guide, it use $ nova flavor-key ssd.large set ssd=true which may cause conflict with ComputeCapabilitiesFilter?? In fact, we should use $ nova flavor-key ssd.large set aggregate_instance_extra_specs:ssd=true instead. Hope someone can fix this problem in officical documents to avoid other people encountering the same problem. -- Luo gangyiluogan...@chinamobile.com__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] ?????? [Nova][Docs] Mislead of Nova Documents in configuringhosts aggregates
Thanks Lana, I have submitted the bug report(https://bugs.launchpad.net/openstack-manuals/+bug/1445285). If you or anyone else can comfirmed the bug, I will submit a patch to fix it. -- Luo gangyiluogan...@cmss.chinamobile.com -- -- ??: Lana Brindley;openst...@lanabrindley.com; : 2015??4??16??(??) 3:41 ??: OpenStack Development Mailing List (not for usage questions)openstack-dev@lists.openstack.org; : Re: [openstack-dev] [Nova] Mislead of Nova Documents in configuringhosts aggregates Hi Luo, Great spot! This looks to be an error in the documentation, rather than in Nova. If you can please include the information in this email in a bug against the documentation, then someone on the docs team can pick it up. If you want to fix the documentation yourself, you can also do that, by checking out the appropriate repo and making the change (just as you would patch code). Help is available on the wiki here: https://wiki.openstack.org/wiki/Documentation/HowTo Hope that helps, Lana On 16/04/15 16:50, Luo Gangyi wrote: Hi, guys, I found a misleading configuration in openstack official documents. When I follow the guide (http://docs.openstack.org/trunk/config-reference/content/section_compute-scheduler.html) to configure nova host aggregates, I got following DEBUG message in nova-schuduler log: 2015-04-16 09:30:47.395 1143 DEBUG nova.scheduler.filters.compute_capabilities_filter [req-66f091c4-c26f-482f-b432-cbd0f2456625 None] (node-132.domain.bcec, node-132.domain.bcec) ram:128519 disk:6240256 io_ops:0 instances:0 fails instance_type extra_specs requirements host_passes /usr/lib/python2.6/site-packages/nova/scheduler/filters/compute_capabilities_filter.py:73 2015-04-16 09:30:47.395 1143 INFO nova.filters [req-66f091c4-c26f-482f-b432-cbd0f2456625 None] Filter ComputeCapabilitiesFilter returned 0 hosts Then I looked the source code, I found this problem was caused by the conflict of ComputeCapabilitiesFilter and AggregateInstanceExtraSpecsFilter if we did not use scope in extra_specs. In official guide, it use $ nova flavor-key /|ssd.large|/set /|ssd=true|/ which may cause conflict with ComputeCapabilitiesFilter?? In fact, we should use $ nova flavor-key /|ssd.large|/set aggregate_instance_extra_specs:/|ssd=true|/ instead. Hope someone can fix this problem in officical documents to avoid other people encountering the same problem. -- Luo Gangyi luogan...@chinamobile.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Lana Brindley Technical Writer Rackspace Cloud Builders Australia http://lanabrindley.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer]Add hardware pollster of memory buffer and cache
Hi guys, I have submit a blueprint of adding hardware memory buffer and cache pollster. Anyone interested in it, Please review https://blueprints.launchpad.net/ceilometer/+spec/hardware-memory-buffer-and-cache-metrics, Thanks All. -- Luo gangyiluogan...@chinamobile.com -- Original -- From: gordon chung;g...@live.ca; Date: Sat, Mar 21, 2015 04:02 AM To: OpenStack Development Mailing List not for usage questionsopenstack-dev@lists.openstack.org; Subject: Re: [openstack-dev] [Ceilometer]Add hardware pollster of memorybuffer and cache this seems reasonable... this might fall into same category ironic generated metrics (and any other source that has their own defined list of metrics beyond ceilometer's list). there was discussion on how to properly handle these cases [1][2] [1] http://eavesdrop.openstack.org/meetings/ceilometer/2015/ceilometer.2015-03-05-15.01.log.html [2] https://review.openstack.org/#/c/130359/ cheers, gord From: lgy...@foxmail.com To: openstack-dev@lists.openstack.org Date: Thu, 19 Mar 2015 16:44:20 +0800 Subject: [openstack-dev] [Ceilometer]Add hardware pollster of memory buffer and cache Hello everyone, I am using Ceilometer to monitor both physical servers and virtutal machines in IAAS. And I found current Ceilometer only support 4 memory oid of SNMP: _memory_total_oid = 1.3.6.1.4.1.2021.4.5.0 _memory_avail_real_oid = 1.3.6.1.4.1.2021.4.6.0 _memory_total_swap_oid = 1.3.6.1.4.1.2021.4.3.0 _memory_avail_swap_oid = 1.3.6.1.4.1.2021.4.4.0 But in practice, memory cache and buffer are also very useful infomation. So I'd like to add two hardware pollster, MemoryCachePollster and MemoryBufferPollster. And I want to know is there anyone else insterested in it and should I submit blueprint on launchpad? Thanks. -- Luo gangyiluogan...@chinamobile.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ceilometer]Add hardware pollster of memory buffer and cache
Hello everyone, I am using Ceilometer to monitor both physical servers and virtutal machines in IAAS. And I found current Ceilometer only support 4 memory oid of SNMP: _memory_total_oid = 1.3.6.1.4.1.2021.4.5.0 _memory_avail_real_oid = 1.3.6.1.4.1.2021.4.6.0 _memory_total_swap_oid = 1.3.6.1.4.1.2021.4.3.0 _memory_avail_swap_oid = 1.3.6.1.4.1.2021.4.4.0 But in practice, memory cache and buffer are also very useful infomation. So I'd like to add two hardware pollster, MemoryCachePollster and MemoryBufferPollster. And I want to know is there anyone else insterested in it and should I submit blueprint on launchpad? Thanks. -- Luo gangyiluogan...@chinamobile.com__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Real world experience with Ceilometerdeployments - Feedback requested
Hi, maishsk I have deployed serveral test environment in my company's labs and each environment has 20-30 servers. Here were the problems I have met. 1. MongoDB consumes too much memory. I use cgroups to restrict the memory used. but if using hard restriction, MongoDB may be terminated by OOM. 2. Billing data and monitoring data share the same table and database. This is very inconvenient! Monitoring data is massive and there is no need to backup monitoring data. On the contrary, billing data is less and important which should be backuped. Mixing these two data together makes things becomes difficult. 3. It posed too much pressure on MQ when we configured monitoring period into 2-5 seconds. So we configured the pipeline using UDP channel. Next month we will build a openstack environment about 100-150 servers. I will record my configuration, performance data and problems and I am willing to sharethese data and experience in the near future. -- Luo gangyiluogan...@chinamobile.com -- Original -- From: Maish Saidel-Keesing;mais...@maishsk.com; Date: Thu, Feb 12, 2015 03:37 AM To: openstack-devopenstack-dev@lists.openstack.org; openstack-operatorsopenstack-operat...@lists.openstack.org; Subject: [openstack-dev] [Ceilometer] Real world experience with Ceilometerdeployments - Feedback requested Is Ceilometer ready for prime time? I would be interested in hearing from people who have deployed OpenStack clouds with Ceilometer, and their experience. Some of the topics I am looking for feedback on are: - Database Size - MongoDB management, Sharding, replica sets etc. - Replication strategies - Database backup/restore - Overall useability - Gripes, pains and problems (things to look out for) - Possible replacements for Ceilometer that you have used instead If you are willing to share - I am sure it will be beneficial to the whole community. Thanks in Advance With best regards, Maish Saidel-Keesing Platform Architect Cisco __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev