[openstack-dev] FW: Change in openstack/masakari-monitors[master]: Introspective Instance Monitoring through QEMU Guest Agent
Thanks again Tushar and Adam for reviewing 534958. Anything else to get Workflow to +1? Thanks. Louie -Original Message- From: Tushar Patil (Code Review) [mailto:rev...@openstack.org] Sent: Tuesday, July 10, 2018 10:21 PM To: Kwan, Louie Cc: Friesen, Chris; Tim Bell; Waines, Greg; Li Yingjun; Sampath Priyankara (samP); wangqiang-bj; Jean-Philippe Evrard; Young, Ken; Tushar Patil; Andrew Beekhof; Abhishek Kekane; zhangyangyang; takahara.kengo; Rikimaru Honjo; Dinesh Bhor; Michele Baldessari; Adam Spiers Subject: Change in openstack/masakari-monitors[master]: Introspective Instance Monitoring through QEMU Guest Agent Tushar Patil has posted comments on this change. ( https://review.openstack.org/534958 ) Change subject: Introspective Instance Monitoring through QEMU Guest Agent .. Patch Set 9: Code-Review+2 LGTM. Thank you. -- To view, visit https://review.openstack.org/534958 To unsubscribe, visit https://review.openstack.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9efc6afc8d476003d3aa7fee8c31bcaa65438674 Gerrit-PatchSet: 9 Gerrit-Project: openstack/masakari-monitors Gerrit-Branch: master Gerrit-Owner: Louie Kwan Gerrit-Reviewer: Abhishek Kekane Gerrit-Reviewer: Adam Spiers Gerrit-Reviewer: Andrew Beekhof Gerrit-Reviewer: Chris Friesen Gerrit-Reviewer: Dinesh Bhor Gerrit-Reviewer: Greg Waines Gerrit-Reviewer: Hieu LE Gerrit-Reviewer: Jean-Philippe Evrard Gerrit-Reviewer: Ken Young Gerrit-Reviewer: Li Yingjun Gerrit-Reviewer: Louie Kwan Gerrit-Reviewer: Michele Baldessari Gerrit-Reviewer: Rikimaru Honjo Gerrit-Reviewer: Sampath Priyankara (samP) Gerrit-Reviewer: Tim Bell Gerrit-Reviewer: Tushar Patil Gerrit-Reviewer: Tushar Patil Gerrit-Reviewer: Zuul Gerrit-Reviewer: takahara.kengo Gerrit-Reviewer: wangqiang-bj Gerrit-Reviewer: zhangyangyang Gerrit-HasComments: No __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [masakari] Introspective Instance Monitoring through QEMU Guest Agent
Thanks Tushar Patil for the +1 for 547118. In regards of the following review: https://review.openstack.org/#/c/534958/ Any more comment? Thanks. Louie From: Tushar Patil (Code Review) [rev...@openstack.org] Sent: Tuesday, July 03, 2018 8:48 PM To: Kwan, Louie Cc: Tim Bell; zhangyanying; Waines, Greg; Li Yingjun; wangqiang-bj; Tushar Patil; Ken Young; NTT system-fault-ci masakari-integration-ci; wangqiang; Abhishek Kekane; takahara.kengo; Rikimaru Honjo; Adam Spiers; Sampath Priyankara (samP); Dinesh Bhor Subject: Change in openstack/masakari[master]: Introspective Instance Monitoring through QEMU Guest Agent Tushar Patil has posted comments on this change. ( https://review.openstack.org/547118 ) Change subject: Introspective Instance Monitoring through QEMU Guest Agent .. Patch Set 3: Workflow+1 -- To view, visit https://review.openstack.org/547118 To unsubscribe, visit https://review.openstack.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I9efc6afc8d476003d3aa7fee8c31bcaa65438674 Gerrit-PatchSet: 3 Gerrit-Project: openstack/masakari Gerrit-Branch: master Gerrit-Owner: Louie Kwan Gerrit-Reviewer: Abhishek Kekane Gerrit-Reviewer: Adam Spiers Gerrit-Reviewer: Dinesh Bhor Gerrit-Reviewer: Greg Waines Gerrit-Reviewer: Hieu LE Gerrit-Reviewer: Ken Young Gerrit-Reviewer: Li Yingjun Gerrit-Reviewer: Louie Kwan Gerrit-Reviewer: NTT system-fault-ci masakari-integration-ci Gerrit-Reviewer: Rikimaru Honjo Gerrit-Reviewer: Sampath Priyankara (samP) Gerrit-Reviewer: Tim Bell Gerrit-Reviewer: Tushar Patil Gerrit-Reviewer: Tushar Patil Gerrit-Reviewer: Zuul Gerrit-Reviewer: takahara.kengo Gerrit-Reviewer: wangqiang Gerrit-Reviewer: wangqiang-bj Gerrit-Reviewer: zhangyanying Gerrit-HasComments: No __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Telemetry] [ceilometer] Ceilometer-file-publisher-compression-csv-format
Hi All, Any weekly meeting for Telemetry? I would like to discuss what we can do for the next step for the following review? https://review.openstack.org/#/c/562768/ Ping in the IRC a few times and please advice the next step. Thanks. Louie From: Kwan, Louie Sent: Friday, May 04, 2018 10:03 AM To: openstack-dev@lists.openstack.org; julien.dan...@enovance.com Subject: [openstack-dev] [Telemetry] [ceilometer] Ceilometer-file-publisher-compression-csv-format Reaching out to Rocky PTL and others. What could be the next step? Thanks. Louie -Original Message- From: Kwan, Louie [mailto:louie.k...@windriver.com] Sent: Monday, April 23, 2018 4:10 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [ceilometer] Ceilometer-file-publisher-compression-csv-format Submitted the following review on April 19, https://review.openstack.org/#/c/562768/ Would like to know who else could be on the reviewer list and anything else for the next step? Thanks. Louie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [masakari] Masakari Project Meeting time
It seems most of the team members are on vacation this week. By the way, 03:00 UTC is fine as well. -Original Message- From: Bhor, Dinesh [mailto:dinesh.b...@nttdata.com] Sent: Wednesday, April 25, 2018 9:09 PM To: Kwan, Louie Cc: Sampath Priyankara (samP); openstack-dev@lists.openstack.org; Young, Ken Subject: Re: [openstack-dev] [masakari] Masakari Project Meeting time +1 This time may not fit for attendees who work in IST time zone as it will 07.30 AM in the morning. Thanks, Dinesh > On Apr 26, 2018, at 12:06 AM, Kwan, Louie wrote: > > Sampath, Dinesh and others, > > It was a good meeting last week. > > As briefly discussed with Sampath, I would like to check whether we can > adjust the meeting time. > > We are at EST time zone, the meeting is right on our midnight time, 12:00 am. > > It will be nice if the meeting can be started ~2 hours earlier e.g. Could it > be started at 02:00: UTC instead? > > Thanks. > Louie > > Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged,confidential, and proprietary data. If you are not the intended recipient,please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [masakari] Masakari Project Meeting time
Sampath, Dinesh and others, It was a good meeting last week. As briefly discussed with Sampath, I would like to check whether we can adjust the meeting time. We are at EST time zone, the meeting is right on our midnight time, 12:00 am. It will be nice if the meeting can be started ~2 hours earlier e.g. Could it be started at 02:00: UTC instead? Thanks. Louie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [masakari] Introspective Instance Monitoring through QEMU Guest Agent
Submitted the following review on January 17, 2018, https://review.openstack.org/#/c/534958/ Would like to know who else could be on the reviewer list ? or anything else is needed for the next step? Also, I am planning to attend our coming Masakari Weekly meeting, April 24, 0400 UTC in #openstack-meeting and would like add an agenda item to follow up how to move the review forward. Thanks. Louie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [ceilometer] Ceilometer-file-publisher-compression-csv-format
Submitted the following review on April 19, https://review.openstack.org/#/c/562768/ Would like to know who else could be on the reviewer list and anything else for the next step? Thanks. Louie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [devstack] stable/queens: How to configure devstack to use openstacksdk===0.11.3 and os-service-types===1.1.0
In the stable/queens branch, since openstacksdk0.11.3 and os-service-types1.1.0 are described in openstack's upper-constraints.txt, https://github.com/openstack/requirements/blob/stable/queens/upper-constraints.txt#L411 https://github.com/openstack/requirements/blob/stable/queens/upper-constraints.txt#L297 If I do > git clone https://git.openstack.org/openstack-dev/devstack -b stable/queens And then stack.sh We will see it is using openstacksdk-0.12.0 and os_service_types-1.2.0 Having said that, we need the older version, how to configure devstack to use openstacksdk===0.11.3 and os-service-types===1.1.0 Thanks. Louie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [python-masakariclient] Installation issues
It may be related to the profile changes? > /usr/local/lib/python2.7/dist-packages/masakariclient/sdk/ha/connection.py(49)create_connection() -> LOG.debug('Connection: %s', conn) (Pdb) n > /usr/local/lib/python2.7/dist-packages/masakariclient/sdk/ha/connection.py(50)create_connection() -> LOG.debug('masakari client initialized: %s', conn.ha) (Pdb) print LOG (Pdb) n ConfigException: ConfigEx...meters',) > /usr/local/lib/python2.7/dist-packages/masakariclient/sdk/ha/connection.py(50)create_connection() -> LOG.debug('masakari client initialized: %s', conn.ha) (Pdb) n > /usr/local/lib/python2.7/dist-packages/masakariclient/sdk/ha/connection.py(51)create_connection() -> except Exception as e: (Pdb) print e *** NameError: name 'e' is not defined (Pdb) n > /usr/local/lib/python2.7/dist-packages/masakariclient/sdk/ha/connection.py(52)create_connection() -> raise e (Pdb) print e Problem with auth parameters (Pdb) From: Kwan, Louie Sent: Friday, March 09, 2018 1:37 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [python-masakariclient] Installation issues Two issues: 1. Just download the latest and got some issues, cannot ? ubuntu@yow-tic-demo1:~$ masakari segment-list ('Problem with auth parameters', ', mode 'w' at 0x7fe5188681e0>) 2. Documentation: http://docs.openstack.org/developer/python-masakariclient ·Not Found ·The documentation is no long valid Checked the bug list, it seems new issues? Any info will be much appreciated. Thanks. Louie Note: sudo su - stack cd /home/stack git clone https://github.com/openstack/python-masakariclient.git cd python-masakariclient/ sudo python setup.py install source ~/admin-openrc.sh # To check the cli is working or not masakari segment-list __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [python-masakariclient] Installation issues
Two issues: 1. Just download the latest and got some issues, cannot ? ubuntu@yow-tic-demo1:~$ masakari segment-list ('Problem with auth parameters', ', mode 'w' at 0x7fe5188681e0>) 2. Documentation: http://docs.openstack.org/developer/python-masakariclient ·Not Found ·The documentation is no long valid Checked the bug list, it seems new issues? Any info will be much appreciated. Thanks. Louie Note: sudo su - stack cd /home/stack git clone https://github.com/openstack/python-masakariclient.git cd python-masakariclient/ sudo python setup.py install source ~/admin-openrc.sh # To check the cli is working or not masakari segment-list __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [masakari] [masakari-monitors] : Masakari notification failed.
Good for now. The issue should be related to some situations that VM stop instance task is taking longer and it seems one of the periodic task is timing out. To avoid the exception, may try to increase some of the timeout values. Or increase the looping interval for retry… Thanks. LK From: Kwan, Louie Sent: Tuesday, February 20, 2018 5:17 PM To: 'OpenStack Development Mailing List (not for usage questions)' Subject: [openstack-dev] [masakari] [masakari-monitors] : Masakari notification failed. Hi Masakari community, I would like to get your help to understand what may be causing the Masakari notification failed. I do get success cases which the engine got the notification, VM got shutdown and rebooted ok. Having said that, there are some cases that the notification failed and it seems there are some conflicts going on. 20% to 40% chance. Feb 20 21:53:21 masakari-2 masakari-engine[3807]: 2018-02-20 21:53:21.517 WARNING masakari.engine.drivers.taskflow.driver [req-ce909151-1afb-4f2f-abf4-f25d54f25c6b service None] Task 'masakari.engine.drivers.taskflow.instance_failure.StopInstanceTask;instance:recovery' (e85dec06-1498-482c-a63a-51f855745c32) transitioned into state 'FAILURE' from state 'RUNNING' Feb 20 21:53:21 masakari-2 masakari-engine[3807]: 1 predecessors (most recent first): Feb 20 21:53:21 masakari-2 masakari-engine[3807]: Flow 'instance_recovery_engine': Conflict: Conflict Is it normal that masakari notification would be failed because of timing or conflicting events? FYI, I only have one VM and one active notification. Enclosed is the log file I got from the engine. I do appreciate if anyone of you can provide some insight what to do with the failure. Any tip where to look at etc? Timeout? Thanks. Louie | notification_uuid| generated_time | status | type | source_host_uuid | payload | +--++--+--+--+--+ | 42ccee84-0ea5-4163-84a5-028a0bb914a3 | 2018-02-20T21:52:03.00 | failed | VM | 66c8b5b9-03f5-4843-8a9c-fa83af807a9b | {u'instance_uuid': u'565da9ba-3c0c-4087-83ca-32a5a1b00a55', u'vir_domain_event': u'STOPPED_FAILED', u'event': u'QEMU_GUEST_AGENT_ERROR'} | | aa4184f3-b002-4ba8-a403-f22ccd4ce6b5 | 2018-02-20T21:42:54.00 | finished | VM | 66c8b5b9-03f5-4843-8a9c-fa83af807a9b | {u'instance_uuid': u'565da9ba-3c0c-4087-83ca-32a5a1b00a55', u'vir_domain_event': u'STOPPED_FAILED', u'event': u'QEMU_GUEST_AGENT_ERROR'} | __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [libvrit] Can QEMU or LIBVIRT know VM is powering-off
When turning off a VM by doing nova stop, the Status and Task State is there for Nova. But can Libvirt / qemu programmatically figure out the 'Task State' that the VM is trying to powering-off ?. For libvirt, it seems only know the "Power State"? Anyway to read the "powering-off" info? +--+--++--+-++ | ID | Name | Status | Task State | Power State | Networks | +--+--++--+-++ | 09d65498-b1fe-4a99-9f43-4c365a79ff36 | c1 | ACTIVE | -| Running | public=172.24.4.6, 2001:db8::3 | | 565da9ba-3c0c-4087-83ca-32a5a1b00a55 | iim1 | ACTIVE | powering-off | Running | public=172.24.4.5, 2001:db8::7 | +--+--++--+-++ +--+--+-++-++ | ID | Name | Status | Task State | Power State | Networks | +--+--+-++-++ | 09d65498-b1fe-4a99-9f43-4c365a79ff36 | c1 | ACTIVE | - | Running | public=172.24.4.6, 2001:db8::3 | | 565da9ba-3c0c-4087-83ca-32a5a1b00a55 | iim1 | SHUTOFF | - | Shutdown | public=172.24.4.5, 2001:db8::7 | Thanks. Louie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [masakari] [masakari-monitors] : Intrusive Instance Monitoring through QEMU Guest Agent Design Update
Hi Sam Make sense and will do option 2. FYI, we are planning to upload the second iim patch within a week and may do the masakari engine in different patch shortly. Thanks for your reply. Louie From: Sam P [mailto:sam47pr...@gmail.com] Sent: Tuesday, February 20, 2018 3:33 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [masakari] [masakari-monitors] : Intrusive Instance Monitoring through QEMU Guest Agent Design Update Hi Louie, Thank you for patch and Sorry for the delay response. I prefer option 2. From Masakari point of view, this is an instance event. Because, even if some thing wrong inside the VM, Masakari only can try to fix it by restart, rebuilt, migrate... etc the VM. Which are the same recovery work flow for instance failures. Therefore, I prefer option 2 rather than option1. Currently, we are discussing how to implement recovery method customization feature [0] in Masakari. With this feature, you may able to call external workflows for certain failure events. For this feature, different failure models required distinguishable events and option 3 will not be appropriate. [0] https://review.openstack.org/#/c/458023/ > 1. define a new type of event for Intrusive Instance monitoring or > 2. add a new event within the INSTANCE_EVENTS as we may eventually integrate > with instance monitoring or >3.simply reuse the LIFECYCLE/STOPPED_FAILED event ( which is what we are >implementing for now.) --- Regards, Sampath On Fri, Feb 16, 2018 at 12:05 AM, Kwan, Louie mailto:louie.k...@windriver.com>> wrote: We submitted the first implementation patch for the following blueprint https://blueprints.launchpad.net/openstack/?searchtext=intrusive-instance-monitoring i.e. https://review.openstack.org/#/c/534958/ The second patch will be pushed within a week time or so. One item we would like to seek clarification among the community is about how we should integrate the notification within the masakari engine. One option is to reuse what has been defined at masakari/engine/instance_events.py. e.g. def masakari_notifier(self, domain_uuid): if self.getJournalObject(domain_uuid).getSentNotification(): LOG.debug('notifier.send_notification Skipped:' + domain_uuid) else: hostname = socket.gethostname() noticeType = ec.EventConstants.TYPE_VM current_time = timeutils.utcnow() event = { 'notification': { 'type': noticeType, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': 'LIFECYCLE', 'instance_uuid': domain_uuid, 'vir_domain_event': 'STOPPED_FAILED' } } } LOG.debug(str(event)) self.notifier.send_notification(CONF.callback.retry_max, CONF.callback.retry_interval, event) self.getJournalObject(domain_uuid).setSentNotification(True) Should we 1. define a new type of event for Intrusive Instance monitoring or 2. add a new event within the INSTANCE_EVENTS as we may eventually integrate with instance monitoring or 3. simply reuse the LIFECYCLE/STOPPED_FAILED event ( which is what we are implementing for now.) One of our reference test case is to detect application meltdown within VM which QEMU may not aware the failure. The recovery should pretty much be the same as LIFECYCLE/STOPPED_FAILED event. What do you think? Thanks. Louie Ntoe: Here is what we got from masakari/engine/instance_events.py These are the events which needs to be processed by masakari in case of instance recovery failure. """ INSTANCE_EVENTS = { # Add more events and vir_domain_events here. 'LIFECYCLE': ['STOPPED_FAILED'], 'IO_ERROR': ['IO_ERROR_REPORT'] } __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [masakari] [notification api] How to clean up or purging of records
Yeee. Tha't is it. Thanks Louie From: Bhor, Dinesh [dinesh.b...@nttdata.com] Sent: Thursday, February 15, 2018 7:09 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [masakari] [notification api] How to clean up or purging of records Hi Kwan Louie, I think you are looking for this: https://review.openstack.org/#/c/487430/ Thank you, Dinesh Bhor From: Kwan, Louie Sent: 16 February 2018 02:46:14 To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [masakari] [notification api] How to clean up or purging of records Hi All, Just wondering, how can we clean up the masakari notification list or purging all old records in the DB? openstack notification list returns too many old records During semi-auto testing, I created a long list of history of records and would like to clean it up and avoid unnecessary actions. Any short term solution is what I am looking for and/or ideas how to extend the CLI is also welcomed so that some of us can extend it later. Thanks, Louie __ Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [masakari] [notification api] How to clean up or purging of records
Hi All, Just wondering, how can we clean up the masakari notification list or purging all old records in the DB? openstack notification list returns too many old records During semi-auto testing, I created a long list of history of records and would like to clean it up and avoid unnecessary actions. Any short term solution is what I am looking for and/or ideas how to extend the CLI is also welcomed so that some of us can extend it later. Thanks, Louie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [automaton] How to extend automaton?
Thanks for the reply. I will take the subclass approach for now. It will be nice if we can dynamically register additional info or even a callback function after building a machine from a state space listing. -LK From: Joshua Harlow [harlo...@fastmail.com] Sent: Wednesday, February 14, 2018 1:05 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [automaton] How to extend automaton? As far a 1, I'd recommend just use functools.partial or make an object with all the extra stuff u want and have that object provide a __call__ method. As far as 2, you might have to subclass the FSM baseclass and add those into the internal data-structure (same for 3 I think); ie this one @ https://github.com/openstack/automaton/blob/master/automaton/machines.py#L186-L191 Of course feel free to do it differently and submit a patch that folks (myself and others) can review. -Josh Kwan, Louie wrote: > https://github.com/openstack/automaton > > Friendly state machines for python. > > A few questions about automaton. > > 1.I would like to know can we addition parameters on on_enter or on_exit > callbacks. Right now, it seems it only allows state and triggered_event. > > a.I have many FSM running for different objects and it is much easier if > I can pass on the some sort of ID back to the callbacks. > > 2.Can we or how can we store extra attribute like last state change > *timestamp*? > > 3.Can we store additional identify info for the FSM object? Would like > to add an */UUID/* > > Thanks. > > Louie > > def print_on_enter(new_state, triggered_event): > > print("Entered '%s' due to '%s'" % (new_state, triggered_event)) > > def print_on_exit(old_state, triggered_event): > > print("Exiting '%s' due to '%s'" % (old_state, triggered_event)) > > # This will contain all the states and transitions that our machine will > > # allow, the format is relatively simple and designed to be easy to use. > > state_space = [ > > { > > 'name': 'stopped', > > 'next_states': { > > # On event 'play' transition to the 'playing' state. > > 'play': 'playing', > > 'open_close': 'opened', > > 'stop': 'stopped', > > }, > > 'on_enter': print_on_enter, > > 'on_exit': print_on_exit, > > }, > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [masakari] [masakari-monitors] : Intrusive Instance Monitoring through QEMU Guest Agent Design Update
We submitted the first implementation patch for the following blueprint https://blueprints.launchpad.net/openstack/?searchtext=intrusive-instance-monitoring i.e. https://review.openstack.org/#/c/534958/ The second patch will be pushed within a week time or so. One item we would like to seek clarification among the community is about how we should integrate the notification within the masakari engine. One option is to reuse what has been defined at masakari/engine/instance_events.py. e.g. def masakari_notifier(self, domain_uuid): if self.getJournalObject(domain_uuid).getSentNotification(): LOG.debug('notifier.send_notification Skipped:' + domain_uuid) else: hostname = socket.gethostname() noticeType = ec.EventConstants.TYPE_VM current_time = timeutils.utcnow() event = { 'notification': { 'type': noticeType, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': 'LIFECYCLE', 'instance_uuid': domain_uuid, 'vir_domain_event': 'STOPPED_FAILED' } } } LOG.debug(str(event)) self.notifier.send_notification(CONF.callback.retry_max, CONF.callback.retry_interval, event) self.getJournalObject(domain_uuid).setSentNotification(True) Should we 1. define a new type of event for Intrusive Instance monitoring or 2. add a new event within the INSTANCE_EVENTS as we may eventually integrate with instance monitoring or 3. simply reuse the LIFECYCLE/STOPPED_FAILED event ( which is what we are implementing for now.) One of our reference test case is to detect application meltdown within VM which QEMU may not aware the failure. The recovery should pretty much be the same as LIFECYCLE/STOPPED_FAILED event. What do you think? Thanks. Louie Ntoe: Here is what we got from masakari/engine/instance_events.py These are the events which needs to be processed by masakari in case of instance recovery failure. """ INSTANCE_EVENTS = { # Add more events and vir_domain_events here. 'LIFECYCLE': ['STOPPED_FAILED'], 'IO_ERROR': ['IO_ERROR_REPORT'] } __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [automaton] How to extend automaton?
https://github.com/openstack/automaton Friendly state machines for python. A few questions about automaton. 1. I would like to know can we addition parameters on on_enter or on_exit callbacks. Right now, it seems it only allows state and triggered_event. a. I have many FSM running for different objects and it is much easier if I can pass on the some sort of ID back to the callbacks. 2. Can we or how can we store extra attribute like last state change timestamp? 3. Can we store additional identify info for the FSM object? Would like to add an UUID Thanks. Louie def print_on_enter(new_state, triggered_event): print("Entered '%s' due to '%s'" % (new_state, triggered_event)) def print_on_exit(old_state, triggered_event): print("Exiting '%s' due to '%s'" % (old_state, triggered_event)) # This will contain all the states and transitions that our machine will # allow, the format is relatively simple and designed to be easy to use. state_space = [ { 'name': 'stopped', 'next_states': { # On event 'play' transition to the 'playing' state. 'play': 'playing', 'open_close': 'opened', 'stop': 'stopped', }, 'on_enter': print_on_enter, 'on_exit': print_on_exit, }, __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zuul] requirements-check FAILURE
I got it now. Thanks all for the info. -Original Message- From: Andreas Jaeger [mailto:a...@suse.com] Sent: Thursday, January 18, 2018 2:43 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Zuul] requirements-check FAILURE On 2018-01-17 23:01, Kwan, Louie wrote: > Would like to add the following module to openstack.masakari project > > https://github.com/pytransitions/transitions > > Got the following error with zuul requirements-check > > Requirement set([Requirement(package=u'transitions', location='', > specifiers='>=0.6.4', markers=u'', comment='', extras=frozenset([]))]) not in > openstack/requirements > > http://logs.openstack.org/88/534888/3/check/requirements-check/edec7bf/ara/ > > Any tip or insight to fix it? Yes, read on how to add it: https://docs.openstack.org/requirements/latest/ Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] requirements-tox-validate-projects FAILURE
Thanks Clark. Got the +1 from zuul. By the way, I also changed to use openstack/automaton instead of transitions. It is all good now. Louie -Original Message- From: Clark Boylan [mailto:cboy...@sapwetik.org] Sent: Thursday, January 18, 2018 5:28 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [requirements] requirements-tox-validate-projects FAILURE On Thu, Jan 18, 2018, at 1:54 PM, Kwan, Louie wrote: > Would like to add the following module to openstack.masakari project > > https://github.com/pytransitions/transitions > > https://review.openstack.org/#/c/534990/ > > requirements-tox-validate-projects failed: > > http://logs.openstack.org/90/534990/6/check/requirements-tox-validate-projects/ed69273/ara/result/4ee4f7a1-456c-4b89-933a-fe282cf534a3/ > > What else need to be done? Reading the log [0] the job failed because python-cratonclient removed its check-requirements job. This was done in https://review.openstack.org/#/c/535344/ as part of the craton retirement and should be fixed on the requirements side by https://review.openstack.org/#/c/535351/. I think a recheck at this point will come back green (so I have done that for you). [0] http://logs.openstack.org/90/534990/6/check/requirements-tox-validate-projects/ed69273/job-output.txt.gz#_2018-01-18_20_07_54_531014 Hope this helps, Clark __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [requirements] requirements-tox-validate-projects FAILURE
Would like to add the following module to openstack.masakari project https://github.com/pytransitions/transitions https://review.openstack.org/#/c/534990/ requirements-tox-validate-projects failed: http://logs.openstack.org/90/534990/6/check/requirements-tox-validate-projects/ed69273/ara/result/4ee4f7a1-456c-4b89-933a-fe282cf534a3/ What else need to be done? Thanks. louie.k...@windriver.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Zuul] requirements-check FAILURE
Would like to add the following module to openstack.masakari project https://github.com/pytransitions/transitions Got the following error with zuul requirements-check Requirement set([Requirement(package=u'transitions', location='', specifiers='>=0.6.4', markers=u'', comment='', extras=frozenset([]))]) not in openstack/requirements http://logs.openstack.org/88/534888/3/check/requirements-check/edec7bf/ara/ Any tip or insight to fix it? Thanks. louie.k...@windriver.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [masakari] problems starting up masakari instance monitoring in devstack @ master
Hi Dinesh, Thanks for the info. 'recovery_method' (choose from 'auto', 'reserved_host', 'auto_priority', 'rh_priority') What should be put for Same as the segment host What should be put as , it seems it accepts any arbitrary values. Any more pointers. Much appreciated. Thank you. Louie Kwan From: Bhor, Dinesh [dinesh.b...@nttdata.com] Sent: Wednesday, December 13, 2017 11:22 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [masakari] problems starting up masakari instance monitoring in devstack @ master Hi Greg, Looks like you don’t have “devstack-masakari” host registered in Masakari database. Currently operator have to add all the hosts from failover-segment manually to Masakari database. You can use below command: Register failover-segment to Masakari database: openstack segment create Register hosts under the created segment to Masakari database: openstack segment host create There is a work in progress on auto compute node registration. Please refer: https://blueprints.launchpad.net/masakari-monitors/+spec/auto-compute-node-registration Thank you, Dinesh Bhor From: Waines, Greg Sent: 13 December 2017 19:26:32 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [masakari] problems starting up masakari instance monitoring in devstack @ master ok ... i changed all the domain related attributes in /etc/masakarimonitors/masakarimonitors.conf to be set to ‘default’ . I seemed to get further, but now get the following error when instancemonitor tries to send a notification: Bad Request (HTTP 400) (Request-ID: req-556c6c4d-0e12-414e-ac8c-8ef3a61d4864), Host with name devstack-masakari could not be found. however: - devstack-masakari is in the /etc/hosts - nova hypervisor-list shows devstack-masakari as a hypervisor hostname - i am running both hostmonitor, processmonitor and instancemonitor any ideas ? see details below, Greg. 2017-12-13 13:50:34.353 7637 INFO masakarimonitors.instancemonitor.libvirt_handler.callback [-] Libvirt Event: type=VM, hostname=devstack-masakari, uuid=6ae0b09b-3e93-4f0c-b81b-fa140636f267, time=2017-12-13 13:50:34.353037, event_id=LIFECYCLE, detail=STOPPED_DESTROYED) 2017-12-13 13:50:34.353 7637 INFO masakarimonitors.ha.masakari [-] Send a notification. {'notification': {'hostname': 'devstack-masakari', 'type': 'VM', 'payload': {'instance_uuid': '6ae0b09b-3e93-4f0c-b81b-fa140636f267', 'vir_domain_event': 'STOPPED_DESTROYED', 'event': 'LIFECYCLE'}, 'generated_time': datetime.datetime(2017, 12, 13, 13, 50, 34, 353037)}} 2017-12-13 13:50:34.461 7637 WARNING masakarimonitors.ha.masakari [-] Retry sending a notification. (HttpException: Bad Request (HTTP 400) (Request-ID: req-556c6c4d-0e12-414e-ac8c-8ef3a61d4864), Host with name devstack-masakari could not be found.): HttpException: HttpException: Bad Request (HTTP 400) (Request-ID: req-556c6c4d-0e12-414e-ac8c-8ef3a61d4864), Host with name devstack-masakari could not be found. ^C2017-12-13 13:50:42.462 7637 INFO oslo_service.service [-] Caught SIGINT signal, instantaneous exiting root@devstack-masakari:~# root@devstack-masakari:~# ping devstack-masakari PING localhost (127.0.0.1) 56(84) bytes of data. 64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.049 ms 64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.022 ms ^C --- localhost ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 999ms rtt min/avg/max/mdev = 0.022/0.035/0.049/0.014 ms root@devstack-masakari:~# stack@devstack-masakari:~$ nova hypervisor-list +--+-+---+-+ | ID | Hypervisor hostname | State | Status | +--+-+---+-+ | 5fb1b09b-e5f5-465a-828a-2101135ff700 | devstack-masakari | up| enabled | +--+-+---+-+ stack@devstack-masakari:~$ From: Greg Waines Reply-To: "openstack-dev@lists.openstack.org" Date: Wednesday, December 13, 2017 at 8:17 AM To: "openstack-dev@lists.openstack.org" Subject: Re: [openstack-dev] [masakari] problems starting up masakari instance monitoring in devstack @ master So i believe the error is: HttpException: HttpException: Expecting to find domain in project. I have attached the /etc/masakarimonitors/masakarimonitors.conf file and the /etc/masakari/masakari.conf file . See the domain related attributes in each file below. Is the Default vs default causing this problem ? Are there other domain related attributes that should be set to default ? stack@devstack-masakari:~$ fgrep -i domain /etc/masakari/masakari.conf project_domain_name = Default user_domain_name = Defa
[openstack-dev] [all] How to use libxml2 with tox
Would like to use libxml2 and having issues for tox. What needs to be included in the requirements.txt file etc. Any tip is much appreciated. Thanks. Louie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev