Re: [Engine-devel] SSH Soft Fencing

[Livnat Peer]

On 06/30/2013 07:35 PM, Barak Azulay wrote:
> 
> 
> - Original Message -
>> From: "Barak Azulay" 
>> To: "Martin Perina" 
>> Cc: "Yair Zaslavsky" , engine-devel@ovirt.org, "Eli 
>> Mesika" 
>> Sent: Thursday, June 27, 2013 8:31:35 PM
>> Subject: Re: SSH Soft Fencing
>>
>>
>>
>> - Original Message -
>>> From: "Eli Mesika" 
>>> To: "Yair Zaslavsky" 
>>> Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
>>> Azulay" 
>>> Sent: Thursday, June 27, 2013 5:55:29 PM
>>> Subject: Re: SSH Soft Fencing
>>>
>>>
>>>
>>> - Original Message -
 From: "Yair Zaslavsky" 
 To: "Eli Mesika" 
 Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
 Azulay" 
 Sent: Thursday, June 27, 2013 5:43:17 PM
 Subject: Re: SSH Soft Fencing



 - Original Message -
> From: "Eli Mesika" 
> To: "Martin Perina" 
> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" ,
> "Barak
> Azulay" 
> Sent: Thursday, June 27, 2013 3:48:39 PM
> Subject: Re: SSH Soft Fencing
>
>
>
> - Original Message -
>> From: "Martin Perina" 
>> To: engine-devel@ovirt.org
>> Cc: "Yair Zaslavsky" , "Barak Azulay"
>> , "Eli Mesika" 
>> Sent: Thursday, June 27, 2013 1:51:06 PM
>> Subject: SSH Soft Fencing
>>
>> Hi,
>>
>> SSH Soft Fencing is a new feature for 3.3 and it tries to restart
>> VDSM
>> using SSH connection on non responsive hosts prior to real fencing.
>> More info can be found at
>>
>> http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
>>
>> In current SSH Soft Fencing implementation the restart VDSM using SSH
>> command is part of standard fencing implementation in
>> VdsNotRespondingTreatmentCommand. But this command is executed only
>> if a host has a valid PM configuration. If host doesn't have a valid
>> PM configuration, the execution of the command is disabled and host
>> state is change to Non Responsive.
>>
>> So my question are:
>>
>> 1) Should SSH Soft Fencing be executed on hosts without valid PM
>>configuration?
>
> I think that the answer should be yes. The vdsm restart will solve most
> of
> problems , so why not using it whether a PM agent is defined or not.
 I agree.
 I would like to say that I also don't like the fact that
 VdsNotRespondingTreatment extends RestartVdsCommand.
 One should ask if "non responding treatment is a restart vds operation"
 or
 maybe RestartVdsCommand is just a step in the non responding treatment
 (inheritance vs containment/delegation).
 I think that VdsNotRespodingTreatment should delegate the call to
 RestartVdsCommand as the 2nd step after issuing the Soft Fencing command.
 Thoughts anyone?
>>>
>>> That would be a nice and needed re-factoring
>>
>> I would say yes - but would add it only with appropriate configuration
>> (enableAutoSoftVdsmRestartWhenNoPMAvailable  I hate the name)
>>
>>  
>>
>>>

>
>>
>> 2) Should VDSM restart using SSH command be reimplemented
>>as standalone command to be usable also in other parts of engine?
>>If 1) is true, I think it will have to be done anyway.

 I agree here.
>
> +1
>>
>> On one hand it makes sense,  but I have several questions on the above:
>> - Who do we think may want to use such a command ?


I believe you'll agree that right encapsulation and decoupling is part
of writing a maintainable code, it is not necessarily about reusing it.

>> - Should (or even can) we limit the use of such command to
>> noneResponsiveTreatment ?
>>

At this point this command would be executed only within the
noneResponsiveTreatment flow.
We don't need to model this in the REST API nor in the UI, decoupling
the vdsm fencing code is just an internal implementation detail.


>> Having general commands available to all code when there is only one specific
>> case we are using it might be a bit riskey,
>> Especially when we talk about restarting something.
> 

I am not sure what is the risk?

> Martin ? Eli? Yair?
> 
> Can you please refer to the issue above ?
> 
> 
>>
>> Thoughts ?
>>
>>
>>
>
>>
>>
>> Martin Perina
>>
>

>>>
>>
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] VNIC profiles

[Livnat Peer]

On 07/01/2013 12:19 AM, Eli Mesika wrote:
> 
> 
> - Original Message -
>> From: "Livnat Peer" 
>> To: "engine-devel" , "Ofri Masad" 
>> Sent: Sunday, June 30, 2013 3:59:37 PM
>> Subject: [Engine-devel] VNIC profiles
>>
>> Hi,
>>
>> We are working on adding VNIC profiles as part of the work to add VNIC QoS.
>>
>> http://www.ovirt.org/Features/Network_QoS#VNIC_Profiles
>>
>> We need to define some of the system behavior followed by this change,
>> here is my take -
>>
>> Editing a profile -
>> 
>> A user should be able to edit the profile properties (including profile
>> name) while VMs are attached and are using this Profile (reference
>> should be done by id).
>>
>> Changing the network though is a bit more tricky as long as we don't
>> have a way to distinguish between running and current configurations I
>> think it could be very confusing to the user. Especially since we
>> support dynamic wiring so the behavior IMO is unpredictable.
>> I think it should be blocked at this point.
>>
>>
>> Edit a VNIC / change a VNIC profile -
>> 
>> Changing the profile a VM is using while the VM is running should behave
>> like dynamic wiring (changing the VM network while it is running).
>>
>>
>> Remove a Profile -
>> ---
>> Is only valid if all VMs that are using this profile are in status down.
> 
> What about HA VMs , are they forced to be down as well for this operation?
> 

If you want to remove the profile you can remove it from the HA VM (hot
unplug NIC for example) and then delete the profile. You can do that
when the VM is up and running.



>> It should update all VMs to point to no profile which should behave like
>> none network today.
>>
>> I see no reason to support a profile on a none network at this point.
>>
>> The above is also relevant for upgrade flow (upgrading none network to
>> point to no profile)
>>
>>
>> Removing a Network -
>> --
>> should remove all profiles on that network
>>
>>
>> VM snapshot/import/export -
>> --
>> We should handle VMs that are pointing to a network directly for b/w
>> compatibility.
>> we need to select first profile that is on that network that the user
>> has permissions on.
>>
>>
>> I assume there are more, comments are welcome
>>
>> Thanks, Livnat
>>
>> ___
>> Engine-devel mailing list
>> Engine-devel@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Spice-devel] SPICE Foreign Menu Using REST

[Tomas Jelinek]

- Original Message -
> From: "Einav Cohen" 
> To: "Tomas Jelinek" , "Vojtech Szocs" 
> Cc: "engine-devel" 
> Sent: Friday, June 28, 2013 6:01:16 PM
> Subject: Re: [Engine-devel] [Spice-devel] SPICE Foreign Menu Using REST
> 
> Hi Tomas,
> 
> > IIUC the RFE addresses only UIPlugins with their metadata so
> > in order to integrate this with the SPICE we nee to either
> > enrich the RFE or to create a UIPlugin which will start the
> > SPICE. I would vote for the second option. What others?
> 
> using a ui-plugin that will start SPICE is an interesting approach - it
> haven't crossed my mind. What I had in mind is to somewhat "generalize" the
> RestApiSessionManager to provide REST-API-session-creation services to the
> entire GUI (e.g. SPICE-console-opening-code), and not only to the ui-plugins
> in particular.
> I think that since SPICE is already quite integrated into oVirt (in our
> business
> logic, dialogs, ...), it would be somewhat weird to use a UI-Plugin in order
> to
> invoke SPICE, just in order to get the REST-API-session-creation capability.
> 
> So I am actually in favor of enriching the RFE (i.e. "generalize" the
> RestApiSessionManager), rather than treating SPICE as a UI-Plugin (in a
> sense).

yes, you are right.

> 
> > Why only for SPICE? I can imagine UIPlugins which could make use of this
> > option.
> 
> the main difference between SPICE and the UIPlugins is that we know in
> advance
> how SPICE is going to use REST API. 

This is not true anymore - SPICE can add whatever it wants to it's menu and 
webadmin/userportal
has no control about it anymore (I'm talking about the non-plugin invocation of 
the SPICE).

> In addition, SPICE is going to do actions
> (e.g.
> Stop/Pause/Attach CD) on the VM only when the user chooses to do so (via the
> SPICE menu), so actions will be done on behalf of the logged-in user, so it
> makes
> sense to provide to SPICE the same credentials as the logged-in user.
> 
> a ui-plugin is a third-party component which we don't know in advance how it
> is
> going to act; moreover, the user that is logging into the web-admin doesn't
> have
> control on it. Imagine a ui-plugin that shuts-down/deletes all VMs
> immediately
> upon login to the web-admin - all VMs will be shutdown/deleted upon the first
> login into the web-admin (after the ui-plugin installation on the engine
> server).
> If the same-credentials-approach will be used, the shutdown/deletion of all
> VMs
> will be done on behalf of the user that happened to be the first one logging
> into
> the web-admin (post ui-plugin installation), which, obviously, is not good.

Well, I did not mean to give this permission to all the plugins - all I wanted 
to
say is to make this configurable per plugin. I can imagine 
also a well-known UI plugin which is trusted by the admin and it does make sense
to make it use the same rights as the logged in user.
I did not mean that each plugin has to have this permissions.

> 
> So to sum-up: I think that
> 
> - the RestApiSessionManager should be "generalized" to provide
> REST-API-session-
> creation services to the entire GUI (e.g. SPICE-console-opening-code), and
> not only
> to the ui-plugins in particular.

completely agree

> 
> - RestApiSessionManager should support creation of both
> same-credentials-session as
> well as other-credentials-session; the same-credentials-session should be
> limited to
> well-known/well-integrated components (e.g. SPICE), whereas the
> other-credentials-session
> can be used by third-party components as well (e.g. ui-plugins).

Who should decide about what is the well-known component? I would let the 
administrator to
configure this. We can provide the defaults (e.g. the same-credentials-session 
will be disabled by
default for ui-plugins but it will be possible to enabled it explicitly for 
some ui-plugins). 

> 
> [I could be wrong - would love to hear what the others GUI maintainers think]
> 
> 
> > - Original Message -
> > From: "Tomas Jelinek" 
> > Sent: Friday, June 28, 2013 1:41:11 AM
> > 
> > 
> > 
> > - Original Message -
> > > From: "Einav Cohen" 
> > > To: "Tomas Jelinek" , "Vojtech Szocs"
> > > 
> > > Cc: "engine-devel" , "Michael Pasternak"
> > > 
> > > Sent: Thursday, June 27, 2013 6:01:13 PM
> > > Subject: Re: [Engine-devel] [Spice-devel] SPICE Foreign Menu Using REST
> > > 
> > > Hi Tomas,
> > > 
> > > > Yes, we can provide the sessionId to authenticate with the REST and the
> > > > vm
> > > > guid is not a problem.
> > > 
> > > Note that in the web-admin, we already have code that generates REST API
> > > session-ID;
> > > this code is being utilized in the ui-plugins infrastructure to allow the
> > > different
> > > ui-plugins to communicate with the rest api.
> > > [one related file is this context is RestApiSessionManager.java in the
> > > web-admin, not
> > > sure if there are others]
> > 
> > Yes, I'm aware of that. This is what I had in mind when was talking about
> > passing the SessionId

[Engine-devel] Build step 'Publish FindBugs analysis results' changed build result to UNSTABLE

[Chen, Wei D]

Is there anyone know the reason for these error message? what can I do to fix 
it?


[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ 
engine-server-ear ---
[INFO] Installing 
/jenkins-workspaces/ovirt_engine_find_bugs_gerrit/ovirt-engine/ear/target/engine.ear
 to 
/home/jenkins/workspace/ovirt_engine_find_bugs_gerrit/.repository/org/ovirt/engine/engine-server-ear/3.3.0-SNAPSHOT/engine-server-ear-3.3.0-SNAPSHOT.ear
[INFO] Installing 
/jenkins-workspaces/ovirt_engine_find_bugs_gerrit/ovirt-engine/ear/pom.xml to 
/home/jenkins/workspace/ovirt_engine_find_bugs_gerrit/.repository/org/ovirt/engine/engine-server-ear/3.3.0-SNAPSHOT/engine-server-ear-3.3.0-SNAPSHOT.pom
[INFO] 
[INFO] --- findbugs-maven-plugin:2.5.2:findbugs (default-cli) @ 
engine-server-ear ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] ovirt-root  SUCCESS [4.719s]
[INFO] oVirt Build Tools root  SUCCESS [0.129s]
[INFO] oVirt checkstyle .. SUCCESS [2.001s]
[INFO] oVirt JBoss Modules Maven Plugin .. SUCCESS [39.769s]
[INFO] oVirt Checkstyle Checks ... SUCCESS [27.944s]
[INFO] oVirt Modules - backend ... SUCCESS [0.054s]
[INFO] oVirt Manager . SUCCESS [0.052s]
[INFO] oVirt DB Scripts .. SUCCESS [0.210s]
[INFO] oVirt Engine dependencies . SUCCESS [2.262s]
[INFO] oVirt Modules - manager ... SUCCESS [1.150s]
[INFO] CSharp Compatibility .. SUCCESS [36.770s]
[INFO] Common Code ... SUCCESS [1:43.000s]
[INFO] Common utilities .. SUCCESS [1:15.054s]
[INFO] Data Access Layer . SUCCESS [1:45.658s]
[INFO] engine scheduler bean . SUCCESS [33.039s]
[INFO] Vds broker  SUCCESS [1:41.045s]
[INFO] Search Backend  SUCCESS [48.983s]
[INFO] Backend Logic @Service bean ... SUCCESS [3:15.813s]
[INFO] oVirt RESTful API Backend Integration . SUCCESS [0.242s]
[INFO] oVirt RESTful API interface ... SUCCESS [0.334s]
[INFO] oVirt Engine API Definition ... SUCCESS [1:04.592s]
[INFO] oVirt Engine API Commom Parent POM  SUCCESS [0.195s]
[INFO] oVirt Engine API Common JAX-RS  SUCCESS [46.398s]
[INFO] oVirt RESTful API Backend Integration Type Mappers  SUCCESS [1:06.535s]
[INFO] oVirt RESTful API Backend Integration JAX-RS Resources  SUCCESS 
[1:45.332s]
[INFO] oVirt RESTful API Backend Integration Webapp .. SUCCESS [1.559s]
[INFO] oVirt Engine Web Root . SUCCESS [38.216s]
[INFO] oVirt Engine Tools  SUCCESS [54.388s]
[INFO] oVirt Modules :: Frontend . SUCCESS [0.034s]
[INFO] oVirt Modules :: Webadmin . SUCCESS [0.042s]
[INFO] oVirt Modules - ui  SUCCESS [0.225s]
[INFO] Extensions for GWT  SUCCESS [27.754s]
[INFO] UI Utils Compatibility (for UICommon) . SUCCESS [32.946s]
[INFO] Frontend for GWT UI Projects .. SUCCESS [42.351s]
[INFO] UICommonWeb ... SUCCESS [2:52.851s]
[INFO] oVirt GWT UI common infrastructure  SUCCESS [2:30.819s]
[INFO] WebAdmin .. SUCCESS [3:36.038s]
[INFO] UserPortal  SUCCESS [1:31.579s]
[INFO] oVirt Server EAR .. SUCCESS [6.661s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 31:41.803s
[INFO] Finished at: Fri Jun 28 12:14:28 EDT 2013
[INFO] Final Memory: 182M/806M
[INFO] 
[FINDBUGS] Collecting findbugs analysis files...
[FINDBUGS] Finding all files that match the pattern **/findbugsXml.xml
[FINDBUGS] Parsing 23 files in 
/home/jenkins/workspace/ovirt_engine_find_bugs_gerrit
[FINDBUGS] Successfully parsed file 
/home/jenkins/workspace/ovirt_engine_find_bugs_gerrit/ovirt-engine/backend/manager/modules/auth/target/findbugsXml.xml
 of module  with 2 warnings.
[FINDBUGS] Successfully parsed file 
/home/jenkins/workspace/ovirt_engine_find_bugs_gerrit/ovirt-engine/backend/manager/modules/bll/target/findbugsXml.xml
 of module  with 177 warnings.
[FINDBUGS] Successfully parsed file 
/home/jenkins/workspace/ovirt_engine_find_bugs_gerrit/ovirt-engine/backend/manager/modules/commo

[Engine-devel] [oVirt jenkins] Weekly report on open tasks for ovirt-engine

[Jenkins ci oVirt Server]

Files scanned: '**/*.java, **/*.py'. 
Strings searched: FIXME | TODO | @deprecated 

 

Report: http://jenkins.ovirt.org/job/ovirt_engine_scan_open_tasks/9/tasksResult/?___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] VNIC profiles

[Eli Mesika]

- Original Message -
> From: "Livnat Peer" 
> To: "engine-devel" , "Ofri Masad" 
> Sent: Sunday, June 30, 2013 3:59:37 PM
> Subject: [Engine-devel] VNIC profiles
> 
> Hi,
> 
> We are working on adding VNIC profiles as part of the work to add VNIC QoS.
> 
> http://www.ovirt.org/Features/Network_QoS#VNIC_Profiles
> 
> We need to define some of the system behavior followed by this change,
> here is my take -
> 
> Editing a profile -
> 
> A user should be able to edit the profile properties (including profile
> name) while VMs are attached and are using this Profile (reference
> should be done by id).
> 
> Changing the network though is a bit more tricky as long as we don't
> have a way to distinguish between running and current configurations I
> think it could be very confusing to the user. Especially since we
> support dynamic wiring so the behavior IMO is unpredictable.
> I think it should be blocked at this point.
> 
> 
> Edit a VNIC / change a VNIC profile -
> 
> Changing the profile a VM is using while the VM is running should behave
> like dynamic wiring (changing the VM network while it is running).
> 
> 
> Remove a Profile -
> ---
> Is only valid if all VMs that are using this profile are in status down.

What about HA VMs , are they forced to be down as well for this operation?

> It should update all VMs to point to no profile which should behave like
> none network today.
> 
> I see no reason to support a profile on a none network at this point.
> 
> The above is also relevant for upgrade flow (upgrading none network to
> point to no profile)
> 
> 
> Removing a Network -
> --
> should remove all profiles on that network
> 
> 
> VM snapshot/import/export -
> --
> We should handle VMs that are pointing to a network directly for b/w
> compatibility.
> we need to select first profile that is on that network that the user
> has permissions on.
> 
> 
> I assume there are more, comments are welcome
> 
> Thanks, Livnat
> 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] SSH Soft Fencing

[Eli Mesika]

- Original Message -
> From: "Barak Azulay" 
> To: "Martin Perina" 
> Cc: "Yair Zaslavsky" , engine-devel@ovirt.org, "Eli 
> Mesika" 
> Sent: Sunday, June 30, 2013 7:35:22 PM
> Subject: Re: SSH Soft Fencing
> 
> 
> 
> - Original Message -
> > From: "Barak Azulay" 
> > To: "Martin Perina" 
> > Cc: "Yair Zaslavsky" , engine-devel@ovirt.org, "Eli
> > Mesika" 
> > Sent: Thursday, June 27, 2013 8:31:35 PM
> > Subject: Re: SSH Soft Fencing
> > 
> > 
> > 
> > - Original Message -
> > > From: "Eli Mesika" 
> > > To: "Yair Zaslavsky" 
> > > Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
> > > Azulay" 
> > > Sent: Thursday, June 27, 2013 5:55:29 PM
> > > Subject: Re: SSH Soft Fencing
> > > 
> > > 
> > > 
> > > - Original Message -
> > > > From: "Yair Zaslavsky" 
> > > > To: "Eli Mesika" 
> > > > Cc: "Martin Perina" , engine-devel@ovirt.org,
> > > > "Barak
> > > > Azulay" 
> > > > Sent: Thursday, June 27, 2013 5:43:17 PM
> > > > Subject: Re: SSH Soft Fencing
> > > > 
> > > > 
> > > > 
> > > > - Original Message -
> > > > > From: "Eli Mesika" 
> > > > > To: "Martin Perina" 
> > > > > Cc: engine-devel@ovirt.org, "Yair Zaslavsky" ,
> > > > > "Barak
> > > > > Azulay" 
> > > > > Sent: Thursday, June 27, 2013 3:48:39 PM
> > > > > Subject: Re: SSH Soft Fencing
> > > > > 
> > > > > 
> > > > > 
> > > > > - Original Message -
> > > > > > From: "Martin Perina" 
> > > > > > To: engine-devel@ovirt.org
> > > > > > Cc: "Yair Zaslavsky" , "Barak Azulay"
> > > > > > , "Eli Mesika" 
> > > > > > Sent: Thursday, June 27, 2013 1:51:06 PM
> > > > > > Subject: SSH Soft Fencing
> > > > > > 
> > > > > > Hi,
> > > > > > 
> > > > > > SSH Soft Fencing is a new feature for 3.3 and it tries to restart
> > > > > > VDSM
> > > > > > using SSH connection on non responsive hosts prior to real fencing.
> > > > > > More info can be found at
> > > > > > 
> > > > > > http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
> > > > > > 
> > > > > > In current SSH Soft Fencing implementation the restart VDSM using
> > > > > > SSH
> > > > > > command is part of standard fencing implementation in
> > > > > > VdsNotRespondingTreatmentCommand. But this command is executed only
> > > > > > if a host has a valid PM configuration. If host doesn't have a
> > > > > > valid
> > > > > > PM configuration, the execution of the command is disabled and host
> > > > > > state is change to Non Responsive.
> > > > > > 
> > > > > > So my question are:
> > > > > > 
> > > > > > 1) Should SSH Soft Fencing be executed on hosts without valid PM
> > > > > >configuration?
> > > > > 
> > > > > I think that the answer should be yes. The vdsm restart will solve
> > > > > most
> > > > > of
> > > > > problems , so why not using it whether a PM agent is defined or not.
> > > > I agree.
> > > > I would like to say that I also don't like the fact that
> > > > VdsNotRespondingTreatment extends RestartVdsCommand.
> > > > One should ask if "non responding treatment is a restart vds operation"
> > > > or
> > > > maybe RestartVdsCommand is just a step in the non responding treatment
> > > > (inheritance vs containment/delegation).
> > > > I think that VdsNotRespodingTreatment should delegate the call to
> > > > RestartVdsCommand as the 2nd step after issuing the Soft Fencing
> > > > command.
> > > > Thoughts anyone?
> > > 
> > > That would be a nice and needed re-factoring
> > 
> > I would say yes - but would add it only with appropriate configuration
> > (enableAutoSoftVdsmRestartWhenNoPMAvailable  I hate the name)
> > 
> >  
> > 
> > > 
> > > > 
> > > > > 
> > > > > > 
> > > > > > 2) Should VDSM restart using SSH command be reimplemented
> > > > > >as standalone command to be usable also in other parts of
> > > > > >engine?
> > > > > >If 1) is true, I think it will have to be done anyway.
> > > > 
> > > > I agree here.
> > > > > 
> > > > > +1
> > 
> > On one hand it makes sense,  but I have several questions on the above:
> > - Who do we think may want to use such a command ?
> > - Should (or even can) we limit the use of such command to
> > noneResponsiveTreatment ?

Yes, I think we should limit that to noneResponsiveTreatment

> > 
> > Having general commands available to all code when there is only one
> > specific
> > case we are using it might be a bit riskey,
> > Especially when we talk about restarting something.

We can keep this internal and not expose it to the user, just implement 
explicitly in non responding treatment 

> 
> Martin ? Eli? Yair?
> 
> Can you please refer to the issue above ?
> 
> 
> > 
> > Thoughts ?
> > 
> > 
> > 
> > > > > 
> > > > > > 
> > > > > > 
> > > > > > Martin Perina
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] development installation issues

[Alon Bar-Lev]

- Original Message -
> From: "Michael Pasternak" 
> To: "engine-devel" 
> Sent: Sunday, June 30, 2013 11:03:06 AM
> Subject: [Engine-devel] development installation issues
> 
> 
> does anyone faced this with new 'development installation'?

Hi Michael,

Can I have the output of:

OTOPI_DEBUG=1 /home/mpastern/Coding/ovirt/ovirt-engine/bin/engine-setup-2

Thanks!

> 
> [mpastern@lpt21-f ovirt-engine]$
> /home/mpastern/Coding/ovirt/ovirt-engine/bin/engine-setup-2
> [ ERROR ] Failed to execute stage 'Booting': 5
> [ INFO  ] Stage: Initializing
>   Setup was run under unprivileged user this will produce development
>   installation do you wish to proceed? (Yes, No) [No]: Yes
> [WARNING] engine-setup-2 is a technical preview, and yet to include all
> functionality that exists in legacy engine-setup. Specifically,
> engine-setup-2 does not support
> upgrade from previous installations.
> [mpastern@lpt21-f ovirt-engine]$
> 
> --
> 
> Michael Pasternak
> RedHat, ENG-Virtualization R&D
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] SSH Soft Fencing

[Livnat Peer]

On 06/30/2013 07:26 PM, Barak Azulay wrote:
> 
> 
> - Original Message -
>> From: "Dan Kenigsberg" 
>> To: "Eli Mesika" 
>> Cc: engine-devel@ovirt.org
>> Sent: Sunday, June 30, 2013 5:40:49 PM
>> Subject: Re: [Engine-devel] SSH Soft Fencing
>>
>> On Thu, Jun 27, 2013 at 08:48:39AM -0400, Eli Mesika wrote:
>>>
>>>
>>> - Original Message -
 From: "Martin Perina" 
 To: engine-devel@ovirt.org
 Cc: "Yair Zaslavsky" , "Barak Azulay"
 , "Eli Mesika" 
 Sent: Thursday, June 27, 2013 1:51:06 PM
 Subject: SSH Soft Fencing

 Hi,

 SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM
 using SSH connection on non responsive hosts prior to real fencing.
 More info can be found at

 http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3

 In current SSH Soft Fencing implementation the restart VDSM using SSH
 command is part of standard fencing implementation in
 VdsNotRespondingTreatmentCommand. But this command is executed only
 if a host has a valid PM configuration. If host doesn't have a valid
 PM configuration, the execution of the command is disabled and host
 state is change to Non Responsive.

 So my question are:

 1) Should SSH Soft Fencing be executed on hosts without valid PM
configuration?
>>>
>>> I think that the answer should be yes. The vdsm restart will solve most of
>>> problems
>>
>> Would you enumerate the problems that would be solved by a vdsm restart
>> (on list, but on the feature page, too)?
>> I am aware of two issues, both are vdsm bugs:
>> - If libvirtd crashes, vdsm not is not restarted unless there are
>>   running VMs
>> - Vdsm had several bugs in its soft prepareForShutdown process, getting
>>   itself stuck there in case of various background storage processes.
>>
>> I think that solving these two issues would be safer and cleaner than
>> introducing `ssh host service vdsmd restart` flow.
>>
>> The first issue is only a matter of untangling some vdsm internal
>> ugliness: whenever a libvirtconnection is produced, it should be wrapped
>> so that it cathces libvirt crashes. Unlike now, where only VM-related
>> libvirtconnection undergo this treatment.
>>
>> The second issue can be avoiding by vdsm resorting to kill-9-ing itself.
>> After all, this is what `service vdsmd restart` ends up doing after a
>> VERY short timeout (2-3 seconds, iicr).
>>
>> I suppose that there are other reasoning for a remote restart, but in
>> general, I think that it's better to have Vdsm "do the right thing" than
>> expecting Engine to control that remotely.
> 
> 
> theoretically you are absolutely right, but this is much more challenging 
> when the platform you are using keeps changing and might introduce unfamiliar 
> behaviors or bugs.
> You have enumerated several issues that we have encountered in the past and 
> were fixed by us or by different components.
> - libvirt related
> - prepareForShutdown
> - ... I even remember some from SuperVDSM
> 
> All the above eventually were handled brutally by the engine and caused the 
> host to be entirely fenced and all running VMs were killed (and the service 
> they gave went down).
> 
> This is about trying to handle an unexpected situation in a more somewhat 
> delicate manner that in most cases will save killing the VMs, in a scenario 
> where the host is going to be fenced anyway 
> 

+1
We can not anticipates our own bugs ;)


> Now the question Martin had raised is whether this functionality should be 
> applied also when a host has no physical Power-Management device, 
> 
> Hopes this provides the info you refereed to.
> 
> 
> Thanks
> Barak Azulay 
> 
> 
>>
>> Regards,
>> Dan.
>>> , so why not using it whether a PM agent is defined or not.
>>>

 2) Should VDSM restart using SSH command be reimplemented
as standalone command to be usable also in other parts of engine?
If 1) is true, I think it will have to be done anyway.
>>>
>>> +1
>>>


 Martin Perina

>>> ___
>>> Engine-devel mailing list
>>> Engine-devel@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>> ___
>> Engine-devel mailing list
>> Engine-devel@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>>
>>
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] SSH Soft Fencing

[Livnat Peer]

On 06/30/2013 07:33 PM, Barak Azulay wrote:
> 
> 
> - Original Message -
>> From: "Livnat Peer" 
>> To: "Yair Zaslavsky" 
>> Cc: engine-devel@ovirt.org
>> Sent: Sunday, June 30, 2013 8:06:28 AM
>> Subject: Re: [Engine-devel] SSH Soft Fencing
>>
>> On 06/30/2013 05:46 AM, Yair Zaslavsky wrote:
>>>
>>>
>>> - Original Message -
 From: "Barak Azulay" 
 To: "Martin Perina" 
 Cc: "Yair Zaslavsky" , engine-devel@ovirt.org, "Eli
 Mesika" 
 Sent: Thursday, June 27, 2013 8:31:35 PM
 Subject: Re: SSH Soft Fencing



 - Original Message -
> From: "Eli Mesika" 
> To: "Yair Zaslavsky" 
> Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
> Azulay" 
> Sent: Thursday, June 27, 2013 5:55:29 PM
> Subject: Re: SSH Soft Fencing
>
>
>
> - Original Message -
>> From: "Yair Zaslavsky" 
>> To: "Eli Mesika" 
>> Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
>> Azulay" 
>> Sent: Thursday, June 27, 2013 5:43:17 PM
>> Subject: Re: SSH Soft Fencing
>>
>>
>>
>> - Original Message -
>>> From: "Eli Mesika" 
>>> To: "Martin Perina" 
>>> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" ,
>>> "Barak
>>> Azulay" 
>>> Sent: Thursday, June 27, 2013 3:48:39 PM
>>> Subject: Re: SSH Soft Fencing
>>>
>>>
>>>
>>> - Original Message -
 From: "Martin Perina" 
 To: engine-devel@ovirt.org
 Cc: "Yair Zaslavsky" , "Barak Azulay"
 , "Eli Mesika" 
 Sent: Thursday, June 27, 2013 1:51:06 PM
 Subject: SSH Soft Fencing

 Hi,

 SSH Soft Fencing is a new feature for 3.3 and it tries to restart
 VDSM
 using SSH connection on non responsive hosts prior to real fencing.
 More info can be found at

 http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3

 In current SSH Soft Fencing implementation the restart VDSM using SSH
 command is part of standard fencing implementation in
 VdsNotRespondingTreatmentCommand. But this command is executed only
 if a host has a valid PM configuration. If host doesn't have a valid
 PM configuration, the execution of the command is disabled and host
 state is change to Non Responsive.

 So my question are:

 1) Should SSH Soft Fencing be executed on hosts without valid PM
configuration?
>>>
>>> I think that the answer should be yes. The vdsm restart will solve most
>>> of
>>> problems , so why not using it whether a PM agent is defined or not.
>> I agree.
>> I would like to say that I also don't like the fact that
>> VdsNotRespondingTreatment extends RestartVdsCommand.
>> One should ask if "non responding treatment is a restart vds operation"
>> or
>> maybe RestartVdsCommand is just a step in the non responding treatment
>> (inheritance vs containment/delegation).
>> I think that VdsNotRespodingTreatment should delegate the call to
>> RestartVdsCommand as the 2nd step after issuing the Soft Fencing
>> command.
>> Thoughts anyone?
>
> That would be a nice and needed re-factoring

 I would say yes - but would add it only with appropriate configuration
 (enableAutoSoftVdsmRestartWhenNoPMAvailable  I hate the name)
>>>
>>> +1 on configuration.
>>> Configuration must reside at host-related entities (i.e - VdsStatic).
>>>
>>> Yair
>>>
>>
>> Why would a user like to avoid fencing VDSM when host becomes
>> non-responsive?
>>
>> I think that adding another configuration option is cumbersome with no
>> real value.
> 
> The problem i'm trying to solve is not about whether this is the right way to 
> solve a case where vdsm is not responsive,
> But it's about changing the expected behavior in certain cases (and not be 
> able to disable such a change).
> 

When adding enhancements many times you change system behavior.
For adding configuration option to enable the previous mode we need to
see if it make sense and if the system users are going to use it,
otherwise we are adding config option with no use.

Ending up with hundreds of configuration options only makes the
configuration cumbersome instead of useful.

> When a user does not configure a PM he does not expect any fencing to happen, 
> and may be he wants to reach a state where VDSM is stuck .
> (even for development purposes ...) and buy doing the automatic SSH fencing 
> we are actually depriving him from thas option.
> 

The developers use case, in this case, does not sound like a good reason
for adding this configuration option but that is only my 2 cents.

> So let's enable him to reach that situation if he desires (by configuration), 
> and put the default to automatically do the SSH + restart.   
> 
> 
>>
>> Livnat
>>

  

>
>>
>

Re: [Engine-devel] SSH Soft Fencing

[Barak Azulay]

- Original Message -
> From: "Barak Azulay" 
> To: "Martin Perina" 
> Cc: "Yair Zaslavsky" , engine-devel@ovirt.org, "Eli 
> Mesika" 
> Sent: Thursday, June 27, 2013 8:31:35 PM
> Subject: Re: SSH Soft Fencing
> 
> 
> 
> - Original Message -
> > From: "Eli Mesika" 
> > To: "Yair Zaslavsky" 
> > Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
> > Azulay" 
> > Sent: Thursday, June 27, 2013 5:55:29 PM
> > Subject: Re: SSH Soft Fencing
> > 
> > 
> > 
> > - Original Message -
> > > From: "Yair Zaslavsky" 
> > > To: "Eli Mesika" 
> > > Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
> > > Azulay" 
> > > Sent: Thursday, June 27, 2013 5:43:17 PM
> > > Subject: Re: SSH Soft Fencing
> > > 
> > > 
> > > 
> > > - Original Message -
> > > > From: "Eli Mesika" 
> > > > To: "Martin Perina" 
> > > > Cc: engine-devel@ovirt.org, "Yair Zaslavsky" ,
> > > > "Barak
> > > > Azulay" 
> > > > Sent: Thursday, June 27, 2013 3:48:39 PM
> > > > Subject: Re: SSH Soft Fencing
> > > > 
> > > > 
> > > > 
> > > > - Original Message -
> > > > > From: "Martin Perina" 
> > > > > To: engine-devel@ovirt.org
> > > > > Cc: "Yair Zaslavsky" , "Barak Azulay"
> > > > > , "Eli Mesika" 
> > > > > Sent: Thursday, June 27, 2013 1:51:06 PM
> > > > > Subject: SSH Soft Fencing
> > > > > 
> > > > > Hi,
> > > > > 
> > > > > SSH Soft Fencing is a new feature for 3.3 and it tries to restart
> > > > > VDSM
> > > > > using SSH connection on non responsive hosts prior to real fencing.
> > > > > More info can be found at
> > > > > 
> > > > > http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
> > > > > 
> > > > > In current SSH Soft Fencing implementation the restart VDSM using SSH
> > > > > command is part of standard fencing implementation in
> > > > > VdsNotRespondingTreatmentCommand. But this command is executed only
> > > > > if a host has a valid PM configuration. If host doesn't have a valid
> > > > > PM configuration, the execution of the command is disabled and host
> > > > > state is change to Non Responsive.
> > > > > 
> > > > > So my question are:
> > > > > 
> > > > > 1) Should SSH Soft Fencing be executed on hosts without valid PM
> > > > >configuration?
> > > > 
> > > > I think that the answer should be yes. The vdsm restart will solve most
> > > > of
> > > > problems , so why not using it whether a PM agent is defined or not.
> > > I agree.
> > > I would like to say that I also don't like the fact that
> > > VdsNotRespondingTreatment extends RestartVdsCommand.
> > > One should ask if "non responding treatment is a restart vds operation"
> > > or
> > > maybe RestartVdsCommand is just a step in the non responding treatment
> > > (inheritance vs containment/delegation).
> > > I think that VdsNotRespodingTreatment should delegate the call to
> > > RestartVdsCommand as the 2nd step after issuing the Soft Fencing command.
> > > Thoughts anyone?
> > 
> > That would be a nice and needed re-factoring
> 
> I would say yes - but would add it only with appropriate configuration
> (enableAutoSoftVdsmRestartWhenNoPMAvailable  I hate the name)
> 
>  
> 
> > 
> > > 
> > > > 
> > > > > 
> > > > > 2) Should VDSM restart using SSH command be reimplemented
> > > > >as standalone command to be usable also in other parts of engine?
> > > > >If 1) is true, I think it will have to be done anyway.
> > > 
> > > I agree here.
> > > > 
> > > > +1
> 
> On one hand it makes sense,  but I have several questions on the above:
> - Who do we think may want to use such a command ?
> - Should (or even can) we limit the use of such command to
> noneResponsiveTreatment ?
> 
> Having general commands available to all code when there is only one specific
> case we are using it might be a bit riskey,
> Especially when we talk about restarting something.

Martin ? Eli? Yair?

Can you please refer to the issue above ?


> 
> Thoughts ?
> 
> 
> 
> > > > 
> > > > > 
> > > > > 
> > > > > Martin Perina
> > > > > 
> > > > 
> > > 
> > 
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] SSH Soft Fencing

[Barak Azulay]

- Original Message -
> From: "Livnat Peer" 
> To: "Yair Zaslavsky" 
> Cc: engine-devel@ovirt.org
> Sent: Sunday, June 30, 2013 8:06:28 AM
> Subject: Re: [Engine-devel] SSH Soft Fencing
> 
> On 06/30/2013 05:46 AM, Yair Zaslavsky wrote:
> > 
> > 
> > - Original Message -
> >> From: "Barak Azulay" 
> >> To: "Martin Perina" 
> >> Cc: "Yair Zaslavsky" , engine-devel@ovirt.org, "Eli
> >> Mesika" 
> >> Sent: Thursday, June 27, 2013 8:31:35 PM
> >> Subject: Re: SSH Soft Fencing
> >>
> >>
> >>
> >> - Original Message -
> >>> From: "Eli Mesika" 
> >>> To: "Yair Zaslavsky" 
> >>> Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
> >>> Azulay" 
> >>> Sent: Thursday, June 27, 2013 5:55:29 PM
> >>> Subject: Re: SSH Soft Fencing
> >>>
> >>>
> >>>
> >>> - Original Message -
>  From: "Yair Zaslavsky" 
>  To: "Eli Mesika" 
>  Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
>  Azulay" 
>  Sent: Thursday, June 27, 2013 5:43:17 PM
>  Subject: Re: SSH Soft Fencing
> 
> 
> 
>  - Original Message -
> > From: "Eli Mesika" 
> > To: "Martin Perina" 
> > Cc: engine-devel@ovirt.org, "Yair Zaslavsky" ,
> > "Barak
> > Azulay" 
> > Sent: Thursday, June 27, 2013 3:48:39 PM
> > Subject: Re: SSH Soft Fencing
> >
> >
> >
> > - Original Message -
> >> From: "Martin Perina" 
> >> To: engine-devel@ovirt.org
> >> Cc: "Yair Zaslavsky" , "Barak Azulay"
> >> , "Eli Mesika" 
> >> Sent: Thursday, June 27, 2013 1:51:06 PM
> >> Subject: SSH Soft Fencing
> >>
> >> Hi,
> >>
> >> SSH Soft Fencing is a new feature for 3.3 and it tries to restart
> >> VDSM
> >> using SSH connection on non responsive hosts prior to real fencing.
> >> More info can be found at
> >>
> >> http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
> >>
> >> In current SSH Soft Fencing implementation the restart VDSM using SSH
> >> command is part of standard fencing implementation in
> >> VdsNotRespondingTreatmentCommand. But this command is executed only
> >> if a host has a valid PM configuration. If host doesn't have a valid
> >> PM configuration, the execution of the command is disabled and host
> >> state is change to Non Responsive.
> >>
> >> So my question are:
> >>
> >> 1) Should SSH Soft Fencing be executed on hosts without valid PM
> >>configuration?
> >
> > I think that the answer should be yes. The vdsm restart will solve most
> > of
> > problems , so why not using it whether a PM agent is defined or not.
>  I agree.
>  I would like to say that I also don't like the fact that
>  VdsNotRespondingTreatment extends RestartVdsCommand.
>  One should ask if "non responding treatment is a restart vds operation"
>  or
>  maybe RestartVdsCommand is just a step in the non responding treatment
>  (inheritance vs containment/delegation).
>  I think that VdsNotRespodingTreatment should delegate the call to
>  RestartVdsCommand as the 2nd step after issuing the Soft Fencing
>  command.
>  Thoughts anyone?
> >>>
> >>> That would be a nice and needed re-factoring
> >>
> >> I would say yes - but would add it only with appropriate configuration
> >> (enableAutoSoftVdsmRestartWhenNoPMAvailable  I hate the name)
> > 
> > +1 on configuration.
> > Configuration must reside at host-related entities (i.e - VdsStatic).
> > 
> > Yair
> > 
> 
> Why would a user like to avoid fencing VDSM when host becomes
> non-responsive?
> 
> I think that adding another configuration option is cumbersome with no
> real value.

The problem i'm trying to solve is not about whether this is the right way to 
solve a case where vdsm is not responsive,
But it's about changing the expected behavior in certain cases (and not be able 
to disable such a change).

When a user does not configure a PM he does not expect any fencing to happen, 
and may be he wants to reach a state where VDSM is stuck .
(even for development purposes ...) and buy doing the automatic SSH fencing we 
are actually depriving him from thas option.

So let's enable him to reach that situation if he desires (by configuration), 
and put the default to automatically do the SSH + restart.   


> 
> Livnat
> 
> >>
> >>  
> >>
> >>>
> 
> >
> >>
> >> 2) Should VDSM restart using SSH command be reimplemented
> >>as standalone command to be usable also in other parts of engine?
> >>If 1) is true, I think it will have to be done anyway.
> 
>  I agree here.
> >
> > +1
> >>
> >> On one hand it makes sense,  but I have several questions on the above:
> >> - Who do we think may want to use such a command ?
> >> - Should (or even can) we limit the use of such command to
> >> noneResponsiveTreatment ?
> >>
> >> Having general commands available to all code when there is only one
> >> sp

Re: [Engine-devel] SSH Soft Fencing

[Barak Azulay]

- Original Message -
> From: "Dan Kenigsberg" 
> To: "Eli Mesika" 
> Cc: engine-devel@ovirt.org
> Sent: Sunday, June 30, 2013 5:40:49 PM
> Subject: Re: [Engine-devel] SSH Soft Fencing
> 
> On Thu, Jun 27, 2013 at 08:48:39AM -0400, Eli Mesika wrote:
> > 
> > 
> > - Original Message -
> > > From: "Martin Perina" 
> > > To: engine-devel@ovirt.org
> > > Cc: "Yair Zaslavsky" , "Barak Azulay"
> > > , "Eli Mesika" 
> > > Sent: Thursday, June 27, 2013 1:51:06 PM
> > > Subject: SSH Soft Fencing
> > > 
> > > Hi,
> > > 
> > > SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM
> > > using SSH connection on non responsive hosts prior to real fencing.
> > > More info can be found at
> > > 
> > > http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
> > > 
> > > In current SSH Soft Fencing implementation the restart VDSM using SSH
> > > command is part of standard fencing implementation in
> > > VdsNotRespondingTreatmentCommand. But this command is executed only
> > > if a host has a valid PM configuration. If host doesn't have a valid
> > > PM configuration, the execution of the command is disabled and host
> > > state is change to Non Responsive.
> > > 
> > > So my question are:
> > > 
> > > 1) Should SSH Soft Fencing be executed on hosts without valid PM
> > >configuration?
> > 
> > I think that the answer should be yes. The vdsm restart will solve most of
> > problems
> 
> Would you enumerate the problems that would be solved by a vdsm restart
> (on list, but on the feature page, too)?
> I am aware of two issues, both are vdsm bugs:
> - If libvirtd crashes, vdsm not is not restarted unless there are
>   running VMs
> - Vdsm had several bugs in its soft prepareForShutdown process, getting
>   itself stuck there in case of various background storage processes.
> 
> I think that solving these two issues would be safer and cleaner than
> introducing `ssh host service vdsmd restart` flow.
> 
> The first issue is only a matter of untangling some vdsm internal
> ugliness: whenever a libvirtconnection is produced, it should be wrapped
> so that it cathces libvirt crashes. Unlike now, where only VM-related
> libvirtconnection undergo this treatment.
> 
> The second issue can be avoiding by vdsm resorting to kill-9-ing itself.
> After all, this is what `service vdsmd restart` ends up doing after a
> VERY short timeout (2-3 seconds, iicr).
> 
> I suppose that there are other reasoning for a remote restart, but in
> general, I think that it's better to have Vdsm "do the right thing" than
> expecting Engine to control that remotely.


theoretically you are absolutely right, but this is much more challenging when 
the platform you are using keeps changing and might introduce unfamiliar 
behaviors or bugs.
You have enumerated several issues that we have encountered in the past and 
were fixed by us or by different components.
- libvirt related
- prepareForShutdown
- ... I even remember some from SuperVDSM

All the above eventually were handled brutally by the engine and caused the 
host to be entirely fenced and all running VMs were killed (and the service 
they gave went down).

This is about trying to handle an unexpected situation in a more somewhat 
delicate manner that in most cases will save killing the VMs, in a scenario 
where the host is going to be fenced anyway 

Now the question Martin had raised is whether this functionality should be 
applied also when a host has no physical Power-Management device, 

Hopes this provides the info you refereed to.


Thanks
Barak Azulay 


> 
> Regards,
> Dan.
> > , so why not using it whether a PM agent is defined or not.
> > 
> > > 
> > > 2) Should VDSM restart using SSH command be reimplemented
> > >as standalone command to be usable also in other parts of engine?
> > >If 1) is true, I think it will have to be done anyway.
> > 
> > +1
> > 
> > > 
> > > 
> > > Martin Perina
> > > 
> > ___
> > Engine-devel mailing list
> > Engine-devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] SSH Soft Fencing

[Dan Kenigsberg]

On Thu, Jun 27, 2013 at 08:48:39AM -0400, Eli Mesika wrote:
> 
> 
> - Original Message -
> > From: "Martin Perina" 
> > To: engine-devel@ovirt.org
> > Cc: "Yair Zaslavsky" , "Barak Azulay" 
> > , "Eli Mesika" 
> > Sent: Thursday, June 27, 2013 1:51:06 PM
> > Subject: SSH Soft Fencing
> > 
> > Hi,
> > 
> > SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM
> > using SSH connection on non responsive hosts prior to real fencing.
> > More info can be found at
> > 
> > http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
> > 
> > In current SSH Soft Fencing implementation the restart VDSM using SSH
> > command is part of standard fencing implementation in
> > VdsNotRespondingTreatmentCommand. But this command is executed only
> > if a host has a valid PM configuration. If host doesn't have a valid
> > PM configuration, the execution of the command is disabled and host
> > state is change to Non Responsive.
> > 
> > So my question are:
> > 
> > 1) Should SSH Soft Fencing be executed on hosts without valid PM
> >configuration?
> 
> I think that the answer should be yes. The vdsm restart will solve most of 
> problems

Would you enumerate the problems that would be solved by a vdsm restart
(on list, but on the feature page, too)?
I am aware of two issues, both are vdsm bugs:
- If libvirtd crashes, vdsm not is not restarted unless there are
  running VMs
- Vdsm had several bugs in its soft prepareForShutdown process, getting
  itself stuck there in case of various background storage processes.

I think that solving these two issues would be safer and cleaner than
introducing `ssh host service vdsmd restart` flow.

The first issue is only a matter of untangling some vdsm internal
ugliness: whenever a libvirtconnection is produced, it should be wrapped
so that it cathces libvirt crashes. Unlike now, where only VM-related
libvirtconnection undergo this treatment.

The second issue can be avoiding by vdsm resorting to kill-9-ing itself.
After all, this is what `service vdsmd restart` ends up doing after a
VERY short timeout (2-3 seconds, iicr).

I suppose that there are other reasoning for a remote restart, but in
general, I think that it's better to have Vdsm "do the right thing" than
expecting Engine to control that remotely.

Regards,
Dan.
> , so why not using it whether a PM agent is defined or not.
> 
> > 
> > 2) Should VDSM restart using SSH command be reimplemented
> >as standalone command to be usable also in other parts of engine?
> >If 1) is true, I think it will have to be done anyway.
> 
> +1 
> 
> > 
> > 
> > Martin Perina
> > 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

[Engine-devel] VNIC profiles

[Livnat Peer]

Hi,

We are working on adding VNIC profiles as part of the work to add VNIC QoS.

http://www.ovirt.org/Features/Network_QoS#VNIC_Profiles

We need to define some of the system behavior followed by this change,
here is my take -

Editing a profile -

A user should be able to edit the profile properties (including profile
name) while VMs are attached and are using this Profile (reference
should be done by id).

Changing the network though is a bit more tricky as long as we don't
have a way to distinguish between running and current configurations I
think it could be very confusing to the user. Especially since we
support dynamic wiring so the behavior IMO is unpredictable.
I think it should be blocked at this point.


Edit a VNIC / change a VNIC profile -

Changing the profile a VM is using while the VM is running should behave
like dynamic wiring (changing the VM network while it is running).


Remove a Profile -
---
Is only valid if all VMs that are using this profile are in status down.
It should update all VMs to point to no profile which should behave like
none network today.

I see no reason to support a profile on a none network at this point.

The above is also relevant for upgrade flow (upgrading none network to
point to no profile)


Removing a Network -
--
should remove all profiles on that network


VM snapshot/import/export -
--
We should handle VMs that are pointing to a network directly for b/w
compatibility.
we need to select first profile that is on that network that the user
has permissions on.


I assume there are more, comments are welcome

Thanks, Livnat

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Guid improvements

[Tal Nisan]

On 06/30/2013 11:13 AM, Yair Zaslavsky wrote:

Well done, should have been done ages ago :)
Now, for the painful rebase of async_task_mgr changes :)
And for the total removal of Guid wrapper usage of java.util.UUID 
directly instead ;)




- Original Message -

From: "Allon Mureinik" 
To: "engine-devel" , "Barak Azulay" 
Cc: "Yair Zaslavsky" , "Michael Pasternak" , 
"Tal Nisan"
, "Ayal Baron" 
Sent: Sunday, June 30, 2013 11:11:30 AM
Subject: Guid improvements

Hi all,

I just merged a couple of improvements to the [N]Guid class [1] to improve
it's performance both CPU-wise and memory-wise, based on a set of benchmarks
presented by Liran.

What this patchset achieves:
1. Clean up the code, so it's easier to understand and use
2. Eliminate the inflation in the memory foot print caused by the getValue()
method
3. Eliminate all the heavy calls to UUID.fromString when creating a new/empty
Guid instance as a default value
4. Note that the cleanups proposed in (1) will have minor performance
benefits (e.g., eliminating useless conditional statements), but I doubt
this would be anything to write home about.

 From a developer's perspective, here's what changed:
1. No more NGuid, just Guid. Both static methods to create a Guid from String
still exist, and are named createGuidFromString and
createGuidFromStringDefaultEmpty.
2. [N]Guid.getValue() was removed, it's no longer needed after (1) was
implemented
3. The Guid() constructor was made private, as it forced a redundant call to
UUID.fromString(String). If you need an empty Guid instance, just use
Guid.Empty
4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was used for
redundant calls to UUID.fromString. If you really, REALLY, need it, just
call Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for a String.
5. All sorts of ways to transform Strings to Guids were removed. If you have
a literal you trust, just use new Guid(String). If you suspect it may be
null, use Guid.createGuidFromString[DefaultEmpty]
6. NewGuid is now called newGuid. We're in Java, not C# :-)


Many thanks to everyone who reviewed this patchset.
You guys rock!


Regards,
Allon


[1]
http://gerrit.ovirt.org/#/q/project:ovirt-engine+branch:master+topic:guid-cleanup,n,z



___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Guid improvements

[Liran Zelkha]

Sure. I agree. I'd be happy to show you the results in the profiler, so we can 
make a correct decision.

On Jun 30, 2013, at 1:23 PM, Michael Pasternak wrote:

> On 06/30/2013 01:15 PM, Michael Pasternak wrote:
>> On 06/30/2013 01:08 PM, Liran Zelkha wrote:
>>> Why synchronization? No need for it. Worst case scenario a put (which 
>>> should be much less common then get) will occur twice on the same key. 
>> 
>> why assuming a best & not worst scenario? don't forget that every new 
>> insertion requires collision resolution
>> which is triggers .equals() on the GUID.
> 
> Liran, don't get me wrong, i'm not against the caching in general, obviously 
> reads > writes so
> actually i'm all with you, just we're going to significantly enlarge a memory 
> footprint so i just want
> to make sure we're on a right track for the worst scenario where engine runs 
> for ages and hashmap reaches
> it's load factor.
> 
>> 
>>> 
>>> On Jun 30, 2013, at 1:00 PM, Michael Pasternak wrote:
>>> 
 On 06/30/2013 12:45 PM, Liran Zelkha wrote:
> All is true. But average UUID.fromString execution is 1675us, and 
> HashMap.put is 78us - so the benefit is clear when we're talking on >100K 
> executions for 10minutes...
 
 even with synchronization? what about ConcurrentHashMap?
 
> 
> 
> On Sun, Jun 30, 2013 at 12:44 PM, Michael Pasternak  > wrote:
> 
>   On 06/30/2013 12:20 PM, Liran Zelkha wrote:
>> I checked such a solution using JProfiler. Creating the GUID object 
>> takes much more CPU cycles that checking the string in the Map.
> 
>   of course it is, but what is the size of map that you checked?, check 
> on worst scenario, i.e map
>   is full of all possible guids,
> 
>   also problem a bit different,java map has a load factor (which is 
> usually 0.75),
>   when ratio increases beyond the load factor, occurs proses called 
> re-hash so that the hash
>   table will double amount of buckets. what can produce a cpu spikes 
> (though it should not happen too often),
>   to avoid this the initial capacity should be greater than the maximum 
> number of entries / the
>   load factor, and this is a huge map
> 
>   so basically this is a tradeoff between time and space costs against 
> the new guid generation.
> 
>> 
>> On Jun 30, 2013, at 12:06 PM, Michael Pasternak wrote:
>> 
>>> On 06/30/2013 11:37 AM, Liran Zelkha wrote:
 Great news.
 Allon - please note that GUID is being recreated from String by both 
 DB calls and by data received from VDSM. It is VERY useful to cache 
 Guid String-->Guid, and not
>   just Empty GUID.
 
 However, as the Guid class runs in GWT as well, you can't use 
 Infinispan and you're limited in the HashMap implementations you can 
 use.
 Personally, I don't think it's a memory leak, as GUID number in the 
 system are finite and not too large.
>>> 
>>> it's large, it's 128-bit random number, it's not about memory leaking, 
>>> but cpu cost,
>>> as you'll face a lot of rehash'ings in the map,
>>> 
>>> i'm not even sure that using indexing in the map can help, worth 
>>> checking.
>>> 
>>> What do you think? Want to add it to your patch?
>>> 
>>> 
 
 On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:
 
> Well done, should have been done ages ago :)
> Now, for the painful rebase of async_task_mgr changes :)
> 
> 
> - Original Message -
>> From: "Allon Mureinik" > >
>> To: "engine-devel" > >, "Barak Azulay" > >
>> Cc: "Yair Zaslavsky" > >, "Michael Pasternak" 
>> mailto:mpast...@redhat.com>>, "Tal Nisan"
>> mailto:tni...@redhat.com>>, "Ayal Baron" 
>> mailto:aba...@redhat.com>>
>> Sent: Sunday, June 30, 2013 11:11:30 AM
>> Subject: Guid improvements
>> 
>> Hi all,
>> 
>> I just merged a couple of improvements to the [N]Guid class [1] to 
>> improve
>> it's performance both CPU-wise and memory-wise, based on a set of 
>> benchmarks
>> presented by Liran.
>> 
>> What this patchset achieves:
>> 1. Clean up the code, so it's easier to understand and use
>> 2. Eliminate the inflation in the memory foot print caused by the 
>> getValue()
>> method
>> 3. Eliminate all the heavy calls to UUID.fromString when creating a 
>> new/empty
>> Guid instance as a default value
>> 4. Note that the cleanups proposed in (1) will have minor performance
>> benefits (e.g., eliminating useless conditi

Re: [Engine-devel] Guid improvements

[Michael Pasternak]

On 06/30/2013 01:15 PM, Michael Pasternak wrote:
> On 06/30/2013 01:08 PM, Liran Zelkha wrote:
>> Why synchronization? No need for it. Worst case scenario a put (which should 
>> be much less common then get) will occur twice on the same key. 
> 
> why assuming a best & not worst scenario? don't forget that every new 
> insertion requires collision resolution
> which is triggers .equals() on the GUID.

Liran, don't get me wrong, i'm not against the caching in general, obviously 
reads > writes so
actually i'm all with you, just we're going to significantly enlarge a memory 
footprint so i just want
to make sure we're on a right track for the worst scenario where engine runs 
for ages and hashmap reaches
it's load factor.

> 
>>
>> On Jun 30, 2013, at 1:00 PM, Michael Pasternak wrote:
>>
>>> On 06/30/2013 12:45 PM, Liran Zelkha wrote:
 All is true. But average UUID.fromString execution is 1675us, and 
 HashMap.put is 78us - so the benefit is clear when we're talking on >100K 
 executions for 10minutes...
>>>
>>> even with synchronization? what about ConcurrentHashMap?
>>>


 On Sun, Jun 30, 2013 at 12:44 PM, Michael Pasternak >>> > wrote:

On 06/30/2013 12:20 PM, Liran Zelkha wrote:
> I checked such a solution using JProfiler. Creating the GUID object takes 
> much more CPU cycles that checking the string in the Map.

of course it is, but what is the size of map that you checked?, check 
 on worst scenario, i.e map
is full of all possible guids,

also problem a bit different,java map has a load factor (which is 
 usually 0.75),
when ratio increases beyond the load factor, occurs proses called 
 re-hash so that the hash
table will double amount of buckets. what can produce a cpu spikes 
 (though it should not happen too often),
to avoid this the initial capacity should be greater than the maximum 
 number of entries / the
load factor, and this is a huge map

so basically this is a tradeoff between time and space costs against 
 the new guid generation.

>
> On Jun 30, 2013, at 12:06 PM, Michael Pasternak wrote:
>
>> On 06/30/2013 11:37 AM, Liran Zelkha wrote:
>>> Great news.
>>> Allon - please note that GUID is being recreated from String by both DB 
>>> calls and by data received from VDSM. It is VERY useful to cache Guid 
>>> String-->Guid, and not
just Empty GUID.
>>>
>>> However, as the Guid class runs in GWT as well, you can't use 
>>> Infinispan and you're limited in the HashMap implementations you can 
>>> use.
>>> Personally, I don't think it's a memory leak, as GUID number in the 
>>> system are finite and not too large.
>>
>> it's large, it's 128-bit random number, it's not about memory leaking, 
>> but cpu cost,
>> as you'll face a lot of rehash'ings in the map,
>>
>> i'm not even sure that using indexing in the map can help, worth 
>> checking.
>>
>> What do you think? Want to add it to your patch?
>>
>>
>>>
>>> On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:
>>>
 Well done, should have been done ages ago :)
 Now, for the painful rebase of async_task_mgr changes :)


 - Original Message -
> From: "Allon Mureinik"  >
> To: "engine-devel"  >, "Barak Azulay"  >
> Cc: "Yair Zaslavsky"  >, "Michael Pasternak" 
> mailto:mpast...@redhat.com>>, "Tal Nisan"
> mailto:tni...@redhat.com>>, "Ayal Baron" 
> mailto:aba...@redhat.com>>
> Sent: Sunday, June 30, 2013 11:11:30 AM
> Subject: Guid improvements
>
> Hi all,
>
> I just merged a couple of improvements to the [N]Guid class [1] to 
> improve
> it's performance both CPU-wise and memory-wise, based on a set of 
> benchmarks
> presented by Liran.
>
> What this patchset achieves:
> 1. Clean up the code, so it's easier to understand and use
> 2. Eliminate the inflation in the memory foot print caused by the 
> getValue()
> method
> 3. Eliminate all the heavy calls to UUID.fromString when creating a 
> new/empty
> Guid instance as a default value
> 4. Note that the cleanups proposed in (1) will have minor performance
> benefits (e.g., eliminating useless conditional statements), but I 
> doubt
> this would be anything to write home about.
>
> From a developer's perspective, here's what changed:
> 1. No more NGuid, just Guid. Both static methods to create a Guid 
> from String
> still exist, and are named cr

Re: [Engine-devel] Guid improvements

[Michael Pasternak]

On 06/30/2013 01:08 PM, Liran Zelkha wrote:
> Why synchronization? No need for it. Worst case scenario a put (which should 
> be much less common then get) will occur twice on the same key. 

why assuming a best & not worst scenario? don't forget that every new insertion 
requires collision resolution
which is triggers .equals() on the GUID.

> 
> On Jun 30, 2013, at 1:00 PM, Michael Pasternak wrote:
> 
>> On 06/30/2013 12:45 PM, Liran Zelkha wrote:
>>> All is true. But average UUID.fromString execution is 1675us, and 
>>> HashMap.put is 78us - so the benefit is clear when we're talking on >100K 
>>> executions for 10minutes...
>>
>> even with synchronization? what about ConcurrentHashMap?
>>
>>>
>>>
>>> On Sun, Jun 30, 2013 at 12:44 PM, Michael Pasternak >> > wrote:
>>>
>>>On 06/30/2013 12:20 PM, Liran Zelkha wrote:
 I checked such a solution using JProfiler. Creating the GUID object takes 
 much more CPU cycles that checking the string in the Map.
>>>
>>>of course it is, but what is the size of map that you checked?, check on 
>>> worst scenario, i.e map
>>>is full of all possible guids,
>>>
>>>also problem a bit different,java map has a load factor (which is 
>>> usually 0.75),
>>>when ratio increases beyond the load factor, occurs proses called 
>>> re-hash so that the hash
>>>table will double amount of buckets. what can produce a cpu spikes 
>>> (though it should not happen too often),
>>>to avoid this the initial capacity should be greater than the maximum 
>>> number of entries / the
>>>load factor, and this is a huge map
>>>
>>>so basically this is a tradeoff between time and space costs against the 
>>> new guid generation.
>>>

 On Jun 30, 2013, at 12:06 PM, Michael Pasternak wrote:

> On 06/30/2013 11:37 AM, Liran Zelkha wrote:
>> Great news.
>> Allon - please note that GUID is being recreated from String by both DB 
>> calls and by data received from VDSM. It is VERY useful to cache Guid 
>> String-->Guid, and not
>>>just Empty GUID.
>>
>> However, as the Guid class runs in GWT as well, you can't use Infinispan 
>> and you're limited in the HashMap implementations you can use.
>> Personally, I don't think it's a memory leak, as GUID number in the 
>> system are finite and not too large.
>
> it's large, it's 128-bit random number, it's not about memory leaking, 
> but cpu cost,
> as you'll face a lot of rehash'ings in the map,
>
> i'm not even sure that using indexing in the map can help, worth checking.
>
> What do you think? Want to add it to your patch?
>
>
>>
>> On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:
>>
>>> Well done, should have been done ages ago :)
>>> Now, for the painful rebase of async_task_mgr changes :)
>>>
>>>
>>> - Original Message -
 From: "Allon Mureinik" >>> >
 To: "engine-devel" >>> >, "Barak Azulay" >>> >
 Cc: "Yair Zaslavsky" >>> >, "Michael Pasternak" 
 mailto:mpast...@redhat.com>>, "Tal Nisan"
 mailto:tni...@redhat.com>>, "Ayal Baron" 
 mailto:aba...@redhat.com>>
 Sent: Sunday, June 30, 2013 11:11:30 AM
 Subject: Guid improvements

 Hi all,

 I just merged a couple of improvements to the [N]Guid class [1] to 
 improve
 it's performance both CPU-wise and memory-wise, based on a set of 
 benchmarks
 presented by Liran.

 What this patchset achieves:
 1. Clean up the code, so it's easier to understand and use
 2. Eliminate the inflation in the memory foot print caused by the 
 getValue()
 method
 3. Eliminate all the heavy calls to UUID.fromString when creating a 
 new/empty
 Guid instance as a default value
 4. Note that the cleanups proposed in (1) will have minor performance
 benefits (e.g., eliminating useless conditional statements), but I 
 doubt
 this would be anything to write home about.

 From a developer's perspective, here's what changed:
 1. No more NGuid, just Guid. Both static methods to create a Guid from 
 String
 still exist, and are named createGuidFromString and
 createGuidFromStringDefaultEmpty.
 2. [N]Guid.getValue() was removed, it's no longer needed after (1) was
 implemented
 3. The Guid() constructor was made private, as it forced a redundant 
 call to
 UUID.fromString(String). If you need an empty Guid instance, just use
 Guid.Empty
 4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was 
 used for
 redundant calls to UUID.fromString. If you r

Re: [Engine-devel] Guid improvements

[Liran Zelkha]

Why synchronization? No need for it. Worst case scenario a put (which should be 
much less common then get) will occur twice on the same key. 

On Jun 30, 2013, at 1:00 PM, Michael Pasternak wrote:

> On 06/30/2013 12:45 PM, Liran Zelkha wrote:
>> All is true. But average UUID.fromString execution is 1675us, and 
>> HashMap.put is 78us - so the benefit is clear when we're talking on >100K 
>> executions for 10minutes...
> 
> even with synchronization? what about ConcurrentHashMap?
> 
>> 
>> 
>> On Sun, Jun 30, 2013 at 12:44 PM, Michael Pasternak > > wrote:
>> 
>>On 06/30/2013 12:20 PM, Liran Zelkha wrote:
>>> I checked such a solution using JProfiler. Creating the GUID object takes 
>>> much more CPU cycles that checking the string in the Map.
>> 
>>of course it is, but what is the size of map that you checked?, check on 
>> worst scenario, i.e map
>>is full of all possible guids,
>> 
>>also problem a bit different,java map has a load factor (which is usually 
>> 0.75),
>>when ratio increases beyond the load factor, occurs proses called re-hash 
>> so that the hash
>>table will double amount of buckets. what can produce a cpu spikes 
>> (though it should not happen too often),
>>to avoid this the initial capacity should be greater than the maximum 
>> number of entries / the
>>load factor, and this is a huge map
>> 
>>so basically this is a tradeoff between time and space costs against the 
>> new guid generation.
>> 
>>> 
>>> On Jun 30, 2013, at 12:06 PM, Michael Pasternak wrote:
>>> 
 On 06/30/2013 11:37 AM, Liran Zelkha wrote:
> Great news.
> Allon - please note that GUID is being recreated from String by both DB 
> calls and by data received from VDSM. It is VERY useful to cache Guid 
> String-->Guid, and not
>>just Empty GUID.
> 
> However, as the Guid class runs in GWT as well, you can't use Infinispan 
> and you're limited in the HashMap implementations you can use.
> Personally, I don't think it's a memory leak, as GUID number in the 
> system are finite and not too large.
 
 it's large, it's 128-bit random number, it's not about memory leaking, but 
 cpu cost,
 as you'll face a lot of rehash'ings in the map,
 
 i'm not even sure that using indexing in the map can help, worth checking.
 
 What do you think? Want to add it to your patch?
 
 
> 
> On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:
> 
>> Well done, should have been done ages ago :)
>> Now, for the painful rebase of async_task_mgr changes :)
>> 
>> 
>> - Original Message -
>>> From: "Allon Mureinik" >> >
>>> To: "engine-devel" >> >, "Barak Azulay" >> >
>>> Cc: "Yair Zaslavsky" >> >, "Michael Pasternak" >> >, "Tal Nisan"
>>> mailto:tni...@redhat.com>>, "Ayal Baron" 
>>> mailto:aba...@redhat.com>>
>>> Sent: Sunday, June 30, 2013 11:11:30 AM
>>> Subject: Guid improvements
>>> 
>>> Hi all,
>>> 
>>> I just merged a couple of improvements to the [N]Guid class [1] to 
>>> improve
>>> it's performance both CPU-wise and memory-wise, based on a set of 
>>> benchmarks
>>> presented by Liran.
>>> 
>>> What this patchset achieves:
>>> 1. Clean up the code, so it's easier to understand and use
>>> 2. Eliminate the inflation in the memory foot print caused by the 
>>> getValue()
>>> method
>>> 3. Eliminate all the heavy calls to UUID.fromString when creating a 
>>> new/empty
>>> Guid instance as a default value
>>> 4. Note that the cleanups proposed in (1) will have minor performance
>>> benefits (e.g., eliminating useless conditional statements), but I doubt
>>> this would be anything to write home about.
>>> 
>>> From a developer's perspective, here's what changed:
>>> 1. No more NGuid, just Guid. Both static methods to create a Guid from 
>>> String
>>> still exist, and are named createGuidFromString and
>>> createGuidFromStringDefaultEmpty.
>>> 2. [N]Guid.getValue() was removed, it's no longer needed after (1) was
>>> implemented
>>> 3. The Guid() constructor was made private, as it forced a redundant 
>>> call to
>>> UUID.fromString(String). If you need an empty Guid instance, just use
>>> Guid.Empty
>>> 4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was 
>>> used for
>>> redundant calls to UUID.fromString. If you really, REALLY, need it, just
>>> call Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for a 
>>> String.
>>> 5. All sorts of ways to transform Strings to Guids were removed. If you 
>>> have
>>> a literal you trust, just use new Guid(String). If you suspect it may be
>>> 

Re: [Engine-devel] Guid improvements

[Michael Pasternak]

On 06/30/2013 12:45 PM, Liran Zelkha wrote:
> All is true. But average UUID.fromString execution is 1675us, and HashMap.put 
> is 78us - so the benefit is clear when we're talking on >100K executions for 
> 10minutes...

even with synchronization? what about ConcurrentHashMap?

> 
> 
> On Sun, Jun 30, 2013 at 12:44 PM, Michael Pasternak  > wrote:
> 
> On 06/30/2013 12:20 PM, Liran Zelkha wrote:
> > I checked such a solution using JProfiler. Creating the GUID object 
> takes much more CPU cycles that checking the string in the Map.
> 
> of course it is, but what is the size of map that you checked?, check on 
> worst scenario, i.e map
> is full of all possible guids,
> 
> also problem a bit different,java map has a load factor (which is usually 
> 0.75),
> when ratio increases beyond the load factor, occurs proses called re-hash 
> so that the hash
> table will double amount of buckets. what can produce a cpu spikes 
> (though it should not happen too often),
> to avoid this the initial capacity should be greater than the maximum 
> number of entries / the
> load factor, and this is a huge map
> 
> so basically this is a tradeoff between time and space costs against the 
> new guid generation.
> 
> >
> > On Jun 30, 2013, at 12:06 PM, Michael Pasternak wrote:
> >
> >> On 06/30/2013 11:37 AM, Liran Zelkha wrote:
> >>> Great news.
> >>> Allon - please note that GUID is being recreated from String by both 
> DB calls and by data received from VDSM. It is VERY useful to cache Guid 
> String-->Guid, and not
> just Empty GUID.
> >>>
> >>> However, as the Guid class runs in GWT as well, you can't use 
> Infinispan and you're limited in the HashMap implementations you can use.
> >>> Personally, I don't think it's a memory leak, as GUID number in the 
> system are finite and not too large.
> >>
> >> it's large, it's 128-bit random number, it's not about memory leaking, 
> but cpu cost,
> >> as you'll face a lot of rehash'ings in the map,
> >>
> >> i'm not even sure that using indexing in the map can help, worth 
> checking.
> >>
> >> What do you think? Want to add it to your patch?
> >>
> >>
> >>>
> >>> On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:
> >>>
>  Well done, should have been done ages ago :)
>  Now, for the painful rebase of async_task_mgr changes :)
> 
> 
>  - Original Message -
> > From: "Allon Mureinik"  >
> > To: "engine-devel"  >, "Barak Azulay"  >
> > Cc: "Yair Zaslavsky"  >, "Michael Pasternak"  >, "Tal Nisan"
> > mailto:tni...@redhat.com>>, "Ayal Baron" 
> mailto:aba...@redhat.com>>
> > Sent: Sunday, June 30, 2013 11:11:30 AM
> > Subject: Guid improvements
> >
> > Hi all,
> >
> > I just merged a couple of improvements to the [N]Guid class [1] to 
> improve
> > it's performance both CPU-wise and memory-wise, based on a set of 
> benchmarks
> > presented by Liran.
> >
> > What this patchset achieves:
> > 1. Clean up the code, so it's easier to understand and use
> > 2. Eliminate the inflation in the memory foot print caused by the 
> getValue()
> > method
> > 3. Eliminate all the heavy calls to UUID.fromString when creating a 
> new/empty
> > Guid instance as a default value
> > 4. Note that the cleanups proposed in (1) will have minor 
> performance
> > benefits (e.g., eliminating useless conditional statements), but I 
> doubt
> > this would be anything to write home about.
> >
> > From a developer's perspective, here's what changed:
> > 1. No more NGuid, just Guid. Both static methods to create a Guid 
> from String
> > still exist, and are named createGuidFromString and
> > createGuidFromStringDefaultEmpty.
> > 2. [N]Guid.getValue() was removed, it's no longer needed after (1) 
> was
> > implemented
> > 3. The Guid() constructor was made private, as it forced a 
> redundant call to
> > UUID.fromString(String). If you need an empty Guid instance, just 
> use
> > Guid.Empty
> > 4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was 
> used for
> > redundant calls to UUID.fromString. If you really, REALLY, need it, 
> just
> > call Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for 
> a String.
> > 5. All sorts of ways to transform Strings to Guids were removed. If 
> you have
> > a literal you trust, just use new Guid(String). If you suspect it 
> may be
> > null, use Guid.createGuidFromString[DefaultEmpty]
> > 6. NewGui

Re: [Engine-devel] Guid improvements

[Liran Zelkha]

All is true. But average UUID.fromString execution is 1675us, and
HashMap.put is 78us - so the benefit is clear when we're talking on >100K
executions for 10minutes...


On Sun, Jun 30, 2013 at 12:44 PM, Michael Pasternak wrote:

> On 06/30/2013 12:20 PM, Liran Zelkha wrote:
> > I checked such a solution using JProfiler. Creating the GUID object
> takes much more CPU cycles that checking the string in the Map.
>
> of course it is, but what is the size of map that you checked?, check on
> worst scenario, i.e map
> is full of all possible guids,
>
> also problem a bit different,java map has a load factor (which is usually
> 0.75),
> when ratio increases beyond the load factor, occurs proses called re-hash
> so that the hash
> table will double amount of buckets. what can produce a cpu spikes (though
> it should not happen too often),
> to avoid this the initial capacity should be greater than the maximum
> number of entries / the
> load factor, and this is a huge map
>
> so basically this is a tradeoff between time and space costs against the
> new guid generation.
>
> >
> > On Jun 30, 2013, at 12:06 PM, Michael Pasternak wrote:
> >
> >> On 06/30/2013 11:37 AM, Liran Zelkha wrote:
> >>> Great news.
> >>> Allon - please note that GUID is being recreated from String by both
> DB calls and by data received from VDSM. It is VERY useful to cache Guid
> String-->Guid, and not just Empty GUID.
> >>>
> >>> However, as the Guid class runs in GWT as well, you can't use
> Infinispan and you're limited in the HashMap implementations you can use.
> >>> Personally, I don't think it's a memory leak, as GUID number in the
> system are finite and not too large.
> >>
> >> it's large, it's 128-bit random number, it's not about memory leaking,
> but cpu cost,
> >> as you'll face a lot of rehash'ings in the map,
> >>
> >> i'm not even sure that using indexing in the map can help, worth
> checking.
> >>
> >> What do you think? Want to add it to your patch?
> >>
> >>
> >>>
> >>> On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:
> >>>
>  Well done, should have been done ages ago :)
>  Now, for the painful rebase of async_task_mgr changes :)
> 
> 
>  - Original Message -
> > From: "Allon Mureinik" 
> > To: "engine-devel" , "Barak Azulay" <
> bazu...@redhat.com>
> > Cc: "Yair Zaslavsky" , "Michael Pasternak" <
> mpast...@redhat.com>, "Tal Nisan"
> > , "Ayal Baron" 
> > Sent: Sunday, June 30, 2013 11:11:30 AM
> > Subject: Guid improvements
> >
> > Hi all,
> >
> > I just merged a couple of improvements to the [N]Guid class [1] to
> improve
> > it's performance both CPU-wise and memory-wise, based on a set of
> benchmarks
> > presented by Liran.
> >
> > What this patchset achieves:
> > 1. Clean up the code, so it's easier to understand and use
> > 2. Eliminate the inflation in the memory foot print caused by the
> getValue()
> > method
> > 3. Eliminate all the heavy calls to UUID.fromString when creating a
> new/empty
> > Guid instance as a default value
> > 4. Note that the cleanups proposed in (1) will have minor performance
> > benefits (e.g., eliminating useless conditional statements), but I
> doubt
> > this would be anything to write home about.
> >
> > From a developer's perspective, here's what changed:
> > 1. No more NGuid, just Guid. Both static methods to create a Guid
> from String
> > still exist, and are named createGuidFromString and
> > createGuidFromStringDefaultEmpty.
> > 2. [N]Guid.getValue() was removed, it's no longer needed after (1)
> was
> > implemented
> > 3. The Guid() constructor was made private, as it forced a redundant
> call to
> > UUID.fromString(String). If you need an empty Guid instance, just use
> > Guid.Empty
> > 4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was
> used for
> > redundant calls to UUID.fromString. If you really, REALLY, need it,
> just
> > call Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for a
> String.
> > 5. All sorts of ways to transform Strings to Guids were removed. If
> you have
> > a literal you trust, just use new Guid(String). If you suspect it
> may be
> > null, use Guid.createGuidFromString[DefaultEmpty]
> > 6. NewGuid is now called newGuid. We're in Java, not C# :-)
> >
> >
> > Many thanks to everyone who reviewed this patchset.
> > You guys rock!
> >
> >
> > Regards,
> > Allon
> >
> >
> > [1]
> >
> http://gerrit.ovirt.org/#/q/project:ovirt-engine+branch:master+topic:guid-cleanup,n,z
> >
>  ___
>  Engine-devel mailing list
>  Engine-devel@ovirt.org
>  http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>
> >>> ___
> >>> Engine-devel mailing list
> >>> Engine-devel@ovirt.org
> >>> http://lists.ovirt

Re: [Engine-devel] Guid improvements

[Michael Pasternak]

On 06/30/2013 12:20 PM, Liran Zelkha wrote:
> I checked such a solution using JProfiler. Creating the GUID object takes 
> much more CPU cycles that checking the string in the Map.

of course it is, but what is the size of map that you checked?, check on worst 
scenario, i.e map
is full of all possible guids,

also problem a bit different,java map has a load factor (which is usually 0.75),
when ratio increases beyond the load factor, occurs proses called re-hash so 
that the hash
table will double amount of buckets. what can produce a cpu spikes (though it 
should not happen too often),
to avoid this the initial capacity should be greater than the maximum number of 
entries / the
load factor, and this is a huge map

so basically this is a tradeoff between time and space costs against the new 
guid generation.

> 
> On Jun 30, 2013, at 12:06 PM, Michael Pasternak wrote:
> 
>> On 06/30/2013 11:37 AM, Liran Zelkha wrote:
>>> Great news.
>>> Allon - please note that GUID is being recreated from String by both DB 
>>> calls and by data received from VDSM. It is VERY useful to cache Guid 
>>> String-->Guid, and not just Empty GUID. 
>>>
>>> However, as the Guid class runs in GWT as well, you can't use Infinispan 
>>> and you're limited in the HashMap implementations you can use. 
>>> Personally, I don't think it's a memory leak, as GUID number in the system 
>>> are finite and not too large.
>>
>> it's large, it's 128-bit random number, it's not about memory leaking, but 
>> cpu cost,
>> as you'll face a lot of rehash'ings in the map,
>>
>> i'm not even sure that using indexing in the map can help, worth checking.
>>
>> What do you think? Want to add it to your patch?
>>
>>
>>>
>>> On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:
>>>
 Well done, should have been done ages ago :)
 Now, for the painful rebase of async_task_mgr changes :)


 - Original Message -
> From: "Allon Mureinik" 
> To: "engine-devel" , "Barak Azulay" 
> 
> Cc: "Yair Zaslavsky" , "Michael Pasternak" 
> , "Tal Nisan"
> , "Ayal Baron" 
> Sent: Sunday, June 30, 2013 11:11:30 AM
> Subject: Guid improvements
>
> Hi all,
>
> I just merged a couple of improvements to the [N]Guid class [1] to improve
> it's performance both CPU-wise and memory-wise, based on a set of 
> benchmarks
> presented by Liran.
>
> What this patchset achieves:
> 1. Clean up the code, so it's easier to understand and use
> 2. Eliminate the inflation in the memory foot print caused by the 
> getValue()
> method
> 3. Eliminate all the heavy calls to UUID.fromString when creating a 
> new/empty
> Guid instance as a default value
> 4. Note that the cleanups proposed in (1) will have minor performance
> benefits (e.g., eliminating useless conditional statements), but I doubt
> this would be anything to write home about.
>
> From a developer's perspective, here's what changed:
> 1. No more NGuid, just Guid. Both static methods to create a Guid from 
> String
> still exist, and are named createGuidFromString and
> createGuidFromStringDefaultEmpty.
> 2. [N]Guid.getValue() was removed, it's no longer needed after (1) was
> implemented
> 3. The Guid() constructor was made private, as it forced a redundant call 
> to
> UUID.fromString(String). If you need an empty Guid instance, just use
> Guid.Empty
> 4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was used 
> for
> redundant calls to UUID.fromString. If you really, REALLY, need it, just
> call Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for a 
> String.
> 5. All sorts of ways to transform Strings to Guids were removed. If you 
> have
> a literal you trust, just use new Guid(String). If you suspect it may be
> null, use Guid.createGuidFromString[DefaultEmpty]
> 6. NewGuid is now called newGuid. We're in Java, not C# :-)
>
>
> Many thanks to everyone who reviewed this patchset.
> You guys rock!
>
>
> Regards,
> Allon
>
>
> [1]
> http://gerrit.ovirt.org/#/q/project:ovirt-engine+branch:master+topic:guid-cleanup,n,z
>
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>> ___
>>> Engine-devel mailing list
>>> Engine-devel@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>
>>
>> -- 
>>
>> Michael Pasternak
>> RedHat, ENG-Virtualization R&D
> 


-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Guid improvements

[Liran Zelkha]

I checked such a solution using JProfiler. Creating the GUID object takes much 
more CPU cycles that checking the string in the Map.

On Jun 30, 2013, at 12:06 PM, Michael Pasternak wrote:

> On 06/30/2013 11:37 AM, Liran Zelkha wrote:
>> Great news.
>> Allon - please note that GUID is being recreated from String by both DB 
>> calls and by data received from VDSM. It is VERY useful to cache Guid 
>> String-->Guid, and not just Empty GUID. 
>> 
>> However, as the Guid class runs in GWT as well, you can't use Infinispan and 
>> you're limited in the HashMap implementations you can use. 
>> Personally, I don't think it's a memory leak, as GUID number in the system 
>> are finite and not too large.
> 
> it's large, it's 128-bit random number, it's not about memory leaking, but 
> cpu cost,
> as you'll face a lot of rehash'ings in the map,
> 
> i'm not even sure that using indexing in the map can help, worth checking.
> 
> What do you think? Want to add it to your patch?
> 
> 
>> 
>> On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:
>> 
>>> Well done, should have been done ages ago :)
>>> Now, for the painful rebase of async_task_mgr changes :)
>>> 
>>> 
>>> - Original Message -
 From: "Allon Mureinik" 
 To: "engine-devel" , "Barak Azulay" 
 
 Cc: "Yair Zaslavsky" , "Michael Pasternak" 
 , "Tal Nisan"
 , "Ayal Baron" 
 Sent: Sunday, June 30, 2013 11:11:30 AM
 Subject: Guid improvements
 
 Hi all,
 
 I just merged a couple of improvements to the [N]Guid class [1] to improve
 it's performance both CPU-wise and memory-wise, based on a set of 
 benchmarks
 presented by Liran.
 
 What this patchset achieves:
 1. Clean up the code, so it's easier to understand and use
 2. Eliminate the inflation in the memory foot print caused by the 
 getValue()
 method
 3. Eliminate all the heavy calls to UUID.fromString when creating a 
 new/empty
 Guid instance as a default value
 4. Note that the cleanups proposed in (1) will have minor performance
 benefits (e.g., eliminating useless conditional statements), but I doubt
 this would be anything to write home about.
 
 From a developer's perspective, here's what changed:
 1. No more NGuid, just Guid. Both static methods to create a Guid from 
 String
 still exist, and are named createGuidFromString and
 createGuidFromStringDefaultEmpty.
 2. [N]Guid.getValue() was removed, it's no longer needed after (1) was
 implemented
 3. The Guid() constructor was made private, as it forced a redundant call 
 to
 UUID.fromString(String). If you need an empty Guid instance, just use
 Guid.Empty
 4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was used 
 for
 redundant calls to UUID.fromString. If you really, REALLY, need it, just
 call Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for a 
 String.
 5. All sorts of ways to transform Strings to Guids were removed. If you 
 have
 a literal you trust, just use new Guid(String). If you suspect it may be
 null, use Guid.createGuidFromString[DefaultEmpty]
 6. NewGuid is now called newGuid. We're in Java, not C# :-)
 
 
 Many thanks to everyone who reviewed this patchset.
 You guys rock!
 
 
 Regards,
 Allon
 
 
 [1]
 http://gerrit.ovirt.org/#/q/project:ovirt-engine+branch:master+topic:guid-cleanup,n,z
 
>>> ___
>>> Engine-devel mailing list
>>> Engine-devel@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>> 
>> ___
>> Engine-devel mailing list
>> Engine-devel@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>> 
> 
> 
> -- 
> 
> Michael Pasternak
> RedHat, ENG-Virtualization R&D

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Guid improvements

[Michael Pasternak]

On 06/30/2013 11:37 AM, Liran Zelkha wrote:
> Great news.
> Allon - please note that GUID is being recreated from String by both DB calls 
> and by data received from VDSM. It is VERY useful to cache Guid 
> String-->Guid, and not just Empty GUID. 
> 
> However, as the Guid class runs in GWT as well, you can't use Infinispan and 
> you're limited in the HashMap implementations you can use. 
> Personally, I don't think it's a memory leak, as GUID number in the system 
> are finite and not too large.

it's large, it's 128-bit random number, it's not about memory leaking, but cpu 
cost,
as you'll face a lot of rehash'ings in the map,

i'm not even sure that using indexing in the map can help, worth checking.

What do you think? Want to add it to your patch?


> 
> On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:
> 
>> Well done, should have been done ages ago :)
>> Now, for the painful rebase of async_task_mgr changes :)
>>
>>
>> - Original Message -
>>> From: "Allon Mureinik" 
>>> To: "engine-devel" , "Barak Azulay" 
>>> 
>>> Cc: "Yair Zaslavsky" , "Michael Pasternak" 
>>> , "Tal Nisan"
>>> , "Ayal Baron" 
>>> Sent: Sunday, June 30, 2013 11:11:30 AM
>>> Subject: Guid improvements
>>>
>>> Hi all,
>>>
>>> I just merged a couple of improvements to the [N]Guid class [1] to improve
>>> it's performance both CPU-wise and memory-wise, based on a set of benchmarks
>>> presented by Liran.
>>>
>>> What this patchset achieves:
>>> 1. Clean up the code, so it's easier to understand and use
>>> 2. Eliminate the inflation in the memory foot print caused by the getValue()
>>> method
>>> 3. Eliminate all the heavy calls to UUID.fromString when creating a 
>>> new/empty
>>> Guid instance as a default value
>>> 4. Note that the cleanups proposed in (1) will have minor performance
>>> benefits (e.g., eliminating useless conditional statements), but I doubt
>>> this would be anything to write home about.
>>>
>>> From a developer's perspective, here's what changed:
>>> 1. No more NGuid, just Guid. Both static methods to create a Guid from 
>>> String
>>> still exist, and are named createGuidFromString and
>>> createGuidFromStringDefaultEmpty.
>>> 2. [N]Guid.getValue() was removed, it's no longer needed after (1) was
>>> implemented
>>> 3. The Guid() constructor was made private, as it forced a redundant call to
>>> UUID.fromString(String). If you need an empty Guid instance, just use
>>> Guid.Empty
>>> 4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was used for
>>> redundant calls to UUID.fromString. If you really, REALLY, need it, just
>>> call Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for a String.
>>> 5. All sorts of ways to transform Strings to Guids were removed. If you have
>>> a literal you trust, just use new Guid(String). If you suspect it may be
>>> null, use Guid.createGuidFromString[DefaultEmpty]
>>> 6. NewGuid is now called newGuid. We're in Java, not C# :-)
>>>
>>>
>>> Many thanks to everyone who reviewed this patchset.
>>> You guys rock!
>>>
>>>
>>> Regards,
>>> Allon
>>>
>>>
>>> [1]
>>> http://gerrit.ovirt.org/#/q/project:ovirt-engine+branch:master+topic:guid-cleanup,n,z
>>>
>> ___
>> Engine-devel mailing list
>> Engine-devel@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] development installation issues

[Moti Asayag]

On 06/30/2013 11:52 AM, Michael Pasternak wrote:
> On 06/30/2013 11:43 AM, Moti Asayag wrote:
>> On 06/30/2013 11:03 AM, Michael Pasternak wrote:
>>>
>>> does anyone faced this with new 'development installation'?
>>>
>>> [mpastern@lpt21-f ovirt-engine]$ 
>>> /home/mpastern/Coding/ovirt/ovirt-engine/bin/engine-setup-2
>>> [ ERROR ] Failed to execute stage 'Booting': 5
>>> [ INFO  ] Stage: Initializing
>>>   Setup was run under unprivileged user this will produce 
>>> development installation do you wish to proceed? (Yes, No) [No]: Yes
>>> [WARNING] engine-setup-2 is a technical preview, and yet to include all 
>>> functionality that exists in legacy engine-setup. Specifically, 
>>> engine-setup-2 does not support
>>> upgrade from previous installations.
>>> [mpastern@lpt21-f ovirt-engine]$
>>>
>>
>> I had the same problem which was solved by upgrading to the latest otopi
>> package.
>>
> 
> thanks moti,
> 
> i'm on latest otopi (1b29c7f06a0b9b6715bd8340396c35ac29bf086c).
> 

It works fine for me with
otopi-1.1.0-0.0.master.20130626.git33d9561.fc17.noarch, taken from
nightly repo.

See mail from Alon Bar-Lev from last week with subject:
"[Engine-devel] [ATN][Action Required] development environemnt users -
otopi migration"
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] development installation issues

[Michael Pasternak]

On 06/30/2013 11:43 AM, Moti Asayag wrote:
> On 06/30/2013 11:03 AM, Michael Pasternak wrote:
>>
>> does anyone faced this with new 'development installation'?
>>
>> [mpastern@lpt21-f ovirt-engine]$ 
>> /home/mpastern/Coding/ovirt/ovirt-engine/bin/engine-setup-2
>> [ ERROR ] Failed to execute stage 'Booting': 5
>> [ INFO  ] Stage: Initializing
>>   Setup was run under unprivileged user this will produce 
>> development installation do you wish to proceed? (Yes, No) [No]: Yes
>> [WARNING] engine-setup-2 is a technical preview, and yet to include all 
>> functionality that exists in legacy engine-setup. Specifically, 
>> engine-setup-2 does not support
>> upgrade from previous installations.
>> [mpastern@lpt21-f ovirt-engine]$
>>
> 
> I had the same problem which was solved by upgrading to the latest otopi
> package.
> 

thanks moti,

i'm on latest otopi (1b29c7f06a0b9b6715bd8340396c35ac29bf086c).

-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] development installation issues

[Moti Asayag]

On 06/30/2013 11:03 AM, Michael Pasternak wrote:
> 
> does anyone faced this with new 'development installation'?
> 
> [mpastern@lpt21-f ovirt-engine]$ 
> /home/mpastern/Coding/ovirt/ovirt-engine/bin/engine-setup-2
> [ ERROR ] Failed to execute stage 'Booting': 5
> [ INFO  ] Stage: Initializing
>   Setup was run under unprivileged user this will produce development 
> installation do you wish to proceed? (Yes, No) [No]: Yes
> [WARNING] engine-setup-2 is a technical preview, and yet to include all 
> functionality that exists in legacy engine-setup. Specifically, 
> engine-setup-2 does not support
> upgrade from previous installations.
> [mpastern@lpt21-f ovirt-engine]$
> 

I had the same problem which was solved by upgrading to the latest otopi
package.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Guid improvements

[Liran Zelkha]

Great news.
Allon - please note that GUID is being recreated from String by both DB calls 
and by data received from VDSM. It is VERY useful to cache Guid String-->Guid, 
and not just Empty GUID. 

However, as the Guid class runs in GWT as well, you can't use Infinispan and 
you're limited in the HashMap implementations you can use. Personally, I don't 
think it's a memory leak, as GUID number in the system are finite and not too 
large. What do you think? Want to add it to your patch?

On Jun 30, 2013, at 11:13 AM, Yair Zaslavsky wrote:

> Well done, should have been done ages ago :)
> Now, for the painful rebase of async_task_mgr changes :)
> 
> 
> - Original Message -
>> From: "Allon Mureinik" 
>> To: "engine-devel" , "Barak Azulay" 
>> 
>> Cc: "Yair Zaslavsky" , "Michael Pasternak" 
>> , "Tal Nisan"
>> , "Ayal Baron" 
>> Sent: Sunday, June 30, 2013 11:11:30 AM
>> Subject: Guid improvements
>> 
>> Hi all,
>> 
>> I just merged a couple of improvements to the [N]Guid class [1] to improve
>> it's performance both CPU-wise and memory-wise, based on a set of benchmarks
>> presented by Liran.
>> 
>> What this patchset achieves:
>> 1. Clean up the code, so it's easier to understand and use
>> 2. Eliminate the inflation in the memory foot print caused by the getValue()
>> method
>> 3. Eliminate all the heavy calls to UUID.fromString when creating a new/empty
>> Guid instance as a default value
>> 4. Note that the cleanups proposed in (1) will have minor performance
>> benefits (e.g., eliminating useless conditional statements), but I doubt
>> this would be anything to write home about.
>> 
>> From a developer's perspective, here's what changed:
>> 1. No more NGuid, just Guid. Both static methods to create a Guid from String
>> still exist, and are named createGuidFromString and
>> createGuidFromStringDefaultEmpty.
>> 2. [N]Guid.getValue() was removed, it's no longer needed after (1) was
>> implemented
>> 3. The Guid() constructor was made private, as it forced a redundant call to
>> UUID.fromString(String). If you need an empty Guid instance, just use
>> Guid.Empty
>> 4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was used for
>> redundant calls to UUID.fromString. If you really, REALLY, need it, just
>> call Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for a String.
>> 5. All sorts of ways to transform Strings to Guids were removed. If you have
>> a literal you trust, just use new Guid(String). If you suspect it may be
>> null, use Guid.createGuidFromString[DefaultEmpty]
>> 6. NewGuid is now called newGuid. We're in Java, not C# :-)
>> 
>> 
>> Many thanks to everyone who reviewed this patchset.
>> You guys rock!
>> 
>> 
>> Regards,
>> Allon
>> 
>> 
>> [1]
>> http://gerrit.ovirt.org/#/q/project:ovirt-engine+branch:master+topic:guid-cleanup,n,z
>> 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Introduce mail

[Dan Kenigsberg]

On Sun, Jun 30, 2013 at 09:16:00AM +0300, Livnat Peer wrote:
> Hi Fellipe,
> 
> We'll be happy to use your help, there is always a lot of work to do :)
> Is there any specific area in virtualization you would like to focus on?
> Network | storage | VMs life-cycle | host life-cycle other?
> 
> To start setting your development environment, take a look at -
> www.ovirt.org/Vdsm_Developers
> 
> 
> Also adding Dan to point any low hanging fruits, or pointers for a
> potentially first task :)

I see that we currently only have 4 EasyFix bugs
https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&component=vdsm&f0=OP&f1=OP&f3=CP&f4=CP&j1=OR&keywords=EasyFix%2C%20&keywords_type=allwords&list_id=1490222&o2=notsubstring&query_format=advanced&v2=rhel-6.2.0

I would very much like to get help testing vdsm on Fedora 19 and fixing
issues that might arrise there. E.g. my http://gerrit.ovirt.org/16059

I find running the tests (unit+functional) and extending them, as a good
starting poing.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Guid improvements

[Yair Zaslavsky]

Well done, should have been done ages ago :)
Now, for the painful rebase of async_task_mgr changes :)


- Original Message -
> From: "Allon Mureinik" 
> To: "engine-devel" , "Barak Azulay" 
> 
> Cc: "Yair Zaslavsky" , "Michael Pasternak" 
> , "Tal Nisan"
> , "Ayal Baron" 
> Sent: Sunday, June 30, 2013 11:11:30 AM
> Subject: Guid improvements
> 
> Hi all,
> 
> I just merged a couple of improvements to the [N]Guid class [1] to improve
> it's performance both CPU-wise and memory-wise, based on a set of benchmarks
> presented by Liran.
> 
> What this patchset achieves:
> 1. Clean up the code, so it's easier to understand and use
> 2. Eliminate the inflation in the memory foot print caused by the getValue()
> method
> 3. Eliminate all the heavy calls to UUID.fromString when creating a new/empty
> Guid instance as a default value
> 4. Note that the cleanups proposed in (1) will have minor performance
> benefits (e.g., eliminating useless conditional statements), but I doubt
> this would be anything to write home about.
> 
> From a developer's perspective, here's what changed:
> 1. No more NGuid, just Guid. Both static methods to create a Guid from String
> still exist, and are named createGuidFromString and
> createGuidFromStringDefaultEmpty.
> 2. [N]Guid.getValue() was removed, it's no longer needed after (1) was
> implemented
> 3. The Guid() constructor was made private, as it forced a redundant call to
> UUID.fromString(String). If you need an empty Guid instance, just use
> Guid.Empty
> 4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was used for
> redundant calls to UUID.fromString. If you really, REALLY, need it, just
> call Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for a String.
> 5. All sorts of ways to transform Strings to Guids were removed. If you have
> a literal you trust, just use new Guid(String). If you suspect it may be
> null, use Guid.createGuidFromString[DefaultEmpty]
> 6. NewGuid is now called newGuid. We're in Java, not C# :-)
> 
> 
> Many thanks to everyone who reviewed this patchset.
> You guys rock!
> 
> 
> Regards,
> Allon
> 
> 
> [1]
> http://gerrit.ovirt.org/#/q/project:ovirt-engine+branch:master+topic:guid-cleanup,n,z
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

[Engine-devel] Guid improvements

[Allon Mureinik]

Hi all,

I just merged a couple of improvements to the [N]Guid class [1] to improve it's 
performance both CPU-wise and memory-wise, based on a set of benchmarks 
presented by Liran.

What this patchset achieves:
1. Clean up the code, so it's easier to understand and use
2. Eliminate the inflation in the memory foot print caused by the getValue() 
method
3. Eliminate all the heavy calls to UUID.fromString when creating a new/empty 
Guid instance as a default value
4. Note that the cleanups proposed in (1) will have minor performance benefits 
(e.g., eliminating useless conditional statements), but I doubt this would be 
anything to write home about.

>From a developer's perspective, here's what changed:
1. No more NGuid, just Guid. Both static methods to create a Guid from String 
still exist, and are named createGuidFromString and 
createGuidFromStringDefaultEmpty.
2. [N]Guid.getValue() was removed, it's no longer needed after (1) was 
implemented
3. The Guid() constructor was made private, as it forced a redundant call to 
UUID.fromString(String). If you need an empty Guid instance, just use Guid.Empty
4. The Guid.EMPTY_GUID_VALUE string constant was removed, as it was used for 
redundant calls to UUID.fromString. If you really, REALLY, need it, just call 
Guid.Empty.getValue() for a UUID or Guid.Empty.toString() for a String.
5. All sorts of ways to transform Strings to Guids were removed. If you have a 
literal you trust, just use new Guid(String). If you suspect it may be null, 
use Guid.createGuidFromString[DefaultEmpty]
6. NewGuid is now called newGuid. We're in Java, not C# :-)


Many thanks to everyone who reviewed this patchset.
You guys rock!


Regards,
Allon


[1] 
http://gerrit.ovirt.org/#/q/project:ovirt-engine+branch:master+topic:guid-cleanup,n,z
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

[Engine-devel] development installation issues

[Michael Pasternak]

does anyone faced this with new 'development installation'?

[mpastern@lpt21-f ovirt-engine]$ 
/home/mpastern/Coding/ovirt/ovirt-engine/bin/engine-setup-2
[ ERROR ] Failed to execute stage 'Booting': 5
[ INFO  ] Stage: Initializing
  Setup was run under unprivileged user this will produce development 
installation do you wish to proceed? (Yes, No) [No]: Yes
[WARNING] engine-setup-2 is a technical preview, and yet to include all 
functionality that exists in legacy engine-setup. Specifically, engine-setup-2 
does not support
upgrade from previous installations.
[mpastern@lpt21-f ovirt-engine]$

-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] SSH Soft Fencing

[Eli Mesika]

- Original Message -
> From: "Livnat Peer" 
> To: "Yair Zaslavsky" 
> Cc: engine-devel@ovirt.org
> Sent: Sunday, June 30, 2013 8:06:28 AM
> Subject: Re: [Engine-devel] SSH Soft Fencing
> 
> On 06/30/2013 05:46 AM, Yair Zaslavsky wrote:
> > 
> > 
> > - Original Message -
> >> From: "Barak Azulay" 
> >> To: "Martin Perina" 
> >> Cc: "Yair Zaslavsky" , engine-devel@ovirt.org, "Eli
> >> Mesika" 
> >> Sent: Thursday, June 27, 2013 8:31:35 PM
> >> Subject: Re: SSH Soft Fencing
> >>
> >>
> >>
> >> - Original Message -
> >>> From: "Eli Mesika" 
> >>> To: "Yair Zaslavsky" 
> >>> Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
> >>> Azulay" 
> >>> Sent: Thursday, June 27, 2013 5:55:29 PM
> >>> Subject: Re: SSH Soft Fencing
> >>>
> >>>
> >>>
> >>> - Original Message -
>  From: "Yair Zaslavsky" 
>  To: "Eli Mesika" 
>  Cc: "Martin Perina" , engine-devel@ovirt.org, "Barak
>  Azulay" 
>  Sent: Thursday, June 27, 2013 5:43:17 PM
>  Subject: Re: SSH Soft Fencing
> 
> 
> 
>  - Original Message -
> > From: "Eli Mesika" 
> > To: "Martin Perina" 
> > Cc: engine-devel@ovirt.org, "Yair Zaslavsky" ,
> > "Barak
> > Azulay" 
> > Sent: Thursday, June 27, 2013 3:48:39 PM
> > Subject: Re: SSH Soft Fencing
> >
> >
> >
> > - Original Message -
> >> From: "Martin Perina" 
> >> To: engine-devel@ovirt.org
> >> Cc: "Yair Zaslavsky" , "Barak Azulay"
> >> , "Eli Mesika" 
> >> Sent: Thursday, June 27, 2013 1:51:06 PM
> >> Subject: SSH Soft Fencing
> >>
> >> Hi,
> >>
> >> SSH Soft Fencing is a new feature for 3.3 and it tries to restart
> >> VDSM
> >> using SSH connection on non responsive hosts prior to real fencing.
> >> More info can be found at
> >>
> >> http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
> >>
> >> In current SSH Soft Fencing implementation the restart VDSM using SSH
> >> command is part of standard fencing implementation in
> >> VdsNotRespondingTreatmentCommand. But this command is executed only
> >> if a host has a valid PM configuration. If host doesn't have a valid
> >> PM configuration, the execution of the command is disabled and host
> >> state is change to Non Responsive.
> >>
> >> So my question are:
> >>
> >> 1) Should SSH Soft Fencing be executed on hosts without valid PM
> >>configuration?
> >
> > I think that the answer should be yes. The vdsm restart will solve most
> > of
> > problems , so why not using it whether a PM agent is defined or not.
>  I agree.
>  I would like to say that I also don't like the fact that
>  VdsNotRespondingTreatment extends RestartVdsCommand.
>  One should ask if "non responding treatment is a restart vds operation"
>  or
>  maybe RestartVdsCommand is just a step in the non responding treatment
>  (inheritance vs containment/delegation).
>  I think that VdsNotRespodingTreatment should delegate the call to
>  RestartVdsCommand as the 2nd step after issuing the Soft Fencing
>  command.
>  Thoughts anyone?
> >>>
> >>> That would be a nice and needed re-factoring
> >>
> >> I would say yes - but would add it only with appropriate configuration
> >> (enableAutoSoftVdsmRestartWhenNoPMAvailable  I hate the name)
> > 
> > +1 on configuration.
> > Configuration must reside at host-related entities (i.e - VdsStatic).
> > 
> > Yair
> > 
> 
> Why would a user like to avoid fencing VDSM when host becomes
> non-responsive?
> 
> I think that adding another configuration option is cumbersome with no
> real value.

I totally agree with Livnat here , restarting vdsm may resolve problems and is 
much less brutal that host restart, should be implemented without any 
configuration IMHO

> 
> Livnat
> 
> >>
> >>  
> >>
> >>>
> 
> >
> >>
> >> 2) Should VDSM restart using SSH command be reimplemented
> >>as standalone command to be usable also in other parts of engine?
> >>If 1) is true, I think it will have to be done anyway.
> 
>  I agree here.
> >
> > +1
> >>
> >> On one hand it makes sense,  but I have several questions on the above:
> >> - Who do we think may want to use such a command ?
> >> - Should (or even can) we limit the use of such command to
> >> noneResponsiveTreatment ?
> >>
> >> Having general commands available to all code when there is only one
> >> specific
> >> case we are using it might be a bit riskey,
> >> Especially when we talk about restarting something.
> >>
> >> Thoughts ?
> >>
> >>
> >>
> >
> >>
> >>
> >> Martin Perina
> >>
> >
> 
> >>>
> >>
> > ___
> > Engine-devel mailing list
> > Engine-devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 
> ___
> Engine-devel