Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not start (RESOLVED)

[Will Dennis]

The vdsmd startup failing problem is now resolved :) The root cause was that 
the /etc/sudoers file (that we maintain a customized version of which thru 
Ansible) was missing the "#includedir /etc/sudoers.d" stanza, which caused a 
sudo problem with the vdsmd startup, since the sudoers include file "50-vdsm" 
wasn't being parsed.

In going back and reviewing the thread, this was the first post that pointed 
out the real problem...

-Original Message-
From: Fabian Deutsch [mailto:fdeutsch at redhat.com] 
Sent: Tuesday, December 01, 2015 12:58 AM
To: Will Dennis
Cc: Simone Tiraboschi; users
Subject: Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not 
start

On Tue, Dec 1, 2015 at 4:52 AM, Will Dennis  wrote:
> Any clues out of the strace of vdsm?

read(9, "sudo: a password is required\n", 4096) = 29

Could it be that sudo is not configured to operate passwordless?

The strat-up can then fail, because sudo requires a ty, but this isn't 
available during service start.

- fabian

> On Nov 25, 2015, at 11:29 AM, Simone Tiraboschi  
> wrote:
> 
>> On Wed, Nov 25, 2015 at 5:22 PM, Willard Dennis  
>> wrote:
>> [root at ovirt-node-01 ~]# sudo -u vdsm /bin/bash
>> bash-4.2$ /usr/share/vdsm/vdsm
>> bash-4.2$ echo $?
>> 1
> 
> Can you please use strace on it?

-Message End-

However, not knowing about the missing sudoers include file, we continued to 
chase a vdsm config problem (uninstall / reinstall vdsm RPM pkgs, & use 
"vdsm-tool configure --force")

Shame on me, it looks like I didn't think to check /var/log/messages for 
"vdsm"-string entries until today, where we found the "vdsm user could not 
manage to run sudo operation: (stderr: ['sudo: sorry, you must have a tty to 
run sudo']). Verify sudoer rules configuration” message that ultimately led to 
the solution :(

However, not sure if it's possible to do so, but could /usr/share/vdsm/vdsm be 
made to emit a error to STDERR if it cannot run correctly? That's the only 
thing I could think of that may have helped me to resolve this earlier with 
what I did do...

Thanks for everyone's kind assistance, this really is a great community! :)

-Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted Engine setup - got " Failed to start service 'ovirt-ha-agent' "

[Will Dennis]

I yum updated my hosts, and it did update ovirt-hosted-engine-ha on all to 
1.3.3.5 (two of my hosts include the one I did the engine install on were 
previously 1.3.3.4, and the third one was 1.3.3.3 for some reason)
Shortly thereafter, I began getting ovirt-hosted-engine state machine emails, 
and when I checked the state of the ovirt-ha-[agent,broker] services, they were 
running. When I got the email saying “EngineStarting-EngineUp”, I checked the 
web UI, and it was available, and I could successfully log into the admin site 
:)

Thanks for your help, and onwards!
W.

On Dec 18, 2015, at 4:55 PM, Simone Tiraboschi 
mailto:stira...@redhat.com>> wrote:

Today we async released ovirt-hosted-engine-ha-1.3.3.5-1 that should fix it.
Can you please check if you are already with that?
If not please update it and manually restart ovirt-ha-broker and ovirt-ha-agent 
services, I'm quite confident the it should be enough.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

Doing a “hosted-engine —setup” on my 2nd oVirt host, and am getting the 
following error:


  --== SYSTEM CONFIGURATION ==--

[WARNING] A configuration file must be supplied to deploy Hosted Engine on an 
additional host.
  The answer file may be fetched from the first host using scp.
  If you do not want to download it automatically you can abort the 
setup answering no to the following question.
  Do you want to scp the answer file from the first host? (Yes, 
No)[Yes]:
  Please provide the FQDN or IP of the first host: 249.26.200.91
  Enter 'root' user password for host 249.26.200.91:
[ ERROR ] Failed to execute stage 'Environment customization': [Errno 2] No 
such file
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file 
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20151218205656.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination


The problem may be slowness of the SSH connection to the 1st host; ever since I 
did the hosted engine setup on the 1st host and it created the ‘ovirtmgmt’ 
bridge et al, it now for some reason takes a long time (~50 sec’s) to SSH into 
the host…
Example: (key exchange was first done from 2nd host to first host)

[root@ovirt-node-02 ~]# time ssh root@ovirt-node-01 uptime
 21:37:33 up 1 day, 10:43,  0 users,  load average: 0.14, 0.15, 0.14

real0m50.185s
user0m0.048s
sys 0m0.006s

So maybe because of that reason, the setup SCP session is timing out?? How can 
I fix this?
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

Hi Joop,

I thought about that, but DNS resolution seems to be fine…


[root@ovirt-node-01 ~]# time dig ovirt-node-02.nec-labs.com

; <<>> DiG 9.9.4-RedHat-9.9.4-29.el7_2.1 <<>> ovirt-node-02.nec-labs.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4574
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;ovirt-node-02.nec-labs.com.  IN  A

;; ANSWER SECTION:
ovirt-node-02.nec-labs.com. 3600 IN A 249.26.200.92

;; Query time: 0 msec
;; SERVER: 249.26.108.11#53(249.26.108.11)
;; WHEN: Sat Dec 19 21:30:42 EST 2015
;; MSG SIZE  rcvd: 71


real  0m0.010s
user  0m0.005s
sys 0m0.005s
[root@ovirt-node-01 ~]# time dig -x 249.26.200.92

; <<>> DiG 9.9.4-RedHat-9.9.4-29.el7_2.1 <<>> -x 249.26.200.92
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65149
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;92.200.15.138.in-addr.arpa.  IN  PTR

;; ANSWER SECTION:
92.200.15.138.in-addr.arpa. 3600 IN PTR ovirt-node-02.nec-labs.com.

;; Query time: 0 msec
;; SERVER: 249.26.108.11#53(249.26.108.11)
;; WHEN: Sat Dec 19 21:31:18 EST 2015
;; MSG SIZE  rcvd: 95


real  0m0.010s
user  0m0.007s
sys 0m0.002s
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

OK, after all the t’shooting I could think to do, out of desperation I put the 
engine VM in maint mode, then rebooted the host. When it came up, I can now SSH 
into it normally (no long delay now.) I saw two state email changes come in 
after the reboot:

- StartState-ReinitializeFSM
- ReinitializeFSM-LocalMaintenance

But the engine VM never started… Do I have to do something at the host (issue a 
command) to get the engine VM to start? (Sorry if I’m ignorant, an oVirt noob 
here…)

-Will

On Dec 19, 2015, at 9:38 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

Hi Joop,

I thought about that, but DNS resolution seems to be fine…


[root@ovirt-node-01 ~]# time dig 
ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com>

; <<>> DiG 9.9.4-RedHat-9.9.4-29.el7_2.1 <<>> 
ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4574
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com>.  IN  A

;; ANSWER SECTION:
ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com>. 3600 IN A 
249.26.200.92

;; Query time: 0 msec
;; SERVER: 249.26.108.11#53(249.26.108.11)
;; WHEN: Sat Dec 19 21:30:42 EST 2015
;; MSG SIZE  rcvd: 71


real  0m0.010s
user  0m0.005s
sys 0m0.005s
[root@ovirt-node-01 ~]# time dig -x 249.26.200.92

; <<>> DiG 9.9.4-RedHat-9.9.4-29.el7_2.1 <<>> -x 249.26.200.92
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65149
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;92.200.15.138.in-addr.arpa.  IN  PTR

;; ANSWER SECTION:
92.200.15.138.in-addr.arpa. 3600 IN PTR 
ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com>.

;; Query time: 0 msec
;; SERVER: 249.26.108.11#53(249.26.108.11)
;; WHEN: Sat Dec 19 21:31:18 EST 2015
;; MSG SIZE  rcvd: 95


real  0m0.010s
user  0m0.007s
sys 0m0.002s

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not start (RESOLVED)

[Will Dennis]

Fair enough… I think the phrase goes “the problem with idiot-proofing is that 
they keep making bigger idiots…” :)

Are you saying that vdsm isn’t a sub-project of oVirt? Who created it then, and 
where else is it used?


On Dec 20, 2015, at 2:07 AM, Yedidyah Bar David 
mailto:d...@redhat.com>> wrote:
Well, I can think of other things as well, such as doing such a test
during rpm/yum install stage, etc., but the question is how far should
we go - there are many other ways to break ovirt, and we can't
possibly consider all of them. That said, I guess patches are welcome
(but you'll need to discuss this with a vdsm maintainer, not me,
probably on gerrit and not here).

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

I have three hosts, each with the following specs:

CPU: (2) Intel Xeon X5450  @ 3.00GHz (8 cores total)
RAM: 24GB
Disk: 2 Gluster volumes (each distributed w/ 3 replicas) for oVirt; engine: 
1.9TB, vmdata: 3.7TB

I have allocated 8192MB RAM to the hosted engine.

-Will


> On Dec 20, 2015, at 3:17 AM, Joop van de Wege  wrote:
> No problem, we were all once.
> 
> 
> Just curious.  What specs sre your host (s) and engine vm?
> 
> Regards, 
> 
> Joop
> 

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

OK, thanks to the assist from Joob, I did startup the engine VM, and took it 
out of maint mode… Then I went to my second oVirt host, and tried 
“hosted-engine —setup” again, and I’m still getting the error message “Failed 
to execute stage 'Environment customization': [Errno 2] No such file”

Screen output of the “hosted-engine —setup” run:
http://ur1.ca/ocr5u

So, I guess that means it can’t find the answer file where it expects it on the 
first host… What do I do now?

-Will



On Dec 19, 2015, at 11:20 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

OK, after all the t’shooting I could think to do, out of desperation I put the 
engine VM in maint mode, then rebooted the host. When it came up, I can now SSH 
into it normally (no long delay now.) I saw two state email changes come in 
after the reboot:

- StartState-ReinitializeFSM
- ReinitializeFSM-LocalMaintenance

But the engine VM never started… Do I have to do something at the host (issue a 
command) to get the engine VM to start? (Sorry if I’m ignorant, an oVirt noob 
here…)

-Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

FYI, the answer file is located at:
/var/lib/ovirt-hosted-engine-setup/answers/answers-20151218124259.conf

There are many other answer files in that directory as well from prior 
unsuccessful runs… Does that matter?


On Dec 20, 2015, at 2:51 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

OK, thanks to the assist from Joob, I did startup the engine VM, and took it 
out of maint mode… Then I went to my second oVirt host, and tried 
“hosted-engine —setup” again, and I’m still getting the error message “Failed 
to execute stage 'Environment customization': [Errno 2] No such file”

Screen output of the “hosted-engine —setup” run:
http://ur1.ca/ocr5u

So, I guess that means it can’t find the answer file where it expects it on the 
first host… What do I do now?

-Will

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

No, it did not complete successfully- the installer threw a error at the end 
"Failed to start service 'ovirt-ha-agent' " during the "Closing up" stage due 
to the (now fixed) bug with the ha-agent systemd startup script So should I 
copy the correct answers log from /var/lib/ovirt-hosted-engine-setup to 
/etc/ovirt-hosted-engine/answers.conf and try the setup again on the second 
host?

-Will



Sent with Good (www.good.com)


-Original Message-
From: Simone Tiraboschi [stira...@redhat.com<mailto:stira...@redhat.com>]
Sent: Monday, December 21, 2015 03:42 AM Eastern Standard Time
To: Yedidyah Bar David
Cc: Will Dennis; users
Subject: Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when 
setting up 2nd host


On Mon, Dec 21, 2015 at 9:33 AM, Yedidyah Bar David 
mailto:d...@redhat.com>> wrote:
On Sun, Dec 20, 2015 at 10:04 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
> FYI, the answer file is located at:
> /var/lib/ovirt-hosted-engine-setup/answers/answers-20151218124259.conf
>
> There are many other answer files in that directory as well from prior 
> unsuccessful runs… Does that matter?

The answerfile is expected to be found in
/etc/ovirt-hosted-engine/answers.conf .
It's written there upon a successful completion of deploy.

Absolutely, I suggest you to start checking for 
/etc/ovirt-hosted-engine/answers.conf on the first host.
If the file is not here please check the setup logs on the first to ensure that 
it successful completed.


It's also always written to /var/lib/ovirt-hosted-engine-setup/answers/ .

You can also copy it by yourself to somefile and run:

hosted-engine --deploy --config-append=somefile

But I'd invest some time understanding why scp fails. Might be bad
name resolution?

>
>
> On Dec 20, 2015, at 2:51 PM, Will Dennis 
> mailto:wden...@nec-labs.com><mailto:wden...@nec-labs.com<mailto:wden...@nec-labs.com>>>
>  wrote:
>
> OK, thanks to the assist from Joob, I did startup the engine VM, and took it 
> out of maint mode… Then I went to my second oVirt host, and tried 
> “hosted-engine —setup” again, and I’m still getting the error message “Failed 
> to execute stage 'Environment customization': [Errno 2] No such file”
>
> Screen output of the “hosted-engine —setup” run:
> http://ur1.ca/ocr5u
>
> So, I guess that means it can’t find the answer file where it expects it on 
> the first host… What do I do now?
>
> -Will
>
> ___
> Users mailing list
> Users@ovirt.org<mailto:Users@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users



--
Didi
___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

OK, that did the trick... 2nd host is joined.

However, when I went to the 3rd host and did the setup, I selected 'glusterfs' 
and gave the path of the engine volume, it came back and incorrectly identified 
it as the first host, instead of an additional host... How does setup determine 
that? I confirmed that on this 3rd host that the engine volume is available and 
has the GUID subfolder of the hosted engine...



Sent with Good (www.good.com)


-Original Message-
From: Simone Tiraboschi [stira...@redhat.com<mailto:stira...@redhat.com>]
Sent: Monday, December 21, 2015 09:28 AM Eastern Standard Time
To: Will Dennis
Cc: Yedidyah Bar David; users
Subject: Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when 
setting up 2nd host



On Mon, Dec 21, 2015 at 3:14 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
No, it did not complete successfully- the installer threw a error at the end 
"Failed to start service 'ovirt-ha-agent' " during the "Closing up" stage due 
to the (now fixed) bug with the ha-agent systemd startup script

If there was an error in the close-up it was not really that successfully :-)

So should I copy the correct answers log from 
/var/lib/ovirt-hosted-engine-setup to /etc/ovirt-hosted-engine/answers.conf and 
try the setup again on the second host?

Yes, that should do the trick



-Will



Sent with Good (www.good.com<http://www.good.com>)


-Original Message-
From: Simone Tiraboschi [stira...@redhat.com<mailto:stira...@redhat.com>]
Sent: Monday, December 21, 2015 03:42 AM Eastern Standard Time
To: Yedidyah Bar David
Cc: Will Dennis; users
Subject: Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when 
setting up 2nd host


On Mon, Dec 21, 2015 at 9:33 AM, Yedidyah Bar David 
mailto:d...@redhat.com>> wrote:
On Sun, Dec 20, 2015 at 10:04 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
> FYI, the answer file is located at:
> /var/lib/ovirt-hosted-engine-setup/answers/answers-20151218124259.conf
>
> There are many other answer files in that directory as well from prior 
> unsuccessful runs… Does that matter?

The answerfile is expected to be found in
/etc/ovirt-hosted-engine/answers.conf .
It's written there upon a successful completion of deploy.

Absolutely, I suggest you to start checking for 
/etc/ovirt-hosted-engine/answers.conf on the first host.
If the file is not here please check the setup logs on the first to ensure that 
it successful completed.


It's also always written to /var/lib/ovirt-hosted-engine-setup/answers/ .

You can also copy it by yourself to somefile and run:

hosted-engine --deploy --config-append=somefile

But I'd invest some time understanding why scp fails. Might be bad
name resolution?

>
>
> On Dec 20, 2015, at 2:51 PM, Will Dennis 
> mailto:wden...@nec-labs.com><mailto:wden...@nec-labs.com<mailto:wden...@nec-labs.com>>>
>  wrote:
>
> OK, thanks to the assist from Joob, I did startup the engine VM, and took it 
> out of maint mode… Then I went to my second oVirt host, and tried 
> “hosted-engine —setup” again, and I’m still getting the error message “Failed 
> to execute stage 'Environment customization': [Errno 2] No such file”
>
> Screen output of the “hosted-engine —setup” run:
> http://ur1.ca/ocr5u
>
> So, I guess that means it can’t find the answer file where it expects it on 
> the first host… What do I do now?
>
> -Will
>
> ___
> Users mailing list
> Users@ovirt.org<mailto:Users@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users



--
Didi
___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

http://ur1.ca/ocstf

On Dec 21, 2015, at 11:58 AM, Simone Tiraboschi 
mailto:stira...@redhat.com>> wrote:

On Mon, Dec 21, 2015 at 5:52 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

However, when I went to the 3rd host and did the setup, I selected 'glusterfs' 
and gave the path of the engine volume, it came back and incorrectly identified 
it as the first host, instead of an additional host... How does setup determine 
that? I confirmed that on this 3rd host that the engine volume is available and 
has the GUID subfolder of the hosted engine...


Can you please attach a log of hosted-engine-setup also from there?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Hosted Engine crash - state = EngineUp-EngineUpBadHealth

[Will Dennis]

OK, another problem :(

I was having the same problem with my second oVirt host that I had with my 
first one, where when I ran “hosted-engine —deploy” on it, after it completed 
successfully, then I was experiencing a ~50sec lag when SSH’ing into the node…

vpnp71:~ will$ time ssh root@ovirt-node-02 uptime
 19:36:06 up 4 days,  8:31,  0 users,  load average: 0.68, 0.70, 0.67

real  0m50.540s
user  0m0.025s
sys 0m0.008s


So, in the oVirt web admin console, I put the "ovirt-node-02” node into 
Maintenance mode, then SSH’d to the server and rebooted it. Sure enough, after 
the server came back up, SSH was fine (no delay), which again was the same 
experience I had had with the first oVirt host. So, I went back to the web 
console, and choose the “Confirm host has been rebooted” option, which I 
thought would be the right action to take after a reboot. The system opened a 
dialog box with a spinner, which never stopped spinning… So finally, I closed 
the dialog box with the upper right (X) symbol, and then for this same host 
choose “Activate” from the menu. It was then I noticed I had recieved a state 
transition email notifying me that "EngineUp-EngineUpBadHealth” and sure 
enough, the web UI was then unresponsive. I checked on the first oVirt host, 
the VM with the name “HostedEngine” is still running, but obviously isn’t 
working… 

So, looks like I need to restart the HostedEngine VM or take whatever action is 
needed to return oVirt to operation… Hate to keep asking this question, but 
what’s the correct action at this point?

Thanks, again,
Will

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

Hi Simone,

It's actually OK that it's not fully supported yet, this is a PoC deployment to 
evaluate oVirt for our needs...

I will provide the VDSM logs shortly & post the URLs.

W.



Sent with Good (www.good.com)


-Original Message-
From: Simone Tiraboschi [stira...@redhat.com<mailto:stira...@redhat.com>]
Sent: Tuesday, December 22, 2015 03:58 AM Eastern Standard Time
To: Will Dennis; Sahina Bose
Cc: Yedidyah Bar David; users
Subject: Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when 
setting up 2nd host



On Tue, Dec 22, 2015 at 2:09 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
http://ur1.ca/ocstf



2015-12-21 11:28:39 DEBUG otopi.plugins.otopi.dialog.human 
dialog.__logString:219 DIALOG:SEND Please specify the full 
shared storage connection path to use (example: host:/path):
2015-12-21 11:28:55 DEBUG otopi.plugins.otopi.dialog.human 
dialog.__logString:219 DIALOG:RECEIVElocalhost:/engine

OK, so you are trying to deploy hosted-engine on GlusterFS in a hyper-converged 
way (using the same hosts for virtualization and for serving GlusterFS). 
Unfortunately I've to advise you that this is not a supported configuration on 
oVirt 3.6 due to different open bugs.
So I'm glad you can help us testing it but I prefer to advise that today that 
schema is not production ready.

In your case it seams that VDSM correctly connects the GlusterFS volume seeing 
all the bricks

2015-12-21 11:28:55 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.nfs 
plugin.execute:936 execute-output: ('/sbin/gluster', '--mode=script', '--xml', 
'volume', 'info', 'engine', '--remote-host=localhost') stdout:


  0
  0
  
  

  
engine
974c9da4-b236-4fc1-b26a-645f14601db8
1
Started
6
3

but then VDSM doesn't find any storage domain there:

otopi.plugins.ovirt_hosted_engine_setup.storage.storage.Plugin._late_customization
2015-12-21 11:29:58 DEBUG 
otopi.plugins.ovirt_hosted_engine_setup.storage.storage 
storage._getExistingDomain:476 _getExistingDomain
2015-12-21 11:29:58 DEBUG 
otopi.plugins.ovirt_hosted_engine_setup.storage.storage 
storage._storageServerConnection:638 connectStorageServer
2015-12-21 11:29:58 DEBUG 
otopi.plugins.ovirt_hosted_engine_setup.storage.storage 
storage._storageServerConnection:701 {'status': {'message': 'OK', 'code': 0}, 
'statuslist': [{'status': 0, 'id': '67ece152-dd66-444c-8d18-4249d1b8f488'}]}
2015-12-21 11:29:58 DEBUG 
otopi.plugins.ovirt_hosted_engine_setup.storage.storage 
storage._getStorageDomainsList:595 getStorageDomainsList
2015-12-21 11:29:59 DEBUG 
otopi.plugins.ovirt_hosted_engine_setup.storage.storage 
storage._getStorageDomainsList:598 {'status': {'message': 'OK', 'code': 0}, 
'domlist': []}

Can you please attach also the correspondent VDSM logs?

Adding Sahina here.


On Dec 21, 2015, at 11:58 AM, Simone Tiraboschi 
mailto:stira...@redhat.com><mailto:stira...@redhat.com<mailto:stira...@redhat.com>>>
 wrote:

On Mon, Dec 21, 2015 at 5:52 PM, Will Dennis 
mailto:wden...@nec-labs.com><mailto:wden...@nec-labs.com<mailto:wden...@nec-labs.com>>>
 wrote:

However, when I went to the 3rd host and did the setup, I selected 'glusterfs' 
and gave the path of the engine volume, it came back and incorrectly identified 
it as the first host, instead of an additional host... How does setup determine 
that? I confirmed that on this 3rd host that the engine volume is available and 
has the GUID subfolder of the hosted engine...


Can you please attach a log of hosted-engine-setup also from there?


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted Engine crash - state = EngineUp-EngineUpBadHealth

[Will Dennis]

The network should *not* be flakey - all hosts are plugged into a Cisco 
Catalyst 4500 switch. I can take a look at the port counters when I have a 
chance, but would not expect intermittent network disruptions.

Will post logs soon and provide URLs.

W.



Sent with Good (www.good.com)


-Original Message-
From: Sahina Bose [sab...@redhat.com<mailto:sab...@redhat.com>]
Sent: Tuesday, December 22, 2015 04:58 AM Eastern Standard Time
To: Simone Tiraboschi; Will Dennis; Dan Kenigsberg
Cc: users
Subject: Re: [ovirt-users] Hosted Engine crash - state = 
EngineUp-EngineUpBadHealth



On 12/22/2015 02:38 PM, Simone Tiraboschi wrote:


On Tue, Dec 22, 2015 at 2:31 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
OK, another problem :(

I was having the same problem with my second oVirt host that I had with my 
first one, where when I ran “hosted-engine —deploy” on it, after it completed 
successfully, then I was experiencing a ~50sec lag when SSH’ing into the node…

vpnp71:~ will$ time ssh root@ovirt-node-02 uptime
 19:36:06 up 4 days,  8:31,  0 users,  load average: 0.68, 0.70, 0.67

real  0m50.540s
user  0m0.025s
sys 0m0.008s


So, in the oVirt web admin console, I put the "ovirt-node-02” node into 
Maintenance mode, then SSH’d to the server and rebooted it. Sure enough, after 
the server came back up, SSH was fine (no delay), which again was the same 
experience I had had with the first oVirt host. So, I went back to the web 
console, and choose the “Confirm host has been rebooted” option, which I 
thought would be the right action to take after a reboot. The system opened a 
dialog box with a spinner, which never stopped spinning… So finally, I closed 
the dialog box with the upper right (X) symbol, and then for this same host 
choose “Activate” from the menu. It was then I noticed I had recieved a state 
transition email notifying me that "EngineUp-EngineUpBadHealth” and sure 
enough, the web UI was then unresponsive. I checked on the first oVirt host, 
the VM with the name “HostedEngine” is still running, but obviously isn’t 
working…

So, looks like I need to restart the HostedEngine VM or take whatever action is 
needed to return oVirt to operation… Hate to keep asking this question, but 
what’s the correct action at this point?


ovirt-ha-agent should always restart it for you after a few minutes but the 
point is that the network configuration seams to be not that stable.

I know from another thread that you are trying to deploy hosted-engine over 
GlusterFS in an hyperconverged way and this, as I said, is currently not 
supported.
I think that it can also requires some specific configuration on network side.

For hyperconverged gluster+engine , it should work without any specific 
configuration on network side. However if the network is flaky, it is possible 
that there are errors with gluster volume access. Could you provide the 
ovirt-ha-agent logs as well as gluster mount logs?


Adding Sahina and Dan here.

Thanks, again,
Will

___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when setting up 2nd host

[Will Dennis]

Yes, OK to attached needed logs to bug report...


From: Simone Tiraboschi [mailto:stira...@redhat.com]
Sent: Tuesday, December 22, 2015 9:27 AM
To: Will Dennis; users
Cc: Sahina Bose; Yedidyah Bar David; Nir Soffer
Subject: Re: [ovirt-users] Cannot retrieve answer file from 1st HE host when 
setting up 2nd host



On Tue, Dec 22, 2015 at 3:06 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
See attached for requested logs


Thanks, the issue is here:
Dec 21 19:40:53 ovirt-node-03 etc-glusterfs-glusterd.vol[1079]: [2015-12-22 
00:40:53.496109] C [MSGID: 106002] 
[glusterd-server-quorum.c:351:glusterd_do_volume_quorum_action] 0-management: 
Server quorum lost for volume engine. Stopping local bricks.
Dec 21 19:40:53 ovirt-node-03 etc-glusterfs-glusterd.vol[1079]: [2015-12-22 
00:40:53.496410] C [MSGID: 106002] 
[glusterd-server-quorum.c:351:glusterd_do_volume_quorum_action] 0-management: 
Server quorum lost for volume vmdata. Stopping local bricks.

So at that point gluster lost its quorum and the fail system got read-only.

On the getStorageDomainsList VDSM internally raises cause the file-system is 
read only:

Thread-141::DEBUG::2015-12-21 
11:29:59,666::fileSD::157::Storage.StorageDomainManifest::(__init__) Reading 
domain in path 
/rhev/data-center/mnt/glusterSD/localhost:_engine/e89b6e64-bd7d-4846-b970-9af32a3295ee
Thread-141::DEBUG::2015-12-21 
11:29:59,666::__init__::320::IOProcessClient::(_run) Starting IOProcess...
Thread-141::DEBUG::2015-12-21 
11:29:59,680::persistentDict::192::Storage.PersistentDict::(__init__) Created a 
persistent dict with FileMetadataRW backend
Thread-141::ERROR::2015-12-21 
11:29:59,686::hsm::2898::Storage.HSM::(getStorageDomainsList) Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/hsm.py", line 2882, in getStorageDomainsList
dom = sdCache.produce(sdUUID=sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 100, in produce
domain.getRealDomain()
  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
return self._cache._realProduce(self._sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 124, in _realProduce
domain = self._findDomain(sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 143, in _findDomain
dom = findMethod(sdUUID)
  File "/usr/share/vdsm/storage/glusterSD.py", line 32, in findDomain
return GlusterStorageDomain(GlusterStorageDomain.findDomainPath(sdUUID))
  File "/usr/share/vdsm/storage/fileSD.py", line 198, in __init__
validateFileSystemFeatures(manifest.sdUUID, manifest.mountpoint)
  File "/usr/share/vdsm/storage/fileSD.py", line 93, in 
validateFileSystemFeatures
oop.getProcessPool(sdUUID).directTouch(testFilePath)
  File "/usr/share/vdsm/storage/outOfProcess.py", line 350, in directTouch
ioproc.touch(path, flags, mode)
  File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 543, in 
touch
self.timeout)
  File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 427, in 
_sendCommand
raise OSError(errcode, errstr)
OSError: [Errno 30] Read-only file system

But instead of reporting a failure to hosted-engine-setup, it reported a 
successfully execution where it wasn't able to find any storage domain there ( 
this one is a real bug, I'm going to open a bug on that, can I attach your logs 
there? ):

Thread-141::INFO::2015-12-21 11:29:59,702::logUtils::51::dispatcher::(wrapper) 
Run and protect: getStorageDomainsList, Return response: {'domlist': []}
Thread-141::DEBUG::2015-12-21 
11:29:59,702::task::1191::Storage.TaskManager.Task::(prepare) 
Task=`96a9ea03-dc13-483e-9b17-b55a759c9b44`::finished: {'domlist': []}
Thread-141::DEBUG::2015-12-21 
11:29:59,702::task::595::Storage.TaskManager.Task::(_updateState) 
Task=`96a9ea03-dc13-483e-9b17-b55a759c9b44`::moving from state preparing -> 
state finished
Thread-141::DEBUG::2015-12-21 
11:29:59,703::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) 
Owner.releaseAll requests {} resources {}
Thread-141::DEBUG::2015-12-21 
11:29:59,703::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) 
Owner.cancelAll requests {}
Thread-141::DEBUG::2015-12-21 
11:29:59,703::task::993::Storage.TaskManager.Task::(_decref) 
Task=`96a9ea03-dc13-483e-9b17-b55a759c9b44`::ref 0 aborting False
Thread-141::INFO::2015-12-21 
11:29:59,704::xmlrpc::92::vds.XMLRPCServer::(_process_requests) Request handler 
for 127.0.0.1:39718<http://127.0.0.1:39718> stopped

And so, cause VDSM doesn't report any existing storage domain, 
hosted-engine-setup assumes that you are going to deploy the first host and so 
your original issue.



From: Simone Tiraboschi [mailto:stira...@redhat.com<mailto:stira...@redhat.com>]
Sent: Tuesday, December 22, 2015 8:56 AM
To: Will Dennis
Cc: Sahina Bose; Yedidyah Bar David

Subject: Re: [ovirt-users] Cann

Re: [ovirt-users] Hosted Engine crash - state = EngineUp-EngineUpBadHealth

[Will Dennis]

Ah OK, my ignorance strikes again... I flushed all the rules on IPtables on 
hosts -01 and -02, and now Gluster seems up and happy...

I saw messages after I flushed IPtables that the engine state was changing, and 
eventually it landed on status: ReinitializeFSM-LocalMaintenance
...but I could not log into the admin website.

I then queried the engine VM state at the CLI, and saw it was thus:

[root@ovirt-node-01 ~]# hosted-engine --vm-status


--== Host 1 status ==--

Status up-to-date  : True
Hostname   : ovirt-node-01
Host ID: 1
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "up", "detail": "paused"}
Score  : 3400
stopped: False
Local maintenance  : False
crc32  : 538868a0
Host timestamp : 214954


--== Host 2 status ==--

Status up-to-date  : True
Hostname   : ovirt-node-02
Host ID: 2
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}
Score  : 0
stopped: False
Local maintenance  : True
crc32  : 419a0c6d
Host timestamp : 53528


So, I issued the command “hosted-engine --vm-shutdown” on host -01, and it 
eventually came down (had to be force-killed per status emails) and then HA 
restarted it :)

Looks like I’m back to being good now... Thanks everyone for the assist, and 
talk to you soon, I’m sure ;)


-Will

From: Simone Tiraboschi [mailto:stira...@redhat.com]
Sent: Tuesday, December 22, 2015 10:23 AM
To: Will Dennis; users
Cc: Sahina Bose; Dan Kenigsberg
Subject: Re: [ovirt-users] Hosted Engine crash - state = 
EngineUp-EngineUpBadHealth


hosted-engine-setup asks:
  iptables was detected on your computer, do you wish setup to 
configure it? (Yes, No)[Yes]:

You have just to say no here.

If you say no it's completely up to you to configure it opening the required 
ports or everything disabling it if you don't care.

The issue with gluster ports is that hosted-engine-setup simply configure 
iptables for what it knows you'll need and on 3.6 it's always assuming that the 
gluster volume is served by external hosts.



From: Sahina Bose [mailto:sab...@redhat.com<mailto:sab...@redhat.com>]
Sent: Tuesday, December 22, 2015 9:19 AM
To: Will Dennis; Simone Tiraboschi; Dan Kenigsberg

Subject: Re: [ovirt-users] Hosted Engine crash - state = 
EngineUp-EngineUpBadHealth


On 12/22/2015 07:47 PM, Sahina Bose wrote:

On 12/22/2015 07:28 PM, Will Dennis wrote:
See attached for requested log files

From gluster logs

[2015-12-22 00:40:53.501341] W [MSGID: 108001] [afr-common.c:3924:afr_notify] 
0-engine-replicate-1: Client-quorum is not met
[2015-12-22 00:40:53.502288] W [socket.c:588:__socket_rwv] 0-engine-client-2: 
readv on 138.15.200.93:49217<http://138.15.200.93:49217> failed (No data 
available)

[2015-12-22 00:41:17.667302] W [fuse-bridge.c:2292:fuse_writev_cbk] 
0-glusterfs-fuse: 3875597: WRITE => -1 (Read-only file system)

Could you check if the gluster ports are open on all nodes?

It's possible you ran into this ? - 
https://bugzilla.redhat.com/show_bug.cgi?id=1288979




From: Sahina Bose [mailto:sab...@redhat.com]
Sent: Tuesday, December 22, 2015 4:59 AM
To: Simone Tiraboschi; Will Dennis; Dan Kenigsberg
Cc: users
Subject: Re: [ovirt-users] Hosted Engine crash - state = 
EngineUp-EngineUpBadHealth


On 12/22/2015 02:38 PM, Simone Tiraboschi wrote:


On Tue, Dec 22, 2015 at 2:31 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
OK, another problem :(

I was having the same problem with my second oVirt host that I had with my 
first one, where when I ran “hosted-engine —deploy” on it, after it completed 
successfully, then I was experiencing a ~50sec lag when SSH’ing into the node…

vpnp71:~ will$ time ssh root@ovirt-node-02 uptime
 19:36:06 up 4 days,  8:31,  0 users,  load average: 0.68, 0.70, 0.67

real  0m50.540s
user  0m0.025s
sys 0m0.008s


So, in the oVirt web admin console, I put the "ovirt-node-02” node into 
Maintenance mode, then SSH’d to the server and rebooted it. Sure enough, after 
the server came back up, SSH was fine (no delay), which again was the same 
experience I had had with the first oVirt host. So, I went back to the web 
console, and choose the “Confirm host has been rebooted” option, which I 
thought would be the right action to take after a reboot. The system opened a 
dialog box with a spinner, which never stopped spinning… So finally, I closed 
the dialog box wi

[ovirt-users] How to run "engine-backup"?

[Will Dennis]

Yay, I *finally* have my 3-host hyper-converged oVirt datacenter stood up :)

[root@ovirt-node-01 ~]# hosted-engine --vm-status


--== Host 1 status ==--

Status up-to-date  : True
Hostname   : ovirt-node-01
Host ID: 1
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Score  : 3400
stopped: False
Local maintenance  : False
crc32  : 65c41ca5
Host timestamp : 217522


--== Host 2 status ==--

Status up-to-date  : True
Hostname   : ovirt-node-02
Host ID: 2
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}
Score  : 3400
stopped: False
Local maintenance  : False
crc32  : a7a599d8
Host timestamp : 56101


--== Host 3 status ==--

Status up-to-date  : True
Hostname   : ovirt-node-03
Host ID: 3
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}
Score  : 3400
stopped: False
Local maintenance  : False
crc32  : 6e138d0b
Host timestamp : 432658



Now in the oVirt webadmin UI down in the "Alerts" section, I am seeing this 
message:

"There is no full backup available, please run engine-backup to prevent data 
loss in case of corruption."

I do not see a "engine-backup" CLI command on my hosts; how does one do this? 
(I have searched ovirt.org to no avail...)

Thanks,
Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] How to run "engine-backup"?

[Will Dennis]

Are there any docs on using this (other than the ‘—help” option text)?

Also, I do not have a GUI environment on my oVirt hosts; how to I display the 
console of the hosted engine VM if I need it?

From: Simone Tiraboschi [mailto:stira...@redhat.com]
Sent: Tuesday, December 22, 2015 11:44 AM
To: Will Dennis
Cc: users
Subject: Re: [ovirt-users] How to run "engine-backup"?

yes, engine-backup is on the engine VM :-)

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] How to run "engine-backup"?

[Will Dennis]

n/m, found this URL cleverly concealed in the output of “—help” ;)

http://www.ovirt.org/Ovirt-engine-backup

Still would be good to know how I could get a console...

From: Will Dennis
Sent: Tuesday, December 22, 2015 1:37 PM
To: 'Simone Tiraboschi'
Cc: users
Subject: RE: [ovirt-users] How to run "engine-backup"?

Are there any docs on using this (other than the ‘—help” option text)?

Also, I do not have a GUI environment on my oVirt hosts; how to I display the 
console of the hosted engine VM if I need it?

From: Simone Tiraboschi [mailto:stira...@redhat.com]
Sent: Tuesday, December 22, 2015 11:44 AM
To: Will Dennis
Cc: users
Subject: Re: [ovirt-users] How to run "engine-backup"?

yes, engine-backup is on the engine VM :-)

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] How to add a Gluster storage domain on hyper-converged?

[Will Dennis]

Hi all,

I have a three-node hyper-converged oVirt datacenter running; now I need to add 
my first storage domain. I had prepped for this before installing oVirt by 
creating two distributed Gluster volumes with 3x replicas (one for the hosted 
engine, one for VM storage) -

[root@ovirt-node-01 ~]# gluster volume info | grep -e "Name" -e "Type" -e 
"Number"
Volume Name: engine
Type: Distributed-Replicate
Number of Bricks: 2 x 3 = 6
Volume Name: vmdata
Type: Distributed-Replicate
Number of Bricks: 2 x 3 = 6

Now I'd like to use the "vmdata" volume for my storage domain. When in webadmin 
I select "New Domain" I get a dialog that lets me select GlusterFS as the 
storage type, but then requires a "Use host:" setting, and a path. Is it 
possible for me to select one of my oVirt hosts (they all have the 'vmdata' 
volume), and then use "localhost:/vmdata" for the path? Or will this not work?

I know this isn't officially supported yet, but if I can get it to work 
somehow, that'd be great :)  It's a non-production (PoC) setup, so the cost of 
failure should be low... That said, I don't want to trash my rig and have to 
redo the whole thing all over ;)

Thanks,
Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Hosted engine vs. hypervisor hosts naming on hyperconverged setup

[Will Dennis]

Hi all,

I have a hyperconverged setup where I have a hosted engine that runs on one of 
three hosts, which are named “ovirt-node-[01,02,03]” —

[root@ovirt-node-01 ~]# hosted-engine --vm-status | grep -e "Hostname" -e 
"Engine"
Hostname   : ovirt-node-01
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-02
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown”}

When I deployed the hosted engine, I gave it a separate hostname/IP, as I 
expected it would need it (hostname = “ovirt-engine-01”) 

However, when I look at the hosts in the oVirt web admin screen, I see that the 
first host has the name “ovirt-engine-01” whereas it has the hostname of 
“ovirt-node-01”

I also notice that even though the 1st host is showing the number of VM’s 
running as “1”, when I click on the “VMs” node, there are no VMs showing.

Not sure of what “should be”, but, would expect that the first host would have 
a name equal to its DNS hostname (like the other two do), and that in VMs I 
would see the engine VM. But is that not how a hosted engine setup works?

And if not, if the engine VM migrates over to another host, will that host gain 
the name "“ovirt-engine-01”?

I ask this now because I want to set up a storage domain on these hosts using 
GlusterFS, and I have to select a host to base the connection on. In the “Use 
Host” dropbox, I currently see the values:

ovirt-node-03
ovirt-node-02
ovirt-engine-01

I would expect that the last entry would be for “ovirt-host-01”, not 
“ovirt-engine-01”… 

I don’t want to set up the storage domain until I figure this out, so as to 
prevent potential breakage...

Thanks,
Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on hyperconverged setup

[Will Dennis]

I meant “ovirt-node-01” below, not “ovirt-host-01”… so the sentence should have 
read:

‘I would expect that the last entry would be for “ovirt-node-01”, not 
“ovirt-engine-01”…’

Sorry for any confusion...

> On Dec 26, 2015, at 11:28 AM, Will Dennis  wrote:
> 
> 
> I would expect that the last entry would be for “ovirt-host-01”, not 
> “ovirt-engine-01”… 
> 

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on hyperconverged setup

[Will Dennis]

Hi Roy,

I did not ever change the engine name… When I deployed it with ‘hosted-engine 
—setup’ I gave it its’ own hostname/IP, assuming it would need one that’s 
different from any of the virtualization hosts. So the virt hosts have names / 
IPs as follows:

ovirt-node-01   xxx.yyy.zzz.91
ovirt-node-02   xxx.yyy.zzz.92
ovirt-node-03   xxx.yyy.zzz.93

…and the engine VM has the hostname / IP as follows:

ovirt-engine-01   xxx.yyy.zzz.95

The engine setup log can be viewed at: http://ur1.ca/od171

I clicked on the “Data Centers” node on the left nav of webadmin, and I see 
that the “Default” DC (the only one I have) has a status of “Uninitialized”… 
Not sure how to initialize it (there’s no menu option for that on the node…)

Thanks,
Will

On Dec 27, 2015, at 1:38 PM, Roy Golan 
mailto:rgo...@redhat.com>> wrote:



On Sat, Dec 26, 2015 at 6:27 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
Hi all,

I have a hyperconverged setup where I have a hosted engine that runs on one of 
three hosts, which are named “ovirt-node-[01,02,03]” —

[root@ovirt-node-01 ~]# hosted-engine --vm-status | grep -e "Hostname" -e 
"Engine"
Hostname   : ovirt-node-01
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-02
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown”}

When I deployed the hosted engine, I gave it a separate hostname/IP, as I 
expected it would need it (hostname = “ovirt-engine-01”)


So when exactly did you changed the hostname?
And what did you pass to the hosted engine install as answers? perhaps share 
the setup log under /var/log/ovirt-hosted-engine-ha

However, when I look at the hosts in the oVirt web admin screen, I see that the 
first host has the name “ovirt-engine-01” whereas it has the hostname of 
“ovirt-node-01”

I also notice that even though the 1st host is showing the number of VM’s 
running as “1”, when I click on the “VMs” node, there are no VMs showing.


You have to have a data center active, and only after that you would be able to 
see your engine VM.  This is a must. First your hosted_storage will be imported 
into the setup automatically followed by the engine vm import.


Not sure of what “should be”, but, would expect that the first host would have 
a name equal to its DNS hostname (like the other two do), and that in VMs I 
would see the engine VM. But is that not how a hosted engine setup works?

And if not, if the engine VM migrates over to another host, will that host gain 
the name "“ovirt-engine-01”?


no. We don't mess with that. The hostname is static in the engine.

I ask this now because I want to set up a storage domain on these hosts using 
GlusterFS, and I have to select a host to base the connection on. In the “Use 
Host” dropbox, I currently see the values:

ovirt-node-03
ovirt-node-02
ovirt-engine-01

I would expect that the last entry would be for “ovirt-host-01”, not 
“ovirt-engine-01”…

I don’t want to set up the storage domain until I figure this out, so as to 
prevent potential breakage...

Thanks,
Will
___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on hyperconverged setup

[Will Dennis]

Did a bit of RTFM this evening, and read in the Admin Guide that:
"The new data center is added to the virtualization environment. It will remain 
Uninitialized until a cluster, host, and storage domain are configured for it; 
use Guide Me to configure these entities."

So, in webadmin I clicked on the Data Centers node, and then clicked on the 
“Guide Me” link; sure enough, in the “There are still unconfigured entities:” 
section, there was a link for “Attach Storage”. When I clicked on it, I saw 
that there was an option to attach the “hosted_storage” storage domain; when I 
tried to do so, it failed, and I got the following events at the bottom of the 
webadmin UI:

Dec 27, 2015 8:58:18 PM  VDSM ovirt-node-03 command failed: Cannot acquire host 
id: (u'e89b6e64-bd7d-4846-b970-9af32a3295ee', SanlockException(22, 'Sanlock 
lockspace add failure', 'Invalid argument'))

Dec 27, 2015 8:58:18 PM  Failed to attach Storage Domains to Data Center 
Default. (User: admin@internal)

Dec 27, 2015 8:58:18 PM  Failed to attach Storage Domain hosted_storage to Data 
Center Default. (User: admin@internal)

So, how to fix this latest issue?


On Dec 27, 2015, at 3:11 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

I clicked on the “Data Centers” node on the left nav of webadmin, and I see 
that the “Default” DC (the only one I have) has a status of “Uninitialized”… 
Not sure how to initialize it (there’s no menu option for that on the node…)


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on hyperconverged setup

[Will Dennis]

Think I found the issue… Something caused the glusterd service on ovirt-host-03 
to segfault… Restarted it, and then went ahead and configured a GlusterFS 
storage domain for Data, which worked, and now the DC status is “Up"… Funny 
thing is, I do not see the “hosted_storage” domain anywhere now, just the Data 
SD that I created (name = vm_storage_1)

There is still nothing showing up in the “VMs” node…


On Dec 27, 2015, at 9:28 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

Did a bit of RTFM this evening, and read in the Admin Guide that:
"The new data center is added to the virtualization environment. It will remain 
Uninitialized until a cluster, host, and storage domain are configured for it; 
use Guide Me to configure these entities."

So, in webadmin I clicked on the Data Centers node, and then clicked on the 
“Guide Me” link; sure enough, in the “There are still unconfigured entities:” 
section, there was a link for “Attach Storage”. When I clicked on it, I saw 
that there was an option to attach the “hosted_storage” storage domain; when I 
tried to do so, it failed, and I got the following events at the bottom of the 
webadmin UI:

Dec 27, 2015 8:58:18 PM  VDSM ovirt-node-03 command failed: Cannot acquire host 
id: (u'e89b6e64-bd7d-4846-b970-9af32a3295ee', SanlockException(22, 'Sanlock 
lockspace add failure', 'Invalid argument'))

Dec 27, 2015 8:58:18 PM  Failed to attach Storage Domains to Data Center 
Default. (User: admin@internal)

Dec 27, 2015 8:58:18 PM  Failed to attach Storage Domain hosted_storage to Data 
Center Default. (User: admin@internal)

So, how to fix this latest issue?


On Dec 27, 2015, at 3:11 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

I clicked on the “Data Centers” node on the left nav of webadmin, and I see 
that the “Default” DC (the only one I have) has a status of “Uninitialized”… 
Not sure how to initialize it (there’s no menu option for that on the node…)



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on hyperconverged setup

[Will Dennis]

The uncompressed engine log for the period in question is 12M... Too large for 
a pastebin...

However, I grepped for ‘domain’ and that culls it down to 18 lines; the results 
are at: http://ur1.ca/od26u


From: Roy Golan [mailto:rgo...@redhat.com]
Sent: Monday, December 28, 2015 3:04 AM
To: Will Dennis
Cc: users
Subject: Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on 
hyperconverged setup



On Mon, Dec 28, 2015 at 5:31 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
Think I found the issue… Something caused the glusterd service on ovirt-host-03 
to segfault… Restarted it, and then went ahead and configured a GlusterFS 
storage domain for Data, which worked, and now the DC status is “Up"… Funny 
thing is, I do not see the “hosted_storage” domain anywhere now, just the Data 
SD that I created (name = vm_storage_1)

There is still nothing showing up in the “VMs” node…


Please share the /var/log/ovirt-engine/engine.log from the engine machine
If you are running 3.6.1 you should see the domain imported automatically.



On Dec 27, 2015, at 9:28 PM, Will Dennis 
mailto:wden...@nec-labs.com><mailto:wden...@nec-labs.com<mailto:wden...@nec-labs.com>>>
 wrote:

Did a bit of RTFM this evening, and read in the Admin Guide that:
"The new data center is added to the virtualization environment. It will remain 
Uninitialized until a cluster, host, and storage domain are configured for it; 
use Guide Me to configure these entities."

So, in webadmin I clicked on the Data Centers node, and then clicked on the 
“Guide Me” link; sure enough, in the “There are still unconfigured entities:” 
section, there was a link for “Attach Storage”. When I clicked on it, I saw 
that there was an option to attach the “hosted_storage” storage domain; when I 
tried to do so, it failed, and I got the following events at the bottom of the 
webadmin UI:

Dec 27, 2015 8:58:18 PM  VDSM ovirt-node-03 command failed: Cannot acquire host 
id: (u'e89b6e64-bd7d-4846-b970-9af32a3295ee', SanlockException(22, 'Sanlock 
lockspace add failure', 'Invalid argument'))

Dec 27, 2015 8:58:18 PM  Failed to attach Storage Domains to Data Center 
Default. (User: admin@internal)

Dec 27, 2015 8:58:18 PM  Failed to attach Storage Domain hosted_storage to Data 
Center Default. (User: admin@internal)

So, how to fix this latest issue?


On Dec 27, 2015, at 3:11 PM, Will Dennis 
mailto:wden...@nec-labs.com><mailto:wden...@nec-labs.com<mailto:wden...@nec-labs.com>>>
 wrote:

I clicked on the “Data Centers” node on the left nav of webadmin, and I see 
that the “Default” DC (the only one I have) has a status of “Uninitialized”… 
Not sure how to initialize it (there’s no menu option for that on the node…)



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on hyperconverged setup

[Will Dennis]

Would like to get this issue settled before I move on… Should I expect to see 
the hosted engine VM showing up in the “VMs” node, or is it a “hidden” VM 
because of it’s special status?

Also, I see that I can change the name of the 1st host if I select it in the 
“Hosts” node, and choose “Edit”… Would it cause any problem if I rename it 
there (how would that effect the engine VM, if at all?)


On Dec 28, 2015, at 8:59 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

The uncompressed engine log for the period in question is 12M... Too large for 
a pastebin...

However, I grepped for ‘domain’ and that culls it down to 18 lines; the results 
are at:http://ur1.ca/od26u


From: Roy Golan [mailto:rgo...@redhat.com]
Sent: Monday, December 28, 2015 3:04 AM
To: Will Dennis
Cc: users
Subject: Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on 
hyperconverged setup



On Mon, Dec 28, 2015 at 5:31 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
There is still nothing showing up in the “VMs” node…


Please share the /var/log/ovirt-engine/engine.log from the engine machine
If you are running 3.6.1 you should see the domain imported automatically.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on hyperconverged setup

[Will Dennis]

Also, just noticed that I’m getting the message “The Hosted Engine Storage 
Domain isn’t Active” in the Events pane, but I do not see that domain anywhere 
now in the Storage node in the Datacenter… What must I do to activate this?

On Dec 29, 2015, at 8:44 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

Would like to get this issue settled before I move on… Should I expect to see 
the hosted engine VM showing up in the “VMs” node, or is it a “hidden” VM 
because of it’s special status?

Also, I see that I can change the name of the 1st host if I select it in the 
“Hosts” node, and choose “Edit”… Would it cause any problem if I rename it 
there (how would that effect the engine VM, if at all?)


On Dec 28, 2015, at 8:59 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

The uncompressed engine log for the period in question is 12M... Too large for 
a pastebin...

However, I grepped for ‘domain’ and that culls it down to 18 lines; the results 
are at:http://ur1.ca/od26u


From: Roy Golan [mailto:rgo...@redhat.com]
Sent: Monday, December 28, 2015 3:04 AM
To: Will Dennis
Cc: users
Subject: Re: [ovirt-users] Hosted engine vs. hypervisor hosts naming on 
hyperconverged setup



On Mon, Dec 28, 2015 at 5:31 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
There is still nothing showing up in the “VMs” node…


Please share the /var/log/ovirt-engine/engine.log from the engine machine
If you are running 3.6.1 you should see the domain imported automatically.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Getting event message "The Hosted Engine Storage Domain isn't Active" every two hours

[Will Dennis]

Thought I’d start a new thread for this issue… Please also refer to my prior 
thread where 
I have the background to this. Again, I am running hosted engine on 
hyperconverged infra. Is this issue possibly related to Bug 
1269768 ?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Getting event message "The Hosted Engine Storage Domain isn't Active" every two hours

[Will Dennis]

This is a (relatively) brand new install of oVirt 3.6, not an upgrade or 
anything...

I will ignore these messages then; hopefully soon there will be a fix that 
allows me to see the HE VM, and stops these messages...

Thanks,
Will

From: Doron Fediuck [mailto:dfedi...@redhat.com]
Sent: Thursday, December 31, 2015 9:53 AM
To: Will Dennis
Cc: users; Golan, Roy
Subject: Re: [ovirt-users] Getting event message "The Hosted Engine Storage 
Domain isn't Active" every two hours

Hi will,
assuming this is a fresh install (please let us know if this is not the case), 
then this is just noise.
In 3.6 the hosted engine VM is being imported into it's own DB, but there's no 
active domain
when you start. This is the cause for the noise, and we're working on improving 
it.

Doron

On Thu, Dec 31, 2015 at 5:03 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
Thought I’d start a new thread for this issue… Please also refer to my prior 
thread<http://lists.ovirt.org/pipermail/users/2015-December/036907.html> where 
I have the background to this. Again, I am running hosted engine on 
hyperconverged infra. Is this issue possibly related to Bug 
1269768<https://bugzilla.redhat.com/show_bug.cgi?id=1269768> ?

___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Configuring another interface for trunked (tagged) VM traffic

[Will Dennis]

Hi all,

Taking the next step on configuring my newly-established oVirt cluster, and 
that would be to set up a trunk (VLAN tagged) connection to each cluster host 
(there are 3) for VM traffic. What I’m looking at is akin to setting up 
vSwitches on VMware, except I have never done this on a VMware cluster, just on 
individual hosts…

Anyhow, I have the following NICs available on my three hosts (conveniently, 
they are the exact same hardware platform):

ovirt-node-01 | success | rc=0 >>
3: enp4s0f0:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
4: enp4s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
5: enp12s0f0:  mtu 1500 qdisc pfifo_fast 
master ovirtmgmt state UP mode DEFAULT qlen 1000
6: enp12s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000

ovirt-node-02 | success | rc=0 >>
3: enp4s0f0:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
4: enp4s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
5: enp12s0f0:  mtu 1500 qdisc pfifo_fast 
master ovirtmgmt state UP mode DEFAULT qlen 1000
6: enp12s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000

ovirt-node-03 | success | rc=0 >>
3: enp4s0f0:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
4: enp4s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
5: enp12s0f0:  mtu 1500 qdisc pfifo_fast 
master ovirtmgmt state UP mode DEFAULT qlen 1000
6: enp12s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000

As you may see, I am using the ‘enp12s0f0’ interface on each host for the 
‘ovirtmgmt’ bridge. This network carries the admin traffic as well as Gluster 
distributed filesystem traffic, but I now want to establish a separate link to 
each host for VM traffic. The ‘ovirtmgmt’ bridge is NOT trunked/tagged, only a 
single VLAN is used. For the VM traffic, I’d like to use the ‘enp4s0f0’ 
interface on each host, and tie them into a logical network named “vm-traffic” 
(or the like) and make that a trunked/tagged interface.

Are there any existing succinct instructions on how to do this? I have been 
reading thru the oVirt Admin Manual’s “Logical Networks” section 
(http://www.ovirt.org/OVirt_Administration_Guide#Logical_Network_Tasks) but it 
hasn’t “clicked” in my mind yet...

Thanks,
Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Configuring another interface for trunked (tagged) VM traffic

[Will Dennis]

I found this following (older) article, that gave me a clue…
http://captainkvm.com/2013/04/maximizing-your-10gb-ethernet-in-kvm/

So I configured the following up in /etc/sysconfig/network-scripts for each of 
my hosts —

[root@ovirt-node-01 network-scripts]# cat ifcfg-enp4s0f0
HWADDR=00:15:17:7B:E9:EA
TYPE=Ethernet
BOOTPROTO=none
NAME=enp4s0f0
UUID=8b006c8c-b5d3-4dae-a1e7-5ca463119be3
ONBOOT=yes
SLAVE=yes
MASTER=bond0

(^^^ same sort of file made for enp4s0f1)

[root@ovirt-node-01 network-scripts]# cat ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="mode=4 miimon=100"

[root@ovirt-node-01 network-scripts]# cat ifcfg-bond0.180
DEVICE=bond0.180
VLAN=yes
BOOTPROTO=static
ONBOOT=yes
BRIDGE=br180

(^^^ same sort of file made for other VLANs)

[root@ovirt-node-03 network-scripts]# cat ifcfg-br180
DEVICE=br180
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
DELAY=0

(^^^ same sort of file made for other bridges)

So that all makes the following sort of device chain:

http://s1096.photobucket.com/user/willdennis/media/ovirt-bond-layout.png.html";
 target="_blank">http://i1096.photobucket.com/albums/g330/willdennis/ovirt-bond-layout.png"; 
border="0" alt="ovirt-bond-layout.png"/>

But then I read this next article:
http://captainkvm.com/2013/04/maximizing-your-10gb-ethernet-in-rhev/

This leads me to believe (if it’s still the same process on current oVirt/RHEV) 
that I could stop with the bond0 setup, and then by tying the networks I 
created for the VLANs of interest (which do have the proper VLAN tags set on 
them) that oVirt would automatically create the needed bond0 VLAN 
sub-interfaces, and the related  per-VLAN bridges.

So, is there a way to tie the oVirt networks to use the bridges I’ve already 
created (they don’t show up in the oVirt webadmin “Setup host networks” dialog) 
or should I just match the oVirt networks with the bond0 interface, and let 
whatever structure oVirt creates happen? (and if so, I guess I’d need to remove 
the bond0 VLAN sub-interfaces, and the related per-VLAN bridges I created?)


On Dec 31, 2015, at 1:56 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

Hi all,

Taking the next step on configuring my newly-established oVirt cluster, and 
that would be to set up a trunk (VLAN tagged) connection to each cluster host 
(there are 3) for VM traffic. What I’m looking at is akin to setting up 
vSwitches on VMware, except I have never done this on a VMware cluster, just on 
individual hosts…

Anyhow, I have the following NICs available on my three hosts (conveniently, 
they are the exact same hardware platform):

ovirt-node-01 | success | rc=0 >>
3: enp4s0f0:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
4: enp4s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
5: enp12s0f0:  mtu 1500 qdisc pfifo_fast 
master ovirtmgmt state UP mode DEFAULT qlen 1000
6: enp12s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000

ovirt-node-02 | success | rc=0 >>
3: enp4s0f0:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
4: enp4s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
5: enp12s0f0:  mtu 1500 qdisc pfifo_fast 
master ovirtmgmt state UP mode DEFAULT qlen 1000
6: enp12s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000

ovirt-node-03 | success | rc=0 >>
3: enp4s0f0:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
4: enp4s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000
5: enp12s0f0:  mtu 1500 qdisc pfifo_fast 
master ovirtmgmt state UP mode DEFAULT qlen 1000
6: enp12s0f1:  mtu 1500 qdisc noop state DOWN mode DEFAULT 
qlen 1000

As you may see, I am using the ‘enp12s0f0’ interface on each host for the 
‘ovirtmgmt’ bridge. This network carries the admin traffic as well as Gluster 
distributed filesystem traffic, but I now want to establish a separate link to 
each host for VM traffic. The ‘ovirtmgmt’ bridge is NOT trunked/tagged, only a 
single VLAN is used. For the VM traffic, I’d like to use the ‘enp4s0f0’ 
interface on each host, and tie them into a logical network named “vm-traffic” 
(or the like) and make that a trunked/tagged interface.

Are there any existing succinct instructions on how to do this? I have been 
reading thru the oVirt Admin Manual’s “Logical Networks” section 
(http://www.ovirt.org/OVirt_Administration_Guide#Logical_Network_Tasks) but it 
hasn’t “clicked” in my mind yet...

Thanks,
Will

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

I have had one of my hosts go into the state “Non Operational” after I rebooted 
it… I also noticed that in the oVirt webadmin UI, the NIC that’s used in the 
‘ovirtmgmt’ network is showing “down”, but in Linux the NIC is operational and 
up, as is the ‘ovirtmgmt’ bridge…

[root@ovirt-node-02 ~]# ip link sh up
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: bond0:  mtu 1500 qdisc noqueue 
state DOWN mode DEFAULT
link/ether 00:15:17:7b:e9:b0 brd ff:ff:ff:ff:ff:ff
3: enp4s0f0:  mtu 1500 qdisc 
pfifo_fast master bond0 state DOWN mode DEFAULT qlen 1000
link/ether 00:15:17:7b:e9:b0 brd ff:ff:ff:ff:ff:ff
4: enp4s0f1:  mtu 1500 qdisc 
pfifo_fast master bond0 state DOWN mode DEFAULT qlen 1000
link/ether 00:15:17:7b:e9:b0 brd ff:ff:ff:ff:ff:ff
5: enp12s0f0:  mtu 1500 qdisc pfifo_fast 
master ovirtmgmt state UP mode DEFAULT qlen 1000
link/ether 00:21:85:35:08:4c brd ff:ff:ff:ff:ff:ff
7: ovirtmgmt:  mtu 1500 qdisc noqueue state UP 
mode DEFAULT
link/ether 00:21:85:35:08:4c brd ff:ff:ff:ff:ff:ff

What should I take a look at first?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

The ‘ovirtmgmt’ network has been & is still placed on a working NIC 
(enp12s0f0)… It’s just that now, oVirt somehow doesn’t *think* it’s working…

http://s1096.photobucket.com/user/willdennis/media/setup-networks.png.html

However, as I showed you in the ‘ip link show up’ output, it is indeed up and 
working.




On Jan 2, 2016, at 8:00 PM, Roy Golan 
mailto:rgo...@redhat.com>> wrote:



On Sun, Jan 3, 2016 at 2:46 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
I have had one of my hosts go into the state “Non Operational” after I rebooted 
it… I also noticed that in the oVirt webadmin UI, the NIC that’s used in the 
‘ovirtmgmt’ network is showing “down”, but in Linux the NIC is operational and 
up, as is the ‘ovirtmgmt’ bridge…


Hosts tab -> Network Interfaces subtab -> click "Setup networks" and make sure 
"ovirtmgmt" is placed on a working nic.

make sure
[root@ovirt-node-02 ~]# ip link sh up
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: bond0:  mtu 1500 qdisc noqueue 
state DOWN mode DEFAULT
link/ether 00:15:17:7b:e9:b0 brd ff:ff:ff:ff:ff:ff
3: enp4s0f0:  mtu 1500 qdisc 
pfifo_fast master bond0 state DOWN mode DEFAULT qlen 1000
link/ether 00:15:17:7b:e9:b0 brd ff:ff:ff:ff:ff:ff
4: enp4s0f1:  mtu 1500 qdisc 
pfifo_fast master bond0 state DOWN mode DEFAULT qlen 1000
link/ether 00:15:17:7b:e9:b0 brd ff:ff:ff:ff:ff:ff
5: enp12s0f0:  mtu 1500 qdisc pfifo_fast 
master ovirtmgmt state UP mode DEFAULT qlen 1000
link/ether 00:21:85:35:08:4c brd ff:ff:ff:ff:ff:ff
7: ovirtmgmt:  mtu 1500 qdisc noqueue state UP 
mode DEFAULT
link/ether 00:21:85:35:08:4c brd ff:ff:ff:ff:ff:ff

What should I take a look at first?

___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

vdsClient output: http://fpaste.org/30/82858714/

The engine.log is very large (4496 lines) so cannot fpaste… Is there a file 
upload service that can be used to share these sorts of things with you?


On Jan 3, 2016, at 5:24 AM, Eliraz Levi 
mailto:el...@redhat.com>> wrote:

Hi how are you?

Can you please send the following information:
1. Run the following command in the host and send it's output:
  vdsClient -s 0 getVdsStats
2. engine.log

Also, please try to refresh caps.

Thanks.
BR'
Eliraz :)


From: "Karli Sjöberg" mailto:karli.sjob...@slu.se>>
To: "Will Dennis" mailto:wden...@nec-labs.com>>
Cc: "users" mailto:users@ovirt.org>>
Sent: Sunday, 3 January, 2016 9:14:04 AM
Subject: Re: [ovirt-users] host status "Non Operational" - how to diagnose & 
fix?




Den 3 jan. 2016 2:43 fm skrev Will Dennis 
mailto:wden...@nec-labs.com>>:

 The ‘ovirtmgmt’ network has been & is still placed on a working NIC 
(enp12s0f0)… It’s just that now, oVirt somehow doesn’t *think* it’s working…

Here's something I wrote a long time ago now, for those times when 
auto-gui-config fluff just won't do:


http://www.ovirt.org/Bonding_VLAN_Bridge

/K

 http://s1096.photobucket.com/user/willdennis/media/setup-networks.png.html

 However, as I showed you in the ‘ip link show up’ output, it is indeed up and 
working.




 On Jan 2, 2016, at 8:00 PM, Roy Golan 
mailto:rgo...@redhat.com><mailto:rgo...@redhat.com>> wrote:



 On Sun, Jan 3, 2016 at 2:46 AM, Will Dennis 
mailto:wden...@nec-labs.com><mailto:wden...@nec-labs.com>>
 wrote:
 I have had one of my hosts go into the state “Non Operational” after I 
rebooted it… I also noticed that in the oVirt webadmin UI, the NIC that’s used 
in the ‘ovirtmgmt’ network is showing “down”, but in Linux the NIC is 
operational and up, as is the ‘ovirtmgmt’ bridge…


 Hosts tab -> Network Interfaces subtab -> click "Setup networks" and make sure 
"ovirtmgmt" is placed on a working nic.

 make sure
 [root@ovirt-node-02 ~]# ip link sh up
 1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode 
DEFAULT
 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 2: bond0:  mtu 1500 qdisc noqueue 
state DOWN mode DEFAULT
 link/ether 00:15:17:7b:e9:b0 brd ff:ff:ff:ff:ff:ff
 3: enp4s0f0:  mtu 1500 qdisc 
pfifo_fast master bond0 state DOWN mode DEFAULT qlen 1000
 link/ether 00:15:17:7b:e9:b0 brd ff:ff:ff:ff:ff:ff
 4: enp4s0f1:  mtu 1500 qdisc 
pfifo_fast master bond0 state DOWN mode DEFAULT qlen 1000
 link/ether 00:15:17:7b:e9:b0 brd ff:ff:ff:ff:ff:ff
 5: enp12s0f0:  mtu 1500 qdisc pfifo_fast 
master ovirtmgmt state UP mode DEFAULT qlen 1000
 link/ether 00:21:85:35:08:4c brd ff:ff:ff:ff:ff:ff
 7: ovirtmgmt:  mtu 1500 qdisc noqueue state 
UP mode DEFAULT
 link/ether 00:21:85:35:08:4c brd ff:ff:ff:ff:ff:ff

 What should I take a look at first?

 ___
 Users mailing list
 Users@ovirt.org<mailto:Users@ovirt.org><mailto:Users@ovirt.org>
 http://lists.ovirt.org/mailman/listinfo/users


 ___
 Users mailing list
 Users@ovirt.org<mailto:Users@ovirt.org>
 http://lists.ovirt.org/mailman/listinfo/users

___

Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

Forgot to cc: list, sorry…

On Jan 3, 2016, at 1:24 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

Digital Ocean droplet and Python SimpleHTTPServer FTW ;)
http://c7-01.thiscant.fail<http://c7-01.thiscant.fail/>



On Jan 3, 2016, at 9:33 AM, Eliraz Levi 
mailto:el...@redhat.com>> wrote:



vdsClient output: http://fpaste.org/30/82858714/

The engine.log is very large (4496 lines) so cannot fpaste… Is there a file 
upload service that can be used to share these sorts of things with you?

Hi Will how are you?
Perhaps you can upload the log to some sort of a cloud? say google and share 
the URL?
I think it will be the fastest way around.
Thanks :)
please send the URL in the mailing list so everybody will be able to follow.
Cheers!
Eliraz :)




___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

Link is working again...

http://c7-01.thiscant.fail



Sent with Good (www.good.com)


-Original Message-
From: Eliraz Levi [el...@redhat.com<mailto:el...@redhat.com>]
Sent: Monday, January 04, 2016 07:17 AM Eastern Standard Time
To: Will Dennis
Subject: Re: [ovirt-users] host status "Non Operational" - how to diagnose & 
fix?


Hi Will :)
The link is broken.
Can you please send a valid one to the list?
thanks :)
Eliraz.

- Original Message -
From: "Will Dennis" 
To: "Eliraz Levi" 
Sent: Sunday, 3 January, 2016 8:23:59 PM
Subject: Re: [ovirt-users] host status "Non Operational" - how to diagnose & 
fix?

Digital Ocean droplet and Python SimpleHTTPServer FTW ;)
http://c7-01.thiscant.fail



On Jan 3, 2016, at 9:33 AM, Eliraz Levi 
mailto:el...@redhat.com>> wrote:



vdsClient output: http://fpaste.org/30/82858714/

The engine.log is very large (4496 lines) so cannot fpaste… Is there a file 
upload service that can be used to share these sorts of things with you?

Hi Will how are you?
Perhaps you can upload the log to some sort of a cloud? say google and share 
the URL?
I think it will be the fastest way around.
Thanks :)
please send the URL in the mailing list so everybody will be able to follow.
Cheers!
Eliraz :)



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

Sorry, link should be working again now...



Sent with Good (www.good.com)


-Original Message-
From: Eliraz Levi [el...@redhat.com<mailto:el...@redhat.com>]
Sent: Monday, January 04, 2016 07:17 AM Eastern Standard Time
To: Will Dennis
Subject: Re: [ovirt-users] host status "Non Operational" - how to diagnose & 
fix?


Hi Will :)
The link is broken.
Can you please send a valid one to the list?
thanks :)
Eliraz.

- Original Message -
From: "Will Dennis" 
To: "Eliraz Levi" 
Sent: Sunday, 3 January, 2016 8:23:59 PM
Subject: Re: [ovirt-users] host status "Non Operational" - how to diagnose & 
fix?

Digital Ocean droplet and Python SimpleHTTPServer FTW ;)
http://c7-01.thiscant.fail



On Jan 3, 2016, at 9:33 AM, Eliraz Levi 
mailto:el...@redhat.com>> wrote:



vdsClient output: http://fpaste.org/30/82858714/

The engine.log is very large (4496 lines) so cannot fpaste… Is there a file 
upload service that can be used to share these sorts of things with you?

Hi Will how are you?
Perhaps you can upload the log to some sort of a cloud? say google and share 
the URL?
I think it will be the fastest way around.
Thanks :)
please send the URL in the mailing list so everybody will be able to follow.
Cheers!
Eliraz :)



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

I put all of the engine logs up there now… Try engine.log-20160103.gz

> On Jan 4, 2016, at 7:48 AM, Eliraz Levi  wrote:
> 
> ok thanks :)
> It looks like you didn't press refresh capabilities.
> I can't learn a lot from this log.
> can you refresh the host's capabilities and then send the log?
> thanks :)
> Eliraz. 
> 
> - Original Message -
> From: "Will Dennis" 
> To: "Eliraz Levi" 
> Sent: Monday, 4 January, 2016 2:22:03 PM
> Subject: RE: [ovirt-users] host status "Non Operational" - how to diagnose & 
> fix?
> 
> If you try it again, should work now... Damn hackers...
> 
> 
> 
> Sent with Good (www.good.com)
> 
> 
> -Original Message-
> From: Eliraz Levi [el...@redhat.com<mailto:el...@redhat.com>]
> Sent: Monday, January 04, 2016 07:17 AM Eastern Standard Time
> To: Will Dennis
> Subject: Re: [ovirt-users] host status "Non Operational" - how to diagnose & 
> fix?
> 
> 
> Hi Will :)
> The link is broken.
> Can you please send a valid one to the list?
> thanks :)
> Eliraz.
> 
> - Original Message -
> From: "Will Dennis" 
> To: "Eliraz Levi" 
> Sent: Sunday, 3 January, 2016 8:23:59 PM
> Subject: Re: [ovirt-users] host status "Non Operational" - how to diagnose & 
> fix?
> 
> Digital Ocean droplet and Python SimpleHTTPServer FTW ;)
> http://c7-01.thiscant.fail
> 
> 
> 
> On Jan 3, 2016, at 9:33 AM, Eliraz Levi 
> mailto:el...@redhat.com>> wrote:
> 
> 
> 
> vdsClient output: http://fpaste.org/30/82858714/
> 
> The engine.log is very large (4496 lines) so cannot fpaste… Is there a file 
> upload service that can be used to share these sorts of things with you?
> 
> Hi Will how are you?
> Perhaps you can upload the log to some sort of a cloud? say google and share 
> the URL?
> I think it will be the fastest way around.
> Thanks :)
> please send the URL in the mailing list so everybody will be able to follow.
> Cheers!
> Eliraz :)
> 
> 

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

I must admit I’m getting a bit weary of fighting oVirt problems at this point… 
Before I move on to deploying any VMs onto my new infra, I’d like to get the 
base infra working…

I’m still experiencing a “Non Operational” problem on my “ovirt-node-02” host:
http://s1096.photobucket.com/user/willdennis/media/ovirt-node-02_problem.png.html

I have pored thru the logs (all the engine logs, plus the syslogs from the 
engine VM + and my three hypervisor/storage hosts) and I can’t pin down why the 
one node is having a problem… Of course with how voluminous all these logs are, 
it’s kind of like looking for a needle in a haystack, and I’m not even sure 
what the needle looks like, or if it’s even a needle :-/

I have also rebooted this host in past days, this also did not fix the problem.

Note that on the screenshot I posted above, that the webadmin hosts screen says 
that -node-01 has one VM running, and the others 0… You’d think that would be 
the HE VM running on there, but it’s actually on -node-02:

$ ansible istgroup-ovirt -f 1 -i prod -u root -m shell -a "hosted-engine 
--vm-status | grep -e '^Hostname' -e '^Engine'"
ovirt-node-01 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}

ovirt-node-02 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}

ovirt-node-03 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown”}

So it looks like the webadmin UI is wrong as well…

It would be awesome if the UI would give a reason for the “Non Operational” 
status somehow… Or if there was a troubleshooter that could be used to analyze 
the problem… As it is, being so new to all of this, I am completely at the 
list’s mercy to figure this out.

This software has such promise, so I’ll keep working thru these issues, but it 
sure hasn’t been a smooth ride so far…


On Jan 4, 2016, at 7:54 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

I put all of the engine logs up there now… Try 
engine.log-20160103.gzhttp://i1096.photobucket.com/albums/g330/willdennis/ovirt-node-02_problem.png
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

Feel like I’m in a bit of an echo chamber here… Is anyone out there? ;) Or have 
I worn out the oVirt crew?

Anyhow, not sure if this is a cause, or an effect, but I noticed tonight that 
the data storage domain (which I’m using Gluster for in a hyperconverged way) 
is not mounted on the problem hypervisor host…

$ ansible istgroup-ovirt -f 1 -i prod -u root -m shell -a "df -h | grep ':’"
ovirt-node-01 | success | rc=0 >>
localhost:/engine 1.9T  3.0G  1.9T   1% 
/rhev/data-center/mnt/glusterSD/localhost:_engine
ovirt-node-01.nec-labs.com<http://ovirt-node-01.nec-labs.com>:/vmdata   
 3.7T   70M  3.7T   1% 
/rhev/data-center/mnt/glusterSD/ovirt-node-01.nec-labs.com<http://ovirt-node-01.nec-labs.com>:_vmdata

ovirt-node-02 | success | rc=0 >>
localhost:/engine 1.9T  3.0G  1.9T   1% 
/rhev/data-center/mnt/glusterSD/localhost:_engine

ovirt-node-03 | success | rc=0 >>
localhost:/engine 1.9T  3.0G  1.9T   1% 
/rhev/data-center/mnt/glusterSD/localhost:_engine
ovirt-node-01.nec-labs.com<http://ovirt-node-01.nec-labs.com>:/vmdata   
  3.7T   70M  3.7T   1% 
/rhev/data-center/mnt/glusterSD/ovirt-node-01.nec-labs.com<http://ovirt-node-01.nec-labs.com>:_vmdata

What causes this mount to occur, and is there a way to trigger the mount 
manually?



On Jan 4, 2016, at 10:47 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

I must admit I’m getting a bit weary of fighting oVirt problems at this point… 
Before I move on to deploying any VMs onto my new infra, I’d like to get the 
base infra working…

I’m still experiencing a “Non Operational” problem on my “ovirt-node-02” host:
http://s1096.photobucket.com/user/willdennis/media/ovirt-node-02_problem.png.html

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

Define “reinstall the host” - do you just mean 'yum remove ovirt* vdsm*’ then 
‘yum install ovirt* vdsm*’, or completely reinstall the OS, reset-up Gluster, 
etc.?

On Jan 6, 2016, at 4:15 AM, Eliraz Levi 
mailto:el...@redhat.com>> wrote:

Hi Will how are you?
The log is first pointing about certifications issues:
2016-01-04 00:02:11,259 ERROR 
[org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer] 
(DefaultQuartzScheduler_Worker-81) [] Failed to get peer certification for host 
'ovirt-node-02': SSL session is invalid
2016-01-04 00:02:11,259 ERROR 
[org.ovirt.engine.core.bll.CertificationValidityChecker] 
(DefaultQuartzScheduler_Worker-81) [] Failed to retrieve peer certifications 
for host 'ovirt-node-02'

So first thing we should do is to try and solve this problem.
Please try to re install the host.
Thanks.
Eliraz :)

- Original Message -
From: "Will Dennis" mailto:wden...@nec-labs.com>>
To: "Eliraz Levi" mailto:el...@redhat.com>>, "users" 
mailto:users@ovirt.org>>
Sent: Tuesday, 5 January, 2016 5:46:23 AM
Subject: Re: [ovirt-users] host status "Non Operational" - how to diagnose & 
fix?

I must admit I’m getting a bit weary of fighting oVirt problems at this point… 
Before I move on to deploying any VMs onto my new infra, I’d like to get the 
base infra working…

I’m still experiencing a “Non Operational” problem on my “ovirt-node-02” host:
http://s1096.photobucket.com/user/willdennis/media/ovirt-node-02_problem.png.html

I have pored thru the logs (all the engine logs, plus the syslogs from the 
engine VM + and my three hypervisor/storage hosts) and I can’t pin down why the 
one node is having a problem… Of course with how voluminous all these logs are, 
it’s kind of like looking for a needle in a haystack, and I’m not even sure 
what the needle looks like, or if it’s even a needle :-/

I have also rebooted this host in past days, this also did not fix the problem.

Note that on the screenshot I posted above, that the webadmin hosts screen says 
that -node-01 has one VM running, and the others 0… You’d think that would be 
the HE VM running on there, but it’s actually on -node-02:

$ ansible istgroup-ovirt -f 1 -i prod -u root -m shell -a "hosted-engine 
--vm-status | grep -e '^Hostname' -e '^Engine'"
ovirt-node-01 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}

ovirt-node-02 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}

ovirt-node-03 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown”}

So it looks like the webadmin UI is wrong as well…

It would be awesome if the UI would give a reason for the “Non Operational” 
status somehow… Or if there was a troubleshooter that could be used to analyze 
the problem… As it is, being so new to all of this, I am completely at the 
list’s mercy to figure this out.

This software has such promise, so I’ll keep working thru these issues, but it 
sure hasn’t been a smooth ride so far…


On Jan 4, 2016, at 7:54 AM, Will Dennis 
mailto:wden...@nec-labs.com><mailto:wden...@nec-labs.com>>
 wrote:

I put all of the engine logs up there now… Try 
engine.log-20160103.gzhttp://i1096.photobucket.com/albums/g330/willdennis/ovirt-node-02_problem.png

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

On Jan 6, 2016, at 7:59 AM, Moti Asayag 
mailto:masa...@redhat.com>> wrote:

In order to see the configuration of 'ovirtmgmt' network please paste the 
output of the following command to be executed on the host:
vdsClient -s 0 getVdsCaps

http://fpaste.org/307742/20853451/


In addition, in order to see the reported status of the networks run and paste 
on the host:
vdsClient -s 0 getVdsStats

http://fpaste.org/307744/45208555/

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

On Jan 6, 2016, at 1:39 AM, Sahina Bose 
mailto:sab...@redhat.com>> wrote:

The reason why the host is Non-operational is usually in the General sub-tab 
for the host.

Ah, did not know that… It does say at the bottom of that pane:

“Host failed to attach one of the Storage Domains attached to it.”

As previously reported to the list last evening, this is true - it has not 
mounted the data SD (which is a Gluster SD.)

Any way to troubleshoot why?
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

I did what Joop suggested (put -node-02 into maint, clear vdsm.log on -node-02, 
clear engine.log on HE, then activate -node-02) and — what do you know, the 
node came up into an operational state! It was able to successfully mount the 
data SD this time:

$ ansible istgroup-ovirt -f 1 -i prod -u root -m shell -a "df -h | grep 
':'"ovirt-node-01 | success | rc=0 >>
localhost:/engine   1.9T  3.0G  1.9T   1% 
/rhev/data-center/mnt/glusterSD/localhost:_engine
ovirt-node-01.nec-labs.com:/vmdata  3.7T   70M  3.7T   1% 
/rhev/data-center/mnt/glusterSD/ovirt-node-01.nec-labs.com:_vmdata

ovirt-node-02 | success | rc=0 >>
localhost:/engine   1.9T  3.0G  1.9T   1% 
/rhev/data-center/mnt/glusterSD/localhost:_engine
ovirt-node-01.nec-labs.com:/vmdata  3.7T   70M  3.7T   1% 
/rhev/data-center/mnt/glusterSD/ovirt-node-01.nec-labs.com:_vmdata

ovirt-node-03 | success | rc=0 >>
localhost:/engine   1.9T  3.0G  1.9T   1% 
/rhev/data-center/mnt/glusterSD/localhost:_engine
ovirt-node-01.nec-labs.com:/vmdata  3.7T   70M  3.7T   1% 
/rhev/data-center/mnt/glusterSD/ovirt-node-01.nec-labs.com:_vmdata

Wonder what the magic was? ;)  I’ll take the result anyways :)

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Webadmin UI not reporting # of VMs correctly

[Will Dennis]

To follow up on this, after the migrations as a result of the troubleshooting, 
the webadmin UI of the hosts in my datacenter now has each host with “1” VM 
running…
https://drive.google.com/file/d/0B88nnCy4LpFMYklDVDhFUV96Y00/view?usp=sharing

However, The only VM that is running currently is the hosted engine, which is 
currently running on host “ovirt-node-03” —

$ ansible istgroup-ovirt -f 1 -i prod -u root -m shell -a "hosted-engine 
--vm-status | grep -e '^Hostname' -e '^Engine'"
ovirt-node-01 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-03
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}

ovirt-node-02 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-03
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}

ovirt-node-03 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-03
Engine status  : {"health": "good", "vm": "up", "detail": 
"up”}

Is this a UI bug of some sort?


On Jan 4, 2016, at 10:47 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

Note that on the screenshot I posted above, that the webadmin hosts screen says 
that -node-01 has one VM running, and the others 0… You’d think that would be 
the HE VM running on there, but it’s actually on -node-02:

$ ansible istgroup-ovirt -f 1 -i prod -u root -m shell -a "hosted-engine 
--vm-status | grep -e '^Hostname' -e '^Engine'"
ovirt-node-01 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}

ovirt-node-02 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown"}

ovirt-node-03 | success | rc=0 >>
Hostname   : ovirt-node-01
Engine status  : {"reason": "bad vm status", "health": 
"bad", "vm": "down", "detail": "down"}
Hostname   : ovirt-node-02
Engine status  : {"health": "good", "vm": "up", "detail": 
"up"}
Hostname   : ovirt-node-03
Engine status  : {"reason": "vm not running on this host", 
"health": "bad", "vm": "down", "detail": "unknown”}

So it looks like the webadmin UI is wrong as well…

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] host status "Non Operational" - how to diagnose & fix?

[Will Dennis]

I actually had the opposite problem - I (not knowing) elected to say “Yes” 
(default answer) to “iptables was detected on your computer, do you wish setup 
to configure it?” which then put in the oVirt iptables rules, which assume the 
standard Gluster TCP ports… Since I am running hyperconverged and had followed 
the instructions found at:
http://www.ovirt.org/Features/Self_Hosted_Engine_Hyper_Converged_Gluster_Support
which ends up changing the Gluster ports, then I experienced a fault with 
Gluster where it lost quorum and went read-only since the firewall on the hosts 
were blocking Gluster communications...


On Jan 6, 2016, at 11:10 AM, Sahina Bose  wrote:

Also, worth checking that glusterd ports are open on the gluster hosts (we had 
an issue where HE install overrides glusterd ports and gluster volume was 
inaccessible)

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Getting event message "The Hosted Engine Storage Domain isn't Active" every two hours

[Will Dennis]

Update on this problem (happily now resolved! :)

Perusing the oVirt Bugzilla tix, I ran across this one - "failed Activating 
hosted engine domain during auto-import on NFS”:
https://bugzilla.redhat.com/show_bug.cgi?id=1290518

In comments #9 and #12, I got the resolution of my problem:

1) In the System node, Storage sub-tab, alt-click on the Unassigned 
hosted_storage SD, and choose Destroy, confirm and OK - SD is removed
2) Within a minute, the hosted_storage SD reappears; it is in status 
“Maintenance”
3) Go to the Data Centers node, Storage sub-tab, click on the hosted_storage SD 
and click the Activate link
4) Within a few seconds, the hosted_storage SD goes into state Active
5) Within a minute or so, the HostedEngine VM appears in the VMs node, and is 
in status Up

This is the sequence of messages in the Events section for all of this:

Jan 6, 2016 9:25:51 PM   Hosted Engine VM was imported successfully

Jan 6, 2016 9:25:51 PM   Starting to import Vm HostedEngine to Data Center 
Default, Cluster Default

Jan 6, 2016 9:25:51 PM   Storage Domain hosted_storage (Data Center Default) 
was activated by admin@internal

Jan 6, 2016 9:24:46 PM   Storage Domain hosted_storage was updated by 
admin@internal

Jan 6, 2016 9:22:35 PM   Hosted Engine storage domain imported successfully

Jan 6, 2016 9:22:35 PM   Storage Domain hosted_storage was attached to Data 
Center Default by SYSTEM

Jan 6, 2016 9:22:35 PM   Failed to activate Storage Domain hosted_storage (Data 
Center Default) by SYSTEM

Jan 6, 2016 9:22:35 PM   This Data center compatibility version does not 
support importing a data domain with its entities (VMs and Templates). The 
imported domain will be imported without them.

Jan 6, 2016 9:22:24 PM   The Hosted Engine Storage Domain doesn't exist. It 
should be imported into the setup.

Jan 6, 2016 9:22:24 PM   Storage Domain hosted_storage was added by SYSTEM

Jan 6, 2016 9:22:08 PM   Storage Domain hosted_storage was forcibly removed by 
admin@internal




On Dec 31, 2015, at 9:53 AM, Doron Fediuck 
mailto:dfedi...@redhat.com>> wrote:

Hi will,
assuming this is a fresh install (please let us know if this is not the case), 
then this is just noise.
In 3.6 the hosted engine VM is being imported into it's own DB, but there's no 
active domain
when you start. This is the cause for the noise, and we're working on improving 
it.

Doron

On Thu, Dec 31, 2015 at 5:03 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
Thought I’d start a new thread for this issue… Please also refer to my prior 
thread<http://lists.ovirt.org/pipermail/users/2015-December/036907.html> where 
I have the background to this. Again, I am running hosted engine on 
hyperconverged infra. Is this issue possibly related to Bug 
1269768<https://bugzilla.redhat.com/show_bug.cgi?id=1269768> ?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] ISO domain question

[Will Dennis]

I’d like to utilize a pre-existing NFS-available ISO repository we have, 
instead of creating an oVirt-specific one. This repo is of course used for 
other purposes, so it cannot be devoted to oVirt alone. We have successfully 
used this NFS resource in VMware, but cannot see how to do the same in oVirt… 
Is it possible? Or must oVirt maintain its own?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] ISO domain question

[Will Dennis]

So would have to create a new SD (I'm using Gluster-based hyperconverged) and 
then create the symlink in the new mount in the subdirectory you named? Maybe 
I'll just rsync the ISO files into there...

Thanks anyhow... Seems like there should be an easier way to consume an 
external existing repository.



Sent with Good (www.good.com)


-Original Message-
From: Maor Lipchuk [mlipc...@redhat.com<mailto:mlipc...@redhat.com>]
Sent: Saturday, January 09, 2016 10:18 AM Eastern Standard Time
To: Will Dennis
Cc: users
Subject: Re: [ovirt-users] ISO domain question


Hi Will,

The ISO storage domain oVirt maintains is configured specifically for oVirt, 
for example don_md/metadata is configured with class ISO.
All the ISO files are stored in a special folder under 
"images/----"
What you can do is to add a new ISO Storage Domain to oVirt, and use a symbolic 
link under the '1' folder to all the ISO files in your pre-existing 
NFS-available ISO.

Please let me know if that answer your question

Regards,
Maor





----- Original Message -
> From: "Will Dennis" 
> To: "users" 
> Sent: Friday, January 8, 2016 2:50:38 PM
> Subject: [ovirt-users] ISO domain question
>
> I’d like to utilize a pre-existing NFS-available ISO repository we have,
> instead of creating an oVirt-specific one. This repo is of course used for
> other purposes, so it cannot be devoted to oVirt alone. We have successfully
> used this NFS resource in VMware, but cannot see how to do the same in
> oVirt… Is it possible? Or must oVirt maintain its own?
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] ISO domain question

[Will Dennis]

Looks like a pretty old request, too… if I may ask, is this a complicated 
technical change, or is it just lower priority than other open issues? Seems 
like if it was an easy change to make, it would be a big win for users…

On Jan 10, 2016, at 3:15 AM, Yedidyah Bar David 
mailto:d...@redhat.com>> wrote:

On Sun, Jan 10, 2016 at 5:09 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:

Seems like there should be an easier way to consume an
external existing repository.

Indeed, and we have an RFE for that:
https://bugzilla.redhat.com/show_bug.cgi?id=1034112

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Not seeing Gluster options on Cluster properties

[Will Dennis]

Hi all,

I am running a hyperconverged setup of oVirt 3.6, where I pre-made the Gluster 
volumes that are used for the hosted engine and the VM storage domains. I have 
seen in screenshots of 3.6 HC setups that there should be options to enable 
Gluster integration in oVirt by checking a box in Cluster properties > General 
tab (“Enable Gluster Service”) but I do not see that checkbox in my Cluster 
properties > General tab. What must I do to enable this integration?

BTW, I see that there is a package "vdsm-gluster-4.17.18-0.el7.centos.noarch” 
installed on all of my oVirt nodes… Is there another package that I am missing?

Thanks,
Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Not seeing Gluster options on Cluster properties

[Will Dennis]

> -Original Message-
> From: Sahina Bose [mailto:sab...@redhat.com] 
> Sent: Tuesday, March 01, 2016 10:01 AM
> To: Will Dennis; users
> Subject: Re: [ovirt-users] Not seeing Gluster options on Cluster properties
>
>
> How did you install the engine? Automatic install via ovirt-engine appliance?
> If so, it's likely that the engine is installed in "Virt" only mode.

Yes, I used the appliance OVF

> Can you also check "engine-config -g AllowClusterWithVirtGlusterEnabled" 
> - this needs to be true.

[root@ovirt-engine-01 ~]# engine-config -g AllowClusterWithVirtGlusterEnabled
AllowClusterWithVirtGlusterEnabled: true version: general

What additional changes need to be made to support Gluster management?
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Not seeing Gluster options on Cluster properties

[Will Dennis]

Ah, although this worked, and I see the Gluster volumes now in the tree & can 
manage them, I see for both volumes I have (“engine” and “vm_data”) that the 
number of bricks is “0”... Why can’t it see the existing bricks?

From: Will Dennis
Sent: Tuesday, March 01, 2016 11:13 AM
To: 'Sahina Bose'; users
Subject: RE: [ovirt-users] Not seeing Gluster options on Cluster properties

Great, that worked – thanks Sahina!


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Not seeing Gluster options on Cluster properties

[Will Dennis]

Great, that worked – thanks Sahina!


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] User cannot add a disk to a VM - permission error

[Will Dennis]

Trying to understand the oVirt user permissions system… I have a user who I 
have granted the “PowerUserRole” role to, with the role description being “User 
Role, allowed to create VMs, Templates and Disks.”

When I log into the User Portal with this user, I try to create a VM with a 
disk, but when I click the OK button, I get the message, “Cannot add Virtual 
Machine Disk. The user doesn’t have permissions to attach Disk Profile to the 
Disk.

What other permission(s) do I need to grant this user in order to have it be 
able to create VMs?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] User cannot add a disk to a VM - permission error

[Will Dennis]

Thanks, that worked...

-Original Message-
From: Ondra Machacek [mailto:omach...@redhat.com] 
Sent: Wednesday, March 09, 2016 2:31 AM
To: Will Dennis; users
Subject: Re: [ovirt-users] User cannot add a disk to a VM - permission error

On 03/09/2016 05:02 AM, Will Dennis wrote:
> Trying to understand the oVirt user permissions system… I have a user who I 
> have granted the “PowerUserRole” role to, with the role description being 
> “User Role, allowed to create VMs, Templates and Disks.”
>
> When I log into the User Portal with this user, I try to create a VM with a 
> disk, but when I click the OK button, I get the message, “Cannot add Virtual 
> Machine Disk. The user doesn’t have permissions to attach Disk Profile to the 
> Disk.
>
> What other permission(s) do I need to grant this user in order to have it be 
> able to create VMs?

See this bz[1]. You should also have 'DiskProfileUser' role on storagedomain 
where you want to create disk.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1209505

>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Active Directory (LDAP) user auth is slow

[Will Dennis]

Hi all,

I have enabled Active Directory authentication for the users in oVirt (via 
engine-manage-domains command using --provider=ad) and, although it works, it 
takes about ~50 sec’s to process a login. I have other OSS software that 
utilizes AD auth, and there is no such lag when processing logins, so I’m 
guessing it’s a problem with the oVirt implementation… Any way to debug why the 
auth process is taking so long?

Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Active Directory (LDAP) user auth is slow

[Will Dennis]

In the RHEV Admin Guide that Martin mentioned, it says:

"Log in to the Administration Portal, and remove all users and groups related 
to the old profile. Users defined in the removed domain will no longer be able 
to authenticate with the Red Hat Enterprise Virtualization Manager. The entries 
for the affected users will remain defined in the Red Hat Enterprise 
Virtualization Manager until they are explicitly removed from the 
Administration Portal.”

I have some VMs running under some AD domain users; if I remove the users from 
the system as above, will I need to remove them from the VM permissions, or is 
that cleaned up as well? And I guess I’ll need to manually re-add the perms 
back after the new directory config is in place? Please advise.

Thanks,
Will

On Mar 21, 2016, at 4:29 AM, Martin Perina 
mailto:mper...@redhat.com>> wrote:



On Mon, Mar 21, 2016 at 8:20 AM, Yedidyah Bar David 
mailto:d...@redhat.com>> wrote:
On Mon, Mar 21, 2016 at 4:47 AM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
> Hi all,
>
> I have enabled Active Directory authentication for the users in oVirt (via 
> engine-manage-domains command using --provider=ad) and, although it works, it 
> takes about ~50 sec’s to process a login. I have other OSS software that 
> utilizes AD auth, and there is no such lag when processing logins, so I’m 
> guessing it’s a problem with the oVirt implementation… Any way to debug why 
> the auth process is taking so long?

This is an old, unmaintained component. You should use the new aaa-ldap one.
Search the list archives for "aaa-ldap" and/or read the README file in the
sources [1]. Best,

[1] 
https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extension-aaa-ldap.git;a=blob;f=README

​You could also take a look at RHEV 3.6 Administration Guide, chapter 13 Users 
and Roles [2]
where you can find detailed steps for common configurations.

Martin Perina

[2] 
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html/Administration_Guide/chap-Users_and_Roles.html
​


>
> Will
> ___
> Users mailing list
> Users@ovirt.org<mailto:Users@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users



--
Didi
___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Active Directory (LDAP) user auth is slow

[Will Dennis]

No worries...

I only had a few VMs to re-assign, so I did it manually...

-Original Message-
From: Karli Sjöberg [mailto:karli.sjob...@slu.se] 
Sent: Thursday, March 24, 2016 11:13 AM
To: Ondra Machacek
Cc: Martin Perina; Will Dennis; users
Subject: SV: [ovirt-users] Active Directory (LDAP) user auth is slow

Sorry about the thread-breakage, OWA...

Från: Ondra Machacek 
Skickat: den 24 mars 2016 15:08
Till: Karli Sjöberg
Kopia: Martin Perina; Will Dennis; users
Ämne: Re: [ovirt-users] Active Directory (LDAP) user auth is slow

On 03/24/2016 03:02 PM, Karli Sjöberg wrote:
>
> Den 24 mars 2016 13:49 skrev Ondra Machacek :
>  >
>  > Hi,
>  >
>  > if you remove user, then also permissions of that user to vms will 
> be  > removed.
>  > And yes, you will have to add all those permissions back to users 
> from  > new profile.
>  >
>  > But, you can try migration tool[1], to migrate all users to new AAA 
> profile.
>  > If you have any problem with it, you can ask.
>
> Ehm, how do you install it? (el6)

yum install -y
https://github.com/machacekondra/ovirt-engine-kerbldap-migration/releases/download/ovirt-engine-kerbldap-migration-1.0.4/ovirt-engine-kerbldap-migration-1.0.4-1.el6ev.noarch.rpm

That worked, plus the migration, but can´t log in since our domain is called 
like 'baz.foo.bar' but our users´s userPrincipalName are just 'u...@foo.bar'. 
How do you configure that with aaa?

/K

>
> /K
>
>  >
>  > Ondra
>  >
>  > [1]
>  >
> https://github.com/machacekondra/ovirt-engine-kerbldap-migration/blob/
> master/README.md
>  >
>  > On 03/24/2016 01:06 PM, Will Dennis wrote:
>  > > In the RHEV Admin Guide that Martin mentioned, it says:
>  > >
>  > > "Log in to the Administration Portal, and remove all users and 
> groups related to the old profile. Users defined in the removed domain 
> will no longer be able to authenticate with the Red Hat Enterprise 
> Virtualization Manager. The entries for the affected users will remain 
> defined in the Red Hat Enterprise Virtualization Manager until they 
> are explicitly removed from the Administration Portal.”
>  > >
>  > > I have some VMs running under some AD domain users; if I remove 
> the users from the system as above, will I need to remove them from 
> the VM permissions, or is that cleaned up as well? And I guess I’ll 
> need to manually re-add the perms back after the new directory config 
> is in place? Please advise.
>  > >
>  > > Thanks,
>  > > Will
>  > >
>  > > On Mar 21, 2016, at 4:29 AM, Martin Perina 
> mailto:mper...@redhat.com>> wrote:
>  > >
>  > >
>  > >
>  > > On Mon, Mar 21, 2016 at 8:20 AM, Yedidyah Bar David 
> mailto:d...@redhat.com>> wrote:
>  > > On Mon, Mar 21, 2016 at 4:47 AM, Will Dennis 
> mailto:wden...@nec-labs.com>> wrote:
>  > >> Hi all,
>  > >>
>  > >> I have enabled Active Directory authentication for the users in 
> oVirt (via engine-manage-domains command using --provider=ad) and, 
> although it works, it takes about ~50 sec’s to process a login. I have 
> other OSS software that utilizes AD auth, and there is no such lag 
> when processing logins, so I’m guessing it’s a problem with the oVirt 
> implementation… Any way to debug why the auth process is taking so long?
>  > >
>  > > This is an old, unmaintained component. You should use the new 
> aaa-ldap one.
>  > > Search the list archives for "aaa-ldap" and/or read the README 
> file in the  > > sources [1]. Best,  > >  > > [1] 
> https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extension-aaa-ldap.git;
> a=blob;f=README
>  > >
>  > > ​You could also take a look at RHEV 3.6 Administration Guide, 
> chapter 13 Users and Roles [2]  > > where you can find detailed steps 
> for common configurations.
>  > >
>  > > Martin Perina
>  > >
>  > > [2]
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtu
> alization/3.6/html/Administration_Guide/chap-Users_and_Roles.html
>  > > ​
>  > >
>  > >
>  > >>
>  > >> Will
>  > >> ___
>  > >> Users mailing list
>  > >> Users@ovirt.org<mailto:Users@ovirt.org>
>  > >> http://lists.ovirt.org/mailman/listinfo/users
>  > >
>  > >
>  > >
>  > > --
>  > > Didi
>  > > ___
>  > > Users mailing list
>  > > Users@ovirt.org<mailto:Users@ovirt.org>
>  > > http://lists.ovirt.org/mailman/listinfo/users
>  > >
>  > >
>  > > ___
>  > > Users mailing list
>  > > Users@ovirt.org
>  > > http://lists.ovirt.org/mailman/listinfo/users
>  > >
>  > ___
>  > Users mailing list
>  > Users@ovirt.org
>  > http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Hosts temporarily in "Non Operational" state after upgrade

[Will Dennis]

Hi all,

Had run updates tonight on my three oVirt hosts (3.6 hyperconverged) on on two 
of them, they went into “non Operational” state for a few minutes each before 
springing back to life… The synopsis was this:

- Ran updates throughout the web Admin UI
...then I got the following series of messages via the “Events” tab in the UI:
- Updates successfully ran
- VDSM “command failed: Heartbeat exceeded” message
- host is not responding message
- "Failed to connect to hosted_storage" message
- “The error message for connection localhost:/engine returned by VDSM was: 
Problem while trying to mount target”
- "Host  reports about one of the Active Storage Domains as Problematic”
- “Host  cannot access the Storage Domain(s) hosted_storage attached to 
the data center Default. Setting host state to Non-Operational.”
- "Detected change in status of brick {…} of volume {…} from DOWN to UP.” (once 
for every brick on the host for every Gluster volume.)
- "Host  was autorecovered.”
- "Status of host  was set to Up."

(BTW, it would be awesome if the UI’s Events log could be copied and pasted… 
Doesn’t work for me at least…)

Duration of outage was ~3 mins per each affected host. Didn’t happen on the 
first host I upgraded, but did on the last two.

I know I’m a little over the bleeding edge running hyperconverged on 3.6 :) 
but, should this behavior be expected?

Also, if I go onto the hosts directly and run a ‘yum update’ after this upgrade 
process (not that I went thru with it, just wanted to see what was available to 
be upgraded) I see a bunch of ovirt-* packages that can be upgraded, which 
didn’t get updated thru the web UI’s upgrade process —
ovirt-engine-sdk-pythonnoarch   3.6.5.0-1.el7.centos 
ovirt-3.6  480 k
ovirt-hosted-engine-ha noarch   1.3.5.3-1.1.el7  
centos-ovirt36 295 k
ovirt-hosted-engine-setup  noarch   1.3.5.0-1.1.el7  
centos-ovirt36 270 k
ovirt-release36noarch   007-1
ovirt-3.6  9.5 k

Are these packages not related to the “Upgrade” process available thru the web 
UI?

FYI, here’s what did get updated thru the web UI “Upgrade” process —
Apr 27 21:36:28 Updated: libvirt-client-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: libvirt-daemon-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: libvirt-daemon-driver-network-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: libvirt-daemon-driver-qemu-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: libvirt-daemon-driver-nwfilter-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: vdsm-infra-4.17.26-1.el7.noarch
Apr 27 21:36:28 Updated: vdsm-python-4.17.26-1.el7.noarch
Apr 27 21:36:28 Updated: vdsm-xmlrpc-4.17.26-1.el7.noarch
Apr 27 21:36:28 Updated: libvirt-daemon-config-nwfilter-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: mom-0.5.3-1.1.el7.noarch
Apr 27 21:36:29 Updated: libvirt-lock-sanlock-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: libvirt-daemon-driver-secret-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: libvirt-daemon-driver-nodedev-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: 
libvirt-daemon-driver-interface-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: libvirt-daemon-driver-storage-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: libvirt-daemon-kvm-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: 1:libguestfs-1.28.1-1.55.el7.centos.2.x86_64
Apr 27 21:36:29 Updated: 1:libguestfs-tools-c-1.28.1-1.55.el7.centos.2.x86_64
Apr 27 21:36:29 Installed: libguestfs-winsupport-7.2-1.el7.x86_64
Apr 27 21:36:29 Updated: vdsm-yajsonrpc-4.17.26-1.el7.noarch
Apr 27 21:36:29 Updated: vdsm-jsonrpc-4.17.26-1.el7.noarch
Apr 27 21:36:29 Installed: unzip-6.0-15.el7.x86_64
Apr 27 21:36:30 Installed: gtk2-2.24.28-8.el7.x86_64
Apr 27 21:36:31 Installed: 1:virt-v2v-1.28.1-1.55.el7.centos.2.x86_64
Apr 27 21:36:31 Updated: safelease-1.0-7.el7.x86_64
Apr 27 21:36:31 Updated: vdsm-hook-vmfex-dev-4.17.26-1.el7.noarch
Apr 27 21:36:32 Updated: vdsm-4.17.26-1.el7.noarch
Apr 27 21:36:32 Updated: vdsm-gluster-4.17.26-1.el7.noarch
Apr 27 21:36:32 Updated: vdsm-cli-4.17.26-1.el7.noarch

Thanks,
Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosts temporarily in "Non Operational" state after upgrade

[Will Dennis]

Bump - can any RHAT folks comment on this?

-Original Message-
From: Will Dennis 
Sent: Wednesday, April 27, 2016 11:00 PM
To: users@ovirt.org
Subject: Hosts temporarily in "Non Operational" state after upgrade

Hi all,

Had run updates tonight on my three oVirt hosts (3.6 hyperconverged) on on two 
of them, they went into “non Operational” state for a few minutes each before 
springing back to life… The synopsis was this:

- Ran updates throughout the web Admin UI ...then I got the following series of 
messages via the “Events” tab in the UI:
- Updates successfully ran
- VDSM “command failed: Heartbeat exceeded” message
- host is not responding message
- "Failed to connect to hosted_storage" message
- “The error message for connection localhost:/engine returned by VDSM was: 
Problem while trying to mount target”
- "Host  reports about one of the Active Storage Domains as Problematic”
- “Host  cannot access the Storage Domain(s) hosted_storage attached to 
the data center Default. Setting host state to Non-Operational.”
- "Detected change in status of brick {…} of volume {…} from DOWN to UP.” (once 
for every brick on the host for every Gluster volume.)
- "Host  was autorecovered.”
- "Status of host  was set to Up."

(BTW, it would be awesome if the UI’s Events log could be copied and pasted… 
Doesn’t work for me at least…)

Duration of outage was ~3 mins per each affected host. Didn’t happen on the 
first host I upgraded, but did on the last two.

I know I’m a little over the bleeding edge running hyperconverged on 3.6 :) 
but, should this behavior be expected?

Also, if I go onto the hosts directly and run a ‘yum update’ after this upgrade 
process (not that I went thru with it, just wanted to see what was available to 
be upgraded) I see a bunch of ovirt-* packages that can be upgraded, which 
didn’t get updated thru the web UI’s upgrade process —
ovirt-engine-sdk-pythonnoarch   3.6.5.0-1.el7.centos 
ovirt-3.6  480 k
ovirt-hosted-engine-ha noarch   1.3.5.3-1.1.el7  
centos-ovirt36 295 k
ovirt-hosted-engine-setup  noarch   1.3.5.0-1.1.el7  
centos-ovirt36 270 k
ovirt-release36noarch   007-1
ovirt-3.6  9.5 k

Are these packages not related to the “Upgrade” process available thru the web 
UI?

FYI, here’s what did get updated thru the web UI “Upgrade” process — Apr 27 
21:36:28 Updated: libvirt-client-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: libvirt-daemon-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: libvirt-daemon-driver-network-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: libvirt-daemon-driver-qemu-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: libvirt-daemon-driver-nwfilter-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:28 Updated: vdsm-infra-4.17.26-1.el7.noarch Apr 27 21:36:28 
Updated: vdsm-python-4.17.26-1.el7.noarch Apr 27 21:36:28 Updated: 
vdsm-xmlrpc-4.17.26-1.el7.noarch Apr 27 21:36:28 Updated: 
libvirt-daemon-config-nwfilter-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: mom-0.5.3-1.1.el7.noarch Apr 27 21:36:29 Updated: 
libvirt-lock-sanlock-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: libvirt-daemon-driver-secret-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: libvirt-daemon-driver-nodedev-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: 
libvirt-daemon-driver-interface-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: libvirt-daemon-driver-storage-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: libvirt-daemon-kvm-1.2.17-13.el7_2.4.x86_64
Apr 27 21:36:29 Updated: 1:libguestfs-1.28.1-1.55.el7.centos.2.x86_64
Apr 27 21:36:29 Updated: 1:libguestfs-tools-c-1.28.1-1.55.el7.centos.2.x86_64
Apr 27 21:36:29 Installed: libguestfs-winsupport-7.2-1.el7.x86_64
Apr 27 21:36:29 Updated: vdsm-yajsonrpc-4.17.26-1.el7.noarch
Apr 27 21:36:29 Updated: vdsm-jsonrpc-4.17.26-1.el7.noarch Apr 27 21:36:29 
Installed: unzip-6.0-15.el7.x86_64 Apr 27 21:36:30 Installed: 
gtk2-2.24.28-8.el7.x86_64 Apr 27 21:36:31 Installed: 
1:virt-v2v-1.28.1-1.55.el7.centos.2.x86_64
Apr 27 21:36:31 Updated: safelease-1.0-7.el7.x86_64 Apr 27 21:36:31 Updated: 
vdsm-hook-vmfex-dev-4.17.26-1.el7.noarch
Apr 27 21:36:32 Updated: vdsm-4.17.26-1.el7.noarch Apr 27 21:36:32 Updated: 
vdsm-gluster-4.17.26-1.el7.noarch Apr 27 21:36:32 Updated: 
vdsm-cli-4.17.26-1.el7.noarch

Thanks,
Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosts temporarily in "Non Operational" state after upgrade

[Will Dennis]

(so noted)   ...or anyone else who knows the answer ;)

-Original Message-
From: Michal Skrivanek [mailto:michal.skriva...@redhat.com] 
Sent: Friday, April 29, 2016 9:02 AM
To: Will Dennis
Cc: users@ovirt.org
Subject: Re: [ovirt-users] Hosts temporarily in "Non Operational" state after 
upgrade


> On 29 Apr 2016, at 14:46, Will Dennis  wrote:
> 
> Bump - can any RHAT folks comment on this?

note oVirt is a community project;-)

> 
> -Original Message-
> From: Will Dennis 
> Sent: Wednesday, April 27, 2016 11:00 PM
> To: users@ovirt.org
> Subject: Hosts temporarily in "Non Operational" state after upgrade
> 
> Hi all,
> 
> Had run updates tonight on my three oVirt hosts (3.6 hyperconverged) on on 
> two of them, they went into “non Operational” state for a few minutes each 
> before springing back to life… The synopsis was this:
> 
> - Ran updates throughout the web Admin UI ...then I got the following series 
> of messages via the “Events” tab in the UI:

what exactly did you do in the UI?

> - Updates successfully ran
> - VDSM “command failed: Heartbeat exceeded” message
> - host is not responding message
> - "Failed to connect to hosted_storage" message
> - “The error message for connection localhost:/engine returned by VDSM was: 
> Problem while trying to mount target”
> - "Host  reports about one of the Active Storage Domains as Problematic”
> - “Host  cannot access the Storage Domain(s) hosted_storage attached to 
> the data center Default. Setting host state to Non-Operational.”
> - "Detected change in status of brick {…} of volume {…} from DOWN to UP.” 
> (once for every brick on the host for every Gluster volume.)
> - "Host  was autorecovered.”
> - "Status of host  was set to Up.”

so..it was not in Maintenance when you run the update?
You should avoid doing that as an update to any package may interfere with 
running guests. E.g. a qemu rpm update can (and likely will) simply kill all 
your VMs, I suppose similarly for Gluster before updating anything the volumes 
should be in some kind of maintenance mode as well

> 
> (BTW, it would be awesome if the UI’s Events log could be copied and pasted… 
> Doesn’t work for me at least…)
> 
> Duration of outage was ~3 mins per each affected host. Didn’t happen on the 
> first host I upgraded, but did on the last two.
> 
> I know I’m a little over the bleeding edge running hyperconverged on 3.6 :) 
> but, should this behavior be expected?
> 
> Also, if I go onto the hosts directly and run a ‘yum update’ after this 
> upgrade process (not that I went thru with it, just wanted to see what was 
> available to be upgraded) I see a bunch of ovirt-* packages that can be 
> upgraded, which didn’t get updated thru the web UI’s upgrade process —
> ovirt-engine-sdk-pythonnoarch   3.6.5.0-1.el7.centos 
> ovirt-3.6  480 k
> ovirt-hosted-engine-ha noarch   1.3.5.3-1.1.el7  
> centos-ovirt36 295 k
> ovirt-hosted-engine-setup  noarch   1.3.5.0-1.1.el7  
> centos-ovirt36 270 k
> ovirt-release36noarch   007-1
> ovirt-3.6  9.5 k
> 
> Are these packages not related to the “Upgrade” process available thru the 
> web UI?
> 
> FYI, here’s what did get updated thru the web UI “Upgrade” process — Apr 27 
> 21:36:28 Updated: libvirt-client-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:28 Updated: libvirt-daemon-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:28 Updated: 
> libvirt-daemon-driver-network-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:28 Updated: libvirt-daemon-driver-qemu-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:28 Updated: 
> libvirt-daemon-driver-nwfilter-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:28 Updated: vdsm-infra-4.17.26-1.el7.noarch Apr 27 21:36:28 
> Updated: vdsm-python-4.17.26-1.el7.noarch Apr 27 21:36:28 Updated: 
> vdsm-xmlrpc-4.17.26-1.el7.noarch Apr 27 21:36:28 Updated: 
> libvirt-daemon-config-nwfilter-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:29 Updated: mom-0.5.3-1.1.el7.noarch Apr 27 21:36:29 Updated: 
> libvirt-lock-sanlock-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:29 Updated: libvirt-daemon-driver-secret-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:29 Updated: 
> libvirt-daemon-driver-nodedev-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:29 Updated: 
> libvirt-daemon-driver-interface-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:29 Updated: 
> libvirt-daemon-driver-storage-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:29 Updated: libvirt-daemon-kvm-1.2.17-13.el7_2.4.x86_64
> Apr 27 21:36:29 Updated: 1:libguestfs-1.28.1-1.55.el7.centos.2.x86_64
> Apr 27 21:36:29 Updated: 1:libguestfs-tools-c-

Re: [ovirt-users] Hosts temporarily in "Non Operational" state after upgrade

[Will Dennis]

Answers inline below...

> From: Michal Skrivanek [mailto:michal.skriva...@redhat.com] 

> what exactly did you do in the UI?
Clicked on the node, and in the bottom pane, clicked on the "Upgrade" link 
showing there (the nodes also had an icon indicating that updates were 
available)

> so..it was not in Maintenance when you run the update?
> You should avoid doing that as an update to any package may interfere with 
> running guests. 
> E.g. a qemu rpm update can (and likely will) simply kill all your VMs, I 
> suppose similarly for Gluster before updating anything 
> the volumes should be in some kind of maintenance mode as well

No, the "Upgrade" link once clicked migrates any running VM off the target node 
onto another node, then sets the target node into Maintenance mode, and then 
performs the updates. Once the updates are completed successfully, it 
re-activates the node and makes it available again. On the second and third 
nodes this coming out of Maintenance process experienced a problem with 
mounting the Gluster storage so it seems, and had the problems I'd indicated.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

Hi all,

Have an interesting problem - I am running a VM in oVirt that is running 
Proxmox VE 4.1 OS, which I have spun up a container on.  The container is set 
for DHCP, and I have verified that it is sending Discover packets as normal, 
and that these packets are making it out of the Proxmox VM to the oVirt bridge 
(which is attached to a VLAN sub-interface of a bond interface.) However, these 
packets do NOT make it past the oVirt bridge. The interesting thing is that the 
Proxmox VM (as well as any other VM I spin up on oVirt) works fine with DHCP. 
(I also have other oVirt VMs instantiated which are using LXD to spin up 
containers, and I have the same problem with those as well.) I checked a bunch 
of stuff, and the only clue I could find is that it seems that the oVirt bridge 
is not learning the MAC for the container on the VM, even though it does learn 
the VM's MAC, but I can capture DHCP traffic coming from the container off the 
'vnet0' interface which is joined to that bridge...

Info:

= off Proxmox VM =

Container's MAC address: 32:62:65:61:65:33

root@proxmox-02:~# ip link sh
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode 
DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0:  mtu 1500 qdisc pfifo_fast master 
vmbr0 state UP mode DEFAULT group default qlen 1000
link/ether 00:1a:4a:16:01:57 brd ff:ff:ff:ff:ff:ff
3: vmbr0:  mtu 1500 qdisc noqueue state UP 
mode DEFAULT group default
link/ether 00:1a:4a:16:01:57 brd ff:ff:ff:ff:ff:ff
7: veth100i0@if6:  mtu 1500 qdisc pfifo_fast 
master vmbr0 state UP mode DEFAULT group default qlen 1000
link/ether fe:50:4f:3c:bd:b8 brd ff:ff:ff:ff:ff:ff link-netnsid 0   <<< 
veth connection to container

root@proxmox-02:~# brctl showmacs vmbr0
port no mac addris local?   ageing timer
  1 00:12:3f:24:a4:54   no   112.88
  1 00:1a:4a:16:01:56   no 0.02
  1 00:1a:4a:16:01:57   yes0.00
  1 00:1a:4a:16:01:57   yes0.00
  1 00:24:50:dd:a2:05   no 1.37
  1 18:03:73:e3:be:5a   no21.04
  1 18:03:73:e3:ca:24   no 4.23
  1 18:03:73:e3:cb:5b   no48.41
  1 18:03:73:e3:cc:e5   no91.93
  1 18:03:73:e3:cd:b8   no   151.04
  1 18:03:73:e3:ce:43   no 0.80
  1 18:03:73:e3:d0:a4   no   290.74
  1 18:03:73:e3:d4:26   no34.06
  1 18:03:73:e3:d5:3d   no 6.36
  1 18:03:73:e4:23:08   no88.76
  1 18:03:73:e4:25:92   no   111.86
  1 18:03:73:e4:26:2f   no 9.54
  1 18:03:73:e4:2b:4c   no   114.86
  1 18:03:73:e4:31:15   no   263.91
  1 18:03:73:e4:6c:19   no 6.36
  1 18:03:73:e4:7e:0a   no   103.06
  1 18:03:73:e8:16:e0   no23.21
  2 32:62:65:61:65:33   no 5.08   <<< container's MAC 
learned on Proxmox bridge
  1 34:17:eb:9b:e0:29   no   265.22
  1 34:17:eb:9b:f8:ea   no   114.86
  1 44:d3:ca:7e:3c:ff   no 0.00
  1 78:2b:cb:3b:ca:b9   no   284.70
  1 78:2b:cb:92:cb:cb   no   279.70
  1 78:2b:cb:93:08:a8   no   287.05
  1 b8:ca:3a:7a:70:63   no 4.83
  1 f8:bc:12:69:bb:a3   no   121.82
  2 fe:50:4f:3c:bd:b8   yes0.00
  2 fe:50:4f:3c:bd:b8   yes0.00

= off oVirt node that has Proxmox VM 

(relevant lines from 'ip link show')
2: bond0:  mtu 1500 qdisc noqueue state 
UP mode DEFAULT
3: enp4s0f0:  mtu 1500 qdisc pfifo_fast 
master bond0 state UP mode DEFAULT qlen 1000
4: enp4s0f1:  mtu 1500 qdisc pfifo_fast 
master bond0 state UP mode DEFAULT qlen 1000
8: bond0.169@bond0:  mtu 1500 qdisc noqueue 
master 169-net state UP mode DEFAULT
10: bond0.180@bond0:  mtu 1500 qdisc noqueue 
master 180-net state UP mode DEFAULT
12: bond0.207@bond0:  mtu 1500 qdisc noqueue 
master 207-net state UP mode DEFAULT
13: 207-net:  mtu 1500 qdisc noqueue state UP 
mode DEFAULT
30: vnet0:  mtu 1500 qdisc pfifo_fast master 
207-net state UNKNOWN mode DEFAULT qlen 500 <<< veth connection to Proxmox VM
31: vnet1:  mtu 1500 qdisc pfifo_fast master 
207-net state UNKNOWN mode DEFAULT qlen 500

[root@ovirt-node-03 ~]# brctl show
bridge name bridge id   STP enabled interfaces
169-net 8000.0015177be9da   no  bond0.169
180-net 8000.0015177be9da   no  bond0.180
207-net 8000.0015177be9da   no  bond0.207
vnet0
vnet1
;

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

That’s in iptables, right? I have iptables disabled on my oVirt nodes...

From: Yaniv Kaul [mailto:yk...@redhat.com]
Sent: Friday, May 06, 2016 3:50 PM
To: Will Dennis
Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

Long shot - you need to disable the EnableMACAntiSpoofingFilterRules .
Y.

On Fri, May 6, 2016 at 8:27 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
Hi all,

Have an interesting problem – I am running a VM in oVirt that is running 
Proxmox VE 4.1 OS, which I have spun up a container on.  The container is set 
for DHCP, and I have verified that it is sending Discover packets as normal, 
and that these packets are making it out of the Proxmox VM to the oVirt bridge 
(which is attached to a VLAN sub-interface of a bond interface.) However, these 
packets do NOT make it past the oVirt bridge. The interesting thing is that the 
Proxmox VM (as well as any other VM I spin up on oVirt) works fine with DHCP. 
(I also have other oVirt VMs instantiated which are using LXD to spin up 
containers, and I have the same problem with those as well.) I checked a bunch 
of stuff, and the only clue I could find is that it seems that the oVirt bridge 
is not learning the MAC for the container on the VM, even though it does learn 
the VM’s MAC, but I can capture DHCP traffic coming from the container off the 
‘vnet0’ interface which is joined to that bridge...

Info:

= off Proxmox VM =

Container's MAC address: 32:62:65:61:65:33

root@proxmox-02:~# ip link sh
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode 
DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0:  mtu 1500 qdisc pfifo_fast master 
vmbr0 state UP mode DEFAULT group default qlen 1000
link/ether 00:1a:4a:16:01:57 brd ff:ff:ff:ff:ff:ff
3: vmbr0:  mtu 1500 qdisc noqueue state UP 
mode DEFAULT group default
link/ether 00:1a:4a:16:01:57 brd ff:ff:ff:ff:ff:ff
7: veth100i0@if6:  mtu 1500 qdisc pfifo_fast 
master vmbr0 state UP mode DEFAULT group default qlen 1000
link/ether fe:50:4f:3c:bd:b8 brd ff:ff:ff:ff:ff:ff link-netnsid 0   <<< 
veth connection to container

root@proxmox-02:~# brctl showmacs vmbr0
port no mac addris local?   ageing timer
  1 00:12:3f:24:a4:54   no   112.88
  1 00:1a:4a:16:01:56   no 0.02
  1 00:1a:4a:16:01:57   yes0.00
  1 00:1a:4a:16:01:57   yes0.00
  1 00:24:50:dd:a2:05   no 1.37
  1 18:03:73:e3:be:5a   no21.04
  1 18:03:73:e3:ca:24   no 4.23
  1 18:03:73:e3:cb:5b   no48.41
  1 18:03:73:e3:cc:e5   no91.93
  1 18:03:73:e3:cd:b8   no   151.04
  1 18:03:73:e3:ce:43   no 0.80
  1 18:03:73:e3:d0:a4   no   290.74
  1 18:03:73:e3:d4:26   no34.06
  1 18:03:73:e3:d5:3d   no 6.36
  1 18:03:73:e4:23:08   no88.76
  1 18:03:73:e4:25:92   no   111.86
  1 18:03:73:e4:26:2f   no 9.54
  1 18:03:73:e4:2b:4c   no   114.86
  1 18:03:73:e4:31:15   no   263.91
  1 18:03:73:e4:6c:19   no 6.36
  1 18:03:73:e4:7e:0a   no   103.06
  1 18:03:73:e8:16:e0   no23.21
  2 32:62:65:61:65:33   no 5.08   <<< container’s MAC 
learned on Proxmox bridge
  1 34:17:eb:9b:e0:29   no   265.22
  1 34:17:eb:9b:f8:ea   no   114.86
  1 44:d3:ca:7e:3c:ff   no 0.00
  1 78:2b:cb:3b:ca:b9   no   284.70
  1 78:2b:cb:92:cb:cb   no   279.70
  1 78:2b:cb:93:08:a8   no   287.05
  1 b8:ca:3a:7a:70:63   no 4.83
  1 f8:bc:12:69:bb:a3   no   121.82
  2 fe:50:4f:3c:bd:b8   yes0.00
  2 fe:50:4f:3c:bd:b8   yes0.00

= off oVirt node that has Proxmox VM 

(relevant lines from ‘ip link show’)
2: bond0:  mtu 1500 qdisc noqueue state 
UP mode DEFAULT
3: enp4s0f0:  mtu 1500 qdisc pfifo_fast 
master bond0 state UP mode DEFAULT qlen 1000
4: enp4s0f1:  mtu 1500 qdisc pfifo_fast 
master bond0 state UP mode DEFAULT qlen 1000
8: bond0.169@bond0<mailto:bond0.169@bond0>:  
mtu 1500 qdisc noqueue master 169-net state UP mode DEFAULT
10: bond0.180@bond0<mailto:bond0.180@bond0>:  
mtu 1500 qdisc noqueue master 180-net state UP mode DEFAULT
12: bond0.207@bond0<mailto:bond0.207@bond0>:  
mtu 1500 qdisc noqueue master 207-net state UP mode DEFAULT
13: 207-net:  mtu 1500 qdisc noqueue state UP 
mode DEFAULT
30: vnet0:  mtu 1500 qdisc pfifo_fast master 
207-net state UNKNOWN mode DEFAULT qlen 500 <<

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

Hi Simone,

Sorry but I'm just getting back to this... Not sure where/how one might 
implement the hook you have indicated... It looks to be VM-specific, but I 
don't see anywhere in either the Admin UI or the User portal UI to set such a 
parameter... Can you give me instructions on how to implement this?

-Original Message-
From: Simone Tiraboschi [mailto:stira...@redhat.com] 
Sent: Monday, May 09, 2016 3:54 AM
To: Yaniv Kaul
Cc: Will Dennis; users@ovirt.org
Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

There is also a specific VDSM hook to address this configuration:
https://github.com/oVirt/vdsm/tree/master/vdsm_hooks/macspoof

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

OK; I searched the oVirt site for 'engine-config' (unfamiliar with it) and 
found the following page:
https://www.ovirt.org/develop/developer-guide/engine/engine-config-examples/

I see the instructions on it for "Adding VM custom properties (macspoof)", so I 
did execute the referenced engine-config line, and then restarted the 
ovirt-engine service. I then went and powered off the VM I'd like to deactivate 
mac spoof filtering on, then went in to the User Portal and clicked "Edit" on 
the VM. However, I can not find a "Custom Properties" dialog anywhere in the VM 
Edit UI (yes, I did enable Advanced options.) Where should I be seeing this?


-Original Message-
From: Yedidyah Bar David [mailto:d...@redhat.com] 
Sent: Tuesday, May 10, 2016 4:31 PM
To: Will Dennis
Cc: Simone Tiraboschi; Yaniv Kaul; users@ovirt.org
Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

On Tue, May 10, 2016 at 8:36 PM, Will Dennis  wrote:
> Hi Simone,
>
> Sorry but I'm just getting back to this... Not sure where/how one might 
> implement the hook you have indicated... It looks to be VM-specific, but I 
> don't see anywhere in either the Admin UI or the User portal UI to set such a 
> parameter... Can you give me instructions on how to implement this?
>

Please check the README that Simone linked to.

You have to add a property using engine-config.
It will appear in the ui when editing a VM, under custom properties.

> -Original Message-
> From: Simone Tiraboschi [mailto:stira...@redhat.com]
> Sent: Monday, May 09, 2016 3:54 AM
> To: Yaniv Kaul
> Cc: Will Dennis; users@ovirt.org
> Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
> in a oVirt VM
>
> There is also a specific VDSM hook to address this configuration:
> https://github.com/oVirt/vdsm/tree/master/vdsm_hooks/macspoof
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users



-- 
Didi
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

Yup, I see it now in the Admin portal VM Edit dialog - thanks!

Will

-Original Message-
From: Michal Skrivanek [mailto:mskri...@redhat.com] 
Sent: Wednesday, May 18, 2016 2:37 AM
To: Yedidyah Bar David
Cc: Will Dennis; users@ovirt.org
Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM



> On 15 May 2016, at 07:24, Yedidyah Bar David  wrote:
> 
>> On Wed, May 11, 2016 at 5:17 PM, Will Dennis  wrote:
>> OK; I searched the oVirt site for 'engine-config' (unfamiliar with it) and 
>> found the following page:
>> https://www.ovirt.org/develop/developer-guide/engine/engine-config-examples/
>> 
>> I see the instructions on it for "Adding VM custom properties (macspoof)", 
>> so I did execute the referenced engine-config line, and then restarted the 
>> ovirt-engine service. I then went and powered off the VM I'd like to 
>> deactivate mac spoof filtering on, then went in to the User Portal and 
>> clicked "Edit" on the VM. However, I can not find a "Custom Properties" 
>> dialog anywhere in the VM Edit UI (yes, I did enable Advanced options.) 
>> Where should I be seeing this?
> 
> Not sure it's visible in the User Portal, might be a matter of permissions.

Iirc it's not in user portal by design. All custom properties are generally 
considered admin-like stuff

> 
> In the admin portal it has, in the same dialog, its own sub menu.
> -- 
> Didi
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

Sadly, having set the 'macspoof' key to 'true' in Custom Properties in the VM 
Edit dialog, then restarting the VM thereafter, I'm still not seeing the 
container's MAC address in the relevant bridge's MAC table in oVirt, and the 
container is failing to lease a DHCP address from the external network... 

Looked at the ebtables filter table, nothing there...
[root@ovirt-node-03 ~]# ebtables -t filter -L
Bridge table: filter

Bridge chain: INPUT, entries: 0, policy: ACCEPT

Bridge chain: FORWARD, entries: 0, policy: ACCEPT

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT


Anyone know where else I could look to troubleshoot?


-----Original Message-
From: Will Dennis 
Sent: Wednesday, May 18, 2016 11:06 AM
To: 'Michal Skrivanek'; Yedidyah Bar David
Cc: users@ovirt.org
Subject: RE: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

Yup, I see it now in the Admin portal VM Edit dialog - thanks!

Will

-Original Message-
From: Michal Skrivanek [mailto:mskri...@redhat.com] 
Sent: Wednesday, May 18, 2016 2:37 AM
To: Yedidyah Bar David
Cc: Will Dennis; users@ovirt.org
Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM



> On 15 May 2016, at 07:24, Yedidyah Bar David  wrote:
> 
>> On Wed, May 11, 2016 at 5:17 PM, Will Dennis  wrote:
>> OK; I searched the oVirt site for 'engine-config' (unfamiliar with it) and 
>> found the following page:
>> https://www.ovirt.org/develop/developer-guide/engine/engine-config-examples/
>> 
>> I see the instructions on it for "Adding VM custom properties (macspoof)", 
>> so I did execute the referenced engine-config line, and then restarted the 
>> ovirt-engine service. I then went and powered off the VM I'd like to 
>> deactivate mac spoof filtering on, then went in to the User Portal and 
>> clicked "Edit" on the VM. However, I can not find a "Custom Properties" 
>> dialog anywhere in the VM Edit UI (yes, I did enable Advanced options.) 
>> Where should I be seeing this?
> 
> Not sure it's visible in the User Portal, might be a matter of permissions.

Iirc it's not in user portal by design. All custom properties are generally 
considered admin-like stuff

> 
> In the admin portal it has, in the same dialog, its own sub menu.
> -- 
> Didi
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

Sorry, got away from this for a while attending to other work, but back at it 
now... Anyone have any ideas on how I may further troubleshoot this issue?

Thanks

-Original Message-
From: Will Dennis 
Sent: Wednesday, May 18, 2016 11:39 AM
To: 'Michal Skrivanek'; 'Yedidyah Bar David'
Cc: 'users@ovirt.org'
Subject: RE: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

Sadly, having set the 'macspoof' key to 'true' in Custom Properties in the VM 
Edit dialog, then restarting the VM thereafter, I'm still not seeing the 
container's MAC address in the relevant bridge's MAC table in oVirt, and the 
container is failing to lease a DHCP address from the external network... 

Looked at the ebtables filter table, nothing there...
[root@ovirt-node-03 ~]# ebtables -t filter -L Bridge table: filter

Bridge chain: INPUT, entries: 0, policy: ACCEPT

Bridge chain: FORWARD, entries: 0, policy: ACCEPT

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT


Anyone know where else I could look to troubleshoot?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

;,
u'shared':u'false',
u'path':u'',
u'type':u'disk'
 },
 {  
u'poolID':u'0001-0001-0001-0001-0138',
u'volumeInfo':{  
   u'domainID':u'4e2358ab-6b51-4d13-9e36-ac5cb271618a',
   u'volType':u'path',
   u'leaseOffset':0,
   u'volumeID':u'278ed347-c79a-4168-b578-1562a08ac551',
   u'leasePath':   
u'/rhev/data-center/mnt/glusterSD/ovirt-node-01.nec-labs.com:_vmdata/4e2358ab-6b51-4d13-9e36-ac5cb271618a/images/aeae1aeb-047e-493f-a70c-f471f2e7245c/278ed347-c79a-4168-b578-1562a08ac551.lease',
   u'imageID':u'aeae1aeb-047e-493f-a70c-f471f2e7245c',
   u'path':   
u'/rhev/data-center/mnt/glusterSD/ovirt-node-01.nec-labs.com:_vmdata/4e2358ab-6b51-4d13-9e36-ac5cb271618a/images/aeae1aeb-047e-493f-a70c-f471f2e7245c/278ed347-c79a-4168-b578-1562a08ac551'
},
u'index':0,
u'iface':u'virtio',
u'apparentsize':u'536870912000',
u'alias':u'virtio-disk0',
u'imageID':u'aeae1aeb-047e-493f-a70c-f471f2e7245c',
u'readonly':u'False',
u'shared':u'false',
u'truesize':u'11320029184',
u'type':u'disk',
u'domainID':u'4e2358ab-6b51-4d13-9e36-ac5cb271618a',
u'reqsize':u'0',
u'format':u'raw',
u'deviceId':u'aeae1aeb-047e-493f-a70c-f471f2e7245c',
u'address':{  
   u'slot':u'0x05',
   u'bus':u'0x00',
   u'domain':u'0x',
   u'type':u'pci',
   u'function':u'0x0'
},
u'device':u'disk',

u'path':u'/rhev/data-center/0001-0001-0001-0001-0138/4e2358ab-6b51-4d13-9e36-ac5cb271618a/images/aeae1aeb-047e-493f-a70c-f471f2e7245c/278ed347-c79a-4168-b578-1562a08ac551',
u'propagateErrors':u'off',
u'optional':u'false',
u'name':u'vda',
u'bootOrder':u'1',
u'volumeID':u'278ed347-c79a-4168-b578-1562a08ac551',
u'specParams':{  

},
u'volumeChain':[  
   {  
  
u'domainID':u'4e2358ab-6b51-4d13-9e36-ac5cb271618a',
  u'volType':u'path',
      u'leaseOffset':0,
  
u'volumeID':u'278ed347-c79a-4168-b578-1562a08ac551',
  u'leasePath':  
u'/rhev/data-center/mnt/glusterSD/ovirt-node-01.nec-labs.com:_vmdata/4e2358ab-6b51-4d13-9e36-ac5cb271618a/images/aeae1aeb-047e-493f-a70c-f471f2e7245c/278ed347-c79a-4168-b578-1562a08ac551.lease',
  
u'imageID':u'aeae1aeb-047e-493f-a70c-f471f2e7245c',
  u'path':  
u'/rhev/data-center/mnt/glusterSD/ovirt-node-01.nec-labs.com:_vmdata/4e2358ab-6b51-4d13-9e36-ac5cb271618a/images/aeae1aeb-047e-493f-a70c-f471f2e7245c/278ed347-c79a-4168-b578-1562a08ac551'
   }
]
 },
 {  
u'device':u'usb',
u'alias':u'usb',
u'type':u'controller',
u'address':{  
   u'slot':u'0x01',
   u'bus':u'0x00',
   u'domain':u'0x',
   u'type':u'pci',
   u'function':u'0x2'
}
 },
 {  
u'device':u'ide',
u'alias':u'ide',
u'type':u'controller',
u'address':{  
   u'slot':u'0x01',
   u'bus':u'0x00',
   u'domain':u'0x',
   u'type':u'pci',
   u'function':u'0x1'
}
 },
 {  
u'device':u'unix',
u'alias':u'channel0',
u'type':u'channel',
u'address':{  
   u'bus':u'0',
   u'controller':u'0',
   u'type':u'virtio-serial',
   u'port':u'1'
}
 },
 {  
u'device':u'unix',
u'alias':u'channel1',
u'type':u'channel',
u'address':{  
   u'bus':u'0',
   u'controller':u'0',
   u'type':u'virtio-serial',
   u'port':u'2'
}
 },
 {  
u'device':u'spicevmc',
u'alias':u'channel2',
u'type':u'channel',
u'address':{  
   u'bus':u'0',
   u'controller':u'0',
   u'type':u'virtio-serial',
   u'port':u'3'
}
 }
  ],
  u'display':u'qxl',
  u'status':u'Up',
  u'timeOffset':u'0',
  u'maxVCpus':u'16',
  u'guestIPs':u'',
  u'statusTime':u'12906968250',
  u'maxMemSlots':16
   }


-Original Message-
From: Yedidyah Bar David [mailto:d...@redhat.com] 
Sent: Tuesday, June 07, 2016 2:03 AM
To: Will Dennis
Cc: users@ovirt.org; Michal Skrivanek
Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

On Tue, Jun 7, 2016 at 12:04 AM, Will Dennis  wrote:
> Sorry, got away from this for a while attending to other work, but back at it 
> now... Anyone have any ideas on how I may further troubleshoot this issue?

Edward later asked you to post the domxml.
You can find it with 'virsh dumpxml vm' or search vdsm logs.

Best,
-- 
Didi
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

The "-r" arg to virsh makes all the difference :)  (much to learn...)

[root@ovirt-node-03 ~]# virsh -r list
 IdName   State

 6 proxmox-01 running
 7 proxmox-02 running

[root@ovirt-node-03 ~]# virsh -r dumpxml proxmox-01

  proxmox-01
  1a49b65b-9950-4b29-8c72-139d5263e11d
  http://ovirt.org/vm/tune/1.0";>

  
  4294967296
  8388608
  8388608
  16
  
1020
  
  

  
  
/machine
  
  

  oVirt
  oVirt Node
  7-2.1511.el7.centos.2.10
  271BC900-7252-11DD-8001-002185350C3C
  1a49b65b-9950-4b29-8c72-139d5263e11d

  
  
hvm


  
  

  
  
Penryn


  

  
  



  
  destroy
  restart
  destroy
  
/usr/libexec/qemu-kvm

  
  
  
  
  
  
  
  
  


  
  
  
  
  aeae1aeb-047e-493f-a70c-f471f2e7245c
  
  
  


  
  


  
  


  
  


  


  
  
  
  
  
  
  
  


  
  
  
  


  
  
  
  


  
  
  




  


  
  
  


  

  


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

Hi Simone,

These are the instructions I followed, which I found at: 
https://www.ovirt.org/develop/developer-guide/engine/engine-config-examples/


Adding VM custom properties (macspoof)
  # engine-config -s "UserDefinedVMProperties=macspoof=(true|false)" 
  # service ovirt-engine restart 

In order to deactivate mac spoof filtering on a VM:

1. Bring down the VM
2. edit the VM
3. Click advanced
4. Custom properties
5. Add a key
6. elect macspoof
7. Type true as the value  <<<
8. Start the VM


I did indeed do this. See the attached UI pic from one of the VMs I edited. So 
I take it that this did not properly alter the 'filterref' attribute properly?


-Original Message-
From: Simone Tiraboschi [mailto:stira...@redhat.com] 
Sent: Wednesday, June 08, 2016 5:58 AM
To: Will Dennis
Cc: jvdw...@xs4all.nl; users@ovirt.org; Michal Skrivanek
Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

On Wed, Jun 8, 2016 at 12:00 AM, Will Dennis  wrote:
> 
> 
>   
>   
>   
>   
>   

The issue is simple here ^^^
Please check if and how you configured the macspoof vdsm hook; please follow 
this guide:
 https://github.com/oVirt/vdsm/tree/master/vdsm_hooks/macspoof


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

FYI, here are the results of 'engine-config -g [...]' on my engine VM:

[root@ovirt-engine-01 ~]# sudo engine-config -g UserDefinedVMProperties
UserDefinedVMProperties:  version: 3.0
UserDefinedVMProperties:  version: 3.1
UserDefinedVMProperties:  version: 3.2
UserDefinedVMProperties:  version: 3.3
UserDefinedVMProperties:  version: 3.4
UserDefinedVMProperties:  version: 3.5
UserDefinedVMProperties: macspoof=(true|false) version: 3.6

[root@ovirt-engine-01 ~]# sudo engine-config -g CustomDeviceProperties
CustomDeviceProperties:  version: 3.0
CustomDeviceProperties:  version: 3.1
CustomDeviceProperties:  version: 3.2
CustomDeviceProperties:  version: 3.3
CustomDeviceProperties: 
{type=interface;prop={SecurityGroups=^(?:(?:[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12},
 *)*[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}|)$}} version: 3.4
CustomDeviceProperties: 
{type=interface;prop={SecurityGroups=^(?:(?:[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12},
 *)*[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}|)$}} version: 3.5
CustomDeviceProperties: 
{type=interface;prop={SecurityGroups=^(?:(?:[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12},
 *)*[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}|)$}} version: 3.6


-Original Message-----
From: Will Dennis 
Sent: Wednesday, June 08, 2016 10:14 AM
To: 'Simone Tiraboschi'
Cc: jvdw...@xs4all.nl; users@ovirt.org; Michal Skrivanek
Subject: RE: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

Hi Simone,

These are the instructions I followed, which I found at: 
https://www.ovirt.org/develop/developer-guide/engine/engine-config-examples/


Adding VM custom properties (macspoof)
  # engine-config -s "UserDefinedVMProperties=macspoof=(true|false)" 
  # service ovirt-engine restart 

In order to deactivate mac spoof filtering on a VM:

1. Bring down the VM
2. edit the VM
3. Click advanced
4. Custom properties
5. Add a key
6. elect macspoof
7. Type true as the value  <<<
8. Start the VM


I did indeed do this. See the attached UI pic from one of the VMs I edited. So 
I take it that this did not properly alter the 'filterref' attribute properly?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

No, I didn't see (notice) that step / instructions anywhere... How to do that?

-Original Message-
From: Simone Tiraboschi [mailto:stira...@redhat.com] 
Sent: Wednesday, June 08, 2016 10:23 AM
To: Will Dennis
Cc: jvdw...@xs4all.nl; users@ovirt.org; Michal Skrivanek
Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM


Did you also correctly installed vdsm-hook-macspoof on all of your 
hosted-engine hosts?
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] FW: virt-in-virt problem: DHCP failing for a container in a oVirt VM

[Will Dennis]

Fwd'g to list to continue to thread...

-Original Message-
From: Will Dennis 
Sent: Wednesday, June 08, 2016 11:55 AM
To: 'Simone Tiraboschi'
Subject: RE: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

OK, I did, and... Still not getting DHCP IP's assigned on the containers 
running in the VM...

Again, I see the DHCP packets coming in on the 'vnet0' int on the oVirt 
hypervisor hosts having the VM which has the containers:
[root@ovirt-node-03 ~]# tcpdump -i vnet0 -vvv -s 1500 '(port 67 or port 68)'
11:13:38.689970 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP 
(17), length 328)
0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request 
from 32:33:32:64:30:65 (oui Unknown), length 300, xid 0xd0f70f67, secs 247, 
Flags [none] (0x)
  Client-Ethernet-Address 32:33:32:64:30:65 (oui Unknown)
  Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Discover
Hostname Option 12, length 9: "blitsnorp"
Parameter-Request Option 55, length 13:
  Subnet-Mask, BR, Time-Zone, Default-Gateway
  Domain-Name, Domain-Name-Server, Option 119, Hostname
  Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
  NTP
END Option 255, length 0
PAD Option 0, length 0, occurs 30

But I do not see the MAC from the container in the relevant oVirt bridge that 
the VM is connected to (in this case, port 3 on the bridge):
[root@ovirt-node-03 ~]# brctl showmacs 207-net | grep "^  3"
  3 00:1a:4a:16:01:57   no 0.13 (VM eth0 MAC)
  3 fe:1a:4a:16:01:57   yes0.00 (veth connection from 
VM)
  3 fe:1a:4a:16:01:57   yes0.00 (why are two showing???)
(notice no entry for MAC 32:33:32:64:30:65 from the container on the VM)



-Original Message-
From: Simone Tiraboschi [mailto:stira...@redhat.com] 
Sent: Wednesday, June 08, 2016 10:54 AM
To: Will Dennis
Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
in a oVirt VM

On Wed, Jun 8, 2016 at 4:50 PM, Will Dennis  wrote:
> OK, installed on all hypervisor hosts (and the engine host as well) - do I 
> need to bounce the VMs having containers now, or not?

Yes, of course: you have to reboot all the involved VMs.

> -Original Message-
> From: Simone Tiraboschi [mailto:stira...@redhat.com]
> Sent: Wednesday, June 08, 2016 10:44 AM
> To: Will Dennis
> Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a container 
> in a oVirt VM
>
> On Wed, Jun 8, 2016 at 4:38 PM, Will Dennis  wrote:
>> OK. This goes on just the engine host, or on the hypervisor hosts? (or both?)
>
> On all the hosted-engine hosts
>
>> Also (and not trying to be insulting here), is this documented anywhere? I 
>> didn't run across this dependency in anything I read...
>
> Good question
>
>> -Original Message-
>> From: Simone Tiraboschi [mailto:stira...@redhat.com]
>> Sent: Wednesday, June 08, 2016 10:31 AM
>> To: Will Dennis
>> Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a 
>> container in a oVirt VM
>>
>> On Wed, Jun 8, 2016 at 4:27 PM, Will Dennis  wrote:
>>> No, I didn't see (notice) that step / instructions anywhere... How to do 
>>> that?
>>>
>>> -Original Message-
>>> From: Simone Tiraboschi [mailto:stira...@redhat.com]
>>> Sent: Wednesday, June 08, 2016 10:23 AM
>>> To: Will Dennis
>>> Cc: jvdw...@xs4all.nl; users@ovirt.org; Michal Skrivanek
>>> Subject: Re: [ovirt-users] virt-in-virt problem: DHCP failing for a 
>>> container in a oVirt VM
>>>
>>>
>>> Did you also correctly installed vdsm-hook-macspoof on all of your 
>>> hosted-engine hosts?
>>
>> yum install vdsm-hook-macspoof
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] GlusterFS native client use with oVirt

[Will Dennis]

Hi all,

Can someone tell me if it's possible or not to utilize GlusterFS mounted as 
native (i.e. FUSE) for a storage domain with oVirt 3.5.x?  I have two nodes 
(with a third I'm thinking of using as well) that are running Gluster, and I've 
created the two volumes needed for hosted engine setup ("engine", "data") on 
them, and mounted them native (not via NFS.) Can this be used with oVirt 3.5.x?

Or is this (from what I now understand) a new feature coming in oVirt 3.6?

Thanks,
Will
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not start

[Will Dennis]

[root@ovirt-node-01 ~]# yum list installed | grep engine
ovirt-engine-sdk-python.noarch 3.6.0.3-1.el7.centos   @ovirt-3.6
ovirt-hosted-engine-ha.noarch  1.3.2.1-1.el7.centos   @ovirt-3.6
ovirt-hosted-engine-setup.noarch   1.3.0-1.el7.centos @ovirt-3.6
[root@ovirt-node-01 ~]#
[root@ovirt-node-01 ~]# yum list installed | grep vdsm
vdsm.noarch4.17.10.1-0.el7.centos @ovirt-3.6
vdsm-cli.noarch4.17.10.1-0.el7.centos @ovirt-3.6
vdsm-gluster.noarch4.17.10.1-0.el7.centos @ovirt-3.6
vdsm-infra.noarch  4.17.10.1-0.el7.centos @ovirt-3.6
vdsm-jsonrpc.noarch4.17.10.1-0.el7.centos @ovirt-3.6
vdsm-python.noarch 4.17.10.1-0.el7.centos @ovirt-3.6
vdsm-xmlrpc.noarch 4.17.10.1-0.el7.centos @ovirt-3.6
vdsm-yajsonrpc.noarch  4.17.10.1-0.el7.centos @ovirt-3.6


From: Roman Mohr [mailto:rm...@redhat.com]
Sent: Wednesday, November 25, 2015 5:23 AM
To: Will Dennis
Cc: users
Subject: Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not 
start

Hi Willard,

Which packages versions are you using?

Could you provide the output of the following commands:
# yum list installed | grep engine
# yum list installed | grep vdsm

Best Regards,
Roman
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not start

[Will Dennis]

Hi Simone,

Looks like the vdsm log is empty...

[root@ovirt-node-01 ~]# ls -l /var/log/vdsm/vdsm.log
-rw-r--r-- 1 vdsm kvm 0 Nov 23 15:41 /var/log/vdsm/vdsm.log

Yes, I did execute the ‘vdsm-tool configure –force’ command before the restart, 
looks like it worked OK, but still the same problem with starting vdsm...



From: Simone Tiraboschi [mailto:stira...@redhat.com] 
Sent: Wednesday, November 25, 2015 5:24 AM
To: Will Dennis
Cc: users
Subject: Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not 
start

Could you please attach vdsm logs (/var/log/vdsm/vdsm.log)?

did you tried  manually running 
 vdsm-tool configure --force
before restarting vdsm?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not start

[Will Dennis]

Any clues out of the strace of vdsm?

From: Willard Dennis [mailto:wden...@nec-labs.com]
Sent: Wednesday, November 25, 2015 12:58 PM
To: Simone Tiraboschi
Cc: users
Subject: Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not 
start

strace output:
https://gist.githubusercontent.com/wdennis/d9913d8064af6a6b2c36/raw/6a33f7d9ba7da72ff9be7a15e2e81e5d06ad0fc5/vdsm-strace.txt


On Nov 25, 2015, at 11:29 AM, Simone Tiraboschi 
mailto:stira...@redhat.com>> wrote:



On Wed, Nov 25, 2015 at 5:22 PM, Willard Dennis 
mailto:wden...@nec-labs.com>> wrote:
[root@ovirt-node-01 ~]# sudo -u vdsm /bin/bash
bash-4.2$ /usr/share/vdsm/vdsm
bash-4.2$ echo $?
1

Can you please use strace on it?


On Nov 25, 2015, at 11:16 AM, Simone Tiraboschi 
mailto:stira...@redhat.com>> wrote:


On Wed, Nov 25, 2015 at 5:09 PM, Willard Dennis 
mailto:wden...@nec-labs.com>> wrote:

On Nov 25, 2015, at 10:37 AM, Simone Tiraboschi 
mailto:stira...@redhat.com>> wrote:



On Wed, Nov 25, 2015 at 4:20 PM, Will Dennis 
mailto:wden...@nec-labs.com>> wrote:
Hi Simone,

Looks like the vdsm log is empty...

[root@ovirt-node-01 ~]# ls -l /var/log/vdsm/vdsm.log
-rw-r--r-- 1 vdsm kvm 0 Nov 23 15:41 /var/log/vdsm/vdsm.log

Yes, I did execute the ‘vdsm-tool configure –force’ command before the restart, 
looks like it worked OK, but still the same problem with starting vdsm...


Can you please try manually run /usr/share/vdsm/vdsm as vdsm user?


[root@ovirt-node-01 ~]# su - vdsm
This account is currently not available.
[root@ovirt-node-01 ~]#
[root@ovirt-node-01 ~]# grep vdsm /etc/passwd
vdsm:x:36:36:Node Virtualization Manager:/var/lib/vdsm:/sbin/nologin
[root@ovirt-node-01 ~]# runuser -l vdsm -c '/usr/share/vdsm/vdsm'
This account is currently not available.


[root@tiramd1 ~]# sudo -u vdsm /bin/bash
bash-4.2$ whoami
vdsm




___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not start

[Will Dennis]

I believe sudo is NOT setup for passwordless (then again, didn't see that in 
any pre-req instructions...) Who would be the sudoer in this case?

-Original Message-
From: Fabian Deutsch [mailto:fdeut...@redhat.com] 
Sent: Tuesday, December 01, 2015 12:58 AM
To: Will Dennis
Cc: Simone Tiraboschi; users
Subject: Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not 
start

On Tue, Dec 1, 2015 at 4:52 AM, Will Dennis  wrote:
> Any clues out of the strace of vdsm?

read(9, "sudo: a password is required\n", 4096) = 29

Could it be that sudo is not configured to operate passwordless?

The strat-up can then fail, because sudo requires a ty, but this isn't 
available during service start.

- fabian



> From: Willard Dennis [mailto:wden...@nec-labs.com]
> Sent: Wednesday, November 25, 2015 12:58 PM
> To: Simone Tiraboschi
> Cc: users
> Subject: Re: [ovirt-users] Problem with hosted engine setup - vsdmd 
> does not start
>
>
>
> strace output:
>
> https://gist.githubusercontent.com/wdennis/d9913d8064af6a6b2c36/raw/6a
> 33f7d9ba7da72ff9be7a15e2e81e5d06ad0fc5/vdsm-strace.txt
>
>
>
>
>
> On Nov 25, 2015, at 11:29 AM, Simone Tiraboschi  wrote:
>
>
>
>
>
>
>
> On Wed, Nov 25, 2015 at 5:22 PM, Willard Dennis 
> wrote:
>
> [root@ovirt-node-01 ~]# sudo -u vdsm /bin/bash
>
> bash-4.2$ /usr/share/vdsm/vdsm
>
> bash-4.2$ echo $?
>
> 1
>
>
>
> Can you please use strace on it?
>
>
>
>
>
> On Nov 25, 2015, at 11:16 AM, Simone Tiraboschi  wrote:
>
>
>
>
>
> On Wed, Nov 25, 2015 at 5:09 PM, Willard Dennis 
> wrote:
>
>
>
> On Nov 25, 2015, at 10:37 AM, Simone Tiraboschi  wrote:
>
>
>
>
>
>
>
> On Wed, Nov 25, 2015 at 4:20 PM, Will Dennis  wrote:
>
> Hi Simone,
>
> Looks like the vdsm log is empty...
>
> [root@ovirt-node-01 ~]# ls -l /var/log/vdsm/vdsm.log
> -rw-r--r-- 1 vdsm kvm 0 Nov 23 15:41 /var/log/vdsm/vdsm.log
>
> Yes, I did execute the ‘vdsm-tool configure –force’ command before the 
> restart, looks like it worked OK, but still the same problem with 
> starting vdsm...
>
>
>
>
>
> Can you please try manually run /usr/share/vdsm/vdsm as vdsm user?
>
>
>
>
>
> [root@ovirt-node-01 ~]# su - vdsm
>
> This account is currently not available.
>
> [root@ovirt-node-01 ~]#
>
> [root@ovirt-node-01 ~]# grep vdsm /etc/passwd
>
> vdsm:x:36:36:Node Virtualization Manager:/var/lib/vdsm:/sbin/nologin
>
> [root@ovirt-node-01 ~]# runuser -l vdsm -c '/usr/share/vdsm/vdsm'
>
> This account is currently not available.
>
>
>
>
>
> [root@tiramd1 ~]# sudo -u vdsm /bin/bash
>
> bash-4.2$ whoami
>
> vdsm
>
>
>
>
>
>
>
>
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>



--
Fabian Deutsch 
RHEV Hypervisor
Red Hat
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not start

[Will Dennis]

Ran the configure the first time and got an error...

root@ovirt-node-01 ~> vdsm-tool configure --force

Checking configuration status...

Current revision of multipath.conf detected, preserving
libvirt is not configured for vdsm yet
FAILED: conflicting vdsm and libvirt-qemu tls configuration.
vdsm.conf with ssl=True requires the following changes:
libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1
qemu.conf: spice_tls=1.

Running configure...
Reconfiguration of sebool is done.
Reconfiguration of passwd is done.
Reconfiguration of libvirt is done.

Done configuring modules to VDSM.

So, made the appropriate edits to libvirtd.conf and qemu.conf

root@ovirt-node-01 ~> grep "^[a-z]" /etc/libvirt/libvirtd.conf
listen_tls = 1
listen_tcp = 0
auth_tcp = "sasl"
auth_unix_rw="sasl"
ca_file="/etc/pki/vdsm/certs/cacert.pem"
cert_file="/etc/pki/vdsm/certs/vdsmcert.pem"
host_uuid="2b026fff-bae2-429f-9386-c60ffc5f3f32"
keepalive_interval=-1
key_file="/etc/pki/vdsm/keys/vdsmkey.pem"
unix_sock_group="qemu"
unix_sock_rw_perms="0770"

root@ovirt-node-01 ~> grep "^[a-z]" /etc/libvirt/qemu.conf
spice_tls = 1
auto_dump_path="/var/log/core"
dynamic_ownership=0
lock_manager="sanlock"
remote_display_port_max=6923
remote_display_port_min=5900
save_image_format="lzop"
spice_tls=1
spice_tls_x509_cert_dir="/etc/pki/vdsm/libvirt-spice"

Then re-ran the config...

root@ovirt-node-01 ~> vdsm-tool configure --force

Checking configuration status...

Current revision of multipath.conf detected, preserving
libvirt is already configured for vdsm
SUCCESS: ssl configured to true. No conflicts

Running configure...
Reconfiguration of sebool is done.
Reconfiguration of libvirt is done.

Done configuring modules to VDSM.

Sadly, vdsmd STILL fails to start...

root@ovirt-node-01 ~> systemctl restart vdsmd
root@ovirt-node-01 ~> systemctl status vdsmd
vdsmd.service - Virtual Desktop Server Manager
   Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled)
   Active: activating (auto-restart) (Result: exit-code) since Wed 2015-12-02 
10:30:47 EST; 1s ago
  Process: 6522 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh --post-stop 
(code=exited, status=0/SUCCESS)
  Process: 6506 ExecStart=/usr/share/vdsm/daemonAdapter -0 /dev/null -1 
/dev/null -2 /dev/null /usr/share/vdsm/vdsm (code=exited, status=1/FAILURE)
  Process: 6433 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start 
(code=exited, status=0/SUCCESS)
Main PID: 6506 (code=exited, status=1/FAILURE)

Dec 02 10:30:47 ovirt-node-01 systemd[1]: Unit vdsmd.service entered failed 
Dec 02 10:30:47 ovirt-node-01 systemd[1]: vdsmd.service holdoff time over, s
Hint: Some lines were ellipsized, use -l to show in full.

:(

From: Simone Tiraboschi [mailto:stira...@redhat.com]
Sent: Wednesday, December 02, 2015 3:57 AM
To: Will Dennis
Cc: Fabian Deutsch; users
Subject: Re: [ovirt-users] Problem with hosted engine setup - vsdmd does not 
start

[snip]

Now can you please now configure it with:
  vdsm-tool configure --force

Then you have to restart it


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[Users] New user to oVirt, and I haz a sad so far...

[Will Dennis (Live.com)]

Hi all, ready for a story? (well, more of a rant, but hopefully it will be a
good UX tale, and may even be entertaining.)

Had one of the groups come to me at work this week and request a OpenStack
setup. When I sat down and discussed their needs, it turns out that they
really only need a multi-hypervisor setup where they can spin up VMs for
their research projects. The VMs should be fairly long-lived, and will have
persistent storage. Their other request is that the storage should be local
on the hypervisor nodes (they plan to use Intel servers with 8-10 2TB drives
for VM storage on each node.) They desire this in order to keep the VM I/O
local - they do not have a SAN of any sort anyhow, and they do not care
about live migration, etc.

In any case, knowing that they did not want to afford a VMware setup (which
is what I'm used to using), I proposed using oVirt to fill their needs,
having heard and read up on it a bit (It's "open-source VMware", right?)
even though I had not used it before (I have however made single-node KVM
hypervisors for their group before, utilizing Open vSwitch, libvirt,
virt-manager etc., so I'm not completely ignorant of KVM/libvirt etc.)

In any case, I took one of their older servers which was already running
CentOS 6.5, installed the requisite packages on it, and in short order had
an engine server up and running (oVirt 3.3.2). That seems to have been the
easy part :-/  Now came the installation of a hypervisor node. I downloaded
and burned an ISO of the latest oVirt node installer
(ovirt-node-iso-3.0.3-1.1.vdsm.fc19.iso) and tried to install it on one of
their target Intel servers. On the 1st try I got to the end of the setup
TUI, invoked the Install link, and was promptly thrown an error (sorry, but
forgot what it was, something like "press X for a command prompt, or
Reboot".) No problem, I rebooted, selected booting off the CD again, waited
until the TUI came up, and when I tried to move past the first screen, it
threw me out to a login prompt. OK, enough of that (the server takes a long
time to reboot, and then boot off the CD) - I then thought I would try it on
a VMware Workstation VM (yes, I get the irony, but VMware wkstn can handle
nested virt, so it's a great testbed platform for OpenStack, etc.) because
that would install a heck of a lot faster. That went a lot better - got the
oVirt node 3.0.3 installed on the first try.

More pain was soon to follow, however.  I logged in and started configuring
the node. The TUI was easy enough - much like an ESXi node ;)  I set the NIC
to IPv4 static, entered in the correct IP info, registered a DNS name for
the IP I had assigned, and then tested pinging the engine, all was good. I
then moved on to the section where you define the engine. I entered in the
FQDN of the engine, verified the key fingerprint, and clicked the "Save and
Register" link at the bottom. That seemed to work, so I completed the rest
of the TUI, and then looked at the oVirt engine web UI. There was my new
node, ready for authorization. I clicked the link to authorize it, and after
a while, the UI came back with "Install Failed" status. Hmmm. So I went back
to the node's TUI, and now some of the screens said that the IP addr was
unconfigured? I went then to the Network screen, and sure enough, the NIC at
the bottom showed "Unconfigured". WTF? So I went and entered in the correct
info back in the IPv4 section, and then arrowed down to the Save link and
clicked it - and the next screen said something like "No info needing
changes, nothing to do." Wh? Went back to the network setup screen, NIC
still showing "Unconfigured" even though the IPv4 info still was there. I
did a ping test at this point from the Ping link on the network setup page,
and what do you know - I could still ping IP's (the engine, the default gw,
etc.) But as I moved around the TUI, other screens still said that the
network was unconfigured. Went back to the Web UI of the engine, put the
host in Maint, then tried to Activate it, still no go - Install Failed. Even
though I had configured the node to allow remote access and set a password,
and also verified via nmap that TCP port 22 on the node was indeed
listening, when I tried to SSH into the node as admin, I immediately got a
"connection closed" message, so that failed as well. Went back to the node's
network setup page, set the IPv4 to "Disabled", saved it, then went back and
set it back to "Static" then re-entered the IPv4 info. Clicked the Save
link, it went thru the setup again, came back with a success, verified with
ping etc. that networking was working on the node. The engine web UI still
said that it could not connect to the node however. So I put the node in
Maint, and then removed it.  I went back to the node, went to the Engine
setup page, and re-did the screen to define the engine on the node. I notice
that after I did this, however, that the node screens went back to saying
that the network was unconfigured. Grr. But the nod

Re: [Users] New user to oVirt, and I haz a sad so far...

[Will Dennis (Live.com)]

Thanks, Joop, for the node platform best practices. I did turn selinux from
"enforcing" to "permissive", and then when I tried to ping the engine by
fqdn, I saw that DNS lookups were failing (even tho resolv.conf looked
correct.) Did a 'yum remove NetworkManager" and then fixed the
/etc/sysconfig/network-scripts/ifcfg-, and then after a reboot, I can
now see the info for the node in the WUI on the manager, although the status
for the node is still "Non Operational". 

 

Where can I find the node install log (on the engine or the node, and name?)
(Sorry for noob status, but I am a quick learner ;)

 

Thanks,

Will

 

From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of
noc
Sent: Friday, January 17, 2014 3:35 AM
To: users@ovirt.org
Subject: Re: [Users] New user to oVirt, and I haz a sad so far...

 

On 17-1-2014 8:53, Gabi C wrote:

've been there! :-D

I mean exactly same issuse you had on Centos, I had on Fedora 19.

Did you disable selinux on nodes? 'cause that's what is causing SSh
connection closing

My setup:

1 engine on vmware  - fedora 19, up-to-date

 

 

2 nodes on IBM x series 3650  - fedora 19 based -oVirt Node - 3.0.3 -
1.1.fc19 with nodes beig in glusterfs cluster also.



Right now, I'm banging my head against "Operation Add-Disk failed to
complete." , message I have got after adding a new virtual machine and try
to addd its disk

 

On Fri, Jan 17, 2014 at 6:08 AM, Will Dennis (Live.com)
 wrote:

Hi all, ready for a story? (well, more of a rant, but hopefully it will be a
good UX tale, and may even be entertaining.)

Had one of the groups come to me at work this week and request a OpenStack
setup. When I sat down and discussed their needs, it turns out that they
really only need a multi-hypervisor setup where they can spin up VMs for
their research projects. The VMs should be fairly long-lived, and will have
persistent storage. Their other request is that the storage should be local
on the hypervisor nodes (they plan to use Intel servers with 8-10 2TB drives
for VM storage on each node.) They desire this in order to keep the VM I/O
local - they do not have a SAN of any sort anyhow, and they do not care
about live migration, etc

@Will
If the installation ends, either with or without error, it will give you a
log location. Upload the log to a paste.bin and mail the link.

@Gabi,
There should be more info in either the vdsm.log on the SPM server or in the
engine.log on the engine server, see above for lettings us know what the
error is.

Having installed oVirt, probably dozens of times, I have some guidelines:
- temporarily disable firewalld/iptables (if all works, enable should still
work, scripts with rules are generated and location is given)
- make selinux permissive either via setenforce 0 (until next boot) or via
/etc/selinux/config ( survives reboots), dont disable it!
- make sure fqdn work in both directions between engine and host(s) (either
using /etc/hosts or DNS)
- make sure NetworkManager is disabled and network enabled

Joop

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [Users] New user to oVirt, and I haz a sad so far...

[Will Dennis (Live.com)]

Docs should be updated with a warning then as to the "alpha" status of the 
ovirt-node product... It was a huge time-sink for me.

- Will

-Original Message-
From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of 
Sven Kieske
Sent: Friday, January 17, 2014 6:05 AM
To: users@ovirt.org; users@ovirt.org
Subject: Re: [Users] New user to oVirt, and I haz a sad so far...

Hi,

my advice would be the following:

for "nodes" use either a "full"
minimal centos or fedora and install ovirt.org repo on top of that.

then everything works.

the "node.isos" provided by ovirt are in pre-alpha state to be honest.

sure you still have to disable selinux
(or better configure it correctly).

But you have no problems with configurations which are not persisted and as a 
bonus you can install $your_favourite_tool on the node, such as tcpdump nmap 
etc. for debugging.

Again: I would strongly advise against useing the node.isos from ovirt.org at 
the moment.

There is just too much stuff not working or bugged, even simple things like 
network configuration.

HTH

--
Mit freundlichen Grüßen / Regards

Sven Kieske

Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen 
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [Users] New user to oVirt, and I haz a sad so far...

[Will Dennis (Live.com)]

OK, I will open a bug; trying now to bring up the node using CentOS 6.5
minimal, will keep good notes this time on what I have to do to get it
connected to the engine.

Thanks,
Will

-Original Message-
From: Itamar Heim [mailto:ih...@redhat.com] 
Sent: Friday, January 17, 2014 3:34 PM
To: Willard Dennis; Einav Cohen
Cc: users@ovirt.org
Subject: Re: [Users] New user to oVirt, and I haz a sad so far...

On 01/17/2014 07:31 PM, Willard Dennis wrote:
> Uuup!  :)
>
> Thanks for your assistance, Einav!
>
> Now onwards

Hi Will,

this should work out of the box.
would help if you can open a bug on this not happening so it will be
verified it does work out of the box.

Thanks,
Itamar

>
>  > Date: Fri, 17 Jan 2014 12:18:30 -0500  > From: eco...@redhat.com  > 
> To: willardden...@live.com  > CC: n...@nieuwland.nl; users@ovirt.org  > 
> Subject: Re: [Users] New user to oVirt, and I haz a sad so far...
>  >
>  > go to the "Network Interface" sub-tab of that node and click the 
> "Setup  > Host Networks" button.
>  > In the dialog that has opened, attach the 'ovirtmgmt' network from 
> the  > right-hand side of the dialog to one of your node's NICs on the 
> left-hand  > side by dragging that network onto the NIC.
>  > click OK.
>  > after a short while, your Host should become 'Up'.
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users