Re: [ovirt-users] oVirt management has lost its SSL.

[Fred Rolland]

Hi,
Can you send the engine log around the time you tried to import the export
domain?
Thanks

On Fri, Nov 3, 2017 at 9:47 PM, ~Stack~  wrote:

> On 11/03/2017 01:17 PM, ~Stack~ wrote:
> > On 11/03/2017 12:48 PM, Alexander Wels wrote:
>
> >> But if all else fails you should be able to create a fresh engine, and
> after
> >> you have added a host, you should be able to import the existing storage
> >> domain (like you noted the VMs are still there).
> >>
> >>
> > Greetings,
> > Thanks, but I've tried that too. Even though it did delete the keystore,
> > I ended up with the exact same error. :-(
> >
> > I'm doing a fresh install right now. I've never done an import like this
> > before. I just connect the fresh install to one of my hosts and I can
> > import the others hosts/vms/configurations?
> >
> > Thanks!
> > ~Stack~
> >
> >
>
> Bender: Are we boned?
> Leela: Yeah, we're boned
>
>
> So I built a new management host from scratch. I added one of my hosts,
> and immediately crashed the vm's running on that hypervisor (they all
> just stopped responding). I don't know why they didn't fail over, but
> they didn't. Oh well. At least the other hypervisor is up!
>
> So I tried following this guide to import my storage domain from the
> section "Disaster Recovery flows" for "Import file Storage Domain".
>
> https://www.ovirt.org/develop/release-management/features/
> storage/importstoragedomain/
>
> Yeah. That didn't work. It says it can't find any other domains to
> import, but if I attempt to create a new one it says it can't because
> there are existing domains!
>
> Well, while I was poking at it the other VM's started acting up (crazy
> high latency and the ovirt logs were really pissed at me). So I shut off
> the ones that still responded, then shut down the other hypervisor. I
> backed up the VM's on my NFS share, and created a new directory for the
> data domain.
>
> Guess I'm rebuilding my environment from scratch. I just hope I can get
> some of the VM's to come back some how. :-/
>
> ~Stack~
>
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] oVirt management has lost its SSL.

[~Stack~]

On 11/03/2017 01:17 PM, ~Stack~ wrote:
> On 11/03/2017 12:48 PM, Alexander Wels wrote:

>> But if all else fails you should be able to create a fresh engine, and after 
>> you have added a host, you should be able to import the existing storage 
>> domain (like you noted the VMs are still there).
>>
>>
> Greetings,
> Thanks, but I've tried that too. Even though it did delete the keystore,
> I ended up with the exact same error. :-(
> 
> I'm doing a fresh install right now. I've never done an import like this
> before. I just connect the fresh install to one of my hosts and I can
> import the others hosts/vms/configurations?
> 
> Thanks!
> ~Stack~
> 
> 

Bender: Are we boned?
Leela: Yeah, we're boned


So I built a new management host from scratch. I added one of my hosts,
and immediately crashed the vm's running on that hypervisor (they all
just stopped responding). I don't know why they didn't fail over, but
they didn't. Oh well. At least the other hypervisor is up!

So I tried following this guide to import my storage domain from the
section "Disaster Recovery flows" for "Import file Storage Domain".

https://www.ovirt.org/develop/release-management/features/storage/importstoragedomain/

Yeah. That didn't work. It says it can't find any other domains to
import, but if I attempt to create a new one it says it can't because
there are existing domains!

Well, while I was poking at it the other VM's started acting up (crazy
high latency and the ovirt logs were really pissed at me). So I shut off
the ones that still responded, then shut down the other hypervisor. I
backed up the VM's on my NFS share, and created a new directory for the
data domain.

Guess I'm rebuilding my environment from scratch. I just hope I can get
some of the VM's to come back some how. :-/

~Stack~




signature.asc
Description: OpenPGP digital signature
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] oVirt management has lost its SSL.

[~Stack~]

On 11/03/2017 12:48 PM, Alexander Wels wrote:
> 
> AFAIC engine-setup will create the files needed. Try running engine-cleanup 
> and maybe it will remove everything needed and then running engine-setup 
> again?
> 
> But if all else fails you should be able to create a fresh engine, and after 
> you have added a host, you should be able to import the existing storage 
> domain (like you noted the VMs are still there).
> 
> 
Greetings,
Thanks, but I've tried that too. Even though it did delete the keystore,
I ended up with the exact same error. :-(

I'm doing a fresh install right now. I've never done an import like this
before. I just connect the fresh install to one of my hosts and I can
import the others hosts/vms/configurations?

Thanks!
~Stack~




signature.asc
Description: OpenPGP digital signature
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] oVirt management has lost its SSL.

[Alexander Wels]

On Friday, November 3, 2017 1:41:00 PM EDT ~Stack~ wrote:
> On 11/03/2017 12:23 PM, Alexander Wels wrote:
> > On Friday, November 3, 2017 1:15:27 PM EDT ~Stack~ wrote:
> >> Greetings,
> >> 
> >> I'm seriously just grasping at straws here. I took a spare hard drive,
> >> tossed it in the management host, and did a fresh install. It did not
> >> like me trying to add it into the existing infrastructure. Tried to dump
> >> the DB from the old to the new, update the passwords, and pretty much
> >> ended up in the same place.
> >> 
> >> I did check the .trustedkeystore and it has the same 1 key as my
> >> original back up. So that isn't the issue.
> >> 
> >> Still poking at it. Would love some thoughts/feedback.
> >> 
> >> Thanks!
> >> ~Stack~
> > 
> > Running engine-setup on the engine machine should re-generate the keys.
> 
> Thanks for the suggestion. I've tried that. Twice. Still the same error.
> 
> "Keystore was tampered with, or password was incorrect."
> 
> From digging around in the logs, it looks like it is trying to access
> /etc/pki/ovirt-engine/.trustedstore but the password found in the ovirt
> configs works just fine. So I know it is not a password issue.
> 
> I've been trying to figure out how that file is created so I can
> possibly generate a new one, but no luck so far.
> 
> Thanks!
> ~Stack~

AFAIC engine-setup will create the files needed. Try running engine-cleanup 
and maybe it will remove everything needed and then running engine-setup 
again?

But if all else fails you should be able to create a fresh engine, and after 
you have added a host, you should be able to import the existing storage 
domain (like you noted the VMs are still there).

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] oVirt management has lost its SSL.

[~Stack~]

On 11/03/2017 12:23 PM, Alexander Wels wrote:
> On Friday, November 3, 2017 1:15:27 PM EDT ~Stack~ wrote:
>> Greetings,
>>
>> I'm seriously just grasping at straws here. I took a spare hard drive,
>> tossed it in the management host, and did a fresh install. It did not
>> like me trying to add it into the existing infrastructure. Tried to dump
>> the DB from the old to the new, update the passwords, and pretty much
>> ended up in the same place.
>>
>> I did check the .trustedkeystore and it has the same 1 key as my
>> original back up. So that isn't the issue.
>>
>> Still poking at it. Would love some thoughts/feedback.
>>
>> Thanks!
>> ~Stack~
>>
> 
> Running engine-setup on the engine machine should re-generate the keys.

Thanks for the suggestion. I've tried that. Twice. Still the same error.

"Keystore was tampered with, or password was incorrect."

From digging around in the logs, it looks like it is trying to access
/etc/pki/ovirt-engine/.trustedstore but the password found in the ovirt
configs works just fine. So I know it is not a password issue.

I've been trying to figure out how that file is created so I can
possibly generate a new one, but no luck so far.

Thanks!
~Stack~




signature.asc
Description: OpenPGP digital signature
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] oVirt management has lost its SSL.

[Alexander Wels]

On Friday, November 3, 2017 1:15:27 PM EDT ~Stack~ wrote:
> Greetings,
> 
> I'm seriously just grasping at straws here. I took a spare hard drive,
> tossed it in the management host, and did a fresh install. It did not
> like me trying to add it into the existing infrastructure. Tried to dump
> the DB from the old to the new, update the passwords, and pretty much
> ended up in the same place.
> 
> I did check the .trustedkeystore and it has the same 1 key as my
> original back up. So that isn't the issue.
> 
> Still poking at it. Would love some thoughts/feedback.
> 
> Thanks!
> ~Stack~
> 

Running engine-setup on the engine machine should re-generate the keys.

> On 11/03/2017 09:30 AM, ~Stack~ wrote:
> > Greetings,
> > 
> > Please, I would greatly appreciate some help/feedback. I'm not sure what
> > else to do.
> > 
> > I reverted the .trustedstore to the only backup I have, and there is one
> > key in it. That too gets flagged by oVirt as having been tampered with
> > (I'm guessing oVirt added something that isn't there any more). The
> > password is correct as I can verify it from the oVirt config file on the
> > command line.
> > 
> > I'm out of ideas on fixing this. What happens to my oVirt hypervisors
> > and VM's if I rebuild the management engine host from scratch?
> > 
> > Thanks!
> > ~Stack~
> > 
> > On 11/02/2017 04:18 PM, ~Stack~ wrote:
> >> Greetings,
> >> 
> >> OS: Scientific Linux 7.4
> >> oVirt: 4.1
> >> Everything fully updated.
> >> 
> >> Everything was working great. I received my new network card today to
> >> upgrade my ovirt management node (physical node; not self-hosted), took
> >> the machine down, swapped the card, and brought it up to many many
> >> errors.
> >> 
> >> Here's the basic break-down of my discoveries.
> >> 
> >> 1) My /etc/pki/ovirt-engine/.trustedstore was corrupt. I had lots of
> >> messages in my engine.log about it being corrupt. Restored from backup,
> >> and oVirt engine was really peeved for not having my domain cert in it
> >> (tons of messages in the engine.log file)...figured out how to add my
> >> domain cert and it seemed OK. Which led me to...
> >> 
> >> 2) My /etc/pki/ovirt-engine/keys/engine.p12 and
> >> /etc/pki/ovirt-engine/keys/apache.p12 are _gone_. Don't have them in my
> >> backups either. This results in a massive java dump when I try to start
> >> the engine service.
> >> 
> >> 3) I noticed that I had
> >> /etc/pki/ovirt-engine/keys/engine.p12.201711021302 which is a time stamp
> >> corresponding to when I shut the node down. Then I noticed, that I was
> >> missing dang near EVERY file in /etc/pki/ovirt-engine but I had an
> >> equivalent file with the ".201711021302" extension. So a touch of bash
> >> and I copied all of my "*.201711021302" files with the proper
> >> user/group/permissions into their base name. Hooray! No more errors in
> >> the log files and all services start!!
> >> 
> >> 4) I open my web browser and head to my management host...and I get this
> >> error:
> >> Keystore was tampered with, or password was incorrect
> >> 
> >> Well...yeah. I had to fix it in step one. :-/
> >> 
> >> I'm not getting anything useful out of my Internet searching. I don't
> >> know what went wrong or why, but my SSL is just borked.
> >> 
> >> Any suggestions? Thoughts? Ideas?
> >> 
> >> Is there a way to just blow away and start over with the SSL _without_
> >> destroying the VM's (which fortunately they all seem to still be
> >> functional!)?
> >> 
> >> Any help would be greatly appreciated.
> >> Thanks!
> >> ~Stack~


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] oVirt management has lost its SSL.

[~Stack~]

Greetings,

I'm seriously just grasping at straws here. I took a spare hard drive,
tossed it in the management host, and did a fresh install. It did not
like me trying to add it into the existing infrastructure. Tried to dump
the DB from the old to the new, update the passwords, and pretty much
ended up in the same place.

I did check the .trustedkeystore and it has the same 1 key as my
original back up. So that isn't the issue.

Still poking at it. Would love some thoughts/feedback.

Thanks!
~Stack~



On 11/03/2017 09:30 AM, ~Stack~ wrote:
> Greetings,
> 
> Please, I would greatly appreciate some help/feedback. I'm not sure what
> else to do.
> 
> I reverted the .trustedstore to the only backup I have, and there is one
> key in it. That too gets flagged by oVirt as having been tampered with
> (I'm guessing oVirt added something that isn't there any more). The
> password is correct as I can verify it from the oVirt config file on the
> command line.
> 
> I'm out of ideas on fixing this. What happens to my oVirt hypervisors
> and VM's if I rebuild the management engine host from scratch?
> 
> Thanks!
> ~Stack~
> On 11/02/2017 04:18 PM, ~Stack~ wrote:
>> Greetings,
>>
>> OS: Scientific Linux 7.4
>> oVirt: 4.1
>> Everything fully updated.
>>
>> Everything was working great. I received my new network card today to
>> upgrade my ovirt management node (physical node; not self-hosted), took
>> the machine down, swapped the card, and brought it up to many many errors.
>>
>> Here's the basic break-down of my discoveries.
>>
>> 1) My /etc/pki/ovirt-engine/.trustedstore was corrupt. I had lots of
>> messages in my engine.log about it being corrupt. Restored from backup,
>> and oVirt engine was really peeved for not having my domain cert in it
>> (tons of messages in the engine.log file)...figured out how to add my
>> domain cert and it seemed OK. Which led me to...
>>
>> 2) My /etc/pki/ovirt-engine/keys/engine.p12 and
>> /etc/pki/ovirt-engine/keys/apache.p12 are _gone_. Don't have them in my
>> backups either. This results in a massive java dump when I try to start
>> the engine service.
>>
>> 3) I noticed that I had
>> /etc/pki/ovirt-engine/keys/engine.p12.201711021302 which is a time stamp
>> corresponding to when I shut the node down. Then I noticed, that I was
>> missing dang near EVERY file in /etc/pki/ovirt-engine but I had an
>> equivalent file with the ".201711021302" extension. So a touch of bash
>> and I copied all of my "*.201711021302" files with the proper
>> user/group/permissions into their base name. Hooray! No more errors in
>> the log files and all services start!!
>>
>> 4) I open my web browser and head to my management host...and I get this
>> error:
>> Keystore was tampered with, or password was incorrect
>>
>> Well...yeah. I had to fix it in step one. :-/
>>
>> I'm not getting anything useful out of my Internet searching. I don't
>> know what went wrong or why, but my SSL is just borked.
>>
>> Any suggestions? Thoughts? Ideas?
>>
>> Is there a way to just blow away and start over with the SSL _without_
>> destroying the VM's (which fortunately they all seem to still be
>> functional!)?
>>
>> Any help would be greatly appreciated.
>> Thanks!
>> ~Stack~
>>
>>
> 
> 
> 





signature.asc
Description: OpenPGP digital signature
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] oVirt management has lost its SSL.

[~Stack~]

Greetings,

Please, I would greatly appreciate some help/feedback. I'm not sure what
else to do.

I reverted the .trustedstore to the only backup I have, and there is one
key in it. That too gets flagged by oVirt as having been tampered with
(I'm guessing oVirt added something that isn't there any more). The
password is correct as I can verify it from the oVirt config file on the
command line.

I'm out of ideas on fixing this. What happens to my oVirt hypervisors
and VM's if I rebuild the management engine host from scratch?

Thanks!
~Stack~
On 11/02/2017 04:18 PM, ~Stack~ wrote:
> Greetings,
> 
> OS: Scientific Linux 7.4
> oVirt: 4.1
> Everything fully updated.
> 
> Everything was working great. I received my new network card today to
> upgrade my ovirt management node (physical node; not self-hosted), took
> the machine down, swapped the card, and brought it up to many many errors.
> 
> Here's the basic break-down of my discoveries.
> 
> 1) My /etc/pki/ovirt-engine/.trustedstore was corrupt. I had lots of
> messages in my engine.log about it being corrupt. Restored from backup,
> and oVirt engine was really peeved for not having my domain cert in it
> (tons of messages in the engine.log file)...figured out how to add my
> domain cert and it seemed OK. Which led me to...
> 
> 2) My /etc/pki/ovirt-engine/keys/engine.p12 and
> /etc/pki/ovirt-engine/keys/apache.p12 are _gone_. Don't have them in my
> backups either. This results in a massive java dump when I try to start
> the engine service.
> 
> 3) I noticed that I had
> /etc/pki/ovirt-engine/keys/engine.p12.201711021302 which is a time stamp
> corresponding to when I shut the node down. Then I noticed, that I was
> missing dang near EVERY file in /etc/pki/ovirt-engine but I had an
> equivalent file with the ".201711021302" extension. So a touch of bash
> and I copied all of my "*.201711021302" files with the proper
> user/group/permissions into their base name. Hooray! No more errors in
> the log files and all services start!!
> 
> 4) I open my web browser and head to my management host...and I get this
> error:
> Keystore was tampered with, or password was incorrect
> 
> Well...yeah. I had to fix it in step one. :-/
> 
> I'm not getting anything useful out of my Internet searching. I don't
> know what went wrong or why, but my SSL is just borked.
> 
> Any suggestions? Thoughts? Ideas?
> 
> Is there a way to just blow away and start over with the SSL _without_
> destroying the VM's (which fortunately they all seem to still be
> functional!)?
> 
> Any help would be greatly appreciated.
> Thanks!
> ~Stack~
> 
> 





signature.asc
Description: OpenPGP digital signature
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] oVirt management has lost its SSL.

[~Stack~]

Greetings,

OS: Scientific Linux 7.4
oVirt: 4.1
Everything fully updated.

Everything was working great. I received my new network card today to
upgrade my ovirt management node (physical node; not self-hosted), took
the machine down, swapped the card, and brought it up to many many errors.

Here's the basic break-down of my discoveries.

1) My /etc/pki/ovirt-engine/.trustedstore was corrupt. I had lots of
messages in my engine.log about it being corrupt. Restored from backup,
and oVirt engine was really peeved for not having my domain cert in it
(tons of messages in the engine.log file)...figured out how to add my
domain cert and it seemed OK. Which led me to...

2) My /etc/pki/ovirt-engine/keys/engine.p12 and
/etc/pki/ovirt-engine/keys/apache.p12 are _gone_. Don't have them in my
backups either. This results in a massive java dump when I try to start
the engine service.

3) I noticed that I had
/etc/pki/ovirt-engine/keys/engine.p12.201711021302 which is a time stamp
corresponding to when I shut the node down. Then I noticed, that I was
missing dang near EVERY file in /etc/pki/ovirt-engine but I had an
equivalent file with the ".201711021302" extension. So a touch of bash
and I copied all of my "*.201711021302" files with the proper
user/group/permissions into their base name. Hooray! No more errors in
the log files and all services start!!

4) I open my web browser and head to my management host...and I get this
error:
Keystore was tampered with, or password was incorrect

Well...yeah. I had to fix it in step one. :-/

I'm not getting anything useful out of my Internet searching. I don't
know what went wrong or why, but my SSL is just borked.

Any suggestions? Thoughts? Ideas?

Is there a way to just blow away and start over with the SSL _without_
destroying the VM's (which fortunately they all seem to still be
functional!)?

Any help would be greatly appreciated.
Thanks!
~Stack~




signature.asc
Description: OpenPGP digital signature
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users