Re: [ovirt-users] Ovirt system reboot no symlink to storage

[David Gossage]

On Sat, Jun 4, 2016 at 11:02 AM, Nir Soffer  wrote:

> On Sat, Jun 4, 2016 at 5:17 PM, David Gossage
>  wrote:
> > This morning I updated my gluster version to 3.7.11 and during this I
> > shutdown ovirt completely and all VM's.  On bringing them back up ovirt
> > seems to have not recreated symlinks to gluster storage
> >
> > Before VM's were created and the path they expected to find the storage
> at
> > as it was created by oVirt was
> >
> >
> /rhev/data-center/0001-0001-0001-0001-00b8/7c73a8dd-a72e-4556-ac88-7f6813131e64
> >
> > which was a symlink to what was mounted at
> > /rhev/data-center/mnt/glusterSD/ccgl1.gl.local:GLUSTER1 once engine
> > activated hosts
> >
> > It never seemed to recreate that symlink though and I ended up just
> doing so
> > manually and after that I could bring up my VM's.
> >
> > If this was caused by some error would I likely find that in the engine
> > log's on the engine vm or on one of the vdsm logs of a host?
> >
> >
> > I'm running oVirt Engine Version: 3.6.1.3-1.el7.centos
>
> Sounds like https://bugzilla.redhat.com/show_bug.cgi?id=1271771
>
> The fix should be available in 3.6.3.
>
> As a general rule, you should run the latest 3.6 version, running
> 3.6.1 is too risky,
> almost as running 3.6.0 :-)
>

I admit it.  I have been lazy.  Been trying to catch up on my updates for
the cluster and just forced myself to come in Saturday morning and do it
starting from storage end.  I'll plan out updating ovirt itself one of
these mornings coming up and see how that goes which I am sure will fix it.

Thanks for pointing the bug report out to me.



> Nir
>
> >
> >
> >
> >
> > David Gossage
> > Carousel Checks Inc. | System Administrator
> > Office 708.613.2284
> >
> > ___
> > Users mailing list
> > Users@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users
> >
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Ovirt system reboot no symlink to storage

[David Gossage]

This morning I updated my gluster version to 3.7.11 and during this I
shutdown ovirt completely and all VM's.  On bringing them back up ovirt
seems to have not recreated symlinks to gluster storage

Before VM's were created and the path they expected to find the storage at
as it was created by oVirt was

/rhev/data-center/0001-0001-0001-0001-00b8/7c73a8dd-a72e-4556-ac88-7f6813131e64

which was a symlink to what was mounted
at /rhev/data-center/mnt/glusterSD/ccgl1.gl.local:GLUSTER1 once engine
activated hosts

It never seemed to recreate that symlink though and I ended up just doing
so manually and after that I could bring up my VM's.

If this was caused by some error would I likely find that in the engine
log's on the engine vm or on one of the vdsm logs of a host?


I'm running oVirt Engine Version: 3.6.1.3-1.el7.centos




*David Gossage*
*Carousel Checks Inc. | System Administrator*
*Office* 708.613.2284
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Documentation: Helper Utilities missing references

[gregor]

Hi,

I found the article to some "Helper Utilities" on the ovirt page [1] but
I can't find the mentioned helpers fkvalidaor.sh, taskcleaner.sh and
unlock_entity.sh.

Does anybody where these are? Especially I need the taskcleaner.sh
because a task is stuck since many days.

cheers
gregor

[1] https://www.ovirt.org/develop/developer-guide/db-issues/helperutilities/
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Sanlock add Lockspace Errors

[InterNetX - Juergen Gotteswinter]

Am 6/3/2016 um 6:37 PM schrieb Nir Soffer:
> On Fri, Jun 3, 2016 at 11:27 AM, InterNetX - Juergen Gotteswinter
>  wrote:
>> What if we move all vm off the lun which causes this error, drop the lun
>> and recreated it. Will we "migrate" the error with the VM to a different
>> lun or could this be a fix?
> 
> This should will fix the ids file, but since we don't know why this corruption
> happened, it may happen again.
>

i am pretty sure to know when / why this happend, after a major outage
with engine gone crazy in fencing hosts + crash / hard reset of the san
this messages occoured the first time.

but i can provide a log package, no problem


> Please open a bug with the log I requested so we can investigate this issue.
> 
> To fix the ids file you don't have to recreate the lun, just
> initialize the ids lv.
> 
> 1. Put the domain to maintenance (via engine)
> 
> No host should access it while you reconstruct the ids file
> 
> 2. Activate the ids lv
> 
> You may need to connect to this iscsi target first, unless you have other
> vgs connected on the same target.
> 
> lvchange -ay sd_uuid/ids
> 
> 3. Initialize the lockspace
> 
> sanlock direct init -s :0:/dev//ids:0
> 
> 4. Deactivate the ids lv
> 
> lvchange -an sd_uuid/ids
> 
> 6. Activate the domain (via engine)
> 
> The domain should become active after a while.
> 

oh, this is great, going to announce an maintance window. Thanks a lot,
this already started to drive me crazy. Will Report after we did this!

> Nir
> 
>>
>> Am 6/3/2016 um 10:08 AM schrieb InterNetX - Juergen Gotteswinter:
>>> Hello David,
>>>
>>> thanks for your explanation of those messages, is there any possibility
>>> to get rid of this? i already figured out that it might be an corruption
>>> of the ids file, but i didnt find anything about re-creating or other
>>> solutions to fix this.
>>>
>>> Imho this occoured after an outage where several hosts, and the iscsi
>>> SAN has been fenced and/or rebooted.
>>>
>>> Thanks,
>>>
>>> Juergen
>>>
>>>
>>> Am 6/2/2016 um 6:03 PM schrieb David Teigland:
 On Thu, Jun 02, 2016 at 06:47:37PM +0300, Nir Soffer wrote:
>> This is a mess that's been caused by improper use of storage, and various
>> sanity checks in sanlock have all reported errors for "impossible"
>> conditions indicating that something catastrophic has been done to the
>> storage it's using.  Some fundamental rules are not being followed.
>
> Thanks David.
>
> Do you need more output from sanlock to understand this issue?

 I can think of nothing more to learn from sanlock.  I'd suggest tighter,
 higher level checking or control of storage.  Low level sanity checks
 detecting lease corruption are not a convenient place to work from.

>>>
>>> ___
>>> Users mailing list
>>> Users@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users