[ovirt-users] Re: i/o wait and slow system

2020-08-28 Thread Darrell Budic
See below: > On Aug 27, 2020, at 3:19 PM, info--- via Users wrote: > > Thank you. Reboot of the engine and afterwards the backup server helped :-) Good deal. > Should I revert some of my previous changes? Reduce the write window size? > - gluster volume set vmstore performance.read-ahead on >

[ovirt-users] Re: i/o wait and slow system

2020-08-27 Thread Darrell Budic
Looks like you’ve got a posix or nfs mount there? Is your gluster storage domain of type GlusterFS? And make sure you restarted the ovirt-engine after enabling LibfgApiSupported, before stopping and restarting the vm. An active libgf mount looks like: > On

[ovirt-users] Re: broker.log not rotating

2020-07-07 Thread Darrell Budic
They’re just log files, so generally safe to delete. You may want to take a look at the huge one though, see what’s up. I had a similar problem that turned out to be a broken HA agent install, cleaned and reinstalled and it went back to the same volume of logs as the others. Now I need to

[ovirt-users] Re: Sometimes paused due to unknown storage error on gluster

2020-03-28 Thread Darrell Budic
Nic, I didn’t see what version of gluster you were running? There was a leak that caused similar behavior for me in early 6.x versions, but it was fixed in 6.6 (I think, you’d have to find it in the bugzilla to be sure) and I havn’t seen this in a while. Not sure it’s exactly your symptoms

[ovirt-users] Re: Speed Issues

2020-03-24 Thread Darrell Budic
Christian, Adding on to Stahil’s notes, make sure you’re using jumbo MTUs on servers and client host nodes. Making sure you’re using appropriate disk schedulers on hosts and VMs is important, worth double checking that it’s doing what you think it is. If you are only HCI, gluster’s

[ovirt-users] Re: paused vm's will not resume

2020-02-18 Thread Darrell Budic
What version of ovirt are you running? What is your storage domain, nfs or gluster? Using libglfapi? How full is your storage domain? If it’s gluster, what type is it and how full are all the bricks? Have you tried stopping and restarting them? Not ideal, but may get you a running system

[ovirt-users] Re: glusterfs

2020-02-14 Thread Darrell Budic
gt; Digital Data Services LLC. > 304.660.9080 > > > -Original Message- > From: Darrell Budic > Sent: Friday, February 14, 2020 2:58 PM > To: eev...@digitaldatatechs.com > Subject: Re: [ovirt-users] Re: glusterfs > > You don’t even need to clean everything ou

[ovirt-users] Re: glusterfs

2020-02-14 Thread Darrell Budic
You can add it in to a running ovirt cluster, it just isn’t as automatic. First you need to enable Gluster in at the cluster settings level for a new or existing cluster. Then either install/reinstall your nodes, or install gluster manually and add vdsm-gluster packages. You can create a stand

[ovirt-users] Re: Enabling Libgfapi in 4.3.8 - VMs won't start

2020-02-13 Thread Darrell Budic
gt;> As far as I understand, only the primary Gluster host is visible to >> libvirt >> when using gfapi, so if that host goes down, all VMs break. I imagine >> you're using a round-robin DNS entry for the primary Gluster host, but >> I'd >> like to be sure. >>

[ovirt-users] Re: Enabling Libgfapi in 4.3.8 - VMs won't start

2020-02-12 Thread Darrell Budic
o <mailto:s.pani...@gmail.com>> wrote: > I used the cockpit-based hc setup and "option rpc-auth-allow-insecure" is > absent from /etc/glusterfs/glusterd.vol. > > I'm going to redo the cluster this week and report back. Thanks for the tip! > > O

[ovirt-users] Re: Enabling Libgfapi in 4.3.8 - VMs won't start

2020-02-10 Thread Darrell Budic
The hosts will still mount the volume via FUSE, but you might double check you set the storage up as Gluster and not NFS. Then gluster used to need some config in glusterd.vol to set option rpc-auth-allow-insecure on I’m not sure if that got added to a hyper converged setup or not, but

[ovirt-users] Re: Emergency :/ No VMs starting

2020-02-02 Thread Darrell Budic
Check the contents of these directories: [root@node03:/rhev/data-center/mnt/glusterSD/node01.dc-dus.dalason.net :_ssd__storage/fec2eb5e-21b5-496b-9ea5-f718b2cb5556/images] # l total 345K drwxr-xr-x. 46 vdsm kvm 8.0K Feb 2 23:18 . drwxr-xr-x. 5 vdsm kvm 64

[ovirt-users] Re: Libgfapi considerations

2019-12-16 Thread Darrell Budic
I use libgfap in production, the performance is worth a couple of quirks for me. - watch major version updates, they’ll silently turn it off because the engine starts using a new version variable - VM/qemu security quirk that resets ownership when the VM quits, was supposedly fixed in 4.3.6 but

[ovirt-users] Re: Ovirt instance having RTO in every 10 minutes

2019-11-15 Thread Darrell Budic
Every 10 minutes & ping loss sounds like your hosted engine may be being restarted by the monitoring agents. Check the Hosted Engine uptime, is it < 10m? Then check the ovirt-HA-agent logs on your hosts and see if you can tell why it’s restarting and correct the issue. > On Nov 15, 2019, at

[ovirt-users] Re: Linux VM keeps crashing on copying files "with lost communication with qemu"

2019-08-02 Thread Darrell Budic
I’ve been seeing similar issues lately (since upgrading to 4.3.4 and later) with gluster storage (and libgfapi), but haven’t pinned to anything particular yet. But libvirt seems to have some issues and vdsmd stops being able to poll disk usage. My VMs don’t crash or stop working, but they do

[ovirt-users] Re: iptables with 4.3+?

2019-07-04 Thread Darrell Budic
I’m in the same boat, puppet managing iptables rules, and was able to continue forcing it on my 4.3.x ovirt systems. Engine-setup complains all the time, but so far it hasn’t broken anything. -Darrell > On Jul 4, 2019, at 9:38 AM, Jordan Conway wrote: > > Hello, > I'm working on migrating

[ovirt-users] Re: Memory ballon question

2019-06-13 Thread Darrell Budic
have the ovirt-guest-agent > up and running. > > Best Regards, > Strahil Nikolov > > On Jun 12, 2019 22:07, Darrell Budic wrote: > Do you have the overt-guest-agent running on your VMs? It’s required for > ballooning to control allocations on the guest side. >

[ovirt-users] Re: Memory ballon question

2019-06-12 Thread Darrell Budic
Do you have the overt-guest-agent running on your VMs? It’s required for ballooning to control allocations on the guest side. > On Jun 12, 2019, at 11:32 AM, Strahil wrote: > > Hello All, > > as a KVM user I know how usefull is the memory balloon and how you can both > increase - and also

[ovirt-users] Re: [ovirt-announce] Re: [ANN] oVirt 4.3.4 First Release Candidate is now available

2019-05-20 Thread Darrell Budic
If someone else already opened - please ping me to mark this one as duplicate. > > Best Regards, > Strahil Nikolov > > > В четвъртък, 16 май 2019 г., 22:27:01 ч. Гринуич+3, Darrell Budic > mailto:bu...@onholyground.com>> написа: > > > On May 16, 2019, at 1:41 PM

[ovirt-users] Re: [ovirt-announce] Re: [ANN] oVirt 4.3.4 First Release Candidate is now available

2019-05-16 Thread Darrell Budic
https://bugzilla.redhat.com/show_bug.cgi?id=1711054 > On May 16, 2019, at 2:17 PM, Nir Soffer wrote: > > On Thu, May 16, 2019 at 10:12 PM Darrell Budic <mailto:bu...@onholyground.com>> wrote: > On May 16, 2019, at 1:41 PM, Nir Soffer <mailto:nsof...@redhat.com>>

[ovirt-users] Re: [ovirt-announce] Re: [ANN] oVirt 4.3.4 First Release Candidate is now available

2019-05-16 Thread Darrell Budic
On May 16, 2019, at 1:41 PM, Nir Soffer wrote: > > On Thu, May 16, 2019 at 8:38 PM Darrell Budic <mailto:bu...@onholyground.com>> wrote: > I tried adding a new storage domain on my hyper converged test cluster > running Ovirt 4.3.3.7 and gluster 6.1. I was able to c

[ovirt-users] Re: [ovirt-announce] Re: [ANN] oVirt 4.3.4 First Release Candidate is now available

2019-05-16 Thread Darrell Budic
I tried adding a new storage domain on my hyper converged test cluster running Ovirt 4.3.3.7 and gluster 6.1. I was able to create the new gluster volume fine, but it’s not able to add the gluster storage domain (as either a managed gluster volume or directly entering values). The created

[ovirt-users] Re: Dropped RX Packets

2019-05-16 Thread Darrell Budic
Check your host for dropped packets as well. I had found that some of my older 10G cards were setting smaller buffers than they could, and using ethtool to set tx and rx buffers to their max values significantly improved things for those cards. And look at your switch to be sure it/they are not

[ovirt-users] Re: Gluster volume heals and after 5 seconds has /dom_md/ids dirty again

2019-05-14 Thread Darrell Budic
Yep, so far so good. Feels like 3.12.15 again, stability wise ;) > On May 14, 2019, at 5:28 AM, Strahil wrote: > > Hi Darrel, > Is ovirrt dealing OK with Gluster 6.X ? > > Best Regards, > Strahil NikolovOn May 13, 2019 18:37, Darrell Budic > wrote: >> &

[ovirt-users] Re: Gluster volume heals and after 5 seconds has /dom_md/ids dirty again

2019-05-13 Thread Darrell Budic
on host node yields no updates available > > On Mon, May 13, 2019 at 1:03 PM Darrell Budic <mailto:bu...@onholyground.com>> wrote: > Ovirt just pulls in the gluster5 repos, if you upgrade now you should get > gluster 5.6 on your nodes. If you’re running them on centos, you can i

[ovirt-users] Re: Gluster volume heals and after 5 seconds has /dom_md/ids dirty again

2019-05-13 Thread Darrell Budic
Ovirt just pulls in the gluster5 repos, if you upgrade now you should get gluster 5.6 on your nodes. If you’re running them on centos, you can install centos-release-gluster6 to go to gluster6. Ovirt NodeNG is a different story, as you mention, but I believe you can still run an update on it to

[ovirt-users] Re: Gluster volume heals and after 5 seconds has /dom_md/ids dirty again

2019-05-13 Thread Darrell Budic
I encountered serious issues with 5.3-5.5 (crashing bricks, multiple brick processes for the same brick causing disconnects and excessive heals). I had better luck with 5.6, although it’s not clear to me if the duplicate brick process issue is still present in that version. I finally jumped to

[ovirt-users] Re: Gluster volume heals and after 5 seconds has /dom_md/ids dirty again

2019-05-13 Thread Darrell Budic
What version of gluster are you running at the moment? > On May 13, 2019, at 10:25 AM, Andreas Elvers > wrote: > > Yes. After a reboot you could have a sync issue for up to a few hours. But > this issue persists now for 24 days. Additionally I see errors in the > glustershd.log of the two

[ovirt-users] Re: Gluster volume heals and after 5 seconds has /dom_md/ids dirty again

2019-05-13 Thread Darrell Budic
I see this sometimes after rebooting a server, and it usually stops happening, generally within a few hours, I’ve never tracked it down further. Don’t know for sure, but I assume it’s related to healing and goes away once everything syncs up. Occasionally it turns out to be a communications

[ovirt-users] Re: Cluster Un-stable since power outage

2019-05-07 Thread Darrell Budic
Was your hyper converged and is this storage gluster based? Your error is DNS related, if a bit odd. Have you checked the resolv.conf configs and confirmed the servers listed there are reachable and responsive? When your hosts are active, are they able to mount all the storage domains they

[ovirt-users] Re: Tuning Gluster Writes

2019-04-15 Thread Darrell Budic
Interesting. Who’s 10g cards and which offload settings did you disable? Did you do that on the servers or the vm host clients or both? > On Apr 15, 2019, at 11:37 AM, Alex McWhirter wrote: >> I went in and disabled TCP offload on all the nics, huge performance boost. >> went from 110MB/s to

[ovirt-users] Re: [Gluster-users] Announcing Gluster release 5.5

2019-03-29 Thread Darrell Budic
I’ve also encounter multiple brick processes (glusterfsd) being spawned per brick directory on gluster 5.5 while upgrading from 3.12.15. In my case, it’s on a stand alone server cluster that doesn’t have ovirt installed, so it seems to be gluster itself. Haven’t had the chance to followup on

[ovirt-users] Re: [Gluster-users] Announcing Gluster release 5.5

2019-03-26 Thread Darrell Budic
seems to be behaving and I was also in the middle of adding disk to my servers, and have to restart them (or at least gluster), so I’m going for it. After I finish this, I’ll test gluster 6 out. -Darrell > On Mar 25, 2019, at 11:04 AM, Darrell Budic wrote: > > I’m not quite done with

[ovirt-users] Re: [Gluster-users] Announcing Gluster release 5.5

2019-03-25 Thread Darrell Budic
o well. -Darrell > On Mar 21, 2019, at 3:39 PM, Strahil wrote: > > Hi Darrel, > > Will it fix the cluster brick sudden death issue ? > > Best Regards, > Strahil Nikolov > > On Mar 21, 2019 21:56, Darrell Budic wrote: > This release of Gluster 5.5 appears

[ovirt-users] Fwd: [Gluster-users] Announcing Gluster release 5.5

2019-03-21 Thread Darrell Budic
This release of Gluster 5.5 appears to fix the gluster 3.12->5.3 migration problems many ovirt users have encountered. I’ll try and test it out this weekend and report back. If anyone else gets a chance to check it out, let us know how it goes! -Darrell > Begin forwarded message: > >

[ovirt-users] Re: Hosted Engine I/O scheduler

2019-03-20 Thread Darrell Budic
> On Mar 20, 2019, at 12:42 PM, Ryan Barry wrote: > > On Wed, Mar 20, 2019, 1:16 PM Darrell Budic <mailto:bu...@onholyground.com>> wrote: > Inline: > >> On Mar 20, 2019, at 4:25 AM, Roy Golan > <mailto:rgo...@redhat.com>> wrote: >> >> On

[ovirt-users] Re: Hosted Engine I/O scheduler

2019-03-20 Thread Darrell Budic
Inline: > On Mar 20, 2019, at 4:25 AM, Roy Golan wrote: > > On Mon, 18 Mar 2019 at 22:14, Darrell Budic <mailto:bu...@onholyground.com>> wrote: > I agree, been checking some of my more disk intensive VMs this morning, > switching them to noop definitely improved respon

[ovirt-users] Re: Hosted Engine I/O scheduler

2019-03-18 Thread Darrell Budic
in my hosts or to switch to > 'cfq'. > After all, I'm using Hyper-converged oVirt and this needs testing. > What I/O scheduler are you using on the host? > > Best Regards, > Strahil Nikolov > > On Mar 18, 2019 19:15, Darrell Budic wrote: > Checked this on min

[ovirt-users] Re: Hosted Engine I/O scheduler

2019-03-18 Thread Darrell Budic
Checked this on mine, see the same thing. Switching the engine to noop definitely feels more responsive. I checked on some VMs as well, it looks like virtio drives (vda, vdb….) get mq-deadline by default, but virtscsi gets noop. I used to think the tuned profile for virtual-guest would set

[ovirt-users] Re: Libgfapisupport messes disk image ownership

2019-03-15 Thread Darrell Budic
You may have this one instead. I just encountered it last night, still seems to be an issue. https://bugzilla.redhat.com/show_bug.cgi?id=1666795 > On Mar 15, 2019, at 4:25 PM, Hesham Ahmed wrote: > > I had reported this here: https://bugzilla.redhat.com/show_bug.cgi?id=1687126 > > Has anyone

[ovirt-users] Re: Are people still experiencing issues with GlusterFS on 4.3x?

2019-03-15 Thread Darrell Budic
Upgrading gluster from version 3.12 or 4.1 (included in ovirt 3.x) to 5.3 (in ovirt 4.3) seems to cause this due to a bug in the gluster upgrade process. It’s an unfortunate side effect fo us upgrading ovirt hyper-converged systems. Installing new should be fine, but I’d wait for gluster to get

[ovirt-users] Re: HC : JBOD or RAID5/6 for NVME SSD drives?

2019-02-25 Thread Darrell Budic
I do similar with ZFS. In fact, I have a mix of large multi-drive ZFS volumes as single bricks, and a few SSDs with xfs as single bricks in other volumes, based on use. From what I’ve gathered watching the lists for a while, some people with lots of single bricks (drives) per node encounter

[ovirt-users] Re: Stuck completing last step of 4.3 upgrade

2019-02-20 Thread Darrell Budic
I was just helping Tristam on #ovirt with a similar problem, we found that his two upgraded nodes were running multiple glusterfsd processes per brick (but not all bricks). His volume & brick files in /var/lib/gluster looked normal, but starting glusterd would often spawn extra fsd processes

[ovirt-users] Re: Q: Is it safe to execute on node "saslpasswd2 -a libvirt username" ?

2019-01-23 Thread Darrell Budic
I’ve done it with no ill effects. Can be useful for troubleshooting or clearing a stuck VM if the engine is down, but I don’t recommend doing much with it if your engine is up and running. > On Jan 23, 2019, at 11:27 AM, Andrei Verovski wrote: > > Hi ! > > Is it safe to execute on oVirt

[ovirt-users] Re: Ovirt hosts running 4.2.3-1.el7 fail to upgrade to CentOS 7.6

2018-12-27 Thread Darrell Budic
Yes, known issue, update your ovirt-release42 first: > On Dec 13, 2018, at 9:08 AM, Sandro Bonazzola wrote: > > I would consider "yum update ovirt-release42" as a better option. It will > provide the missing nbdkit dependency. > It has been fixed in oVirt 4.2.7 release. > > On Dec 27,

[ovirt-users] Re: Running different minor versions in different clusters?

2018-12-18 Thread Darrell Budic
It survives and continues to work fine. I’ve been in this situation during upgrades when I haven’t been able to do all my clusters at the same time. > On Dec 18, 2018, at 12:17 PM, Florian Schmid wrote: > > Hi, > > does nobody has a clue on this? I would need a clear statement about that. ;)

[ovirt-users] Re: Gluster with two ovirt nodes

2018-12-13 Thread Darrell Budic
I would recommend just putting a gluster Arbiter on the 3rd node, then you can use normal ovirt tools more easily. If you really want to do this, I wouldn’t bother with ctdb. I used to do it, switch to a simpler DNS trick, just put entries in your hosts file with the storage ip of both nodes,

[ovirt-users] Re: Memory usage inclines

2018-12-13 Thread Darrell Budic
the host itself in the engine, > it says under "General" that > Max free Memory for scheduling new VMs: 413360 MB > > So maybe it's some sort of caching that's using the memory. > > > /tony > > On Wed, 2018-12-12 at 09:59 -0600, Darrell Budic wrote:

[ovirt-users] Re: Memory usage inclines

2018-12-12 Thread Darrell Budic
, (I've tried that earlier) the only thing > that works is to stop all vm's, reboot all hosts at the same time and > start vm's again. Then memory usage shown in the dashboard slowly > increases over time again. > > /tony > > > > > > > On Tue, 2018-12-1

[ovirt-users] Re: Memory usage inclines

2018-12-11 Thread Darrell Budic
4208 S 1.0 0.0 > 368:03.80 > libvirtd > 6902 qemu 20 0 9110480 8.0g 13580 S 1.0 1.6 1787:57 > qemu-kvm > 7249 qemu 20 0 4913084 1.6g 13712 S 0.7 0.3 1367:32 > qemu-kvm > > > It looks

[ovirt-users] Re: Memory usage inclines

2018-12-10 Thread Darrell Budic
Grab a shell on your hosts and check top memory use quick. Could be VDSMD, in which case restarting the process will give you a temp fix. If you’re running hyperconvered, check your gluster version, there was a leak in versions 3.12.7 - 3.1.12 or so, updating ovirt/gluster is the best fix for

[ovirt-users] Re: Hyperconverged Ovirt + ZFS

2018-11-16 Thread Darrell Budic
> of a VM. COW filesystems have downsides over time. > > > > > On Thu, Nov 15, 2018 at 6:09 PM Darrell Budic <mailto:bu...@onholyground.com>> wrote: > I did this in the past and didn’t have any trouble with gluster/ZFS, but > 4.2.x probably does more validat

[ovirt-users] Re: Hyperconverged Ovirt + ZFS

2018-11-15 Thread Darrell Budic
I did this in the past and didn’t have any trouble with gluster/ZFS, but 4.2.x probably does more validation. I recommend these settings on your zfs volumes, I set mine at the root(v0 here) and let them inherit: required: v0xattr sa local v0acltype

[ovirt-users] Re: [ovirt-announce] [ANN] oVirt 4.2.7 async update is now available

2018-11-13 Thread Darrell Budic
Re 1647032, isn’t server.allow-insecure=on required for libgfapi? Or has that been worked around in a different way? -Darrell > On Nov 13, 2018, at 9:46 AM, Sandro Bonazzola wrote: > > The oVirt Team has just released a new version of the following packages: > - ovirt-engine > -

[ovirt-users] Re: Affinity rules in ovirt

2018-10-14 Thread Darrell Budic
VM to VM affinity will try and run the VMs on the same host if positive, and different hosts if negative. VM to Host affinity will try and run the VM on a specific set of Hosts if positive, and not on those hosts if negative. Enforcing will keep the scheduler from launching a VM if it can’t

[ovirt-users] Re: VM stuck in paused mode with Cluster Compatibility Version 3.6 on 4.2 cluster

2018-09-20 Thread Darrell Budic
I had something similar happen while upgrading. Didn’t find a way to fix the configs on the fly, but was able to un-pause the VMs using virsh, then proceed to handle the ovirt portions. Probably work for you as well. > From: Marco Lorenzo Crociani > Subject: [ovirt-users] VM stuck in paused

[ovirt-users] Re: Upgraded host, engine now won't boot

2018-09-04 Thread Darrell Budic
[ovirt-users] Upgraded host, engine now won't boot > Date: September 3, 2018 at 10:21:01 PM CDT > To: Darrell Budic > Cc: users > > Ok, finally got it...Had to get a terminal ready with the virsh command and > guess what the instance number was, and then run suspend right aft

[ovirt-users] Re: Upgraded host, engine now won't boot

2018-09-03 Thread Darrell Budic
> Date: September 3, 2018 at 1:08:27 PM CDT > To: Darrell Budic > Cc: users > > Unfortunately, I seem unable to get connected to the console early enough to > actually see a kernel list. > > I've tried the hosted-engine --start-vm-paused command, but it just starts i

[ovirt-users] Re: Upgraded host, engine now won't boot

2018-09-03 Thread Darrell Budic
again. > From: Jim Kusznir > Subject: Re: [ovirt-users] Upgraded host, engine now won't boot > Date: September 3, 2018 at 12:45:22 PM CDT > To: Darrell Budic > Cc: users > > Thanks to Jayme who pointed me to the --add-console-password hosted-engine > command to set a passwor

[ovirt-users] Re: Upgraded host, engine now won't boot

2018-09-02 Thread Darrell Budic
It’s definitely not starting, you’ll have to see if you can figure out why. A couple things to try: - Check "virsh list" and see if it’s running, or paused for storage. (google "virsh saslpasswd2 ” if you need to

[ovirt-users] Re: Next Gluster Updates?

2018-08-29 Thread Darrell Budic
3.12.13 is now showing up in the storage repo. I can confirm it fixes the leak I’ve been seeing since 3.12.9 (upgraded one of my nodes and ran it overnight). Hurray! > From: Sahina Bose > Subject: [ovirt-users] Re: Next Gluster Updates? > Date: August 28, 2018 at 3:28:27 AM CDT > To: Robert

[ovirt-users] Re: Weird Memory Leak Issue

2018-08-29 Thread Darrell Budic
There’s a memory leak in gluster 3.12.9 - 3.12.12 on fuse mounted volumes, sounds like what you’re seeing. The fix is in 3.12.13, which should be showing up today or tomorrow in the centos repos (currently available from the testing repo). I’ve been running it overnight on one host to test,

[ovirt-users] Re: Tuning and testing GlusterFS performance

2018-08-05 Thread Darrell Budic
servers.. > From: Jayme > Subject: Re: [ovirt-users] Tuning and testing GlusterFS performance > Date: August 5, 2018 at 2:21:00 PM EDT > To: William Dossett > Cc: Darrell Budic; users > > I can't imagine too many probs with such a minor update I've been doing > updates o

[ovirt-users] Re: Tuning and testing GlusterFS performance

2018-08-05 Thread Darrell Budic
It set these by default? cluster.shd-wait-qlength: 1 cluster.shd-max-threads: 8 In my experience, these are WAY too high and will degrade performance to the point of causing problems on decently used volumes during a heal. If these are being set by the HCI installer, I’d recommend changing

[ovirt-users] Re: Ovirt cluster unstable; gluster to blame (again)

2018-07-09 Thread Darrell Budic
I encountered this after upgrading clients to 3.12.9 as well. It’s not present in 3.12.8 or 3.12.6. I’ve added some data I had to that bug, can produce more if needed. Forgot to mention my server cluster is at 3.12.9, and is not showing any problems, it’s just the clients. A test cluster on

[ovirt-users] Re: Ovirt cluster unstable; gluster to blame (again)

2018-07-06 Thread Darrell Budic
Jim- In additional to my comments on the gluster-users list (go conservative on your cluster-shd settings for all volumes), I have one ovirt specific one that can help you in the situation you’re in, at least if you’re seeing the same client side memory use issue I am on gluster 3.12.9+. Since

[ovirt-users] Re: Installing 3rd party watchdogs?

2018-06-06 Thread Darrell Budic
No, things like zabbix_agent or snmpd are fine. You just don’t want anything else to reboot a node the is under the control of an ovirt engine, leave the to the engine. > From: femi adegoke > Subject: [ovirt-users] Installing 3rd party watchdogs? > Date: June 6, 2018 at 3:12:49 AM CDT > To:

Re: [ovirt-users] Issues with ZFS volume creation

2018-04-02 Thread Darrell Budic
Try it with —force, if the disks have any kind of partition table on them, zfs will not allow you to over-write them by default. If it’s still complaining about the disks being in use, it’s probably mutlipathd grabbing them. multipath -l or multipath -ll will show the to you. You may be able

Re: [ovirt-users] Ovirt vm's paused due to storage error

2018-03-30 Thread Darrell Budic
settings when done, and really don’t test extreme settings in production! Back to upgrading my test cluster so I can properly abuse things like this. -Darrell > From: Darrell Budic <bu...@onholyground.com> > Subject: Re: [ovirt-users] Ovirt vm's paused due to storage error > Date: Marc

Re: [ovirt-users] gluster self-heal takes cluster offline

2018-03-23 Thread Darrell Budic
What version of ovirt and gluster? Sounds like something I just saw with gluster 3.12.x, are you using libgfapi or just fuse mounts? > From: Sahina Bose > Subject: Re: [ovirt-users] gluster self-heal takes cluster offline > Date: March 23, 2018 at 1:26:01 AM CDT > To: Jim

Re: [ovirt-users] Ovirt vm's paused due to storage error

2018-03-22 Thread Darrell Budic
I’ve also encounter something similar on my setup, ovirt 3.1.9 with a gluster 3.12.3 storage cluster. All the storage domains in question are setup as gluster volumes & sharded, and I’ve enabled libgfapi support in the engine. It’s happened primarily to VMs that haven’t been restarted to switch

Re: [ovirt-users] Ovirt with ZFS+ Gluster

2018-03-19 Thread Darrell Budic
Most of this is still valid if getting a bit long in the tooth: https://docs.gluster.org/en/latest/Administrator%20Guide/Gluster%20On%20ZFS/ I’ve got it running on several production clusters. I’m using the zfsol 0.7.6 kmod installation myself. I use a zvol per brick, and only one brick per

Re: [ovirt-users] ovirt and gateway behavior

2018-02-06 Thread Darrell Budic
I’ve seen this sort of happen on my systems, the gateway ip goes down for some reason, and the engine restarts repeatedly, rending it unusable, even though it’s on the same ip subnet as all the host boxes and can still talk to the VDSMs. In my case, it doesn’t hurt the cluster or DC, but it’s

Re: [ovirt-users] [ANN] oVirt 4.1.9 Release is now available

2018-01-24 Thread Darrell Budic
Couple questions about a fixed ‘bug' in the release notes for this: Does https://bugzilla.redhat.com/show_bug.cgi?id=1517237 mean that ovirt will no longer use libgfapi for any VMs, or is it just removing the check box from the storage GUI? If it is removing the capacity, I have several

Re: [ovirt-users] OVS not running / logwatch error after upgrade from 4.0.6 to 4.1.8

2018-01-19 Thread Darrell Budic
OVS is an optional tech preview in 4.1.x, you don’t need it. It is annoying about the logwatch errors though… I think I created the directory to avoid the errors, I forgot exactly what it was, sorry. > From: Derek Atkins > Subject: [ovirt-users] OVS not running / logwatch

Re: [ovirt-users] Problems with some vms

2018-01-14 Thread Darrell Budic
What version of gluster are you running? I’ve seen a few of these since moving my storage cluster to 12.3, but still haven’t been able to determine what’s causing it. Seems to be happening most often on VMs that haven’t been switches over to libgfapi mounts yet, but even one of those has paused

Re: [ovirt-users] Some major problems after 4.2 upgrade, could really use some assistance

2018-01-11 Thread Darrell Budic
Were you running gluster under you shared storage? If so, you probably need to setup ganesha nfs yourself. If not, check your ha-agent logs and make sure it’s mounting the storage properly and check for errors. Good luck! > From: Jayme > Subject: Re: [ovirt-users] Some major

Re: [ovirt-users] Non-responsive host, VM's are still running - how to resolve?

2017-11-14 Thread Darrell Budic
Try restarting vdsmd from the shell, “systemctl restart vdsmd”. > From: Artem Tambovskiy > Subject: [ovirt-users] Non-responsive host, VM's are still running - how to > resolve? > Date: November 14, 2017 at 11:23:32 AM CST > To: users > > Apparently, i lost the

Re: [ovirt-users] Enabling libgfapi disk access with oVirt 4.2

2017-11-09 Thread Darrell Budic
You do need to stop the VMs and restart them, not just issue a reboot. I havn’t tried under 4.2 yet, but it works in 4.1.6 that way for me. > From: Alessandro De Salvo > Subject: Re: [ovirt-users] Enabling libgfapi disk access with oVirt 4.2 > Date: November 9,

Re: [ovirt-users] How to best view remote console via macosX

2017-11-03 Thread Darrell Budic
https://www.ovirt.org/develop/release-management/features/virt/novnc-console/ > From: Jayme Fall <ja...@silverorange.com> > Subject: Re: [ovirt-users] How to best view remote console via macosX > Date: November 3, 2017 at 5:24:24 PM CDT > To: Darrell Budic > > How is web

Re: [ovirt-users] How to best view remote console via macosX

2017-11-03 Thread Darrell Budic
I find using the web based VNC is the simplest from my mac. You can extract data from a console.vv file and open it with any VNC software on a mac, even Screen Sharing, but you have to enter the IP & port manually. I’m not aware of any spice solutions at this moment, but i haven’t looked for

Re: [ovirt-users] [ANN] oVirt 4.2.0 First Beta Release is now available for testing

2017-11-01 Thread Darrell Budic
From: Greg Sheremeta > Subject: Re: [ovirt-users] [ANN] oVirt 4.2.0 First Beta Release is now > available for testing > Date: November 1, 2017 at 11:21:52 AM CDT > To: Robert Story > Cc: FERNANDO FREDIANI; users > > > I'd argue > that oVirt, particularly the admin portal,

Re: [ovirt-users] [ANN] oVirt 4.2.0 First Beta Release is now available for testing

2017-10-31 Thread Darrell Budic
. >> >> We will however consider bringing this back if there will be high demand. >> >> Thanks for the feedback! >> Oved >> >> On Oct 31, 2017 7:50 PM, "Darrell Budic" <bu...@onholyground.com >> <mailto:bu...@onholyground.com>> wrote:

Re: [ovirt-users] [ANN] oVirt 4.2.0 First Beta Release is now available for testing

2017-10-31 Thread Darrell Budic
Agreed. I use the right click functionality all the time and will miss it. With 70+ VMs, I may check status in a mobile interface, but I’m never going to use it for primary work. Please prioritize ease of use on Desktop over Mobile! > From: FERNANDO FREDIANI >

Re: [ovirt-users] VM resource allocation and IO Threads

2017-10-30 Thread Darrell Budic
Best explanation I’ve found is https://wiki.mikejung.biz/KVM_/_Xen#virtio-blk_iothreads_.28x-data-plane.29 If you google a bit, you’ll find some more under QEMU topics, I saw some discussion of threads and queues in virtio-scsi, but that seems to be a slightly different thing than this

Re: [ovirt-users] Debugging warning messages about bonding mode 4

2017-10-06 Thread Darrell Budic
That looks like the normal state for a LACP bond, but it does record some churn (bond renegotiations, I believe). So it probably bounced once or twice coming up. Maybe a slow switch, maybe a switch relying on dynamic bonding instead of static bonds, and taking longer to establish. For the

Re: [ovirt-users] More than one mgmt network possible?

2017-09-11 Thread Darrell Budic
From personal experience, if you want it in the same Cluster as other servers, it needs to be on the same mgmt network. If you put it in it’s own cluster, it can have it’s own mgmt network. The engine needs IP connectivity, obviously. I have a DC running with 3 clusters, 2 in the same

Re: [ovirt-users] ovirt-hosted-engine state transition messages

2017-07-23 Thread Darrell Budic
, but web interface was a bit slow before the restart. This was 4.1.3 now. Added some ram to the Hosted Engine, but looks like I need to restart it and will probably wait until I’m back for that. > On Jul 18, 2017, at 9:22 AM, Darrell Budic <bu...@onholyground.com> wrote: >

Re: [ovirt-users] ovirt-hosted-engine state transition messages

2017-07-18 Thread Darrell Budic
I had some of this going on recently under 4.1.2, started with one or two warning messages, then a flood of them. Did the upgrade to 4.1.3 and haven’t seen it yet, but it’s only been a few days so far. A java process was consuming much CPU, and the DataWarehouse appears to not be collecting

[ovirt-users] vdsm (4.1) restarts glusterd when activating a node, even if it's already running

2017-07-02 Thread Darrell Budic
Upgrading some nodes today, and noticed that vdsmd restarts glusterd on a node when it activates it. This is causing a short break in healing when the shd gets disconnected, forcing some extra healing when the healing process reports “Transport Endpoint Disconnected” (N/A in the ovirt gui).

[ovirt-users] vdsm changing disk scheduler when starting, configurable?

2017-07-02 Thread Darrell Budic
It seems vdsmd under 4.1.x (or something under it’s control) changes the disk schedulers when it starts or a host node is activated, and I’d like to avoid this. Is it preventable? Or configurable anywhere? This was probably happening under earlier version, but I just noticed it while upgrading

Re: [ovirt-users] Very poor GlusterFS performance

2017-06-19 Thread Darrell Budic
Chris- You probably need to head over to gluster-us...@gluster.org for help with performance issues. That said, what kind of performance are you getting, via some form or testing like bonnie++ or even dd runs? Raw bricks vs gluster performance is useful to

Re: [ovirt-users] Lost our HostedEngineVM

2017-03-22 Thread Darrell Budic
From a hosted engine host shell, it’s: hosted-engine —vm-start hosted-engine —vm-status is also useful. If you restored your storage (that include the hosted engine storage) after rebooting that host, you should try restarting ovirt-ha-agent & ovirt-ha-broker, or just restart the machine and

Re: [ovirt-users] oVirt 4 and 10GbE NICs

2017-03-14 Thread Darrell Budic
> On Mar 14, 2017, at 7:54 AM, FERNANDO FREDIANI > wrote: > > Isn't the traffic shown on the dashboard based in 1Gbps always, even if the > hosts have 10Gb interfaces ? > Yep, all dirt interfaces show as 1Gb. > Is there anywhere in oVirt config files or Database

Re: [ovirt-users] Replicated Glusterfs on top of ZFS

2017-03-03 Thread Darrell Budic
Why are you using an arbitrator if all your HW configs are identical? I’d use a true replica 3 in this case. Also in my experience with gluster and vm hosting, the ZIL/slog degrades write performance unless it’s a truly dedicated disk. But I have 8 spinners backing my ZFS volumes, so trying to

Re: [ovirt-users] upgrading from 3.6 -> 4.1, vm restarts at 4.0 mandatory?

2017-02-26 Thread Darrell Budic
-Updating_the_oVirt_Environment/> > > Regards, > Fred > > On Fri, Feb 24, 2017 at 10:44 PM, Darrell Budic <bu...@onholyground.com > <mailto:bu...@onholyground.com>> wrote: > I’m upgrading my main cluster from 3.6 to 4.1, and I’m currently at 4.0. I’ve > upg

[ovirt-users] upgrading from 3.6 -> 4.1, vm restarts at 4.0 mandatory?

2017-02-24 Thread Darrell Budic
I’m upgrading my main cluster from 3.6 to 4.1, and I’m currently at 4.0. I’ve upgraded the cluster and datacenter compatibility versions to 4.0, and now all my VMs are pending restart to update their configs to 4.0. My question is “Do I need to do this here, or can I go ahead and update the

Re: [ovirt-users] gpu passthrough

2017-02-21 Thread Darrell Budic
I think you need to click the “down arrow” under the top box, should move the GPU into the “Host Devices to be attached” box, then hit Ok to make the assignment. > On Feb 21, 2017, at 3:26 AM, qinglong.d...@horebdata.cn wrote: > > Hi, all: > I want to assign the gpu card of one host to

Re: [ovirt-users] Request for feedback on your db vacuum status

2016-12-13 Thread Darrell Budic
12 December 2016 at 20:31, Darrell Budic <bu...@onholyground.com > <mailto:bu...@onholyground.com>> wrote: > Here’s mine: http://paste.fedoraproject.org/505443/14815674/ > <http://paste.fedoraproject.org/505443/14815674/> > > This engine has been around since at

Re: [ovirt-users] ovirtmgmt manual bridge cannot be used in ovirt 4.0

2016-11-28 Thread Darrell Budic
I’m having trouble with that plan and I’m not even trying to put the ovirtmgmt bridge on an existing bond. I have a pre-existing gluster setup with gluster running on a bonded interface. The ovirtmgmt should go on a different interface, but even that fails on the bond with the “torn down

  1   2   >