[ovirt-users] Re: [Gluster-users] Announcing Gluster release 5.5

2019-03-28 Thread Leo David
Olaf, thank you very much for this feedback, I was just about to upgrade my 12 nodes 4.2.8 production cluster. And it seem so that you speared me of a lot of trouble. Though, I thought that 4.3.1 comes with gluster 5.5 which has been solved the issues, and the upgrade procedure works seemless. Not

[ovirt-users] Re: [Gluster-users] Announcing Gluster release 5.5

2019-03-28 Thread olaf . buitelaar
Forgot one more issue with ovirt, on some hypervisor nodes we also run docker, it appears vdsm tries to get an hold of the interfaces docker creates/removes and this is spamming the vdsm and engine logs with; Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19]

[ovirt-users] Re: [Gluster-users] Announcing Gluster release 5.5

2019-03-28 Thread olaf . buitelaar
Dear All, I wanted to share my experience upgrading from 4.2.8 to 4.3.1. While previous upgrades from 4.1 to 4.2 etc. went rather smooth, this one was a different experience. After first trying a test upgrade on a 3 node setup, which went fine. i headed to upgrade the 9 node production

[ovirt-users] Re: Gluster VM image Resync Time

2019-03-28 Thread Krutika Dhananjay
On Thu, Mar 28, 2019 at 2:28 PM Krutika Dhananjay wrote: > Gluster 5.x does have two important performance-related fixes that are not > part of 3.12.x - > i. in shard-replicate interaction - > https://bugzilla.redhat.com/show_bug.cgi?id=1635972 > Sorry, wrong bug-id. This should be

[ovirt-users] Re: Gluster VM image Resync Time

2019-03-28 Thread Krutika Dhananjay
Gluster 5.x does have two important performance-related fixes that are not part of 3.12.x - i. in shard-replicate interaction - https://bugzilla.redhat.com/show_bug.cgi?id=1635972 ii. in qemu-gluster-fuse interaction - https://bugzilla.redhat.com/show_bug.cgi?id=1635980 The two fixes do improve

[ovirt-users] Re: Gluster VM image Resync Time

2019-03-28 Thread Strahil
Hi Krutika, I have noticed some performance penalties (10%-15%) when using sharing in v3.12 . What is the situation now with 5.5 ? Best Regards, Strahil NikolovOn Mar 28, 2019 08:56, Krutika Dhananjay wrote: > > Right. So Gluster stores what are called "indices" for each modified file (or >

[ovirt-users] Re: Gluster VM image Resync Time

2019-03-28 Thread Krutika Dhananjay
Right. So Gluster stores what are called "indices" for each modified file (or shard) under a special hidden directory of the "good" bricks at $BRICK_PATH/.glusterfs/indices/xattrop. When the offline brick comes back up, the file corresponding to each index is healed, and then the index deleted to

[ovirt-users] Re: Gluster VM image Resync Time

2019-03-28 Thread Indivar Nair
Hi Krutika, So how does the Gluster node know which shards were modified after it went down? Do the other Gluster nodes keep track of it? Regards, Indivar Nair On Thu, Mar 28, 2019 at 9:45 AM Krutika Dhananjay wrote: > Each shard is a separate file of size equal to value of >

[ovirt-users] Re: Gluster VM image Resync Time

2019-03-27 Thread Krutika Dhananjay
Each shard is a separate file of size equal to value of "features.shard-block-size". So when a brick/node was down, only those shards belonging to the VM that were modified will be sync'd later when the brick's back up. Does that answer your question? -Krutika On Wed, Mar 27, 2019 at 7:48 PM

[ovirt-users] Re: Gluster VM image Resync Time

2019-03-27 Thread Sahina Bose
On Wed, Mar 27, 2019 at 7:40 PM Indivar Nair wrote: > > Hi Strahil, > > Ok. Looks like sharding should make the resyncs faster. > > I searched for more info on it, but couldn't find much. > I believe it will still have to compare each shard to determine whether there > are any changes that need

[ovirt-users] Re: Gluster VM image Resync Time

2019-03-27 Thread Indivar Nair
Hi Strahil, Ok. Looks like sharding should make the resyncs faster. I searched for more info on it, but couldn't find much. I believe it will still have to compare each shard to determine whether there are any changes that need to be replicated. Am I right? Regards, Indivar Nair On Wed, Mar

[ovirt-users] Re: Gluster VM image Resync Time

2019-03-27 Thread Strahil
By default ovirt uses 'sharding' which splits the files into logical chunks. This greatly reduces healing time, as VM's disk is not always completely overwritten and only the shards that are different will be healed. Maybe you should change the default shard size. Best Regards, Strahil

[ovirt-users] Re: [Gluster-users] Re: VM disk corruption with LSM on Gluster

2019-03-27 Thread Sander Hoentjen
Hi Krutika, Leo, Sounds promising. I will test this too, and report back tomorrow (or maybe sooner, if corruption occurs again). -- Sander On 27-03-19 10:00, Krutika Dhananjay wrote: > This is needed to prevent any inconsistencies stemming from buffered > writes/caching file data during live

[ovirt-users] Re: [Gluster-users] Announcing Gluster release 5.5

2019-03-26 Thread Darrell Budic
Following up on this, my test/dev cluster is now completely upgraded to ovirt 4.3.2-1 and gluster5.5 and I’ve bumped the op-version on the gluster volumes. It’s behaving normally and gluster is happy, no excessive healing or crashing bricks. I did encounter

[ovirt-users] Re: [Gluster-users] Announcing Gluster release 5.5

2019-03-25 Thread Darrell Budic
I’m not quite done with my test upgrade to ovirt 4.3.x with gluster 5.5, but so far it’s looking good. I have NOT encountered the upgrade bugs listed as resolved in the 5.5 release notes. Strahil, I didn’t encounter the brick death issue and don’t have a bug ID handy for it, but so far I

[ovirt-users] Re: "gluster-ansible-roles is not installed on Host" error on Cockpit

2019-03-11 Thread Gobinda Das
I just used below repo in centos7 and able to install latest *gluster-ansible-roles-1.0.4-4.el7.noarch* [sac-gluster-ansible] name=Copr repo for gluster-ansible owned by sac baseurl= https://copr-be.cloud.fedoraproject.org/results/sac/gluster-ansible/epel-7-$basearch/ type=rpm-md

[ovirt-users] Re: "gluster-ansible-roles is not installed on Host" error on Cockpit

2019-03-11 Thread Sahina Bose
We do have an updated rpm gluster-ansible-roles. +Sachidananda URS On Sun, Mar 10, 2019 at 7:00 PM Hesham Ahmed wrote: > > sac-gluster-ansible is there and is enabled: > > [sac-gluster-ansible] > enabled=1 > name = Copr repo for gluster-ansible owned by sac > baseurl = >

[ovirt-users] Re: "gluster-ansible-roles is not installed on Host" error on Cockpit

2019-03-10 Thread Hesham Ahmed
sac-gluster-ansible is there and is enabled: [sac-gluster-ansible] enabled=1 name = Copr repo for gluster-ansible owned by sac baseurl = https://copr-be.cloud.fedoraproject.org/results/sac/gluster-ansible/epel-7-$basearch/ type = rpm-md skip_if_unavailable = False gpgcheck = 1 gpgkey =

[ovirt-users] Re: "gluster-ansible-roles is not installed on Host" error on Cockpit

2019-03-10 Thread Strahil
Check if you have a repo called sac-gluster-ansible. Best Regards, Strahil NikolovOn Mar 10, 2019 08:21, Hesham Ahmed wrote: > > On a new 4.3.1 oVirt Node installation, when trying to deploy HCI > (also when trying adding a new gluster volume to existing clusters) > using Cockpit, an error is

[ovirt-users] Re: Gluster messages after upgrade to 4.3.1

2019-03-01 Thread Sandro Bonazzola
Il giorno ven 1 mar 2019 alle ore 12:57 Jayme ha scritto: > These are both reported bugs > yes, just adding some pointers: > > On Fri, Mar 1, 2019 at 7:34 AM Stefano Danzi wrote: > >> Hello, >> >> I've just upgrade to version 4.3.1 and I can see this message in gluster >> log of all my host

[ovirt-users] Re: Gluster messages after upgrade to 4.3.1

2019-03-01 Thread Jayme
These are both reported bugs On Fri, Mar 1, 2019 at 7:34 AM Stefano Danzi wrote: > Hello, > > I've just upgrade to version 4.3.1 and I can see this message in gluster > log of all my host (running oVirt Node): > > The message "E [MSGID: 101191] > [event-epoll.c:671:event_dispatch_epoll_worker]

[ovirt-users] Re: Gluster setup Problem

2019-02-26 Thread Matthew Roth
I got the Gluster working but I can’t finish setting up the engine it fails and says can’t query dns for engine and then can’t remove the storage domain without redoing everything all over again. On Tue, Feb 26, 2019 at 2:27 AM Parth Dhanjal wrote: > Hey Matthew! > > Can you please provide me

[ovirt-users] Re: Gluster setup Problem

2019-02-25 Thread Parth Dhanjal
Hey Matthew! Can you please provide me with the following to help you debug the issue that you are facing? 1. oVirt and gdeploy version 2. /var/log/messages file 3. /root/.gdeploy file On Mon, Feb 25, 2019 at 1:23 PM Parth Dhanjal wrote: > Hey Matthew! > > Can you please provide which oVirt

[ovirt-users] Re: Gluster - performance.strict-o-direct and other performance tuning in different storage backends

2019-02-25 Thread Leo David
Thank you Krutika, Does it mean that turning that setting off, i have chances to get into data corruption ? It seems to have a pretty big impact on vm performance.. On Mon, Feb 25, 2019, 12:40 Krutika Dhananjay wrote: > Gluster's write-behind translator by default buffers writes for flushing >

[ovirt-users] Re: Gluster - performance.strict-o-direct and other performance tuning in different storage backends

2019-02-25 Thread Krutika Dhananjay
Gluster's write-behind translator by default buffers writes for flushing to disk later, *even* when the file is opened with O_DIRECT flag. Not honoring O_DIRECT could mean a reader from another client could be READing stale data from bricks because some WRITEs may not yet be flushed to disk.

[ovirt-users] Re: Gluster setup Problem

2019-02-24 Thread Parth Dhanjal
Hey Matthew! Can you please provide which oVirt and gdeploy version have you installed? Regards Parth Dhanjal On Mon, Feb 25, 2019 at 12:56 PM Sahina Bose wrote: > +Gobinda Das +Dhanjal Parth can you please check? > > On Fri, Feb 22, 2019 at 11:52 PM Matthew Roth wrote: > > > > I have 3

[ovirt-users] Re: Gluster setup Problem

2019-02-24 Thread Sahina Bose
+Gobinda Das +Dhanjal Parth can you please check? On Fri, Feb 22, 2019 at 11:52 PM Matthew Roth wrote: > > I have 3 servers, Node 1 is 3tb /dev/sda, Node 2, 3tb /dev/sdb, node3 3tb > /dev/sdb > > I start the process for gluster deployment. I change node 1 to sda and all > the other ones to

[ovirt-users] Re: Gluster version on oVirt 4.3.0

2019-02-07 Thread Simon Coter
> On Feb 7, 2019, at 11:55 AM, supo...@logicworks.pt wrote: > > Hi, > > What Glusterfs version should I use with oVirt 4.3.0 ? Gluster 5.2 is the release used by oVirt 4.3 Simon > > Thanks > > -- > Jose Ferradeira > http://www.logicworks.pt >

[ovirt-users] Re: Gluster version on oVirt 4.3.0

2019-02-07 Thread Sandro Bonazzola
Il giorno gio 7 feb 2019 alle ore 11:57 ha scritto: > Hi, > > What Glusterfs version should I use with oVirt 4.3.0 ? > 4.3.0 is using Gluster 5 > > Thanks > > -- > -- > Jose Ferradeira > http://www.logicworks.pt > ___ >

[ovirt-users] Re: [Gluster-users] VMs paused - unknown storage error - Stale file handle - distribute 2 - replica 3 volume with sharding

2019-01-17 Thread olaf . buitelaar
Hi Marco, It looks like I'm suffering form the same issue, see; https://lists.gluster.org/pipermail/gluster-users/2019-January/035602.html I've included a simple github gist there, which you can run on the machines with the stale shards. However i haven't tested the full purge, it works well

[ovirt-users] Re: Gluster Disk Full

2018-12-18 Thread suporte
Hi, It's possible to remove a gluster folder and files (corresponding to a disk) from the command line? Thanks José From: supo...@logicworks.pt To: "users" Sent: Monday, December 17, 2018 11:46:54 PM Subject: [ovirt-users] Gluster Disk Full Hi, I have a gluster volume with disk

[ovirt-users] Re: [Gluster-users] VMs paused - unknown storage error - Stale file handle - distribute 2 - replica 3 volume with sharding

2018-12-13 Thread Marco Lorenzo Crociani
Hi, is there a way to recover file from "Stale file handle" errors? Here some of the tests we have done: - compared the extended attributes of all of the three replicas of the involved shard. Found identical attributes. - compared SHA512 message digest of all of the three replicas of the

[ovirt-users] Re: Gluster with two ovirt nodes

2018-12-13 Thread Darrell Budic
I would recommend just putting a gluster Arbiter on the 3rd node, then you can use normal ovirt tools more easily. If you really want to do this, I wouldn’t bother with ctdb. I used to do it, switch to a simpler DNS trick, just put entries in your hosts file with the storage ip of both nodes,

[ovirt-users] Re: Gluster snapshots on hyperconverged with VDO ?

2018-10-17 Thread Sahina Bose
On Wed, Oct 17, 2018 at 7:15 PM wrote: > Thank you for this information. > I guess I should at least wait for that bug to be resolved before > deploying in production. Do you have the bugzilla reference so I could > track it? > https://bugzilla.redhat.com/show_bug.cgi?id=1600156

[ovirt-users] Re: Gluster snapshots on hyperconverged with VDO ?

2018-10-17 Thread guillaume . pavese
Thank you for this information. I guess I should at least wait for that bug to be resolved before deploying in production. Do you have the bugzilla reference so I could track it? ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to

[ovirt-users] Re: Gluster snapshots on hyperconverged with VDO ?

2018-10-17 Thread Sahina Bose
On Wed, Oct 17, 2018 at 6:42 PM wrote: > Hi, > > Anyone with experience with vdo on Hyperconverged with ovirt 4.2.7? > Should I force thin provisioning for lv in gdeploy's conf in order to have > working gluster snapshots? > > I am not sure about the status of dedup in ovirt 4.2/hyperconverged.

[ovirt-users] Re: Gluster snapshots on hyperconverged with VDO ?

2018-10-17 Thread guillaume . pavese
Hi, Anyone with experience with vdo on Hyperconverged with ovirt 4.2.7? Should I force thin provisioning for lv in gdeploy's conf in order to have working gluster snapshots? I am not sure about the status of dedup in ovirt 4.2/hyperconverged. Supported, advised or not?

[ovirt-users] Re: Gluster JSON-RPC errors

2018-10-08 Thread Maton, Brett
Bug filed https://bugzilla.redhat.com/show_bug.cgi?id=1637302 On Mon, 8 Oct 2018 at 11:21, Sahina Bose wrote: > Thanks for reporting this issue. Can you please log a bug report on this? > > On Mon, Oct 8, 2018 at 3:20 PM Kaustav Majumder > wrote: > >> Hi, >> You can find the related logs in

[ovirt-users] Re: Gluster JSON-RPC errors

2018-10-08 Thread Sahina Bose
Thanks for reporting this issue. Can you please log a bug report on this? On Mon, Oct 8, 2018 at 3:20 PM Kaustav Majumder wrote: > Hi, > You can find the related logs in supervdsm.log > > MainProcess|jsonrpc/4::DEBUG::2018-10-05 > 06:05:18,038::commands::65::root::(execCmd) /usr/bin/taskset

[ovirt-users] Re: Gluster JSON-RPC errors

2018-10-08 Thread Kaustav Majumder
Hi, You can find the related logs in supervdsm.log MainProcess|jsonrpc/4::DEBUG::2018-10-05 06:05:18,038::commands::65::root::(execCmd) /usr/bin/taskset --cpu-list 0-3 /usr/sbin/gluster --mode=script volume heal gv0 info --xml (cwd None) MainProcess|jsonrpc/4::ERROR::2018-10-05

[ovirt-users] Re: Gluster JSON-RPC errors

2018-10-08 Thread Piotr Kliczewski
This error was raised on vdsm side here [1]. I was unable to find 'getiterator' in vdsm code based. Please provide gluster related logs. This error means that 'bool' object had no attribute 'getiterator' and the call failed with runtime issue. Thanks, Piotr [1]

[ovirt-users] Re: Gluster JSON-RPC errors

2018-10-07 Thread Kaustav Majumder
Hi , I don't see any errors in the vdsm logs you have sent. Can you forward engine.log as well On Fri, Oct 5, 2018 at 11:56 AM Sahina Bose wrote: > Can you provide the vdsm.log and supervdsm.log with the relevant log. > Adding Kaustav to look into this > > On Fri, Oct 5, 2018 at 11:00 AM Maton,

[ovirt-users] Re: Gluster JSON-RPC errors

2018-10-05 Thread Sahina Bose
Can you provide the vdsm.log and supervdsm.log with the relevant log. Adding Kaustav to look into this On Fri, Oct 5, 2018 at 11:00 AM Maton, Brett wrote: > > I'm seeing the following errors appear in the event log every 10 minutes > for each participating host in the gluster cluster > >

[ovirt-users] Re: Gluster Issues

2018-09-14 Thread Paolo Margara
Hi, there was a memory leak in the gluster client that is fixed in release 3.12.13 (https://github.com/gluster/glusterdocs/blob/master/docs/release-notes/3.12.13.md). What version of gluster are you using? Paolo Il 11/09/2018 16:51, Endre Karlson ha scritto: > Hi, we are seeing some issues

[ovirt-users] Re: Gluster deployment - Arbiter volume - different disk size - volumes sizes

2018-09-12 Thread Donny Davis
I just use engine and data in mine On Wed, Sep 12, 2018, 1:33 AM femi adegoke wrote: > For the engine, you will need at least 58 GB. I always use 62 just to be > on the safe side. If you use 50, your install will fail. > > You don't need an ISO domain. ISO files can be stored in "data" or >

[ovirt-users] Re: Gluster deployment - Arbiter volume - different disk size - volumes sizes

2018-09-12 Thread femi adegoke
For the engine, you will need at least 58 GB. I always use 62 just to be on the safe side. If you use 50, your install will fail. You don't need an ISO domain. ISO files can be stored in "data" or "vmstore". Each vm you create should have 2 disks, 1 for the o/s & 1 for the data. The o/s disk

[ovirt-users] Re: Gluster deployment - Arbiter volume - different disk size - volumes sizes

2018-09-12 Thread Jayme
You don't really need a data and vmstore. Vmstore I believe iaeamt to be the new iso domain but even it is not needed as all data domains act the same. You can use a seperate data and vmstore domain because it will give you greater flexibility in terms of backing up thr volumes so you can choose

[ovirt-users] Re: Gluster clients intermittently hang until first gluster server in a Replica 1 Arbiter 1 cluster is rebooted, server error: 0-management: Unlocking failed & client error: bailing out

2018-09-02 Thread Sam McLeod
Sorry, please ignore, incorrect mailing list (doh!) -- Sam McLeod (protoporpoise on IRC) https://twitter.com/s_mcleod https://smcleod.net Words are my own opinions and do not necessarily represent those of my employer or partners. On Mon, 3 Sep 2018, at 12:30 PM, Sam McLeod wrote: > We've got

[ovirt-users] Re: Gluster Deployment hangs on enabling or disabling chronyd service

2018-07-23 Thread Sakhi Hadebe
Hi, The problem is solved. I found that the problem was with Ansible, it couldn't ssh (SSH Error) to one of the nodes. With that fixed, it installed the oVirt successfully. Thank you for your support On Tue, Jul 17, 2018 at 2:05 PM, Gobinda Das wrote: > Hi Sakhi, > Can you please provide

[ovirt-users] Re: Gluster Deployment hangs on enabling or disabling chronyd service

2018-07-17 Thread Gobinda Das
Hi Sakhi, Can you please provide engine log and ovirt-host-deploy log ? You mentioned that, you have attached log but unfortunately I can't find the attachment. On Tue, Jul 17, 2018 at 3:12 PM, Sakhi Hadebe wrote: > Hi, > > Why is gluster deployment hangs on enabling or disabling the chronyd >

[ovirt-users] Re: Gluster not synicng changes between nodes for engine

2018-06-18 Thread Hanson Turner
Ok, So removing one downed node cleared all the non syncing issues. In the mean time, when that one node was coming back, it seems to have corrupted the hosted-engine vm. Remote-Viewer nodeip:5900, the console shows: Probing EDD (edd=off to disable)... ok Doesn't matter which of the three

[ovirt-users] Re: Gluster Deployment: Repositories, Packages

2018-06-14 Thread femi adegoke
Yes Greg, I thought the same thing...like there is no way the folks at RH would make a page & expect us to leave it blank!! ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement:

[ovirt-users] Re: Gluster Deployment: Repositories, Packages

2018-06-14 Thread Greg Sheremeta
Hm, that is confusing. There should be some wording there to clarify, perhaps via info icons / tooltips. cc'ing our UXD lead Liz. Thanks for raising it. Greg On Thu, Jun 14, 2018 at 2:56 AM, Karli Sjöberg wrote: > On Wed, 2018-06-13 at 23:47 -0700, femi adegoke wrote: > > Forgot to attach

[ovirt-users] Re: Gluster Deployment: Repositories, Packages

2018-06-14 Thread Karli Sjöberg
On Wed, 2018-06-13 at 23:47 -0700, femi adegoke wrote: > Forgot to attach picture. > > On 2018-06-13 23:43, femi adegoke wrote: > > In Step 2 of the HE deployment what should be filled in here? Nothing, if you don´t have any special packages that you´d like to add. /K > > Repositories: ?? > >

[ovirt-users] Re: Gluster Deployment: Repositories, Packages

2018-06-14 Thread femi adegoke
Forgot to attach picture. On 2018-06-13 23:43, femi adegoke wrote: In Step 2 of the HE deployment what should be filled in here? Repositories: ?? Packages: ?? ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-30 Thread Johan Bernhardsson
Is storage working as it should? Does the gluster mount point respond as it should? Can you write files to it? Does the physical drives say that they are ok? Can you write (you shouldn't bypass gluster mount point but you need to test the drives) to the physical drives? For me this sounds

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-30 Thread Jim Kusznir
At the moment, it is responding like I would expect. I do know I have one failed drive on one brick (hardware failure, OS removed drive completely; the underlying /dev/sdb is gone). I have a new disk on order (overnight), but that is also one brick of one volume that is replica 3, so I would

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-30 Thread Sahina Bose
On Wed, May 30, 2018 at 10:42 AM, Jim Kusznir wrote: > hosted-engine --deploy failed (would not come up on my existing gluster > storage). However, I realized no changes were written to my existing > storage. So, I went back to trying to get my old engine running. > > hosted-engine --vm-status

[ovirt-users] Re: Gluster quorum

2018-05-30 Thread Demeter Tibor
Dear Jim, Thank you for your help, now it's working again!!! :) Have a nice day! Regards, Tibor - 2018. máj.. 29., 23:57, Jim Kusznir írta: > I had the same problem when I upgraded to 4.2. I found that if I went to the > brick in the UI and selected it, there was a "start" button

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-30 Thread Jim Kusznir
hosted-engine --deploy failed (would not come up on my existing gluster storage). However, I realized no changes were written to my existing storage. So, I went back to trying to get my old engine running. hosted-engine --vm-status is now taking a very long time (5+minutes) to return, and it

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Jim Kusznir
Well, things went from bad to very, very bad It appears that during one of the 2 minute lockups, the fencing agents decided that another node in the cluster was down. As a result, 2 of the 3 nodes were simultaneously reset with fencing agent reboot. After the nodes came back up, the engine

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Krutika Dhananjay
Adding Ravi to look into the heal issue. As for the fsync hang and subsequent IO errors, it seems a lot like https://bugzilla.redhat.com/show_bug.cgi?id=1497156 and Paolo Bonzini from qemu had pointed out that this would be fixed by the following commit: commit

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Jim Kusznir
I also finally found the following in my system log on one server: [10679.524491] INFO: task glusterclogro:14933 blocked for more than 120 seconds. [10679.525826] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [10679.527144] glusterclogro D 97209832bf40 0

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Jim Kusznir
I think this is the profile information for one of the volumes that lives on the SSDs and is fully operational with no down/problem disks: [root@ovirt2 yum.repos.d]# gluster volume profile data info Brick: ovirt2.nwfiber.com:/gluster/brick2/data --

[ovirt-users] Re: Gluster quorum

2018-05-29 Thread Jim Kusznir
I had the same problem when I upgraded to 4.2. I found that if I went to the brick in the UI and selected it, there was a "start" button in the upper-right of the gui. clicking that resolved this problem a few minutes later. I had to repeat for each volume that showed a brick down for which

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Jim Kusznir
Thank you for your response. I have 4 gluster volumes. 3 are replica 2 + arbitrator. replica bricks are on ovirt1 and ovirt2, arbitrator on ovirt3. The 4th volume is replica 3, with a brick on all three ovirt machines. The first 3 volumes are on an SSD disk; the 4th is on a Seagate SSHD (same

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Jim Kusznir
Due to the cluster spiraling downward and increasing customer complaints, I went ahead and finished the upgrade of the nodes to ovirt 4.2 and gluster 3.12. It didn't seem to help at all. I DO have one brick down on ONE of my 4 gluster filesystems/exports/whatever. The other 3 are fully

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Alex K
I would check disks status and accessibility of mount points where your gluster volumes reside. On Tue, May 29, 2018, 22:28 Jim Kusznir wrote: > On one ovirt server, I'm now seeing these messages: > [56474.239725] blk_update_request: 63 callbacks suppressed > [56474.239732] blk_update_request:

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Jim Kusznir
On one ovirt server, I'm now seeing these messages: [56474.239725] blk_update_request: 63 callbacks suppressed [56474.239732] blk_update_request: I/O error, dev dm-2, sector 0 [56474.240602] blk_update_request: I/O error, dev dm-2, sector 3905945472 [56474.241346] blk_update_request: I/O error,

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Jim Kusznir
I see in messages on ovirt3 (my 3rd machine, the one upgraded to 4.2): May 29 11:54:41 ovirt3 ovs-vsctl: ovs|1|db_ctl_base|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) May 29 11:54:51 ovirt3 ovs-vsctl:

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Sahina Bose
Do you see errors reported in the mount logs for the volume? If so, could you attach the logs? Any issues with your underlying disks. Can you also attach output of volume profiling? On Wed, May 30, 2018 at 12:13 AM, Jim Kusznir wrote: > Ok, things have gotten MUCH worse this morning. I'm

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-29 Thread Jim Kusznir
Ok, things have gotten MUCH worse this morning. I'm getting random errors from VMs, right now, about a third of my VMs have been paused due to storage issues, and most of the remaining VMs are not performing well. At this point, I am in full EMERGENCY mode, as my production services are now

[ovirt-users] Re: Gluster problems, cluster performance issues

2018-05-28 Thread Sahina Bose
[Adding gluster-users to look at the heal issue] On Tue, May 29, 2018 at 9:17 AM, Jim Kusznir wrote: > Hello: > > I've been having some cluster and gluster performance issues lately. I > also found that my cluster was out of date, and was trying to apply updates > (hoping to fix some of

[ovirt-users] Re: Gluster quorum

2018-05-28 Thread Demeter Tibor
Hi, Ok I will try it. In this case, is it possible to remove and re-add a host that member of HA gluster ? This is an another task, but I need to separate my gluster network from my ovirtmgmt network. What is the recommended way for do this? It is not important now, but I need to do in

[ovirt-users] Re: Gluster quorum

2018-05-28 Thread Sahina Bose
On Mon, May 28, 2018 at 4:47 PM, Demeter Tibor wrote: > Dear Sahina, > > Yes, exactly. I can check that check box, but I don't know how is safe > that. Is it safe? > It is safe - if you can ensure that only one host is put into maintenance at a time. > > I want to upgrade

[ovirt-users] Re: Gluster quorum

2018-05-28 Thread Demeter Tibor
Dear Sahina, Yes, exactly. I can check that check box, but I don't know how is safe that. Is it safe? I want to upgrade all of my host. If it will done, then the monitoring will work perfectly? Thanks. R. Tibor - 2018. máj.. 28., 10:09, Sahina Bose írta: > On

[ovirt-users] Re: Gluster quorum

2018-05-28 Thread Sahina Bose
On Mon, May 28, 2018 at 1:06 PM, Demeter Tibor wrote: > Hi, > > Somebody could answer to my question please? > It is very important for me, I could no finish my upgrade process (from > 4.1 to 4.2) since 9th May! > Can you explain how the upgrade process is blocked due to

[ovirt-users] Re: Gluster quorum

2018-05-28 Thread Demeter Tibor
Hi, Somebody could answer to my question please? It is very important for me, I could no finish my upgrade process (from 4.1 to 4.2) since 9th May! Meanwhile - I don't know why - one of my two gluster volume seems UP (green) on the GUI. So, now only one is down. I need help. What can I

[ovirt-users] Re: Gluster quorum

2018-05-23 Thread Demeter Tibor
Hi, I've updated again to the latest version, but there are no changes. All of bricks on my first node are down in the GUI (in console are ok) An Interesting thing, the "Self-Heal info" column show "OK" for all hosts and all bricks, but "Space used" column is zero for all hosts/bricks. Can I

[ovirt-users] Re: Gluster quorum

2018-05-23 Thread Denis Chaplygin
Hello! On Tue, May 22, 2018 at 11:10 AM, Demeter Tibor wrote: > > Is there any changes with this bug? > > Still I haven't finish my upgrade process that i've started on 9th may:( > > Please help me if you can. > > Looks like all required patches are already merged, so

[ovirt-users] Re: Gluster quorum

2018-05-22 Thread Demeter Tibor
Dear Sahina, Is there any changes with this bug? Still I haven't finish my upgrade process that i've started on 9th may:( Please help me if you can. Thanks Tibor - 2018. máj.. 18., 9:29, Demeter Tibor írta: > Hi, > I have to update the engine again? >

[ovirt-users] Re: Gluster quorum

2018-05-18 Thread Demeter Tibor
Hi, I have to update the engine again? Thanks, R Tibor - 2018. máj.. 18., 6:47, Sahina Bose írta: > Thanks for reporting this. [ https://gerrit.ovirt.org/91375 | > https://gerrit.ovirt.org/91375 ] fixes this. I've re-opened bug [ >

[ovirt-users] Re: Gluster quorum

2018-05-17 Thread Sahina Bose
Thanks for reporting this. https://gerrit.ovirt.org/91375 fixes this. I've re-opened bug https://bugzilla.redhat.com/show_bug.cgi?id=1574508 On Thu, May 17, 2018 at 10:12 PM, Demeter Tibor wrote: > Hi, > > 4.2.4-0.0.master.20180515183442.git00e1340.el7.centos > > Firstly, I

[ovirt-users] Re: Gluster quorum

2018-05-17 Thread Demeter Tibor
Hi, 4.2.4-0.0.master.20180515183442.git00e1340.el7.centos Firstly, I did a yum update "ovirt-*-setup*" second, I have ran engine-setup to upgrade. I didn't remove the old repos, just installed the nightly repo. Thank you again, Regards, Tibor - 2018. máj.. 17., 15:02, Sahina

[ovirt-users] Re: Gluster quorum

2018-05-17 Thread Sahina Bose
It doesn't look like the patch was applied. Still see the same error in engine.log "Error while refreshing brick statuses for volume 'volume1' of cluster 'C6220': null"\ Did you use engine-setup to upgrade? What's the version of ovirt-engine currently installed? On Thu, May 17, 2018 at 5:10 PM,

[ovirt-users] Re: Gluster quorum

2018-05-17 Thread Sahina Bose
[+users] Can you provide the engine.log to see why the monitoring is not working here. thanks! On Wed, May 16, 2018 at 2:08 PM, Demeter Tibor wrote: > Hi, > > Meanwhile, I did the upgrade engine, but the gluster state is same on my > first node. > I've attached some

[ovirt-users] Re: Gluster quorum

2018-05-15 Thread Sahina Bose
On Tue, May 15, 2018 at 1:28 PM, Demeter Tibor wrote: > Hi, > > Could you explain how can I use this patch? > You can use the 4.2 nightly to test it out - http://resources.ovirt.org/pub/yum-repo/ovirt-release42-snapshot.rpm > R, > Tibor > > > - 2018. máj.. 14., 11:18,

[ovirt-users] Re: Gluster quorum

2018-05-15 Thread Demeter Tibor
Hi, Could you explain how can I use this patch? R, Tibor - 2018. máj.. 14., 11:18, Demeter Tibor írta: > Hi, > Sorry for my question, but can you tell me please how can I use this patch? > Thanks, > Regards, > Tibor > - 2018. máj.. 14., 10:47, Sahina Bose

[ovirt-users] Re: Gluster quorum

2018-05-14 Thread Demeter Tibor
Hi, Sorry for my question, but can you tell me please how can I use this patch? Thanks, Regards, Tibor - 2018. máj.. 14., 10:47, Sahina Bose írta: > On Sat, May 12, 2018 at 1:14 PM, Demeter Tibor < [ mailto:tdeme...@itsmart.hu > | > tdeme...@itsmart.hu ] > wrote:

[ovirt-users] Re: Gluster quorum

2018-05-14 Thread Sahina Bose
On Sat, May 12, 2018 at 1:14 PM, Demeter Tibor wrote: > Hi, > > Could someone help me please ? I can't finish my upgrade process. > https://gerrit.ovirt.org/91164 should fix the error you're facing. Can you elaborate why this is affecting the upgrade process? > Thanks >

[ovirt-users] Re: Gluster quorum

2018-05-14 Thread Demeter Tibor
Meanwhile i just changed my gluster network to 10.104.0.0/24 but does not happend anything. Regards, Tibor - 2018. máj.. 14., 9:49, Demeter Tibor írta: > Hi, > Yes, I have a gluster network, but it's "funny" because that is the > 10.105.0.x/24. :( Also, the

[ovirt-users] Re: Gluster quorum

2018-05-14 Thread Demeter Tibor
Hi, Yes, I have a gluster network, but it's "funny" because that is the 10.105.0.x/24. :( Also, the n4.itsmart.cloud is mean 10.104.0.4. The 10.104.0.x/24 is my ovirtmgmt network. However, the 10.104.0.x is accessable from all hosts. What should I do? Thanks, R Tibor - 2018.

[ovirt-users] Re: Gluster quorum

2018-05-12 Thread Doug Ingham
The two key errors I'd investigate are these... 2018-05-10 03:24:21,048+02 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn] > (DefaultQuartzScheduler8) [7715ceda] Could not associate brick '10.104.0.1: > /gluster/brick/brick1' of volume

[ovirt-users] Re: Gluster quorum

2018-05-12 Thread Demeter Tibor
Hi, Could someone help me please ? I can't finish my upgrade process. Thanks R Tibor - 2018. máj.. 10., 12:51, Demeter Tibor írta: > Hi, > I've attached the vdsm and supervdsm logs. But I don't have engine.log here, > because that is on hosted engine vm. Should

[ovirt-users] Re: gluster dispersed volume provisioning

2018-05-11 Thread Sahina Bose
On Wed, May 9, 2018 at 12:56 PM, wrote: > Hi, have a quick question regarding ovirt UI and provisioning of gluster > volumes. > I've found an old thread - https://lists.ovirt.org/ > pipermail/users/2015-February/064602.html - where it's said that creating > dispersed volumes

[ovirt-users] Re: Gluster quorum

2018-05-10 Thread Sahina Bose
There's a bug here. Can you log one attaching this engine.log and also vdsm.log & supervdsm.log from n3.itsmart.cloud On Thu, May 10, 2018 at 3:35 PM, Demeter Tibor wrote: > Hi, > > I found this: > > > 2018-05-10 03:24:19,096+02 INFO

[ovirt-users] Re: Gluster quorum

2018-05-10 Thread Demeter Tibor
Hi, I found this: 2018-05-10 03:24:19,096+02 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler7) [43f4eaec] FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return:

[ovirt-users] Re: Gluster quorum

2018-05-10 Thread Sahina Bose
This doesn't affect the monitoring of state. Any errors in vdsm.log? Or errors in engine.log of the form "Error while refreshing brick statuses for volume" On Thu, May 10, 2018 at 2:33 PM, Demeter Tibor wrote: > Hi, > > Thank you for your fast reply :) > > > 2018-05-10

[ovirt-users] Re: Gluster quorum

2018-05-10 Thread Demeter Tibor
Hi, Thank you for your fast reply :) 2018-05-10 11:01:51,574+02 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (DefaultQuartzScheduler6) [7f01fc2d] START, GlusterServersListVDSCommand(HostName = n2.itsmart.cloud,

[ovirt-users] Re: Gluster quorum

2018-05-10 Thread Sahina Bose
Could you check the engine.log if there are errors related to getting GlusterVolumeAdvancedDetails ? On Thu, May 10, 2018 at 2:02 PM, Demeter Tibor wrote: > Dear Ovirt Users, > I've followed up the self-hosted-engine upgrade documentation, I upgraded > my 4.1 system to

<    1   2   3   4   5   >