On 2/25/19 3:49 AM, Sahina Bose wrote:
I've filed Bug 1682925 <https://bugzilla.redhat.com/show_bug.cgi?id=1682925> with the requested logs during the time frame I experienced issues. Sorry for the delay, I was out of the office Friday and this morning.On Thu, Feb 21, 2019 at 11:11 PM Jason P. Thomas <[email protected]> wrote:On 2/20/19 5:33 PM, Darrell Budic wrote: I was just helping Tristam on #ovirt with a similar problem, we found that his two upgraded nodes were running multiple glusterfsd processes per brick (but not all bricks). His volume & brick files in /var/lib/gluster looked normal, but starting glusterd would often spawn extra fsd processes per brick, seemed random. Gluster bug? Maybe related to https://secure-web.cisco.com/1zutsPlj0TjKvDiGvxmw5PZZPUoEtpkcJqhpWGvx2-fJoEMzSTBS7hSNqb4ZmBoELYn-OnLEiRZHRpA5XY5mJr_7RfSDll1EaXkWR3M-r-rqGPXA4X2JDx5sZIgvjQ2RJeEYd_rZJAV0FE2Z7aTuyPvla0k6cqyL02QWOj-RjkRXgfrSECActBehc3ggEc95SMQ068Z6cbtEdIb09cLN6qDHuWz-vs9NqstdyJQQZ8n_BDEVOw5MURbMF54EWQto914O2Pq11buymdl_wDrA2SCi31sSraVfhD9r0DTFuNqd8ibxbYbdIeKtg87JFMy3EYj8zsOlZxtsNWlbz4UMo6q0BMTcHGfrQhY_sMTWL0HlKsbonv9m8bZQsoQ7LNrQzfkEZBQgHnyNRE1_97d2ZjfcQAtZJJsIGVN4ddEp82FrJd6zhSXnfrcLiaNDAEUBi/https%3A%2F%2Fbugzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D1651246, but I’m helping debug this one second hand… Possibly related to the brick crashes? We wound up stopping glusterd, killing off all the fsds, restarting glusterd, and repeating until it only spawned one fsd per brick. Did that to each updated server, then restarted glusterd on the not-yet-updated server to get it talking to the right bricks. That seemed to get to a mostly stable gluster environment, but he’s still seeing 1-2 files listed as needing healing on the upgraded bricks (but not the 3.12 brick). Mainly the DIRECT_IO_TEST and one of the dom/ids files, but he can probably update that. Did manage to get his engine going again, waiting to see if he’s stable now. Anyway, figured it was worth posting about so people could check for multiple brick processes (glusterfsd) if they hit this stability issue as well, maybe find common ground. Note: also encountered https://secure-web.cisco.com/1eMDo5MMs_aJ6TOHBckwoG7rBeXsvPoHI01UBZ8YJ2GnI4ds5muAWuLwC7w0uUVSuhnic-rltGTzC_FGPYYigedn-1_Z90J0qkvaSCJNLr_gm2gj0ujAk5G31zF1tv-YHZSUx6PBm2tG8GpplIWiwN5N1TBqfjKEszVer6Yxrtkjl7ZPqnZlTBeGiNKBLa5zz9JFmvAhuXtFodfy6Dh-mIbFQZ1IdeIh6j37hhTfYLAhG_r-tsIFSIc8V9x_q3-PE7OA68lQ9w7dmFteOLHkhND8mvhmI1sWlV0iVg3jx9ll8a4mBSsng4Thf1lJGdBvT_vXxriv8LKHp1vh86r_u3gdbs65hWQKhhc7Z2zJJvh9yXw7lk7qF4n-hIlyIS2p4Yzgtn94ExpgvhQM1q9HVB7-SJ2YMhGJGfrAu3lFFesUMtqRPbY9TTUAXxMREwjth/https%3A%2F%2Fbugzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D1348434 trying to get his engine back up, restarting libvirtd let us get it going again. Maybe un-needed if he’d been able to complete his third node upgrades, but he got stuck before then, so... -Darrell Stable is a relative term. My unsynced entries total for each of my 4 volumes changes drastically (with the exception of the engine volume, it pretty much bounces between 1 and 4). The cluster has been "healing" for 18 hours or so and only the unupgraded HC node has healed bricks. I did have the problem that some files/directories were owned by root:root. These VMs did not boot until I changed ownership to 36:36. Even after 18 hours, there's anywhere from 20-386 entries in vol heal info for my 3 non engine bricks. Overnight I had one brick on one volume go down on one HC node. When I bounced glusterd, it brought up a new fsd process for that brick. I killed the old one and now vol status reports the right pid on each of the nodes. This is quite the debacle. If I can provide any info that might help get this debacle moving in the right direction, let me know.Can you provide the gluster brick logs and glusterd logs from the servers (from /var/log/glusterfs/). Since you mention that heal seems to be stuck, could you also provide the heal logs from /var/log/glusterfs/glustershd.log If you can log a bug with these logs, that would be great - please use https://secure-web.cisco.com/1u1JXUFgLqfmecw2r1a0ZiPWiR0AlgZ1-8A3Ax1bpyCysNvH6JpfsdOiPfzxYx9Y8wB0zTlGHTNbWzR2Qf_Y3ElsXtJKdRfYllHpXeCSMSHQKq-jQzK83503i24c4jYADgwiPhWVUuk-3K9nVr_NTDrVKPu8KShG2UH9sFakcBsTMC6xTAaHTLJJxHH_PYCmjG9XECovJOX7_LYjOIJOd-npFm_fUlWtTwXick9NaOggfdhESof7KhbdleZWP36--TaXZ3a9c26rFkogI0kVuv-8Eex7blbGFBuoCyqabs0W1fczoluYJ30xx5kAMqM5tSb36QQHRtQbhEPFD1_x9uUqwE3CFm_W7towsYiSqeY3J32aAGIliiddohLhr5a8IpQYaeHHd6Wwp0RZun_zucnEtEq7uZY8QMMMrZLDh6iA0YLntokb0xRhh_dodk8dZ/https%3A%2F%2Fbugzilla.redhat.com%2Fenter_bug.cgi%3Fproduct%3DGlusterFS to log the bug.
Jason
Jason aka Tristam On Feb 14, 2019, at 1:12 AM, Sahina Bose <[email protected]> wrote: On Thu, Feb 14, 2019 at 2:39 AM Ron Jerome <[email protected]> wrote: Can you be more specific? What things did you see, and did you report bugs? I've got this one: https://secure-web.cisco.com/13CTqpVLySHJsr0ExmooX7-Akp0iAcz1X7qikVLAWiuwjZERnApT0H_bG_3sBVc5S1k2wHsm48kCMoxnr0XUp3owKQg90cq-lHceW49zjRyF4B6WHsdrwYNR2oh4CDFxX91mXnWWp2rZhLmuHVPuqdasu2YJvJif1uymfmESrO4qB9Bmv_Nhr4WcOgTE5cGGjCJwi1y1HnBpzUtC0GXvOq0RGwQNzvkiP0vCAcWO0gaTHMya5EDT1UfRHVu6oKHD4BuiGXtp3HANc2bu2noOrDnUONyJEtzZosCxDNmTnDBbdvccsluRVi_1p2np0eVzKw-T4AQRjZBi2TDxpCC2JhjZnSD0rUr0gvWA6oFF3jzObm3ABooGJ9Ac16qRId03pohIy4nxBy843wkJHxSv33cVkFb6KmftvQzMJMwc_o_B68aHhWgA1zcXXS1-IHKCj/https%3A%2F%2Fbugzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D1649054 and this one: https://secure-web.cisco.com/1zutsPlj0TjKvDiGvxmw5PZZPUoEtpkcJqhpWGvx2-fJoEMzSTBS7hSNqb4ZmBoELYn-OnLEiRZHRpA5XY5mJr_7RfSDll1EaXkWR3M-r-rqGPXA4X2JDx5sZIgvjQ2RJeEYd_rZJAV0FE2Z7aTuyPvla0k6cqyL02QWOj-RjkRXgfrSECActBehc3ggEc95SMQ068Z6cbtEdIb09cLN6qDHuWz-vs9NqstdyJQQZ8n_BDEVOw5MURbMF54EWQto914O2Pq11buymdl_wDrA2SCi31sSraVfhD9r0DTFuNqd8ibxbYbdIeKtg87JFMy3EYj8zsOlZxtsNWlbz4UMo6q0BMTcHGfrQhY_sMTWL0HlKsbonv9m8bZQsoQ7LNrQzfkEZBQgHnyNRE1_97d2ZjfcQAtZJJsIGVN4ddEp82FrJd6zhSXnfrcLiaNDAEUBi/https%3A%2F%2Fbugzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D1651246 and I've got bricks randomly going offline and getting out of sync with the others at which point I've had to manually stop and start the volume to get things back in sync. Thanks for reporting these. Will follow up on the bugs to ensure they're addressed. Regarding brciks going offline - are the brick processes crashing? Can you provide logs of glusterd and bricks. Or is this to do with ovirt-engine and brick status not being in sync? _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://secure-web.cisco.com/1nEZk1HmmOVHI5jAXqISs80YRB-SbMsrunxvZ9XJpVCVvb-FUkEv84wZRHCniIFIRkCZMgrF8cVcLUivR_ZOhnC9oUOUtjawCGGbA1Q9QqcpwjmlA0nNn94klqBKhlSe4JqC1JZ_EIuEiJsTiuSe4-H4T43pXcHw7mfIht3pbxvMh8hIFExJqH-Uwh4PQcur4ostxaolE2sVaBJUt4jJXCnDW8M-I_vJ72Diy6queQsiHPkXt-i0X1kiFYwNDSdyaeOAZHz7SSSgp7tz0AOy9a9412bEaP_xR_tmLg8OrmR_nut2slftsxTbPngFDCCYC7sYMbCy6XUrJ_yTlQAKMIvNwsO-E1xTfUvzSazxTEOOVht_CW3bysC04V2yn2P0OUuFVGRV2TH1ZRRFF0B3dEcPV7N47DeB66NbnigXY9_V-krcg4AfxfgvdEVkVkjcM/https%3A%2F%2Fwww.ovirt.org%2Fsite%2Fprivacy-policy%2F oVirt Code of Conduct: https://secure-web.cisco.com/12Y4LEt9YRdw8K3OrOj6uPi27aZWSQ67hHP7OnuT4WeYOwwFDt4a48qNVpqj7GEge08wrxcMnxuDXjlHRA7tp5VzyRHUvAK_utFxTlbMXJKcQ-UxYr7uukErJFyOtxvKAGa0iCj3ApJNw8faeL9xWpw5ZQtRgwXgfQSagS3HNyUgQbW5E9hdejIq-L7BMncpk8Gy_HpB-YWsL9H4txk_F5GOOSmRaQMfOr8yPB8T9nKoFBW1xJwK6cSWH5qFNCEnCNjT_Y1LpbLNNuXJnnz2GV8wltwE0wutQ2Eszp4y9tMnXxKCSxtggtfYq9UMB0ubE0bjmlfVSb7KgFNg1ducA3DhCB4fuioSyiuhLwkkmwmRlVSU0m46F1YszgccR3BJMFQybeImIhqdI3taNuY-FOa_ChDX5bklaG1WQmffIJ03icDuTyDKF01ggvTHGVcch/https%3A%2F%2Fwww.ovirt.org%2Fcommunity%2Fabout%2Fcommunity-guidelines%2F List Archives: https://secure-web.cisco.com/1ZrjEv46RcxAUmSm9tzKCuYdZL9varvvxSuqJiTCcGl2sv0gexon9lOCJBU_5FRbbQgiBAaniDmjD8PdMyX5k3_2kd9bDxEB_9qVHdZL_143jIxRubQRLPd7Rp3FjFC77gndpAwT76GisHddgZFuZW5cgKaoqY1kHCGPTB5szu3ZuIJWfb706K4nR8zQQq7ZxiFI4D5VOJf9bsRmzCIJ1dIwxPdZkdVSwcpd3uuFOsyn4Nv2GC2cA3mW6BfGStlPxiQhBvCzQl0wWs_CwF0Sh1hsnkLkxUslGVXMN0XN2N9glhBNxzS8nLyJ6VQC_GQ_XpWfkYzDJvskXlrIwuWOxL8JC-x6dVwsQlf5lo1Qm37evCWS_De86p3TgylUmai6TNJccS2C2JNpGyOJgUXWJAAOwxGd3SDGyz1vIa0blRUhzoYaV7BV4UZHKpT1McLCN/https%3A%2F%2Flists.ovirt.org%2Farchives%2Flist%2Fusers%40ovirt.org%2Fmessage%2F3RVMLCRK4BWCSBTWVXU2JTIDBWU7WEOP%2F _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://secure-web.cisco.com/1nEZk1HmmOVHI5jAXqISs80YRB-SbMsrunxvZ9XJpVCVvb-FUkEv84wZRHCniIFIRkCZMgrF8cVcLUivR_ZOhnC9oUOUtjawCGGbA1Q9QqcpwjmlA0nNn94klqBKhlSe4JqC1JZ_EIuEiJsTiuSe4-H4T43pXcHw7mfIht3pbxvMh8hIFExJqH-Uwh4PQcur4ostxaolE2sVaBJUt4jJXCnDW8M-I_vJ72Diy6queQsiHPkXt-i0X1kiFYwNDSdyaeOAZHz7SSSgp7tz0AOy9a9412bEaP_xR_tmLg8OrmR_nut2slftsxTbPngFDCCYC7sYMbCy6XUrJ_yTlQAKMIvNwsO-E1xTfUvzSazxTEOOVht_CW3bysC04V2yn2P0OUuFVGRV2TH1ZRRFF0B3dEcPV7N47DeB66NbnigXY9_V-krcg4AfxfgvdEVkVkjcM/https%3A%2F%2Fwww.ovirt.org%2Fsite%2Fprivacy-policy%2F oVirt Code of Conduct: https://secure-web.cisco.com/12Y4LEt9YRdw8K3OrOj6uPi27aZWSQ67hHP7OnuT4WeYOwwFDt4a48qNVpqj7GEge08wrxcMnxuDXjlHRA7tp5VzyRHUvAK_utFxTlbMXJKcQ-UxYr7uukErJFyOtxvKAGa0iCj3ApJNw8faeL9xWpw5ZQtRgwXgfQSagS3HNyUgQbW5E9hdejIq-L7BMncpk8Gy_HpB-YWsL9H4txk_F5GOOSmRaQMfOr8yPB8T9nKoFBW1xJwK6cSWH5qFNCEnCNjT_Y1LpbLNNuXJnnz2GV8wltwE0wutQ2Eszp4y9tMnXxKCSxtggtfYq9UMB0ubE0bjmlfVSb7KgFNg1ducA3DhCB4fuioSyiuhLwkkmwmRlVSU0m46F1YszgccR3BJMFQybeImIhqdI3taNuY-FOa_ChDX5bklaG1WQmffIJ03icDuTyDKF01ggvTHGVcch/https%3A%2F%2Fwww.ovirt.org%2Fcommunity%2Fabout%2Fcommunity-guidelines%2F List Archives: https://secure-web.cisco.com/1KzN5tLsiJy3-g9yYcOx5d8B1oOe9PrxbORUgNL0FrN3Wvl5CEeqieipEL02acXE_4xoVFlh9qNWma7TL5fiyZ_BYZVpwjI1YBblytzfxp8miWnUxBOPF-LCiIWSpuNZfViTu06wbxDXH6dqwHqBe0-qHjr_XVX0F5KLEr88UBTyiU-9gbF1qGE9sZ6bT7LIF67AGb_t-wGaL74q8egeIIjYYtZRn9HSt0cvNZ4iXWPEusWMkRbtglg4SCIR8K7qKcw2qXXkI4ilBHrKsXNMTEvzl7mnJ8SE-BbQWMDdF0erjvgnYJ93eywGsEGPxu_mrBeLEBX8woxDnZMLBlqL4v8r-pqoSamCBqkVlEQFjcP4cUZzXy4itzHRjPeg9WIdk3ca0opxWho8KatLWRGFqP09TaJ8Jq707hjDNDhCWn7f8RP9xIgobuXdo_bRyY3K4/https%3A%2F%2Flists.ovirt.org%2Farchives%2Flist%2Fusers%40ovirt.org%2Fmessage%2F4PKJSVDIH3V4H7Q2RKS2C4ZUMWDODQY6%2F _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://secure-web.cisco.com/1ubMaXUij250PN8zKVQvmo6NUYWPOdVDirkU4lwkRkpCkQix6ZJlGJiEF1lWy8_04u2Ems0FwTKbgPFhm06jfILR59nJDNUIiCeN5YkYj0RU-r9UbaWrCmz_uLZJISuevoC0ELHCC121je2k5qatuVVcZL3XrG4eOeOFlhAd7riOB_HVcTdkWXGXF5hw6IiQj4E33rY5vEP9waE6nkhZO6bk08CLKlYrPyVF0o8d1-X8ntzhjWIE311h2ZNlu9KFarFqe5cckSGvVh1UiHQ3AKuBPZAvPKIH7KXsL6iFBNG-pJm-uVP27ZUnoeEQaG8kAVc6jW43e7fxfBUvrzmiFlQyD2o3HBrNNlbtHGjYU5Wy3Ao2H09QtCReoIaypCYbwS6Di3wqgY0lNuxB7swSo1vziW4Uez_j5sRmSl43UgIXzzjoeu4gWwRyfeteXo88x/https%3A%2F%2Fwww.ovirt.org%2Fsite%2Fprivacy-policy%2F oVirt Code of Conduct: https://secure-web.cisco.com/1HjeIIkwx_NRkoCsnonfHu87z-MFaPfE3HOMBJ02Mzwyj-9AxzEENIuSMb_cTt98gAuZrWnSWq26-hUbz4lqcziFPWDUWOpWeYyBfQFYYld79cH960SfEhrOi44Gl9GDPCs27iXPJ1Kpxbp0t3iyi0HmC9QqLoXswWm8sIRPgvg4g1q9sSRKmrTyqylP8-MEETXdMXW-SwYeQT0I-_w1GH9VHOuy6cYf8bqaAwYFtAQ_TDrJX0atMmNh1bqDF3BLKxeXePEZCwqondC9a5ovB9-FzZcpUHrT4YK6gOIng55mdlAj6j-6lyw9N5gNXtfz9oq5DH78nE15q6iFyyEVG58pbrUje45FJdy9WsRRvNttcFbzgtb5E5-RtoFgdIYf5fJfchr0o1NVNHWpb1beyhLeM8_fS1Pzy-Fo8m0r_ZcYtOQ1WOdfE5fs5QRz2UVVZ/https%3A%2F%2Fwww.ovirt.org%2Fcommunity%2Fabout%2Fcommunity-guidelines%2F List Archives: https://secure-web.cisco.com/1XcKrt1wH3y9o2mcDXqQa9v-MXc1VugRHkrHz1HJwNk-1Mv89pcENMjNLP_TEZ99urLjX0r-Njjx1RP-mFIsJ_OOvLqjsx1fHATqYdaQf4kPCSF2q9mQeT69waxxb6pMsr12XPMv8rLYkx4aW2OstK2D-qPwT8zq5VxhKu-BnlokI1iS8eE08BgzugQl5Z471i_6Huk6h9jYCYxWW7lPu3OMBmRtlsV_wIshfnu1Cslu_sAOh_44DsDsfPswlNOHzVWX7bS67AKwhr7Ic-QUeew3FJXQL_JnMXZstYWxXgZhK48wgp1CNMhjhva4OiBwm6eKnvMjB6_IYQQSbjO0qg9MHHQQ01BfgJmM2uWLdzAeK7e7S_JKPndAQVyg2LOhECm4JX8GEUEC6a4zM1WbXoA-Zp-vUvOMfzeM-DWmIvqfyojx-yRwDcI8r7HtcJvhE/https%3A%2F%2Flists.ovirt.org%2Farchives%2Flist%2Fusers%40ovirt.org%2Fmessage%2FDYFZAC4BPJNGZP3PEZ6ZP2AB3C3JVAFM%2F _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://secure-web.cisco.com/1nEZk1HmmOVHI5jAXqISs80YRB-SbMsrunxvZ9XJpVCVvb-FUkEv84wZRHCniIFIRkCZMgrF8cVcLUivR_ZOhnC9oUOUtjawCGGbA1Q9QqcpwjmlA0nNn94klqBKhlSe4JqC1JZ_EIuEiJsTiuSe4-H4T43pXcHw7mfIht3pbxvMh8hIFExJqH-Uwh4PQcur4ostxaolE2sVaBJUt4jJXCnDW8M-I_vJ72Diy6queQsiHPkXt-i0X1kiFYwNDSdyaeOAZHz7SSSgp7tz0AOy9a9412bEaP_xR_tmLg8OrmR_nut2slftsxTbPngFDCCYC7sYMbCy6XUrJ_yTlQAKMIvNwsO-E1xTfUvzSazxTEOOVht_CW3bysC04V2yn2P0OUuFVGRV2TH1ZRRFF0B3dEcPV7N47DeB66NbnigXY9_V-krcg4AfxfgvdEVkVkjcM/https%3A%2F%2Fwww.ovirt.org%2Fsite%2Fprivacy-policy%2F oVirt Code of Conduct: https://secure-web.cisco.com/12Y4LEt9YRdw8K3OrOj6uPi27aZWSQ67hHP7OnuT4WeYOwwFDt4a48qNVpqj7GEge08wrxcMnxuDXjlHRA7tp5VzyRHUvAK_utFxTlbMXJKcQ-UxYr7uukErJFyOtxvKAGa0iCj3ApJNw8faeL9xWpw5ZQtRgwXgfQSagS3HNyUgQbW5E9hdejIq-L7BMncpk8Gy_HpB-YWsL9H4txk_F5GOOSmRaQMfOr8yPB8T9nKoFBW1xJwK6cSWH5qFNCEnCNjT_Y1LpbLNNuXJnnz2GV8wltwE0wutQ2Eszp4y9tMnXxKCSxtggtfYq9UMB0ubE0bjmlfVSb7KgFNg1ducA3DhCB4fuioSyiuhLwkkmwmRlVSU0m46F1YszgccR3BJMFQybeImIhqdI3taNuY-FOa_ChDX5bklaG1WQmffIJ03icDuTyDKF01ggvTHGVcch/https%3A%2F%2Fwww.ovirt.org%2Fcommunity%2Fabout%2Fcommunity-guidelines%2F List Archives: https://secure-web.cisco.com/1VHG2GaLSYhMlJRaVcbioIc6zcifQKoE2LPKzsOAGjti4Ap5QZ3b3WdbQuF0X8M-cLTB4lZl0bRSDya58Xo9_Rlnip9HKy0kS57ryEHpoSdfqpK7k6LgS4OIxool9RR0pABjePLUFayXBXJHa5MngW_G0YRdOJTXqFZDAOY9R6RH0C6ONqJYfMnNV50WAkL0_cxu2GbfHUtgkHO6BPVtaenTbdY0Sao6z0BDS2L_2zIxKbdBF8AnmAWjZGcdqLPIZG4LqKwzg_fA3WbmqGexOXtUDRBOPM0QK6EXiZBEJkrmdFi1o1P6Fx_3U1Bx8_-3tTzlA6JI97bLAx2hI8R_1dMj-NIU6IcowP0VIJKdL2ZwPSRnB2Hp_fEsL-SHMoXhAUkdeks6_5iHaL0IEdJDys6jYTI2Tl2JNY5inTSl2imNzERYoytpX0VJlyv-r0OjX/https%3A%2F%2Flists.ovirt.org%2Farchives%2Flist%2Fusers%40ovirt.org%2Fmessage%2FO2WI5F77DVZ3HZORNNXOIWFLMP4GYERV%2F_______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://secure-web.cisco.com/1nEZk1HmmOVHI5jAXqISs80YRB-SbMsrunxvZ9XJpVCVvb-FUkEv84wZRHCniIFIRkCZMgrF8cVcLUivR_ZOhnC9oUOUtjawCGGbA1Q9QqcpwjmlA0nNn94klqBKhlSe4JqC1JZ_EIuEiJsTiuSe4-H4T43pXcHw7mfIht3pbxvMh8hIFExJqH-Uwh4PQcur4ostxaolE2sVaBJUt4jJXCnDW8M-I_vJ72Diy6queQsiHPkXt-i0X1kiFYwNDSdyaeOAZHz7SSSgp7tz0AOy9a9412bEaP_xR_tmLg8OrmR_nut2slftsxTbPngFDCCYC7sYMbCy6XUrJ_yTlQAKMIvNwsO-E1xTfUvzSazxTEOOVht_CW3bysC04V2yn2P0OUuFVGRV2TH1ZRRFF0B3dEcPV7N47DeB66NbnigXY9_V-krcg4AfxfgvdEVkVkjcM/https%3A%2F%2Fwww.ovirt.org%2Fsite%2Fprivacy-policy%2F oVirt Code of Conduct: https://secure-web.cisco.com/12Y4LEt9YRdw8K3OrOj6uPi27aZWSQ67hHP7OnuT4WeYOwwFDt4a48qNVpqj7GEge08wrxcMnxuDXjlHRA7tp5VzyRHUvAK_utFxTlbMXJKcQ-UxYr7uukErJFyOtxvKAGa0iCj3ApJNw8faeL9xWpw5ZQtRgwXgfQSagS3HNyUgQbW5E9hdejIq-L7BMncpk8Gy_HpB-YWsL9H4txk_F5GOOSmRaQMfOr8yPB8T9nKoFBW1xJwK6cSWH5qFNCEnCNjT_Y1LpbLNNuXJnnz2GV8wltwE0wutQ2Eszp4y9tMnXxKCSxtggtfYq9UMB0ubE0bjmlfVSb7KgFNg1ducA3DhCB4fuioSyiuhLwkkmwmRlVSU0m46F1YszgccR3BJMFQybeImIhqdI3taNuY-FOa_ChDX5bklaG1WQmffIJ03icDuTyDKF01ggvTHGVcch/https%3A%2F%2Fwww.ovirt.org%2Fcommunity%2Fabout%2Fcommunity-guidelines%2F List Archives: https://secure-web.cisco.com/1Ww442-q18Ni2CsDxBntfOz7M8DrE6wAzkG6fJ0CeCJ_JjiG2Jj2sDnREIN6c36XZohKFl2t_TIKfmUO6cLx_zLuRJyuOuEtQlNX99ZDXvPW_x8GYCfgKkagx6orVXSs4NjUgrs-ItYc3HRwYLWsDE16v7wf67n2CyDQpYOWFsoy3uGQWd0wYZeAEurFTp1uKTXAqh4XGNKZ-cX0vyh9tNobtVdG8sjQIgFzeJi4U-dVPWftzGqTTESrDvVspHx8gS8bl2IVl9J_8EqKn4ap82ntaNqUm9ZcXW0L9mLBhtKSYuQbwGavA9Bkth0zMWwMLhnTtgAe2SmmqFWyycqCnod5r9Ucwcazzi-Qkz-w6kK0EEVkoXSft7HR45sL1wS2P1v27x2ilToTZ50SJbxrI-j1ZjOWv5_yEwLbtREqMk5CLg2g20_V4JsjphsXKYrkK/https%3A%2F%2Flists.ovirt.org%2Farchives%2Flist%2Fusers%40ovirt.org%2Fmessage%2FUG7XXWITXUUJYMAXWYZUXJFQGSYWRND6%2F
_______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/XV36WWSSWYQEOPDDJMIHLXIPWO5DC6DO/
