Re: [ovirt-users] Request for feedback on your db vacuum status

2016-12-13 Thread Darrell Budic 
Whups, bad reply to, no problem copying the list.

It's a self hosted system, currently with two clusters and 9 active hosts.74 
VMs, yes. It’s had a few more clusters over time, and a few more hosts, 
including some removed and re-added when upgrading from centos 6 to 7. But only 
9 at the moment. One cluster of 6 with most of the vms, using external Gluster 
servers not managed by ovirt, and one cluster of 3 which are also gluster 
servers as well as hypervisors. 7 vms currently on the smaller cluster.


> On Dec 13, 2016, at 3:36 AM, Roy Golan  wrote:
> On 12 December 2016 at 20:31, Darrell Budic  > wrote:
> Here’s mine: http://paste.fedoraproject.org/505443/14815674/ 
> 
> 
> This engine has been around since at 3.3, maybe 3.2, currently on 3.6 because 
> I haven’t had time to arrange the OS upgrade from centos 6 to 7 for the 
> engine host yet.
> 
> 
> Thank you very much Darrell! your vacuum seems boring (good!) and the db 
> seems healthy.  Can you reply to the list that you sent the feedback so 
> everyone will have a chance to look at it? also how big is your setup? 
> judging by the output is it 74 vms and 42 hosts?  
> 
>> On Dec 8, 2016, at 8:18 AM, Roy Golan > > wrote:
>> 
>> Hi all,
>> 
>> Following the thread about vacuum tool [1] I would like to gather some 
>> feedback about your deployment's db vacuum status The info is completely 
>> anonymous and function running it is a read only reporting one and should 
>> have little or no effect on the db.
>> 
>> The result can be pretty verbose  but again will not disclose sensitive 
>> info. Anyway review it before pasting it. It should look something like 
>> that(a snippet of one table):
>> 
>> INFO:  vacuuming "pg_catalog.pg_ts_template"
>> INFO:  index "pg_ts_template_tmplname_index" now contains 5 row versions in 
>> 2 pages
>> DETAIL:  0 index row versions were removed.
>> 0 index pages have been deleted, 0 are currently reusable.
>> CPU 0.00s/0.00u sec elapsed 0.00 sec.
>> 
>> 
>> 1. sudo su - postgres  -c "psql engine -c 'vacuum verbose'" &> 
>> /tmp/vacuum.log
>> 
>> 2. review the /tmp/vacuum.log
>> 
>> 3. paste it to http://paste.fedoraproject.org/ 
>>  and reply with the link here
>> 
>> 
>> [1] http://lists.ovirt.org/pipermail/devel/2016-December/014484.html 
>> 
>> 
>> 
>> Thanks,
>> Roy
>> ___
>> Users mailing list
>> Users@ovirt.org 
>> http://lists.ovirt.org/mailman/listinfo/users 
>> 
> 
> 

___
Users mailing list
Users@ovirt.org
http://lists.phx.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Request for feedback on your db vacuum status

2016-12-12 Thread Baptiste Agasse 
Hi, 

- Le 12 Déc 16, à 13:54, Shirly Radco  a écrit : 

> Hi Baptiste,

> Thank you very much for your reply.

> I understand that you updated your DWH to collect every 60 seconds instead of
> 20.
> I'm the oVirt DWH maintainer and I would really appreciate if you can share 
> what
> led you to this decision?
> And some details on your setup.

> Do you have it installed on the same machine as the engine or on a remote one?
> Is your database remote or local?
> What is the scale of you environment ? Number of hosts/vms...

> This will may help us with the bug Roy mentioned.

>From my mind, it was the ovirt_engine_history DB, i don't remember if there 
>was one or more tables that reported a lot of disk space usage. A full vacuum 
>corrected this size issue. 
For the bugzilla mentioned, i saw it and i applied the sampling suggestion to 
see if the DB grows more slowly. 

For our environment we have today (and growing) 
* 4 DC 
* 5 Clusters 
* 9 Storages domains (iscsi) 
* About 360 virtual disks in storage domains 
* 13 Hosts (growing) 
* About 250 VMs (growing) 

* The engine + DWH + DB server are all on the same server (hosted engine) 
* DB Size is about 3.2 GB (after the vacuum) 
* As all was on the same box, the engine setup via appliance was preferred and 
it was not possible to customize the size of the appliance at install/update, 
we wanted to keep the DB size as small as possible, but with some history. I 
saw that the engine appliance size will be customizable soon, so we will maybe 
extend the engine disk at update and keep a little bit more history or decrease 
the sampling interval again. 

Have a nice day. 

Regards. 

> Best regards,
> Shirly Radco
> BI Software Engineer
> Red Hat Israel Ltd.
> 34 Jerusalem Road
> Building A, 4th floor
> Ra'anana, Israel 4350109

> On Thu, Dec 8, 2016 at 5:01 PM, Baptiste Agasse <
> baptiste.aga...@lyra-network.com > wrote:

>> - Le 8 Déc 16, à 15:18, Roy Golan < rgo...@redhat.com > a écrit :

>>> Hi all,

>>> Following the thread about vacuum tool [1] I would like to gather some 
>>> feedback
>>> about your deployment's db vacuum status The info is completely anonymous 
>>> and
>>> function running it is a read only reporting one and should have little or 
>>> no
>>> effect on the db.

>>> The result can be pretty verbose but again will not disclose sensitive info.
>>> Anyway review it before pasting it. It should look something like that(a
>>> snippet of one table):

>>> INFO: vacuuming "pg_catalog.pg_ts_template"
>>> INFO: index "pg_ts_template_tmplname_index" now contains 5 row versions in 2
>>> pages
>>> DETAIL: 0 index row versions were removed.
>>> 0 index pages have been deleted, 0 are currently reusable.
>>> CPU 0.00s/0.00u sec elapsed 0.00 sec.

>>> 1. sudo su - postgres -c "psql engine -c 'vacuum verbose'" &> 
>>> /tmp/vacuum.log

>>> 2. review the /tmp/vacuum.log

>>> 3. paste it to http://paste.fedoraproject.org/ and reply with the link here

>>> [1] http://lists.ovirt.org/pipermail/devel/2016-December/014484.html

>>> Thanks,
>>> Roy

>>> ___
>>> Users mailing list
>>> Users@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users

>> http://paste.fedoraproject.org/501769/48120789/

>> But, we run a full vacuum about one month ago that have free about 8GB of 
>> space
>> and we set DWH_SAMPLING=60 to decrease data size of DWH (install is ~ 1y and
>> half old, updated from 3.5 to 3.6 to 4.0).

>> Have a nice day.

>> Regards.

>> --
>> Baptiste

>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users

-- 
Baptiste 
___
Users mailing list
Users@ovirt.org
http://lists.phx.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Request for feedback on your db vacuum status

2016-12-12 Thread Shirly Radco 
Hi Baptiste,

Thank you very much for your reply.

I understand that you updated your DWH to collect every 60 seconds instead
of 20.
I'm the oVirt DWH maintainer and I would really appreciate if you can share
what led you to this decision?
And some details on your setup.

Do you have it installed on the same machine as the engine or on a remote
one?
Is your database remote or local?
What is the scale of you environment ? Number of hosts/vms...

This will may help us with the bug Roy mentioned.



Best regards,

Shirly Radco

BI Software Engineer
Red Hat Israel Ltd.
34 Jerusalem Road
Building A, 4th floor
Ra'anana, Israel 4350109


On Thu, Dec 8, 2016 at 5:01 PM, Baptiste Agasse <
baptiste.aga...@lyra-network.com> wrote:

>
>
> - Le 8 Déc 16, à 15:18, Roy Golan  a écrit :
>
> Hi all,
>
> Following the thread about vacuum tool [1] I would like to gather some
> feedback about your deployment's db vacuum status The info is completely
> anonymous and function running it is a read only reporting one and should
> have little or no effect on the db.
>
> The result can be pretty verbose  but again will not disclose sensitive
> info. Anyway review it before pasting it. It should look something like
> that(a snippet of one table):
>
> INFO:  vacuuming "pg_catalog.pg_ts_template"
> INFO:  index "pg_ts_template_tmplname_index" now contains 5 row versions
> in 2 pages
> DETAIL:  0 index row versions were removed.
> 0 index pages have been deleted, 0 are currently reusable.
> CPU 0.00s/0.00u sec elapsed 0.00 sec.
>
>
> 1. sudo su - postgres  -c "psql engine -c 'vacuum verbose'" &>
> /tmp/vacuum.log
>
> 2. review the /tmp/vacuum.log
>
> 3. paste it to http://paste.fedoraproject.org/ and reply with the link
> here
>
>
> [1] http://lists.ovirt.org/pipermail/devel/2016-December/014484.html
>
>
> Thanks,
> Roy
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
> http://paste.fedoraproject.org/501769/48120789/
>
> But, we run a full vacuum about one month ago that have free about 8GB of
> space and we set DWH_SAMPLING=60 to decrease data size of DWH (install is ~
> 1y and half old, updated from 3.5 to 3.6 to 4.0).
>
> Have a nice day.
>
> Regards.
>
> --
> Baptiste
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@ovirt.org
http://lists.phx.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Request for feedback on your db vacuum status

2016-12-12 Thread Roy Golan 
On 8 December 2016 at 17:01, Baptiste Agasse <
baptiste.aga...@lyra-network.com> wrote:

>
>
> - Le 8 Déc 16, à 15:18, Roy Golan  a écrit :
>
> Hi all,
>
> Following the thread about vacuum tool [1] I would like to gather some
> feedback about your deployment's db vacuum status The info is completely
> anonymous and function running it is a read only reporting one and should
> have little or no effect on the db.
>
> The result can be pretty verbose  but again will not disclose sensitive
> info. Anyway review it before pasting it. It should look something like
> that(a snippet of one table):
>
> INFO:  vacuuming "pg_catalog.pg_ts_template"
> INFO:  index "pg_ts_template_tmplname_index" now contains 5 row versions
> in 2 pages
> DETAIL:  0 index row versions were removed.
> 0 index pages have been deleted, 0 are currently reusable.
> CPU 0.00s/0.00u sec elapsed 0.00 sec.
>
>
> 1. sudo su - postgres  -c "psql engine -c 'vacuum verbose'" &>
> /tmp/vacuum.log
>
> 2. review the /tmp/vacuum.log
>
> 3. paste it to http://paste.fedoraproject.org/ and reply with the link
> here
>
>
> [1] http://lists.ovirt.org/pipermail/devel/2016-December/014484.html
>
>
> Thanks,
> Roy
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
> http://paste.fedoraproject.org/501769/48120789/
>
> But, we run a full vacuum about one month ago that have free about 8GB of
> space and we set DWH_SAMPLING=60 to decrease data size of DWH (install is ~
> 1y and half old, updated from 3.5 to 3.6 to 4.0).
>
> Have a nice day.
>
> Regards.
>
> --
> Baptiste
>
>
Baptiste thank you very much for submitting this, very helpful.
I wonder if it was the engine or dwh db who took most of the 8gb.
FYI there is a discussion on increasing the interval to 60 on  *Bug 1395608*
 - DWH sampling is too
high
___
Users mailing list
Users@ovirt.org
http://lists.phx.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Request for feedback on your db vacuum status

2016-12-08 Thread Baptiste Agasse 
- Le 8 Déc 16, à 15:18, Roy Golan  a écrit : 

> Hi all,

> Following the thread about vacuum tool [1] I would like to gather some 
> feedback
> about your deployment's db vacuum status The info is completely anonymous and
> function running it is a read only reporting one and should have little or no
> effect on the db.

> The result can be pretty verbose but again will not disclose sensitive info.
> Anyway review it before pasting it. It should look something like that(a
> snippet of one table):

> INFO: vacuuming "pg_catalog.pg_ts_template"
> INFO: index "pg_ts_template_tmplname_index" now contains 5 row versions in 2
> pages
> DETAIL: 0 index row versions were removed.
> 0 index pages have been deleted, 0 are currently reusable.
> CPU 0.00s/0.00u sec elapsed 0.00 sec.

> 1. sudo su - postgres -c "psql engine -c 'vacuum verbose'" &> /tmp/vacuum.log

> 2. review the /tmp/vacuum.log

> 3. paste it to http://paste.fedoraproject.org/ and reply with the link here

> [1] http://lists.ovirt.org/pipermail/devel/2016-December/014484.html

> Thanks,
> Roy

> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

http://paste.fedoraproject.org/501769/48120789/ 

But, we run a full vacuum about one month ago that have free about 8GB of space 
and we set DWH_SAMPLING=60 to decrease data size of DWH (install is ~ 1y and 
half old, updated from 3.5 to 3.6 to 4.0). 

Have a nice day. 

Regards. 

-- 
Baptiste 
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Request for feedback on your db vacuum status

2016-12-08 Thread Roy Golan 
Hi all,

Following the thread about vacuum tool [1] I would like to gather some
feedback about your deployment's db vacuum status The info is completely
anonymous and function running it is a read only reporting one and should
have little or no effect on the db.

The result can be pretty verbose  but again will not disclose sensitive
info. Anyway review it before pasting it. It should look something like
that(a snippet of one table):

INFO:  vacuuming "pg_catalog.pg_ts_template"
INFO:  index "pg_ts_template_tmplname_index" now contains 5 row versions in
2 pages
DETAIL:  0 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.


1. sudo su - postgres  -c "psql engine -c 'vacuum verbose'" &>
/tmp/vacuum.log

2. review the /tmp/vacuum.log

3. paste it to http://paste.fedoraproject.org/ and reply with the link here


[1] http://lists.ovirt.org/pipermail/devel/2016-December/014484.html


Thanks,
Roy
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users