Re: [GENERAL] Huge Pages - setting the right value

[pinker]

standard hugepages, transparent are disabled.
They were set exactly following the procedure from postgres documentation.



--
View this message in context: 
http://www.postgresql-archive.org/Huge-Pages-setting-the-right-value-tp5952972p5966064.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Huge Pages - setting the right value

[Andrew Kerber]

Yes, those should always be disabled using tuned or other methods. Using
tuned is described here (second method).  I think the grub.conf method
described is unique to RHEL/OEL/CENTOS.
http://houseofbrick.com/disabling-transparent-hugepages-using-tuned/



On Sun, Jun 11, 2017 at 5:00 PM, Lucas Possamai 
wrote:

>
>
> 2017-06-12 9:52 GMT+12:00 Andrew Kerber :
>
>> Was that transparent hugepages or standard hugepages?  databases commonly
>> have problems dealing with transparent hugepages.
>>
>>
>
>
> IN my case, it was the Transparent Hugepages
>
>
> Lucas
>



-- 
Andrew W. Kerber

'If at first you dont succeed, dont take up skydiving.'

Re: [GENERAL] Huge Pages - setting the right value

[Lucas Possamai]

2017-06-12 9:52 GMT+12:00 Andrew Kerber :

> Was that transparent hugepages or standard hugepages?  databases commonly
> have problems dealing with transparent hugepages.
>
>


IN my case, it was the Transparent Hugepages


Lucas

Re: [GENERAL] Huge Pages - setting the right value

[Andrew Kerber]

Was that transparent hugepages or standard hugepages?  databases commonly
have problems dealing with transparent hugepages.

On Sun, Jun 11, 2017 at 4:39 PM, Lucas Possamai 
wrote:

> 2017-06-12 7:52 GMT+12:00 Andrew Kerber :
>
>> I am sure it does not.
>>
>> Sent from my iPhone
>>
>> > On Jun 11, 2017, at 10:50 AM, pinker  wrote:
>> >
>> > Andrew Kerber wrote
>> >> I can't give you an absolutely authoritative answer, but because of the
>> >> way hugepages are implemented and allocated, I can't think how they
>> could
>> >> be used for other processes.  Linux hugepages are either 2m or 1g, far
>> too
>> >> large for any likely processes to require. They cannot be allocated in
>> >> partial pages.
>> >
>> > thank you for your help.
>> > My system is using 2MB pages for shared buffers. I have checked and one
>> of
>> > my processes has used 606788kB of memory, so potentially could use ~ 300
>> > huge pages, but does postgres can use it for non shared memory?
>> >
>> >
>> >
>> >
>> > --
>> > View this message in context: http://www.postgresql-archive.
>> org/Huge-Pages-setting-the-right-value-tp5952972p5965963.html
>> > Sent from the PostgreSQL - general mailing list archive at Nabble.com.
>> >
>> >
>> > --
>> > Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
>> > To make changes to your subscription:
>> > http://www.postgresql.org/mailpref/pgsql-general
>>
>>
>> --
>> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-general
>>
>
>
>
> In my case, we had the HugePages enabled but not configured in our Master
> DB Server. When we increased the server resources (More RAM & CPU) we had
> lots of issues with HugePages. Specially I/O ones. Had to disabled it.
>
> Running Ubuntu 14.04 Server @ Amazon.
>
>
> Lucas
>
>


-- 
Andrew W. Kerber

'If at first you dont succeed, dont take up skydiving.'

Re: [GENERAL] Huge Pages - setting the right value

[Lucas Possamai]

2017-06-12 7:52 GMT+12:00 Andrew Kerber :

> I am sure it does not.
>
> Sent from my iPhone
>
> > On Jun 11, 2017, at 10:50 AM, pinker  wrote:
> >
> > Andrew Kerber wrote
> >> I can't give you an absolutely authoritative answer, but because of the
> >> way hugepages are implemented and allocated, I can't think how they
> could
> >> be used for other processes.  Linux hugepages are either 2m or 1g, far
> too
> >> large for any likely processes to require. They cannot be allocated in
> >> partial pages.
> >
> > thank you for your help.
> > My system is using 2MB pages for shared buffers. I have checked and one
> of
> > my processes has used 606788kB of memory, so potentially could use ~ 300
> > huge pages, but does postgres can use it for non shared memory?
> >
> >
> >
> >
> > --
> > View this message in context: http://www.postgresql-archive.
> org/Huge-Pages-setting-the-right-value-tp5952972p5965963.html
> > Sent from the PostgreSQL - general mailing list archive at Nabble.com.
> >
> >
> > --
> > Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-general
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>



In my case, we had the HugePages enabled but not configured in our Master
DB Server. When we increased the server resources (More RAM & CPU) we had
lots of issues with HugePages. Specially I/O ones. Had to disabled it.

Running Ubuntu 14.04 Server @ Amazon.


Lucas

Re: [GENERAL] Huge Pages - setting the right value

[Andrew Kerber]

I am sure it does not. 

Sent from my iPhone

> On Jun 11, 2017, at 10:50 AM, pinker  wrote:
> 
> Andrew Kerber wrote
>> I can't give you an absolutely authoritative answer, but because of the
>> way hugepages are implemented and allocated, I can't think how they could
>> be used for other processes.  Linux hugepages are either 2m or 1g, far too
>> large for any likely processes to require. They cannot be allocated in
>> partial pages.
> 
> thank you for your help. 
> My system is using 2MB pages for shared buffers. I have checked and one of
> my processes has used 606788kB of memory, so potentially could use ~ 300
> huge pages, but does postgres can use it for non shared memory?
> 
> 
> 
> 
> --
> View this message in context: 
> http://www.postgresql-archive.org/Huge-Pages-setting-the-right-value-tp5952972p5965963.html
> Sent from the PostgreSQL - general mailing list archive at Nabble.com.
> 
> 
> -- 
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Huge Pages - setting the right value

[pinker]

Andrew Kerber wrote
> I can't give you an absolutely authoritative answer, but because of the
> way hugepages are implemented and allocated, I can't think how they could
> be used for other processes.  Linux hugepages are either 2m or 1g, far too
> large for any likely processes to require. They cannot be allocated in
> partial pages.

thank you for your help. 
My system is using 2MB pages for shared buffers. I have checked and one of
my processes has used 606788kB of memory, so potentially could use ~ 300
huge pages, but does postgres can use it for non shared memory?




--
View this message in context: 
http://www.postgresql-archive.org/Huge-Pages-setting-the-right-value-tp5952972p5965963.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Huge Pages - setting the right value

[Andrew Kerber]

I can't give you an absolutely authoritative answer, but because of the way 
hugepages are implemented and allocated, I can't think how they could be used 
for other processes.  Linux hugepages are either 2m or 1g, far too large for 
any likely processes to require. They cannot be allocated in partial pages.

Sent from my iPad

> On Jun 11, 2017, at 10:02 AM, pinker  wrote:
> 
> We are experiencing some performance issues because of high CPU load. So I
> would like to ask one more time. The exact question is:
> Does PostgreSQL can use huge pages for processes or only for shared buffers?
> (Does it make any sense to set the number of huge pages above the
> shared_buffers?)
> Any help or suggestions would be much appreciated! 
> 
> 
> 
> --
> View this message in context: 
> http://www.postgresql-archive.org/Huge-Pages-setting-the-right-value-tp5952972p5965956.html
> Sent from the PostgreSQL - general mailing list archive at Nabble.com.
> 
> 
> -- 
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Huge Pages - setting the right value

[pinker]

We are experiencing some performance issues because of high CPU load. So I
would like to ask one more time. The exact question is:
Does PostgreSQL can use huge pages for processes or only for shared buffers?
(Does it make any sense to set the number of huge pages above the
shared_buffers?)
Any help or suggestions would be much appreciated! 



--
View this message in context: 
http://www.postgresql-archive.org/Huge-Pages-setting-the-right-value-tp5952972p5965956.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Huge Pages - setting the right value

[pinker]

W dniu 2017-03-30 11:45:55 użytkownik pinker  napisał:
> Hi,
> I'm currently testing performance with and without huge pages. Documentation
> says that in order to estimate the number of huge pages needed one should
> check the postmaster's VmPeak value. I wonder if it's only postmaster memory
> usage what's matters? Or I could get better estimation from the most memory
> intensive postgres process - not necessarly postmaster? I'm using following
> command to check it:
> for i in $(ps -ef | grep postgres|awk '{print $2}'); do grep ^VmPeak
> /proc/${i}/status|awk '{print $2}' >> log; done; sort -n -r log | head -1
> 
> I'm asking because some other process takes 606788kB while postmaster only
> 280444kB.
> 
> 
> 
> --
> View this message in context: 
> http://www.postgresql-archive.org/Huge-Pages-setting-the-right-value-tp5952972.html
> Sent from the PostgreSQL - general mailing list archive at Nabble.com.
> 
> 
> -- 
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
> 


or maybe sum of all processes?
I assume that memory allocated by postmaster means shared buffers, so if one 
wants to huge pages beeing used for sorting as well then should set some bigger 
number of huge pages in the kernel? Is it a right assumption?





-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general