Hi,
On 03/03/14 21:03, Heikki Linnakangas wrote:
> I spotted this in section "17.4.1 Shared Memory and Semaphores":
>
> >Linux
> >
> >The default maximum segment size is 32 MB, and the default maximum total
> > size is 2097152 pages. A page is almost always 4096 bytes except in unusual
> >
On 03/03/2014 11:34 AM, Christian Kruse wrote:
Hi,
Attached is a patch with the updated documentation (now uses
consistently huge pages) as well as a renamed GUC, consistent wording
(always use huge pages) as well as renamed variables.
Hmm, I wonder if that could now be misunderstood to have
Hi,
On 28/02/14 17:58, Peter Geoghegan wrote:
> On Fri, Feb 28, 2014 at 9:43 AM, Heikki Linnakangas
> wrote:
> > Hmm, I wonder if that could now be misunderstood to have something to do
> > with the PostgreSQL page size? Maybe add the word "memory" or "operating
> > system" in the first sentence
Hi,
> >Attached is a patch with the updated documentation (now uses
> >consistently huge pages) as well as a renamed GUC, consistent wording
> >(always use huge pages) as well as renamed variables.
>
> Hmm, I wonder if that could now be misunderstood to have something to do
> with the PostgreSQL
On Fri, Feb 28, 2014 at 9:43 AM, Heikki Linnakangas
wrote:
> Hmm, I wonder if that could now be misunderstood to have something to do
> with the PostgreSQL page size? Maybe add the word "memory" or "operating
> system" in the first sentence in the docs, like this: "Enables/disables the
> use of hu
On 02/27/2014 09:34 AM, Christian Kruse wrote:
Hi,
On 26/02/14 13:13, Alvaro Herrera wrote:
There's one thing that rubs me the wrong way about all this
functionality, which is that we've named it "huge TLB pages". That is
wrong -- the TLB pages are not huge. In fact, as far as I understand,
Hi,
On 27/02/14 08:35, Christian Kruse wrote:
> Hi Peter,
Sorry, Stephen of course – it was definitely to early.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
pgpm6lYpan4Df.pgp
Description: PGP signatur
Hi Peter,
thank you for your nice words, much appreciated. I'm sorry that I was
so whiny about this in the last post.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
pgpxVj8SJRDQS.pgp
Description: PGP sign
Hi,
On 26/02/14 13:13, Alvaro Herrera wrote:
>
> There's one thing that rubs me the wrong way about all this
> functionality, which is that we've named it "huge TLB pages". That is
> wrong -- the TLB pages are not huge. In fact, as far as I understand,
> the TLB doesn't have pages at all. It's
Christian,
Thanks for working on all of this and dealing with the requests for
updates and changes, as well as for dealing very professionally with an
inappropriate and incorrect remark. Unfortunately, mailing lists can
make communication difficult and someone's knee-jerk reaction (not
referring
On 02/26/2014 06:13 PM, Alvaro Herrera wrote:
There's one thing that rubs me the wrong way about all this
functionality, which is that we've named it "huge TLB pages". That is
wrong -- the TLB pages are not huge. In fact, as far as I understand,
the TLB doesn't have pages at all. It's the pag
There's one thing that rubs me the wrong way about all this
functionality, which is that we've named it "huge TLB pages". That is
wrong -- the TLB pages are not huge. In fact, as far as I understand,
the TLB doesn't have pages at all. It's the pages that are huge, but
those pages are not TLB pa
Hi,
On 26/02/14 14:34, Heikki Linnakangas wrote:
> That still says "The setting is ignored on other systems". That's not quite
> true: as explained later in the section, if you set huge_tlb_pages=on and
> the platform doesn't support it, the server will refuse to start.
I added a sentence about i
On 02/26/2014 10:35 AM, Christian Kruse wrote:
On 25/02/14 19:28, Andres Freund wrote:
I think all that's needed is to cut the first paragraphs that generally
explain what huge pages are in some detail from the text and make sure
the later paragraphs don't refer to the earlier ones.
Attached y
Hi Peter,
after a night of sleep I'm still not able to swallow the pill. To be
honest I'm a little bit angry about this accusation.
I didn't mean to copy from the Debian wiki and after re-checking the
text again I'm still convinced that I didn't.
Of course the text SAYS something similar, but th
Hi,
On 25/02/14 19:28, Andres Freund wrote:
> I think all that's needed is to cut the first paragraphs that generally
> explain what huge pages are in some detail from the text and make sure
> the later paragraphs don't refer to the earlier ones.
Attached you will find a new version of the patch.
On 2014-02-25 13:21:46 -0500, Tom Lane wrote:
> Bruce Momjian writes:
> > On Tue, Feb 25, 2014 at 12:18:02PM -0500, Peter Eisentraut wrote:
> >> As I mentioned, I would just cut those introductory parts out.
>
> > Should we link to the Debian wiki content?
>
> -1. We generally don't link to our
Bruce Momjian writes:
> On Tue, Feb 25, 2014 at 12:18:02PM -0500, Peter Eisentraut wrote:
>> As I mentioned, I would just cut those introductory parts out.
> Should we link to the Debian wiki content?
-1. We generally don't link to our *own* wiki in our SGML docs, let alone
things that aren't e
On Tue, Feb 25, 2014 at 12:18:02PM -0500, Peter Eisentraut wrote:
> On 2/25/14, 11:08 AM, Christian Kruse wrote:
> > Hi,
> >
> > On 25/02/14 17:01, Andres Freund wrote:
> >>> And if we did, we shouldn't copy it verbatim from the Debian wiki
> >>> without attribution.
> >>
> >> Is it actually? A qu
On 2/25/14, 11:08 AM, Christian Kruse wrote:
> Hi,
>
> On 25/02/14 17:01, Andres Freund wrote:
>>> And if we did, we shouldn't copy it verbatim from the Debian wiki
>>> without attribution.
>>
>> Is it actually? A quick comparison doesn't show that many similarities?
>> Christian?
>
> Not as far
Hi,
On 25/02/14 17:01, Andres Freund wrote:
> > And if we did, we shouldn't copy it verbatim from the Debian wiki
> > without attribution.
>
> Is it actually? A quick comparison doesn't show that many similarities?
> Christian?
Not as far as I know. But of course, as I wrote the text I _also_
(t
Hi,
On 25/02/14 10:29, Peter Eisentraut wrote:
> I don't think we should be explaining the basics of OS memory management
> in our documentation.
Well, I'm confused. I thought that's exactly what has been asked.
> And if we did, we shouldn't copy it verbatim from the Debian wiki
> without attrib
On 2014-02-25 10:29:32 -0500, Peter Eisentraut wrote:
> On 1/30/14, 2:28 AM, Christian Kruse wrote:
> > after I finally got documentation compilation working I updated the
> > patch to be syntactically correct. You will find it attached.
>
> I don't think we should be explaining the basics of OS m
On Tue, Feb 25, 2014 at 10:29 AM, Peter Eisentraut wrote:
> And if we did, we shouldn't copy it verbatim from
> the Debian wiki without attribution.
That is seriously not cool.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hacker
On 1/30/14, 2:28 AM, Christian Kruse wrote:
> after I finally got documentation compilation working I updated the
> patch to be syntactically correct. You will find it attached.
I don't think we should be explaining the basics of OS memory management
in our documentation. And if we did, we should
Hi,
after I finally got documentation compilation working I updated the
patch to be syntactically correct. You will find it attached.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
diff --git a/doc/src/sgml
Hi,
On 29/01/14 22:17, Heikki Linnakangas wrote:
> Thanks. There are more cases of that in InternalIpcMemoryCreate, they ought
> to be fixed as well. And should also grep the rest of the codebase for more
> instances of that. And this needs to be back-patched.
I'm way ahead of you ;-) Working on
On 01/29/2014 09:59 PM, Christian Kruse wrote:
Hi,
On 29/01/14 21:36, Heikki Linnakangas wrote:
[…]
Fix pushed.
You are right. Thanks. But there is another bug, see
<20140128154307.gc24...@defunct.ch>
ff. Attached you will find a patch fixing that.
Thanks. There are more cases of that in
Hi,
On 29/01/14 21:36, Heikki Linnakangas wrote:
> […]
> Fix pushed.
You are right. Thanks. But there is another bug, see
<20140128154307.gc24...@defunct.ch>
ff. Attached you will find a patch fixing that.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQ
On 01/29/2014 09:18 PM, Christian Kruse wrote:
Hi,
On 29/01/14 10:11, Jeff Janes wrote:
I'm getting this warning now with gcc (GCC) 4.4.7:
Interesting. I don't get that warning. But the compiler is (formally)
right.
pg_shmem.c: In function 'PGSharedMemoryCreate':
pg_shmem.c:332: warning: 'a
Hi,
On 29/01/14 10:11, Jeff Janes wrote:
> I'm getting this warning now with gcc (GCC) 4.4.7:
Interesting. I don't get that warning. But the compiler is (formally)
right.
> pg_shmem.c: In function 'PGSharedMemoryCreate':
> pg_shmem.c:332: warning: 'allocsize' may be used uninitialized in this
>
Hi,
On 29/01/14 14:12, Heikki Linnakangas wrote:
> The documentation is still lacking. We should explain somewhere how to set
> nr.hugepages, for example. The "Managing Kernel Resources" section ought to
> mention setting. Could I ask you to work on that, please?
Of course! Attached you will find
On Wed, Jan 29, 2014 at 4:12 AM, Heikki Linnakangas wrote:
> On 01/28/2014 06:11 PM, Christian Kruse wrote:
>
>> Hi,
>>
>> attached you will find a new version of the patch, ported to HEAD,
>> fixed the mentioned bug and - hopefully - dealing the the remaining
>> issues.
>>
>
> Thanks, I have com
On Tue, Jan 28, 2014 at 5:58 AM, Christian Kruse
wrote:
> Hi,
>
> On 28/01/14 13:51, Heikki Linnakangas wrote:
>> Oh darn, I remembered we had already committed this, but clearly not. I'd
>> love to still get this into 9.4. The latest patch (hugepages-v5.patch) was
>> pretty much ready for commit,
On 01/29/2014 04:01 PM, Vik Fearing wrote:
On 01/29/2014 01:12 PM, Heikki Linnakangas wrote:
The documentation is still lacking.
The documentation is indeed lacking since it breaks the build.
doc/src/sgml/config.sgml contains the line
normal allocation if that fails. With on tag.
Trivi
On 01/29/2014 01:12 PM, Heikki Linnakangas wrote:
> On 01/28/2014 06:11 PM, Christian Kruse wrote:
>> Hi,
>>
>> attached you will find a new version of the patch, ported to HEAD,
>> fixed the mentioned bug and - hopefully - dealing the the remaining
>> issues.
>
> Thanks, I have committed this now.
On 01/28/2014 06:11 PM, Christian Kruse wrote:
Hi,
attached you will find a new version of the patch, ported to HEAD,
fixed the mentioned bug and - hopefully - dealing the the remaining
issues.
Thanks, I have committed this now.
The documentation is still lacking. We should explain somewhere
Hi,
attached you will find a new version of the patch, ported to HEAD,
fixed the mentioned bug and - hopefully - dealing the the remaining
issues.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
diff --git a
Hi,
On 15/11/13 15:17, Heikki Linnakangas wrote:
> I spent some time whacking this around, new patch version attached. I moved
> the mmap() code into a new function, that leaves the PGSharedMemoryCreate
> more readable.
I think there's a bug in this version of the patch. Have a look at
this:
+
Hi,
On 28/01/14 13:51, Heikki Linnakangas wrote:
> Oh darn, I remembered we had already committed this, but clearly not. I'd
> love to still get this into 9.4. The latest patch (hugepages-v5.patch) was
> pretty much ready for commit, except for documentation.
I'm working on it. I ported it to HEA
On 01/27/2014 09:20 PM, Alvaro Herrera wrote:
Heikki Linnakangas wrote:
I spent some time whacking this around, new patch version attached.
I moved the mmap() code into a new function, that leaves the
PGSharedMemoryCreate more readable.
Did this patch go anywhere?
Oh darn, I remembered we h
Heikki Linnakangas wrote:
> I spent some time whacking this around, new patch version attached.
> I moved the mmap() code into a new function, that leaves the
> PGSharedMemoryCreate more readable.
Did this patch go anywhere?
Someone just pinged me about a kernel scalability problem in Linux with
At 2013-11-21 22:14:35 +0100, and...@2ndquadrant.com wrote:
>
> I'd certainly want a setting that errors out if it cannot get the
> memory using hugetables.
OK, then the current try/on/off settings are fine.
I'm better today, so I'll read the patch Heikki posted and see what more
needs to be done
On 2013-11-21 16:24:56 -0500, Robert Haas wrote:
> > What about
> > huge_tlb_pages={off,try}
> >
> > Or maybe
> > huge_tlb_pages={off,try,require}
>
> I'd spell "require" as "on", or at least accept that as a synonym.
That's off,try, on is what the patch currently implements, Abhijit just
was arg
On Thu, Nov 21, 2013 at 4:09 PM, Alvaro Herrera
wrote:
> Abhijit Menon-Sen wrote:
>> At 2013-11-15 15:17:32 +0200, hlinnakan...@vmware.com wrote:
>
>> > But I'm not wedded to the idea if someone objects; a log message might
>> > also be reasonable: "LOG: huge TLB pages are not supported on this
>>
On 2013-11-21 18:09:38 -0300, Alvaro Herrera wrote:
> Abhijit Menon-Sen wrote:
> > At 2013-11-15 15:17:32 +0200, hlinnakan...@vmware.com wrote:
>
> > > But I'm not wedded to the idea if someone objects; a log message might
> > > also be reasonable: "LOG: huge TLB pages are not supported on this
>
Abhijit Menon-Sen wrote:
> At 2013-11-15 15:17:32 +0200, hlinnakan...@vmware.com wrote:
> > But I'm not wedded to the idea if someone objects; a log message might
> > also be reasonable: "LOG: huge TLB pages are not supported on this
> > platform, but huge_tlb_pages was 'on'"
>
> Put that way, I
At 2013-11-15 15:17:32 +0200, hlinnakan...@vmware.com wrote:
>
> I spent some time whacking this around, new patch version attached.
Thanks.
> But I'm not wedded to the idea if someone objects; a log message might
> also be reasonable: "LOG: huge TLB pages are not supported on this
> platform, bu
I was recently running some tests with huge page tables. I ran them on two
different architectures: x86 and PPC64.
I saw some discussion going on over here so thought of sharing.
I was using 3 Cores, 8GB RAM, 2 LUN for filesystem (1 for dbfiles and 1 for
logfiles) for these tests...
I had dedicat
On 30.10.2013 19:11, Andres Freund wrote:
On 2013-10-30 22:39:20 +0530, Abhijit Menon-Sen wrote:
At 2013-10-30 11:04:36 -0400, t...@sss.pgh.pa.us wrote:
As a compromise, perhaps we can unconditionally round the size up to be
a multiple of 2MB? […]
That sounds reasonably painless to me.
He
On Wed, Oct 30, 2013 at 12:51 PM, Sergey Konoplev wrote:
> On Wed, Oct 30, 2013 at 12:17 PM, Alvaro Herrera
> wrote:
>>> > I wasn't talking about a built-in support. It was about an ability (a
>>> > way) to back sh_buf with hugepages.
>>>
>>> Then what you need is to set
>>> dynamic_shared_memory
On Wed, Oct 30, 2013 at 12:17 PM, Alvaro Herrera
wrote:
>> > I wasn't talking about a built-in support. It was about an ability (a
>> > way) to back sh_buf with hugepages.
>>
>> Then what you need is to set
>> dynamic_shared_memory_type = sysv
>> in postgresql.conf.
>
> The above is mistaken -- th
Alvaro Herrera escribió:
> Sergey Konoplev escribió:
> > I wasn't talking about a built-in support. It was about an ability (a
> > way) to back sh_buf with hugepages.
>
> Then what you need is to set
> dynamic_shared_memory_type = sysv
> in postgresql.conf.
The above is mistaken -- there's no w
On Wed, Oct 30, 2013 at 11:50 AM, Alvaro Herrera
wrote:
>> >> There were an ability to back shared memory with hugepages when using
>> >> <=9.2. I use it on ~30 servers for several years and it brings 8-17%
>> >> of performance depending on the memory size. Here you will find
>> >> several paragra
Sergey Konoplev escribió:
> On Wed, Oct 30, 2013 at 8:11 AM, Tom Lane wrote:
> > Sergey Konoplev writes:
> >> There were an ability to back shared memory with hugepages when using
> >> <=9.2. I use it on ~30 servers for several years and it brings 8-17%
> >> of performance depending on the memor
On Wed, Oct 30, 2013 at 8:11 AM, Tom Lane wrote:
> Sergey Konoplev writes:
>> On Tue, Oct 29, 2013 at 9:31 PM, Tom Lane wrote:
>>> Say what? There's never been any hugepages support in Postgres.
>
>> There were an ability to back shared memory with hugepages when using
>> <=9.2. I use it on ~30
On 2013-10-30 22:39:20 +0530, Abhijit Menon-Sen wrote:
> At 2013-10-30 11:04:36 -0400, t...@sss.pgh.pa.us wrote:
> >
> > > As a compromise, perhaps we can unconditionally round the size up to be
> > > a multiple of 2MB? […]
> >
> > That sounds reasonably painless to me.
>
> Here's a patch that d
At 2013-10-30 11:04:36 -0400, t...@sss.pgh.pa.us wrote:
>
> > As a compromise, perhaps we can unconditionally round the size up to be
> > a multiple of 2MB? […]
>
> That sounds reasonably painless to me.
Here's a patch that does that and adds a DEBUG1 log message when we try
with MAP_HUGETLB and
Sergey Konoplev writes:
> On Tue, Oct 29, 2013 at 9:31 PM, Tom Lane wrote:
>> Say what? There's never been any hugepages support in Postgres.
> There were an ability to back shared memory with hugepages when using
> <=9.2. I use it on ~30 servers for several years and it brings 8-17%
> of perfo
Abhijit Menon-Sen writes:
> As a compromise, perhaps we can unconditionally round the size up to be
> a multiple of 2MB? That way, we can use huge pages more often, but also
> avoid putting in a lot of code and effort into the workaround and waste
> only a little space (if any at all).
That sound
At 2013-10-30 00:10:39 -0700, da...@fetter.org wrote:
>
> How about documenting that 2MB is the quantum (OK, we'll say
> "indivisible unit" or "smallest division" or something) and failing
> with a message to that effect if someone tries to set it otherwise?
I don't think you understand the proble
On Wed, Oct 30, 2013 at 10:16:57AM +0530, Abhijit Menon-Sen wrote:
> At 2013-10-24 16:06:19 +0300, hlinnakan...@vmware.com wrote:
> >
> > Let's get rid of the rounding.
>
> I share Andres's concern that the bug is present in various recent
> kernels that are going to stick around for quite some ti
On Tue, Oct 29, 2013 at 11:08:05PM -0700, Sergey Konoplev wrote:
> On Tue, Oct 29, 2013 at 9:31 PM, Tom Lane wrote:
> > Sergey Konoplev writes:
> >> On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen
> >> wrote:
> >>> This is a slightly reworked version of the patch submitted by Richard
> >>>
On Tue, Oct 29, 2013 at 9:31 PM, Tom Lane wrote:
> Sergey Konoplev writes:
>> On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen
>> wrote:
>>> This is a slightly reworked version of the patch submitted by Richard
>>> Poole last month, which was based on Christian Kruse's earlier patch.
>
>> Is
At 2013-10-24 19:00:28 +0200, and...@2ndquadrant.com wrote:
>
> I think we should log when we tried to use hugepages but fell back to
> plain mmap, currently it's hard to see whether they are used.
Good idea, thanks. I'll do this in the next patch I post (which will be
after we reach some consensu
At 2013-10-24 16:06:19 +0300, hlinnakan...@vmware.com wrote:
>
> Let's get rid of the rounding.
I share Andres's concern that the bug is present in various recent
kernels that are going to stick around for quite some time. Given
the rather significant performance gain, I think it's worth doing
som
Sergey Konoplev writes:
> On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen
> wrote:
>> This is a slightly reworked version of the patch submitted by Richard
>> Poole last month, which was based on Christian Kruse's earlier patch.
> Is it possible that this patch will be included in a minor v
Hi,
On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen
wrote:
> This is a slightly reworked version of the patch submitted by Richard
> Poole last month, which was based on Christian Kruse's earlier patch.
Is it possible that this patch will be included in a minor version of
9.3? IMHO hugepage
On Thu, Oct 24, 2013 at 1:00 PM, Andres Freund wrote:
> On 2013-10-24 16:06:19 +0300, Heikki Linnakangas wrote:
>> On 24.10.2013 09:03, Abhijit Menon-Sen wrote:
>> >One open question is what to do about rounding up the size. It should
>> >not be necessary, but for the fairly recent bug described a
On 2013-10-24 16:06:19 +0300, Heikki Linnakangas wrote:
> On 24.10.2013 09:03, Abhijit Menon-Sen wrote:
> >One open question is what to do about rounding up the size. It should
> >not be necessary, but for the fairly recent bug described at the link
> >in the comment (https://bugzilla.kernel.org/sh
On Thu, Oct 24, 2013 at 9:06 AM, Heikki Linnakangas
wrote:
> * the documentation should perhaps mention that the setting only has an
> effect if POSIX shared memory is used. That's the default on Linux, but we
> will try to fall back to SystemV shared memory if it fails.
This is true for dynamic
On 24.10.2013 09:03, Abhijit Menon-Sen wrote:
This is a slightly reworked version of the patch submitted by Richard
Poole last month, which was based on Christian Kruse's earlier patch.
Thanks.
With huge_tlb_pages=off, this is the best result I got:
tps = 8680.771068 (including connecti
At 2013-10-24 11:33:13 +0530, a...@2ndquadrant.com wrote:
>
> >From /proc/$pid/status, VmPTE was 2880kb with huge_tlb_pages=off, and
> 56kb with it turned on.
(VmPTE is the size of the process's page tables.)
-- Abhijit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To m
Hi.
This is a slightly reworked version of the patch submitted by Richard
Poole last month, which was based on Christian Kruse's earlier patch.
Apart from doing various minor cleanups and documentation fixes, I also
tested this patch against HEAD on a machine with 256GB of RAM. Here's an
overview
74 matches
Mail list logo
