Hi,
On 03/03/14 21:03, Heikki Linnakangas wrote:
I spotted this in section 17.4.1 Shared Memory and Semaphores:
Linux
The default maximum segment size is 32 MB, and the default maximum total
size is 2097152 pages. A page is almost always 4096 bytes except in unusual
kernel
Hi,
Attached is a patch with the updated documentation (now uses
consistently huge pages) as well as a renamed GUC, consistent wording
(always use huge pages) as well as renamed variables.
Hmm, I wonder if that could now be misunderstood to have something to do
with the PostgreSQL page
Hi,
On 28/02/14 17:58, Peter Geoghegan wrote:
On Fri, Feb 28, 2014 at 9:43 AM, Heikki Linnakangas
hlinnakan...@vmware.com wrote:
Hmm, I wonder if that could now be misunderstood to have something to do
with the PostgreSQL page size? Maybe add the word memory or operating
system in the
On 03/03/2014 11:34 AM, Christian Kruse wrote:
Hi,
Attached is a patch with the updated documentation (now uses
consistently huge pages) as well as a renamed GUC, consistent wording
(always use huge pages) as well as renamed variables.
Hmm, I wonder if that could now be misunderstood to have
On 02/27/2014 09:34 AM, Christian Kruse wrote:
Hi,
On 26/02/14 13:13, Alvaro Herrera wrote:
There's one thing that rubs me the wrong way about all this
functionality, which is that we've named it huge TLB pages. That is
wrong -- the TLB pages are not huge. In fact, as far as I understand,
On Fri, Feb 28, 2014 at 9:43 AM, Heikki Linnakangas
hlinnakan...@vmware.com wrote:
Hmm, I wonder if that could now be misunderstood to have something to do
with the PostgreSQL page size? Maybe add the word memory or operating
system in the first sentence in the docs, like this: Enables/disables
Hi,
On 27/02/14 08:35, Christian Kruse wrote:
Hi Peter,
Sorry, Stephen of course – it was definitely to early.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training Services
pgpm6lYpan4Df.pgp
Description: PGP signature
Hi,
On 25/02/14 19:28, Andres Freund wrote:
I think all that's needed is to cut the first paragraphs that generally
explain what huge pages are in some detail from the text and make sure
the later paragraphs don't refer to the earlier ones.
Attached you will find a new version of the patch.
Hi Peter,
after a night of sleep I'm still not able to swallow the pill. To be
honest I'm a little bit angry about this accusation.
I didn't mean to copy from the Debian wiki and after re-checking the
text again I'm still convinced that I didn't.
Of course the text SAYS something similar, but
On 02/26/2014 10:35 AM, Christian Kruse wrote:
On 25/02/14 19:28, Andres Freund wrote:
I think all that's needed is to cut the first paragraphs that generally
explain what huge pages are in some detail from the text and make sure
the later paragraphs don't refer to the earlier ones.
Attached
Hi,
On 26/02/14 14:34, Heikki Linnakangas wrote:
That still says The setting is ignored on other systems. That's not quite
true: as explained later in the section, if you set huge_tlb_pages=on and
the platform doesn't support it, the server will refuse to start.
I added a sentence about it.
There's one thing that rubs me the wrong way about all this
functionality, which is that we've named it huge TLB pages. That is
wrong -- the TLB pages are not huge. In fact, as far as I understand,
the TLB doesn't have pages at all. It's the pages that are huge, but
those pages are not TLB
On 02/26/2014 06:13 PM, Alvaro Herrera wrote:
There's one thing that rubs me the wrong way about all this
functionality, which is that we've named it huge TLB pages. That is
wrong -- the TLB pages are not huge. In fact, as far as I understand,
the TLB doesn't have pages at all. It's the
Christian,
Thanks for working on all of this and dealing with the requests for
updates and changes, as well as for dealing very professionally with an
inappropriate and incorrect remark. Unfortunately, mailing lists can
make communication difficult and someone's knee-jerk reaction (not
referring
Hi,
On 26/02/14 13:13, Alvaro Herrera wrote:
There's one thing that rubs me the wrong way about all this
functionality, which is that we've named it huge TLB pages. That is
wrong -- the TLB pages are not huge. In fact, as far as I understand,
the TLB doesn't have pages at all. It's the
Hi Peter,
thank you for your nice words, much appreciated. I'm sorry that I was
so whiny about this in the last post.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training Services
pgpxVj8SJRDQS.pgp
Description: PGP
On 1/30/14, 2:28 AM, Christian Kruse wrote:
after I finally got documentation compilation working I updated the
patch to be syntactically correct. You will find it attached.
I don't think we should be explaining the basics of OS memory management
in our documentation. And if we did, we
On 2014-02-25 10:29:32 -0500, Peter Eisentraut wrote:
On 1/30/14, 2:28 AM, Christian Kruse wrote:
after I finally got documentation compilation working I updated the
patch to be syntactically correct. You will find it attached.
I don't think we should be explaining the basics of OS memory
Hi,
On 25/02/14 10:29, Peter Eisentraut wrote:
I don't think we should be explaining the basics of OS memory management
in our documentation.
Well, I'm confused. I thought that's exactly what has been asked.
And if we did, we shouldn't copy it verbatim from the Debian wiki
without
Hi,
On 25/02/14 17:01, Andres Freund wrote:
And if we did, we shouldn't copy it verbatim from the Debian wiki
without attribution.
Is it actually? A quick comparison doesn't show that many similarities?
Christian?
Not as far as I know. But of course, as I wrote the text I _also_
(that's
On 2/25/14, 11:08 AM, Christian Kruse wrote:
Hi,
On 25/02/14 17:01, Andres Freund wrote:
And if we did, we shouldn't copy it verbatim from the Debian wiki
without attribution.
Is it actually? A quick comparison doesn't show that many similarities?
Christian?
Not as far as I know. But
On Tue, Feb 25, 2014 at 12:18:02PM -0500, Peter Eisentraut wrote:
On 2/25/14, 11:08 AM, Christian Kruse wrote:
Hi,
On 25/02/14 17:01, Andres Freund wrote:
And if we did, we shouldn't copy it verbatim from the Debian wiki
without attribution.
Is it actually? A quick comparison
Bruce Momjian br...@momjian.us writes:
On Tue, Feb 25, 2014 at 12:18:02PM -0500, Peter Eisentraut wrote:
As I mentioned, I would just cut those introductory parts out.
Should we link to the Debian wiki content?
-1. We generally don't link to our *own* wiki in our SGML docs, let alone
things
On 2014-02-25 13:21:46 -0500, Tom Lane wrote:
Bruce Momjian br...@momjian.us writes:
On Tue, Feb 25, 2014 at 12:18:02PM -0500, Peter Eisentraut wrote:
As I mentioned, I would just cut those introductory parts out.
Should we link to the Debian wiki content?
-1. We generally don't link
On 01/28/2014 06:11 PM, Christian Kruse wrote:
Hi,
attached you will find a new version of the patch, ported to HEAD,
fixed the mentioned bug and - hopefully - dealing the the remaining
issues.
Thanks, I have committed this now.
The documentation is still lacking. We should explain somewhere
On 01/29/2014 01:12 PM, Heikki Linnakangas wrote:
On 01/28/2014 06:11 PM, Christian Kruse wrote:
Hi,
attached you will find a new version of the patch, ported to HEAD,
fixed the mentioned bug and - hopefully - dealing the the remaining
issues.
Thanks, I have committed this now.
The
On 01/29/2014 04:01 PM, Vik Fearing wrote:
On 01/29/2014 01:12 PM, Heikki Linnakangas wrote:
The documentation is still lacking.
The documentation is indeed lacking since it breaks the build.
doc/src/sgml/config.sgml contains the line
normal allocation if that fails. With
On Tue, Jan 28, 2014 at 5:58 AM, Christian Kruse
christ...@2ndquadrant.com wrote:
Hi,
On 28/01/14 13:51, Heikki Linnakangas wrote:
Oh darn, I remembered we had already committed this, but clearly not. I'd
love to still get this into 9.4. The latest patch (hugepages-v5.patch) was
pretty much
On Wed, Jan 29, 2014 at 4:12 AM, Heikki Linnakangas hlinnakan...@vmware.com
wrote:
On 01/28/2014 06:11 PM, Christian Kruse wrote:
Hi,
attached you will find a new version of the patch, ported to HEAD,
fixed the mentioned bug and - hopefully - dealing the the remaining
issues.
Thanks, I
Hi,
On 29/01/14 14:12, Heikki Linnakangas wrote:
The documentation is still lacking. We should explain somewhere how to set
nr.hugepages, for example. The Managing Kernel Resources section ought to
mention setting. Could I ask you to work on that, please?
Of course! Attached you will find a
Hi,
On 29/01/14 10:11, Jeff Janes wrote:
I'm getting this warning now with gcc (GCC) 4.4.7:
Interesting. I don't get that warning. But the compiler is (formally)
right.
pg_shmem.c: In function 'PGSharedMemoryCreate':
pg_shmem.c:332: warning: 'allocsize' may be used uninitialized in this
On 01/29/2014 09:18 PM, Christian Kruse wrote:
Hi,
On 29/01/14 10:11, Jeff Janes wrote:
I'm getting this warning now with gcc (GCC) 4.4.7:
Interesting. I don't get that warning. But the compiler is (formally)
right.
pg_shmem.c: In function 'PGSharedMemoryCreate':
pg_shmem.c:332: warning:
Hi,
On 29/01/14 21:36, Heikki Linnakangas wrote:
[…]
Fix pushed.
You are right. Thanks. But there is another bug, see
20140128154307.gc24...@defunct.ch
ff. Attached you will find a patch fixing that.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL
On 01/29/2014 09:59 PM, Christian Kruse wrote:
Hi,
On 29/01/14 21:36, Heikki Linnakangas wrote:
[…]
Fix pushed.
You are right. Thanks. But there is another bug, see
20140128154307.gc24...@defunct.ch
ff. Attached you will find a patch fixing that.
Thanks. There are more cases of that in
Hi,
On 29/01/14 22:17, Heikki Linnakangas wrote:
Thanks. There are more cases of that in InternalIpcMemoryCreate, they ought
to be fixed as well. And should also grep the rest of the codebase for more
instances of that. And this needs to be back-patched.
I'm way ahead of you ;-) Working on
Hi,
after I finally got documentation compilation working I updated the
patch to be syntactically correct. You will find it attached.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training Services
diff --git
On 01/27/2014 09:20 PM, Alvaro Herrera wrote:
Heikki Linnakangas wrote:
I spent some time whacking this around, new patch version attached.
I moved the mmap() code into a new function, that leaves the
PGSharedMemoryCreate more readable.
Did this patch go anywhere?
Oh darn, I remembered we
Hi,
On 28/01/14 13:51, Heikki Linnakangas wrote:
Oh darn, I remembered we had already committed this, but clearly not. I'd
love to still get this into 9.4. The latest patch (hugepages-v5.patch) was
pretty much ready for commit, except for documentation.
I'm working on it. I ported it to HEAD
Hi,
On 15/11/13 15:17, Heikki Linnakangas wrote:
I spent some time whacking this around, new patch version attached. I moved
the mmap() code into a new function, that leaves the PGSharedMemoryCreate
more readable.
I think there's a bug in this version of the patch. Have a look at
this:
+
Hi,
attached you will find a new version of the patch, ported to HEAD,
fixed the mentioned bug and - hopefully - dealing the the remaining
issues.
Best regards,
--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training Services
diff --git
Heikki Linnakangas wrote:
I spent some time whacking this around, new patch version attached.
I moved the mmap() code into a new function, that leaves the
PGSharedMemoryCreate more readable.
Did this patch go anywhere?
Someone just pinged me about a kernel scalability problem in Linux with
At 2013-11-21 22:14:35 +0100, and...@2ndquadrant.com wrote:
I'd certainly want a setting that errors out if it cannot get the
memory using hugetables.
OK, then the current try/on/off settings are fine.
I'm better today, so I'll read the patch Heikki posted and see what more
needs to be done
On 2013-11-21 18:09:38 -0300, Alvaro Herrera wrote:
Abhijit Menon-Sen wrote:
At 2013-11-15 15:17:32 +0200, hlinnakan...@vmware.com wrote:
But I'm not wedded to the idea if someone objects; a log message might
also be reasonable: LOG: huge TLB pages are not supported on this
platform,
On 2013-11-21 16:24:56 -0500, Robert Haas wrote:
What about
huge_tlb_pages={off,try}
Or maybe
huge_tlb_pages={off,try,require}
I'd spell require as on, or at least accept that as a synonym.
That's off,try, on is what the patch currently implements, Abhijit just
was arguing for
Abhijit Menon-Sen wrote:
At 2013-11-15 15:17:32 +0200, hlinnakan...@vmware.com wrote:
But I'm not wedded to the idea if someone objects; a log message might
also be reasonable: LOG: huge TLB pages are not supported on this
platform, but huge_tlb_pages was 'on'
Put that way, I have to
On Thu, Nov 21, 2013 at 4:09 PM, Alvaro Herrera
alvhe...@2ndquadrant.com wrote:
Abhijit Menon-Sen wrote:
At 2013-11-15 15:17:32 +0200, hlinnakan...@vmware.com wrote:
But I'm not wedded to the idea if someone objects; a log message might
also be reasonable: LOG: huge TLB pages are not
At 2013-11-15 15:17:32 +0200, hlinnakan...@vmware.com wrote:
I spent some time whacking this around, new patch version attached.
Thanks.
But I'm not wedded to the idea if someone objects; a log message might
also be reasonable: LOG: huge TLB pages are not supported on this
platform, but
I was recently running some tests with huge page tables. I ran them on two
different architectures: x86 and PPC64.
I saw some discussion going on over here so thought of sharing.
I was using 3 Cores, 8GB RAM, 2 LUN for filesystem (1 for dbfiles and 1 for
logfiles) for these tests...
I had
On 30.10.2013 19:11, Andres Freund wrote:
On 2013-10-30 22:39:20 +0530, Abhijit Menon-Sen wrote:
At 2013-10-30 11:04:36 -0400, t...@sss.pgh.pa.us wrote:
As a compromise, perhaps we can unconditionally round the size up to be
a multiple of 2MB? […]
That sounds reasonably painless to me.
On Tue, Oct 29, 2013 at 9:31 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Sergey Konoplev gray...@gmail.com writes:
On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen a...@2ndquadrant.com
wrote:
This is a slightly reworked version of the patch submitted by Richard
Poole last month, which was
On Tue, Oct 29, 2013 at 11:08:05PM -0700, Sergey Konoplev wrote:
On Tue, Oct 29, 2013 at 9:31 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Sergey Konoplev gray...@gmail.com writes:
On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen a...@2ndquadrant.com
wrote:
This is a slightly reworked
On Wed, Oct 30, 2013 at 10:16:57AM +0530, Abhijit Menon-Sen wrote:
At 2013-10-24 16:06:19 +0300, hlinnakan...@vmware.com wrote:
Let's get rid of the rounding.
I share Andres's concern that the bug is present in various recent
kernels that are going to stick around for quite some time.
At 2013-10-30 00:10:39 -0700, da...@fetter.org wrote:
How about documenting that 2MB is the quantum (OK, we'll say
indivisible unit or smallest division or something) and failing
with a message to that effect if someone tries to set it otherwise?
I don't think you understand the problem.
Abhijit Menon-Sen a...@2ndquadrant.com writes:
As a compromise, perhaps we can unconditionally round the size up to be
a multiple of 2MB? That way, we can use huge pages more often, but also
avoid putting in a lot of code and effort into the workaround and waste
only a little space (if any at
Sergey Konoplev gray...@gmail.com writes:
On Tue, Oct 29, 2013 at 9:31 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Say what? There's never been any hugepages support in Postgres.
There were an ability to back shared memory with hugepages when using
=9.2. I use it on ~30 servers for several years
At 2013-10-30 11:04:36 -0400, t...@sss.pgh.pa.us wrote:
As a compromise, perhaps we can unconditionally round the size up to be
a multiple of 2MB? […]
That sounds reasonably painless to me.
Here's a patch that does that and adds a DEBUG1 log message when we try
with MAP_HUGETLB and fail
On 2013-10-30 22:39:20 +0530, Abhijit Menon-Sen wrote:
At 2013-10-30 11:04:36 -0400, t...@sss.pgh.pa.us wrote:
As a compromise, perhaps we can unconditionally round the size up to be
a multiple of 2MB? […]
That sounds reasonably painless to me.
Here's a patch that does that and
On Wed, Oct 30, 2013 at 8:11 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Sergey Konoplev gray...@gmail.com writes:
On Tue, Oct 29, 2013 at 9:31 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Say what? There's never been any hugepages support in Postgres.
There were an ability to back shared memory with
Sergey Konoplev escribió:
On Wed, Oct 30, 2013 at 8:11 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Sergey Konoplev gray...@gmail.com writes:
There were an ability to back shared memory with hugepages when using
=9.2. I use it on ~30 servers for several years and it brings 8-17%
of performance
On Wed, Oct 30, 2013 at 11:50 AM, Alvaro Herrera
alvhe...@2ndquadrant.com wrote:
There were an ability to back shared memory with hugepages when using
=9.2. I use it on ~30 servers for several years and it brings 8-17%
of performance depending on the memory size. Here you will find
several
Alvaro Herrera escribió:
Sergey Konoplev escribió:
I wasn't talking about a built-in support. It was about an ability (a
way) to back sh_buf with hugepages.
Then what you need is to set
dynamic_shared_memory_type = sysv
in postgresql.conf.
The above is mistaken -- there's no way to
On Wed, Oct 30, 2013 at 12:17 PM, Alvaro Herrera
alvhe...@2ndquadrant.com wrote:
I wasn't talking about a built-in support. It was about an ability (a
way) to back sh_buf with hugepages.
Then what you need is to set
dynamic_shared_memory_type = sysv
in postgresql.conf.
The above is
On Wed, Oct 30, 2013 at 12:51 PM, Sergey Konoplev gray...@gmail.com wrote:
On Wed, Oct 30, 2013 at 12:17 PM, Alvaro Herrera
alvhe...@2ndquadrant.com wrote:
I wasn't talking about a built-in support. It was about an ability (a
way) to back sh_buf with hugepages.
Then what you need is to set
Hi,
On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen a...@2ndquadrant.com
wrote:
This is a slightly reworked version of the patch submitted by Richard
Poole last month, which was based on Christian Kruse's earlier patch.
Is it possible that this patch will be included in a minor version of
Sergey Konoplev gray...@gmail.com writes:
On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen a...@2ndquadrant.com
wrote:
This is a slightly reworked version of the patch submitted by Richard
Poole last month, which was based on Christian Kruse's earlier patch.
Is it possible that this
At 2013-10-24 16:06:19 +0300, hlinnakan...@vmware.com wrote:
Let's get rid of the rounding.
I share Andres's concern that the bug is present in various recent
kernels that are going to stick around for quite some time. Given
the rather significant performance gain, I think it's worth doing
At 2013-10-24 19:00:28 +0200, and...@2ndquadrant.com wrote:
I think we should log when we tried to use hugepages but fell back to
plain mmap, currently it's hard to see whether they are used.
Good idea, thanks. I'll do this in the next patch I post (which will be
after we reach some consensus
Hi.
This is a slightly reworked version of the patch submitted by Richard
Poole last month, which was based on Christian Kruse's earlier patch.
Apart from doing various minor cleanups and documentation fixes, I also
tested this patch against HEAD on a machine with 256GB of RAM. Here's an
At 2013-10-24 11:33:13 +0530, a...@2ndquadrant.com wrote:
From /proc/$pid/status, VmPTE was 2880kb with huge_tlb_pages=off, and
56kb with it turned on.
(VmPTE is the size of the process's page tables.)
-- Abhijit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make
On 24.10.2013 09:03, Abhijit Menon-Sen wrote:
This is a slightly reworked version of the patch submitted by Richard
Poole last month, which was based on Christian Kruse's earlier patch.
Thanks.
With huge_tlb_pages=off, this is the best result I got:
tps = 8680.771068 (including
On Thu, Oct 24, 2013 at 9:06 AM, Heikki Linnakangas
hlinnakan...@vmware.com wrote:
* the documentation should perhaps mention that the setting only has an
effect if POSIX shared memory is used. That's the default on Linux, but we
will try to fall back to SystemV shared memory if it fails.
This
On 2013-10-24 16:06:19 +0300, Heikki Linnakangas wrote:
On 24.10.2013 09:03, Abhijit Menon-Sen wrote:
One open question is what to do about rounding up the size. It should
not be necessary, but for the fairly recent bug described at the link
in the comment
On Thu, Oct 24, 2013 at 1:00 PM, Andres Freund and...@2ndquadrant.com wrote:
On 2013-10-24 16:06:19 +0300, Heikki Linnakangas wrote:
On 24.10.2013 09:03, Abhijit Menon-Sen wrote:
One open question is what to do about rounding up the size. It should
not be necessary, but for the fairly recent
73 matches
Mail list logo
