On Sun, 17 Apr 2011 21:06:17 -0400, Robert Haas wrote:
On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura
rsmog...@softperience.eu wrote:
Each process has simple mirror of shared descriptors.
I believe that modifications to buffer content may be only done
when holding
exclusive lock (with some
Tom Lane t...@sss.pgh.pa.us Sunday 17 April 2011 01:35:45
=?utf-8?q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes:
No, no, no :) I wanted to do this, but from above reason I skipped it. I
swap VM pages, I do remap, in place where the shared buffer was I put
mmaped page, and in
=?utf-8?q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes:
Tom Lane t...@sss.pgh.pa.us Sunday 17 April 2011 01:35:45
... Huh? Are you saying that you ask the kernel to map each individual
shared buffer separately? I can't believe that's going to scale to
realistic applications.
No,
Robert Haas robertmh...@gmail.com writes:
... But please, everyone feel free to continue bashing me for
wanting a readable patch with some understandable submission notes.
What he said. All this obsessing over whether the mmap patch could or
should have been run through pgindent is missing the
Tom Lane t...@sss.pgh.pa.us Sunday 17 April 2011 17:48:56
=?utf-8?q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes:
Tom Lane t...@sss.pgh.pa.us Sunday 17 April 2011 01:35:45
... Huh? Are you saying that you ask the kernel to map each individual
shared buffer separately? I
On Sunday 17 April 2011 19:26:31 Radosław Smogura wrote:
Kernel merges vm_structs. So mappings are compacted. I'm not kernel
specialist, but skipping memory consumption, for not compacted mappings,
kernel uses btrees for dealing with TLB, so it should not matter if there
is 100 vm_structs
Robert,
Actually, I'd walk through fire for a 10% performance improvement if
it meant only a *risk* to stability.
Depends on the degree of risk. MMAP has the potential to introduce instability
into areas of the code which have been completely reliable for years. Adding
20 new coredump
Joshua Berkus j...@agliodbs.com writes:
I, for one, am glad he did this work. We've discussed MMAP in the code off
and on for years, but nobody wanted to do the work to test it. Now someone
has, and we can decide whether it's worth pursuing based on the numbers.
Well, the troubling issue
On Sun, Apr 17, 2011 at 11:48 AM, Tom Lane t...@sss.pgh.pa.us wrote:
=?utf-8?q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes:
Tom Lane t...@sss.pgh.pa.us Sunday 17 April 2011 01:35:45
... Huh? Are you saying that you ask the kernel to map each individual
shared buffer separately? I
Andres Freund and...@anarazel.de Sunday 17 April 2011 20:02:11
On Sunday 17 April 2011 19:26:31 Radosław Smogura wrote:
Kernel merges vm_structs. So mappings are compacted. I'm not kernel
specialist, but skipping memory consumption, for not compacted mappings,
kernel uses btrees for dealing
On Sunday 17 April 2011 22:09:24 Radosław Smogura wrote:
I only know Phenom has 4096 entries I think and this covers 16MB of
memory.
The numbers I cited where intels before and after core2.
Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your
Robert Haas robertmh...@gmail.com Sunday 17 April 2011 22:01:55
On Sun, Apr 17, 2011 at 11:48 AM, Tom Lane t...@sss.pgh.pa.us wrote:
=?utf-8?q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes:
Tom Lane t...@sss.pgh.pa.us Sunday 17 April 2011 01:35:45
... Huh? Are you saying that
On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura
rsmog...@softperience.eu wrote:
Each process has simple mirror of shared descriptors.
I believe that modifications to buffer content may be only done when holding
exclusive lock (with some simple exceptions) (+ MVCC), actually I saw only two
Robert Haas robertmh...@gmail.com Monday 18 April 2011 03:06:17
On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura
rsmog...@softperience.eu wrote:
Each process has simple mirror of shared descriptors.
I believe that modifications to buffer content may be only done when
holding
On Apr 16, 2011, at 1:48 AM, Greg Smith g...@2ndquadrant.com wrote:
P.S. You know what else I feel should earn an automatic rejection without any
reviewer even looking at the code?
Greg is absolutely right. And to the two he listed, let me add another of my
own gripes: failing to provide
Greg Smith g...@2ndquadrant.com writes:
Reading one's own diff and reflecting on what you've changed is one of
the extremely underappreciated practices of good open-source software
development. Minimizing the size of that diff is perhaps the most
important thing someone can do in order to
Tom Lane wrote:
* On the other side of the coin, I have seen many a patch that was
written to minimize the length of the diff to the detriment of
readability or maintainability of the resulting code, and that's *not*
a good tradeoff.
Sure. that's possible. But based on the reviews I've
Robert Haas wrote:
The OP says that this patch maintains the WAL-before-data rule without any
explanation of how it accomplishes that seemingly quite amazing feat. I assume
I'm going to have to read this patch at some point to refute this assertion,
and I think that sucks.
I don't think
On Sat, Apr 16, 2011 at 7:24 AM, Robert Haas robertmh...@gmail.com wrote:
The OP says that this patch maintains the WAL-before-data rule without any
explanation of how it accomplishes that seemingly quite amazing feat. I
assume I'm going to have to read this patch at some point to refute
Greg Stark gsst...@mit.edu Saturday 16 April 2011 13:00:19
On Sat, Apr 16, 2011 at 7:24 AM, Robert Haas robertmh...@gmail.com wrote:
The OP says that this patch maintains the WAL-before-data rule without
any explanation of how it accomplishes that seemingly quite amazing
feat. I assume I'm
On Sat, Apr 16, 2011 at 8:48 AM, Greg Smith g...@2ndquadrant.com wrote:
Joshua Berkus wrote:
Guys, can we *please* focus on the patch for now, rather than the
formatting, which is fixable with sed?
Never, and that's not true. Heikki was being nice; I wouldn't have even
slogged through it
Greg Stark gsst...@mit.edu writes:
What he did, I gather, is treat the mmapped buffers as a read-only
copy of the data. To actually make any modifications he copies it into
shared buffers and treats them like normal. When the buffers get
flushed from memory they get written and then the
On Fri, 2011-04-15 at 12:32 +0200, Radosław Smogura wrote:
I didn't included this, as diff, because of ~150kb size (mainly
configure scripts, which are included in SVC). Due to this, You may
download it from
http://softperience.eu/downloads/pg_mmap_20110415.diff.bz2 (Legal:
Work
Radoslaw,
I think 10% is quite good, as my stand-alone test of mmap vs. read
shown that
speed up of copying 100MB data to mem may be from ~20ms to ~100ms
(depends on
destination address). Of course deeper, system test simulating real
usage will
say more. In any case after good deals with
On Fri, Apr 15, 2011 at 3:12 PM, Radosław Smogura
rsmog...@softperience.eu wrote:
On Fri, 15 Apr 2011 14:33:37 +0300, Heikki Linnakangas wrote:
The patch is quite hard to read because of random whitespace changes
and other stylistic issues, but I have a couple of high-level
questions on the
Tom Lane t...@sss.pgh.pa.us Saturday 16 April 2011 17:02:32
Greg Stark gsst...@mit.edu writes:
What he did, I gather, is treat the mmapped buffers as a read-only
copy of the data. To actually make any modifications he copies it into
shared buffers and treats them like normal. When the
Radosław Smogura wrote:
Yes, but, hmm... in Netbeans I had really long gaps (probably 8
spaces, from tabs), so deeper ifs, comments at the and of variables,
went of out my screen. I really wanted to not format this, but
sometimes I needed.
The guide at
=?utf-8?q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes:
No, no, no :) I wanted to do this, but from above reason I skipped it. I swap
VM pages, I do remap, in place where the shared buffer was I put mmaped page,
and in place where mmaped page was I put shared page (in certain
On Sat, Apr 16, 2011 at 2:33 PM, Joshua Berkus j...@agliodbs.com wrote:
Well, given the risks to durability and stability associated with using MMAP,
I doubt anyone would even consider it for a 10% throughput improvement.
However, I don't think the test you used demonstrates the best case
On Sat, Apr 16, 2011 at 9:31 PM, Robert Haas robertmh...@gmail.com wrote:
On Sat, Apr 16, 2011 at 2:33 PM, Joshua Berkus j...@agliodbs.com wrote:
Well, given the risks to durability and stability associated with using
MMAP, I doubt anyone would even consider it for a 10% throughput
On 15.04.2011 13:32, Radosław Smogura wrote:
If I may, I want to share some concept to use mmap in PG. It's far, far
away from perfect, but it's keeps WAL before data. As well I crated
table, with index, inserted few values, and I done vacuum full on this
table. Db inits welcome from orginal
On Fri, 15 Apr 2011 14:33:37 +0300, Heikki Linnakangas wrote:
On 15.04.2011 13:32, Radosław Smogura wrote:
If I may, I want to share some concept to use mmap in PG. It's far,
far
away from perfect, but it's keeps WAL before data. As well I crated
table, with index, inserted few values, and I
On 04/15/2011 08:12 AM, Radosław Smogura wrote:
The patch is quite hard to read because of random whitespace changes
and other stylistic issues, but I have a couple of high-level
questions on the design:
Yes, but, hmm... in Netbeans I had really long gaps (probably 8
spaces, from tabs), so
Radoslaw,
10% improvement isn't very impressive from a switch to mmap. What workload did
you test with? What I'd really like to see is testing with databases which are
50%, 90% and 200% the size of RAM ... that's where I'd expect the greatest gain
from limiting copying.
Netbeans is
Joshua Berkus j...@agliodbs.com Friday 15 April 2011 18:55:04
Radoslaw,
10% improvement isn't very impressive from a switch to mmap. What workload
did you test with? What I'd really like to see is testing with databases
which are 50%, 90% and 200% the size of RAM ... that's where I'd
Joshua Berkus wrote:
Guys, can we *please* focus on the patch for now, rather than the formatting,
which is fixable with sed?
Never, and that's not true. Heikki was being nice; I wouldn't have even
slogged through it long enough to ask the questions he did before
kicking it back as
36 matches
Mail list logo
