On Thu, Mar 15, 2012 at 5:52 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
When all those changes are put together, the patched version now beats or
matches the current code in the RAM drive tests, except that the
single-client case is still about 10% slower. I added the new
On 21.03.2012 13:14, Fujii Masao wrote:
PANIC: space reserved for WAL record does not match what was
written, CurrPos: C/0, EndPos: B/FF00
So I think that the patch would have a bug which handles WAL boundary wrongly.
Thanks for the testing! These WAL boundary issues are really
On Wed, Mar 21, 2012 at 7:52 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
So, although none of the issues alone is a show-stopper, but considering all
these things together, I'm starting to feel that this needs to be pushed to
9.3. Thoughts?
I think I agree. I like the
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
... although none of the issues alone is a show-stopper, but considering
all these things together, I'm starting to feel that this needs to be
pushed to 9.3. Thoughts?
Agreed. In particular, I think you are right that it'd be
On Wed, Mar 21, 2012 at 10:59 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
... although none of the issues alone is a show-stopper, but considering
all these things together, I'm starting to feel that this needs to be
pushed to 9.3.
On Fri, Mar 9, 2012 at 2:45 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Thanks!
BTW, I haven't forgotten about the recovery bugs Jeff found earlier. I'm
planning to do a longer run with his test script - I only run it for about
1000 iterations - to see if I can
On 09.03.2012 12:04, Heikki Linnakangas wrote:
I've been doing some performance testing with this, using a simple C
function that just inserts a dummy WAL record of given size. I'm not
totally satisfied. Although the patch helps with scalability at 3-4
concurrent backends doing WAL insertions,
On Tue, Mar 13, 2012 at 1:59 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On Fri, Mar 9, 2012 at 2:45 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Thanks!
BTW, I haven't forgotten about the recovery bugs Jeff found earlier. I'm
planning to do a longer run with his test
On 07.03.2012 17:28, Tom Lane wrote:
Simon Riggssi...@2ndquadrant.com writes:
On Wed, Mar 7, 2012 at 3:04 PM, Tom Lanet...@sss.pgh.pa.us wrote:
Alvaro Herreraalvhe...@commandprompt.com writes:
So they are undoubtely rare. Not sure if as rare as Higgs bosons.
Even if they're rare,
On Fri, Mar 9, 2012 at 7:04 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Here's an updated patch. It now only loops once per segment that a record
crosses. Plus a lot of other small cleanup.
Thanks! But you forgot to attach the patch.
I've been doing some performance
On Tue, Mar 6, 2012 at 8:32 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
On 06.03.2012 17:12, Tom Lane wrote:
How long is the current locked code exactly --- does it contain a loop?
Perhaps best if you take a look for yourself, the
Excerpts from Simon Riggs's message of mié mar 07 05:35:44 -0300 2012:
On Tue, Mar 6, 2012 at 8:32 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
On 06.03.2012 17:12, Tom Lane wrote:
How long is the current locked code exactly --- does
Alvaro Herrera alvhe...@commandprompt.com writes:
Just to keep things in perspective -- For a commit record to reach one
megabyte, it would have to be a transaction that drops over 43k tables.
Or have 64k smgr inval messages (for example, a TRUNCATE might send half
a dozen of these messages).
On Wed, Mar 7, 2012 at 3:04 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Alvaro Herrera alvhe...@commandprompt.com writes:
Just to keep things in perspective -- For a commit record to reach one
megabyte, it would have to be a transaction that drops over 43k tables.
Or have 64k smgr inval messages
Simon Riggs si...@2ndquadrant.com writes:
On Wed, Mar 7, 2012 at 3:04 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Alvaro Herrera alvhe...@commandprompt.com writes:
So they are undoubtely rare. Not sure if as rare as Higgs bosons.
Even if they're rare, having a major performance hiccup when one
On Mon, Mar 5, 2012 at 8:50 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
That particular issue would be very hard to hit in practice, so I don't know
if this could explain the recovery failures that Jeff saw. I got the test
script running (thanks for that Jeff!), but
On Tue, Mar 6, 2012 at 2:17 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
On 21.02.2012 13:19, Fujii Masao wrote:
In some places, the spinlock insertpos_lck is taken while another
spinlock info_lck is being held. Is this OK? What if
On Tue, Mar 6, 2012 at 1:50 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
+ * An xlog-switch record consumes all the remaining space on the
+ * WAL segment. We have already reserved it for us, but we still
need
+ * to make sure it's been allocated and
On 06.03.2012 14:52, Fujii Masao wrote:
On Tue, Mar 6, 2012 at 2:17 AM, Tom Lanet...@sss.pgh.pa.us wrote:
Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes:
On 21.02.2012 13:19, Fujii Masao wrote:
In some places, the spinlock insertpos_lck is taken while another
spinlock info_lck
On Tue, Mar 6, 2012 at 10:07 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
I admit that block is longer than any of our existing spinlock blocks.
However, it's important for performance. I tried using a lwlock earlier, and
that negated the gains. So if that's a serious
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
On 06.03.2012 14:52, Fujii Masao wrote:
This also strikes me that the usage of the spinlock insertpos_lck might
not be OK in ReserveXLogInsertLocation() because a few dozen instructions
can be performed while holding the
On 06.03.2012 17:12, Tom Lane wrote:
Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes:
On 06.03.2012 14:52, Fujii Masao wrote:
This also strikes me that the usage of the spinlock insertpos_lck might
not be OK in ReserveXLogInsertLocation() because a few dozen instructions
can be
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
On 06.03.2012 17:12, Tom Lane wrote:
How long is the current locked code exactly --- does it contain a loop?
Perhaps best if you take a look for yourself, the function is called
ReserveXLogInsertLocation() in patch. It calls a
On Wed, Mar 7, 2012 at 5:32 AM, Tom Lane t...@sss.pgh.pa.us wrote:
What I suggest is that it should not be necessary to crawl forward one
page at a time to figure out how many pages will be needed to store N
bytes worth of WAL data. You're basically implementing a division
problem as repeated
On 20.02.2012 08:00, Amit Kapila wrote:
I was trying to understand this patch and had few doubts:
1. In PerformXLogInsert(), why there is need to check freespace when
already during ReserveXLogInsertLocation(), the space is reserved. Is
it possible that the record size is more than actually
On 21.02.2012 13:19, Fujii Masao wrote:
On Sat, Feb 18, 2012 at 12:36 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Attached is a new version, fixing that, and off-by-one bug you pointed out
in the slot wraparound handling. I also moved code around a bit, I think
this new
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
On 21.02.2012 13:19, Fujii Masao wrote:
In some places, the spinlock insertpos_lck is taken while another
spinlock info_lck is being held. Is this OK? What if unfortunately
inner spinlock takes long to be taken?
Hmm, that's only
On Tue, Feb 21, 2012 at 5:34 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Tue, Feb 21, 2012 at 8:19 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Sat, Feb 18, 2012 at 12:36 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Attached is a new version, fixing that, and
On Sat, Feb 18, 2012 at 12:36 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Attached is a new version, fixing that, and off-by-one bug you pointed out
in the slot wraparound handling. I also moved code around a bit, I think
this new division of labor between the XLogInsert
On Tue, Feb 21, 2012 at 8:19 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Sat, Feb 18, 2012 at 12:36 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Attached is a new version, fixing that, and off-by-one bug you pointed out
in the slot wraparound handling. I also moved
On Sun, Feb 19, 2012 at 3:01 AM, Jeff Janes jeff.ja...@gmail.com wrote:
I've tested your v9 patch. I no longer see any inconsistencies or
lost transactions in the recovered database. But occasionally I get
databases that fail to recover at all.
It has always been with the exact same failed
Janes; Robert Haas; PostgreSQL-development
Subject: Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work
outside WALInsertLock)
On 17.02.2012 07:27, Fujii Masao wrote:
Got another problem: when I ran pg_stop_backup to take an online
backup, it got stuck until I had generated new WAL
On Fri, Feb 17, 2012 at 7:36 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 17.02.2012 07:27, Fujii Masao wrote:
Got another problem: when I ran pg_stop_backup to take an online backup,
it got stuck until I had generated new WAL record. This happens because,
in the
On 16.02.2012 13:31, Fujii Masao wrote:
On Thu, Feb 16, 2012 at 6:15 PM, Fujii Masaomasao.fu...@gmail.com wrote:
BTW, when I ran the test on my Ubuntu, I could not reproduce the problem.
I could reproduce the problem only in MacOS.
+ nextslot = Insert-nextslot;
+ if
On Thu, Feb 16, 2012 at 5:02 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 15.02.2012 18:52, Fujii Masao wrote:
On Thu, Feb 16, 2012 at 1:01 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Are you still seeing this failure with the latest patch I
On Thu, Feb 16, 2012 at 6:15 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Thu, Feb 16, 2012 at 5:02 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 15.02.2012 18:52, Fujii Masao wrote:
On Thu, Feb 16, 2012 at 1:01 AM, Heikki Linnakangas
On Mon, Feb 13, 2012 at 8:37 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 13.02.2012 01:04, Jeff Janes wrote:
Attached is my quick and dirty attempt to set XLP_FIRST_IS_CONTRECORD.
I have no idea if I did it correctly, in particular if calling
GetXLogBuffer(CurrPos)
On 13.02.2012 19:13, Fujii Masao wrote:
On Mon, Feb 13, 2012 at 8:37 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 13.02.2012 01:04, Jeff Janes wrote:
Attached is my quick and dirty attempt to set XLP_FIRST_IS_CONTRECORD.
I have no idea if I did it correctly, in
On Thu, Feb 16, 2012 at 1:01 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 13.02.2012 19:13, Fujii Masao wrote:
On Mon, Feb 13, 2012 at 8:37 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 13.02.2012 01:04, Jeff Janes wrote:
Attached is my
On 15.02.2012 18:52, Fujii Masao wrote:
On Thu, Feb 16, 2012 at 1:01 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Are you still seeing this failure with the latest patch I posted
(http://archives.postgresql.org/message-id/4f38f5e5.8050...@enterprisedb.com)?
Yes. Just to
On Mon, Feb 13, 2012 at 8:37 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 13.02.2012 01:04, Jeff Janes wrote:
Attached is my quick and dirty attempt to set XLP_FIRST_IS_CONTRECORD.
I have no idea if I did it correctly, in particular if calling
GetXLogBuffer(CurrPos)
On Thu, Feb 9, 2012 at 3:02 AM, Fujii Masao masao.fu...@gmail.com wrote:
On Thu, Feb 9, 2012 at 7:25 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Thu, Feb 9, 2012 at 3:32 AM, Jeff Janes jeff.ja...@gmail.com wrote:
After applying this patch and then forcing crashes, upon recovery the
On Thu, Feb 9, 2012 at 3:32 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On Wed, Feb 1, 2012 at 11:46 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 31.01.2012 17:35, Fujii Masao wrote:
On Fri, Jan 20, 2012 at 11:11 PM, Heikki Linnakangas
On Thu, Feb 9, 2012 at 7:25 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Thu, Feb 9, 2012 at 3:32 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On Wed, Feb 1, 2012 at 11:46 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 31.01.2012 17:35, Fujii Masao wrote:
On Fri, Jan
On Wed, Feb 1, 2012 at 11:46 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 31.01.2012 17:35, Fujii Masao wrote:
On Fri, Jan 20, 2012 at 11:11 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 20.01.2012 15:32, Robert Haas wrote:
On Sat, Jan 14,
On Fri, Jan 20, 2012 at 11:11 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 20.01.2012 15:32, Robert Haas wrote:
On Sat, Jan 14, 2012 at 9:32 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Here's another version of the patch to make XLogInsert less
On Sat, Jan 14, 2012 at 9:32 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Here's another version of the patch to make XLogInsert less of a bottleneck
on multi-CPU systems. The basic idea is the same as before, but several bugs
have been fixed, and lots of misc. clean up has
On Fri, Jan 20, 2012 at 2:11 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 20.01.2012 15:32, Robert Haas wrote:
On Sat, Jan 14, 2012 at 9:32 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Here's another version of the patch to make XLogInsert less
48 matches
Mail list logo
