[HACKERS] Opportunity for a Radical Changes in Database Software

2007-10-25 Thread Dan
k better if the master or master log entry moves from machine to machine with a commit coinciding with a disk write on each machine? Any other ideas? It seems to be a problem worth pondering since in-memory databases are possible. Thanks Dan ---(e

Re: [HACKERS] autoconf taking forever?

2001-10-18 Thread Dan
I always found with new machines and configure scripts is if gethostname does not resolve then the autoconfig will hang. I would make sure your /etc/resolve.conf /etc/hosts , hostname domainname are setup right and resolve from the command line.man gethostbyname On Thu, 18 Oct 2001, Tom L

Re: [HACKERS] compiling libpq++ on Solaris with Sun SPRO6U2 (fixed

2001-10-17 Thread Dan
whats wrong with kill -9' the postmaster works fine for me hahahaa. > Date: Wed, 17 Oct 2001 22:34:47 +0200 (CEST) > From: Peter Eisentraut <[EMAIL PROTECTED]> > To: Denis A Ustimenko <[EMAIL PROTECTED]> > Cc: [EMAIL PROTECTED] > Subject: Re: [HACKERS] compiling libpq++ on Solaris with Sun SP

Re: [HACKERS] Replication toolkit added to repository

2000-12-20 Thread Dan
rojects? Have you already explored working together? Many, including me, are chomping at the bit for an opensource replication ability in pgsql. Thx, -Dan.

[HACKERS] PGCon 2016 CFP - one week left

2016-01-12 Thread Dan Langille
Hello There is one week left in the PGCon CFP. Details below. Please submit. Thanks. PGCon 2016 will be on 17-21 May 2016 at University of Ottawa. * 17-18 (Tue-Wed) tutorials * 19 & 20 (Thu-Fri) talks - the main part of the conference * 17 & 21 (Wed & Sat) The Developer Unconference & the Us

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-10-03 Thread Wood, Dan
I’ve just started looking at this again after a few weeks break. There is a tangled web of issues here. With the community fix we get a corrupted page(invalid redirect ptr from indexed item). The cause of that is: pruneheap.c: /* * Check the tuple XMIN agai

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-10-03 Thread Wood, Dan
| 49155 | 36963 | 36961 7 | (0,7) | 8032 | 11010 | 32771 | 36961 | 0 (7 rows) On 10/3/17, 6:20 PM, "Peter Geoghegan" wrote: On Tue, Oct 3, 2017 at 6:09 PM, Wood, Dan wrote: > I’ve just started looking at this again after a fe

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-10-04 Thread Wood, Dan
Whatever you do make sure to also test 250 clients running lock.sql. Even with the communities fix plus YiWen’s fix I can still get duplicate rows. What works for “in-block” hot chains may not work when spanning blocks. Once nearly all 250 clients have done their updates and everybody is waiti

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-10-05 Thread Wood, Dan
9 AM, Wood, Dan wrote: > Whatever you do make sure to also test 250 clients running lock.sql. Even with the communities fix plus YiWen’s fix I can still get duplicate rows. What works for “in-block” hot chains may not work when spanning blocks. Interesting. Which version did you t

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-10-08 Thread Wood, Dan
I’m unclear on what is being repro’d in 9.6. Are you getting the duplicate rows problem or just the reindex problem? Are you testing with asserts enabled(I’m not)? If you are getting the dup rows consider the code in the block in heapam.c that starts with the comment “replace multi by update

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-10-10 Thread Wood, Dan
I found one glitch with our merge of the original dup row fix. With that corrected AND Alvaro’s Friday fix things are solid. No dup’s. No index corruption. Thanks so much. On 10/10/17, 7:25 PM, "Michael Paquier" wrote: On Tue, Oct 10, 2017 at 11:14 PM, Alvaro Herrera wrote: > I

[HACKERS] PGCon 2017 registration now open

2017-04-17 Thread Dan Langille
gistration is now open at http://www.pgcon.org/2017/registration.php <http://www.pgcon.org/2017/registration.php> -- Dan Langille - BSDCan / PGCon d...@langille.org <mailto:d...@langille.org>

[HACKERS] PGCon 2014 - last chance

2014-01-19 Thread Dan Langille
<http://www.pgcon.org/2014/submissions.php> -- Dan Langille - http://langille.org signature.asc Description: Message signed with OpenPGP using GPGMail

Re: [HACKERS] SSI freezing bug

2013-10-03 Thread Dan Ports
ful thought is needed here... > It seems to me that a change such as you are now suggesting is > likely to be too invasive to back-patch.  Do you agree that it > would make sense to apply the patch I have proposed, back to 9.1, > and then consider any alternative as 9.4 material? I agr

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Dan Ports
gt; page-level lock, which would also create unnecessary conflicts. Dan -- Dan R. K. PortsUW CSEhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 16-bit page checksums for 9.2

2012-01-26 Thread Dan Scales
rue, right? bufmgr.c, line 1914 - bufToWrite no longer exists. You could revert changes from 1912-1920 localbuf.c, line 203 - as mentioned below, this comment is no longer relevant, you are checksumming local buffers now Dan - Original Message - From: "Noah Misch" T

[HACKERS] PGCon 2012 Call for Papers - reminder

2012-01-26 Thread Dan Langille
n 2012 are available from: <http://www.pgcon.org/2012/submissions.php> -- Dan Langille - http://langille.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 16-bit page checksums for 9.2

2012-01-27 Thread Dan Scales
e checksum calculation (PageSetVerificationInfo) to mdextend() (or preferably smgrextend()) as well? Otherwise, you won't be checksumming a bunch of the new pages. Dan - Original Message - From: "Robert Haas" To: "Dan Scales" Cc: "Noah Misch" , "Heikki

[HACKERS] PGCon 2012 Call for Papers - extension

2012-01-29 Thread Dan Langille
<http://www.pgcon.org/2012/papers.php> Instructions for submitting a proposal to PGCon 2012 are available from: <http://www.pgcon.org/2012/submissions.php> -- Dan Langille - http://langille.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] double writes using "double-write buffer" approach [WIP]

2012-02-03 Thread Dan Scales
more efficient. Can you let me know what the shared_buffers and RAM sizes were for your pgbench run? I can try running the same workload. If the size of shared_buffers is especially small compared to RAM, then we should increase the size of shared_buffers when using double_writes. Thanks, D

Re: [HACKERS] double writes using "double-write buffer" approach [WIP]

2012-02-05 Thread Dan Scales
the double-write files can be stored on. Thanks, Dan - Original Message - From: "Robert Haas" To: "Dan Scales" Cc: "PG Hackers" Sent: Friday, February 3, 2012 1:48:54 PM Subject: Re: [HACKERS] double writes using "double-write buffer" app

Re: [HACKERS] double writes using "double-write buffer" approach [WIP]

2012-02-06 Thread Dan Scales
f double writes were in use, they might be automatically switched over to full page writes for the duration of the base backup. And the double write file should not be part of the base backup. Dan - Original Message - From: "Fujii Masao" To: "Dan Scales" Cc: &qu

Re: [HACKERS] double writes using "double-write buffer" approach [WIP]

2012-02-08 Thread Dan Scales
gh. I did write the code so that any process can write a completed batch if the batch is not currently being flushed (so as to deal with crashes by backends). Having the backends flush the batches as they fill them up was just simpler for a first prototype. Dan - Original Message ----- Fro

Re: [HACKERS] RFC: Making TRUNCATE more "MVCC-safe"

2012-02-11 Thread Dan Ports
hat the combination of that code and this patch would ensure full serializability for TRUNCATE operations. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] SSI rw-conflicts and 2PC

2012-02-13 Thread Dan Ports
one we could plausibly backpatch to 9.1. But if the extra aborts after recovery seem too expensive, we may want to consider one of the other options for future releases. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSI rw-conflicts and 2PC

2012-02-14 Thread Dan Ports
gy if we wanted to go that route. I hadn't really considered it because I'm not that familiar with the xlog code (plus, the commit record already contains a variable length field, making it that much more difficult to add another). Dan -- Dan R. K. Ports MIT CSAIL

Re: [HACKERS] SSI rw-conflicts and 2PC

2012-02-14 Thread Dan Ports
On Tue, Feb 14, 2012 at 09:27:58AM -0600, Kevin Grittner wrote: > Heikki Linnakangas wrote: > > On 14.02.2012 04:57, Dan Ports wrote: > >> The easiest answer would be to just treat every prepared > >> transaction found during recovery as though it had a conflict in >

[HACKERS] possible new option for wal_sync_method

2012-02-16 Thread Dan Scales
nterest in this change, or comments on its usefulness/correctness? It would be just an extra option for wal_sync_method that users can try out and has benefits for certain configurations. Dan diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 266c0de..a830a01 1

Re: [HACKERS] possible new option for wal_sync_method

2012-02-16 Thread Dan Scales
options for ext3. I don't think that I have seen a significant different when the DBT2 workload for ext3 option data=ordered. I will measure all these numbers again tonight, but with barrier=0, so as to try to confirm that the write flush itself isn't costing a lot for this configuratio

Re: [HACKERS] possible new option for wal_sync_method

2012-03-02 Thread Dan Scales
block device would do the trick of flushing the disk cache. Dan - Original Message - From: "Andres Freund" To: pgsql-hackers@postgresql.org Cc: "Dan Scales" Sent: Monday, February 27, 2012 12:43:49 PM Subject: Re: [HACKERS] possible new option for wal_sync_method Hi,

[HACKERS] a slightly stale comment

2012-03-06 Thread Dan Ports
turb a comment just before its 19th birthday, the bit about two-phase locking and serializability hasn't been correct since around 1999 (when MVCC was added). :-) Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] a slightly stale comment

2012-03-07 Thread Dan Ports
;t right either -- they may not be serializable, but that isn't the reason why. I don't particularly object to the warning that "the tests in this routine are correct" (although indeed the fact that they've changed over the years does seem to belie it). So I

[HACKERS] [PATCH] Never convert n_distinct < 2 values to a ratio when computing stats

2012-03-25 Thread Dan McGee
This is a bit of a corner case in all honesty, but if you have a short table (under 20 rows), the 10% heuristic used that decides whether distinct values scale with the row count will result in rather odd values for stadistinct in pg_statistic, such as '-0.2' or '-0.67', rather than the expecte

[HACKERS] PGCon 2015

2015-05-04 Thread Dan Langille
want to be there. Don't leave it much longer. — Dan Langille http://langille.org/ signature.asc Description: Message signed with OpenPGP using GPGMail

[HACKERS] HEADS UP: PGCon 2015 is in June

2014-09-27 Thread Dan Langille
HEADS UP. PGCon 2015 will be in June. That’s a few weeks later than in previous years. — Dan Langille signature.asc Description: Message signed with OpenPGP using GPGMail

[HACKERS] Validating CHECK constraints with SPI

2014-10-29 Thread Dan Robinson
rue already if you do the same with enable_indexscan off, but that requires knowing that PostgreSQL is going to do the seq scan no matter what.) Would y'all be open to a patch that made this change? Best, -Dan

Re: [HACKERS] Validating CHECK constraints with SPI

2014-10-29 Thread Dan Robinson
On Wed, Oct 29, 2014 at 7:17 AM, Alvaro Herrera wrote: > Dan Robinson wrote: > > Hi all, > > > > If I'm reading correctly in src/backend/commands/tablecmds.c, it looks > like > > PostgreSQL does a full table scan in validateCheckConstraint and in the &

[HACKERS] PGCon 2016 call for papers

2016-01-03 Thread Dan Langille
In case you've overlooked it, you have about two weeks to submit your proposal. PGCon 2016 will be on 17-21 May 2016 at University of Ottawa. * 17-18 (Tue-Wed) tutorials * 19 & 20 (Thu-Fri) talks - the main part of the conference * 17 & 21 (Wed & Sat) The Developer Unconference & the User Unconfe

[HACKERS] PGCon 2013 - call for papers

2012-12-08 Thread Dan Langille
php> Instructions for submitting a proposal to PGCon 2013 are available from: <http://www.pgcon.org/2013/submissions.php> -- Dan Langille - http://langille.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.po

Re: [HACKERS] Strange errors from 9.2.1 and 9.2.2 (I hope I'm missing something obvious)

2012-12-16 Thread Dan Scott
On Dec 11, 2012 9:28 PM, "David Gould" wrote: > > Thank you. I got the example via cut and paste from email and pasted it > into psql on different hosts. od tells me it ends each line with: > > \n followed by 0xC2 0xA0 and then normal spaces. The C2A0 thing is > apparently NO-BREAK SPACE. Invi

[HACKERS] PGCon 2013 - CFP & unconference day

2013-01-03 Thread Dan Langille
by developers and users of PostgreSQL. Be sure to submit your proposal soon because time is running out. -- Dan Langille - http://langille.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] serializable read only deferrable

2010-12-09 Thread Dan Ports
o pay to get a snapshot that can never see an anomaly. > Pseudo-code of idea (conveniently ignoring locking issues and > non-serializable transactions): This seems reasonable to me. Let me know if you need help implementing it; I have some spare cycles right now. Dan -- Dan R. K. Ports

Re: [HACKERS] SSI patch(es)

2011-01-10 Thread Dan Ports
On Sat, Jan 08, 2011 at 10:20:22PM -0600, Kevin Grittner wrote: > One thing that would help a lot besides code review is performance > testing. I did some months ago and I know Dan booked time on MIT > benchmarking systems and got good numbers, but with the refactoring > it would be

[HACKERS] PGCon 2011 Call for Papers - reminder

2011-01-11 Thread Dan Langille
e from: <http://www.pgcon.org/2011/submissions.php> -- Dan Langille - http://langille.org/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSI and 2PC

2011-01-12 Thread Dan Ports
On Tue, Jan 11, 2011 at 12:34:44PM -0600, Kevin Grittner wrote: > Agreed; but I am starting to get concerned about whether this > particular area can be completed by the start of the CF. I might > run a few days over on 2PC support. Unless ... Dan? Could you look > into this while

Re: [HACKERS] SSI patch version 12

2011-01-17 Thread Dan Ports
f I'm not mistaken (Kevin?), we can eliminate SERIALIZABLEXACTTAG altogether and not bother putting the vxid in the sxact. While we're at it, it probably wouldn't hurt to rename SerializableXactHashLock to PredTranLock or something, since there's no SerializableXactHash any

Re: [HACKERS] SSI patch version 12

2011-01-17 Thread Dan Ports
evel anymore. Oh, right, we do use it to generate pg_lock_status. I forgot about that particular tendril of code that extends outside of predicate.c. > Dan, if you're not already working on these, I'll take them, so you > can focus on 2PC. All yours! But if you feel like renaming

Re: [HACKERS] SSI patch version 12

2011-01-17 Thread Dan Ports
that as well as I might. Any thoughts on it? I'd been meaning to ask about that too. I couldn't think of any reason it would need to be volatile. Why do you think it might need to be? Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-

Re: [HACKERS] SSI and Hot Standby

2011-01-19 Thread Dan Ports
d REPEATABLE READ is exactly the same as the current (9.0) SERIALIZABLE behavior. But it's definitely something that should be addressed in documentation. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Dan Ports
mechanism for identifying snapshots where serialization failures like these will never happen. If we pass that information to the slave and allow it to run transactions only on those snapshots, serializability is safe. Hopefully that made some more sense... Dan -- Dan R. K. Ports

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Dan Ports
all happened at the master, T2 would get rolled back when it tries to do its INSERT. (I just tried it.) But if T3 happened on the slave, the master doesn't know that it read both tables, nor does the slave know at the time it's executing T3 that it's going to conflict with T2. Dan

Re: [HACKERS] SSI patch version 14

2011-01-25 Thread Dan Ports
till have relevant predicate > locks? When a SerializableXact gets summarized to the SLRU, its predicate locks aren't released; they're transferred to the dummy OldCommittedSxact. > I'll keep working on this patch. I hope I can be of some help getting > this committed, because

Re: [HACKERS] leaking lots of unreferenced inodes (pg_xlog files?), maybe after moving tables and indexes to tablespace on different volume

2013-03-13 Thread Dan Thomas
now we've been upgrading things in the hope that the problem will go away, but since we've got one server up to fbsd9.1/pg9.2.3 and still seeing the problem we're a little stumped. Any ideas about how we can go about debugging this would be appreciated. Thanks, Dan On 13 March 2013 07

Re: [HACKERS] lwlock contention with SSI

2013-04-10 Thread Dan Ports
ALIZABLEXACTs and RWConflicts, the SerializableXidHash table, the latest SxactCommitSeqno and SxactGlobalXmin, etc. I'm trying to swap back in my notes about how to address this. It is bound to be a substantial project, however. Dan -- Dan R. K. PortsUW CSE

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-10 Thread Dan Ports
instead of COMMIT, T2 would be allowed to proceed. Dan -- Dan R. K. PortsUW CSEhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Doc typo: lexems -> lexemes

2012-09-11 Thread Dan Scott
I ran across a minor typo while reviewing the full-text search documentation. Attached is a patch to address the one usage of "lexems" in a sea of "lexemes". diff --git a/doc/src/sgml/textsearch.sgml b/doc/src/sgml/textsearch.sgml new file mode 100644 index 978aa54..5305198 *** a/doc/src/sgml/text

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-11 Thread Dan Ports
transactions to execute concurrently!) What I was getting at in my previous mail was that there aren't any situations where COMMIT will return a serialization failure for a read-only transaction. Dan -- Dan R. K. PortsUW CSEhttp://drkp.net/ -- Sent via pg

Re: [HACKERS] plpgsql gram.y make rule

2012-09-24 Thread Dan Scott
a specific file or directory beyond a renaming event. git blame will show you the commit that renamed the file, by default, but then you can request the revision prior to that using the commit hash || '^', for example. "git blame 2fb6cc90^ -- src/backend/parser/gram.y" to work

Re: [HACKERS] Extending range of to_tsvector et al

2012-09-30 Thread Dan Scott
think they should behave - and we could start building some test cases as a first step? -- Dan Scott Laurentian University -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Doc patch, normalize search_path in index

2012-09-30 Thread Dan Scott
at indexing "search path"-the-concept is useful for translations, and the Japanese translation includes an index (I couldn't find the index for the French translation). -- Dan Scott Laurentian University -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Extending range of to_tsvector et al

2012-09-30 Thread Dan Scott
Hi John: On Sun, Sep 30, 2012 at 11:45 PM, john knightley wrote: > Dear Dan, > > thank you for your reply. > > The OS I am using is Ubuntu 12.04, with PostgreSQL 9.1.5 installed on > a utf8 local > > A short 5 line dictionary file is sufficient to test:- >

[HACKERS] PGCon 2010 - registered yet?

2010-04-23 Thread Dan Langille
. -- Dan Langille - http://langille.org/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Serializable snapshot isolation error logging

2010-09-20 Thread Dan S
Hi ! I wonder if the SSI implementation will give some way of detecting the cause of a serialization failure. Something like the deadlock detection maybe where you get the sql-statements involved. Best Regards Dan S

Re: [HACKERS] Serializable snapshot isolation error logging

2010-09-20 Thread Dan S
f, take an explicit lock on and so on to reduce serialization failure rate. If holding a list of the involved transactions turns out to be expensive, maybe one should be able to turn it on by a GUC only when you have a problem and need the extra information to track it down. Best Regards Dan S 2010/

Re: [HACKERS] Serializable snapshot isolation error logging

2010-09-21 Thread Dan S
y for concurrency. But what else if there is no way of knowing what the slow transaction conflicts against. As things with concurrency involved have a tendency to pop up in production and not in test I think it is important to start thinking about them as soon as possible. Best Regards Dan S

Re: [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-04 Thread Dan Ports
he other major bottleneck they ran into was a kernel one: reading from the heap file requires a couple lseek operations, and Linux acquires a mutex on the inode to do that. The proper place to fix this is certainly in the kernel but it may be possible to work around in Postgres. Dan -- Dan R. K

Re: [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-04 Thread Dan Ports
he lock-free structures helpe with both, so the impact of changing NUM_LOCK_PARTITIONS was less interesting. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscripti

Re: [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-04 Thread Dan Ports
ocks without needing an exclusive LWLock, and acquiring shared LWLocks without spinlocks if possible. I think the buffer cache manager is the next bottleneck after the row/table lock manager. Seems like it would also be a good candidate for similar techniques, but that's totally uninformed

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Dan Ports
s because they did not have the time/motivation to get the work into a polished, acceptable state, and in part because of the reputation of the community. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsq

[HACKERS] SSI non-serializalbe UPDATE performance (was: getting to beta)

2011-04-22 Thread Dan Ports
page locked, so it's not able to take any relevant SIREAD locks. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSI non-serializable UPDATE performance

2011-04-24 Thread Dan Ports
On Sat, Apr 23, 2011 at 08:54:31AM -0500, Kevin Grittner wrote: > Even though this didn't show any difference in Dan's performance > tests, it seems like reasonable insurance against creating a new > bottleneck in very high concurrency situations. > > Dan, do you have a

Re: [HACKERS] SSI non-serializable UPDATE performance

2011-04-27 Thread Dan Ports
a serializable transaction? That is correct, now. (Well, other than having to check whether a serializable transaction is running, the cost of which is truly negligible.) Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-04-27 Thread Dan Ports
tion of keeping locks past backend termination for 2PC prepared transactions, but it would be hard to apply here. The most practical solutions I see here are to, when acquiring an AccessExclusive lock, either wait for any associated SIREAD locks to go away, or to promote them to relation level. Dan -

Re: [HACKERS] SSI non-serializable UPDATE performance

2011-04-28 Thread Dan Ports
serializable transaction after that, we know it can't take any SIREAD locks on the same target while we're holding the buffer page lock. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] SSI non-serializable UPDATE performance

2011-04-29 Thread Dan Ports
if you think it helps. Patch attached. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c index fe0e458..694e87d 100644 --- a/src/backend/storage/lmgr/predicate.c +++ b/src/back

Re: [HACKERS] Predicate locking

2011-05-02 Thread Dan Ports
the initial SSI development has been to get something that's functionally correct and stable before optimizing it. I'm hoping to get some time to work on index-key locking for 9.2, as I expect it will make a significant performance difference. Dan -- Dan R. K. Ports MIT CSA

[HACKERS] patch: fix race in SSI's CheckTargetForConflictsIn

2011-05-04 Thread Dan Ports
t any need to worry about concurrent updates to the target's lock list. The resulting code is also simpler. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c index 3309d07..c2

[HACKERS] Re: Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Dan Ports
ywhere between there and where it is now. Do you have any preference? Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] patch: fix race in SSI's CheckTargetForConflictsIn

2011-05-06 Thread Dan Ports
refer the != NULL myself). I believe I've seen both elsewhere, though... Will update the patch. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://

Re: [HACKERS] patch: fix race in SSI's CheckTargetForConflictsIn

2011-05-08 Thread Dan Ports
On Fri, May 06, 2011 at 10:49:22PM -0400, Dan Ports wrote: > Will update the patch. Updated patch (in response to Robert's comments) attached. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend

Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

2011-05-21 Thread Dan Ports
) I just posted them at http://drkp.net/drkp/papers/ssi-pgcon11-slides.pdf ...and they'll be linked from the Serializable wiki page as soon as I remember how to edit it. :-) Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers maili

Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

2011-05-23 Thread Dan Ports
We check whether the conflict has caused the writer to become a pivot, but only if neither the reader or writer is committed. Why is that last condition there? In this case, the reader (T4) has committed but the writer (T1) hasn't. It would be worth making sure there aren't any other cases

Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

2011-05-23 Thread Dan Ports
conflict) That means T0 committed before T3 started, and therefore there can't be a rw-conflict from T3 to T0. In both cases, we didn't need the T1 -> T3 edge. Does that make sense to you? Unless anyone can poke a hole in that reasoning, I think we can get rid of the code to handle

Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

2011-05-24 Thread Dan Ports
However, because the rw-conflict can't be pointing to a transaction that precedes T1 in the serial order, it won't create a cycle. In other words, there are serialization failures that won't happen anymore, but they were false positives. Dan -- Dan R. K. Ports MIT CSAIL

Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

2011-06-02 Thread Dan Ports
On Wed, Jun 01, 2011 at 05:09:09PM -0500, Kevin Grittner wrote: > I won't be shocked if Dan can come up with a shorter proof, but I'm > confident this one is solid. Well, so happens I wrote a proof on the airplane today, before I saw your mail. It's actually quite straigh

Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

2011-06-02 Thread Dan Ports
en to be the transaction in the cycle with the earliest commit time. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-05 Thread Dan Ports
ons. This combination of conditions seems quite unlikely and I have a hard time getting too worked up about it. Occasional false positives are already a fact of life when using SSI. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing l

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-07 Thread Dan Ports
On Tue, Jun 07, 2011 at 07:45:43PM -0500, Kevin Grittner wrote: > During testing of the SSI DDL changes I noticed that a REINDEX INDEX > created a predicate lock on page 0 of the index. Really? That surprises me, and I couldn't reproduce it just now. Dan -- Dan R. K. Ports

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-08 Thread Dan Ports
the HeapScanDesc to indicate this. Actually, setting rs_relpredicatelocked has exactly that effect, so we ought to be able to use that (but might want to change the name). Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing

Re: [HACKERS] SSI heap_insert and page-level predicate locks

2011-06-08 Thread Dan Ports
d expect inserts to be tested against them. I'm not sure whether we'd actually do this, but we wanted to keep the option open during development. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] SSI work for 9.1

2011-06-08 Thread Dan Ports
ples by TID > doesn't lock not-found "gaps" the way an indexed access would. I can take care of this one and some other README-SSI changes. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] SSI work for 9.1

2011-06-08 Thread Dan Ports
g vacuum and recovery. The call in _bt_get_endpoint() seems unnecessary, because after it returns, _bt_endpoint() takes the same lock. The only other callers of _bt_get_endpoint() are _bt_pagedel() and _bt_insert_parent(), neither of which should take predicate locks. I've updated the pat

Re: [HACKERS] SSI work for 9.1

2011-06-09 Thread Dan Ports
unless you can get to it first, in which case I wouldn't complain) Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSI work for 9.1

2011-06-09 Thread Dan Ports
On Thu, Jun 09, 2011 at 01:30:27PM -0400, Dan Ports wrote: > On Thu, Jun 09, 2011 at 07:06:18AM -0500, Kevin Grittner wrote: > > Sounds reasonable, but why did you pass the snapshot to the > > PredicateLockPage() call but not the PredicateLockRelation() call? > > Oversig

Re: [HACKERS] Small SSI issues

2011-06-10 Thread Dan Ports
; > warnings.) > > I believe we can drop it, I'll double-check. Yes, dropping it seems like the thing to do. It's been on my list for a while. We are not really getting anything out of declaring it volatile since we cast the volatile qualifier away most of the time. Dan --

Re: [HACKERS] Small SSI issues

2011-06-11 Thread Dan Ports
pointing to) is in backend-local memory, so it won't change under us. The volatile qualifier (as written) doesn't help with that anyway, it attaches to the data being pointed to, not the pointer itself. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ --

Re: [HACKERS] SSI patch renumbered existing 2PC resource managers??

2011-06-13 Thread Dan Ports
On Mon, Jun 13, 2011 at 10:22:19PM +0300, Heikki Linnakangas wrote: > As far as I can tell it was for purely cosmetic reasons, to have lock > and predicate lock lines together. Yes, that is the only reason. Dan -- Dan R. K. Ports MIT CSAILhttp://dr

Re: [HACKERS] SSI patch renumbered existing 2PC resource managers??

2011-06-13 Thread Dan Ports
On Mon, Jun 13, 2011 at 03:33:24PM -0400, Tom Lane wrote: > We can either change that now, or undo the > unnecessary change in existing RM IDs. I vote for the latter. Sounds good to me. I'd offer a patch, but it'd probably take you longer to apply than to make the change yoursel

[HACKERS] patch: update README-SSI

2011-06-15 Thread Dan Ports
future tense, probably because they were written long ago), and remove a couple items from the "R&D Issues" list that have since been addressed. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ diff --git a/src/backend/storage/lmgr/README-SSI

Re: [HACKERS] patch: update README-SSI

2011-06-16 Thread Dan Ports
. > I don't see how there can be a ww-dependency between T0 and Tin. There > can't be a rw-conflict because Tin is read-only, so surely there can't > be a ww-conflict either? Yes, it can only be a wr-conflict. Good catch. Dan -- Dan R. K. Ports MIT CSAIL

  1   2   3   4   >