[HACKERS] WIP -- renaming implicit sequences

2012-01-11 Thread Thomas Munro
locking levels? What should I read to understand the rules of locking? Have I failed to handle errors? Have I made memory ownership mistakes? Thanks! Thomas Munro diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c index 335bdc6..cd6318e 100644 --- a/src/backend

Re: [HACKERS] WIP -- renaming implicit sequences

2012-01-12 Thread Thomas Munro
On 12 January 2012 00:58, Tom Lane t...@sss.pgh.pa.us wrote: Thomas Munro mu...@ip9.org writes: Here is an unfinished patch to implement something which appears on the TODO list under ALTER: automatic renaming of sequences created with serial when the table and column names change.  I've often

Re: [HACKERS] Multithread Query Planner

2012-01-14 Thread Thomas Munro
On 13 January 2012 20:14, Frederico zepf...@gmail.com wrote: I'm trying to develop a multithread planner, and some times is raised a exception of access memory. I was a bit confused about what you are trying to do -- somehow use concurrency during the planning phase, or during execution (maybe

Re: [HACKERS] WIP -- renaming implicit sequences

2012-01-14 Thread Thomas Munro
On 12 January 2012 00:58, Tom Lane t...@sss.pgh.pa.us wrote: Hmm ... this seems a bit inconsistent with the fact that we got rid of automatic renaming of indexes a year or three back.  Won't renaming of serials have all the same problems that caused us to give up on renaming indexes? I was

[HACKERS] SKIP LOCKED DATA

2012-01-15 Thread Thomas Munro
that HeapTupleSelfUpdated is treated -- that is, goto lnext to fetch the next tuple. 4. Probably some changes to handle table-level locks too. 5. Probably many other things that I'm not aware of right now and won't discover until I dig/ask further and/or run into a brick wall! Useful? Doable? Thanks, Thomas Munro

Re: [HACKERS] SKIP LOCKED DATA

2012-01-17 Thread Thomas Munro
On 16 January 2012 08:06, Ilya Kosmodemiansky hydrobi...@gmail.com wrote: That is quite useful feature to implement smth. like message queues based on database and so on. Now there is possibility to jump over luck of such feature in Postgres using current advisory lock implementation

Re: [HACKERS] WIP -- renaming implicit sequences

2012-01-21 Thread Thomas Munro
On 19/01/2012, Robert Haas robertmh...@gmail.com wrote: rhaas=# alter sequence foo_a_seq rename to bob; ALTER SEQUENCE If somebody renames the table or the column at this point, it's a good bet that they *don't* want bob renamed. FWIW the patch addresses this case and wouldn't try to rename

Re: [HACKERS] SKIP LOCKED DATA

2012-02-04 Thread Thomas Munro
On 16 January 2012 21:30, Josh Berkus j...@agliodbs.com wrote: Useful, yes.  Harder than it looks, probably.  I tried to mock up a version of this years ago for a project where I needed it, and ran into all kinds of race conditions. Can you remember any details about those race conditions?

[HACKERS] pg_regress running for ~10 hours using 100% CPU

2012-09-30 Thread Thomas Munro
Hi Buildfarm machine lyrebird, clang 2.9, amd64, Debian 6, 'HEAD'. I have been unsuccessful in finding out what it's doing using gdb: pg_animal@asterix:~/work/HEAD/pgsql$ gdb --pid=10681 GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version

Re: [HACKERS] Should select 'nan'::float = 'nan'::float; return false as per IEEE 754

2012-10-28 Thread Thomas Munro
On 28 October 2012 09:43, Hannu Krosing ha...@2ndquadrant.com wrote: This is how PostgreSQL currently works - test=# select 'NaN'::float = 'NaN'::float as must_be_false; must_be_false -- t (1 row) I think that PostgreSQL's behaviour of comparing two NaN-s as equal is wrong and

Re: [HACKERS] Should select 'nan'::float = 'nan'::float; return false as per IEEE 754

2012-10-28 Thread Thomas Munro
On 28 October 2012 10:37, Hannu Krosing ha...@2ndquadrant.com wrote: On 10/28/2012 11:21 AM, Thomas Munro wrote: On 28 October 2012 09:43, Hannu Krosing ha...@2ndquadrant.com wrote: This is how PostgreSQL currently works - test=# select 'NaN'::float = 'NaN'::float as must_be_false

Re: [HACKERS] Covering Indexes

2012-06-30 Thread Thomas Munro
On 28 June 2012 14:02, Rob Wultsch wult...@gmail.com wrote: On Thu, Jun 28, 2012 at 8:16 AM, David E. Wheeler da...@justatheory.com wrote: I'm particularly intrigued by covering indexes. For example:    CREATE INDEX cover1 ON table1(a,b) COVERING(c,d); IRC MS SQL also allow unindexed

[HACKERS] NOTIFY with tuples

2011-12-13 Thread Thomas Munro
thought I'd present this idea and see if it's a duplicate effort, or someone has a much better idea, or it is instantly shot down in flames for technical or other reasons, before investing further in it. Thanks! Thomas Munro [1] http://en.wikipedia.org/wiki/StreamSQL [2] http://de.wikipedia.org

Re: [HACKERS] NOTIFY with tuples

2011-12-13 Thread Thomas Munro
On 14 December 2011 04:21, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Dec 13, 2011 at 6:30 PM, Thomas Munro mu...@ip9.org wrote: I imagine a very simple system like this, somehow built on top of the existing NOTIFY infrastructure: I'm not sure

Re: [HACKERS] NOTIFY with tuples

2011-12-14 Thread Thomas Munro
On 14 December 2011 15:10, Merlin Moncure mmonc...@gmail.com wrote: As to the wider point I'm wondering why you can't layer your API on top of existing facilities (tables, notifications, etc). PGQ (have you seen that?) does this and it's an absolute marvel.  Meaning, I bet you could do this

Re: [HACKERS] Adding IEEE 754:2008 decimal floating point and hardware support for it

2013-06-12 Thread Thomas Munro
On 12 June 2013 00:56, Craig Ringer cr...@2ndquadrant.com wrote: The main thing I'm wondering is how/if to handle backward compatibility with the existing NUMERIC and its DECIMAL alias, or whether adding new DECIMAL32, DECIMAL64, and DECIMAL128 types would be more appropriate. I'd love to

Re: [HACKERS] Re: Adding IEEE 754:2008 decimal floating point and hardware support for it

2013-06-20 Thread Thomas Munro
). Finally, I recently ran into a 3rd software implementation of 754-2008: libmpdec (the other two being IBM decNumber and Intel's library), but I haven't looked into it yet. Thomas Munro

Re: [HACKERS] Re: Adding IEEE 754:2008 decimal floating point and hardware support for it

2013-06-20 Thread Thomas Munro
On 20 June 2013 08:05, Thomas Munro mu...@ip9.org wrote: On 20 June 2013 06:45, Craig Ringer cr...@2ndquadrant.com wrote: If the performance isn't interesting it may still be worth adding for compliance reasons, but if we can only add IEEE-compliant decimal FP by using non-SQL-standard type

[HACKERS] CLUSTER FREEZE

2013-10-23 Thread Thomas Munro
Hi I noticed that CLUSTER doesn't have a FREEZE option. Here is a patch to add that, for consistency with VACUUM. Is it useful? Thanks Thomas Munro cluster-freeze.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] CLUSTER FREEZE

2013-10-24 Thread Thomas Munro
On 24 October 2013 05:58, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Oct 24, 2013 at 4:58 AM, Thomas Munro mu...@ip9.org wrote: Hi I noticed that CLUSTER doesn't have a FREEZE option. Here is a patch to add that, for consistency with VACUUM. Is it useful? I wonder why anyone

Re: [HACKERS] CLUSTER FREEZE

2013-10-25 Thread Thomas Munro
attach an alternative patch that makes CLUSTER *always* freeze, without any option (but doesn't affect VACUUM FULL in the same way). I will post both alternatives to the commitfest app since there seems to be some disagreement about whether tuple freezing should be an optional. Thanks Thomas Munro

[HACKERS] Compression of tables

2013-12-09 Thread Thomas Munro
Hi I have been wondering what the minimum useful heap table compression system would be for Postgres, in order to reduce disk footprint of large mostly static datasets. Do you think an approach similar to the static row-level compression of DB2 could make sense? I imagine something like this:

Re: [HACKERS] Compression of tables

2013-12-14 Thread Thomas Munro
On 10 December 2013 15:15, Merlin Moncure mmonc...@gmail.com wrote: I doubt you'll ever see generally heap compressed data in the way you're thinking: postgres has a strong informal policy of not implementing features which are dubious and or excessively complicated with limited benefit,

[HACKERS] Cluster name in ps output

2014-05-05 Thread Thomas Munro
Hi When running more than one cluster I often find myself looking at the output of 'iotop' or other tools wondering which cluster's wal receiver process or checkpointer process etc I'm seeing. Obviously it's easy enough to find out (for example by looking at a tree view in htop/ps that shows the

Re: [HACKERS] Cluster name in ps output

2014-05-05 Thread Thomas Munro
On 5 May 2014 10:10, Andres Freund and...@2ndquadrant.com wrote: Hi, On 2014-05-05 10:00:34 +, Thomas Munro wrote: When running more than one cluster I often find myself looking at the output of 'iotop' or other tools wondering which cluster's wal receiver process or checkpointer

Re: [HACKERS] Cluster name in ps output

2014-05-05 Thread Thomas Munro
On 5 May 2014 10:49, Thomas Munro mu...@ip9.org wrote: On 5 May 2014 10:10, Andres Freund and...@2ndquadrant.com wrote: I guess the question is where this should be available as well. At the very least I'd want to reference it in log_line_prefix as well? Good idea, I will look

Re: [HACKERS] Cluster name in ps output

2014-05-05 Thread Thomas Munro
-name-in-ps-v3-b.patch. I will try to add these to the open commitfest, and see if there is something I can usefully review in return. I verified that SHOW cluster_name works as expected and you can't change it with SET. Thanks, Thomas Munro diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-05-16 Thread Thomas Munro
shorter is sweeter * I used the term wait_policy and an enum, Simon used waitMode and an int * I had noWait and skipLocked travelling separately in some places, Simon had a single parameter, which is much better Best regards, Thomas Munro

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-05-19 Thread Thomas Munro
not experienced at benchmarking and I don't claim that this particular workload or configuration is particularly sensible or representative of anything but it might give some idea of the motivation. Best regards, Thomas Munro

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-05-23 Thread Thomas Munro
I think I prefer NOWAIT EXCLUDE (perhaps with NOWAIT ABORT as a long version of the existing NOWAIT behaviour for contrast). Or adding just one new keyword: NOWAIT SKIP SKIP LOCK Regards, Thomas Munro

Re: [HACKERS] Cluster name in ps output

2014-06-26 Thread Thomas Munro
incorrect static initialization with a string literal was intended to make sure logging would work before the GUC machinery has finished setting up default values, but that no longer applies.) Regards, Thomas Munro diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index e3d1c62

Re: [HACKERS] Cluster name in ps output

2014-06-29 Thread Thomas Munro
done so, we should add a double colon after the cluster name, so it's not: postgres: server1 stats collector process but postgres: server1: stats collector process +1 Best regards, Thomas Munro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] DISTINCT with btree skip scan

2014-07-04 Thread Thomas Munro
to send such half-baked early hacking phase code to the list, but thought it would make sense to demo the concept and then seek advice, warnings, cease and desist notices etc before pressing on down that route!) I would be most grateful for any feedback you might have. Best regards, Thomas Munro

Re: [HACKERS] DISTINCT with btree skip scan

2014-07-05 Thread Thomas Munro
On 5 July 2014 02:03, Vik Fearing vik.fear...@dalibo.com wrote: [1] http://wiki.postgresql.org/wiki/Loose_indexscan Thanks. It talks about DISTINCT, and also about using index when you don't have the leading column in your WHERE clause (as well as MySQL (loose), at least Oracle (skip), SQLite

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-07-24 Thread Thomas Munro
On 24 July 2014 00:52, Thomas Munro mu...@ip9.org wrote: On 23 July 2014 13:15, David Rowley dgrowle...@gmail.com wrote: I'm also wondering about this block of code in general: if (erm-waitPolicy == RWP_WAIT) wait_policy = LockWaitBlock; else if (erm-waitPolicy == RWP_SKIP ) wait_policy

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-07-26 Thread Thomas Munro
On 24 July 2014 00:52, Thomas Munro mu...@ip9.org wrote: On 23 July 2014 13:15, David Rowley dgrowle...@gmail.com wrote: I'm also wondering about this block of code in general: if (erm-waitPolicy == RWP_WAIT) wait_policy = LockWaitBlock; else if (erm-waitPolicy == RWP_SKIP ) wait_policy

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-07-26 Thread Thomas Munro
On 26 July 2014 15:43, Tom Lane t...@sss.pgh.pa.us wrote: Thomas Munro mu...@ip9.org writes: I couldn't find an existing reasonable place to share a single wait policy enumeration between parser/planner/executor and the heap access module, and I get the feeling that it would be unacceptable

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-07-27 Thread Thomas Munro
On 27 July 2014 14:31, David Rowley dgrowle...@gmail.com wrote: On Sun, Jul 27, 2014 at 4:49 AM, Thomas Munro mu...@ip9.org wrote: Here is a new version of the patch with a single enum LockWaitPolicy defined in utils/lockwaitpolicy.h. That seems much cleaner A few more comments: You seem

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-07-28 Thread Thomas Munro
On 27 July 2014 23:19, Thomas Munro mu...@ip9.org wrote: On the subject of isolation tests, I think skip-locked.spec is only producing schedules that reach third of the three 'return HeapTupleWouldBlock' statements in heap_lock_tuple. I will follow up with some more thorough isolation tests

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-07-30 Thread Thomas Munro
regards Thomas Munro diff --git a/src/test/isolation/expected/nowait-2.out b/src/test/isolation/expected/nowait-2.out new file mode 100644 index 000..6e24bbb --- /dev/null +++ b/src/test/isolation/expected/nowait-2.out @@ -0,0 +1,43 @@ +Parsed test spec with 2 sessions + +starting permutation: s1a s2a

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-08-01 Thread Thomas Munro
this I can't see any other problems with the patch and I'd be very inclined, once the above are fixed up to mark the patch ready for commiter. Good work Thanks for all the guidance, I appreciate it! My review karma account is now well overdrawn. Best regards, Thomas Munro diff --git a/doc/src

Re: [HACKERS] pgcrypto: PGP signatures

2014-08-21 Thread Thomas Munro
loop in extract_signatures, but for (;;) in pullf_discard? It looks like the latter is much more common in the source tree.) Best regards, Thomas Munro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-08-24 Thread Thomas Munro
that locked the updated version? It might be difficult to hit this case or construct an isolationtester spec file though, since there's a narrow window you need to race to. Hmm. I will look into this, thanks. Best regards, Thomas Munro diff --git a/src/backend/access/heap/heapam.c b/src/backend

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-08-24 Thread Thomas Munro
On 24 August 2014 22:04, Thomas Munro mu...@ip9.org wrote: On 22 August 2014 23:02, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Did you consider heap_lock_updated_tuple? A rationale for saying it doesn't need to pay attention to the wait policy is: if you're trying to lock-skip-locked

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-08-25 Thread Thomas Munro
On 25 August 2014 02:57, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: The difficulty of course will be testing all these racy cases reproducibly... Does this help? http://www.postgresql.org/message-id/51fb4305.3070...@2ndquadrant.com The useful trick there is forcing

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-08-27 Thread Thomas Munro
On 27 August 2014 17:18, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: On 25 August 2014 02:57, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: The difficulty of course will be testing all these racy cases reproducibly... Does this help? http

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-08-28 Thread Thomas Munro
On 28 August 2014 00:25, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: Thanks, I hadn't seen this, I should have checked the archives better. I have actually already updated my patch to handle EvalPlanQualFetch with NOWAIT and SKIP LOCKED with isolation specs, see

[HACKERS] Multithreaded SIGPIPE race in libpq on Solaris

2014-08-28 Thread Thomas Munro
}; sigemptyset(sigpipe_sigset); sigaddset(sigpipe_sigset, SIGPIPE); - sigwait(sigpipe_sigset, signo); + sigtimedwait(sigpipe_sigset, siginfo, timeout); } } Does this make any sense? Best regards, Thomas

Re: [HACKERS] Multithreaded SIGPIPE race in libpq on Solaris

2014-08-28 Thread Thomas Munro
. It does seem more likely the crashing database triggered this somehow though, perhaps in combination with something else the client app was doing, though I can't think what it could be that would eat another thread's SIGPIPE in between the sigpending and sigwait syscalls. Best regards, Thomas

Re: [HACKERS] Multithreaded SIGPIPE race in libpq on Solaris

2014-08-29 Thread Thomas Munro
On 29 August 2014 01:04, Thomas Munro mu...@ip9.org wrote: On 28 August 2014 23:45, Tom Lane t...@sss.pgh.pa.us wrote: I don't claim to be an expert on this stuff, but I had the idea that multithreaded environments were supposed to track signal state per-thread not just per-process, precisely

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-08-30 Thread Thomas Munro
On 28 August 2014 00:25, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: I haven't yet figured out how to get get into a situation where heap_lock_updated_tuple_rec waits. Well, as I think I said in the first post I mentioned this, maybe there is no such situation. In any

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-09-06 Thread Thomas Munro
On 31 August 2014 01:36, Thomas Munro mu...@ip9.org wrote: On 28 August 2014 00:25, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: I haven't yet figured out how to get get into a situation where heap_lock_updated_tuple_rec waits. Well, as I think I said in the first post

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-09-11 Thread Thomas Munro
On 10 September 2014 14:47, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: @@ -2022,7 +2030,7 @@ EvalPlanQualFetch(EState *estate, Relation relation, int lockmode, bool noWait, */ test = heap_lock_tuple(relation, tuple

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-09-14 Thread Thomas Munro
On 12 September 2014 03:56, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: But to reach the case you mentioned, it would need to get past that (xmax is not a valid transaction) but then the tuple would need to be locked by another session before heap_lock_tuple is called

Re: [HACKERS] NEXT VALUE FOR sequence

2014-10-02 Thread Thomas Munro
On 3 October 2014 00:01, Thomas Munro mu...@ip9.org wrote: On 2 October 2014 14:48, Tom Lane t...@sss.pgh.pa.us wrote: Thomas Munro mu...@ip9.org writes: SQL:2003 introduced the function NEXT VALUE FOR sequence. Google tells me that at least DB2, SQL Server and a few niche databases

Re: [HACKERS] NEXT VALUE FOR sequence

2014-10-04 Thread Thomas Munro
On 3 October 2014 00:18, Tom Lane t...@sss.pgh.pa.us wrote: Thomas Munro mu...@ip9.org writes: I suppose one approach would be to use command IDs as the scope. The spec clearly says one value per row, not one per statement; so command ID is very definitely not the right thing. I think

Re: [HACKERS] DISTINCT with btree skip scan

2014-10-31 Thread Thomas Munro
Best regards, Thomas Munro diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c index 53cf96f..5f10d7f 100644 --- a/src/backend/access/index/indexam.c +++ b/src/backend/access/index/indexam.c @@ -29,6 +29,7 @@ * index_can_return - does index support index-only scans

Re: [HACKERS] WALWriteLock contention

2015-05-17 Thread Thomas Munro
High Performance in chapter 4 (page 79). -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] CLUSTER on brin indexes

2015-05-14 Thread Thomas Munro
know these facts, then you know something potentially useful about the physical order of the tuples within those blocks. (It may be extremely difficult to make use of that information, I have no idea, so maybe this is crazy talk.) -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-04 Thread Thomas Munro
On Fri, Jun 5, 2015 at 11:47 AM, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Fri, Jun 5, 2015 at 9:29 AM, Robert Haas robertmh...@gmail.com wrote: Here's a new version with some more fixes and improvements: - SetOffsetVacuumLimit was failing to set MultiXactState-oldestOffset when

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-04 Thread Thomas Munro
. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-03 Thread Thomas Munro
2048 DETAIL: Could not read from file pg_multixact/offsets/ at offset 8192: Undefined error: 0. FATAL: could not access status of transaction 131072 DETAIL: Could not open file pg_multixact/offsets/0002: No such file or directory. But, yeah, this isn't the bug we're looking for. -- Thomas

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Thomas Munro
On Fri, Jun 5, 2015 at 1:47 PM, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Fri, Jun 5, 2015 at 11:47 AM, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Fri, Jun 5, 2015 at 9:29 AM, Robert Haas robertmh...@gmail.com wrote: Here's a new version with some more fixes

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-05 Thread Thomas Munro
On Sat, Jun 6, 2015 at 10:12 AM, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Sat, Jun 6, 2015 at 9:45 AM, Dan Langille d...@langille.org wrote: I noticed this today on my 9.4.2 server running on FreeBSD 10.1: Jun 5 18:59:40 slocum postgres[986]: [3957-1] LOG: could not truncate

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-05 Thread Thomas Munro
://www.postgresql.org/message-id/CAEepm=0DqAtnM=23oq44bbnwvn3g6+dxx+s5g4jrbp-vy8g...@mail.gmail.com I will try to repro this. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-05 Thread Thomas Munro
On Sat, Jun 6, 2015 at 1:25 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: My idea was that if I could get oldestXact == next XID in TruncateSUBSTRANS, then TransactionIdToPage(oldestXact) for a value of oldestXact that happens to be immediately after a page boundary

[HACKERS] Checkpoints vs restartpoints

2015-06-09 Thread Thomas Munro
recover time for lower write IO? I was wondering about this in the context of the recent multixact work, since such configurations could leave you with different SLRU files on disk which in some versions might change the behaviour in interesting ways. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-07 Thread Thomas Munro
On Sat, Jun 6, 2015 at 4:51 PM, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Sat, Jun 6, 2015 at 1:25 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: My idea was that if I could get oldestXact == next XID in TruncateSUBSTRANS, then TransactionIdToPage

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-07 Thread Thomas Munro
On Mon, Jun 8, 2015 at 12:29 PM, Thomas Munro thomas.mu...@enterprisedb.com wrote: Here's a repro script and a suggested patch. Argh... I realised immediately after sending this that subtransaction truncation doesn't even use the oldest XID computed by vacuum, it uses GetOldestXmin (the oldest

[HACKERS] get_relation_info comment out of sync

2015-06-21 Thread Thomas Munro
Hi The comment for get_relation_info should probably include serverid in the list of rel members that it can update (see attached). -- Thomas Munro http://www.enterprisedb.com get_relation_info-comment.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Inheritance planner CPU and memory usage change since 9.3.2

2015-06-20 Thread Thomas Munro
30751 Q4 500 500 1503 252003 252003 251502 126002501 -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dblink: add polymorphic functions - review

2015-06-18 Thread Thomas Munro
. I think you have to subscribe first so that you can log into it. If you start here and browse to the message in the archive, you'll see a 'Mail this message to ...' link at the bottom: https://lists.postgresql.org/mj/mj_wwwusr/domain=postgresql.org -- Thomas Munro http://www.enterprisedb.com

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-16 Thread Thomas Munro
On Wed, Jun 17, 2015 at 6:58 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thomas Munro wrote: Thanks. As mentioned elsewhere in the thread, I discovered that the same problem exists for page boundaries, with a different error message. I've tried the attached repro scripts on 9.3.0

[HACKERS] Inheritance planner CPU and memory usage change since 9.3.2

2015-06-17 Thread Thomas Munro
for some users after an upgrade. [1] http://www.postgresql.org/message-id/8c9acaa.1f453.14c0da0402f.coremail.chjis...@163.com [2] http://www.postgresql.org/message-id/flat/20141107185824.2513.53...@wrigleys.postgresql.org#20141107185824.2513.53...@wrigleys.postgresql.org -- Thomas Munro http

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-06-12 Thread Thomas Munro
://www.postgresql.org/message-id/CAEepm=0DqAtnM=23oq44bbnwvn3g6+dxx+s5g4jrbp-vy8g...@mail.gmail.com [2] https://commitfest.postgresql.org/5/265/ [3] http://www.postgresql.org/message-id/1274373980-sup-3...@alvh.no-ip.org -- Thomas Munro http://www.enterprisedb.com repro-bogus-multixact-error.sh Description

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Thomas Munro
ordering that causes trouble, but I don't yet see why it would break if you replay the WAL from the backup label checkpoint (and I think the repro would take days/weeks to run...) -- Thomas Munro http://www.enterprisedb.com copy-after-truncation.sh Description: Bourne shell script copy-before

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-05-28 Thread Thomas Munro
, filename); -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-05-29 Thread Thomas Munro
we encounter multixacts in tuple headers (updating, locking or vacuuming). If you have truncated multixacts referenced in your tuples then you have a different form of corruption than the pg_upgrade-tramples-on-oldestMultiXactId case we're trying to handle gracefully here. -- Thomas Munro http

Re: [HACKERS] Multixid hindsight design

2015-06-01 Thread Thomas Munro
in a minor release. What about prepared transactions? They can lock rows FOR SHARE that survive server restarts. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread Thomas Munro
suggestion pg_xact for that? It fits alongside pg_multixact. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-05-29 Thread Thomas Munro
On Sat, May 30, 2015 at 10:48 AM, Andres Freund and...@anarazel.de wrote: On 2015-05-30 10:41:01 +1200, Thomas Munro wrote: On Sat, May 30, 2015 at 10:29 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, May 29, 2015 at 5:14 PM, Josh Berkus j...@agliodbs.com wrote: Just saw what looks

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-05-29 Thread Thomas Munro
think any of the code that is affected here exists in 9.2, or that any of the fixes involved were back-patched to 9.2. So that might be something else altogether. Not only that, the pg_multixact/members *directory* is reported missing, which is a different problem entirely. -- Thomas Munro http

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-05-29 Thread Thomas Munro
. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-06-30 Thread Thomas Munro
it. +1 I was thinking recently that it would be very cool to see some estimation of the progress of VACUUM and CLUSTER in a view similar to pg_stat_activity, or the ps title. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Rework the way multixact truncations work

2015-06-29 Thread Thomas Munro
think saw some extra autovacuum activity that I didn't expect in a simple scenario, but I'm not sure and will continue looking tomorrow. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Solaris testers wanted for strxfrm() behavior

2015-06-28 Thread Thomas Munro
-- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Solaris testers wanted for strxfrm() behavior

2015-06-28 Thread Thomas Munro
On Mon, Jun 29, 2015 at 10:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: Thomas Munro thomas.mu...@enterprisedb.com writes: Just by the way, I wonder if this was that bug: https://illumos.org/issues/1594 Oooh. Might or might not be *same* bug, but it sure looks like it could have the right

Re: [HACKERS] query in PostgreSQL

2015-08-14 Thread Thomas Munro
internals are discussed. The best place to start might be pgsql-general mailing list or the #postgresql IRC channel, where you could explain what you're trying to do and post the errors you're seeing etc and hopefully get some help. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql

[HACKERS] Documentation tweak for row-valued expressions and null

2015-07-26 Thread Thomas Munro
Hi I wonder if it might be worth adding a tiny note to the manual to point out that the special logic for row-valued-expression IS [ NOT ] NULL doesn't apply anywhere else that we handle nulls or talk about [non]-null values in the manual. See attached. -- Thomas Munro http

[HACKERS] WAL test/verification tool

2015-07-17 Thread Thomas Munro
Hi I have heard rumours of a tool that could verify or compare the effects of applying WAL records for testing/development purposes, but I've been unable to track it down or find out if it was publicly released. Does anyone know the status of that or what it was called? Thanks, -- Thomas

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-23 Thread Thomas Munro
On Sat, Oct 24, 2015 at 6:19 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Sun, Oct 18, 2015 at 9:12 PM, Thomas Munro > <thomas.mu...@enterprisedb.com> wrote: >> Thanks for taking a look at this! The word count returned by >> get_previous_words was incorrect

Re: [HACKERS] bugs and bug tracking

2015-10-21 Thread Thomas Munro
DÅ©ng" on this page: https://granicus.if.org/pgbugs/13691 -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-21 Thread Thomas Munro
and there is interest in the patch, I'll keep doing that.) -- Thomas Munro http://www.enterprisedb.com tab-complete-macrology-v6.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-11 Thread Thomas Munro
On Wed, Nov 11, 2015 at 9:42 PM, Heikki Linnakangas <hlinn...@iki.fi> wrote: > On 11/11/2015 10:23 AM, Simon Riggs wrote: > >> On 11 November 2015 at 05:37, Thomas Munro <thomas.mu...@enterprisedb.com >> > >> wrote: >> >> Many sites use hot sta

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-11 Thread Thomas Munro
On Thu, Nov 12, 2015 at 12:10 AM, Ants Aasma <ants.aa...@eesti.ee> wrote: > On Wed, Nov 11, 2015 at 11:22 AM, Thomas Munro > <thomas.mu...@enterprisedb.com> wrote: > > On Wed, Nov 11, 2015 at 9:42 PM, Heikki Linnakangas <hlinn...@iki.fi> > wrote: > >>

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-11-11 Thread Thomas Munro
New version attached, merging recent changes. -- Thomas Munro http://www.enterprisedb.com tab-complete-v8.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-12 Thread Thomas Munro
On Fri, Nov 13, 2015 at 1:16 AM, Simon Riggs <si...@2ndquadrant.com> wrote: > On 11 November 2015 at 09:22, Thomas Munro <thomas.mu...@enterprisedb.com> > wrote: > > >> 1. Reader waits with exposed LSNs, as Heikki suggests. This is what >> BerkeleyDB does i

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-16 Thread Thomas Munro
lls which wouldn't play nicely with signals (whether from 'kill' or SetLatch) and can deal with network IO with the lowest possible latency. I would like to help test/review that, if that could be useful. The SIGUSR1 code in the WalReceiverMain and WalRecvWakeup in this patch works well enough for

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-15 Thread Thomas Munro
On Sun, Nov 15, 2015 at 11:41 PM, Simon Riggs <si...@2ndquadrant.com> wrote: > On 12 November 2015 at 18:25, Thomas Munro <thomas.mu...@enterprisedb.com> > wrote: > > >> I don't want to get bogged down in details, while we're talking about >> the 30,000 foo

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-11-15 Thread Thomas Munro
se this can be more sane/effective/maintainable by > refactoring. But before that issue, I'm not confident at all that > this is really a alternative with *gigantic* improvement. > > Any opinions? > It's an interesting idea to use regular expressions, but it's a shame to move the patterns so far away from the actions they trigger. -- Thomas Munro http://www.enterprisedb.com

  1   2   3   4   5   6   7   8   9   >