[HACKERS] minor spelling error fix (btis -> bits)

Attached please find a minor spelling error fix, changing "btis" to "bits". -- Jon From f590a6dce6677bc5b8a409d40fd651ecb69b27bb Mon Sep 17 00:00:00 2001 From: Jon Nelson <jnel...@jamponi.net> Date: Sun, 23 Mar 2014 08:23:48 -0500 Subject: [PATCH] - fix minor spelli

Re: [HACKERS] [PATCH] guc-ify the formerly hard-coded MAX_SEND_SIZE to max_wal_send

On Thu, Mar 16, 2017 at 9:59 AM, David Steele <da...@pgmasters.net> wrote: > On 1/9/17 11:33 PM, Jon Nelson wrote: > > > > On Sat, Jan 7, 2017 at 7:48 PM, Jim Nasby <jim.na...@bluetreble.com > > <mailto:jim.na...@bluetreble.com>> wrote: > > >

Re: [HACKERS] Faster methods for getting SPI results

On Thu, Mar 2, 2017 at 10:03 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 12/20/16 23:14, Jim Nasby wrote: > > I've been looking at the performance of SPI calls within plpython. > > There's a roughly 1.5x difference from equivalent python code just in > > pulling data out

Re: [HACKERS] [PATCH] guc-ify the formerly hard-coded MAX_SEND_SIZE to max_wal_send

d. I have verified that the GUC properly grumps about values greater than XLOG_SEG_SIZE / 1024 or smaller than 4. -- Jon From 7286e290daec32e12260e9e1e44a490c13ed2245 Mon Sep 17 00:00:00 2001 From: Jon Nelson <jnel...@jamponi.net> Date: Mon, 9 Jan 2017 20:00:41 -0600 Subject: [PATCH] guc-ify the f

Re: [HACKERS] macaddr 64 bit (EUI-64) datatype support

On Tue, Nov 22, 2016 at 8:42 AM, Tom Lane wrote: > Haribabu Kommi writes: > > Any suggestions for the name to be used for the new datatype the can > > work for both 48 and 64 bit MAC addresses? > > The precedent of int4/int8/float4/float8 is that

Re: [HACKERS] Change pg_cancel_*() to ignore current backend

On May 20, 2015 6:43 AM, David Steele da...@pgmasters.net wrote: On 5/20/15 1:40 AM, Jim Nasby wrote: On 5/19/15 9:19 PM, Fabrízio de Royes Mello wrote: We could add a second parameter to the current functions: allow_own_pid DEFAULT false. To me that seems better than an

Re: [HACKERS] Change pg_cancel_*() to ignore current backend

On Wed, May 20, 2015 at 9:09 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think backwards compatibility probably trumps that argument. I have no objection to providing a different call that behaves this way, but changing the behavior of existing applications will face a *much* higher barrier to

[HACKERS] CTE that result in repeated sorting of the data

I was watching a very large recursive CTE get built today and this CTE involves on the order of a dozen or so loops joining the initial table against existing tables. It struck me that - every time through the loop the tables were sorted and then joined and that it would be much more efficient if

Re: [HACKERS] CTE that result in repeated sorting of the data

On Thu, May 15, 2014 at 4:50 PM, David G Johnston david.g.johns...@gmail.com wrote: Jon Nelson-14 wrote I was watching a very large recursive CTE get built today and this CTE involves on the order of a dozen or so loops joining the initial table against existing tables. It struck me

Re: [HACKERS] PoC: Duplicate Tuple Elidation during External Sort for DISTINCT

What - if anything - do I need to do to get this on the commitfest list for the next commitfest? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PoC: Duplicate Tuple Elidation during External Sort for DISTINCT

What are my next steps here? I believe the concept is sound, the code is appears to work and doesn't crash, and the result does show a performance win in most cases (sometimes a big win). It's also incomplete, at least insofar as it doesn't interface with the cost models at all, etc... -- Jon

Re: [HACKERS] PoC: Duplicate Tuple Elidation during External Sort for DISTINCT

On Wed, Jan 22, 2014 at 3:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeremy Harris j...@wizmail.org writes: On 22/01/14 03:53, Tom Lane wrote: Jon Nelson jnelson+pg...@jamponi.net writes: - in createplan.c, eliding duplicate tuples is enabled if we are creating a unique plan which involves

[HACKERS] PoC: Duplicate Tuple Elidation during External Sort for DISTINCT

Greetings -hackers: I have worked up a patch to PostgreSQL which elides tuples during an external sort. The primary use case is when sorted input is being used to feed a DISTINCT operation. The idea is to throw out tuples that compare as identical whenever it's convenient, predicated on the

Re: [HACKERS] PoC: Duplicate Tuple Elidation during External Sort for DISTINCT

On Tue, Jan 21, 2014 at 9:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jon Nelson jnelson+pg...@jamponi.net writes: A rough summary of the patch follows: - a GUC variable enables or disables this capability - in nodeAgg.c, eliding duplicate tuples is enabled if the number of distinct columns

Re: [HACKERS] 9.4 regression

On Thu, Oct 24, 2013 at 5:41 AM, Thom Brown t...@linux.com wrote: On 5 September 2013 22:24, Bruce Momjian br...@momjian.us wrote: On Mon, Aug 19, 2013 at 09:27:57PM -0400, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: I vote for adapting the patch to additionally zero

Re: [HACKERS] 9.4 regression

On Fri, Aug 16, 2013 at 3:57 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Aug 15, 2013 at 12:08:57PM -0500, Jon Nelson wrote: Where are we on this issue? I've been able to replicate it pretty easily with PostgreSQL and continue to look into it. I've contacted Theodore Ts'o and have

Re: [HACKERS] 9.4 regression

Where are we on this issue? I've been able to replicate it pretty easily with PostgreSQL and continue to look into it. I've contacted Theodore Ts'o and have gotten some useful information, however I'm unable to replicate the behavior with the test program (even one that's been modified). What

[HACKERS] pg_ctl initdb takes options, but pg_ctl --help doesn't document them?

Taking a look at PostgreSQL HEAD today, I noticed that pg_ctl documents that pg_ctl initdb takes OPTIONS but doesn't document them (unlike for start and others). Is this intentional? -- Jon -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] 9.4 regression

A follow-up. I found this thread that seems to explain some things: http://comments.gmane.org/gmane.comp.file-systems.ext4/33024 Short version: if we are writing into the middle of the newly-fallocated file on ext4 (with extents) the extent tree can fragment badly, causing poor performance due

Re: [HACKERS] 9.4 regression

On Thu, Aug 8, 2013 at 2:50 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 08/08/2013 05:28 PM, Jon Nelson wrote: ... Just an idea - can you check if using a fillfactor different form 100 changes anything pgbench -s 20 -p 54320 -d pgb -F 90 -i pgbench -j 80 -c 80 -T 120 -p 54320 pgb

Re: [HACKERS] 9.4 regression

On Thu, Aug 8, 2013 at 4:24 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Aug 8, 2013 at 04:12:06PM -0500, Jon Nelson wrote: On Thu, Aug 8, 2013 at 2:50 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 08/08/2013 05:28 PM, Jon Nelson wrote: ... Just an idea - can you check if using

Re: [HACKERS] 9.4 regression

On Thu, Aug 8, 2013 at 4:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jon Nelson jnelson+pg...@jamponi.net writes: At this point I'm convinced that the issue is a pathological case in ext4. The performance impact disappears as soon as the unwritten extent(s) are written to with real data. Thus

Re: [HACKERS] 9.4 regression

On Thu, Aug 8, 2013 at 5:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jon Nelson jnelson+pg...@jamponi.net writes: On Thu, Aug 8, 2013 at 4:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Does your test program use all the same writing options that the real WAL writes do (like O_DIRECT)? I believe so

Re: [HACKERS] 9.4 regression

On Thu, Aug 8, 2013 at 5:25 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: On Thu, Aug 8, 2013 at 5:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jon Nelson jnelson+pg...@jamponi.net writes: On Thu, Aug 8, 2013 at 4:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Does your test program use all the same

Re: [HACKERS] 9.4 regression

On Thu, Aug 8, 2013 at 9:27 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-08-08 16:12:06 -0500, Jon Nelson wrote: ... At this point I'm convinced that the issue is a pathological case in ext4. The performance impact disappears as soon as the unwritten extent(s) are written

Re: [HACKERS] 9.4 regression

On Wed, Aug 7, 2013 at 11:21 AM, Thom Brown t...@linux.com wrote: Hi all, I recently tried a simple benchmark to see how far 9.4 had come since 8.4, but I discovered that I couldn't get 9.4 to even touch 8.4 for performance. After checking 9.2 and 9.3 (as per Kevin Grittner's suggestion), I

Re: [HACKERS] 9.4 regression

On Wed, Aug 7, 2013 at 12:42 PM, Thom Brown t...@linux.com wrote: On 7 August 2013 17:54, Jon Nelson jnelson+pg...@jamponi.net wrote: On Wed, Aug 7, 2013 at 11:21 AM, Thom Brown t...@linux.com wrote: Hi all, I recently tried a simple benchmark to see how far 9.4 had come since 8.4, but I

Re: [HACKERS] 9.4 regression

On Wed, Aug 7, 2013 at 4:18 PM, Kevin Grittner kgri...@ymail.com wrote: Thom Brown t...@linux.com wrote: pgbench -j 80 -c 80 -T 3600 269e78: 606.268013 8800d8: 779.583129 I have also been running some tests and - as yet - they are inconclusive. What I can say about them so far is that - at

Re: [HACKERS] 9.4 regression

On Wed, Aug 7, 2013 at 10:05 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-08-07 20:23:55 +0100, Thom Brown wrote: 269e78 was the commit immediately after 8800d8, so it appears that introduced the regression. Use posix_fallocate() for new WAL files, where available. This is

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Fri, Jul 5, 2013 at 2:23 AM, Greg Smith g...@2ndquadrant.com wrote: On 7/5/13 2:50 AM, Jeff Davis wrote: So, my simple conclusion is that glibc emulation should be about the same as what we're doing now, so there's no reason to avoid it. That means, if posix_fallocate() is present, we

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Sun, Jun 30, 2013 at 11:52 PM, Greg Smith g...@2ndquadrant.com wrote: On 6/30/13 9:28 PM, Jon Nelson wrote: The performance of the latter (new) test sometimes seems to perform worse and sometimes seems to perform better (usually worse) than either of the other two. In all cases

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Sun, Jun 30, 2013 at 5:55 PM, Greg Smith g...@2ndquadrant.com wrote: pwrite(4, \0, 1, 16769023)= 1 pwrite(4, \0, 1, 16773119)= 1 pwrite(4, \0, 1, 16777215)= 1 That's glibc helpfully converting your call to posix_fallocate into small writes, because

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Sun, Jun 30, 2013 at 6:49 PM, Greg Smith g...@2ndquadrant.com wrote: On 5/28/13 10:00 PM, Jon Nelson wrote: A note: The attached test program uses *fsync* instead of *fdatasync* after calling fallocate (or writing out 16MB of zeroes), per an earlier suggestion. I tried this out

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Fri, Jun 14, 2013 at 12:06 PM, Jeff Davis pg...@j-davis.com wrote: On Sat, 2013-05-25 at 13:55 -0500, Jon Nelson wrote: Ack. I've revised the patch to always have the GUC (for now), default to false, and if configure can't find posix_fallocate (or the user disables it by way

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Sun, Jun 16, 2013 at 9:59 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: On Fri, Jun 14, 2013 at 12:06 PM, Jeff Davis pg...@j-davis.com wrote: On Sat, 2013-05-25 at 13:55 -0500, Jon Nelson wrote: .. * You check for the presence of posix_fallocate at configure time, but don't #ifdef

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

There hasn't been much activity here recently. I'm curious, then, if there are questions that I can answer. It may be useful to summarize some things here: - the purpose of the patch is to use posix_fallocate when creating new WAL files, because it's (usually) much quicker - using

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Tue, Jun 11, 2013 at 12:49 PM, Stefan Drees ste...@drees.name wrote: On 2013-06-11 19:45 CEST, Greg Smith wrote: On 6/11/13 12:22 PM, Merlin Moncure wrote: Personally I think this patch should go in regardless -- the concerns made IMNSHO are specious. That's nice, but we have this

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Tue, May 28, 2013 at 9:19 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, May 28, 2013 at 10:15 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-05-28 10:03:58 -0400, Robert Haas wrote: On Sat, May 25, 2013 at 2:55 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: The biggest

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Tue, May 28, 2013 at 10:36 AM, Greg Smith g...@2ndquadrant.com wrote: On 5/28/13 11:12 AM, Jon Nelson wrote: It opens a new file, fallocates 16MB, calls fdatasync. Outside of the run for performance testing, I think it would be good at this point to validate that there is really a 16MB

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Thu, May 16, 2013 at 7:05 PM, Greg Smith g...@2ndquadrant.com wrote: On 5/16/13 9:16 AM, Jon Nelson wrote: Am I doing this the right way? Should I be posting the full patch each time, or incremental patches? There are guidelines for getting your patch in the right format at https

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Wed, May 15, 2013 at 10:36 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: On Wed, May 15, 2013 at 10:17 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Jon Nelson escribió: On Wed, May 15, 2013 at 4:46 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: That's true. I originally wrote

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Tue, May 14, 2013 at 9:43 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, May 13, 2013 at 9:54 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: Pertinent to another thread titled [HACKERS] corrupt pages detected by enabling checksums I hope to explore the possibility of using fallocate

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Wed, May 15, 2013 at 4:34 PM, Andres Freund and...@2ndquadrant.com wrote: Hi, On 2013-05-15 16:26:15 -0500, Jon Nelson wrote: I have written up a patch to use posix_fallocate in new WAL file creation, including configuration by way of a GUC variable, but I've not contributed

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Wed, May 15, 2013 at 4:46 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: On Wed, May 15, 2013 at 4:34 PM, Andres Freund and...@2ndquadrant.com wrote: .. Some where quick comments, without thinking about this: Thank you for the kind feedback. * needs a configure check for posix_fallocate

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Wed, May 15, 2013 at 10:17 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Jon Nelson escribió: On Wed, May 15, 2013 at 4:46 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: That's true. I originally wrote the patch using fallocate(2). What would be appropriate here? Should I switch

Re: [HACKERS] corrupt pages detected by enabling checksums

On Mon, May 13, 2013 at 7:49 AM, k...@rice.edu k...@rice.edu wrote: On Sun, May 12, 2013 at 07:41:26PM -0500, Jon Nelson wrote: On Sun, May 12, 2013 at 3:46 PM, Jim Nasby j...@nasby.net wrote: On 5/10/13 1:06 PM, Jeff Janes wrote: Of course the paranoid DBA could turn off

Re: [HACKERS] corrupt pages detected by enabling checksums

On Mon, May 13, 2013 at 8:32 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-05-12 19:41:26 -0500, Jon Nelson wrote: On Sun, May 12, 2013 at 3:46 PM, Jim Nasby j...@nasby.net wrote: On 5/10/13 1:06 PM, Jeff Janes wrote: Of course the paranoid DBA could turn off restart_after_crash

[HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

Pertinent to another thread titled [HACKERS] corrupt pages detected by enabling checksums I hope to explore the possibility of using fallocate (or posix_fallocate) for new WAL file creation. Most modern Linux filesystems support fast fallocate/posix_fallocate, reducing extent fragmentation (where

Re: [HACKERS] corrupt pages detected by enabling checksums

On Sun, May 12, 2013 at 3:46 PM, Jim Nasby j...@nasby.net wrote: On 5/10/13 1:06 PM, Jeff Janes wrote: Of course the paranoid DBA could turn off restart_after_crash and do a manual investigation on every crash, but in that case the database would refuse to restart even in the case where it

Re: [HACKERS] Considering Gerrit for CFs

On Fri, Feb 8, 2013 at 1:43 PM, Phil Sorber p...@omniti.com wrote: On Fri, Feb 8, 2013 at 10:20 AM, Peter Eisentraut pete...@gmx.net wrote: On 2/8/13 5:23 AM, Magnus Hagander wrote: But do you have any actual proof that the problem is in we loose reviewers because we're relying on email?

Re: [HACKERS] question: foreign key constraints and AccessExclusive locks

On Sun, Jan 6, 2013 at 4:14 AM, Simon Riggs si...@2ndquadrant.com wrote: On 6 January 2013 03:08, Jon Nelson jnelson+pg...@jamponi.net wrote: When adding a foreign key constraint on tableA which references tableB, why is an AccessExclusive lock on tableB necessary? Wouldn't a lock

[HACKERS] question: foreign key constraints and AccessExclusive locks

When adding a foreign key constraint on tableA which references tableB, why is an AccessExclusive lock on tableB necessary? Wouldn't a lock that prevents writes be sufficient, or does PostgreSQL have to modify *both* tables in some fashion? I'm using PostgreSQL 8.4 on Linux. -- Jon -- Sent

Re: [HACKERS] xmalloc = pg_malloc

On Wed, Oct 3, 2012 at 11:36 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: xmalloc, xstrdup, etc. are pretty common names for functions that do alloc-or-die (another possible naming scheme ;-) ). The naming pg_malloc etc. on the other hand suggests that the

Re: [HACKERS] Posix Shared Mem patch

On Thu, Jun 28, 2012 at 6:05 AM, Magnus Hagander mag...@hagander.net wrote: On Thu, Jun 28, 2012 at 7:00 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 27, 2012 at 9:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jun 27, 2012 at 12:00 AM,

Re: [HACKERS] Posix Shared Mem patch

On Thu, Jun 28, 2012 at 8:57 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 28, 2012 at 9:47 AM, Jon Nelson jnelson+pg...@jamponi.net wrote: Why not just mmap /dev/zero (MAP_SHARED but not MAP_ANONYMOUS)?  I seem to think that's what I did when I needed this functionality oh so many