Re: [HACKERS] Measuring replay lag

2017-02-16 Thread Abhijit Menon-Sen
Hi Thomas. At 2017-02-15 00:48:41 +1300, thomas.mu...@enterprisedb.com wrote: > > Here is a new version with the buffer on the sender side as requested. This looks good. > + write_lag > + interval > + Estimated time taken for recent WAL records to be written on this > + standby

Re: [HACKERS] [COMMITTERS] pgsql: Update copyright for 2017

2017-01-03 Thread Abhijit Menon-Sen
At 2017-01-03 18:46:32 +0100, mag...@hagander.net wrote: > > Thoughts? I think we should stop making wholesale changes to copyright notices every year. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.o

Re: [HACKERS] pg_basebackups and slots

2016-12-15 Thread Abhijit Menon-Sen
At 2016-12-16 07:32:24 +0100, mag...@hagander.net wrote: > > I really think the default should be "what most people want", and not > "whatever is compatible with a mode that was necessary because we > lacked infrastructure". Very much agreed. -- Abhijit -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-11-04 Thread Abhijit Menon-Sen
At 2016-11-04 10:35:05 +, si...@2ndquadrant.com wrote: > > Since the "lots of parameters" approach is almost exactly what we have > now, I think we should do that first, get this patch committed and > then sit back and discuss an improved API and see what downsides it > introduces. Thanks, I a

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-10-31 Thread Abhijit Menon-Sen
another update very soon. >From 231ccefbc99c07f00560f4e88a961db186db704e Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Mon, 31 Oct 2016 11:32:51 +0530 Subject: Convert recovery.conf settings to GUCs Based on unite_recoveryconf_postgresqlconf_v3.patch by Fujii Masao. --- co

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-06 Thread Abhijit Menon-Sen
At 2016-09-06 10:56:53 +0200, p...@2ndquadrant.com wrote: > > So +1 on the recovery_target = 'xid:xxx' idea. OK, I'll make it work that way. New patch in a couple of days. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: htt

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-06 Thread Abhijit Menon-Sen
At 2016-09-06 14:40:54 +0900, michael.paqu...@gmail.com wrote: > > my best advice here is to make all those recovery_target_* parameters > PGC_POSTMASTER so as they are loaded only once when the server starts, > and then we define the recovery target type used in the startup > process instead of tr

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-05 Thread Abhijit Menon-Sen
ed order (which would also break the current last-mention-wins behaviour). Thoughts? (I've added Fujii to the Cc: list, in case he has any comments, since this is based on his earlier patch.) -- Abhijit commit c20d735648f5ea867fda5afc499a63d3877536a3 Author: Abhijit Menon-Sen Date: Thu Sep 1 0

Re: [HACKERS] LSN as a recovery target

2016-09-04 Thread Abhijit Menon-Sen
At 2016-09-04 07:02:01 +0100, si...@2ndquadrant.com wrote: > > > By the way, what has been committed does not include the patch > > adding the parsing context in case of an error as wanted upthread. > > Perhaps that's not worth adding now as there is the GUC refactoring > > potentially happening fo

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Abhijit Menon-Sen
At 2016-09-01 15:51:11 +0900, michael.paqu...@gmail.com wrote: > > - (errmsg("starting point-in-time recovery to XID %u", > - recoveryTargetXid))); > User loses information if those logs are removed. Agreed. I'm revising the patch right now, and I decide

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-08-31 Thread Abhijit Menon-Sen
s of the original patch). Rather than trying to untangle that right now, I'm posting what I have as-is, and I'll post an updated version tomorrow. -- Abhijit commit 999c0c2632f0d4c20d20b9ac30cd258305f74bac Author: Abhijit Menon-Sen Date: Wed Aug 31 22:18:07 2016 +05

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-07-21 Thread Abhijit Menon-Sen
At 2016-07-21 12:46:29 -0400, robertmh...@gmail.com wrote: > > I join with others in thinking it's a reasonable contrib module. I don't like the use of the term "empty" to describe an UPDATE or DELETE without a WHERE clause. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] 9.6 and fsync=off

2016-04-28 Thread Abhijit Menon-Sen
a checkpoint when we see that its value has changed, but if I did any work on it (which I have a vague memory of), I can't find it now. Sorry. Do you want a patch along those lines now, or is it too late? -- Abhijit >From 1768680b672bcb037446230323cabcf9960f7f9a Mon Sep 17 00:00:00 2001

Re: [HACKERS] 9.6 and fsync=off

2016-04-27 Thread Abhijit Menon-Sen
Here's a patch just to help things along. -- Abhijit diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index f3e3de0..353de2e 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -

Re: [HACKERS] 9.6 and fsync=off

2016-04-27 Thread Abhijit Menon-Sen
At 2016-04-27 17:58:08 +0800, cr...@2ndquadrant.com wrote: > > #fsync = on # turns forced synchronization on or > off I suggest:# provide crash safety by flushing disk writes # (Disabling this c

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-04-13 Thread Abhijit Menon-Sen
At 2016-04-12 09:00:57 -0400, robertmh...@gmail.com wrote: > > On Mon, Apr 11, 2016 at 1:17 PM, Andres Freund wrote: > > > > 3) Actually handle the case of the last open segment not being > >RELSEG_SIZE properly in _mdfd_getseg() - mdnblocks() does so. > > #3 seems like it's probably about 15

Re: [HACKERS] Pglogical questions and problems

2016-04-13 Thread Abhijit Menon-Sen
At 2016-04-13 03:46:21 -0700, j...@commandprompt.com wrote: > > ALTER DATABASE ADD NODE; > ALTER SCHEMA SUBSCRIBE ALL; > CREATE REPLICATION SET; > > But I am unaware if that is possible within the constraints of the > extensions API. It is not possible. -- Abhijit -- Sent via pgsql-hackers ma

Re: [HACKERS] Incomplete startup packet errors

2016-04-13 Thread Abhijit Menon-Sen
At 2016-04-13 10:02:22 +0200, mag...@hagander.net wrote: > > I wonder if it would make sense to only log that error if *at least > one byte* has been received and then it becomes empty. Yes, it would be very nice to eliminate that logspam, as you say. -- Abhijit -- Sent via pgsql-hackers maili

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-05 Thread Abhijit Menon-Sen
At 2016-04-05 18:45:58 -0300, alvhe...@2ndquadrant.com wrote: > > I changed the regression test a bit more, so please recheck. Looks good, thank you. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-05 Thread Abhijit Menon-Sen
OK, thanks for the clarification. Here's the earlier patch, but with the relevant added docs and tests retained. -- Abhijit >From dfb6ded15246ec65cc911864bfcff285eef1c4d4 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Tue, 5 Apr 2016 11:55:09 +0530 Subject: =?UTF-8?q?Implement=

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-05 Thread Abhijit Menon-Sen
At 2016-04-05 12:33:56 +0530, a...@2ndquadrant.com wrote: > > Álvaro: I did document and test the extra types you added, but now > that I think about it a bit more, it's hard to argue that it's useful > to have a table, for example, depend on an extension. There's really > nothing about a table tha

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-05 Thread Abhijit Menon-Sen
Álvaro: I did document and test the extra types you added, but now that I think about it a bit more, it's hard to argue that it's useful to have a table, for example, depend on an extension. There's really nothing about a table that "doesn't work without" an extension. -- Abhijit -- Sent via pg

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-04 Thread Abhijit Menon-Sen
if there's anything else you'd like to see here. Thanks again for your review and suggestions. -- Abhijit >From d4882446de50c49c6563dac6dbdd386c01f9477a Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Tue, 5 Apr 2016 11:55:09 +0530 Subject: =?UTF-8?q?Implement=20"ALTE

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-04 Thread Abhijit Menon-Sen
At 2016-04-04 18:55:03 -0300, alvhe...@2ndquadrant.com wrote: > > At this point I think we're missing user-level docs for the additional > clause in each ALTER command. Thanks for having a look. Now that you're happy with the grammar, I'll write the remaining docs and resubmit the patch later toda

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-04 Thread Abhijit Menon-Sen
At 2016-04-04 17:28:07 +0900, masao.fu...@gmail.com wrote: > > Barring any objections, I'll commit this patch. No objections, just a minor wording tweak: doc/src/sgml/config.sgml: "The synchronous standbys will be the standbys that their names appear early in this list" should be "The synchronou

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-01 Thread Abhijit Menon-Sen
At 2016-03-29 10:15:51 -0400, da...@pgmasters.net wrote: > > Either way it looks like you need to post a patch with more > documentation - do you know when you'll have that ready? Here it is. (I was actually looking for other potential callers, but I couldn't find any. There are some places that

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-25 Thread Abhijit Menon-Sen
At 2016-03-24 22:48:51 +0530, a...@2ndquadrant.com wrote: > > > I think I would like to see code implement both alternatives to see > > which one is least ugly. Maybe a third idea will manifest itself > > upon seeing those. > > Here's the first one. ExecAlterObjectDependsStmt() looks like this:

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-24 Thread Abhijit Menon-Sen
At 2016-03-24 12:31:16 -0300, alvhe...@2ndquadrant.com wrote: > > In other words I think the conclusion here is that we must use > qualified_name in the new production rather than switching the old > production to any_name. Makes sense. > I think I would like to see code implement both alternativ

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-23 Thread Abhijit Menon-Sen
Hi. I implemented "ALTER FUNCTION … DEPENDS ON EXTENSION" using a new node (AlterObjectDependsStmt), and tried to add "ALTER TRIGGER … DEPENDS ON EXTENSION" (partly because I wanted to make sure the code could support multiple object types, partly because it's convenient in this particular use cas

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-21 Thread Abhijit Menon-Sen
At 2016-03-21 15:43:09 -0400, robertmh...@gmail.com wrote: > > I also think we should allow a function to depend on multiple > extensions, as Alvaro mentions downthread. I'm working on an updated patch, will post shortly. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-21 Thread Abhijit Menon-Sen
red. -- Abhijit >From 722f23b75369f035f094753e47663c50580b052c Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Tue, 1 Mar 2016 06:44:28 +0530 Subject: =?UTF-8?q?Add=20DEPENDENCY=5FAUTO=5FEXTENSION/ALTER=20FUNCTION=20?= =?UTF-8?q?=E2=80=A6=20DEPENDS=20ON=20EXTENSION=20=E2=80=A6?= MIME-V

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-21 Thread Abhijit Menon-Sen
At 2016-03-21 13:04:33 +0300, a.korot...@postgrespro.ru wrote: > > I'm not sure why we want to make new dependency type by ALTER FUNCTION > command, not ALTER EXTENSION? It's a matter of semantics. It means something very different than what an 'e' dependency means. The extension doesn't own the f

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-20 Thread Abhijit Menon-Sen
At 2016-03-19 17:46:25 -0300, alvhe...@2ndquadrant.com wrote: > > I don't think the first patch is acceptable standalone -- we need both > things together. OK. > But in reality, pg_depend handling is mixed up with other changes all > over the place. Yes, I noticed that. :-/ > Anyway I think thi

Re: [HACKERS] pg_rewind just doesn't fsync *anything*?

2016-03-09 Thread Abhijit Menon-Sen
At 2016-03-10 08:35:43 +0100, michael.paqu...@gmail.com wrote: > > > I guess the easiest fix would be to shell out to initdb -s? > > What do you mean? I am not sure what initdb has to do with that as we > have no need for it in pg_rewind. initdb -S/--sync-only fsyncs everything in the data direct

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-02-29 Thread Abhijit Menon-Sen
At 2016-02-29 19:56:07 -0600, jim.na...@bluetreble.com wrote: > > I don't see why this would be limited to just functions. […] Am I > missing something? No, you are not missing anything. The specific problem I was trying to solve involved a function, so I sketched out a solution for functions. Onc

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-02-29 Thread Abhijit Menon-Sen
ring the pg_proc entry for a function, so the new code didn't fit. Ultimately, ExecAlterExtensionContentsStmt() was the closest match, so that's where I did it. Comments welcome. I'll add this patch to the CF. -- Abhijit >From 9835f0990a015431393d608c8710d9effe301c9d Mon Sep 17 00:00:0

Re: [HACKERS] Writing new unit tests with PostgresNode

2016-02-21 Thread Abhijit Menon-Sen
At 2016-02-22 07:45:37 +, e...@xs4all.nl wrote: > > I think what's needed is: > > use 5.008_008; That is meant to fail if the code is run on a version of Perl older than 5.8.8, not fail if the code uses language features not present in 5.8.8. It is little help for those trying to maintain ba

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-01-17 Thread Abhijit Menon-Sen
At 2016-01-16 12:18:53 -0500, robertmh...@gmail.com wrote: > > This seems like one manifestation of the more general problem that we > don't have any real idea what objects a function definition depends > on. Yes. I'm proposing to address a part of that problem by allowing extension dependencies

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-01-16 Thread Abhijit Menon-Sen
Right, here's another try. The extension does trigger-based DML auditing. You install it using CREATE EXTENSION and then call one of its functions to enable auditing for a particular table. That function will create a customised trigger function based on the table's columns and a trigger that uses

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-01-15 Thread Abhijit Menon-Sen
I'm sorry, it wasn't clear from my earlier post that the triggers are dependent on a function provided by the extension. So when you do CREATE EXTENSION foo, it creates foo_somefunc() that RETURNS TRIGGER. Later, a trigger is created (somehow; in this case it is by some other function in the exten

[HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-01-14 Thread Abhijit Menon-Sen
Hi. I'm looking at an extension that creates some triggers (on user tables) dynamically (i.e., not during CREATE EXTENSION, but afterwards). The author has two problems with it: * «DROP EXTENSION ext» won't work without adding CASCADE, which is an (admittedly relatively minor) inconvenience to

Re: [HACKERS] [PATCH] introduce XLogLockBlockRangeForCleanup()

2015-06-23 Thread Abhijit Menon-Sen
BufTableInsert() later in BufferAlloc. On the other hand, if I don't make BufferAlloc call the _shared function, then we end up with a new few-line special-case function plus either a self-contained new branch in ReadBuffer_common, or a bunch of RBM_CACHE_ONLY tests. That doesn't really

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-12 Thread Abhijit Menon-Sen
At 2015-06-11 14:38:03 +0900, langote_amit...@lab.ntt.co.jp wrote: > > > On the other hand, I don't like the idea of doing (3) by adding > > command line arguments to pg_basebackup and adding a new option to > > the command. I don't think that level of "flexibility" is justified; > > it would also

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-11 Thread Abhijit Menon-Sen
At 2015-06-11 14:28:36 +0900, michael.paqu...@gmail.com wrote: > > After spending the night thinking about that, honestly, I think that > we should go with (2) and keep the base backup as light-weight as > possible and not bother about a GUC. OK. Then the patch I posted earlier should be sufficien

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Abhijit Menon-Sen
At 2015-06-10 13:22:27 -0400, robertmh...@gmail.com wrote: > > I'm not clear on which of these options you are voting for: > > (1) include pg_log in pg_basebackup as we do currently > (2) exclude it > (3) add a switch controlling whether or not it gets excluded > > I can live with (3), but I bet

Re: [HACKERS] skipping pg_log in basebackup

2015-06-07 Thread Abhijit Menon-Sen
At 2015-06-08 13:09:02 +0900, michael.paqu...@gmail.com wrote: > > It seems to be that: > http://www.postgresql.org/message-id/cahgqgwh0okz6ckpjkcwojga3ejwffm1enrmro3dkdoteaai...@mail.gmail.com (Note that this is about calculating the wrong size, whereas my bug is about the file being too large to

[HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-07 Thread Abhijit Menon-Sen
Aren't we leaking statrelpath? >From 8db162c1385b1cdd2b0e666975b76aa814f09f35 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Mon, 27 Apr 2015 12:58:52 +0530 Subject: Skip files in pg_log during basebackup --- src/backend/replication/basebackup.c | 29 + 1 f

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-06-02 Thread Abhijit Menon-Sen
At 2015-06-01 23:35:23 -0500, htf...@gmail.com wrote: > > No, it won't prevent the incredibly stupid from doing incredibly > stupid things, nothing will. I hate to speechify, but I think we should try hard to avoid framing such questions in terms of "incredibly stupid" people and the things they m

Re: [HACKERS] pg_xlog -> pg_xjournal?

2015-05-31 Thread Abhijit Menon-Sen
At 2015-05-31 13:46:33 -0400, t...@sss.pgh.pa.us wrote: > > just always create pg_xlog as a symlink to pg_xjournal during initdb. At first glance, the Subject: of this thread made me think that *was* Joel's proposal. :-) But I think it's a great idea, and worth doing. I think "pg_journal" (no "x"

[HACKERS] [PATCH, TRIVIAL] don't specify S_IRUSR|S_IWUSR without O_CREAT

2015-05-29 Thread Abhijit Menon-Sen
Just for the record: a minor nit I noticed yesterday. -- Abhijit >From 07353c86483f7e26d44a9bbe94b32315537cee73 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Fri, 29 May 2015 23:15:15 +0530 Subject: The file mode is ignored without O_CREAT, so set it to 0 --- src/backend/storage/f

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-29 Thread Abhijit Menon-Sen
At 2015-05-29 13:14:18 -0400, t...@sss.pgh.pa.us wrote: > > Pushed with minor revisions. Thanks, looks good. > Since we're only logging the failures anyway, I think it is reasonable > to log a complaint for any unwritable file in the data directory. Sounds reasonable, patch attached. ETXTBSY ha

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-28 Thread Abhijit Menon-Sen
t;From 26bb25e99a2954381587bbfe296a50b19a7f047c Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Thu, 28 May 2015 22:02:29 +0530 Subject: Make initdb -S behave like xlog.c:fsync_pgdata() In particular, it should silently skip unreadable/unexpected files in the data directory and follo

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-28 Thread Abhijit Menon-Sen
c should be changed: s/links/entries/. >From 8e69433fae0b4f51879793f6594c76b99d764818 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Thu, 28 May 2015 22:02:29 +0530 Subject: Make initdb -S behave like xlog.c:fsync_pgdata() In particular, it should silently skip unreadable/unexpected files in the data directory and follo

[HACKERS] [PATCH] readlink missing nul-termination in pg_rewind

2015-05-28 Thread Abhijit Menon-Sen
atter), and does not follow pg_xlog if it's a symlink (which probably does). If you want, I can submit a trivial patch for the latter. >From d1e5cbea21bb916253bce2ad189deb1924864508 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Thu, 28 May 2015 21:03:50 +0530 Subject: readlink(

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-28 Thread Abhijit Menon-Sen
separate patch to initdb with the corresponding changes, which I will post after dinner and a bit more testing. -- Abhijit >From 491dcb8c7203fd992dd9c441f5463a75c88e6f52 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Thu, 28 May 2015 17:22:18 +0530 Subject: Skip unreadable files in fsy

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-27 Thread Abhijit Menon-Sen
At 2015-05-27 23:41:29 -0400, t...@sss.pgh.pa.us wrote: > > What about turning ReadDir into a wrapper around a ReadDirExtended > function that adds an elevel argument? Sounds reasonable, doing. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-27 Thread Abhijit Menon-Sen
At 2015-05-27 20:22:18 -0400, t...@sss.pgh.pa.us wrote: > > I doubt that that (not using AllocateDir) […] (Disregarding per your followup.) > What I think is a reasonable compromise is to treat AllocateDir > failure as nonfatal, but to continue using ReadDir etc which means > that any post-open f

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-26 Thread Abhijit Menon-Sen
At 2015-05-27 11:46:39 +0530, a...@2ndquadrant.com wrote: > > I'm trying a couple of approaches to that (e.g. using readdir directly > instead of ReadDir), but other suggestions are welcome. Here's what that looks like, but not yet fully tested. -- Abhijit diff --git a/src/backend/access/transam/

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-26 Thread Abhijit Menon-Sen
At 2015-05-26 22:44:03 +0200, and...@anarazel.de wrote: > > So what I propose is: > 1) Remove the automatic symlink following > 2) Follow pg_tbspc/*, pg_xlog if it's a symlink, fix the latter in >initdb -S > 3) Add a elevel argument to walkdir(), return if AllocateDir() fails, >continue for

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-26 Thread Abhijit Menon-Sen
At 2015-05-26 22:44:03 +0200, and...@anarazel.de wrote: > > So, this was discussed in the following thread, starting at: > http://archives.postgresql.org/message-id/20150403163232.GA28444%40eldon.alvh.no-ip.org Sorry, I didn't see this before replying. > There are no other places we it's "allowed

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-26 Thread Abhijit Menon-Sen
At 2015-05-26 19:07:20 +0200, and...@anarazel.de wrote: > > Abhijit, do you recall why the code was changed to follow all symlinks > in contrast to explicitly going through the tablespaces as initdb -S > does? I'm pretty sure early versions of the patch pretty much had a > verbatim copy of the init

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-25 Thread Abhijit Menon-Sen
At 2015-05-26 03:54:51 +0200, and...@anarazel.de wrote: > > Say a symlink goes to a binary, which is currently being executed: > ETXTBSY. Or the file is in a readonly filesystem: EROFS. So we'd > need to ignore a lot of errors, possibly ignoring valid ones. Right. That's why I started out by bein

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-12 Thread Abhijit Menon-Sen
d you've made some other changes too.) Thank you. -- Abhijit >From 57d597f176294ebfe30efa6d73569505cbb41e31 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Wed, 13 May 2015 09:19:07 +0530 Subject: Rename pgstatapprox to pgstattuple_approx --- contrib/pgstattuple/pgstatapprox

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-11 Thread Abhijit Menon-Sen
gstatapprox_heap(). Thank you. -- Abhijit P.S. What, if anything, should be done about the complicated and likely not very useful skip-only-min#-blocks logic in lazy_scan_heap? >From 0a99fc61d36e3a3ff4003db95e5c299a5f07a850 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Fri, 26 Dec 2014

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-11 Thread Abhijit Menon-Sen
Abhijit >From 421267bba8394255feed8f9b9424d25d0be9f979 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Mon, 11 May 2015 15:54:48 +0530 Subject: Make pgstattuple use heap_form_tuple instead of BuildTupleFromCStrings --- contrib/pgstattuple/pgstatin

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-09 Thread Abhijit Menon-Sen
At 2015-05-09 02:20:51 +0200, and...@anarazel.de wrote: > > > + * Abhijit Menon-Sen > > + * Portions Copyright (c) 2001,2002Tatsuo Ishii (from pgstattuple) > > I think for new extension we don't really add authors and such anymore. OK, I'll get rid of the

Re: [HACKERS] initdb -S and tablespaces

2015-05-01 Thread Abhijit Menon-Sen
At 2015-05-01 09:57:28 -0400, robertmh...@gmail.com wrote: > > If you don't object to this version, I'll commit it. Looks fine to me, thank you. As for the non-backpatchable 0002, I agree with Andres that it should be included in 9.5; but I take it you're still not convinced? Should I add that to

Re: [HACKERS] initdb -S and tablespaces

2015-05-01 Thread Abhijit Menon-Sen
d of some place that's actually > appropriate. OK, moved to storage/file/fd.c (0001*). -- Abhijit >From 088b80eb0825339eca688e4347a4ef547edcadbb Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Thu, 6 Nov 2014 00:45:56 +0530 Subject: Recursively fsync PGDATA at startup after a crash

Re: [HACKERS] initdb -S and tablespaces

2015-04-30 Thread Abhijit Menon-Sen
At 2015-04-30 16:56:17 -0700, t...@sss.pgh.pa.us wrote: > > As for the notion that this needs to be back-patched, I would say no. Not even just the "fsync after crash" part? I could separate that out from the control file changes and try to eliminate the duplication. I think that would be worth ba

Re: [HACKERS] initdb -S and tablespaces

2015-04-30 Thread Abhijit Menon-Sen
At 2015-04-30 15:37:44 -0400, robertmh...@gmail.com wrote: > > 1. It doesn't do that. As soon as we fsync the data directory, we >reset the flag. That's not what "ever disabled" means to me. Could you suggest an acceptable alternative wording? I can't immediately think of anything better tha

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-04-27 Thread Abhijit Menon-Sen
At 2015-01-30 21:36:42 +0100, and...@2ndquadrant.com wrote: > > > Here's an alternative approach. I think it generally is superior and > > going in the right direction, but I'm not sure it's backpatchable. > > > > It basically consists out of: > > 1) Make GetLockConflicts() actually work. > > alr

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-23 Thread Abhijit Menon-Sen
58dcdca1e410ef9e95b5e Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Fri, 26 Dec 2014 12:37:13 +0530 Subject: Add pgstatbloat to pgstattuple --- contrib/pgstattuple/Makefile | 4 +- contrib/pgstattuple/pgstatbloat.c | 389 + con

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-23 Thread Abhijit Menon-Sen
At 2015-04-24 07:22:27 +0530, amit.kapil...@gmail.com wrote: > > Few minor issues: > 1. > warning on windows > > \pgstatbloat.c(193): warning C4715: 'pgstatbloat' : not all control paths > return a value This is because the function ends with an ereport(ERROR, …). What would you suggest here? Jus

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-22 Thread Abhijit Menon-Sen
comments. -- Abhijit >From 3edb5426292d6097cb66339b865e99bf4f766646 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Fri, 26 Dec 2014 12:37:13 +0530 Subject: Add pgstatbloat to pgstattuple --- contrib/pgstattuple/Makefile | 4 +- contrib/pgstattuple/pgsta

fix xlogdump percentage display (was Re: [HACKERS] Replication identifiers, take 4)

2015-04-17 Thread Abhijit Menon-Sen
At 2015-04-17 10:54:51 +0200, and...@anarazel.de wrote: > > (The FPI percentage display above is arguably borked. Interesting.) Sorry for the trouble. Patch attached. -- Abhijit >From 1e5c5d5948030e8ff6ccdd2309a97fb1e116d8e2 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Fr

Re: [HACKERS] initdb -S and tablespaces

2015-04-16 Thread Abhijit Menon-Sen
Hi. Here's a variation of the earlier patch that follows all links in PGDATA. Does this look more like what you had in mind? -- Abhijit >From d86888b0d2f5a3a57027d26ce050a3bbb58670d3 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Thu, 6 Nov 2014 00:45:56 +0530 Subject: Recursive

Re: [HACKERS] initdb -S and tablespaces

2015-04-14 Thread Abhijit Menon-Sen
At 2015-04-15 11:40:34 +0530, a...@2ndquadrant.com wrote: > > Still, thanks to the example in basebackup.c, I've got most of a patch > that should follow any symlinks in PGDATA. I notice that src/bin/pg_rewind/copy_fetch.c has a traverse_datadir() function that does what we want (but it recurses i

Re: [HACKERS] initdb -S and tablespaces

2015-04-14 Thread Abhijit Menon-Sen
At 2015-04-06 10:16:36 -0300, alvhe...@2ndquadrant.com wrote: > > Well, we have many things that can be set as symlinks; some you can > have initdb create the links for you (initdb --xlogdir), others you > can move manually. Looking at sendDir() in backend/replication/basebackup.c, I notice that t

Re: [HACKERS] initdb -S and tablespaces

2015-04-06 Thread Abhijit Menon-Sen
Hi Álvaro. Thanks for taking a look at the patch. At 2015-04-03 13:32:32 -0300, alvhe...@2ndquadrant.com wrote: > > But then, since the name is already telling us that it's all about > PGDATA, maybe we can remove the "recursively" part of the name? Sure, that makes sense too. Since you and Andre

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-03 Thread Abhijit Menon-Sen
eparate patch. -- Abhijit >From d11b84627438fca12955dfad77042be0a27c9acc Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Fri, 26 Dec 2014 12:37:13 +0530 Subject: Add pgstatbloat to pgstattuple --- contrib/pgstattuple/Makefile | 4 +- contrib/pgstattuple/p

Re: [HACKERS] What exactly is our CRC algorithm?

2015-04-02 Thread Abhijit Menon-Sen
At 2015-04-03 00:33:10 +0300, hlinn...@iki.fi wrote: > > I came up with the attached. I like it very much. src/port/Makefile has (note src/srv): +# pg_crc32c_sse42.o and its _src.o version need CFLAGS_SSE42 +pg_crc32c_sse42.o: CFLAGS+=$(CFLAGS_SSE42) +pg_crc32c_sse42_srv.o: CFLAGS+=$

Re: [HACKERS] What exactly is our CRC algorithm?

2015-04-02 Thread Abhijit Menon-Sen
At 2015-04-02 17:58:23 +0300, hlinn...@iki.fi wrote: > > We're only using inline assembly to force producing SSE 4.2 code, even > when -msse4.2 is not used. That feels wrong. Why? It feels OK to me (and to Andres, per earlier discussions about exactly this topic). Doing it this way allows the bina

Re: [HACKERS] What exactly is our CRC algorithm?

2015-04-02 Thread Abhijit Menon-Sen
At 2015-03-25 19:18:51 +0200, hlinn...@iki.fi wrote: > > I think we'll need a version check there. […] > > You want to write that or should I? I'm not familiar with MSVC at all, so it would be nice if you did it. > How do you like this latest version of the patch otherwise? I'm sorry, but I'm s

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-03-31 Thread Abhijit Menon-Sen
e diff --git a/contrib/pgstattuple/pgstatbloat.c b/contrib/pgstattuple/pgstatbloat.c new file mode 100644 index 000..15c2cb9 --- /dev/null +++ b/contrib/pgstattuple/pgstatbloat.c @@ -0,0 +1,346 @@ +/* + * contrib/pgstattuple/pgstatbloat.c + * + * Abhijit Menon-Sen + * Portions Copyright (c) 2001,20

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-03-22 Thread Abhijit Menon-Sen
At 2015-02-24 11:22:41 -0500, da...@pgmasters.net wrote: > > Patch v3 is attached. > > […] > > +/* Log class enum used to represent bits in auditLogBitmap */ > +enum LogClass > +{ > + LOG_NONE = 0, > + > + /* SELECT */ > + LOG_READ = (1 << 0), > + > + /* INSERT, UPDATE, DELETE, TRUN

Re: [HACKERS] MD5 authentication needs help -SCRAM

2015-03-18 Thread Abhijit Menon-Sen
As a followup, I spoke to an IETF friend who's used and implemented both SRP and SCRAM. He agrees that SRP is cryptographically solid, that it's significantly more difficult to implement (and therefore has a bit of a monoculture risk overall, though of course that wouldn't apply to us if we were to

Re: [HACKERS] MD5 authentication needs help -SCRAM

2015-03-18 Thread Abhijit Menon-Sen
At 2015-03-14 09:44:02 +0200, hlinn...@iki.fi wrote: > > Perhaps it would be time to restart the discussion on standardizing > SRP as a SASL mechanism in IETF. I haven't seen much evidence that there's any interest in doing this; in fact, I can't remember the author of the draft you pointed to bei

Re: [HACKERS] initdb -S and tablespaces

2015-03-10 Thread Abhijit Menon-Sen
to do about that. Is this about what you had in mind? -- Abhijit >From bb2b5f130525dd44a10eab6829b9802b8f6f7eed Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen Date: Thu, 6 Nov 2014 00:45:56 +0530 Subject: Recursively fsync PGDATA at startup if needed It's needed if we need to perf

Re: [HACKERS] MD5 authentication needs help -SCRAM

2015-03-09 Thread Abhijit Menon-Sen
At 2015-03-09 13:52:10 +0200, hlinn...@iki.fi wrote: > > Do you have any insight on why the IETF working group didn't choose a > PAKE protocol instead of or in addition to SCRAM, when SCRAM was > standardized? Hi Heikki. It was a long time ago, but I recall that SRP was patent-encumbered: https:

Re: [HACKERS] MD5 authentication needs help -SCRAM

2015-03-09 Thread Abhijit Menon-Sen
At 2015-03-08 12:48:44 -0700, j...@agliodbs.com wrote: > > Since SCRAM has been brought up a number of times here, I thought > I'd loop in the PostgreSQL contributor who is co-author of the SCRAM > standard to see if he has anything to say about implementing SCRAM as > a built-in auth method for Po

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-02-17 Thread Abhijit Menon-Sen
At 2015-02-17 13:01:46 -0500, sfr...@snowman.net wrote: > > I have to admit that I'm confused by this. Patches don't stabilise > through sitting in the archives, they stabilise when the comments are > being addressed, the patch updated, and further comments are > addressing less important issues.

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-02-17 Thread Abhijit Menon-Sen
At 2015-02-17 10:52:55 -0500, sfr...@snowman.net wrote: > > From the old thread, David had offered to submit a pull request if > there was interest and I didn't see any response... For whatever it's worth, that's because I've been away from work, and only just returned. I had it on my list to look

Re: [HACKERS] What exactly is our CRC algorithm?

2015-02-11 Thread Abhijit Menon-Sen
At 2015-02-11 13:20:29 +0200, hlinnakan...@vmware.com wrote: > > I don't follow. I didn't change configure at all, compared to your > patch. OK, I extrapolated a little too much. Your patch didn't actually include crc_instructions.h; from the name of the #define, I imagined you planned to move the

Re: [HACKERS] What exactly is our CRC algorithm?

2015-02-11 Thread Abhijit Menon-Sen
At 2015-02-10 14:30:51 +0200, hlinnakan...@vmware.com wrote: > > I propose that we add a new header file in src/port, let's call it > crc_instructions.h […] the point of the crc_instructions.h header > is to hide the differences between compilers and architectures. Moving the assembly code/compile

Re: [HACKERS] What exactly is our CRC algorithm?

2015-02-09 Thread Abhijit Menon-Sen
At 2015-02-09 12:52:41 +0200, hlinnakan...@vmware.com wrote: > > Ok, I've committed a patch that just moves the existing code to > common/pg_crc.c […] Thanks, looks good. > Attached is a rebased version of the slicing-by-8 patch. Looks OK too. > Do you have access to big-endian hardware to test

Re: [HACKERS] What exactly is our CRC algorithm?

2015-02-08 Thread Abhijit Menon-Sen
At 2015-02-08 18:46:30 +0200, hlinnakan...@vmware.com wrote: > > So I propose to move pg_crc.c to src/common, and move the tables > from pg_crc_tables.h directly to pg_crc.c. Thoughts? Sounds fine to me. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make c

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-27 17:00:27 -0600, jim.na...@bluetreble.com wrote: > > It would be best to get this into a commit fest so it's not lost. It's there already. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.o

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-26 18:47:29 -0600, jim.na...@bluetreble.com wrote: > > Anything happen with this? Nothing so far. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-26 17:45:52 -0500, robertmh...@gmail.com wrote: > > > Based on the recent emails, it appears there's been a shift of > > preference to having it be in-core […] > > Well, I'm not sure that anyone else here agreed with me on that Sure, an in-core AUDIT command would be great. Stephen has

Re: [HACKERS] pg_basebackup fails with long tablespace paths

2015-01-23 Thread Abhijit Menon-Sen
At 2014-12-24 08:10:46 -0500, pete...@gmx.net wrote: > > As a demo for how this might look, attached is a wildly incomplete > patch to produce GNU long-link headers. Hi Peter. In what way exactly is this patch wildly incomplete? (I ask because it's been added to the current CF). -- Abhijit --

  1   2   3   4   5   >