Re: [HACKERS] Measuring replay lag

2017-02-16 Thread Abhijit Menon-Sen
Hi Thomas. At 2017-02-15 00:48:41 +1300, thomas.mu...@enterprisedb.com wrote: > > Here is a new version with the buffer on the sender side as requested. This looks good. > + write_lag > + interval > + Estimated time taken for recent WAL records to be written on this > + standby

Re: [HACKERS] [COMMITTERS] pgsql: Update copyright for 2017

2017-01-03 Thread Abhijit Menon-Sen
At 2017-01-03 18:46:32 +0100, mag...@hagander.net wrote: > > Thoughts? I think we should stop making wholesale changes to copyright notices every year. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] pg_basebackups and slots

2016-12-15 Thread Abhijit Menon-Sen
At 2016-12-16 07:32:24 +0100, mag...@hagander.net wrote: > > I really think the default should be "what most people want", and not > "whatever is compatible with a mode that was necessary because we > lacked infrastructure". Very much agreed. -- Abhijit -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-11-04 Thread Abhijit Menon-Sen
At 2016-11-04 10:35:05 +, si...@2ndquadrant.com wrote: > > Since the "lots of parameters" approach is almost exactly what we have > now, I think we should do that first, get this patch committed and > then sit back and discuss an improved API and see what downsides it > introduces. Thanks, I

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-10-31 Thread Abhijit Menon-Sen
0f4e88a961db186db704e Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen <a...@2ndquadrant.com> Date: Mon, 31 Oct 2016 11:32:51 +0530 Subject: Convert recovery.conf settings to GUCs Based on unite_recoveryconf_postgresqlconf_v3.patch by Fujii Masao. --- contrib/pg_standby/pg_standby.c

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-06 Thread Abhijit Menon-Sen
At 2016-09-06 10:56:53 +0200, p...@2ndquadrant.com wrote: > > So +1 on the recovery_target = 'xid:xxx' idea. OK, I'll make it work that way. New patch in a couple of days. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-06 Thread Abhijit Menon-Sen
At 2016-09-06 14:40:54 +0900, michael.paqu...@gmail.com wrote: > > my best advice here is to make all those recovery_target_* parameters > PGC_POSTMASTER so as they are loaded only once when the server starts, > and then we define the recovery target type used in the startup > process instead of

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-05 Thread Abhijit Menon-Sen
Cc: list, in case he has any comments, since this is based on his earlier patch.) -- Abhijit commit c20d735648f5ea867fda5afc499a63d3877536a3 Author: Abhijit Menon-Sen <a...@2ndquadrant.com> Date: Thu Sep 1 09:01:04 2016 +0530 Convert recovery.conf settings to G

Re: [HACKERS] LSN as a recovery target

2016-09-04 Thread Abhijit Menon-Sen
At 2016-09-04 07:02:01 +0100, si...@2ndquadrant.com wrote: > > > By the way, what has been committed does not include the patch > > adding the parsing context in case of an error as wanted upthread. > > Perhaps that's not worth adding now as there is the GUC refactoring > > potentially happening

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Abhijit Menon-Sen
At 2016-09-01 15:51:11 +0900, michael.paqu...@gmail.com wrote: > > - (errmsg("starting point-in-time recovery to XID %u", > - recoveryTargetXid))); > User loses information if those logs are removed. Agreed. I'm revising the patch right now, and I

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-08-31 Thread Abhijit Menon-Sen
the original patch). Rather than trying to untangle that right now, I'm posting what I have as-is, and I'll post an updated version tomorrow. -- Abhijit commit 999c0c2632f0d4c20d20b9ac30cd258305f74bac Author: Abhijit Menon-Sen <a...@2ndquadrant.com> Date: Wed Aug 31 22:18:07 2016 +0530

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-07-21 Thread Abhijit Menon-Sen
At 2016-07-21 12:46:29 -0400, robertmh...@gmail.com wrote: > > I join with others in thinking it's a reasonable contrib module. I don't like the use of the term "empty" to describe an UPDATE or DELETE without a WHERE clause. -- Abhijit -- Sent via pgsql-hackers mailing list

Re: [HACKERS] 9.6 and fsync=off

2016-04-29 Thread Abhijit Menon-Sen
en we see that its value has changed, but if I did any work on it (which I have a vague memory of), I can't find it now. Sorry. Do you want a patch along those lines now, or is it too late? -- Abhijit >From 1768680b672bcb037446230323cabcf9960f7f9a Mon Sep 17 00:00:00 2001 From: Abhij

Re: [HACKERS] 9.6 and fsync=off

2016-04-27 Thread Abhijit Menon-Sen
Here's a patch just to help things along. -- Abhijit diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index f3e3de0..353de2e 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@

Re: [HACKERS] 9.6 and fsync=off

2016-04-27 Thread Abhijit Menon-Sen
At 2016-04-27 17:58:08 +0800, cr...@2ndquadrant.com wrote: > > #fsync = on # turns forced synchronization on or > off I suggest:# provide crash safety by flushing disk writes # (Disabling this

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-04-13 Thread Abhijit Menon-Sen
At 2016-04-12 09:00:57 -0400, robertmh...@gmail.com wrote: > > On Mon, Apr 11, 2016 at 1:17 PM, Andres Freund wrote: > > > > 3) Actually handle the case of the last open segment not being > >RELSEG_SIZE properly in _mdfd_getseg() - mdnblocks() does so. > > #3 seems like

Re: [HACKERS] Pglogical questions and problems

2016-04-13 Thread Abhijit Menon-Sen
At 2016-04-13 03:46:21 -0700, j...@commandprompt.com wrote: > > ALTER DATABASE ADD NODE; > ALTER SCHEMA SUBSCRIBE ALL; > CREATE REPLICATION SET; > > But I am unaware if that is possible within the constraints of the > extensions API. It is not possible. -- Abhijit -- Sent via pgsql-hackers

Re: [HACKERS] Incomplete startup packet errors

2016-04-13 Thread Abhijit Menon-Sen
At 2016-04-13 10:02:22 +0200, mag...@hagander.net wrote: > > I wonder if it would make sense to only log that error if *at least > one byte* has been received and then it becomes empty. Yes, it would be very nice to eliminate that logspam, as you say. -- Abhijit -- Sent via pgsql-hackers

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-05 Thread Abhijit Menon-Sen
At 2016-04-05 18:45:58 -0300, alvhe...@2ndquadrant.com wrote: > > I changed the regression test a bit more, so please recheck. Looks good, thank you. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-05 Thread Abhijit Menon-Sen
OK, thanks for the clarification. Here's the earlier patch, but with the relevant added docs and tests retained. -- Abhijit >From dfb6ded15246ec65cc911864bfcff285eef1c4d4 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen <a...@2ndquadrant.com> Date: Tue, 5 Apr 2016 11:55:09 +0530 Subject

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-05 Thread Abhijit Menon-Sen
At 2016-04-05 12:33:56 +0530, a...@2ndquadrant.com wrote: > > Álvaro: I did document and test the extra types you added, but now > that I think about it a bit more, it's hard to argue that it's useful > to have a table, for example, depend on an extension. There's really > nothing about a table

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-05 Thread Abhijit Menon-Sen
Álvaro: I did document and test the extra types you added, but now that I think about it a bit more, it's hard to argue that it's useful to have a table, for example, depend on an extension. There's really nothing about a table that "doesn't work without" an extension. -- Abhijit -- Sent via

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-05 Thread Abhijit Menon-Sen
anything else you'd like to see here. Thanks again for your review and suggestions. -- Abhijit >From d4882446de50c49c6563dac6dbdd386c01f9477a Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen <a...@2ndquadrant.com> Date: Tue, 5 Apr 2016 11:55:09 +0530 Subject: =?UTF-8?q?Implement=20"

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-04 Thread Abhijit Menon-Sen
At 2016-04-04 18:55:03 -0300, alvhe...@2ndquadrant.com wrote: > > At this point I think we're missing user-level docs for the additional > clause in each ALTER command. Thanks for having a look. Now that you're happy with the grammar, I'll write the remaining docs and resubmit the patch later

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-04 Thread Abhijit Menon-Sen
At 2016-04-04 17:28:07 +0900, masao.fu...@gmail.com wrote: > > Barring any objections, I'll commit this patch. No objections, just a minor wording tweak: doc/src/sgml/config.sgml: "The synchronous standbys will be the standbys that their names appear early in this list" should be "The

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-04-01 Thread Abhijit Menon-Sen
At 2016-03-29 10:15:51 -0400, da...@pgmasters.net wrote: > > Either way it looks like you need to post a patch with more > documentation - do you know when you'll have that ready? Here it is. (I was actually looking for other potential callers, but I couldn't find any. There are some places that

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-25 Thread Abhijit Menon-Sen
At 2016-03-24 22:48:51 +0530, a...@2ndquadrant.com wrote: > > > I think I would like to see code implement both alternatives to see > > which one is least ugly. Maybe a third idea will manifest itself > > upon seeing those. > > Here's the first one. ExecAlterObjectDependsStmt() looks like this:

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-24 Thread Abhijit Menon-Sen
At 2016-03-24 12:31:16 -0300, alvhe...@2ndquadrant.com wrote: > > In other words I think the conclusion here is that we must use > qualified_name in the new production rather than switching the old > production to any_name. Makes sense. > I think I would like to see code implement both

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-23 Thread Abhijit Menon-Sen
Hi. I implemented "ALTER FUNCTION … DEPENDS ON EXTENSION" using a new node (AlterObjectDependsStmt), and tried to add "ALTER TRIGGER … DEPENDS ON EXTENSION" (partly because I wanted to make sure the code could support multiple object types, partly because it's convenient in this particular use

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-21 Thread Abhijit Menon-Sen
At 2016-03-21 15:43:09 -0400, robertmh...@gmail.com wrote: > > I also think we should allow a function to depend on multiple > extensions, as Alvaro mentions downthread. I'm working on an updated patch, will post shortly. -- Abhijit -- Sent via pgsql-hackers mailing list

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-21 Thread Abhijit Menon-Sen
bhijit >From 722f23b75369f035f094753e47663c50580b052c Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen <a...@2ndquadrant.com> Date: Tue, 1 Mar 2016 06:44:28 +0530 Subject: =?UTF-8?q?Add=20DEPENDENCY=5FAUTO=5FEXTENSION/ALTER=20FUNCTION=20?= =?UTF-8?q?=E2=80=A6=20DEPENDS=20ON=20EXTENSION=20=

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-21 Thread Abhijit Menon-Sen
At 2016-03-21 13:04:33 +0300, a.korot...@postgrespro.ru wrote: > > I'm not sure why we want to make new dependency type by ALTER FUNCTION > command, not ALTER EXTENSION? It's a matter of semantics. It means something very different than what an 'e' dependency means. The extension doesn't own the

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-03-21 Thread Abhijit Menon-Sen
At 2016-03-19 17:46:25 -0300, alvhe...@2ndquadrant.com wrote: > > I don't think the first patch is acceptable standalone -- we need both > things together. OK. > But in reality, pg_depend handling is mixed up with other changes all > over the place. Yes, I noticed that. :-/ > Anyway I think

Re: [HACKERS] pg_rewind just doesn't fsync *anything*?

2016-03-09 Thread Abhijit Menon-Sen
At 2016-03-10 08:35:43 +0100, michael.paqu...@gmail.com wrote: > > > I guess the easiest fix would be to shell out to initdb -s? > > What do you mean? I am not sure what initdb has to do with that as we > have no need for it in pg_rewind. initdb -S/--sync-only fsyncs everything in the data

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-02-29 Thread Abhijit Menon-Sen
At 2016-02-29 19:56:07 -0600, jim.na...@bluetreble.com wrote: > > I don't see why this would be limited to just functions. […] Am I > missing something? No, you are not missing anything. The specific problem I was trying to solve involved a function, so I sketched out a solution for functions.

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-02-29 Thread Abhijit Menon-Sen
code didn't fit. Ultimately, ExecAlterExtensionContentsStmt() was the closest match, so that's where I did it. Comments welcome. I'll add this patch to the CF. -- Abhijit >From 9835f0990a015431393d608c8710d9effe301c9d Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen <a...@2ndquadrant.com> Date:

Re: [HACKERS] Writing new unit tests with PostgresNode

2016-02-21 Thread Abhijit Menon-Sen
At 2016-02-22 07:45:37 +, e...@xs4all.nl wrote: > > I think what's needed is: > > use 5.008_008; That is meant to fail if the code is run on a version of Perl older than 5.8.8, not fail if the code uses language features not present in 5.8.8. It is little help for those trying to maintain

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-01-17 Thread Abhijit Menon-Sen
At 2016-01-16 12:18:53 -0500, robertmh...@gmail.com wrote: > > This seems like one manifestation of the more general problem that we > don't have any real idea what objects a function definition depends > on. Yes. I'm proposing to address a part of that problem by allowing extension dependencies

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-01-16 Thread Abhijit Menon-Sen
Right, here's another try. The extension does trigger-based DML auditing. You install it using CREATE EXTENSION and then call one of its functions to enable auditing for a particular table. That function will create a customised trigger function based on the table's columns and a trigger that

Re: [HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-01-15 Thread Abhijit Menon-Sen
I'm sorry, it wasn't clear from my earlier post that the triggers are dependent on a function provided by the extension. So when you do CREATE EXTENSION foo, it creates foo_somefunc() that RETURNS TRIGGER. Later, a trigger is created (somehow; in this case it is by some other function in the

[HACKERS] dealing with extension dependencies that aren't quite 'e'

2016-01-14 Thread Abhijit Menon-Sen
Hi. I'm looking at an extension that creates some triggers (on user tables) dynamically (i.e., not during CREATE EXTENSION, but afterwards). The author has two problems with it: * «DROP EXTENSION ext» won't work without adding CASCADE, which is an (admittedly relatively minor) inconvenience to

Re: [HACKERS] [PATCH] introduce XLogLockBlockRangeForCleanup()

2015-06-23 Thread Abhijit Menon-Sen
special-case access. I'm open to suggestions. -- Abhijit P.S. For reference, I've attached the patch that I posted earlier, rebased to apply to master. commit 4cbbbd242b7ee6333e1d1a09794aa1cd9d39035f Author: Abhijit Menon-Sen a...@2ndquadrant.com Date: Tue Jun 23 13:38:01 2015 +0530 Introduce

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-12 Thread Abhijit Menon-Sen
At 2015-06-11 14:38:03 +0900, langote_amit...@lab.ntt.co.jp wrote: On the other hand, I don't like the idea of doing (3) by adding command line arguments to pg_basebackup and adding a new option to the command. I don't think that level of flexibility is justified; it would also make it

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-11 Thread Abhijit Menon-Sen
At 2015-06-11 14:28:36 +0900, michael.paqu...@gmail.com wrote: After spending the night thinking about that, honestly, I think that we should go with (2) and keep the base backup as light-weight as possible and not bother about a GUC. OK. Then the patch I posted earlier should be sufficient.

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Abhijit Menon-Sen
At 2015-06-10 13:22:27 -0400, robertmh...@gmail.com wrote: I'm not clear on which of these options you are voting for: (1) include pg_log in pg_basebackup as we do currently (2) exclude it (3) add a switch controlling whether or not it gets excluded I can live with (3), but I bet most

[HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-07 Thread Abhijit Menon-Sen
? From 8db162c1385b1cdd2b0e666975b76aa814f09f35 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Mon, 27 Apr 2015 12:58:52 +0530 Subject: Skip files in pg_log during basebackup --- src/backend/replication/basebackup.c | 29 + 1 file changed, 29

Re: [HACKERS] skipping pg_log in basebackup

2015-06-07 Thread Abhijit Menon-Sen
At 2015-06-08 13:09:02 +0900, michael.paqu...@gmail.com wrote: It seems to be that: http://www.postgresql.org/message-id/cahgqgwh0okz6ckpjkcwojga3ejwffm1enrmro3dkdoteaai...@mail.gmail.com (Note that this is about calculating the wrong size, whereas my bug is about the file being too large to

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-02 Thread Abhijit Menon-Sen
At 2015-06-01 23:35:23 -0500, htf...@gmail.com wrote: No, it won't prevent the incredibly stupid from doing incredibly stupid things, nothing will. I hate to speechify, but I think we should try hard to avoid framing such questions in terms of incredibly stupid people and the things they might

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-05-31 Thread Abhijit Menon-Sen
At 2015-05-31 13:46:33 -0400, t...@sss.pgh.pa.us wrote: just always create pg_xlog as a symlink to pg_xjournal during initdb. At first glance, the Subject: of this thread made me think that *was* Joel's proposal. :-) But I think it's a great idea, and worth doing. I think pg_journal (no x) is

[HACKERS] [PATCH, TRIVIAL] don't specify S_IRUSR|S_IWUSR without O_CREAT

2015-05-29 Thread Abhijit Menon-Sen
Just for the record: a minor nit I noticed yesterday. -- Abhijit From 07353c86483f7e26d44a9bbe94b32315537cee73 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Fri, 29 May 2015 23:15:15 +0530 Subject: The file mode is ignored without O_CREAT, so set it to 0 --- src

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-29 Thread Abhijit Menon-Sen
At 2015-05-29 13:14:18 -0400, t...@sss.pgh.pa.us wrote: Pushed with minor revisions. Thanks, looks good. Since we're only logging the failures anyway, I think it is reasonable to log a complaint for any unwritable file in the data directory. Sounds reasonable, patch attached. ETXTBSY has

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-28 Thread Abhijit Menon-Sen
8e69433fae0b4f51879793f6594c76b99d764818 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Thu, 28 May 2015 22:02:29 +0530 Subject: Make initdb -S behave like xlog.c:fsync_pgdata() In particular, it should silently skip unreadable/unexpected files in the data directory and follow symlinks only for pg_xlog

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-28 Thread Abhijit Menon-Sen
Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Thu, 28 May 2015 22:02:29 +0530 Subject: Make initdb -S behave like xlog.c:fsync_pgdata() In particular, it should silently skip unreadable/unexpected files in the data directory and follow symlinks only for pg_xlog and under

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-28 Thread Abhijit Menon-Sen
will post after dinner and a bit more testing. -- Abhijit From 491dcb8c7203fd992dd9c441f5463a75c88e6f52 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Thu, 28 May 2015 17:22:18 +0530 Subject: Skip unreadable files in fsync_pgdata; follow only expected symlinks --- src

[HACKERS] [PATCH] readlink missing nul-termination in pg_rewind

2015-05-28 Thread Abhijit Menon-Sen
), and does not follow pg_xlog if it's a symlink (which probably does). If you want, I can submit a trivial patch for the latter. From d1e5cbea21bb916253bce2ad189deb1924864508 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Thu, 28 May 2015 21:03:50 +0530 Subject: readlink

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-27 Thread Abhijit Menon-Sen
At 2015-05-27 11:46:39 +0530, a...@2ndquadrant.com wrote: I'm trying a couple of approaches to that (e.g. using readdir directly instead of ReadDir), but other suggestions are welcome. Here's what that looks like, but not yet fully tested. -- Abhijit diff --git

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-27 Thread Abhijit Menon-Sen
At 2015-05-27 20:22:18 -0400, t...@sss.pgh.pa.us wrote: I doubt that that (not using AllocateDir) […] (Disregarding per your followup.) What I think is a reasonable compromise is to treat AllocateDir failure as nonfatal, but to continue using ReadDir etc which means that any post-open

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-27 Thread Abhijit Menon-Sen
At 2015-05-27 23:41:29 -0400, t...@sss.pgh.pa.us wrote: What about turning ReadDir into a wrapper around a ReadDirExtended function that adds an elevel argument? Sounds reasonable, doing. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-26 Thread Abhijit Menon-Sen
At 2015-05-26 19:07:20 +0200, and...@anarazel.de wrote: Abhijit, do you recall why the code was changed to follow all symlinks in contrast to explicitly going through the tablespaces as initdb -S does? I'm pretty sure early versions of the patch pretty much had a verbatim copy of the initdb

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-26 Thread Abhijit Menon-Sen
At 2015-05-26 22:44:03 +0200, and...@anarazel.de wrote: So, this was discussed in the following thread, starting at: http://archives.postgresql.org/message-id/20150403163232.GA28444%40eldon.alvh.no-ip.org Sorry, I didn't see this before replying. There are no other places we it's allowed to

Re: [HACKERS] fsync-pgdata-on-recovery tries to write to more files than previously

2015-05-25 Thread Abhijit Menon-Sen
At 2015-05-26 03:54:51 +0200, and...@anarazel.de wrote: Say a symlink goes to a binary, which is currently being executed: ETXTBSY. Or the file is in a readonly filesystem: EROFS. So we'd need to ignore a lot of errors, possibly ignoring valid ones. Right. That's why I started out by being

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-12 Thread Abhijit Menon-Sen
changes too.) Thank you. -- Abhijit From 57d597f176294ebfe30efa6d73569505cbb41e31 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Wed, 13 May 2015 09:19:07 +0530 Subject: Rename pgstatapprox to pgstattuple_approx --- contrib/pgstattuple/pgstatapprox.c| 4

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-11 Thread Abhijit Menon-Sen
(tupdesc, values, nulls); /* make the tuple into a datum */ return HeapTupleGetDatum(tuple); -- 1.9.1 From 0be1d1d49703ff9704255495cedb91c097fbeaa0 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Fri, 26 Dec 2014 12:37:13 +0530 Subject: Add pgstatbloat

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-11 Thread Abhijit Menon-Sen
about the complicated and likely not very useful skip-only-min#-blocks logic in lazy_scan_heap? From 0a99fc61d36e3a3ff4003db95e5c299a5f07a850 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Fri, 26 Dec 2014 12:37:13 +0530 Subject: Add pgstatapprox to pgstattuple

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-09 Thread Abhijit Menon-Sen
At 2015-05-09 02:20:51 +0200, and...@anarazel.de wrote: + * Abhijit Menon-Sen a...@2ndquadrant.com + * Portions Copyright (c) 2001,2002Tatsuo Ishii (from pgstattuple) I think for new extension we don't really add authors and such anymore. OK, I'll get rid of the boilerplate. Hm

Re: [HACKERS] initdb -S and tablespaces

2015-05-01 Thread Abhijit Menon-Sen
At 2015-05-01 09:57:28 -0400, robertmh...@gmail.com wrote: If you don't object to this version, I'll commit it. Looks fine to me, thank you. As for the non-backpatchable 0002, I agree with Andres that it should be included in 9.5; but I take it you're still not convinced? Should I add that to

Re: [HACKERS] initdb -S and tablespaces

2015-05-01 Thread Abhijit Menon-Sen
actually appropriate. OK, moved to storage/file/fd.c (0001*). -- Abhijit From 088b80eb0825339eca688e4347a4ef547edcadbb Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Thu, 6 Nov 2014 00:45:56 +0530 Subject: Recursively fsync PGDATA at startup after a crash This is so

Re: [HACKERS] initdb -S and tablespaces

2015-04-30 Thread Abhijit Menon-Sen
At 2015-04-30 15:37:44 -0400, robertmh...@gmail.com wrote: 1. It doesn't do that. As soon as we fsync the data directory, we reset the flag. That's not what ever disabled means to me. Could you suggest an acceptable alternative wording? I can't immediately think of anything better than

Re: [HACKERS] initdb -S and tablespaces

2015-04-30 Thread Abhijit Menon-Sen
At 2015-04-30 16:56:17 -0700, t...@sss.pgh.pa.us wrote: As for the notion that this needs to be back-patched, I would say no. Not even just the fsync after crash part? I could separate that out from the control file changes and try to eliminate the duplication. I think that would be worth

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-04-27 Thread Abhijit Menon-Sen
At 2015-01-30 21:36:42 +0100, and...@2ndquadrant.com wrote: Here's an alternative approach. I think it generally is superior and going in the right direction, but I'm not sure it's backpatchable. It basically consists out of: 1) Make GetLockConflicts() actually work. already

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-23 Thread Abhijit Menon-Sen
At 2015-04-24 07:22:27 +0530, amit.kapil...@gmail.com wrote: Few minor issues: 1. warning on windows \pgstatbloat.c(193): warning C4715: 'pgstatbloat' : not all control paths return a value This is because the function ends with an ereport(ERROR, …). What would you suggest here? Just

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-23 Thread Abhijit Menon-Sen
2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Fri, 26 Dec 2014 12:37:13 +0530 Subject: Add pgstatbloat to pgstattuple --- contrib/pgstattuple/Makefile | 4 +- contrib/pgstattuple/pgstatbloat.c | 389 + contrib/pgstattuple

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-22 Thread Abhijit Menon-Sen
3edb5426292d6097cb66339b865e99bf4f766646 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Fri, 26 Dec 2014 12:37:13 +0530 Subject: Add pgstatbloat to pgstattuple --- contrib/pgstattuple/Makefile | 4 +- contrib/pgstattuple/pgstatbloat.c | 387

Re: [HACKERS] initdb -S and tablespaces

2015-04-16 Thread Abhijit Menon-Sen
Hi. Here's a variation of the earlier patch that follows all links in PGDATA. Does this look more like what you had in mind? -- Abhijit From d86888b0d2f5a3a57027d26ce050a3bbb58670d3 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Thu, 6 Nov 2014 00:45:56 +0530 Subject

Re: [HACKERS] initdb -S and tablespaces

2015-04-15 Thread Abhijit Menon-Sen
At 2015-04-06 10:16:36 -0300, alvhe...@2ndquadrant.com wrote: Well, we have many things that can be set as symlinks; some you can have initdb create the links for you (initdb --xlogdir), others you can move manually. Looking at sendDir() in backend/replication/basebackup.c, I notice that the

Re: [HACKERS] initdb -S and tablespaces

2015-04-15 Thread Abhijit Menon-Sen
At 2015-04-15 11:40:34 +0530, a...@2ndquadrant.com wrote: Still, thanks to the example in basebackup.c, I've got most of a patch that should follow any symlinks in PGDATA. I notice that src/bin/pg_rewind/copy_fetch.c has a traverse_datadir() function that does what we want (but it recurses

Re: [HACKERS] initdb -S and tablespaces

2015-04-06 Thread Abhijit Menon-Sen
Hi Álvaro. Thanks for taking a look at the patch. At 2015-04-03 13:32:32 -0300, alvhe...@2ndquadrant.com wrote: But then, since the name is already telling us that it's all about PGDATA, maybe we can remove the recursively part of the name? Sure, that makes sense too. Since you and Andres

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-03 Thread Abhijit Menon-Sen
d11b84627438fca12955dfad77042be0a27c9acc Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Fri, 26 Dec 2014 12:37:13 +0530 Subject: Add pgstatbloat to pgstattuple --- contrib/pgstattuple/Makefile | 4 +- contrib/pgstattuple/pgstatbloat.c | 340

Re: [HACKERS] What exactly is our CRC algorithm?

2015-04-02 Thread Abhijit Menon-Sen
At 2015-03-25 19:18:51 +0200, hlinn...@iki.fi wrote: I think we'll need a version check there. […] You want to write that or should I? I'm not familiar with MSVC at all, so it would be nice if you did it. How do you like this latest version of the patch otherwise? I'm sorry, but I'm still

Re: [HACKERS] What exactly is our CRC algorithm?

2015-04-02 Thread Abhijit Menon-Sen
At 2015-04-03 00:33:10 +0300, hlinn...@iki.fi wrote: I came up with the attached. I like it very much. src/port/Makefile has (note src/srv): +# pg_crc32c_sse42.o and its _src.o version need CFLAGS_SSE42 +pg_crc32c_sse42.o: CFLAGS+=$(CFLAGS_SSE42) +pg_crc32c_sse42_srv.o:

Re: [HACKERS] What exactly is our CRC algorithm?

2015-04-02 Thread Abhijit Menon-Sen
At 2015-04-02 17:58:23 +0300, hlinn...@iki.fi wrote: We're only using inline assembly to force producing SSE 4.2 code, even when -msse4.2 is not used. That feels wrong. Why? It feels OK to me (and to Andres, per earlier discussions about exactly this topic). Doing it this way allows the binary

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-03-31 Thread Abhijit Menon-Sen
/pgstattuple/pgstatbloat.c b/contrib/pgstattuple/pgstatbloat.c new file mode 100644 index 000..15c2cb9 --- /dev/null +++ b/contrib/pgstattuple/pgstatbloat.c @@ -0,0 +1,346 @@ +/* + * contrib/pgstattuple/pgstatbloat.c + * + * Abhijit Menon-Sen a...@2ndquadrant.com + * Portions Copyright (c) 2001,2002

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-03-22 Thread Abhijit Menon-Sen
At 2015-02-24 11:22:41 -0500, da...@pgmasters.net wrote: Patch v3 is attached. […] +/* Log class enum used to represent bits in auditLogBitmap */ +enum LogClass +{ + LOG_NONE = 0, + + /* SELECT */ + LOG_READ = (1 0), + + /* INSERT, UPDATE, DELETE, TRUNCATE */ +

Re: [HACKERS] MD5 authentication needs help -SCRAM

2015-03-18 Thread Abhijit Menon-Sen
At 2015-03-14 09:44:02 +0200, hlinn...@iki.fi wrote: Perhaps it would be time to restart the discussion on standardizing SRP as a SASL mechanism in IETF. I haven't seen much evidence that there's any interest in doing this; in fact, I can't remember the author of the draft you pointed to being

Re: [HACKERS] MD5 authentication needs help -SCRAM

2015-03-18 Thread Abhijit Menon-Sen
As a followup, I spoke to an IETF friend who's used and implemented both SRP and SCRAM. He agrees that SRP is cryptographically solid, that it's significantly more difficult to implement (and therefore has a bit of a monoculture risk overall, though of course that wouldn't apply to us if we were

Re: [HACKERS] initdb -S and tablespaces

2015-03-10 Thread Abhijit Menon-Sen
? -- Abhijit From bb2b5f130525dd44a10eab6829b9802b8f6f7eed Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Thu, 6 Nov 2014 00:45:56 +0530 Subject: Recursively fsync PGDATA at startup if needed It's needed if we need to perform crash recovery or if fsync was disabled at some

Re: [HACKERS] MD5 authentication needs help -SCRAM

2015-03-09 Thread Abhijit Menon-Sen
At 2015-03-09 13:52:10 +0200, hlinn...@iki.fi wrote: Do you have any insight on why the IETF working group didn't choose a PAKE protocol instead of or in addition to SCRAM, when SCRAM was standardized? Hi Heikki. It was a long time ago, but I recall that SRP was patent-encumbered:

Re: [HACKERS] MD5 authentication needs help -SCRAM

2015-03-09 Thread Abhijit Menon-Sen
At 2015-03-08 12:48:44 -0700, j...@agliodbs.com wrote: Since SCRAM has been brought up a number of times here, I thought I'd loop in the PostgreSQL contributor who is co-author of the SCRAM standard to see if he has anything to say about implementing SCRAM as a built-in auth method for

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-02-17 Thread Abhijit Menon-Sen
At 2015-02-17 13:01:46 -0500, sfr...@snowman.net wrote: I have to admit that I'm confused by this. Patches don't stabilise through sitting in the archives, they stabilise when the comments are being addressed, the patch updated, and further comments are addressing less important issues. The

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-02-17 Thread Abhijit Menon-Sen
At 2015-02-17 10:52:55 -0500, sfr...@snowman.net wrote: From the old thread, David had offered to submit a pull request if there was interest and I didn't see any response... For whatever it's worth, that's because I've been away from work, and only just returned. I had it on my list to look

Re: [HACKERS] What exactly is our CRC algorithm?

2015-02-11 Thread Abhijit Menon-Sen
At 2015-02-10 14:30:51 +0200, hlinnakan...@vmware.com wrote: I propose that we add a new header file in src/port, let's call it crc_instructions.h […] the point of the crc_instructions.h header is to hide the differences between compilers and architectures. Moving the assembly code/compiler

Re: [HACKERS] What exactly is our CRC algorithm?

2015-02-09 Thread Abhijit Menon-Sen
At 2015-02-09 12:52:41 +0200, hlinnakan...@vmware.com wrote: Ok, I've committed a patch that just moves the existing code to common/pg_crc.c […] Thanks, looks good. Attached is a rebased version of the slicing-by-8 patch. Looks OK too. Do you have access to big-endian hardware to test

Re: [HACKERS] What exactly is our CRC algorithm?

2015-02-08 Thread Abhijit Menon-Sen
At 2015-02-08 18:46:30 +0200, hlinnakan...@vmware.com wrote: So I propose to move pg_crc.c to src/common, and move the tables from pg_crc_tables.h directly to pg_crc.c. Thoughts? Sounds fine to me. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-26 17:45:52 -0500, robertmh...@gmail.com wrote: Based on the recent emails, it appears there's been a shift of preference to having it be in-core […] Well, I'm not sure that anyone else here agreed with me on that Sure, an in-core AUDIT command would be great. Stephen has

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-26 18:47:29 -0600, jim.na...@bluetreble.com wrote: Anything happen with this? Nothing so far. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-27 17:00:27 -0600, jim.na...@bluetreble.com wrote: It would be best to get this into a commit fest so it's not lost. It's there already. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Perl coding error in msvc build system?

2015-01-23 Thread Abhijit Menon-Sen
At 2014-06-03 22:30:50 -0400, pete...@gmx.net wrote: I'm not sure whether the following coding actually detects any errors: Solution.pm: open(P, cl /? 21 |) || die cl command not found; Since nobody with a Windows system has commented, I'm just writing to say that from a Perl

Re: [HACKERS] pg_basebackup fails with long tablespace paths

2015-01-23 Thread Abhijit Menon-Sen
At 2014-12-24 08:10:46 -0500, pete...@gmx.net wrote: As a demo for how this might look, attached is a wildly incomplete patch to produce GNU long-link headers. Hi Peter. In what way exactly is this patch wildly incomplete? (I ask because it's been added to the current CF). -- Abhijit --

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-21 Thread Abhijit Menon-Sen
[After a discussion on IRC with Stephen…] At 2015-01-20 21:47:02 -0500, sfr...@snowman.net wrote: Review the opening of this email though and consider that we could look at what privileges has the audit role granted to the current role? as defining what is to be audited. Right, I understand

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-20 Thread Abhijit Menon-Sen
At 2015-01-20 20:36:39 -0500, robertmh...@gmail.com wrote: I think this is throwing the baby out with the bathwater. Neither C functions nor all-or-nothing are going to be of any practical use. Do you see some approach that has a realistic chance of making 9.5 and would also actually be worth

  1   2   3   4   5   >