Re: [HACKERS] Replication connection URI?

2014-11-25 Thread Heikki Linnakangas
On 11/25/2014 05:11 PM, Heikki Linnakangas wrote: On 11/24/2014 06:05 PM, Alex Shulgin wrote: Heikki Linnakangas writes: It appears that replication connection doesn't support URI but only the traditional conninfo string. src/backend/replication/libpqwalreceiver/libpqwalreceiver.c:99: in li

Re: [HACKERS] Replication connection URI?

2014-11-25 Thread Heikki Linnakangas
On 11/24/2014 06:05 PM, Alex Shulgin wrote: Heikki Linnakangas writes: It appears that replication connection doesn't support URI but only the traditional conninfo string. src/backend/replication/libpqwalreceiver/libpqwalreceiver.c:99: in libpqrcv_connect(): snprintf(conninfo_repl, si

Re: Missing OOM checks in libpq (was Re: [HACKERS] Replication connection URI?)

2014-11-25 Thread Alex Shulgin
Heikki Linnakangas writes: >>> >>> I think we need to fix all of those, and backpatch. Per attached. >> >> Yikes! Looks sane to me. > > Ok thanks, committed. It didn't apply cleanly to 9.0, 9.1 and 9.2, so > the patch for those branches looks a bit different. Great. Are you looking at the actu

Re: Missing OOM checks in libpq (was Re: [HACKERS] Replication connection URI?)

2014-11-25 Thread Heikki Linnakangas
On 11/25/2014 01:37 PM, Alex Shulgin wrote: Heikki Linnakangas writes: On 11/24/2014 06:05 PM, Alex Shulgin wrote: The first patch is not on topic, I just spotted this missing check. *** a/src/interfaces/libpq/fe-connect.c --- b/src/interfaces/libpq/fe-connect.c *** conninfo_a

Re: Missing OOM checks in libpq (was Re: [HACKERS] Replication connection URI?)

2014-11-25 Thread Alex Shulgin
Heikki Linnakangas writes: > On 11/24/2014 06:05 PM, Alex Shulgin wrote: >> The first patch is not on topic, I just spotted this missing check. > >> *** a/src/interfaces/libpq/fe-connect.c >> --- b/src/interfaces/libpq/fe-connect.c >> *** conninfo_array_parse(const char *const * >> *

Missing OOM checks in libpq (was Re: [HACKERS] Replication connection URI?)

2014-11-25 Thread Heikki Linnakangas
On 11/24/2014 06:05 PM, Alex Shulgin wrote: The first patch is not on topic, I just spotted this missing check. *** a/src/interfaces/libpq/fe-connect.c --- b/src/interfaces/libpq/fe-connect.c *** conninfo_array_parse(const char *const * *** 4402,4407 --- 4402,4415

Re: [HACKERS] Replication connection URI?

2014-11-24 Thread Alex Shulgin
Alex Shulgin writes: > Heikki Linnakangas writes: >>> >>> It appears that replication connection doesn't support URI but only the >>> traditional conninfo string. >>> >>> src/backend/replication/libpqwalreceiver/libpqwalreceiver.c:99: in >>> libpqrcv_connect(): >>> >>> snprintf(conninfo_r

Re: [HACKERS] Replication connection URI?

2014-11-24 Thread Alex Shulgin
Heikki Linnakangas writes: >> >> It appears that replication connection doesn't support URI but only the >> traditional conninfo string. >> >> src/backend/replication/libpqwalreceiver/libpqwalreceiver.c:99: in >> libpqrcv_connect(): >> >> snprintf(conninfo_repl, sizeof(conninfo_repl), >>

Re: [HACKERS] Replication connection URI?

2014-11-24 Thread Heikki Linnakangas
On 11/24/2014 02:41 PM, Alex Shulgin wrote: Hackers, It appears that replication connection doesn't support URI but only the traditional conninfo string. src/backend/replication/libpqwalreceiver/libpqwalreceiver.c:99: in libpqrcv_connect(): snprintf(conninfo_repl, sizeof(conninfo_repl),

Re: [HACKERS] Replication identifiers, take 3

2014-10-29 Thread Simon Riggs
On 2 October 2014 09:49, Heikki Linnakangas wrote: >> What I've previously suggested (and which works well in BDR) is to add >> the internal id to the XLogRecord struct. There's 2 free bytes of >> padding that can be used for that purpose. > > > Adding a field to XLogRecord for this feels wrong.

Re: [HACKERS] Replication identifiers, take 3

2014-10-04 Thread Jim Nasby
On 10/2/14, 7:28 AM, Robert Haas wrote: On Thu, Oct 2, 2014 at 4:49 AM, Heikki Linnakangas wrote: >An origin column in the table itself helps tremendously to debug issues with >the replication system. In many if not most scenarios, I think you'd want to >have that extra column, even if it's no

Re: [HACKERS] Replication identifiers, take 3

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 4:49 AM, Heikki Linnakangas wrote: > An origin column in the table itself helps tremendously to debug issues with > the replication system. In many if not most scenarios, I think you'd want to > have that extra column, even if it's not strictly required. I like a lot of wha

Re: [HACKERS] Replication identifiers, take 3

2014-10-02 Thread Andres Freund
On 2014-10-02 11:49:31 +0300, Heikki Linnakangas wrote: > On 09/23/2014 09:24 PM, Andres Freund wrote: > >I've previously started two threads about replication identifiers. Check > >http://archives.postgresql.org/message-id/20131114172632.GE7522%40alap2.anarazel.de > >and > >http://archives.postgre

Re: [HACKERS] Replication identifiers, take 3

2014-10-02 Thread Heikki Linnakangas
On 09/23/2014 09:24 PM, Andres Freund wrote: I've previously started two threads about replication identifiers. Check http://archives.postgresql.org/message-id/20131114172632.GE7522%40alap2.anarazel.de and http://archives.postgresql.org/message-id/20131211153833.GB25227%40awork2.anarazel.de . Th

Re: [HACKERS] Replication identifiers, take 3

2014-09-29 Thread Robert Haas
On Sat, Sep 27, 2014 at 12:11 PM, Steve Singer wrote: > If we were now increasing the WAL record size anyway for some unrelated > reason, would we be willing to increase it by a further 2 bytes for the node > identifier? Obviously not. Otherwise Andres would be proposing to put an OID in there i

Re: [HACKERS] Replication identifiers, take 3

2014-09-29 Thread Andres Freund
On 2014-09-27 12:11:16 -0400, Steve Singer wrote: > On 09/26/2014 06:05 PM, Andres Freund wrote: > >On 2014-09-26 14:57:12 -0400, Robert Haas wrote: > >Sure, it'll possibly not be trivial to move them elsewhere. On the other > >hand, the padding bytes have been unused for 8+ years without somebody

Re: [HACKERS] Replication identifiers, take 3

2014-09-27 Thread Steve Singer
On 09/26/2014 10:21 AM, Andres Freund wrote: On 2014-09-26 09:53:09 -0400, Robert Haas wrote: On Fri, Sep 26, 2014 at 5:05 AM, Andres Freund wrote: Let me try to summarize the information requirements for each of these things. For #1, you need to know, after crash recovery, for each standby,

Re: [HACKERS] Replication identifiers, take 3

2014-09-27 Thread Steve Singer
On 09/26/2014 06:05 PM, Andres Freund wrote: On 2014-09-26 14:57:12 -0400, Robert Haas wrote: Sure, it'll possibly not be trivial to move them elsewhere. On the other hand, the padding bytes have been unused for 8+ years without somebody laying "claim" on them but "me". I don't think it's a good

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Andres Freund
On 2014-09-26 14:57:12 -0400, Robert Haas wrote: > On Fri, Sep 26, 2014 at 12:32 PM, Andres Freund > wrote: > >> Huh? That's just to say that the unused bit space is, in fact, > >> unused. But so what? We've always been very careful about using up > >> things like infomask bits, because there

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Robert Haas
On Fri, Sep 26, 2014 at 12:32 PM, Andres Freund wrote: >> Huh? That's just to say that the unused bit space is, in fact, >> unused. But so what? We've always been very careful about using up >> things like infomask bits, because there are only so many bits >> available, and when they're gone th

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Andres Freund
On 2014-09-26 11:02:16 -0400, Robert Haas wrote: > On Fri, Sep 26, 2014 at 10:55 AM, Andres Freund > wrote: > > On 2014-09-26 10:40:37 -0400, Robert Haas wrote: > >> On Fri, Sep 26, 2014 at 10:21 AM, Andres Freund > >> wrote: > >> > As explained above this isn't happening on the level of indivi

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Robert Haas
On Fri, Sep 26, 2014 at 10:55 AM, Andres Freund wrote: > On 2014-09-26 10:40:37 -0400, Robert Haas wrote: >> On Fri, Sep 26, 2014 at 10:21 AM, Andres Freund >> wrote: >> > As explained above this isn't happening on the level of individual AMs. >> >> Well, that's even worse. You want to grab 100

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Andres Freund
On 2014-09-26 10:40:37 -0400, Robert Haas wrote: > On Fri, Sep 26, 2014 at 10:21 AM, Andres Freund > wrote: > > As explained above this isn't happening on the level of individual AMs. > > Well, that's even worse. You want to grab 100% of the available > generic bitspace applicable to all record

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Robert Haas
On Fri, Sep 26, 2014 at 10:21 AM, Andres Freund wrote: > As explained above this isn't happening on the level of individual AMs. Well, that's even worse. You want to grab 100% of the available generic bitspace applicable to all record types for purposes specific to logical decoding (and it's sti

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Andres Freund
On 2014-09-26 09:53:09 -0400, Robert Haas wrote: > On Fri, Sep 26, 2014 at 5:05 AM, Andres Freund wrote: > >> Let me try to summarize the information requirements for each of these > >> things. For #1, you need to know, after crash recovery, for each > >> standby, the last commit LSN which the cl

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Robert Haas
On Fri, Sep 26, 2014 at 5:05 AM, Andres Freund wrote: >> Let me try to summarize the information requirements for each of these >> things. For #1, you need to know, after crash recovery, for each >> standby, the last commit LSN which the client has confirmed via a >> feedback message. > > I'm not

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Andres Freund
On 2014-09-25 22:44:49 -0400, Robert Haas wrote: > Thanks for this write-up. > > On Tue, Sep 23, 2014 at 2:24 PM, Andres Freund wrote: > > 1) The ability Monitor how for replication has progressed in a > >crashsafe manner to allow it to restart at the right point after > >errors/crashes. >

Re: [HACKERS] Replication identifiers, take 3

2014-09-26 Thread Petr Jelinek
On 26/09/14 04:44, Robert Haas wrote: On Tue, Sep 23, 2014 at 2:24 PM, Andres Freund wrote: Note that it depends on the replication solution whether these external identifiers need to be coordinated across systems or not. I think it's *good* if we don't propose a solution for that - different

Re: [HACKERS] Replication identifiers, take 3

2014-09-25 Thread Robert Haas
Thanks for this write-up. On Tue, Sep 23, 2014 at 2:24 PM, Andres Freund wrote: > 1) The ability Monitor how for replication has progressed in a >crashsafe manner to allow it to restart at the right point after >errors/crashes. > 2) Efficiently identify the origin of individual changes an

Re: [HACKERS] replication commands and log_statements

2014-09-12 Thread Fujii Masao
On Wed, Sep 10, 2014 at 7:39 PM, Fujii Masao wrote: > Thanks for reviewing the patch! > > On Wed, Sep 10, 2014 at 4:57 PM, Heikki Linnakangas > wrote: >> On 08/28/2014 11:38 AM, Fujii Masao wrote: >>> >>> On Thu, Jun 19, 2014 at 5:29 PM, Ian Barwick wrote: - minor rewording for the des

Re: [HACKERS] replication commands and log_statements

2014-09-10 Thread Fujii Masao
Thanks for reviewing the patch! On Wed, Sep 10, 2014 at 4:57 PM, Heikki Linnakangas wrote: > On 08/28/2014 11:38 AM, Fujii Masao wrote: >> >> On Thu, Jun 19, 2014 at 5:29 PM, Ian Barwick wrote: >>> >>> - minor rewording for the description, mentioning that statements will >>>still be logged

Re: [HACKERS] replication commands and log_statements

2014-09-10 Thread Heikki Linnakangas
On 08/28/2014 11:38 AM, Fujii Masao wrote: On Thu, Jun 19, 2014 at 5:29 PM, Ian Barwick wrote: - minor rewording for the description, mentioning that statements will still be logged as DEBUG1 even if parameter set to 'off' (might prevent reports of the kind "I set it to 'off', why am I st

Re: [HACKERS] replication commands and log_statements

2014-08-28 Thread Fujii Masao
On Thu, Jun 19, 2014 at 5:29 PM, Ian Barwick wrote: > On 12/06/14 20:37, Fujii Masao wrote: >> >> On Wed, Jun 11, 2014 at 11:55 PM, Tom Lane wrote: >>> >>> Andres Freund writes: Your wish just seems like a separate feature to me. Including replication commands in 'all' seems corre

Re: [HACKERS] replication commands and log_statements

2014-08-27 Thread Robert Haas
On Tue, Aug 26, 2014 at 7:17 AM, Fujii Masao wrote: > On Wed, Aug 20, 2014 at 1:14 PM, Michael Paquier > wrote: >> On Wed, Aug 20, 2014 at 2:06 AM, Robert Haas wrote: >>> >>> On Sat, Aug 16, 2014 at 10:27 AM, Amit Kapila >>> wrote: >>> > I think ideally it would have been better if we could hav

Re: [HACKERS] replication commands and log_statements

2014-08-26 Thread Fujii Masao
On Wed, Aug 20, 2014 at 1:14 PM, Michael Paquier wrote: > > > > On Wed, Aug 20, 2014 at 2:06 AM, Robert Haas wrote: >> >> On Sat, Aug 16, 2014 at 10:27 AM, Amit Kapila >> wrote: >> > I think ideally it would have been better if we could have logged >> > replication commands under separate log_le

Re: [HACKERS] replication commands and log_statements

2014-08-19 Thread Michael Paquier
On Wed, Aug 20, 2014 at 2:06 AM, Robert Haas wrote: > On Sat, Aug 16, 2014 at 10:27 AM, Amit Kapila > wrote: > > I think ideally it would have been better if we could have logged > > replication commands under separate log_level, but as still there > > is no consensus on extending log_statement

Re: [HACKERS] replication commands and log_statements

2014-08-19 Thread Robert Haas
On Sat, Aug 16, 2014 at 10:27 AM, Amit Kapila wrote: > I think ideally it would have been better if we could have logged > replication commands under separate log_level, but as still there > is no consensus on extending log_statement and nobody is even > willing to pursue, it seems okay to go ahea

Re: [HACKERS] replication commands and index terms

2014-08-17 Thread Fujii Masao
On Fri, Aug 15, 2014 at 11:25 AM, Fujii Masao wrote: > On Fri, Aug 15, 2014 at 4:59 AM, Robert Haas wrote: >> On Thu, Aug 14, 2014 at 12:59 PM, Fujii Masao wrote: >>> Since I sometimes try to search the replication command in the index, >>> I'd feel inclined to expose all those commands as index

Re: [HACKERS] replication commands and log_statements

2014-08-16 Thread Amit Kapila
On Thu, Aug 14, 2014 at 7:19 PM, Stephen Frost wrote: > * Amit Kapila (amit.kapil...@gmail.com) wrote: > > On Thu, Aug 14, 2014 at 5:56 AM, Stephen Frost wrote: > > > Regarding this, I'm generally in the camp that says to just include it > > > in 'all' and be done with it- for now. > > > > Okay,

Re: [HACKERS] replication commands and index terms

2014-08-14 Thread Fujii Masao
On Fri, Aug 15, 2014 at 4:59 AM, Robert Haas wrote: > On Thu, Aug 14, 2014 at 12:59 PM, Fujii Masao wrote: >> Since I sometimes try to search the replication command in the index, >> I'd feel inclined to expose all those commands as index terms... > > +1. Attached patch exposes replication comma

Re: [HACKERS] replication commands and index terms

2014-08-14 Thread Robert Haas
On Thu, Aug 14, 2014 at 12:59 PM, Fujii Masao wrote: > Since I sometimes try to search the replication command in the index, > I'd feel inclined to expose all those commands as index terms... +1. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Se

Re: [HACKERS] replication commands and log_statements

2014-08-14 Thread Stephen Frost
Amit, * Amit Kapila (amit.kapil...@gmail.com) wrote: > On Thu, Aug 14, 2014 at 5:56 AM, Stephen Frost wrote: > > Regarding this, I'm generally in the camp that says to just include it > > in 'all' and be done with it- for now. > > Okay, but tomorrow if someone wants to implement a patch to log >

Re: [HACKERS] replication commands and log_statements

2014-08-13 Thread Amit Kapila
On Thu, Aug 14, 2014 at 5:56 AM, Stephen Frost wrote: > * Amit Kapila (amit.kapil...@gmail.com) wrote: > Oh, to be clear- I agree that logging of queries executed through SPI is > desirable; I'd certainly like to have that capability without having to > go through the auto-explain module or write

Re: [HACKERS] replication commands and log_statements

2014-08-13 Thread Michael Paquier
On Thu, Aug 14, 2014 at 10:40 AM, Fujii Masao wrote: > On Thu, Aug 14, 2014 at 9:26 AM, Stephen Frost wrote: >> * Amit Kapila (amit.kapil...@gmail.com) wrote: >>> On Wed, Aug 13, 2014 at 4:24 AM, Stephen Frost wrote: >>> > Not entirely sure what you're referring to as 'internally generated' >>>

Re: [HACKERS] replication commands and log_statements

2014-08-13 Thread Fujii Masao
On Thu, Aug 14, 2014 at 9:26 AM, Stephen Frost wrote: > * Amit Kapila (amit.kapil...@gmail.com) wrote: >> On Wed, Aug 13, 2014 at 4:24 AM, Stephen Frost wrote: >> > Not entirely sure what you're referring to as 'internally generated' >> > here.. >> >> Here 'internally generated' means that user d

Re: [HACKERS] replication commands and log_statements

2014-08-13 Thread Stephen Frost
* Amit Kapila (amit.kapil...@gmail.com) wrote: > On Wed, Aug 13, 2014 at 4:24 AM, Stephen Frost wrote: > > Not entirely sure what you're referring to as 'internally generated' > > here.. > > Here 'internally generated' means that user doesn't execute those > statements, rather the replication/bac

Re: [HACKERS] replication commands and log_statements

2014-08-12 Thread Amit Kapila
On Wed, Aug 13, 2014 at 4:24 AM, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Aug 12, 2014 at 10:07:34AM +0530, Amit Kapila wrote: > > > One difference is that replication commands are internally generated > > > commands. Do we anywhere else log such internally gen

Re: [HACKERS] replication commands and log_statements

2014-08-12 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Tue, Aug 12, 2014 at 10:07:34AM +0530, Amit Kapila wrote: > > On Tue, Aug 12, 2014 at 9:29 AM, Fujii Masao wrote: > > > On Tue, Aug 12, 2014 at 2:34 AM, Robert Haas > > > wrote: > > > > > > > > If you have a user devoted to it, I suppose that's true

Re: [HACKERS] replication commands and log_statements

2014-08-12 Thread Bruce Momjian
On Tue, Aug 12, 2014 at 10:07:34AM +0530, Amit Kapila wrote: > On Tue, Aug 12, 2014 at 9:29 AM, Fujii Masao wrote: > > On Tue, Aug 12, 2014 at 2:34 AM, Robert Haas wrote: > > > > > > If you have a user devoted to it, I suppose that's true.  I still > > > think it shouldn't get munged together lik

Re: [HACKERS] replication commands and log_statements

2014-08-11 Thread Amit Kapila
On Tue, Aug 12, 2014 at 9:29 AM, Fujii Masao wrote: > On Tue, Aug 12, 2014 at 2:34 AM, Robert Haas wrote: > > > > If you have a user devoted to it, I suppose that's true. I still > > think it shouldn't get munged together like that. > > Why do we need to treat only replication commands as specia

Re: [HACKERS] replication commands and log_statements

2014-08-11 Thread Fujii Masao
On Tue, Aug 12, 2014 at 2:34 AM, Robert Haas wrote: > On Fri, Aug 8, 2014 at 1:04 PM, Fujii Masao wrote: >> On Sat, Aug 9, 2014 at 12:29 AM, Robert Haas wrote: >>> On Fri, Aug 8, 2014 at 11:00 AM, Bruce Momjian wrote: On Fri, Aug 8, 2014 at 08:51:13AM -0400, Robert Haas wrote: > On Th

Re: [HACKERS] replication commands and log_statements

2014-08-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Aug 8, 2014 at 1:04 PM, Fujii Masao wrote: > > You can do that by executing > > "ALTER ROLE SET log_statement TO 'all'". > > If you don't use the replication user to execute SQL statements, > > no SQL statements are logged in that setting. >

Re: [HACKERS] replication commands and log_statements

2014-08-11 Thread Robert Haas
On Fri, Aug 8, 2014 at 1:04 PM, Fujii Masao wrote: > On Sat, Aug 9, 2014 at 12:29 AM, Robert Haas wrote: >> On Fri, Aug 8, 2014 at 11:00 AM, Bruce Momjian wrote: >>> On Fri, Aug 8, 2014 at 08:51:13AM -0400, Robert Haas wrote: On Thu, Aug 7, 2014 at 12:07 PM, Abhijit Menon-Sen wrote:

Re: [HACKERS] replication commands and log_statements

2014-08-08 Thread Fujii Masao
On Sat, Aug 9, 2014 at 12:29 AM, Robert Haas wrote: > On Fri, Aug 8, 2014 at 11:00 AM, Bruce Momjian wrote: >> On Fri, Aug 8, 2014 at 08:51:13AM -0400, Robert Haas wrote: >>> On Thu, Aug 7, 2014 at 12:07 PM, Abhijit Menon-Sen >>> wrote: >>> > At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com

Re: [HACKERS] replication commands and log_statements

2014-08-08 Thread Robert Haas
On Fri, Aug 8, 2014 at 11:00 AM, Bruce Momjian wrote: > On Fri, Aug 8, 2014 at 08:51:13AM -0400, Robert Haas wrote: >> On Thu, Aug 7, 2014 at 12:07 PM, Abhijit Menon-Sen >> wrote: >> > At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com wrote: >> >> That is, we log replication commands only whe

Re: [HACKERS] replication commands and log_statements

2014-08-08 Thread Bruce Momjian
On Fri, Aug 8, 2014 at 08:51:13AM -0400, Robert Haas wrote: > On Thu, Aug 7, 2014 at 12:07 PM, Abhijit Menon-Sen > wrote: > > At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com wrote: > >> That is, we log replication commands only when log_statement is set to > >> all. Neither new parameter is

Re: [HACKERS] replication commands and log_statements

2014-08-08 Thread Robert Haas
On Thu, Aug 7, 2014 at 12:07 PM, Abhijit Menon-Sen wrote: > At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com wrote: >> That is, we log replication commands only when log_statement is set to >> all. Neither new parameter is introduced nor log_statement is >> redefined as a list. > > That sounds

Re: [HACKERS] replication commands and log_statements

2014-08-07 Thread Abhijit Menon-Sen
At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com wrote: > > That is, we log replication commands only when log_statement is set to > all. Neither new parameter is introduced nor log_statement is > redefined as a list. That sounds good to me. -- Abhijit -- Sent via pgsql-hackers mailing list

Re: [HACKERS] replication commands and log_statements

2014-08-07 Thread Fujii Masao
On Wed, Jul 2, 2014 at 4:26 AM, Abhijit Menon-Sen wrote: > Hi. > > Do we have any consensus about what to do with these two patches? > > 1. Introduce a "log_replication_command" setting. > 2. Change log_statement to be a list of tokens. > > If I understand correctly, there weren't any strong objec

Re: [HACKERS] replication commands and log_statements

2014-07-01 Thread Abhijit Menon-Sen
Hi. Do we have any consensus about what to do with these two patches? 1. Introduce a "log_replication_command" setting. 2. Change log_statement to be a list of tokens. If I understand correctly, there weren't any strong objections to the former, but the situation is less clear when it comes to t

Re: [HACKERS] replication identifier format

2014-06-23 Thread Robert Haas
On Mon, Jun 23, 2014 at 11:28 AM, Andres Freund wrote: >> Oh, great. Somehow I missed the fact that that had been addressed. I >> had assumed that we still needed global identifiers in which case I >> think they'd need to be 64+ bits (preferably more like 128). If they >> only need to be locall

Re: [HACKERS] replication identifier format

2014-06-23 Thread Andres Freund
On 2014-06-23 10:45:51 -0400, Robert Haas wrote: > On Mon, Jun 23, 2014 at 10:11 AM, Andres Freund > wrote: > >> > Why? Users and other systems only ever see the external ID. Everything > >> > leaving the system is converted to the external form. The short id > >> > basically is only used in shar

Re: [HACKERS] replication commands and log_statements

2014-06-23 Thread Robert Haas
On Mon, Jun 23, 2014 at 11:15 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> Similarly, >> building a logging facility that meets the needs of real users is >> going to require a configuration method more flexible than a total >> order with four choices. I happen to th

Re: [HACKERS] replication commands and log_statements

2014-06-23 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > Similarly, > building a logging facility that meets the needs of real users is > going to require a configuration method more flexible than a total > order with four choices. I happen to think a list of comma-separated > tokens is a pretty good choice

Re: [HACKERS] replication commands and log_statements

2014-06-23 Thread Robert Haas
On Fri, Jun 20, 2014 at 9:48 AM, Tom Lane wrote: > Fujii Masao writes: >> OK, I've just implemented the patch (attached) which does this, i.e., >> redefines >> log_statement as a list. Thanks to the patch, log_statement can be set >> to "none", >> "ddl", "mod", "dml", "all", and any combinations

Re: [HACKERS] replication identifier format

2014-06-23 Thread Robert Haas
On Mon, Jun 23, 2014 at 10:11 AM, Andres Freund wrote: >> > Why? Users and other systems only ever see the external ID. Everything >> > leaving the system is converted to the external form. The short id >> > basically is only used in shared memory and in wal records. For both >> > using longer str

Re: [HACKERS] replication identifier format

2014-06-23 Thread Andres Freund
On 2014-06-23 10:09:49 -0400, Robert Haas wrote: > On Wed, Jun 18, 2014 at 12:46 PM, Andres Freund > wrote: > > On 2014-06-18 12:36:13 -0400, Robert Haas wrote: > >> > I actually don't think any of the discussions I was involved in had the > >> > externally visible version of replication identifi

Re: [HACKERS] replication identifier format

2014-06-23 Thread Robert Haas
On Wed, Jun 18, 2014 at 12:46 PM, Andres Freund wrote: > On 2014-06-18 12:36:13 -0400, Robert Haas wrote: >> > I actually don't think any of the discussions I was involved in had the >> > externally visible version of replication identifiers limited to 16bits? >> > If you are referring to my patch

Re: [HACKERS] replication commands and log_statements

2014-06-20 Thread Michael Paquier
On Fri, Jun 20, 2014 at 10:48 PM, Tom Lane wrote: > Fujii Masao writes: >> OK, I've just implemented the patch (attached) which does this, i.e., >> redefines >> log_statement as a list. Thanks to the patch, log_statement can be set >> to "none", >> "ddl", "mod", "dml", "all", and any combination

Re: [HACKERS] replication commands and log_statements

2014-06-20 Thread Tom Lane
Fujii Masao writes: > OK, I've just implemented the patch (attached) which does this, i.e., > redefines > log_statement as a list. Thanks to the patch, log_statement can be set > to "none", > "ddl", "mod", "dml", "all", and any combinations of them. The meanings of > "none", "ddl", "mod" and "all

Re: [HACKERS] replication commands and log_statements

2014-06-20 Thread Fujii Masao
On Thu, Jun 12, 2014 at 10:55 PM, Robert Haas wrote: > On Wed, Jun 11, 2014 at 7:42 AM, Magnus Hagander wrote: >>> Replication commands like IDENTIFY_COMMAND are not logged even when >>> log_statements is set to all. Some users who use log_statements to >>> audit *all* statements might dislike th

Re: [HACKERS] replication commands and log_statements

2014-06-19 Thread Ian Barwick
On 12/06/14 20:37, Fujii Masao wrote: On Wed, Jun 11, 2014 at 11:55 PM, Tom Lane wrote: Andres Freund writes: Your wish just seems like a separate feature to me. Including replication commands in 'all' seems correct independent of the desire for a more granular control. No, I think I've got

Re: [HACKERS] replication identifier format

2014-06-18 Thread Andres Freund
On 2014-06-18 12:36:13 -0400, Robert Haas wrote: > > I actually don't think any of the discussions I was involved in had the > > externally visible version of replication identifiers limited to 16bits? > > If you are referring to my patch, 16bits was just the width of the > > *internal* name that s

Re: [HACKERS] replication commands and log_statements

2014-06-12 Thread Robert Haas
On Wed, Jun 11, 2014 at 7:42 AM, Magnus Hagander wrote: >> Replication commands like IDENTIFY_COMMAND are not logged even when >> log_statements is set to all. Some users who use log_statements to >> audit *all* statements might dislike this current situation. So I'm >> thinking to change log_stat

Re: [HACKERS] replication commands and log_statements

2014-06-12 Thread Fujii Masao
On Wed, Jun 11, 2014 at 11:55 PM, Tom Lane wrote: > Andres Freund writes: >> Your wish just seems like a separate feature to me. Including >> replication commands in 'all' seems correct independent of the desire >> for a more granular control. > > No, I think I've got to vote with the other side

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Tom Lane
Andres Freund writes: > Your wish just seems like a separate feature to me. Including > replication commands in 'all' seems correct independent of the desire > for a more granular control. No, I think I've got to vote with the other side on that. The reason we can have log_statement as a scalar

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 14:50:34 +0200, Magnus Hagander wrote: > On Wed, Jun 11, 2014 at 2:35 PM, Andres Freund > wrote: > > > On 2014-06-11 14:22:43 +0200, Magnus Hagander wrote: > > > Yes, but how would you specify for example "i want DDL and all > > replication > > > commands" (which is quite a reasonab

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Magnus Hagander
On Wed, Jun 11, 2014 at 2:35 PM, Andres Freund wrote: > On 2014-06-11 14:22:43 +0200, Magnus Hagander wrote: > > Yes, but how would you specify for example "i want DDL and all > replication > > commands" (which is quite a reasonable thing to log, I believe). If you > > actually require it to be s

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 14:22:43 +0200, Magnus Hagander wrote: > Yes, but how would you specify for example "i want DDL and all replication > commands" (which is quite a reasonable thing to log, I believe). If you > actually require it to be set to "all", most people won't have any use at > all for it... Th

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Magnus Hagander
On Wed, Jun 11, 2014 at 2:17 PM, Andres Freund wrote: > On 2014-06-11 13:42:58 +0200, Magnus Hagander wrote: > > On Wed, Jun 11, 2014 at 12:34 PM, Fujii Masao > wrote: > > > > > Hi, > > > > > > Replication commands like IDENTIFY_COMMAND are not logged even when > > > log_statements is set to all

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 13:42:58 +0200, Magnus Hagander wrote: > On Wed, Jun 11, 2014 at 12:34 PM, Fujii Masao wrote: > > > Hi, > > > > Replication commands like IDENTIFY_COMMAND are not logged even when > > log_statements is set to all. Some users who use log_statements to > > audit *all* statements might

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Magnus Hagander
On Wed, Jun 11, 2014 at 12:34 PM, Fujii Masao wrote: > Hi, > > Replication commands like IDENTIFY_COMMAND are not logged even when > log_statements is set to all. Some users who use log_statements to > audit *all* statements might dislike this current situation. So I'm > thinking to change log_st

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 19:34:25 +0900, Fujii Masao wrote: > Hi, > > Replication commands like IDENTIFY_COMMAND are not logged even when > log_statements is set to all. Some users who use log_statements to > audit *all* statements might dislike this current situation. So I'm > thinking to change log_stateme

Re: [HACKERS] replication protocol documentation inconsistencies

2014-06-02 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, May 28, 2014 at 6:51 PM, Andres Freund wrote: > > On 2014-05-21 07:29:53 -0400, Peter Eisentraut wrote: > >> More generally, it is weird that the message formats are described > >> there, even though the rest of the protocol documentation only mentions > >> the messag

Re: [HACKERS] replication protocol documentation inconsistencies

2014-06-02 Thread Robert Haas
On Wed, May 28, 2014 at 6:51 PM, Andres Freund wrote: > On 2014-05-21 07:29:53 -0400, Peter Eisentraut wrote: >> Looking at >> http://www.postgresql.org/docs/devel/static/protocol-replication.html >> under START_REPLICATION it goes >> >> """ >> The payload of each CopyData message from server to t

Re: [HACKERS] replication protocol documentation inconsistencies

2014-05-28 Thread Andres Freund
Hi, On 2014-05-21 07:29:53 -0400, Peter Eisentraut wrote: > Looking at > http://www.postgresql.org/docs/devel/static/protocol-replication.html > under START_REPLICATION it goes > > """ > The payload of each CopyData message from server to the client contains > a message of one of the following fo

Re: [HACKERS] Replication slots and footguns

2014-03-18 Thread Robert Haas
On Thu, Mar 13, 2014 at 9:07 PM, Josh Berkus wrote: > On 03/13/2014 05:28 PM, Robert Haas wrote: >> Well we may have kind of hosed ourselves, because the in-memory data >> structures that represent the data structure have an in_use flag that >> indicates whether the structure is allocated at all,

Re: [HACKERS] Replication slots and footguns

2014-03-13 Thread Josh Berkus
On 03/13/2014 05:28 PM, Robert Haas wrote: > Well we may have kind of hosed ourselves, because the in-memory data > structures that represent the data structure have an in_use flag that > indicates whether the structure is allocated at all, and then an > active flag that indicates whether some back

Re: [HACKERS] Replication slots and footguns

2014-03-13 Thread Robert Haas
On Thu, Mar 13, 2014 at 8:09 PM, Josh Berkus wrote: > On 03/13/2014 05:01 PM, Robert Haas wrote: >> On Thu, Mar 13, 2014 at 6:45 PM, Josh Berkus wrote: >>> On 03/13/2014 01:17 PM, Robert Haas wrote: I think "in use" is just as clear as active, and I think the text Andres proposed previo

Re: [HACKERS] Replication slots and footguns

2014-03-13 Thread Josh Berkus
On 03/13/2014 05:01 PM, Robert Haas wrote: > On Thu, Mar 13, 2014 at 6:45 PM, Josh Berkus wrote: >> On 03/13/2014 01:17 PM, Robert Haas wrote: >>> I think "in use" is just as clear as active, and I think the text >>> Andres proposed previously reads a whole lot more nicely than this: >>> >>> repli

Re: [HACKERS] Replication slots and footguns

2014-03-13 Thread Robert Haas
On Thu, Mar 13, 2014 at 6:45 PM, Josh Berkus wrote: > On 03/13/2014 01:17 PM, Robert Haas wrote: >> I think "in use" is just as clear as active, and I think the text >> Andres proposed previously reads a whole lot more nicely than this: >> >> replication slot "%s" is in use by another backend > >

Re: [HACKERS] Replication slots and footguns

2014-03-13 Thread Josh Berkus
On 03/13/2014 01:17 PM, Robert Haas wrote: > I think "in use" is just as clear as active, and I think the text > Andres proposed previously reads a whole lot more nicely than this: > > replication slot "%s" is in use by another backend Then we should change the column name in the pg_stat_replicat

Re: [HACKERS] Replication slots and footguns

2014-03-13 Thread Robert Haas
On Thu, Mar 13, 2014 at 1:03 PM, Josh Berkus wrote: > On 03/13/2014 04:07 AM, Andres Freund wrote: >> On 2014-03-12 13:34:47 -0700, Josh Berkus wrote: >>> On 03/12/2014 12:34 PM, Robert Haas wrote: >> Urgh. That error message looks susceptible to improvement. How about: rep

Re: [HACKERS] Replication slots and footguns

2014-03-13 Thread Josh Berkus
On 03/13/2014 04:07 AM, Andres Freund wrote: > On 2014-03-12 13:34:47 -0700, Josh Berkus wrote: >> On 03/12/2014 12:34 PM, Robert Haas wrote: > Urgh. That error message looks susceptible to improvement. How about: >>> >>> replication slot "%s" cannot be dropped because it is currently

Re: [HACKERS] Replication slots and footguns

2014-03-13 Thread Andres Freund
On 2014-03-12 13:34:47 -0700, Josh Berkus wrote: > On 03/12/2014 12:34 PM, Robert Haas wrote: > >>> Urgh. That error message looks susceptible to improvement. How about: > >>> >> > >>> >> replication slot "%s" cannot be dropped because it is currently in use > >> > > >> > I think that'd require d

Re: [HACKERS] Replication slots and footguns

2014-03-12 Thread Josh Berkus
On 03/12/2014 04:52 PM, Thom Brown wrote: > On 12 March 2014 23:17, Michael Paquier wrote: >> On Thu, Mar 13, 2014 at 5:45 AM, Thom Brown wrote: >>> I'm not clear on why would dropping an active replication slot would >>> solve disk space problems related to WAL. I thought it was inactive >>> sl

Re: [HACKERS] Replication slots and footguns

2014-03-12 Thread Thom Brown
On 12 March 2014 23:17, Michael Paquier wrote: > On Thu, Mar 13, 2014 at 5:45 AM, Thom Brown wrote: >> On 12 March 2014 19:00, Josh Berkus wrote: >>> All: >>> >>> I was just reading Michael's explanation of replication slots >>> (http://michael.otacoo.com/postgresql-2/postgres-9-4-feature-highli

Re: [HACKERS] Replication slots and footguns

2014-03-12 Thread Michael Paquier
On Thu, Mar 13, 2014 at 5:45 AM, Thom Brown wrote: > On 12 March 2014 19:00, Josh Berkus wrote: >> All: >> >> I was just reading Michael's explanation of replication slots >> (http://michael.otacoo.com/postgresql-2/postgres-9-4-feature-highlight-replication-slots/) >> and realized there was somet

Re: [HACKERS] Replication slots and footguns

2014-03-12 Thread Thom Brown
On 12 March 2014 19:00, Josh Berkus wrote: > All: > > I was just reading Michael's explanation of replication slots > (http://michael.otacoo.com/postgresql-2/postgres-9-4-feature-highlight-replication-slots/) > and realized there was something which had completely escaped me in the > pre-commit di

Re: [HACKERS] Replication slots and footguns

2014-03-12 Thread Josh Berkus
On 03/12/2014 12:34 PM, Robert Haas wrote: >>> Urgh. That error message looks susceptible to improvement. How about: >>> >> >>> >> replication slot "%s" cannot be dropped because it is currently in use >> > >> > I think that'd require duplicating some code between acquire and drop, >> > but how a

<    1   2   3   4   5   6   >