Re: [HACKERS] pglogical - logical replication contrib module

2016-01-16 Thread Steve Singer
On 01/15/2016 12:07 PM, Petr Jelinek wrote: That's bug, fixed. Can you posted an updated patch with whatever fixes you have so far made? There are several statuses the table goes through, during the COPY it's in synchronizing status, so next logical step seemed to be synchronized. Maybe it

Re: [HACKERS] pglogical - logical replication contrib module

2016-01-22 Thread Steve Singer
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested This reply will covers a 10,000 foot level review of the feature (some of my

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-02 Thread Steve Singer
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: not tested Documentation:tested, failed Here is some more review +- `pglogical.replication_set_add_table

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-02 Thread Steve Singer
On 01/26/2016 10:43 AM, Craig Ringer wrote: On 23 January 2016 at 11:17, Steve Singer <mailto:st...@ssinger.info>> wrote: ** Schema changes involving rewriting big tables Sometimes you have a DDL change on a large table that will involve a table rewrite and the be

Re: [HACKERS] Logical Replication WIP

2016-09-05 Thread Steve Singer
On 08/31/2016 04:51 PM, Petr Jelinek wrote: Hi, and one more version with bug fixes, improved code docs and couple more tests, some general cleanup and also rebased on current master for the start of CF. To get the 'subscription' TAP tests to pass I need to set export PGTZ=+02 Shouldn'

Re: [HACKERS] Logical Replication WIP

2016-09-05 Thread Steve Singer
On 09/05/2016 03:58 PM, Steve Singer wrote: On 08/31/2016 04:51 PM, Petr Jelinek wrote: Hi, and one more version with bug fixes, improved code docs and couple more tests, some general cleanup and also rebased on current master for the start of CF. A few more things I noticed when

Re: [HACKERS] Logical Replication WIP

2016-09-18 Thread Steve Singer
On 09/08/2016 06:59 PM, Petr Jelinek wrote: - the CREATE SUBSCRIPTION also tries to check if the specified connection connects back to same db (although that check is somewhat imperfect) and if it gets stuck on create slot it should be normally cancelable (that should solve the issue Steve

Re: [HACKERS] Logical Replication WIP

2016-09-18 Thread Steve Singer
if the specified connection connects back to same db (although that check is somewhat imperfect) and if it gets stuck on create slot it should be normally cancelable (that should solve the issue Steve Singer had) - fixed the tests to work in any timezone - added DDL regress tests for subscription

Re: [HACKERS] Logical Replication WIP

2016-09-20 Thread Steve Singer
On Tue, 20 Sep 2016, Peter Eisentraut wrote: On 9/18/16 4:17 PM, Steve Singer wrote: I think if we want to prevent the creation of subscriptions that point to self, then we need to create a magic token when the postmaster starts and check for that when we connect. So more of a running

Re: [HACKERS] pgbench regression test failure

2017-11-11 Thread Steve Singer
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: not tested Spec compliant: not tested Documentation:not tested This causes the pgbench tests to fail (consistently) with not ok 194 - p

Re: [HACKERS] Logical Replication WIP

2016-10-30 Thread Steve Singer
On 10/24/2016 09:22 AM, Petr Jelinek wrote: Hi, attached is updated version of the patch. There are quite a few improvements and restructuring, I fixed all the bugs and basically everything that came up from the reviews and was agreed on. There are still couple of things missing, ie column type

Re: [HACKERS] Logical Replication WIP

2016-11-05 Thread Steve Singer
On 10/31/2016 06:38 AM, Petr Jelinek wrote: There are some fundamental issues with initial sync that need to be discussed on list but this one is not known. I'll try to convert this to test case (seems like useful one) and fix it, thanks for the report. In meantime I realized I broke the last p

Re: [HACKERS] Logical Replication WIP

2016-11-13 Thread Steve Singer
On 10/31/2016 06:38 AM, Petr Jelinek wrote: On 31/10/16 00:52, Steve Singer wrote: There are some fundamental issues with initial sync that need to be discussed on list but this one is not known. I'll try to convert this to test case (seems like useful one) and fix it, thanks for the r

Re: [HACKERS] Logical Replication WIP

2016-11-20 Thread Steve Singer
On Sun, 20 Nov 2016, Petr Jelinek wrote: On 13/11/16 23:02, Steve Singer wrote: There is one exception though: *** 195,214 A conflict will produce an error and will stop the replication; it ! must be resolved manually by the user. ! The resolution can

[HACKERS] 9.4 logical replication - walsender keepalive replies

2014-06-30 Thread Steve Singer
In 9.4 we've the below block of code to walsender.c as /* * We only send regular messages to the client for full decoded * transactions, but a synchronous replication and walsender shutdown * possibly are waiting for a later location. So we send pings * containing the flush location every no

[HACKERS] 9.4 pg_control corruption

2014-07-08 Thread Steve Singer
I've encountered a corrupt pg_control file on my 9.4 development cluster. I've mostly been using the cluster for changeset extraction / slony testing. This is a 9.4 (currently commit 6ad903d70a440e + a walsender change discussed in another thread) but would have had the initdb done with an

Re: [HACKERS] 9.4 pg_control corruption

2014-07-08 Thread Steve Singer
On 07/08/2014 10:14 PM, Tom Lane wrote: Steve Singer writes: I've encountered a corrupt pg_control file on my 9.4 development cluster. I've mostly been using the cluster for changeset extraction / slony testing. This is a 9.4 (currently commit 6ad903d70a440e + a walsender change

Re: [HACKERS] 9.4 logical replication - walsender keepalive replies

2014-07-14 Thread Steve Singer
On 07/06/2014 10:11 AM, Andres Freund wrote: Hi Steve, Right. I thought about this for a while, and I think we should change two things. For one, don't request replies here. It's simply not needed, as this isn't dealing with timeouts. For another don't just check ->flush < sentPtr but also &&

Re: [HACKERS] 9.4 logical replication - walsender keepalive replies

2014-08-11 Thread Steve Singer
On 07/14/2014 01:19 PM, Steve Singer wrote: On 07/06/2014 10:11 AM, Andres Freund wrote: Hi Steve, Right. I thought about this for a while, and I think we should change two things. For one, don't request replies here. It's simply not needed, as this isn't dealing with timeou

[HACKERS] 9.4 logical decoding assertion

2014-08-14 Thread Steve Singer
I hit the following on 9.4 testing logical decoding. TRAP: FailedAssertion("!(prev_first_lsn < cur_txn->first_lsn)", File: "reorderbuffer.c", Line: 618) LOG: server process (PID 3801) was terminated by signal 6: Aborted Unfortunately I don't have a core file and I haven't been able to repro

Re: [HACKERS] 9.4 logical decoding assertion

2014-08-15 Thread Steve Singer
On 08/15/2014 09:42 AM, Andres Freund wrote: On 2014-08-14 16:03:08 -0400, Steve Singer wrote: I hit the following on 9.4 testing logical decoding. TRAP: FailedAssertion("!(prev_first_lsn < cur_txn->first_lsn)", File: "reorderbuffer.c", Line: 618) LOG: ser

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

2017-04-16 Thread Steve Singer
On 04/10/2017 01:28 PM, Fujii Masao wrote: Hi, src/backend/replication/logical/launcher.c * Worker started and attached to our shmem. This check is safe * because only launcher ever starts the workers, so nobody can steal * the worker slot. The tablesync

[HACKERS] 10 beta docs: different replication solutions

2017-07-30 Thread Steve Singer
We don't seem to describe logical replication on https://www.postgresql.org/docs/10/static/different-replication-solutions.html The attached patch adds a section. Steve diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml index 138bdf2..1329d1f 100644 --- a/

Re: [HACKERS] 10 beta docs: different replication solutions

2017-07-31 Thread Steve Singer
On Mon, 31 Jul 2017, Merlin Moncure wrote: On Sun, Jul 30, 2017 at 8:34 PM, Steve Singer wrote: We don't seem to describe logical replication on https://www.postgresql.org/docs/10/static/different-replication-solutions.html The attached patch adds a section. This is a good catch.

Re: [HACKERS] snapshot too old, configured by time

2015-09-08 Thread Steve Singer
On 08/31/2015 10:07 AM, Kevin Grittner wrote: Kevin, I've started to do a review on this patch but I am a bit confused with some of what I am seeing. The attached testcase fails I replace the cursor in your test case with direct selects from the table. I would have expected this to generat

Re: [HACKERS] record identical operator

2013-09-18 Thread Steve Singer
On 09/18/2013 11:39 AM, Stephen Frost wrote: * Kevin Grittner (kgri...@ymail.com) wrote: = and <> aren't listed above even though they do a byte-for-byte comparison because, well, I guess because we have chosen to treat two UTF8 strings which produce the same set of glyphs using different bytes

Re: [HACKERS] record identical operator - Review

2013-09-19 Thread Steve Singer
On 09/12/2013 06:27 PM, Kevin Grittner wrote: Attached is a patch for a bit of infrastructure I believe to be necessary for correct behavior of REFRESH MATERIALIZED VIEW CONCURRENTLY as well as incremental maintenance of matviews. Here is attempt at a review of the v1 patch. There has been a h

Re: [HACKERS] record identical operator - Review

2013-09-20 Thread Steve Singer
On 09/20/2013 08:38 AM, Kevin Grittner wrote: Did you look at the record_eq and record_cmp functions which exist before this patch? Is there a reason we should do it one way for the default operators and another way for this alternative? Do you think we should change record_eq and record_cmp t

Re: [HACKERS] logical changeset generation v6

2013-09-21 Thread Steve Singer
On 09/20/2013 06:33 AM, Andres Freund wrote: Hi, The points I find daunting are the semantics, like: * How do we control whether a standby is allowed prevent WAL file removal. What if archiving is configured? * How do we control whether a standby is allowed to peg xmin? * How long do we pe

Re: [HACKERS] logical changeset generation v6

2013-09-24 Thread Steve Singer
On 09/24/2013 11:21 AM, Andres Freund wrote: Not having a consumer of the walsender interface included sounds like a bad idea to me, even if it were only useful for testing. Now, you could argue it should be in /contrib - and I wouldn't argue against that except it shares code with the rest of sr

Re: [HACKERS] logical changeset generation v6

2013-09-25 Thread Steve Singer
On 09/17/2013 10:31 AM, Andres Freund wrote: This patch set now fails to apply because of the commit "Rename various "freeze multixact" variables". And I am even partially guilty for that patch... Rebased patches attached. While testing the logical replication changes against my WIP logical s

Re: [HACKERS] logical changeset generation v6

2013-09-25 Thread Steve Singer
On 09/25/2013 11:08 AM, Andres Freund wrote: On 2013-09-25 11:01:44 -0400, Steve Singer wrote: On 09/17/2013 10:31 AM, Andres Freund wrote: This patch set now fails to apply because of the commit "Rename various "freeze multixact" variables". And I am even partially g

Re: [HACKERS] logical changeset generation v6

2013-09-26 Thread Steve Singer
On 09/25/2013 01:20 PM, Steve Singer wrote: On 09/25/2013 11:08 AM, Andres Freund wrote: On 2013-09-25 11:01:44 -0400, Steve Singer wrote: On 09/17/2013 10:31 AM, Andres Freund wrote: This patch set now fails to apply because of the commit "Rename various "freeze multixact" va

Re: [HACKERS] logical changeset generation v6

2013-09-27 Thread Steve Singer
On 09/26/2013 02:47 PM, Steve Singer wrote: I've determined that when in this test the walsender seems to be hitting this when it is decode the transactions that are behind the slonik commands to add tables to replication (set add table, set add sequence). This is before the SUBSCRIB

Re: [HACKERS] logical changeset generation v6.1

2013-09-28 Thread Steve Singer
On 09/27/2013 11:44 AM, Andres Freund wrote: I'm encountering a make error: Gah. Lastminute changes. Always the same... Updated patch attached. Greetings, Andres Freund I'm still encountering an error in the make. make clean . .make[3]: Entering directory `/usr/local/src/postgresql/src/bi

Re: [HACKERS] logical changeset generation v6

2013-09-28 Thread Steve Singer
On 09/27/2013 05:18 PM, Andres Freund wrote: Hi Steve, On 2013-09-27 17:06:59 -0400, Steve Singer wrote: I've determined that when in this test the walsender seems to be hitting this when it is decode the transactions that are behind the slonik commands to add tables to replication (se

Re: [HACKERS] record identical operator - Review

2013-09-29 Thread Steve Singer
On 09/28/2013 03:03 PM, Kevin Grittner wrote: +To support matching of rows which include elements without a default +B-tree operator class, the following operators are defined for composite +type comparison: +*=, +*<>, +*<, +*<=, +*>, and +*>=. +These oper

Re: [HACKERS] logical changeset generation v6.2

2013-10-01 Thread Steve Singer
On 09/30/2013 06:44 PM, Andres Freund wrote: Hi, The series from friday was a bit too buggy - obviously I was too tired. So here's a new one: With this series I've also noticed #2 0x007741a7 in ExceptionalCondition ( conditionName=conditionName@entry=0x7c2908 "!(!(tuple->t_infomas

Re: [HACKERS] record identical operator - Review

2013-10-03 Thread Steve Singer
On 09/30/2013 09:08 AM, Kevin Grittner wrote: Steve Singer wrote: How about To support matching of rows which include elements without a default B-tree operator class, the following operators are defined for composite type comparison

Re: [HACKERS] logical changeset generation v6.2

2013-10-03 Thread Steve Singer
On 10/03/2013 12:38 PM, Andres Freund wrote: Does your code use SELECT FOR UPDATE/SHARE on system or treat_as_catalog tables? Greetings, Andres Freund Yes. It declares sl_table and sl_sequence and sl_set as catalog. It does a SELECT .. from @NAMESPACE@.sl_table T, @NAMESPACE@.sl_set S

Re: [HACKERS] logical changeset generation v6.2

2013-10-07 Thread Steve Singer
On 10/03/2013 04:00 PM, Andres Freund wrote: Ok, there were a couple of bugs because I thought mxacts wouldn't need to be supported. So far your testcase doesn't crash the database anymore - it spews some internal errors though, so I am not sure if it's entirely fixed for you. Thanks for testin

Re: [HACKERS] logical changeset generation v6.1

2013-10-08 Thread Steve Singer
On 10/07/2013 09:32 AM, Andres Freund wrote: Todo: * rename treat_as_catalog_table, after agreeing on the new name * rename remaining timetravel function names * restrict SuspendDecodingSnapshots usage to RelationInitPhysicalAddr, that ought to be enough. * add InLogicalDecoding() function. *

Re: [HACKERS] logical changeset generation v6.5

2013-11-09 Thread Steve Singer
On 11/05/2013 10:21 AM, Andres Freund wrote: Hi, Attached to this mail and in the xlog-decoding-rebasing-remapping branch in my git[1] repository you can find the next version of the patchset that: * Fixes full table rewrites of catalog tables using the method Robert prefers (which is to log

Re: [HACKERS] logical changeset generation v6.5

2013-11-09 Thread Steve Singer
On 11/09/2013 05:42 PM, Andres Freund wrote: On 2013-11-09 17:36:49 -0500, Steve Singer wrote: On 11/05/2013 10:21 AM, Andres Freund wrote: Hi, Attached to this mail and in the xlog-decoding-rebasing-remapping branch in my git[1] repository you can find the next version of the patchset that

Re: [HACKERS] logical changeset generation v6.5

2013-11-10 Thread Steve Singer
On 11/10/2013 09:41 AM, Andres Freund wrote: Still give me the following: update disorder.do_inventory set ii_in_stock=2 where ii_id=251; UPDATE 1 test1=# LOG: tuple in table with oid: 35122 without primary key Hm. Could it be that you still have an older "test_decoding" plugin lying around? Th

Re: [HACKERS] snapshot too old, configured by time

2015-11-08 Thread Steve Singer
On 10/15/2015 05:47 PM, Kevin Grittner wrote: All other issues raised by Álvaro and Steve have been addressed, except for this one, which I will argue against: I've been looking through the updated patch In snapmgr.c + * XXX: If we can trust a read of an int64 value to be atomic, we can s

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-24 Thread Steve Singer
On 05/28/2013 04:41 PM, Szymon Guz wrote: Hi, I've got a patch. This is for a plpython enhancement. There is an item at the TODO list http://wiki.postgresql.org/wiki/Todo#Server-Side_Languages "Fix loss of information during conversion of numeric type to Python float" This patch uses a deci

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-25 Thread Steve Singer
On 06/25/2013 06:42 AM, Szymon Guz wrote: Hi, I've attached a new patch. I've fixed all the problems you've found, except for the efficiency problem, which has been described in previous email. thanks, Szymon This version of the patch addresses the issues I mentioned. Thanks for looki

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-26 Thread Steve Singer
On 06/26/2013 04:47 PM, Szymon Guz wrote: Attached patch has all changes against trunk code. There is added a function for conversion from Postgres numeric to Python Decimal. The Decimal type is taken from cdecimal.Decimal, if it is available. It is an external library, quite fast, but may

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-28 Thread Steve Singer
On 06/27/2013 05:04 AM, Szymon Guz wrote: On 27 June 2013 05:21, Steve Singer <mailto:st...@ssinger.info>> wrote: On 06/26/2013 04:47 PM, Szymon Guz wrote: Hi Steve, thanks for the changes. You're idea about common code for decimal and cdecimal is good, however not

Re: [HACKERS] changeset generation v5-01 - Patches & git tree

2013-07-05 Thread Steve Singer
On 07/05/2013 08:03 AM, Andres Freund wrote: On 2013-06-27 21:52:03 -0700, Kevin Grittner wrote: Tried that, too, and problem persists. The log shows the last commit on your branch as 022c2da1873de2fbc93ae524819932719ca41bdb. Ok. I think I have a slight idea what's going on. Could you check w

Re: [HACKERS] changeset generation v5-01 - Patches & git tree

2013-07-05 Thread Steve Singer
On 07/05/2013 09:34 AM, Andres Freund wrote: On 2013-07-05 09:28:45 -0400, Steve Singer wrote: On 07/05/2013 08:03 AM, Andres Freund wrote: On 2013-06-27 21:52:03 -0700, Kevin Grittner wrote: Tried that, too, and problem persists. The log shows the last commit on your branch as

Re: [HACKERS] changeset generation v5-01 - Patches & git tree

2013-07-05 Thread Steve Singer
On 06/14/2013 06:51 PM, Andres Freund wrote: The git tree is at: git://git.postgresql.org/git/users/andresfreund/postgres.git branch xlog-decoding-rebasing-cf4 http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=shortlog;h=refs/heads/xlog-decoding-rebasing-cf4 We discussed i

Re: [HACKERS] Replication identifiers, take 3

2014-09-27 Thread Steve Singer
On 09/26/2014 06:05 PM, Andres Freund wrote: On 2014-09-26 14:57:12 -0400, Robert Haas wrote: Sure, it'll possibly not be trivial to move them elsewhere. On the other hand, the padding bytes have been unused for 8+ years without somebody laying "claim" on them but "me". I don't think it's a good

Re: [HACKERS] Replication identifiers, take 3

2014-09-27 Thread Steve Singer
On 09/26/2014 10:21 AM, Andres Freund wrote: On 2014-09-26 09:53:09 -0400, Robert Haas wrote: On Fri, Sep 26, 2014 at 5:05 AM, Andres Freund wrote: Let me try to summarize the information requirements for each of these things. For #1, you need to know, after crash recovery, for each standby,

Re: [HACKERS] [PATCH] HINT: pg_hba.conf changed since last config reload

2014-10-19 Thread Steve Singer
On 10/16/2014 11:34 PM, Craig Ringer wrote: Given the generally positive reception to this, here's a patch. The first patch adds an errhint_log , akin to the current errdetail_log, so we can send a different HINT to the server log than we do to the client. The patch behaves as you describe.

Re: [HACKERS] Moving of INT64_FORMAT to c.h

2014-10-22 Thread Steve Singer
On 10/16/2014 08:08 AM, Andres Freund wrote: On 2014-10-16 08:04:17 -0400, Jan Wieck wrote: Hi, PostgreSQL has for ages defined INT64_FORMAT and UINT64_FORMAT in pg_config.h. This commit http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=ce486056ecd28050 moved those definitions to

Re: [HACKERS] Moving of INT64_FORMAT to c.h

2014-10-22 Thread Steve Singer
On 10/22/2014 04:17 PM, Robert Haas wrote: Slonik used to include postgre_fe.h but back in 2011 or so we stopped doing so because it was causing issues (I think on win32 builds) That seems like something we ought to consider fixing, but obviously we'd need more details. When I'll try to find

[HACKERS] "snapshot too large" error when initializing logical replication (9.4)

2014-10-25 Thread Steve Singer
I sometimes get the error "snapshot too large" from my logical replication walsender process when in response to a CREATE_REPLICATION_SLOT. This is in SnapBuildExportSnapshot in snapbuild.c newxcnt is 212 at that point I have max_connections = 200 procArray->maxProcs=212 Should we be testing

[HACKERS] logical decoding - reading a user catalog table

2014-10-25 Thread Steve Singer
My logical decoding plugin is occasionally getting this error "could not resolve cmin/cmax of catalog tuple" I get this when my output plugin is trying to read one of the user defined catalog tables (user_catalog_table=true) I am not sure if this is a bug in the time-travel support in the l

Re: [HACKERS] logical decoding - reading a user catalog table

2014-10-28 Thread Steve Singer
On 10/28/2014 01:31 PM, Andres Freund wrote: On 2014-10-25 18:18:07 -0400, Steve Singer wrote: My logical decoding plugin is occasionally getting this error "could not resolve cmin/cmax of catalog tuple" I get this when my output plugin is trying to read one of the user defin

Re: [HACKERS] "snapshot too large" error when initializing logical replication (9.4)

2014-10-30 Thread Steve Singer
On 10/28/2014 01:27 PM, Andres Freund wrote: Hi, On 2014-10-25 18:09:36 -0400, Steve Singer wrote: I sometimes get the error "snapshot too large" from my logical replication walsender process when in response to a CREATE_REPLICATION_SLOT. Yes. That's possible if 'too much

Re: [HACKERS] tracking commit timestamps

2014-11-05 Thread Steve Singer
On 11/05/2014 11:23 AM, Jim Nasby wrote: Except that commit time is not guaranteed unique *even on a single system*. That's my whole point. If we're going to bother with all the commit time machinery it seems really silly to provide a way to uniquely order every commit. Clearly trying to u

Re: [HACKERS] tracking commit timestamps

2014-11-05 Thread Steve Singer
On 11/05/2014 05:43 PM, Andres Freund wrote: On 2014-11-05 17:17:05 -0500, Steve Singer wrote: Imo that's essentially a different feature. What you essentially would need here is a 'commit sequence number' - but no timestamps. And probably to be useful that number has to be 8

Re: [HACKERS] tracking commit timestamps

2014-11-09 Thread Steve Singer
On 11/07/2014 07:07 PM, Petr Jelinek wrote: The list of what is useful might be long, but we can't have everything there as there are space constraints, and LSN is another 8 bytes and I still want to have some bytes for storing the "origin" or whatever you want to call it there, as that's the

Re: [HACKERS] tracking commit timestamps

2014-11-10 Thread Steve Singer
On 11/10/2014 08:39 AM, Petr Jelinek wrote: On 09/11/14 17:57, Steve Singer wrote: On 11/07/2014 07:07 PM, Petr Jelinek wrote: The list of what is useful might be long, but we can't have everything there as there are space constraints, and LSN is another 8 bytes and I still want to have

Re: [HACKERS] logical decoding - reading a user catalog table

2014-11-13 Thread Steve Singer
On 11/13/2014 02:44 PM, Andres Freund wrote: H I've pushed a fix for a bug that could possibly also cause this. Although it'd be odd that it always hits the user catalog table. Except if your tests mostly modify the slony tables, but do not do much DDL otherwise? The test I was running doesn

Re: [HACKERS] tracking commit timestamps

2014-11-14 Thread Steve Singer
On 11/14/2014 08:21 PM, Simon Riggs wrote: The requested information is already available, as discussed. Logical decoding adds commit ordering for *exactly* the purpose of using it for replication, available to all solutions. This often requested feature has now been added and doesn't need to be

Re: [HACKERS] logical decoding - reading a user catalog table

2014-11-16 Thread Steve Singer
On 11/13/2014 02:44 PM, Andres Freund wrote: Hi Steve, If it still happens, could you send me instructions of how to reproduce the problem after cloning the necessary source repositories? It's quite hard to validate a possible fix otherwise. 1. Install PG 9.4 2. Perform an initdb max_con

Re: [HACKERS] logical decoding - reading a user catalog table

2014-11-17 Thread Steve Singer
On 11/16/2014 04:49 PM, Steve Singer wrote: I installed things following the above steps on a different system than my usual development laptop and I have been unable to reproduce the error so for (on that system). But I am still able to reproduce it on occasion on my normal development

Re: [HACKERS] logical decoding - reading a user catalog table

2014-11-17 Thread Steve Singer
On 11/17/2014 10:37 AM, Andres Freund wrote: On 2014-11-13 22:23:02 -0500, Steve Singer wrote: Also since updating (to 2c267e47afa4f9a7c) I've seen a assertion failure in a normal client connection, not the walsender #3 0x006b4978 in GetSerializableTransactionSnapsh

Re: [HACKERS] logical decoding - reading a user catalog table

2014-11-17 Thread Steve Singer
On 11/17/2014 11:34 AM, Andres Freund wrote: Hi, Kevin: CCed you, because it doesn't really look like a logical decoding related issue. On 2014-11-17 11:25:40 -0500, Steve Singer wrote: On 11/17/2014 10:37 AM, Andres Freund wrote: On 2014-11-13 22:23:02 -0500, Steve Singer wrote:

Re: [HACKERS] tracking commit timestamps

2014-11-19 Thread Steve Singer
On 11/19/2014 08:22 AM, Alvaro Herrera wrote: I think we're overblowing the pg_upgrade issue. Surely we don't need to preserve commit_ts data when upgrading across major versions; and pg_upgrade is perfectly prepared to remove old data when upgrading (actually it just doesn't copy it; consider

Re: [HACKERS] pglogical - logical replication contrib module

2016-01-08 Thread Steve Singer
On 12/31/2015 06:34 PM, Petr Jelinek wrote: Hi, I'd like to submit the replication solution which is based on the pglogical_output [1] module (which is obviously needed for this to compile). Hi, make check gives me for extra in ../../contrib/pglogical_output contrib/pglogical; do make

Re: [HACKERS] pglogical - logical replication contrib module

2016-01-09 Thread Steve Singer
On 12/31/2015 06:34 PM, Petr Jelinek wrote: Hi, I'd like to submit the replication solution which is based on the pglogical_output [1] module (which is obviously needed for this to compile). The pglogical contrib module provides extension which does the master-slave logical replication base

Re: [HACKERS] pglogical - logical replication contrib module

2016-01-10 Thread Steve Singer
On 01/09/2016 01:30 PM, Steve Singer wrote: On 12/31/2015 06:34 PM, Petr Jelinek wrote: I'm not really sure what to do to 'recover' my cluster at this point so I'll send this off and rebuild my cluster and start over. I had a setup test1--->test2 (with 2 tables

Re: [HACKERS] [PATCH 09/14] Adjust all *Satisfies routines to take a HeapTuple instead of a HeapTupleHeader

2012-11-16 Thread Steve Singer
On 12-11-14 08:17 PM, Andres Freund wrote: For the regular satisfies routines this is needed in prepareation of logical decoding. I changed the non-regular ones for consistency as well. The naming between htup, tuple and similar is rather confused, I could not find any consistent naming anywhere

Re: [HACKERS] logical changeset generation v3 - Source for Slony

2012-11-19 Thread Steve Singer
First, you can add me to the list of people saying 'wow', I'm impressed. The approach I am taking to reviewing this to try and answer the following question 1) How might a future version of slony be able to use logical replication as described by your patch and design documents and what woul

Re: [HACKERS] logical changeset generation v3 - Source for Slony

2012-11-19 Thread Steve Singer
On 12-11-18 11:07 AM, Andres Freund wrote: Hi Steve! I think we should provide some glue code to do this, otherwise people will start replicating all the bugs I hacked into this... More seriously: I think we should have support code here, no user will want to learn the intracacies of feedback m

Re: [HACKERS] [PATCH 11/14] Introduce wal decoding via catalog timetravel

2012-12-02 Thread Steve Singer
On 12-11-14 08:17 PM, Andres Freund wrote: I am getting errors like the following when I try to use either your test_decoding plugin or my own (which does even less than yours) LOG: database system is ready to accept connections LOG: autovacuum launcher started WARNING: connecting to WARNI

Re: [HACKERS] [PATCH 11/14] Introduce wal decoding via catalog timetravel

2012-12-03 Thread Steve Singer
On 12-12-03 07:42 AM, Andres Freund wrote: Two things: 1) Which exact options are you using for pg_receivellog? Not "-d replication" by any chance? Yes that is exactly what I'md doing. Using a real database name instead makes this go away. Thanks This seems to produce exactly that kind of

Re: [HACKERS] [PATCH 11/14] Introduce wal decoding via catalog timetravel

2012-12-03 Thread Steve Singer
On 12-12-03 09:48 AM, Andres Freund wrote: On 2012-12-03 09:35:55 -0500, Steve Singer wrote: On 12-12-03 07:42 AM, Andres Freund wrote: Yes that is exactly what I'md doing. Using a real database name instead makes this go away. Was using "replication" an accident or do you

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-11 Thread Steve Singer
On 12-12-11 06:52 PM, Andres Freund wrote: Hi, Problem 1: One problem I see is that while exporting a snapshot solves the visibility issues of the table's contents it does not protect against schema changes. I am not sure whether thats a problem. If somebody runs a CLUSTER or something like

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-13 Thread Steve Singer
On 12-12-12 06:20 AM, Andres Freund wrote: Possible solutions: 1) INIT_LOGICAL_REPLICATION waits for an answer from the client that confirms that logical replication initialization is finished. Before that the walsender connection cannot be used for anything else. 2) we remove the snapshot as so

[HACKERS] AIX buildfarm member

2013-01-11 Thread Steve Singer
The only animal in the buildfarm running AIX is grebe (http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=grebe&br=HEAD) It is likely that the server running this animal will go away sometime this year and the machine replacing it isn't running AIX. If someone else in the community is

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-19 Thread Steve Singer
On 13-01-09 03:07 PM, Tom Lane wrote: Andres Freund writes: Well, I *did* benchmark it as noted elsewhere in the thread, but thats obviously just machine (E5520 x 2) with one rather restricted workload (pgbench -S -jc 40 -T60). At least its rather palloc heavy. Here are the numbers: before: #10

Re: [HACKERS] logical changeset generation v4

2013-01-19 Thread Steve Singer
On 13-01-14 08:38 PM, Andres Freund wrote: Hi everyone, Here is the newest version of logical changeset generation. 2) Currently the logical replication infrastructure assigns a 'slot-id' when a new replica is setup. That slot id isn't really nice (e.g. "id-321578-3"). It also requires that [

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-21 Thread Steve Singer
On 13-01-21 02:28 AM, Andres Freund wrote: I haven't removed it from the patch afaik, so it would be great to get a profile here! Its "only" for xlogdump, but that tool helped me immensely and I don't want to maintain it independently... Here is the output from tprof Here is the baseline: Av

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-21 Thread Steve Singer
On 13-01-21 12:15 PM, Andres Freund wrote: On 2013-01-21 11:59:18 -0500, Steve Singer wrote: On 13-01-21 02:28 AM, Andres Freund wrote: I haven't removed it from the patch afaik, so it would be great to get a profile here! Its "only" for xlogdump, but that tool helped me immens

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Steve Singer
On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an output plugin as well. Steve, how far did you get there? I was able to get something

Re: [HACKERS] Event Triggers: adding information

2013-01-24 Thread Steve Singer
On 13-01-24 05:43 AM, Dimitri Fontaine wrote: Robert Haas writes: On Mon, Jan 21, 2013 at 12:27 PM, Dimitri Fontaine wrote: - Context exposing and filtering I'm not feeling very sanguine about any of this. I feel strongly that we should try to do something that's more principled than ju

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-26 Thread Steve Singer
On 13-01-24 11:15 AM, Steve Singer wrote: On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an output plugin as well. Steve, how far did you get

Re: [HACKERS] Event Triggers: adding information

2013-01-27 Thread Steve Singer
On 13-01-26 11:11 PM, Robert Haas wrote: On Fri, Jan 25, 2013 at 11:58 AM, Dimitri Fontaine wrote: My understanding is that if the command string we give to event triggers is ambiguous (sub-object names, schema qualifications, etc), it comes useless for logical replication use. I'll leave it to

Re: [HACKERS] logical changeset generation v4

2013-01-27 Thread Steve Singer
s Freund -- Andres Freundhttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services >From 176087bacec6cbf0b86e4ffeb918f41b4a5b8d7a Mon Sep 17 00:00:00 2001 From: Steve Singer Date: Sun, 27 Jan 2013 12:24:33 -0500 Subject: [PATCH] allow p

Re: [HACKERS] Patch to add a primary key using an existing index

2010-11-25 Thread Steve Singer
On 10-11-22 03:24 PM, Steve Singer wrote: On 10-11-22 09:37 AM, Gurjeet Singh wrote: On Sat, Nov 20, 2010 at 9:00 AM, Steve Singer Almost fixed. I still get an unexpected difference. ! DETAIL: cannot create PRIMARY KEY/UNIQUE constraint with a non-unique index. CREATE UNIQUE INDEX rpi_idx2

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Steve Singer
On 10-12-06 06:56 PM, Greg Smith wrote: Tom Lane wrote: The various testing that's been reported so far is all for Linux and thus doesn't directly address the question of whether other kernels will have similar performance properties. Survey of some popular platforms: So my guess is that

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-07 Thread Steve Singer
On 10-12-06 09:00 PM, Josh Berkus wrote: Steve, If you tell me which options to pgbench and which .conf file settings you'd like to see I can probably arrange to run some tests on AIX. Compile and run test_fsync in PGSRC/src/tools/fsync. Attached are runs against two different disk sub-sys

Re: [HACKERS] Patch to add a primary key using an existing index

2011-01-16 Thread Steve Singer
ould this be a call to pstrdup Feature Test - I wasn't able to find any issues in my testing I'm marking this as returned with feedback pending your answer on the possible memory leak above but I think the patch is very close to being ready. Steve Singer diff --git

Re: [HACKERS] log_hostname and pg_stat_activity

2011-01-18 Thread Steve Singer
Here is my review for this patch Submission Review -Patch applies cleanly -Patch does not include documentation changes. At a minimum: update the table that lists what pg_stat_activity and pg_stat_replication includes in monitoring.sgml but I propose more below. -No tests

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-27 Thread Steve Singer
On 13-01-24 11:15 AM, Steve Singer wrote: On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an output plugin as well. Steve, how far did you get

  1   2   3   >