[GENERAL] Insert behavior in transaction

2005-06-19 Thread Andres
one of them to finish. Thanks in advance. Andres ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster

Re: [GENERAL] Insert behavior in transaction

2005-06-19 Thread Andres
But, does it have solution ?? You are right, the second insert imply a unique-key violation, but what to do in that cases? Thanks for your help and attention! Tom Lane writes: > "Andres" <[EMAIL PROTECTED]> writes: >> I did this simple test and it fails too. >

Re: [GENERAL] shared_buffers formula

2015-03-03 Thread Andres Freund
the shared_buffers referenced in the formula is the number of buffers - whereas when you specify it using a size unit (like MB, GB,...) that amount of memory is divided by the size of a page. So you're off by a factor of 8192. Greetings, Andres Freund -- Andres Freund

Re: [GENERAL] shared_buffers formula

2015-03-04 Thread Andres Freund
nd > even swap is used. What could be the reason of postgres using so much > memory? Presumably it's used by the OS to cache IO? Are you investigating an actual problem with memory consumption or just wondering? Greetings, Andres Freund -- Andres Freund h

Re: [GENERAL] BDR - triggers on receiving node?

2015-03-26 Thread Andres Freund
t;standby" and do the cache invalidation via that. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make chan

Re: [GENERAL] Waiting on ExclusiveLock on extension

2015-04-17 Thread Andres Freund
e instead of one. I don't really agree that that's the most important bit. See http://archives.postgresql.org/message-id/20150329185619.GA29062%40alap3.anarazel.de Greetings, Andres Freund -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-05-29 Thread Andres Freund
On 2015-05-29 15:49:53 -0400, Bruce Momjian wrote: > I think we need to step back and look at the brain power required to > unravel the mess we have made regarding multi-xact and fixes. (I bet > few people can even remember which multi-xact fixes went into which > releases --- I can't.) Instead o

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-05-29 Thread Andres Freund
up needing to do a) even in the backbranches. Am I missing something? This problem isn't conflicting with most of the fixes you describe, so I'll continue with reviewing those. Greetings, Andres Freund -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-05-30 Thread Andres Freund
On 2015-05-30 00:52:37 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > I considered for a second whether the solution for that could be to not > > truncate while inconsistent - but I think that doesn't solve anything as > > then we can end up with directori

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-01 Thread Andres Freund
e actual bug? I'm more worried about the cases where we didn't ever actually "badly wrap around" (i.e. overwrite needed data); but where that's not clear on the standby because the base backup isn't in a consistent state. Greetings, Andres Freund -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-01 14:22:32 -0400, Robert Haas wrote: > On Mon, Jun 1, 2015 at 4:58 AM, Andres Freund wrote: > > The lack of WAL logging actually has caused problems in the 9.3.3 (?) > > era, where we didn't do any truncation during recovery... > > Right, but now we're

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-02 11:16:22 -0400, Robert Haas wrote: > I'm having trouble figuring out what to do about this. I mean, the > essential principle of this patch is that if we can't count on > relminmxid, datminmxid, or the control file to be accurate, we can at > least look at what is present on the disk

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-02 11:29:24 -0400, Robert Haas wrote: > On Tue, Jun 2, 2015 at 8:56 AM, Andres Freund wrote: > > But what *definitely* looks wrong to me is that a TruncateMultiXact() in > > this scenario now (since a couple weeks ago) does a > > SimpleLruReadPage_ReadOnly() in

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
ems like somebody needs to do it. I'm willing to invest the time to develop an initial version, but I'll need help evaluating it. I don't have many testing resources available atm, and I'm not going to trust stuff I developed while travelling by just looking at the code. Greeti

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
On 2015-06-02 11:49:56 -0400, Robert Haas wrote: > On Tue, Jun 2, 2015 at 11:44 AM, Andres Freund wrote: > > On 2015-06-02 11:37:02 -0400, Robert Haas wrote: > >> The exact circumstances under which we're willing to replace a > >> relminmxid with a newly-c

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
ndary, right? I'm not sure that's actually ok; because the value at the beginning of the segment can very well end up being a 0, as MaybeExtendOffsetSlru() will have filled the page with zeros. I think that should be harmless, the worst that can happen is that oldestOffset errorneously is 0, which should be correct, even though possibly overly conservative, in these cases. Greetings, Andres Freund -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-02 Thread Andres Freund
y happen after a upgrade from < 9.3. And in that case we initialize nextOffset to 0. That ought to safe us? Greetings, Andres Freund -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-03 Thread Andres Freund
de is. We must be missing some crucial detail ... I might have missed it in this already long thread. Could you share a bunch of details about hte case? It'd be very interesting to see the contents of the backup label (to see where start/end are), the contents of the initial checkpoint (to se

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-03 Thread Andres Freund
On 2015-06-03 15:01:46 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > That's not necessarily the case though, given how the code currently > > works. In a bunch of places the SLRUs are accessed *before* having been > > made consistent by WAL replay. Especia

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-04 Thread Andres Freund
o also return false in case of a InvalidMultiXactId - that'll be returned if the page has been zeroed. Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Andres Freund
On 2015-06-05 11:43:45 -0400, Tom Lane wrote: > Robert Haas writes: > > On Fri, Jun 5, 2015 at 2:20 AM, Noah Misch wrote: > >> I read through this version and found nothing to change. I encourage other > >> hackers to study the patch, though. The surrounding code is

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Andres Freund
committing today to a release wrap on Monday, > I don't see it happening till after PGCon. I wonder if, with all the recent, err, training, we could wrap it on Tuesday instead. Independent of the truncation rework going in or not, an additional work day to go over all the changes and

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-05 Thread Andres Freund
On June 5, 2015 10:02:37 PM GMT+02:00, Robert Haas wrote: >On Fri, Jun 5, 2015 at 2:47 PM, Andres Freund >wrote: >> On 2015-06-05 14:33:12 -0400, Tom Lane wrote: >>> Robert Haas writes: >>> > 1. The problem that we might truncate an SLRU members page away >

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-07 Thread Andres Freund
On 2015-06-05 20:47:33 +0200, Andres Freund wrote: > On 2015-06-05 14:33:12 -0400, Tom Lane wrote: > > Robert Haas writes: > > > 1. The problem that we might truncate an SLRU members page away when > > > it's in the buffers, but not drop it from the buffers, leadin

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Andres Freund
On 2015-06-05 16:56:18 -0400, Tom Lane wrote: > Andres Freund writes: > > On June 5, 2015 10:02:37 PM GMT+02:00, Robert Haas > > wrote: > >> I think we would be foolish to rush that part into the tree. We > >> probably got here in the first place by rushing

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Andres Freund
On 2015-06-08 15:15:04 +0200, Andres Freund wrote: > 1) the autovacuum trigger logic isn't perfect yet. I.e. especially with > autovacuum=off you can get into situations where emergency vacuums > aren't started when necessary. This is particularly likely to happen >

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Andres Freund
On June 8, 2015 7:06:31 PM GMT+02:00, Alvaro Herrera wrote: >Andres Freund wrote: > >> A first version to address this problem can be found appended to this >> email. >> >> Basically it does: >> * Whenever more than MULTIXACT_MEMBER_SAFE_THRESHOLD are u

Re: [HACKERS] [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

2015-06-08 Thread Andres Freund
tted email, than when sent from a full desktop. That's why I added the notice... Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] CLOG read problem after pg_basebackup

2015-06-21 Thread Andres Freund
70" at offset 237568: Success. > > (the clog file differed in each case of course..) How exactly are you starting the the standby after the basebackups? Any chance you removed backup.label? This sounds like the typical symptoms of doing that. Greetings, Andres Freund -- Sent via pgsq

Re: [GENERAL] cascading replication and replication slots.

2015-06-23 Thread Andres Freund
On 2015-06-22 13:32:23 +, Leif Gunnar Erlandsen wrote: > Is it possible to use a replication_slot for a downstream-server when setting > up cascading replication on 9.4 yes. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://w

Re: [GENERAL] Lots of stuck queries after upgrade to 9.4

2015-07-20 Thread Andres Freund
LogInsertionsToFinish() call in XLogFlush(). Spiros, I guess you have commit_delay enabled? If so, does disabling it "fix" the issue? Regards, Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Lots of stuck queries after upgrade to 9.4

2015-07-20 Thread Andres Freund
On 2015-07-20 13:22:42 +0200, Andres Freund wrote: > Hm. The problem seems to be the WaitXLogInsertionsToFinish() call in > XLogFlush(). These are the relevant stack traces: db9lock/debuglog-commit.txt #2 0x7f7405bd44f4 in LWLockWaitForVar (l=0x7f70f2ab6680, valptr=0x7f70f2ab66a0,

Re: [GENERAL] Lots of stuck queries after upgrade to 9.4

2015-07-20 Thread Andres Freund
Heikki, On 2015-07-20 13:27:12 +0200, Andres Freund wrote: > On 2015-07-20 13:22:42 +0200, Andres Freund wrote: > > Hm. The problem seems to be the WaitXLogInsertionsToFinish() call in > > XLogFlush(). > > These are the relevant stack traces: > db9lock/

Re: [GENERAL] Lots of stuck queries after upgrade to 9.4

2015-07-20 Thread Andres Freund
Hi, On 2015-07-20 15:58:33 +0300, Spiros Ioannou wrote: > Happened again, another backtrace from a COMMIT process. I changed the > commit_delay to 0 (it was 4000 to help with our storage) and will report > back. What hardware & OS is this happening on? Regards, Andres -- S

Re: [GENERAL] Lots of stuck queries after upgrade to 9.4

2015-07-20 Thread Andres Freund
On 2015-07-20 17:00:52 +0300, Spiros Ioannou wrote: > FYI we have an 9.3.5 with commit_delay = 4000 and commit_siblings = 5 with > a 8TB dataset which seems fine. (Runs on different - faster hardware > though). 9.4 has a different xlog insertion algorithm (scaling much better), so that unfortunate

Re: [GENERAL] Delete rule does not prevent truncate

2015-07-23 Thread Andres Freund
On 2015-07-23 12:57:20 +0100, Tim Smith wrote: > Thus, I should not have to use a trigger for TRUNCATE because the "each > row" concept does not apply. Plus it makes perfect sense to want to > transform the truncate command and transform into ignore That'd entirely defeat the point of TRUNCATE

Re: [GENERAL] Delete rule does not prevent truncate

2015-07-24 Thread Andres Freund
his thread die. Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Logical decoding off of a replica?

2015-07-30 Thread Andres Freund
recent progress towards allowing that, but I still consider it not to be too hard to implement. Regards, Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Grouping sets, cube and rollup

2015-08-26 Thread Andres Freund
i.e. you can't patch postgres and then continue with an existing data directory. - Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] bdr download

2015-08-31 Thread Andres Freund
this list is what to use for BDR reports > > and discussions, so this report is in the right place. > > Huh, why did we decide that when the community doesn't control any of > it? That doesn't make any sense. http://archives.postgresql.org/message-id/%2053A2AA64.9040709%4

Re: [GENERAL] bdr download

2015-08-31 Thread Andres Freund
On 2015-08-31 18:41:19 +0200, Andres Freund wrote: > http://archives.postgresql.org/message-id/%2053A2AA64.9040709%402ndquadrant.com http://archives.postgresql.org/message-id/53A2AA64.9040709%402ndquadrant.com There was a space too much in the link... Greetings, Andres Freund -- Sent

Re: [GENERAL] FDW and BDR

2015-09-02 Thread Andres Freund
ikely, nobody removed the error check. Either way it should be simple to implement. Greetings, Andres Freund -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] in defensive of zone_reclaim_mode on linux

2015-09-06 Thread Andres Freund
the negative impact of transparent hugepages being mitigated to some degree by zone reclaim mode (which'll avoid some cross-node transfers). Greetings, Andres Freund -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Drop or alter column under load give ERROR #42804 structure of query does not match function result type:

2015-10-12 Thread Andres Freund
n undesirable way. (In the case where our own xact * modifies the rel, the relcache update happens via * CommandCounterIncrement, not here.) */ if (res != LOCKACQUIRE_ALREADY_HELD) AcceptInvalidationMessages(); } I've not investigated what

Re: [GENERAL] Drop or alter column under load give ERROR #42804 structure of query does not match function result type:

2015-10-12 Thread Andres Freund
of the "users" type when the table is used, but there's no locking preventing the columns to be dropped while the function is used. So what happens is that 1) the function is parsed & planned 2) DROP COLUMN is executed 3) the contained statement is executed 4) a mismatch between

Re: [GENERAL] Drop or alter column under load give ERROR #42804 structure of query does not match function result type:

2015-10-12 Thread Andres Freund
;s[1] example, the above sequence of events does not fail. It > fails in Victors's case when the server is under load, so it seems there is > another factor in play. The above sequence is only problematic if 2) happens exactly between 1) and 3), which is not particularly likely given that 1)

Re: [GENERAL] BDR workers exiting?

2015-10-12 Thread Andres Freund
34532,1,""","" There'll possibly be an error message on the other node about ending the connection. Do you use SSL? If so, try disabling renegotiation. Regards, Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] ERROR: tablespace "archive2" is not empty

2015-10-19 Thread Andres Freund
g query. The files on disk are relefilenodes not oids. Try WHERE pg_relation_filenode(oid) IN ... Greetings, Andres Freund -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] postgresql 9.4 streaming replication

2015-10-19 Thread Andres Freund
> /var/log/standby.log' > primary_slot_name='standby1' pg_standby is for a "warm standby" - instead of signalling an error if an archive file does not exist it'll sleep. Thus this node will never enter streaming replication. Use cp or something instead. Gre

Re: [GENERAL] ERROR: tablespace "archive2" is not empty

2015-10-19 Thread Andres Freund
On 2015-10-19 11:14:33 +0200, Josip Rodin wrote: > On Mon, Oct 19, 2015 at 11:06:59AM +0200, Andres Freund wrote: > > Hi, > > > > On 2015-10-19 10:49:11 +0200, Josip Rodin wrote: > > > % sudo -H -u postgres psql mydb -c "SELECT oid, relname, relkind FROM >

Re: [GENERAL] what change in postgres 9.5 improvements for multi-CPU machines

2016-08-16 Thread Andres Freund
.3.5 。 Can it solve the spinlock problem, if > upgrade it to 9.5 ? It's quite possible that the upgrade helps. But without additional data it's hard to say. The change is that postgres internal reader/writer lock now, in many cases, avoid the use of a spinlock, relying on atomic

Re: [GENERAL] What limits Postgres performance when the whole database lives in cache?

2016-09-02 Thread Andres Freund
er outage and not lose all your data. > > Stonebraker's new stuff is cool, but it is NOT designed to survive > total power failure. > > Two totally different design concepts. It's apples and oranges to compare > them. I don't think they're that fundame

Re: [GENERAL] Nonblocking libpq + openssl = ?

2016-09-16 Thread Andres Freund
e dislike this bit, and I've complained about it before. But it is noteworthy that it's nearly impossible to hit these days, due to ssl-renegotiation support having been ripped out. That's what could trigger openssl to require writes upon reads. Greetings, Andres Freund -- Sent v

Re: [GENERAL] Nonblocking libpq + openssl = ?

2016-09-16 Thread Andres Freund
On 2016-09-17 03:12:53 +0300, Nikolai Zhubr wrote: > 17.09.2016 2:05, Andres Freund: > [...] > > Well, it's not pretty. I quite dislike this bit, and I've complained > > about it before. But it is noteworthy that it's nearly impossible to > > hit these

Re: [GENERAL] Lock contention in TransactionIdIsInProgress()

2016-10-08 Thread Andres Freund
On 2016-10-07 08:36:12 -0500, Merlin Moncure wrote: > Won't happen. Only bugs get ported back Hopefully we don't port bugs back all that often. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-ge

Re: [GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-10 Thread Andres Freund
emoryDetach is getting > called more than once, but I'm not sure how that would happen. Can you > characterize where this happens more precisely? What nondefault settings > have you got in postgresql.conf? Hm. Could that be from the DSM code? - Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] out-of-order XID insertion in KnownAssignedXids

2016-10-18 Thread Andres Freund
produce any kind of log mesages > / details that would be helpful. Could you use pg_xlogdump to dump the WAL file on which replay failed? And then attach the output in a compressed manner? Greetings, Andres Freund -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To m

Re: [GENERAL] out-of-order XID insertion in KnownAssignedXids

2016-10-20 Thread Andres Freund
to be ok as long as there's no checkpoints while taking the base backups (or when the control file was copied early enough). But as soon as a second checkpoint happens before the control file is copied... Fredrik, how did you end up removing the label? Greetings, Andres Freund -- Sent via p

Re: [GENERAL] hot_standby_feedback

2016-11-28 Thread Andres Freund
y, the only option is to cancel it. > > Is that explanation correct? No. That just means that we don't update the value more frequently. The value reported is a "horizon" meaning that nothing older than the reported value can be accessed. Greetings, Andres Freund --

Re: [GENERAL] Is is safe to use SPI in multiple threads?

2016-12-09 Thread Andres Freund
On 2016-12-09 16:52:05 +0800, Qiu Xiafei wrote: > 1. Is there a way to use SPI in multi-thread style? No. > 2. Another option is to use libpq, like normal clients do. Is libpq as > efficient as SPI? No. - Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org

Re: [GENERAL] logical decoding output plugin

2016-12-10 Thread Andres Freund
ack (no catalog access there), but you can do it the first time through the change callback. For lookups the most complete lookup is to use RangeVarGetRelid() to get the oid. There's other variants, but that's the easiest approach. Do you have to care about the table being renamed? G

Re: [GENERAL] Causeless CPU load waves in backend, on windows, 9.5.5 (EDB binary).

2017-01-31 Thread Andres Freund
stead of 9.5 and report back. > > Exactly same trouble with 9.4 (Specifically EDB 9.4.10-1 win32) That's good to know, less because of 519b0757, more because of the latch changes - but they also went in in 9.5... > Would it make sense to check some even older ones? Could you use pr

Re: [GENERAL] Causeless CPU load waves in backend, on windows, 9.5.5 (EDB binary).

2017-02-03 Thread Andres Freund
On 2017-02-01 01:02:11 +0300, Nikolai Zhubr wrote: > 31.01.2017 19:51, Andres Freund: > [...] > > > Exactly same trouble with 9.4 (Specifically EDB 9.4.10-1 win32) > > > > That's good to know, less because of 519b0757, more because of the latch > >

Re: [GENERAL] Causeless CPU load waves in backend, on windows, 9.5.5 (EDB binary).

2017-02-03 Thread Andres Freund
On 2017-02-03 22:17:55 +0300, Nikolai Zhubr wrote: > 03.02.2017 20:29, Andres Freund: > [...] > > > > Could you use process monitor or such to see what the process is doing > > > > while using a lot of CPU? > > > > > > I'm not sure how t

Re: [GENERAL] full_page_writes on SSD?

2015-11-24 Thread Andres Freund
On 2015-11-24 13:09:58 -0600, Kevin Grittner wrote: > On Tue, Nov 24, 2015 at 12:48 PM, Marcin Mańk wrote: > > > if SSDs have 4kB/8kB sectors, and we'd make the Postgres page > > size equal to the SSD page size, do we still need full_page_writes? > > If an OS write of the PostgreSQL page size ha

Re: [GENERAL] Old source code needed

2015-11-27 Thread Andres Freund
intended (expected) behavior or a compiller bug (Being on Gentoo, > compiller bug scary me a lot). This has been fixed later in the 9.0 branch = but as you want to checkout a specific tag, that's not goign to help you... What are you actually trying to do? Andres -- Sent via pgsql-gen

Re: [GENERAL] Shared system resources

2016-01-04 Thread Andres Freund
On 2015-12-23 11:55:36 -0500, George Neuner wrote: > With sufficient privileges, a debugger-like process can attach and > examine the memory of a running - or just terminated - process, but it > won't have access to discarded (unmapped) memory. You just have to load a kernel module to do so - it's

Re: [GENERAL] A motion

2016-01-23 Thread Andres Freund
On 2016-01-23 15:31:02 -0800, Joshua D. Drake wrote: > With respect Adrian, that is a motion that never stands a chance. If you > don't want to read it, set up a filter that sends it right to the round > file. It'd help if there weren't six, but one thread... -- Sent via pgsql-general mailing l

Re: [GENERAL] 2x Time difference between first and subsequent run of the same query on fresh established connection (on linux, with perf data included, all query data in the shared buffers) on postgre

2016-02-25 Thread Andres Freund
1,77% postgres [kernel.kallsyms] [k] page_add_file_rmap >1,66% postgres postgres [.] _bt_checkkeys >1,27% postgres postgres [.] LWLockRelease Looks like page faults are part of the proble here. Upgrade to 9.4 and use them unfortunately is the best suggest

Re: [GENERAL] Confusing with commit time usage in logical decoding

2016-03-02 Thread Andres Freund
commit_time returned was always > 0. > Could you help me by indicating me what could be wrong in my case? Any > missing parameters set? That was a bug introduced recently (9.5). The issue was discussed in http://archives.postgresql.org/message-id/56D42918.1010108%40postgrespro.ru , and a

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-16 Thread Andres Freund
ion like that if we made postmaster use a *different* pipe as stderr than the one we're handing to normal backends. If postmaster created a new pipe and closed the read end whenever forking a syslogger, we should get EPIPEs when writing after syslogger died and could fall back to proper stderr or

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-16 Thread Andres Freund
On 2017-11-17 11:09:56 +0900, Michael Paquier wrote: > On Fri, Nov 17, 2017 at 10:50 AM, Andres Freund wrote: > > On 2017-11-06 15:35:03 -0500, Tom Lane wrote: > >> David Pacheco writes: > >> > I ran into what appears to be a deadlock in the logging subsystem. It &

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-16 Thread Andres Freund
, which > I'm not quite sure of ... but it does seem like a situation that could > arise from time to time. Hm. I'm a bit scared about that - it doesn't seem that inconceivable that various backends log humongous multi-line messages, leading to syslogger *actually* taking up a

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-16 Thread Andres Freund
On November 16, 2017 7:06:23 PM PST, Tom Lane wrote: >Andres Freund writes: >> On 2017-11-16 21:39:49 -0500, Tom Lane wrote: >>> What might be worth thinking about is allowing the syslogger process >to >>> inherit the postmaster's OOM-kill-proofness settin

Re: [GENERAL] mild modification to pg_dump

2017-11-17 Thread Andres Freund
On 2017-11-17 18:56:45 -0300, marcelo wrote: > Truly, I'm catched in a very big app, so I have no time to read all > the docs. People on this list also have jobs. - Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscri

Re: [GENERAL] ERROR row is too big size 9336, exceeds size 8160 when populating record with tsquery and tsvector fields

2010-11-18 Thread Andres Freund
; restriction? Looks like you tried to index the text field itself with a btree? Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] slow building index and reference after Sybase to Pg

2011-02-28 Thread Andres Freund
this case. > maintenance_work_mem = 2GB 1GB is the max value taking effect. I would also suggest setting wal_sync_method=fdatasync You haven't configured shared_buffers at all? I would suggest setting it to 2GB or such. Andres -- Sent via pgsql-general mailing list (pgsql-general@p

Re: [GENERAL] slow building index and reference after Sybase to Pg

2011-02-28 Thread Andres Freund
ld be useful: * pg version * kernel version * distribution Andres -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Shared memory changes in 9.4?

2014-05-26 Thread Andres Freund
or. I still think dynamic_shared_memory_type should default to 'none' because of such problems. > Any idea > what may have changed? I suspect the solution here is tweaking some LXC > setting, but I was hoping someone can point me in the right direction. It sounds like you

Re: [GENERAL] Backups over slave instead master?

2014-05-29 Thread Andres Freund
bel a backup that's been created won't necessarily start up from the right point. >From 9.2. you can simply use pg_basebackup from standby servers though. That does all the necessary things internally. Greetings, Andres Freund -- Andres Freund http://

Re: [GENERAL] out of memory errors

2014-06-16 Thread Andres Freund
tio at all? Can you show /proc/meminfo's contents? My guess is that the CommitLimit is too low... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-general mailing list

Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column

2014-06-17 Thread Andres Freund
ecessary to > re-do the transfer of the existing dump file. Any chance there was windows in the mix somewhere? IIRC there were bugs with large dumps there for a while... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Su

Re: [GENERAL] Repeated semop calls

2014-06-27 Thread Andres Freund
ostgres with -fno-omit-frame-pointers and use perf record -g. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Bi-Directional replication client awareness

2014-07-12 Thread Andres Freund
MMIT has safely been replicated to other systems. That's supported by BDR today. The more complex thing is support for performing transactions that, when the COMMIT returns successfully, can't have conflicts. That's *not* yet supported, but we're thinking of implementing it

Re: [GENERAL] Bi-Directional replication client awareness

2014-07-12 Thread Andres Freund
On 2014-07-12 13:23:08 +0200, Martin Gudmundsson wrote: > > 12 jul 2014 kl. 12:33 skrev Andres Freund : > > > On 2014-07-12 18:22:30 +0800, Craig Ringer wrote: > >> On 07/12/2014 02:24 PM, Martin Gudmundsson wrote: > >>> Any ideas giving BDR an option

Re: [GENERAL] Bi-Directional replication client awareness

2014-07-12 Thread Andres Freund
or not. I'd like that too, but I don't think that's going to happen. There's just too many caveats (CAP theorem et al) to make that generally useful/applicable. I think we (as in postgres) will probably get the ability to run individual transactions in such a mode, but

Re: [GENERAL] We've been affected by a pg_upgrade bug. What do we do next?

2014-07-16 Thread Andres Freund
. What exactly is it whining about? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscript

Re: [GENERAL] WAL receive process dies

2014-08-29 Thread Andres Freund
ceive WAL from. Are you sure primary_conninfo / the spiped tunnel connects to the right postgres instance? Did you possibly have a failover or something like that? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Tr

Re: [GENERAL] WAL receive process dies

2014-08-29 Thread Andres Freund
* one of the above as it has a flush position of '174/B76D16A8' - not something that's really possible when the node actually is at '177/2E80C9F8'. Could you run, on the standby that's having problems, the following command: psql 'host=127.0.0.1 port=5445

Re: [GENERAL] BDR DML Only

2014-09-16 Thread Andres Freund
ally adding any missing DDL on the > "slave". As Christoph wrote this pretty much is already possible. Even if it could be prettier. What I'd like to know is what the exact use case you want it for is. Could you quickly explain? Greetings, Andres Freund -- Andres Freu

Re: [GENERAL] BDR Error recovery

2014-09-16 Thread Andres Freund
he UI for it better. All you can currently do is to skip the transaction doing the DDL using pg_replication_identifier_advance(). Which requires some fiddling around. It's on our roadmap to make a) skipping the replication of a transaction easier b) allow to skip individual problematic actio

Re: [GENERAL] BDR DML Only

2014-09-16 Thread Andres Freund
anges with 1 column, others with 2. In neither case it's clear how to replicate these. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-general mailing list (p

Re: [GENERAL] BDR DML Only

2014-09-16 Thread Andres Freund
bdr-pg/REL9_4_STABLE) for bdr 9.4. For which no instructions about compiling exist yet :( > But I will keep on testing because the end result will be very valuable. Cool! Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [GENERAL] pg_multixact issues

2014-09-17 Thread Andres Freund
On 2014-09-17 17:46:05 +0530, Dev Kumkar wrote: > On one my machine the pg_multixact directory size has grown up to 5 GB and > am not sure how to clean up this directory. Which version of postgres are you using? Greetings, Andres Freund -- Andres Freund http

Re: [GENERAL] [SQL] pg_multixact issues

2014-09-18 Thread Andres Freund
imits would help in cleaning up > pg_multixact? Can you show pg_controldata output and the output of 'SELECT oid, datname, relfrozenxid, age(relfrozenxid), relminmxid FROM pg_database;'? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ Pos

Re: [GENERAL] [SQL] pg_multixact issues

2014-09-19 Thread Andres Freund
On 2014-09-18 22:52:57 +0530, Dev Kumkar wrote: > On Thu, Sep 18, 2014 at 6:20 PM, Dev Kumkar wrote: > > > On Thu, Sep 18, 2014 at 4:03 PM, Andres Freund > > wrote: > > > >> I don't think that's relevant for you. > >> > >> Did y

Re: [GENERAL] fmgr_oldstyle in extensions

2014-09-23 Thread Andres Freund
argument, and from there things naturally stop working. > > So, I’m wondering what I could have done that is causing my calls to route > through fmgr_oldstyle instead of the usual path? You forgot to add a PG_FUNCTION_INFO_V1(yourfunc); for the function. I really, really think

Re: [GENERAL] BDR Global Sequences

2014-09-30 Thread Andres Freund
ad 100M rows now so > you should get ready for that."? Not yet, but we're planning to add that. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-general mailin

Re: [GENERAL] Integrating C++ singletons into postgresql extensions???

2014-10-18 Thread Andres Freund
ost appropriate to map the logging onto postgres' internal logging. That should easily be possible. The most important thing to be aware of here is that you need to take *great* care that exception, *NEVER* permeate through postgres code. Doing so will cause bad things. Greetings, Andres Fr

Re: [GENERAL] pg killed by oom-killer, "invalid contrecord length 2190 at A6C/331AAA90" on slaves

2014-10-28 Thread Andres Freund
d 9.3.5 on the slave that doesn't have the problem. Is this > something that was fixed in 9.3.5? We have really no information to answer that question accurately. So you really need to provide logs and such. Greetings, Andres Freund -- Andres Freund http://www.

  1   2   3   >