Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

was indeed fairly easy. > > I got stalled when I feature-crept myself by realizing I wanted it > snapshotted to WAL so it could fix the PITR and replication issues. And > then properly bogged down when I realized that on the standby I'd want > *both* the stats from the standby (while i

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

- it's file backed mmap, rather than anonymous mmap. To my knowledge there's no good portable way to use anonymous mmap to share memory across processes unless established before a fork(). > If so, why isn't choose_dsm_implementation() trying it; and if not, > why are we carrying it? I thin

Re: [HACKERS] Orphaned files in base/[oid]

, seek out valid files, and delete the rest? I think the fix here is to call RemovePgTempFiles() during crash-restarts, instead of just full starts. The previously stated need to be able to inspect temp files after a crash can be less impactfully fulfilled with restart_after_crash = false. Greetings, Andr

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

On 2017-08-14 12:21:30 -0400, Robert Haas wrote: > >> ... If somebody has a system where no other form of shared > >> memory, works dynamic_shared_memory_type = mmap is still a thing, so > >> the use case for "none" seems very thin indeed. I'd vote for just > >> ripping it out in v11. > > > >

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

Hi, On 2017-08-14 11:46:10 -0400, Robert Haas wrote: > On Sun, Aug 13, 2017 at 9:59 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Andres Freund <and...@anarazel.de> writes: > >> You both are obviously right. Another point of potential concern could > >> be

[HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

think it'd be pretty interesting to look at replacing parts of the > >>> stats collector machinery with something DHT-based. > > On Sat, Aug 12, 2017 at 11:30 PM, Andres Freund <and...@anarazel.de> wrote: > >> That seems to involve a lot more than this though, given that cu

Re: [HACKERS] POC: Sharing record typmods between backends

Hi, On 2017-08-11 20:39:13 +1200, Thomas Munro wrote: > Please find attached a new patch series. Review for 0001: I think you made a few long lines even longer, like: @@ -1106,11 +1106,11 @@ pltcl_trigger_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,

Re: [HACKERS] Patches I'm thinking of pushing shortly

On 2017-08-13 17:43:10 -0400, Robert Haas wrote: > On Sun, Aug 13, 2017 at 5:24 PM, Tom Lane wrote: > >> I'd vote for including this in v10. There doesn't seem to be any > >> downside to this: it's a no brainer to avoid our exploding hash table > >> case when we can see it

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

not good from an interruptability perspective, but better than blocking for the entire connection establishment. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

Hi, On 2017-08-11 18:11:03 +0530, Augustine, Jobin wrote: > Appears that patch is not helping. > Errors like below are still appearing in the log > === > 2017-08-11 12:22:35 UTC [2840]: [1-1] user=,db=,app=,client= FATAL: could > not connect to the primary server:

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

On 2017-08-11 20:56:22 -0700, Noah Misch wrote: > > > If nobody volunteers, you could always resolve this by reverting 1e8a850 > > > and > > > successors. > > > > I think you're blaming the victim. Our current theory about the cause > > of this is that on Windows, WaitLatchOrSocket cannot be

Re: [HACKERS] POC: Sharing record typmods between backends

On 2017-08-12 22:52:57 -0400, Robert Haas wrote: > On Fri, Aug 11, 2017 at 9:55 PM, Andres Freund <and...@anarazel.de> wrote: > > Well, most of the potential usecases for dsmhash I've heard about so > > far, don't actually benefit much from incremental growth. In nearly all &g

Re: [HACKERS] POC: Sharing record typmods between backends

do a pass through your your submission... > On Tue, Jul 25, 2017 at 10:09 PM, Andres Freund <and...@anarazel.de> wrote: > > It does concern me that we're growing yet another somewhat different > > hashtable implementation. Yet I don't quite see how we could avoid > > it

Re: [HACKERS] POC: Sharing record typmods between backends

On 2017-08-11 11:14:44 -0400, Robert Haas wrote: > On Fri, Aug 11, 2017 at 4:39 AM, Thomas Munro > wrote: > > OK. Now it's ds_hash_table.{c,h}, where "ds" stands for "dynamic > > shared". Better? If we were to do other data structures in DSA > > memory they could

Re: [HACKERS] WIP: Failover Slots

bably a lot closer to zero than craig is. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

Hi, On 2017-07-21 19:32:02 +0300, Marina Polyakova wrote: > Here is the third version of the patch for pgbench thanks to Fabien Coelho > comments. As in the previous one, transactions with serialization and > deadlock failures are rolled back and retried until they end successfully or > their

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

On 2017-08-10 18:51:07 -0700, Noah Misch wrote: > On Tue, Aug 08, 2017 at 07:25:37PM -0400, Peter Eisentraut wrote: > > On 8/7/17 21:06, Noah Misch wrote: > > >> That would fit. Until v10 (commit 1e8a850), PQconnectStart() had no > > >> in-tree > > >> callers outside of libpq itself. > > >

Re: [HACKERS] Thoughts on unit testing?

(like we already kinda have for EXEC_BACKEND style subprocesses), that loads a shared library and invokes an entry point in it. That'd then be invoked at some defined point in time. Alternatively you could just have a bgworker that does its thing at startup and then shuts down the cluster... Greetings,

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

On 2017-08-08 19:25:37 -0400, Peter Eisentraut wrote: > On 8/7/17 21:06, Noah Misch wrote: > >> That would fit. Until v10 (commit 1e8a850), PQconnectStart() had no > >> in-tree > >> callers outside of libpq itself. > > [Action required within three days. This is a generic notification.] > > >

Re: [HACKERS] Error : undefined symbol : LWLockAssign in 9.6.3

've CCed him, maybe he can comment on his plans around imcs. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Infrastructure for JIT compiling tuple deforming

Thoughts? Better ideas? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

Hi, On 2017-08-07 17:30:13 -0400, Tom Lane wrote: > Meh. The lack of field complaints about this doesn't indicate to me that > we have a huge problem, and in any case, just increasing NUM_RESERVED_FDS > would do nothing for the system-wide limits. Howso? Via count_usable_fds() we test for

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

On 2017-08-07 17:05:06 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-08-07 16:52:42 -0400, Tom Lane wrote: > >> No, I don't think so. If you're depending on the NUM_RESERVED_FDS > >> headroom for anything meaningful, *yo

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

On 2017-08-07 16:52:42 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > These days there's a number of other consumers of > > fds. E.g. postgres_fdw, epoll, ... All these aren't accounted for by > > fd.c. > > > Given how close max_files

[HACKERS] max_files_per_processes vs others uses of file descriptors

of fds. E.g. postgres_fdw, epoll, ... All these aren't accounted for by fd.c. Given how close max_files_per_process is to the default linux limit of 1024 fds, I wonder if we shouldn't increase NUM_RESERVED_FDS by quite a bit? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list

Re: [HACKERS] snapbuild woes

On 2017-06-09 09:25:34 +0200, Antonin Houska wrote: > Andres Freund <and...@anarazel.de> wrote: > > > Looking at 0001: > > - GetOldestSafeDecodingTransactionId() only guarantees to return an xid > > safe for decoding (note how procArray->replication_slot_catalo

Re: [HACKERS] Draft release notes up for review

Hi Tom, On 2017-08-04 18:41:12 -0400, Tom Lane wrote: > I've committed the first-draft release notes for 9.6.4 at > https://git.postgresql.org/pg/commitdiff/03378c4da598840b0520a53580dd7713c95f21c8 I just pushed a 9.4 specific bugfix. Do you want me to fix up the release notes after you

Re: [HACKERS] PostgreSQL not setting OpenSSL session id context?

h backend has it's own local memory. I.e. any cache state that openssl would maintain wouldn't be useful. If you want to take advantage of features around this you really need to cache tickets in shared memory... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] Hash Functions

On 2017-08-03 17:57:37 -0400, Robert Haas wrote: > On Thu, Aug 3, 2017 at 5:50 PM, Andres Freund <and...@anarazel.de> wrote: > > On 2017-08-03 17:43:44 -0400, Robert Haas wrote: > >> For me, the basic point here is that we need a set of hash functions > >> for has

Re: [HACKERS] Hash Functions

ubt we want to end up with different hashfunctions for sharding, partitioning, hashjoins (which seems to form a hierarchy). Having a working hash-combine function, or even better a hash API that can continue to use the hash's internal state, seems a more scalable solution. Greetings, Andres Freund

Re: [HACKERS] Hash Functions

Hi, On 2017-08-03 17:09:41 -0400, Robert Haas wrote: > On Thu, Jun 1, 2017 at 2:25 PM, Andres Freund <and...@anarazel.de> wrote: > > Just to clarify: I don't think it's a problem to do so for integers and > > most other simple scalar types. There's plenty hash algorithms t

Re: [HACKERS] Tightly packing expressions

Hi Doug, On 2017-08-03 18:01:06 +, Douglas Doole wrote: > Back when you were getting ready to commit your faster expressions, I > volunteered to look at switching from an array of fixed sized steps to > tightly packed structures. ( >

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On 2017-02-10 07:52:57 -0500, Robert Haas wrote: > On Thu, Feb 9, 2017 at 6:38 PM, Thomas Munro > > Up until two minutes ago I assumed that policy would leave only two > > possibilities: you attach to the DSM segment and attach to the > > SharedBufFileManager successfully or you attach to the DSM

Re: [HACKERS] Why does logical replication launcher exit with exit code 1?

On 2017-08-02 17:09:10 -0400, Tom Lane wrote: > Even if we fix that, though, I think it is reasonable to downgrade it to > DEBUG1. We did that already for other standard background processes such > as the autovac launcher, and it's not apparent to me why the logrep > launcher should be chattier.

Re: [HACKERS] Why does logical replication launcher exit with exit code 1?

On 2017-08-02 16:52:01 -0400, Robert Haas wrote: > On Wed, Aug 2, 2017 at 4:38 PM, Peter Eisentraut > wrote: > > Maybe it doesn't need to be logged at all (other than perhaps as DEBUG)? > > A few months ago, people were complaining about too many messages about

Re: [HACKERS] POC: Cache data in GetSnapshotData()

Hi, I think this patch should have a "cover letter" explaining what it's trying to achieve, how it does so and why it's safe/correct. I think it'd also be good to try to show some of the worst cases of this patch (i.e. where the caching only adds overhead, but never gets used), and some of the

Re: [HACKERS] Why does logical replication launcher exit with exit code 1?

On 2017-08-01 20:37:07 -0400, Robert Haas wrote: > On Tue, Aug 1, 2017 at 7:03 PM, Andres Freund <and...@anarazel.de> wrote: > > On 2017-08-02 10:58:32 +1200, Thomas Munro wrote: > >> When I shut down a cluster that isn't using logical replication, it > >> alwa

Re: [HACKERS] Why does logical replication launcher exit with exit code 1?

On 2017-08-02 12:14:18 +1200, Thomas Munro wrote: > On Wed, Aug 2, 2017 at 11:03 AM, Andres Freund <and...@anarazel.de> wrote: > > On 2017-08-02 10:58:32 +1200, Thomas Munro wrote: > >> When I shut down a cluster that isn't using logical replication, it > >> alwa

Re: [HACKERS] Why does logical replication launcher exit with exit code 1?

On 2017-08-02 11:19:39 +1200, Gavin Flower wrote: > Returning zero to indicate success is a holdover to the time computers could > only run one program at a time. At the end of the code there was a jump > table of 4 byte entries. The first entry with a displacement of zero was > the location to

Re: [HACKERS] Why does logical replication launcher exit with exit code 1?

Hi, On 2017-08-02 10:58:32 +1200, Thomas Munro wrote: > When I shut down a cluster that isn't using logical replication, it > always logs a line like the following. So do the build farm members I > looked at. I didn't see anything about this in the open items list -- > isn't it a bug? > >

Re: [HACKERS] [PATCH] A hook for session start

On 2017-08-01 15:37:40 -0400, Peter Eisentraut wrote: > On 7/21/17 12:59, Robert Haas wrote: > > That's an exceedingly-weak argument for rejecting this patch. The > > fact that you can probably hack around the lack of a hook for most > > reasonable use cases is not an argument for having a hook

Re: [HACKERS] Better way to handle suppression of CASCADE detail messages

On 2017-08-01 13:48:34 -0400, Robert Haas wrote: > On Tue, Aug 1, 2017 at 1:39 PM, Andres Freund <and...@anarazel.de> wrote: > > Oid is probably not good enough - with parallel tests and such it's not > > necessarily predicable. Even less so when the tests are run against an

Re: [HACKERS] Better way to handle suppression of CASCADE detail messages

such it's not necessarily predicable. Even less so when the tests are run against an existing cluster. Sorting by name would probably be better... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] [PATCH v3] pg_progress() SQL function to monitor progression of long running SQL queries/utilities

On 2017-08-01 19:11:55 +0200, Michael Paquier wrote: > I think that Depesz makes use of a non-default format for its > explain.depesz.com, or he would have a hard time maintaining a > deparsing API for its application. Hm? e.d.c accepts the text explain format, so I'm unclear on what you're

[HACKERS] Re: [BUGS] BUG #14758: Segfault with logical replication on a function index

On 2017-07-31 09:40:34 +0900, Masahiko Sawada wrote: > Moved to -hackers. > > On Sat, Jul 29, 2017 at 4:35 AM, Scott Milliken wrote: > > Thank you Masahiko! I've tested and confirmed that this patch fixes the > > problem. > > > > Thank you for the testing. This issue should

Re: [HACKERS] Parallel Hash take II

From: Thomas Munro Date: Wed 26 Jul 2017 19:58:20 NZST Subject: [PATCH] Add support for parallel-aware hash joins. Hi, WRT the main patch: - Echoing concerns from other threads (Robert: ping): I'm doubtful that it makes sense to size the number of parallel

Re: [HACKERS] Parallel Hash take II

ut modelling the cost of shared memory access didn't > work out. Hm. You say, "didn't work out" - could you expand a bit on that? I'm quite doubtful that justaccounting for barriers will be good enough. > I'll report on performance separately. Looking forward to that ;) Greetin

Re: [HACKERS] POC: Sharing record typmods between backends

Hi, diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index 9fd7b4e019b..97c0125a4ba 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -337,17 +337,75 @@ DecrTupleDescRefCount(TupleDesc tupdesc) {

Re: [HACKERS] Persistent wait event sets and socket changes

documenting it. It doesn't work, right. I'm not sure I see the point of adding comments explaining that a nonexistant interface doesn't exist? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] segfault in HEAD when too many nested functions call

Hi, On 2017-07-29 16:14:08 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > [ 0002-Move-ExecProcNode-from-dispatch-to-function-pointer-.patch ] > > Here's a reviewed version of this patch. Thanks! I pushed both now. > I added dummy E

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-29 14:20:32 -0400, Tom Lane wrote: > Here's a reviewed version of this patch. Thanks! > * I think you put ExecScan's CFI in the wrong place; AFAICT yours > only covers its fast path. Sure - but the old path already has a CFI? And it has to be inside the loop, because well, the loop

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-28 09:29:58 -0700, Noah Misch wrote: > On Thu, Jul 27, 2017 at 10:08:57PM -0700, Andres Freund wrote: > > For me that means the policy isn't quite right. It's not like I can > > force Tom to review the patch at a specific date. But the thread has > > been prog

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-27 22:04:59 -0700, Noah Misch wrote: > On Thu, Jul 27, 2017 at 09:49:18PM -0700, Andres Freund wrote: > > On 2017-07-27 21:46:57 -0700, Noah Misch wrote: > > > On Thu, Jul 27, 2017 at 02:29:32AM +, Noah Misch wrote: > > > > On Mon, Jul 24, 2017 at 0

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-27 21:46:57 -0700, Noah Misch wrote: > On Thu, Jul 27, 2017 at 02:29:32AM +, Noah Misch wrote: > > On Mon, Jul 24, 2017 at 08:04:30AM +0100, Andres Freund wrote: > > > > > > > > > On July 24, 2017 7:10:19 AM GMT+01:00, Noah Misch <n...@le

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-26 16:28:38 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-07-26 15:03:37 -0400, Tom Lane wrote: > >> Hm, that seems kinda backwards to me; I was envisioning the checks > >> moving to the callees not the callers. I t

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-26 15:03:37 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > I've moved the CHECK_FOR_INTERRUPTS() to the callsites. That > > unsurprisingly ends up being somewhat verbose, and there's a bunch of > > minor judgement calls where exactly

Re: [HACKERS] segfault in HEAD when too many nested functions call

Hi, On 2017-07-24 13:27:58 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > >> * I think the comments need more work. Am willing to make a pass over > >> that if you want. > > > That'd be good, but let's wait till we have something mor

Re: [HACKERS] Create language syntax is not proper in pg_dumpall and not working using pg_upgrade

On 2017-07-25 13:18:25 -0400, Robert Haas wrote: > On Tue, Jul 25, 2017 at 1:13 PM, Andres Freund <and...@anarazel.de> wrote: > > On 2017-07-25 13:10:11 -0400, Robert Haas wrote: > >> On Tue, Jul 25, 2017 at 1:06 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >>

Re: [HACKERS] Create language syntax is not proper in pg_dumpall and not working using pg_upgrade

On 2017-07-25 13:10:11 -0400, Robert Haas wrote: > On Tue, Jul 25, 2017 at 1:06 PM, Tom Lane wrote: > >> Is this assumption, like, documented someplace? > > > > Uh, right there? > > I don't think we can expect end-users to read the code comments to > determine whether their

Re: [HACKERS] POC: Sharing record typmods between backends

On 2017-07-10 21:39:09 +1200, Thomas Munro wrote: > Here's a new version that introduces a per-session DSM segment to hold > the shared record typmod registry (and maybe more things later). You like to switch it up. *.patchset.tgz??? ;) It does concern me that we're growing yet another somewhat

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-24 13:27:58 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > >> I seriously, seriously, seriously dislike that. You practically might as > >> well put miscadmin.h into postgres.h. Instead, what do you think of > >> requiring t

Re: [HACKERS] segfault in HEAD when too many nested functions call

Hi, On 2017-07-21 20:17:54 -0400, Tom Lane wrote: > > I dislike having the miscadmin.h include in executor.h - but I don't > > quite see a better alternative. > > I seriously, seriously, seriously dislike that. You practically might as > well put miscadmin.h into postgres.h. Instead, what do

Re: [HACKERS] segfault in HEAD when too many nested functions call

On July 24, 2017 7:10:19 AM GMT+01:00, Noah Misch <n...@leadboat.com> wrote: >On Tue, Jul 18, 2017 at 01:04:10PM -0700, Andres Freund wrote: >> Ok, I'll flesh out the patch till Thursday. But I do think we're >going >> to have to do something about the back branches, to

Re: [HACKERS] [COMMITTERS] pgsql: Add a Gather executor node.

On July 21, 2017 5:34:32 PM GMT+02:00, Alvaro Herrera wrote: >Robert Haas wrote: > >> I think those top-of-file lists are just annoying, and would prefer >to >> rip them out entirely rather than spend time maintaining them. > >+1 I'm quite sympathetic to that view.

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

Hi, (please don't top-reply on this list) On 2017-07-19 14:04:39 +0900, Yugo Nagata wrote: > On Tue, 18 Jul 2017 10:10:49 -0400 > Tom Lane wrote: > > Thank you for your comments. I understand the problem of my proposal > patch. Does that mean you're trying to rewrite it in

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-18 16:53:43 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > ... If we were to go this route we'd have to probably move > > the callback assignment into the ExecInit* routines, and possibly > > replace the switch in ExecInitNode() with

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-18 15:45:53 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > Attached is a trivial patch implementing 1) and a proof-of-concept hack > > for 2). > > The comment you made previously, as well as the proposed commit message > for 0001,

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

es in PGXACT or such, without any locking, and send it only to those that have outstanding ones. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

Hi, On 2017-07-18 21:49:27 +0900, Yugo Nagata wrote: > After a DML is perfomed, the statistics is sent to pgstat by > pgsent_report_sent(). > However, the mininum time between stas file update is PGSTAT_STAT_INTERVAL, > so if > a few DMLs are performed with short interval, some statistics could

Re: [HACKERS] segfault in HEAD when too many nested functions call

On 2017-07-17 23:04:43 +0200, Julien Rouhaud wrote: > On 17/07/2017 16:57, Andres Freund wrote: > > The latter obviously isn't ready, but might make clearer what I'm > > thinking about. > > It did for me, and FWIW I like this approach. Cool. > > If we were

Re: [HACKERS] segfault in HEAD when too many nested functions call

Hi, On 2017-07-17 00:26:29 -0700, Andres Freund wrote: > I'm less convinced of that, due to the overhead argument. I think > there's a couple ways around that however: > > 1) We could move ExecProcNode() to be callback based. The first time a >node is executed a "wrapper

Re: [HACKERS] segfault in HEAD when too many nested functions call

s are similar enough to ExecProcNode()'s allowing us to put a check_stack_depth() into the former. That seems like it's required anyway, since in many cases that's going to trigger stack-depth exhaustion first anyway (unless we hit it in parse analysis, which also seems quite common).

Re: [HACKERS] building libpq.a static library

On 2017-07-12 23:55:56 +0100, Greg Stark wrote: > Fwiw I think the real problem is that building static libraries > "properly" requires different compiler options -- notably they're not > normally built with -fPIC. So that means building every object twice > which kind of breaks make's build model

Re: [HACKERS] Rust bindings to pgtypes lib

On 2017-07-07 12:54:33 +0200, Michael Meskes wrote: > > Some people use http://libpqtypes.esilo.com/ > > Never before saw this. It does not seem to have more in common than the > name, though. It has binary to text conversion functionality for various types. I don't exactly know what Kaare

Re: [HACKERS] Rust bindings to pgtypes lib

ch of the source tree would I have to carve out? > > Or from another perspective; how do other language (if any) solve this? > > I'm not aware of any other language binding for pgtypeslib. Some people use http://libpqtypes.esilo.com/ Greetings, Andres Freund -- Sent via pgsql-ha

Re: [BUGS] [HACKERS] Segmentation fault in libpq

Hi, On 2017-07-02 20:58:52 +0200, Michal Novotný wrote: > thank you all for your advice. I've been investigating this a little more > and finally it turned out it's not a bug in libpq although I got confused > by going deep as several libpq functions. The bug was really on our side > after trying

Re: [HACKERS] Apparent walsender bug triggered by logical replication

y send back something once a transaction is complete, and until that happens we'll just block waiting for more WAL. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reducing pg_ctl's reaction time

On June 29, 2017 10:19:46 AM PDT, Jeff Janes <jeff.ja...@gmail.com> wrote: >On Tue, Jun 27, 2017 at 11:59 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> I wrote: >> > Andres Freund <and...@anarazel.de> writes: >> >> On 2017-06-26 17:38:03 -0400,

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

/include/pg_config.h.in > +++ b/src/include/pg_config.h.in > @@ -167,6 +167,9 @@ > /* Define to 1 if you have the header file. */ > #undef HAVE_EDITLINE_READLINE_H > > +/* Define to 1 if you have the `fallocate' function. */ > +#undef HAVE_FALLOCATE > + > /* Define to 1 if you have the `fdatasync' function. */ > #undef HAVE_FDATASYNC Needs pg_config.h.win32 adjustment... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reducing pg_ctl's reaction time

On 2017-06-28 13:31:27 -0400, Tom Lane wrote: > I'm not hearing anyone speaking against doing this now, so I'm going > to go ahead with it. Cool. > While looking this over again, I got worried about the fact that pg_ctl > is #including "miscadmin.h". That's a pretty low-level backend header >

Re: [HACKERS] Fast promotion not used when doing a recovery_target PITR restore?

On 2017-06-28 06:04:23 +0900, Michael Paquier wrote: > On Wed, Jun 28, 2017 at 3:44 AM, Andres Freund <and...@anarazel.de> wrote: > > I'm far from convinced by this. By now WAL replay with checkpointer, > > bgwriter, etc. active is actually *more* tested t

Re: [HACKERS] Reducing pg_ctl's reaction time

tptr++ = '\n'; > + } > + > + /* >* Write or rewrite the target line. >*/ > snprintf(destptr, destbuffer + sizeof(destbuffer) - destptr, > "%s\n", str); Not this patches fault, and shouldn't be changed now, but this is a fairly weird way to manage and update this file. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] memory layouts for binary search in nbtree

On 2017-06-27 11:13:38 -0700, Peter Geoghegan wrote: > On Tue, Jun 27, 2017 at 11:04 AM, Andres Freund <and...@anarazel.de> wrote: > > > > On 2017-06-27 10:57:15 -0700, Peter Geoghegan wrote: > >> I looked at this again recently. I wrote a patch to prove to myself

Re: [HACKERS] Fast promotion not used when doing a recovery_target PITR restore?

On 2017-06-23 10:56:07 +0900, Michael Paquier wrote: > > And even there it might actually be > > a pretty good idea to not force a full checkpoint - getting up fast > > after a crash is kinda important.. > > But not that. Crash recovery is designed to be simple and robust, with > only the

Re: [HACKERS] memory layouts for binary search in nbtree

On 2016-05-19 19:38:02 -0700, Peter Geoghegan wrote: > On Wed, May 18, 2016 at 6:25 AM, Andres Freund <and...@anarazel.de> wrote: > > currently we IIRC use linearly sorted datums for the search in > > individual btree nodes. Not surprisingly that's often one of the

Re: [HACKERS] memory layouts for binary search in nbtree

significant share of the time, but that's easy enough with index nested loop joins and such. I do recall seeing it recently-ish in a number of tpc-h queries. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Reducing pg_ctl's reaction time

On 2017-06-26 17:38:03 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-06-26 17:30:30 -0400, Tom Lane wrote: > >> No, I don't like that at all. Has race conditions against updates > >> coming from the startup process. &g

Re: [HACKERS] Reducing pg_ctl's reaction time

On 2017-06-26 17:30:30 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > It'd be quite possible to address the race-condition by moving the > > updating of the control file to postmaster, to the > > CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STA

Re: [HACKERS] Reducing pg_ctl's reaction time

Hi, On 2017-06-26 16:49:07 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > Arguably we could and should improve the logic when the server has > > started, right now it's pretty messy because we never treat a standby as > > up if hot_stand

Re: [HACKERS] Reducing pg_ctl's reaction time

On 2017-06-26 16:26:00 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-06-26 16:19:16 -0400, Tom Lane wrote: > >> Sure, what do you think an appropriate behavior would be? > > > It'd not be unreasonble to check pg_control first,

Re: [HACKERS] Reducing pg_ctl's reaction time

On 2017-06-26 16:19:16 -0400, Tom Lane wrote: > Jeff Janes writes: > > The 10 fold increase in log spam during long PITR recoveries is a bit > > unfortunate. > > > 9153 2017-06-26 12:55:40.243 PDT FATAL: the database system is starting up > > 9154 2017-06-26 12:55:40.345

Re: [HACKERS] Another reason why the recovery tests take a long time

On 2017-06-26 13:42:52 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-06-26 12:32:00 -0400, Tom Lane wrote: > >> ... But I wonder whether it's intentional that the old > >> walreceiver dies in the first place. That FATAL exit looks

Re: [HACKERS] Another reason why the recovery tests take a long time

Hi, On 2017-06-26 12:32:00 -0400, Tom Lane wrote: > I've found another edge-case bug through investigation of unexpectedly > slow recovery test runs. It goes like this: > > * While streaming from master to slave, test script shuts down master > while slave is left running. We soon restart the

Re: [HACKERS] subscription worker signalling wal writer too much

e > anyway. s/completely new patch/completely new page/? In my opinion we actually *do* want to write (but not flush!) out such pages, so I'm not sure I agree with that logic. Have to think about this some more... Greetings, Andres Freund >From 50a3e62b16b752837f5a388e3259fee2bca6c2

Re: [HACKERS] Logical replication: stuck spinlock at ReplicationSlotRelease

No idea why I've done it like that, makes no sense. Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix a typo in snapmgr.c

On 2017-06-23 19:21:57 +0100, Simon Riggs wrote: > On 23 June 2017 at 08:23, Simon Riggs <si...@2ndquadrant.com> wrote: > > On 23 June 2017 at 08:21, Andres Freund <and...@anarazel.de> wrote: > >> On 2017-06-07 10:17:31 -0700, Andres Freund wrote: > >>> On

Re: [HACKERS] REPLICA IDENTITY FULL

On 2017-06-23 13:05:21 -0400, Alvaro Herrera wrote: > Tom Lane wrote: > > Peter Eisentraut writes: > > > Any thoughts about keeping datumAsEqual() as a first check? I did some > > > light performance tests, but it was inconclusive. > > > > Seems like it would

Re: [HACKERS] Fix a typo in snapmgr.c

On 2017-06-07 10:17:31 -0700, Andres Freund wrote: > On 2017-05-08 09:12:13 -0400, Tom Lane wrote: > > Simon Riggs <si...@2ndquadrant.com> writes: > > > So rearranged code a little to keep it lean. > > > > Didn't you break it with that? As it now stands,

[HACKERS] Dynamic instrumentation of lwlock wait times (lwlock flamegraphs)

tack - ... I think it might be interesting to collect a few of these somewhere centrally once halfway mature. Maybe in src/tools or such. Greetings, Andres Freund [1] https://github.com/iovisor/bcc #!/usr/bin/python from __future__ import print_function from bcc import BPF from time import s

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

On 2017-06-22 10:41:41 -0700, Andres Freund wrote: > On 2017-02-02 12:18:22 -0800, Jimmy Yih wrote: > > In the above pull request, Heikki also mentions that a similar scenario can > > happen during palloc() as well... which is similar to what we saw in > > Greenplu

<    1   2   3   4   5   6   7   8   9   10   >