Re: [HACKERS] Restricting maximum keep segments by repslots

sed even after we purge from local pg_xlog, albeit at a performance cost. I'm thinking of this mainly for logical slots. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] ArrayLists instead of List (for some things)

On 3 November 2017 at 12:41, David Rowley <david.row...@2ndquadrant.com> wrote: > On 3 November 2017 at 03:26, Craig Ringer <cr...@2ndquadrant.com> wrote: >> On 2 November 2017 at 22:22, David Rowley <david.row...@2ndquadrant.com> >> wrote: >>> Maybe, but

Re: [HACKERS] SSL and Encryption

This is probably off topic for pgsql-hackers. For password crypto please go read the SCRAM thread and the PostgreSQL 10 release notes. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing lis

Re: [HACKERS] Client Connection redirection support for PostgreSQL

n doesn't have your prepared statements, any WITH HOLD cursors, temp tables, etc you were working with. Strangeness ensues. But we now have a session-intent stuff though. So we could possibly do it at session level. Backends used just for a redirect would be pretty expensive though. -- Craig R

Re: [HACKERS] proposal: schema variables

e-id/flat/CAMsr%2BYF0G8_FehQyFS8gSfnEer9OPsMOvpfniDJOVGQzJzHzsw%40mail.gmail.com#camsr+yf0g8_fehqyfs8gsfneer9opsmovpfnidjovgqzjzh...@mail.gmail.com It'd be nice if you summarised any outcomes from that and addressed it, rather than taking this as a new topic. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL De

Re: [HACKERS] Custom compression methods

some concerns about what happened with pg_upgrade, with consuming precious toast bits, and a few other things. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

Re: [HACKERS] ArrayLists instead of List (for some things)

th ADTs offers multiple implementations of list data structures, often wrapped with a common API. Java's Collections, the STL, you name it. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers

Re: [HACKERS] ArrayLists instead of List (for some things)

jacked up the List API and driven a new implementation underneath > once before. Maybe it's time to do that again. I know some systems use hybrid linked array-lists, where linked list cells are multi-element. https://en.wikipedia.org/wiki/Unrolled_linked_list I don't have much experience with the

Re: [HACKERS] Mapping MERGE onto CTEs (Re: MERGE SQL Statement for PG11)

use functions that aren't immutable, > they may be considered for inlining. > https://www.postgresql.org/message-id/5351711493487...@web53g.yandex.ru Yep. All theoretical though, I don't think anyone (myself included) stumped up a patch. -- Craig Ringer http://www.

Re: [HACKERS] Statement-level rollback

og spam too. And unlike some of the prior protocol tweaks I've been interested in, it'd be client-initiated so it should be pretty safe. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers maili

Re: [HACKERS] MERGE SQL Statement for PG11

atement would be MERGE CONCURRENTLY ... and when you removed the WHEN NOT MATCHED clause it'd ERROR because that's no longer able to be done with the same concurrency-safe semantics? I don't know if this would be helpful TBH, or if it would negate Simon's compatibility goals. Just another

Re: [HACKERS] Statement-level rollback

atement level rollback we might still have some optimisation opportunities. Downside is that it needs support in each client driver. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing li

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

materialize into a temp table / tuplestore / etc. It's not clear to me why an unbounded portal fetch, using the tcp socket windows and buffers for flow control, isn't sufficient. Tomas, can you explain the use case a bit more? -- Craig Ringer http://www.2ndQuadrant.com/ Postgr

Re: [HACKERS] proposal: extend shm_mq to support more use cases

ing significant cost on existing users. And it'd make it way simpler to build a layer on top for a 1:m 2-way comms system like Ildus is talking about. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsq

Re: [HACKERS] Re: Anyone have experience benchmarking very high effective_io_concurrency on NVME's?

ame in the relevant threads, so you know that. To save others the time, see: * https://lwn.net/Articles/724198/ * https://lwn.net/Articles/671649/ -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hack

Re: [HACKERS] Dynamic result sets from procedures

ts from batch query executions. Instead, the multiple result set emulation requires the caller to 'getObject' the 'refcursor' field's result-object, then cast it to ResultSet, and treat it as a new (nested) result set. True multiple result sets would be exposed in PgJDBC via getMor

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

he socket buffering take care of things, reading results only when it wants them, and letting the server block when the windows are full. That's not to say that SQL-level cursor support wouldn't be nice. I'm just trying to better understand what it's solving. -- Craig Ringer h

Re: [HACKERS] Reading timeline from pg_control on replication slave

On 28 October 2017 at 06:09, Michael Paquier wrote: > On Fri, Oct 27, 2017 at 1:04 AM, Andrey Borodin wrote: >> I'm working on backups from replication salve in WAL-G [0] >> Backups used to use result of pg_walfile_name(pg_start_backup(...)). Call

Re: [HACKERS] logical decoding of two-phase transactions

hils has done some significant work on this patch. Hopefully he'll be able to share it. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make change

Re: [HACKERS] Timeline ID in backup_label file

e start LSN when working on > backup methods. > > I would find interesting to add at the bottom of the backup_label file > a new field called "START TIMELINE: %d" to put this information in a > more understandable shape. Any opinions? Strong "yes" from me. -- Crai

Re: [HACKERS] legitimacy of using PG_TRY , PG_CATCH , PG_END_TRY in C function

On 23 October 2017 at 16:16, Craig Ringer <cr...@2ndquadrant.com> wrote: > On 23 October 2017 at 08:30, John Lumby <johnlu...@hotmail.com> wrote: > >> All works but not perfectly -- at COMMIT, resource_owner issues >> relcache reference leak messages about relatio

Re: [HACKERS] legitimacy of using PG_TRY , PG_CATCH , PG_END_TRY in C function

ostgresMain. At some point I'd really like to expose that in a more general way so it can be used from background workers. Right now AFAICS most background workers have to cope with errors with a proc_exit(1) and getting restarted to try again. Not ideal. -- Craig Ringer http:

Re: [HACKERS] How to determine that a TransactionId is really aborted?

> I'm just thinking out lout here, but the context is likely something along > the lines of externally storing all transaction ids, and periodically asking > Postgres if they're known-to-be-aborted-by-all-transactions -- one at a time. I think Peter is asking "why?". -- Craig

Re: [HACKERS] Re: Is anything preventing us from allowing write to foreign tables from standby?

les from standbys. It >> works from the first glance for me. > > > No interest yet, but no objections too :-) > I'm going to add this to next commitfest. Superficially at least, it sounds like a good idea. We should only need a virtual xid when we're working with foreign tables since we don't

Re: [HACKERS] Query got Killed with CTE.

1); > > You produced a huge cross-product by the looks, and psql ran of of RAM buffering the result. The OOM killer fired (check 'dmesg' to confirm) and killed psql. The server noticed psql going away, and reported the fact. None of this is surprising. What's the problem here? --

Re: [HACKERS] SIGSEGV in BRIN autosummarize

the same configure options as you find in the rpm spec. That can be easier when iterating tests and builds. Since the patches are separate, you can skip the tarball and clone the same tag from git instead. Then apply the rpm patches as separate commits. That's typically what I'll do, make

Re: [HACKERS] Determine state of cluster (HA)

On 17 October 2017 at 01:02, Joshua D. Drake <j...@commandprompt.com> wrote: > On 10/15/2017 07:39 PM, Craig Ringer wrote: >> >> On 13 October 2017 at 08:50, Joshua D. Drake <j...@commandprompt.com> wrote: >>> >>> -Hackers, >>> >>> I

Re: [HACKERS] Determine state of cluster (HA)

On 13 October 2017 at 08:50, Joshua D. Drake wrote: > -Hackers, > > I had a long call with a firm developing front end proxy/cache/HA for > Postgres today. Essentially the software is a replacement for PGPool in > entirety but also supports analytics etc... When I was

Re: [HACKERS] Slow synchronous logical replication

those WAl records which is safe to skip > (insert/update/delete records not affecting this publication). That sounds like a giant layering violation too. I suggest focusing on reducing the amount of work done when reading WAL, not trying to jump over whole ranges of WAL. -- Craig Ringer

Re: [HACKERS] show precise repos version for dev builds?

t. We send server_version, but not server_version_num, as GUC_REPORT. So if a client wants server_version_num it has to do another round trip to query for it. - In pg_config, where we don't expose any --version-num only --version. -- Craig Ringer http://www.2ndQuadrant.com/ Post

Re: [HACKERS] Slow synchronous logical replication

point wal > -sender to the proper LSN without decoding huge part of WAL. > But it seems to be not so easy to implement. Sounds like confusing layering violations to me. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &a

Re: [HACKERS] show precise repos version for dev builds?

ware will fail in a variety of exciting ways, some more subtle than others. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to y

Re: [HACKERS] Slow synchronous logical replication

On 9 October 2017 at 15:37, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: > Thank you for explanations. > > On 08.10.2017 16:00, Craig Ringer wrote: >> >> I think it'd be helpful if you provided reproduction instructions, >> test programs, etc,

Re: [HACKERS] Latches API on windows

need to do it by having your extension load via shared_preload_libraries, registering its latch in shmem_startup_hook . But ... that's an off-the-cuff guess. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sen

Re: [HACKERS] Help required to debug pg_repack breaking logical replication

ut start by studying src/backend/access/heap/rewriteheap.c . Notably logical_end_heap_rewrite, logical_rewrite_heap_tuple, logical_begin_heap_rewrite. At a wild "I haven't read any of the relevant code in detail yet" stab in the dark, pg_repack is failing to do the bookkeeping required by logic

Re: [HACKERS] Slow synchronous logical replication

tion. Hopefully that gives you something to look into, anyway. Maybe you'll be inspired to work on parallelized logical decoding :) -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] Still another race condition in recovery TAP tests

we grow the list of modules they must first install. Meh, I don't buy that. At worst, all we have to do is provide a script that fetches them, from distro repos if possible, and failing that from CPAN. With cpanminus, that's pretty darn simple too. -- Craig Ringer http://www.2

Re: [HACKERS] [PATCH] A hook for session start

can be truly horrible when a user has hundreds of plpgsql functions that all like to make temp tables. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

[HACKERS] PATCH: Expose generate_qualified_relation_name functionality

ingMode(). (Prompted by https://dba.stackexchange.com/q/187788/7788) -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From 011cd6e08d6d29854db637d6beb8709615f376cb Mon Sep 17 00:00:00 2001 From: Craig Ringer <cr...@2ndquadrant.com&

Re: [HACKERS] fork()-safety, thread-safety

On 6 October 2017 at 08:06, Andres Freund <and...@anarazel.de> wrote: > On 2017-10-06 07:59:40 +0800, Craig Ringer wrote: >> The only thing that gets me excited about a threaded postgres is the >> ability to have a PL/Java, PL/Mono etc that don't suck. We could do >&

Re: [HACKERS] fork()-safety, thread-safety

make it.) The only thing that gets me excited about a threaded postgres is the ability to have a PL/Java, PL/Mono etc that don't suck. We could do some really cool things that just aren't practical right now. Not compelling to a wide audience, really. -- Craig Ringer http:

Re: [HACKERS] Postgresql gives error that role goes not exists while it exists

On 3 October 2017 at 20:47, Euler Taveira wrote: > > 2017-10-03 5:49 GMT-03:00 Nick Dro : > > Can someone assists with the issue posted on StackOverflow? > > > >

Re: [HACKERS] [PATCH] Off-by-one error in logical slot resource retention

patch so we can get it > further > > in the process? > > Since this patch has been in Waiting for Author state for the duration of > the > commitfest without moving, I’m marking it Returned with Feedback. If > there is > still interest in pursuing this patch, please re-su

Re: [HACKERS] pg_prepared_xact_status

opposed, I just don't really see the point. I'm not seeing where it'd come in useful. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] pg_prepared_xact_status

If we get better about that, then we might need some way to ask Pg to keep extra clog. But for now it works well enough. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] pg_prepared_xact_status

It can't do anything else, since if it unilaterally commits or rolls back it might later find out that the nodes on the other side of the network partition or whatever made the opposite decision and, boom! -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] pg_prepared_xact_status

resolver; then I think it's probably not going to get far. I could see a full DTC resolver in postgres one day, once we have things like working in-core logical rep based multi-master with 2PC support. But that's a looong way off. -- Craig Ringer http://www.2ndQuadrant.com/

Re: [HACKERS] Built-in plugin for logical decoding output

/shims using user catalogs. Not impossible, but probably irritating and verbose. And you'd have none of the DDL required to manage it, so you'd need SQL-function equivalents. I suspect you'd be better off tweaking pglogical to speak the same protocol as pg10, since the pgoutput protocol is an evo

Re: [HACKERS] Built-in plugin for logical decoding output

ted, we can likely use pgoutput for that to some extent at least. I think the pressing need is json, going by the zillion plugins out there for it. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Built-in plugin for logical decoding output

That said, I'm all in favour of a generic json output plugin that shares infrastructure with logical replication, so people who are on inflexible environments have a fallback option. I just don't care to write it. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Built-in plugin for logical decoding output

t;>> Postgres and maintain it themselves. >>> >>> https://www.2ndquadrant.com/en/resources/2ndqpostgres/ >>> >> >> This doesn't seem like a good way to argue. >> >> > Sorry, that wasn't supposed to be negative. My point was t

Re: [HACKERS] Built-in plugin for logical decoding output

lives easier and I don't much care for that. Personally I'd be more friendly toward Amazon / Google / etc wanting us to include things for their convenience if they actually usefully contributed to development and maintenance of Pg. -- Craig Ringer

Re: [HACKERS] Built-in plugin for logical decoding output

col connection is *right there*... -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Built-in plugin for logical decoding output

ones. Why? https://github.com/apigee-labs/transicator/tree/master/pgoutput > No README? Why did this need to be invented, rather than using an existing plugin? I don't mind, I mean, it's great that you're using the plugin infrastructure and using postgres. I'm just curious what bottled-

Re: [HACKERS] Assertion failure when the non-exclusive pg_stop_backup aborted.

ens when waiting for the > archives to be done, and the session flag is set to NONE at this > point. > Another one to watch out for is that elog(...) and ereport(...) invoke CHECK_FOR_INTERRUPTS. That's given me exciting surprises before when combined with assertion checking and various exit cleanup hooks. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Error: dsa_area could not attach to a segment that has been freed

On 21 September 2017 at 05:50, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: > On Thu, Sep 21, 2017 at 12:59 AM, Robert Haas <robertmh...@gmail.com> > wrote: > > On Wed, Sep 20, 2017 at 5:54 AM, Craig Ringer <cr...@2ndquadrant.com> > wrote: > >&

Re: [HACKERS] Error: dsa_area could not attach to a segment that has been freed

On 20 September 2017 at 17:52, Craig Ringer <cr...@2ndquadrant.com> wrote: > On 20 September 2017 at 16:55, Thomas Munro <thomas.mu...@enterprisedb.com > > wrote: > >> On Wed, Sep 20, 2017 at 6:14 PM, Gaddam Sai Ram >> <gaddamsaira...@zohocorp.com> wrote:

Re: [HACKERS] Error: dsa_area could not attach to a segment that has been freed

u're using a big > long lived DSA area you have nothing like that. We need, IMO, a DSA-backed heirachical MemoryContext system. We can't use the exact MemoryContext API as-is due to the need for far pointers though :( -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

[HACKERS] Utilities for quoting literals and identifiers in Pg TAP tests

Hi all Here's a little utility class I wrote for value and identifier quoting for use in TAP tests. Might be handy for others. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services PGValues.pm Description: Perl pro

Re: [HACKERS] src/test/subscription/t/002_types.pl hanging on particular environment

On 20 September 2017 at 12:16, Craig Ringer <cr...@2ndquadrant.com> wrote: > The thought I had in mind upthread was to get rid of logicalrep slots >> in favor of expanding the underlying bgworker slot with some additional >> fields that would carry whatever extra info we ne

Re: [HACKERS] src/test/subscription/t/002_types.pl hanging on particular environment

On 20 September 2017 at 12:06, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Wed, Sep 20, 2017 at 9:23 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Craig Ringer <cr...@2ndquadrant.com> writes: > >> On 19 September 2017 at 18:04, Petr Jelinek < >

Re: [HACKERS] src/test/subscription/t/002_types.pl hanging on particular environment

On 20 September 2017 at 11:53, Tom Lane <t...@sss.pgh.pa.us> wrote: > Craig Ringer <cr...@2ndquadrant.com> writes: > > On 19 September 2017 at 18:04, Petr Jelinek < > petr.jeli...@2ndquadrant.com> > > wrote: > >> If you are asking why they are not

Re: [HACKERS] Show backtrace when tap tests fail

On 20 September 2017 at 06:36, David Steele <da...@pgmasters.net> wrote: > > I just use: > > $SIG{__DIE__} = sub {Carp::confess @_}; > That's what I patched into my TestLib.pm too, until I learned of Carp::Always. I'd rather have Carp::Always, but it's definitely an OK

Re: [HACKERS] src/test/subscription/t/002_types.pl hanging on particular environment

postmaster; and (b) the postmaster remembering bgworker registrations across crash restart with no way to tell it not to. Maybe Petr remembers the details? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] src/test/subscription/t/002_types.pl hanging on particular environment

help. At some point we'll also want to be able to enumerate background workers and get handles for existing workers. Also, let background workers recover from errors without exiting, which means factoring a bunch of stuff out of PostgresMain. But both of those are bigger jobs. -- Craig Ringer

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

y plan is to emit raw flags by default, so FROZEN would't be shown at all, only COMMITTED|INVALID. If the bool to decode combined flags is set, then it'll show things like FROZEN, and hide COMMITTED|INVALID. Similar for other combos. -- Craig Ringer http://www.2ndQuadrant.com/ P

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 13 September 2017 at 13:44, Vaishnavi Prabakaran < vaishnaviprabaka...@gmail.com> wrote: > Thanks for explaining. Will change this too in next version. > > Thankyou, a lot, for picking up this patch. -- Craig Ringer http://www.2ndQuadrant.com/ Postgre

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

PQbatchSyncQueue >> > + >> > + >> >> I wonder why this isn't framed as PQbatchIssue/Send/...()? Syncing seems >> to mostly make sense from a protocol POV. >> >> > Renamed to PQbatchCommitQueue. > > Per above, strong -1 on that. Bu

Re: [HACKERS] WIP: Failover Slots

On 14 August 2017 at 11:56, Craig Ringer <cr...@2ndquadrant.com> wrote: > > I don't want to block failover slots on decoding on standby just because > decoding on standby would be nice to have. > However, during discussion with Tomas Munro a point has come up that does bl

Re: [HACKERS] [PATCH] Fix drop replication slot blocking instead of returning error

. > > > > Sorry, I was thinking that the new behavior was needed for internal > > future functions since the doc wasn't changed. > > FWIW, I also don't think it's ok to just change the behaviour > unconditionally and without a replacement for existing behaviour. Seems like it just needs a new argument nowait DEFAULT false Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Custom allocators in libpq

ations and pass it to a connect function. I pretty much always just put the user's original connstring in 'dbname' and set expand_dbname = true instead. It might make sense to have any new function accept PQconninfoOption*. Or a variant of PQconninfoParse that populates k/v arrays with 'n' extra

Re: [HACKERS] Make pg_regress print a connstring with sockdir

On 28 August 2017 at 19:45, Tom Lane <t...@sss.pgh.pa.us> wrote: > Craig Ringer <cr...@2ndquadrant.com> writes: > > It's a pain having to find the postmaster command line to get the port > > pg_regress started a server on. We print the port in the pg_regress >

Re: [HACKERS] psql --batch

ll bytes aren't usable for all scripts, and nothing else cannot also be output in the data its self. No easy answers there. In cases where I expect that to be an issue I sometimes use \COPY ... TO STDOUT WITH (FORMAT CSV) though. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] psql --batch

mplaining about, >> but it would be ok as the long option version if -B is added, and it is >> auto-completion friendly. > > > This doesn't really address the original issue though, that it's far from obvious how to easily and correctly script psql. I guess there's always the option of a docs patch for that. *shrug* I'll see what others have to say. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Make pg_regress print a connstring with sockdir

On 28 August 2017 at 15:19, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Aug 28, 2017 at 4:07 PM, Craig Ringer <cr...@2ndquadrant.com> > wrote: > > == starting postmaster== > > running with PID 30235; connec

Re: [HACKERS] Make pg_regress print a connstring with sockdir

:08, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Aug 28, 2017 at 2:28 PM, Craig Ringer <cr...@2ndquadrant.com> > wrote: > > It's a pain having to find the postmaster command line to get the port > > pg_regress started a server on. We print the port in the pg_

Re: [HACKERS] psql --batch

wo, though, as I can at least think of some cases where you might want it. X: skip .psqlrc Reliable, portable scripted psql shouldn't be using the local .psqlrc IMO. It's likely to just break things in exciting ways. But I can see it being reasonable to require this option to be s

[HACKERS] Make pg_regress print a connstring with sockdir

? If you'd prefer nicer wording at the expense of two lines, maybe running with PID 16409 connection string: 'port=50848 host=/tmp/blah' -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From 5b242d9af911a88c084a74bf49904b941173

[HACKERS] psql --batch

Hi all I find myself regurgitating the incantation psql -qAtX -v ON_ERRORS_STOP=1 quite a bit. It's not ... super friendly. It strikes me that we could possibly benefit from a 'psql --batch' option. Thoughts? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] WIP: Separate log file for extension

anisms could be left as stubs initially. So the outcome would be the same, just without the assumption of specific file name and output mechanism baked in. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] shm_mq_wait_internal gets stuck forever on fast shutdown

rsight re applying shm_mq to non-bgworker endpoints. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] shm_mq_wait_internal gets stuck forever on fast shutdown

On 21 August 2017 at 10:57, Craig Ringer <cr...@2ndquadrant.com> wrote: > Hi all > > I've noticed a possible bug / design limitation where shm_mq_wait_internal > sleep in a latch wait forever, and the postmaster gets stuck waiting for > the bgworker the wait

[HACKERS] shm_mq_wait_internal gets stuck forever on fast shutdown

er looping should continue, to be passed to shm_mq_attach. So if you can't supply a bgw handle, you supply that instead. Provide a shm_mq_set_handle equivalent for it too. Any objections to the last approach? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24

Re: [HACKERS] Updating line length guidelines

nally I'd be fine with 100 or so, but when I'm using buffers side by side, or when I'm working in poor conditions where I've set my terminal to "giant old people text" sizes, I remember the advantages of a width limit. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] [RFC] What would be difficult to make data models pluggable for making PostgreSQL a multi-model database?

el. With some kind of interaction so the sub-planner for the other model could know to generate a different sub-plan based on the context of the outer plan. I have no idea how that could work. But I think you have about zero chance of achieving what you want by going straight there. Focus on

Re: [HACKERS] Function to move the position of a replication slot

think it's quite a useful > function to be used by an automated system. E.g. to ensure enough, but > not too much, WAL is available for a tertiary standby both on the actual > primary and a failover node. > I strongly agree. If you really need to move a physical slot back (why?) you can do i

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

r other multi-purpose flags we have macros, and I think it'd make sense to use them here too. Eschew direct use of HEAP_XMIN_COMMITTED, HEAP_XMIN_INVALID and HEAP_XMIN_FROZEN in tests. Instead, consistently use HeapXminIsFrozen(), HeapXminIsCommitted(), and HeapXminIsInvalid() or something like th

Re: [HACKERS] Function to move the position of a replication slot

ut plugin that filters out all xacts, or by lower level use of the decoding code. Reasonable? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

we should not do is interpret the bitmasks (omitting some of > the information) assuming all the bits were set correctly. I agree, and the patch already does half of this: it can output just the raw bit flags, or it can interpret them to show HEAP_XMIN_FROZEN etc. So the required change, whi

Re: [HACKERS] What users can do with custom ICU collations in Postgres 10

On 15 August 2017 at 10:16, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Aug 15, 2017 at 11:10 AM, Craig Ringer <cr...@2ndquadrant.com> > wrote: > > Ooh, this finally gives us a path toward case-insensitive default > database > > collation

Re: [HACKERS] What users can do with custom ICU collations in Postgres 10

umbers come first, which we see here: > > Ooh, this finally gives us a path toward case-insensitive default database collation via CLDR caseLevel. http://userguide.icu-project.org/collation http://www.unicode.org/reports/tr35/tr35-collation.html#Algorithm_Case That *definitely* should be do

Re: [HACKERS] make async slave to wait for lsn to be replayed

: That ship sailed a long time ago unfortunately, they're all over pg_stat_replication and pg_replication_slots and so on. They're already routinely used for monitoring replication lag in bytes, waiting for a peer to catch up, etc. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

On 15 August 2017 at 09:11, Moon Insung <moon_insung...@lab.ntt.co.jp> wrote: > Dear Craig Ringer > > > > Frist, thank you for implementing the necessary function. > > > > but, i have some question. > > > > question 1) vacuum freeze hint bits >

Re: [HACKERS] WIP: Failover Slots

much actual code overlap beyond minor things like both wanting a way to query slot state on the upstream. Both *could* use decoding on standby to advance slot positions, but for the physical case that's just a slower (and unfinished) way to do what we already have, wheras it's necessary for logical failover. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Thoughts on unit testing?

ests to run them. It's not strict unit testing as the rest of Pg's APIs aren't mocked away, but it's very practical small-unit integration testing that helps catch issues. I wouldn't mind having an easier and nicer way to do that built in to Pg, but don't have many ideas about practical, low-m

Re: [HACKERS] WIP: Failover Slots

257 ) To actually use the slot once decoding on standby is supported: a decoding client on "C" can consume xacts and cause slot "X" to advance catalog_xmin, confirmed_flush_lsn, etc. walreceiver on "C" will tell walsender on "B" about the new slot state, and it'll get synced up-tree, then B will tell A. Since slot is already marked permanent, state won't get copied back down-tree, that only happens once when slot is first fully created on master. Some node "D" can exist as a phys rep of "C". If C fails and is replace with D, admin can promote the down-mirror slot on "D" to an owned slot. Make sense? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Thoughts on unit testing?

heavily on 'static' and move toward higher level visibility (https://gcc.gnu.org/wiki/Visibility: -fvisibility=hidden and __attribute__((dllexport)) ). It'd make it easier not to forget needed PGDLLEXPORTs, let us hide stuff we consider really internal but still share across a few files, etc.) -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] [TRAP: FailedAssertion] causing server to crash

On 10 August 2017 at 23:25, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Aug 7, 2017 at 2:06 AM, Craig Ringer <cr...@2ndquadrant.com> > wrote: > > I think so - specifically, that it's a leftover from a revision where the > > xid limit was advanced before

Re: [HACKERS] WIP: Failover Slots

On 9 August 2017 at 23:42, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Aug 8, 2017 at 4:00 AM, Craig Ringer <cr...@2ndquadrant.com> > wrote: > >> - When a standby connects to a master, it can optionally supply a list > >> of slot names t

Re: [HACKERS] Walsender timeouts and large transactions

> > + } > > + if (now <= TimestampTzPlusMilliseconds(last_reply_timestamp, > wal_sender_timeout / 2)) > > + return; > > + } > > > > If not, what problem prevents? > > We should do CHECK_FOR_INTERRUPTS() i

  1   2   3   4   5   6   7   8   9   10   >