[HACKERS] [PATCH] Generalized JSON output functions

Hi, Hackers! Attached is a patch against master to generalize the JSON-producing functions in utils/adt/json.c and to provide a set of callbacks which can be overridden the same way that is already provided for *parsing* JSON. The motivation behind this to be able to produce specially-crafted

[HACKERS] Handle PGRES_COPY_BOTH in psql for logical replication?

Hello Hackers, At the moment one can open up a replication connection using psql and use all of the commands available on this special type of connection, except for START_REPLICATION: $ psql -d dbname=test replication=database psql (9.4.1) Type help for help. test=# IDENTIFY_SYSTEM;

Re: [HACKERS] Handle PGRES_COPY_BOTH in psql for logical replication?

On Fri, Jun 5, 2015 at 10:22 AM, Andres Freund and...@anarazel.de wrote: Maybe I'm missing something, which functions do you have in mind exactly? pg_logical_slot_get_changes() etc? Oh, totally forgot about these. However there are two significant differences between using the functions

Re: [HACKERS] Handle PGRES_COPY_BOTH in psql for logical replication?

On Fri, Jun 5, 2015 at 9:26 AM, Andres Freund and...@anarazel.de wrote: On 2015-06-05 09:20:41 +0200, Shulgin, Oleksandr wrote: Hello Hackers, At the moment one can open up a replication connection using psql and use all of the commands available on this special type of connection

Re: [HACKERS] Streaming replication for psycopg2

On Tue, Jun 2, 2015 at 2:23 PM, Shulgin, Oleksandr oleksandr.shul...@zalando.de wrote: Hello, I've submitted a patch to psycopg2 to support streaming replication protocol (COPY_BOTH): https://github.com/psycopg/psycopg2/pull/322 It would be great if more people had a chance to take a look

[HACKERS] Fix logical decoding sendtime update

Hi Hackers, it turns out, that the code in WalSndWriteData is setting the timestamp of the replication message just *after* it has been sent out to the client, thus the sendtime field always reads as zero. Attached is a trivial patch to fix this. The physical replication path already does the

[HACKERS] Streaming replication for psycopg2

Hello, I've submitted a patch to psycopg2 to support streaming replication protocol (COPY_BOTH): https://github.com/psycopg/psycopg2/pull/322 It would be great if more people had a chance to take a look and provide feedback about it. In particular, please see example usage at this github

Re: [HACKERS] [PATCH] Generalized JSON output functions

On Sat, May 23, 2015 at 3:03 AM, Ryan Pedela rped...@datalanche.com wrote: On Fri, May 22, 2015 at 10:51 AM, Merlin Moncure mmonc...@gmail.com wrote: On Fri, May 22, 2015 at 9:43 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Andrew Dunstan wrote: On 05/20/2015 09:16 AM, Shulgin

Re: [HACKERS] pg_dump quietly ignore missing tables - is it bug?

On Fri, May 22, 2015 at 6:09 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-05-21 16:48 GMT+02:00 Oleksandr Shulgin oleksandr.shul...@zalando.de : I think this is a bit over-engineered (apart from the fact that processSQLNamePattern is also used in two dozen of places in

Re: [HACKERS] pg_dump quietly ignore missing tables - is it bug?

On Fri, May 22, 2015 at 6:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Oleksandr Shulgin oleksandr.shul...@zalando.de writes: I think this is a bit over-engineered (apart from the fact that processSQLNamePattern is also used in two dozen of places in psql/describe.c and all of them must be

Re: [HACKERS] pg_dump quietly ignore missing tables - is it bug?

On Fri, May 22, 2015 at 6:32 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-05-22 18:30 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de: On Fri, May 22, 2015 at 6:09 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-05-21 16:48 GMT+02:00 Oleksandr Shulgin

Re: [HACKERS] count_nulls(VARIADIC any)

On Thu, Aug 13, 2015 at 9:25 AM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-08-13 9:21 GMT+02:00 Marko Tiikkaja ma...@joh.to: On 8/13/15 9:18 AM, Shulgin, Oleksandr wrote: nnulls() I think I'd prefer num_nulls() over that. can be what about similar twin function num_nonulls

Re: [HACKERS] count_nulls(VARIADIC any)

On Thu, Aug 13, 2015 at 2:19 AM, David G. Johnston david.g.johns...@gmail.com wrote: On Wed, Aug 12, 2015 at 4:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Geoghegan p...@heroku.com writes: On Wed, Aug 12, 2015 at 10:30 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: The name

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Wed, Aug 19, 2015 at 4:45 PM, ''Victor Wagner *EXTERN*' *EXTERN*' *EXTERN* vi...@wagner.pp.ru wrote: On 2015.08.19 at 15:35:17 +0100, Simon Riggs wrote: I think we do need some way of saying that a readonly connection is OK. So I had such thing in my propsal (boolean parameter

Re: [HACKERS] deparsing utility commands

On Thu, Aug 20, 2015 at 4:28 PM, Shulgin, Oleksandr oleksandr.shul...@zalando.de wrote: Which gets deparsed as: ALTER TABLE cwi_test DROP CONSTRAINT cwi_uniq_idx, ADD CONSTRAINT cwi_replaced_pkey PRIMARY KEY USING INDEX cwi_replaced_pkey; The problem is that during constraint

Re: [HACKERS] Streaming replication for psycopg2

On Thu, Jun 4, 2015 at 5:49 PM, Shulgin, Oleksandr oleksandr.shul...@zalando.de wrote: On Tue, Jun 2, 2015 at 2:23 PM, Shulgin, Oleksandr oleksandr.shul...@zalando.de wrote: Hello, I've submitted a patch to psycopg2 to support streaming replication protocol (COPY_BOTH): https

Re: [HACKERS] deparsing utility commands

On Wed, Jul 29, 2015 at 12:44 PM, Shulgin, Oleksandr oleksandr.shul...@zalando.de wrote: On Tue, May 12, 2015 at 12:25 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: David Steele wrote: I have reviewed and tested this patch and everything looks good to me. It also looks like you

Re: [HACKERS] Don'st start streaming after creating a slot in pg_receivexlog

On Wed, Jul 29, 2015 at 10:02 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/29/2015 10:58 AM, Michael Paquier wrote: On Wed, Jul 29, 2015 at 4:51 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/29/2015 10:37 AM, Andres Freund wrote: Heikki complained about pg_receivexlog

Re: [HACKERS] deparsing utility commands

On Tue, May 12, 2015 at 12:25 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: David Steele wrote: I have reviewed and tested this patch and everything looks good to me. It also looks like you added better coverage for schema DDL, which is a welcome addition. Thanks -- I have pushed

Re: [HACKERS] [PATCH] Generalized JSON output functions

On July 13 I wrote: Yes, but I think the plugin is the right place to do it. What is more, this won't actually prevent you completely from producing non-ECMAScript compliant JSON, since json or jsonb values containing offending numerics won't be caught, AIUI. But a fairly simple to write

Re: [HACKERS] replication slot restart_lsn initialization

On Fri, Aug 14, 2015 at 9:54 AM, Andres Freund and...@anarazel.de wrote: On 2015-08-14 16:44:44 +0900, Michael Paquier wrote: Commit 6fcd8851, which is the result of this thread, is not touching the replication protocol at all. This looks like an oversight to me: we should be a maximum

Re: [HACKERS] [PATCH] Generalized JSON output functions

On Jul 17, 2015 4:31 PM, Andrew Dunstan and...@dunslane.net wrote: On 07/17/2015 10:11 AM, Andrew Dunstan wrote: On 07/17/2015 08:20 AM, Shulgin, Oleksandr wrote: This patch makes Postgres core more complex Yes, it does. But, that was not the purpose, obviously. :-) while not really

Re: [HACKERS] [PATCH] Generalized JSON output functions

On Jul 17, 2015 12:23 AM, Ryan Pedela rped...@datalanche.com wrote: On Thu, Jul 16, 2015 at 11:51 AM, Robert Haas robertmh...@gmail.com wrote: I don't understand these issues in great technical depth, but if somebody is arguing that it's OK for PostgreSQL to be difficult to use for a certain

Re: [HACKERS] [PATCH] Generalized JSON output functions

Yes, but I think the plugin is the right place to do it. What is more, this won't actually prevent you completely from producing non-ECMAScript compliant JSON, since json or jsonb values containing offending numerics won't be caught, AIUI. Ah, that's a good catch indeed. But a fairly simple to

Re: [HACKERS] [PATCH] Generalized JSON output functions

On Sun, Jul 12, 2015 at 8:39 PM, Pavel Stehule pavel.steh...@gmail.com wrote: The thing is - it's not only about whitespace, otherwise I would probably not bother with the generic interface. For my original problem, there is simply no way to do this correctly in an extension w/o copying over

Re: [HACKERS] Patch: Implement failover on libpq connect level.

On Mon, Oct 26, 2015 at 10:02 PM, Christopher Browne wrote: > > On 26 October 2015 at 16:25, Peter Eisentraut wrote: > >> On 10/14/15 6:41 AM, Victor Wagner wrote: >> > 1. It is allowed to specify several hosts in the connect string, either >> > in URL-style

Re: [HACKERS] pg_basebackup and replication slots

On Mon, Oct 26, 2015 at 12:58 PM, Joshua D. Drake wrote: > Hello, > > The fact that pg_basebackup doesn't use replicaiton slots, is that a > technical limitation or just a, "we need a patch"? > I believe it does, but only in master so far, not even in 9.5:

Re: [HACKERS] Erroneous cost estimation for nested loop join

On Mon, Nov 9, 2015 at 11:08 AM, wrote: > > - cost parameter calibration: random_page_cost = 92.89 > This demands some explanation and raises question of value of seq_page_cost parameter -- I don't see anything about it your mail. -- Alex

Re: [HACKERS] pglogical_output - a general purpose logical decoding output plugin

On Mon, Nov 2, 2015 at 1:17 PM, Craig Ringer wrote: > Hi all > > I'd like to submit pglogical_output for inclusion in the 9.6 series as > a contrib. Yay, that looks pretty advanced! :-) Still digesting... -- Alex

Re: [HACKERS] extend pgbench expressions with functions

On Fri, Sep 18, 2015 at 10:21 AM, Fabien COELHO wrote: > > Hello Kyotaro-san, > > My description should have been obscure. Indeed the call tree is >> finite for *sane* expression node. But it makes infinit call for >> a value of expr->etype unknown by both evalDouble and >>

Re: [HACKERS] SuperUser check in pg_stat_statements

On Mon, Oct 19, 2015 at 5:15 PM, rajan wrote: > Hello, > > When monitoring using pg_stat_satements I see that postgres by default > conceals queries executed by other users from the user who is selecting the > pg_stat_statements view. > > I have edited the

Re: [HACKERS] How to import PostgreSQL 9.2.4 dump to PostgreSQL 9.4.5?

On Thu, Oct 15, 2015 at 11:27 AM, FattahRozzaq wrote: > Hi guys, > > I'm running some test. > > > However, I'm stuck in restoring a PostgreSQL 9.2.4 dump to PostgreSQL > 9.4.5 database. > I'm doing the backup using pg_dump version PostgreSQL 9.2.4 in text > file (SQL dump). >

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 7:52 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > > This is not a change of the direction, but rather of the approach. > Hitting a process with a signal and hoping it will produce a meaningful > response in all circumstances with

Re: [HACKERS] Database schema diff

On Tue, Oct 13, 2015 at 5:48 PM, Michal Novotny < michal.novo...@trustport.com> wrote: > Hi guys, > > I would like to ask you whether is there any tool to be able to compare > database schemas ideally no matter what the column order is or to dump > database table with ascending order of all

Re: [HACKERS] Patch: Implement failover on libpq connect level.

On Wed, Oct 14, 2015 at 12:41 PM, Victor Wagner wrote: > On 2015.08.18 at 07:18:50 +0300, Victor Wagner wrote: > > > Rationale > > = > > > > Since introduction of the WAL-based replication into the PostgreSQL, it > is > > possible to create high-availability and

Re: [HACKERS] [PATCH] Generalized JSON output functions

On Fri, Jul 10, 2015 at 5:16 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Well, one could call it premature pessimization due to dynamic call overhead. IMO, the fact that json_out_init_context() sets the value callback to json_out_value is an implementation detail, the other parts of

Re: [HACKERS] [PATCH] Generalized JSON output functions

On Jul 11, 2015 6:19 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-07-11 18:02 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de: On Fri, Jul 10, 2015 at 5:16 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Well, one could call it premature pessimization due to dynamic

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-10 14:34 GMT+02:00 Pavel Stehule pavel.steh...@gmail.com: Hi I am sending review of this patch: 1. I reread a previous discussion and almost all are for this patch (me too) 2. I have to fix a typo in hstore_io.c function (update attached), other (patching, regress tests)

Re: [HACKERS] [PATCH] Generalized JSON output functions

On Fri, Jul 10, 2015 at 4:04 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2. why you did indirect call via JsonOutContext? What is benefit dst.value(dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid, false); instead json_out_value(dst, ) For consistency. Even though we

Re: [HACKERS] [PATCH] Generalized JSON output functions

we talking about output - I can imagine, so there is only two possibilities - plain join, and pretty formatted join (but with only one style). This makes sense. Postgres core really only needs to support the minimum styles necessary for core requirements. This means raw unformatted json

Re: [HACKERS] psql - better support pipe line

On Fri, Aug 28, 2015 at 3:29 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 8/27/15 8:37 AM, Pavel Stehule wrote: I don't think we can detect and remove the default values from this output in a reliable way? This is pretty difficult - any parameter can be important, and hard to

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Fri, Aug 28, 2015 at 6:10 PM, Teodor Sigaev teo...@sigaev.ru wrote: +1 for bringing the jdbc driver URI syntax into libpq, so that all interfaces can be optionally specified this way. This doesn't preclude the use of ipfailover, in fact it might be work well together. If you don't like it,

Re: [HACKERS] to_json(NULL) should to return JSON null instead NULL

On Sat, Aug 29, 2015 at 3:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 08/29/2015 08:47 AM, Shulgin, Oleksandr wrote: Given there were no loud complaints about this, the current behavior is appropriate for most users, the rest can still work

Re: [HACKERS] to_json(NULL) should to return JSON null instead NULL

On Sat, Aug 29, 2015 at 8:39 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hi currently JSON support NULL value - but we doesn't convert NULL to JSON correctly postgres=# select to_json(NULL::text) IS NULL; ┌──┐ │ ?column? │ ╞══╡ │ t│ └──┘ (1 row)

Re: [HACKERS] psql - better support pipe line

On Fri, Aug 28, 2015 at 9:52 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 8/28/15 3:58 AM, Shulgin, Oleksandr wrote: It occurs to me the most flexible thing that could be done here would be providing a libpq function that spits out JSON connection parameters and have psql

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Sat, Aug 29, 2015 at 5:44 PM, Andres Freund and...@anarazel.de wrote: On 2015-08-29 17:33:22 +0200, Shulgin, Oleksandr wrote: Probably using SIGUSR2 would be more appropriate, but I'm not sure if there are other extensions out there that might be already using it for some other reason

[HACKERS] On-demand running query plans using auto_explain and signals

Hello, The other day I've rediscovered the exciting idea of using signals to dump a backend's currently executed query plan, which, it turns out, was already proposed by Pavel and Simon in these threads:

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Aug 29, 2015 7:31 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-08-29 18:36 GMT+02:00 Andres Freund and...@anarazel.de: On 2015-08-29 18:27:59 +0200, Pavel Stehule wrote: 2015-08-29 18:25 GMT+02:00 Shulgin, Oleksandr oleksandr.shul...@zalando.de Good point. There's still

Re: [HACKERS] psql - better support pipe line

On Wed, Aug 26, 2015 at 5:44 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Hi 2015-08-25 17:21 GMT+02:00 Tom Lane t...@sss.pgh.pa.us: Jim Nasby jim.na...@bluetreble.com writes: What I've had problems with is trying to correlate psql specified connection attributes with things like

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Aug 31, 2015 at 12:35 PM, Pavel Stehule wrote: > >>> >>> http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com >>> >> >> Ah, thanks! Somehow I've missed this mail. You didn't add the patch to >> a commitfest

Re: [HACKERS] On-demand running query plans using auto_explain and signals

> > Do you still have the code somewhere around? Did it see production use? >>> >> I sent it to mailing list year ago > > > http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com > Ah, thanks! Somehow I've missed this mail. You didn't add the

[HACKERS] Adding since-version tags to the docs?

Hello, I often find it pity that our docs are missing any information on since when a certain GUC setting, SQL-level command or function was introduced. Clicking through the "this page in other versions" links at the top of a webpage does help, but you still need to do some guessing (binary

Re: [HACKERS] Adding since-version tags to the docs?

On Mon, Aug 31, 2015 at 4:01 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > "Shulgin, Oleksandr" <oleksandr.shul...@zalando.de> writes: > > I often find it pity that our docs are missing any information on since > > when a certain GUC setting, SQL-lev

Re: [HACKERS] Adding since-version tags to the docs?

On Mon, Aug 31, 2015 at 3:30 PM, Andres Freund <and...@anarazel.de> wrote: > Hi, > > On 2015-08-31 13:06:04 +0200, Shulgin, Oleksandr wrote: > > I often find it pity that our docs are missing any information on since > > when a certain GUC setting, SQL-level command

Re: [HACKERS] Adding since-version tags to the docs?

On Mon, Aug 31, 2015 at 3:17 PM, Anastasia Lubennikova < a.lubennik...@postgrespro.ru> wrote: > 31.08.2015 14:06, Shulgin, Oleksandr пишет: > > Hello, > > I often find it pity that our docs are missing any information on since > when a certain GUC setting, SQL-l

Re: [HACKERS] On-demand running query plans using auto_explain and signals

> > I'd say we should hide the so-designed pg_cmdstatus() interface behind >> more friendly calls like pg_explain_backend() and pg_backend_progress() to >> give some naming examples, to remove the need for magic numbers in the >> second arg. >> > > I had similar idea - this is good enough for

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Tue, Sep 1, 2015 at 8:12 PM, Andres Freund wrote: > On 2015-09-01 14:07:19 -0400, Robert Haas wrote: > > But I think it's quite wrong to assume that the infrastructure for > > this is available and usable everywhere, because in my experience, > > that's far from the case.

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule wrote: > >> But do we really need the slots mechanism? Would it not be OK to just >> let the LWLock do the sequencing of concurrent requests? Given that we >> only going to use one message queue per cluster, there's not

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Thu, Sep 3, 2015 at 6:02 PM, Robert Haas wrote: > > Maybe someday we should have all that, but I think for right now > that's complicating things unnecessarily. I think the best proposal > so far is to allow the host=X option to be repeated multiple times. > If you

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 3:07 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Wed, Sep 2, 2015 at 3:04 PM, Pavel Stehule <pavel.steh...@gmail.com> > wrote: >> >> >>> Well, maybe I'm missing something, but sh_mq_create() will just >&

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 4, 2015 at 6:11 AM, Pavel Stehule wrote: > Sorry, but I still don't see how the slots help this issue - could you >> please elaborate? >> > with slot (or some similiar) there is not global locked resource. If I'll > have a time at weekend I'll try to write

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 8, 2015 at 6:58 AM, Pavel Stehule wrote: >> >> But you will still lock on the slots list to find an unused one. How is that substantially different from what I'm doing? > > It is not necessary - you can use similar technique to what it does PGPROC. I am

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Wed, Sep 2, 2015 at 9:00 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Sep 2, 2015 at 4:52 AM, Shulgin, Oleksandr > <oleksandr.shul...@zalando.de> wrote: > > On Tue, Sep 1, 2015 at 8:12 PM, Andres Freund <and...@anarazel.de> > wrote: > >>

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 11:16 AM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > > > 2015-09-02 11:01 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Tue, Sep 1, 2015 at 7:02 PM, Pavel Stehule <pavel.steh...@gmail.com> >> wrote

Re: [HACKERS] On-demand running query plans using auto_explain and signals

> > I'm not familiar with the shared memory handling, but could we not >> allocate just enough shared memory to fit the data we're going to write >> instead of the fixed 8k? It's not that we cannot know the length of the >> resulting plan text in advance. >> > > the shared memory cannot be reused

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Sep 3, 2015 10:14 PM, "Pavel Stehule" wrote: >>> >>> Please find attached a v3. >>> >>> It uses a shared memory queue and also has the ability to capture plans nested deeply in the call stack. Not sure about using the executor hook, since this is not an extension...

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Sep 3, 2015 7:30 PM, "Robert Haas" wrote: > > All of these objections seem pretty thin to me. I'd accept any of > them as a reason for preferring one alternative over another, but I > don't accept that the presence of a few problems of this magnitude > means we should

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 2:56 PM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > > > 2015-09-02 12:36 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Wed, Sep 2, 2015 at 11:16 AM, Pavel Stehule <pavel.steh...@gmail.com> >> wro

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 2, 2015 at 3:04 PM, Pavel Stehule wrote: > > >> Well, maybe I'm missing something, but sh_mq_create() will just overwrite >> the contents of the struct, so it doesn't care about sender/receiver: only >> sh_mq_set_sender/receiver() do. >> > > if you create

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 8, 2015 at 11:49 AM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > > >> The real problem could be if the process that was signaled to connect > to the message queue never handles the interrupt, and we keep waiting > forever in shm_mq_receive().

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 9, 2015 at 8:36 AM, Pavel Stehule wrote: > >> Please find attached v4. >> > > It is better > One important thing to notice, and probably deserves a code comment, that any modification of the slot fields done by the info sender backend must be done before

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 9, 2015 at 10:22 AM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Wed, Sep 9, 2015 at 8:36 AM, Pavel Stehule <pavel.steh...@gmail.com> > wrote: > >> >>> Please find attached v4. >>> >> >> It is better >>

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 14, 2015 at 11:53 AM, Tomas Vondra <tomas.von...@2ndquadrant.com > wrote: > > On 09/14/2015 10:23 AM, Shulgin, Oleksandr wrote: > >> On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra >> <tomas.von...@2ndquadrant.com <mailto:tomas.

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 14, 2015 at 2:11 PM, Tomas Vondra wrote: > >> Now the backend that has been signaled on the second call to >> pg_cmdstatus (it can be either some other backend, or the backend B >> again) will not find an unprocessed slot, thus it will not try to >>

Re: [HACKERS] On-demand running query plans using auto_explain and signals

> > Well, I didn't attach the updated patch (doing that now). > This time for real. Sorry, it's Monday :-p diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c index 32ac58f..2e3beaf 100644 --- a/src/backend/storage/ipc/ipci.c +++ b/src/backend/storage/ipc/ipci.c @@ -43,6

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 14, 2015 at 3:09 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Mon, Sep 14, 2015 at 2:11 PM, Tomas Vondra < > tomas.von...@2ndquadrant.com> wrote: > >> >>> Now the backend that has been signaled on the second call to >>&

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 15, 2015 at 11:00 AM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Mon, Sep 14, 2015 at 7:27 PM, Pavel Stehule <pavel.steh...@gmail.com> > wrote: > >> >> 2015-09-14 18:46 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.sh

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 14, 2015 at 7:27 PM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > > 2015-09-14 18:46 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> >> I have a radical proposal to remove the need for locking: make the >> CmdStat

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Wed, Sep 16, 2015 at 8:07 PM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > > 2015-09-16 16:31 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> >> I've added the timeout parameter to the pg_cmdstatus call, and more >> import

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 2:14 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Sep 16, 2015 at 10:31 AM, Shulgin, Oleksandr > <oleksandr.shul...@zalando.de> wrote: > > I've also decided we really ought to suppress any possible ERROR level > > messag

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 12:06 PM, Pavel Stehule wrote: > >> That won't work really well with something like I use to do when testing >> this patch, namely: >> >> postgres=# select pid, array(select pg_cmdstatus(pid, 1, 10)) from >> pg_stat_activity where

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 5:16 PM, Pavel Stehule wrote: > 2015-09-17 16:46 GMT+02:00 Robert Haas : > >> >> Second, using a shm_mq manipulates the state of the process latch. I >> don't think you can make the assumption that it's safe to reset the >>

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra wrote: > Hi, > > I did a quick initial review of this patch today, so here are my comments > so far: > Hi Tomas, First of all, thanks for the review! - ipcs.c should include utils/cmdstatus.h (the compiler complains

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 28, 2015 at 7:09 PM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > > 2015-09-28 12:37 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> >>> I didn't propose too different solution. There is only one difference - >>

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 12:28 AM, Jim Nasby <jim.na...@bluetreble.com> wrote: > On 9/18/15 5:05 AM, Shulgin, Oleksandr wrote: > >> >> So this has to be the responsibility of the reply sending backend in the >> end: to create and release the DSM *at some point*.

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 12:57 PM, Andres Freund <and...@anarazel.de> wrote: > On 2015-09-25 19:13:13 +0200, Shulgin, Oleksandr wrote: > > the auto_explain contrib module. I now propose the most simple thing > > possible in my opinion: a new GUC option for auto_explain. I

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 25, 2015 at 7:13 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > > Some implementation details: > > For every backend that might be running (MaxBackends) we reserve a > dsm_handle slot in the addins shared memory. When the new option is turned

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Mon, Sep 28, 2015 at 12:05 PM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > > 2015-09-28 12:01 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Sun, Sep 27, 2015 at 8:05 AM, Pavel Stehule <pavel.steh...@gmail.com> >> wr

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Sun, Sep 27, 2015 at 8:05 AM, Pavel Stehule wrote: the preparing of content before execution is interesting idea, that can be > used more. The almost queries and plans are not too big, so when the size > of content is not too big - less than 1MB, then can be used one

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 7:01 PM, Simon Riggs <si...@2ndquadrant.com> wrote: > On 25 September 2015 at 12:13, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > > >> I now believe that use of ProcessInterrupts() in the recently proposed >> design a

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Tue, Sep 29, 2015 at 8:34 PM, Simon Riggs <si...@2ndquadrant.com> wrote: > On 29 September 2015 at 12:52, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > > >> Hitting a process with a signal and hoping it will produce a meaningful >> r

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Thu, Sep 17, 2015 at 10:13 PM, Robert Haas wrote: > On Thu, Sep 17, 2015 at 11:16 AM, Pavel Stehule > wrote: > > >> Second, using a shm_mq manipulates the state of the process latch. I > >> don't think you can make the assumption that it's

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 18, 2015 at 11:25 AM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > 2015-09-18 10:59 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> >> If we take the per-backend slot approach the locking seems unnecessary >> and the

Re: [HACKERS] Calculage avg. width when operator = is missing

On Wed, Sep 23, 2015 at 3:21 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > "Shulgin, Oleksandr" <oleksandr.shul...@zalando.de> writes: > > On Tue, Sep 22, 2015 at 11:56 PM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > >> It l

Re: [HACKERS] Calculage avg. width when operator = is missing

On Thu, Sep 24, 2015 at 12:30 AM, Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> Personally I think Alvaro's position is unduly conservative: to the > extent > >> that plans change it'd likely be for the better. But I'm not

Re: [HACKERS] Calculage avg. width when operator = is missing

On Sep 22, 2015 8:58 PM, "Andrew Dunstan" <and...@dunslane.net> wrote: > > > > On 09/22/2015 12:16 PM, Shulgin, Oleksandr wrote: >> >> Hi Hackers, >> >> I've recently stumbled upon a problem with table bloat estimation in case there are co

Re: [HACKERS] Calculage avg. width when operator = is missing

On Tue, Sep 22, 2015 at 11:17 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Shulgin, Oleksandr wrote: > > On Sep 22, 2015 8:58 PM, "Andrew Dunstan" <and...@dunslane.net> wrote: > > > > Yes, "/revenons/ à /nos moutons/." You can

Re: [HACKERS] Calculage avg. width when operator = is missing

On Tue, Sep 22, 2015 at 11:56 PM, Alvaro Herrera wrote: > Tom Lane wrote: > > > Should we consider this HEAD-only, or a back-patchable bug fix? > > Or perhaps compromise on HEAD + 9.5? > > It looks like a bug to me, but I think it might destabilize approved > execution

[HACKERS] Calculage avg. width when operator = is missing

Hi Hackers, I've recently stumbled upon a problem with table bloat estimation in case there are columns of type JSON. The quick bloat estimation queries use sum over pg_statistic.stawidth of table's columns, but in case of JSON the corresponding entry is never created by the ANALYZE command due

Re: [HACKERS] On-demand running query plans using auto_explain and signals

On Fri, Sep 18, 2015 at 7:04 PM, Robert Haas wrote: > > Frankly, I think you guys are making this out to be way more > complicated than it really is. Basically, I think the process being > queried should publish a DSM via a slot it owns. The recipient is > responsible

  1   2   >