Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

On 2016/02/22 20:13, Rushabh Lathia wrote: I did another round of review for the latest patch and well as performed the sanity test, and haven't found any functional issues. Found couple of issue, see in-line comments for the same. Thanks! On Thu, Feb 18, 2016 at 3:15 PM, Etsuro Fujita

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

Robert Haas writes: > On Tue, Feb 23, 2016 at 2:06 AM, Tom Lane wrote: >> Robert Haas writes: >>> Foreign tables are supposed to be categorically excluded from >>> parallelism. Not sure why that's not working in this instance.

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

On Tue, Feb 23, 2016 at 2:06 AM, Tom Lane wrote: > Robert Haas writes: >>> Foreign tables are supposed to be categorically excluded from >>> parallelism. Not sure why that's not working in this instance. > > BTW, I wonder where you think that's

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

Thomas Munro writes: > On Tue, Feb 23, 2016 at 4:03 AM, Tom Lane wrote: >> I've not looked at the test case to see if this is exactly what's >> going wrong, but it's pretty easy to see how there might be a problem: >> consider a STABLE

Re: [HACKERS] [PATH] Correct negative/zero year in to_date/to_timestamp

On 2/22/16, Thomas Munro wrote: > On Tue, Feb 23, 2016 at 11:58 AM, Vitaly Burovoy > wrote: >> Hello, Hackers! >> >> I'm writing another patch and while I was trying to cover corner cases >> I found that to_date and to_timestamp work wrong

Re: [HACKERS] Writing new unit tests with PostgresNode

On Tue, Feb 23, 2016 at 12:29 AM, Alvaro Herrera wrote: > Craig Ringer wrote: > >> > +=pod >> > + >> > +=head2 Set up a node >> > pod format... Do we really want that? Considering that those modules >> > are only aimed at being dedicated for in-core testing, I would say

Re: format() changes discussion (was: Re: [HACKERS] psql metaqueries with \gexec)

On 2/22/16 5:16 PM, Corey Huinker wrote: (One thing I had to come up with was processing of arrays, which you also see in that example JSON -- it's the specifiers that have a colon inside the {}. The part after the colon is used as separator between the array elements, and each

Re: [HACKERS] psql metaqueries with \gexec

On 2/22/16 5:13 PM, Alvaro Herrera wrote: Jim Nasby wrote: On 2/22/16 11:47 AM, Alvaro Herrera wrote: Pavel Stehule wrote: The design of the "format" function is not closed. Try to send prototype and patch. The possibility to do PostgreSQL customization was strong reason why we didn't

Re: [HACKERS] [PATH] Correct negative/zero year in to_date/to_timestamp

On Tue, Feb 23, 2016 at 11:58 AM, Vitaly Burovoy wrote: > Hello, Hackers! > > I'm writing another patch and while I was trying to cover corner cases > I found that to_date and to_timestamp work wrong if year in input > value is zero or negative: > > postgres=# SELECT >

Re: [HACKERS] Convert pltcl from strings to objects

On 2/18/16 6:26 AM, Victor Wagner wrote: On Tue, 9 Feb 2016 16:23:21 -0600 There is suspicious place at the end of compile_pltcl_fuction function, where you've put comment that old prodesc cannot be deallocated, because it might be used by other call. It seems that reference counting mechanism

Re: [HACKERS] Sanity checking for ./configure options?

On 2/5/16 10:08 AM, David Fetter wrote: On Wed, Feb 03, 2016 at 06:02:57PM -0600, Jim Nasby wrote: I just discovered that ./configure will happily accept '--with-pgport=' (I was actually doing =$PGPORT, and didn't realize $PGPORT was empty). What you end up with is a compile error in guc.c,

format() changes discussion (was: Re: [HACKERS] psql metaqueries with \gexec)

> > (One thing I had to come up with was processing of arrays, which you > also see in that example JSON -- it's the specifiers that have a colon > inside the {}. The part after the colon is used as separator between > the array elements, and each element is expanded separately.) > > I'm

Re: [HACKERS] psql metaqueries with \gexec

Jim Nasby wrote: > On 2/22/16 11:47 AM, Alvaro Herrera wrote: > >Pavel Stehule wrote: > > > >>The design of the "format" function is not closed. Try to send prototype > >>and patch. The possibility to do PostgreSQL customization was strong reason > >>why we didn't implemented "sprintf" and we

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

On Tue, Feb 23, 2016 at 4:03 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 22, 2016 at 11:02 AM, Thomas Munro >> wrote: >>> The postgres_fdw failure is a visibility-of-my-own-uncommitted-work >>> problem. The

Re: [HACKERS] [PATH] Correct negative/zero year in to_date/to_timestamp

On 2/22/16, Vitaly Burovoy wrote: > Testings, complains, advice, comment improvements are very appreciated. The patch seems simple, but it can lead to a discussion, so I've added it to CF. [CF]https://commitfest.postgresql.org/9/533/ -- Best regards, Vitaly Burovoy

[HACKERS] [PATH] Correct negative/zero year in to_date/to_timestamp

Hello, Hackers! I'm writing another patch and while I was trying to cover corner cases I found that to_date and to_timestamp work wrong if year in input value is zero or negative: postgres=# SELECT postgres-# y || '-06-01' as src postgres-# ,CASE WHEN y>0 THEN ('00'||y||'-06-01') WHEN

Re: [HACKERS] psql metaqueries with \gexec

On 2/22/16 11:47 AM, Alvaro Herrera wrote: Pavel Stehule wrote: The design of the "format" function is not closed. Try to send prototype and patch. The possibility to do PostgreSQL customization was strong reason why we didn't implemented "sprintf" and we implemented "format". Probably not

Re: [HACKERS] about google summer of code 2016

On 22/02/16 23:34, Tom Lane wrote: =?UTF-8?Q?=c3=81lvaro_Hern=c3=a1ndez_Tortosa?= writes: On 22/02/16 05:10, Tom Lane wrote: Another variable is that your answers might depend on what format you assume the client is trying to convert from/to. (It's presumably not text

Re: [HACKERS] about google summer of code 2016

=?UTF-8?Q?=c3=81lvaro_Hern=c3=a1ndez_Tortosa?= writes: > On 22/02/16 05:10, Tom Lane wrote: >> Another variable is that your answers might depend on what format you >> assume the client is trying to convert from/to. (It's presumably not >> text JSON, but then what is it?) >

Re: [HACKERS] about google summer of code 2016

On 22/02/16 05:10, Tom Lane wrote: Heikki Linnakangas writes: On 19/02/16 10:10, �lvaro Hernández Tortosa wrote: Oleg and I discussed recently that a really good addition to a GSoC item would be to study whether it's convenient to have a binary serialization format for

Re: [HACKERS] about google summer of code 2016

On 21/02/16 21:15, Heikki Linnakangas wrote: On 19/02/16 10:10, Álvaro Hernández Tortosa wrote: Oleg and I discussed recently that a really good addition to a GSoC item would be to study whether it's convenient to have a binary serialization format for jsonb over the wire. Some argue

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

Robert Haas writes: >> Foreign tables are supposed to be categorically excluded from >> parallelism. Not sure why that's not working in this instance. BTW, I wonder where you think that's supposed to be enforced, because I sure can't find any such logic. I suppose that

Re: [HACKERS] checkpointer continuous flushing - V18

Random updates on 16 tables which total to 1.1GB of data, so this is in buffer, no significant "read" traffic. (1) with 16 tablespaces (1 per table) on 1 disk : 680.0 tps per second avg, stddev [ min q1 median d3 max ] <=300tps 679.6 ± 750.4 [0.0, 317.0, 371.0, 438.5, 2724.0] 19.5%

Re: [HACKERS] Add generate_series(date,date) and generate_series(date,date,integer)

> > > > > Given that counterexample, I think we not only shouldn't back-patch such > a > > change but should reject it altogether. > > Ouch, good point. The overflows are a different problem that we had > better address though (still on my own TODO list)... > So I should remove the bounds

Re: [HACKERS] psql metaqueries with \gexec

> > In the mean time, update patch attached. > > Really attached this time. diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c index 9750a5b..5ca769f 100644 --- a/src/bin/psql/command.c +++ b/src/bin/psql/command.c @@ -849,6 +849,13 @@ exec_command(const char *cmd, status

Re: [HACKERS] psql metaqueries with \gexec

On Mon, Feb 22, 2016 at 11:30 AM, Corey Huinker wrote: > On Mon, Feb 22, 2016 at 10:08 AM, Daniel Verite > wrote: > >> Corey Huinker wrote: >> >> > ...and query text visibility, and result visibility, and error handling, >> > etc. In

Re: [HACKERS] psql metaqueries with \gexec

Pavel Stehule wrote: > The design of the "format" function is not closed. Try to send prototype > and patch. The possibility to do PostgreSQL customization was strong reason > why we didn't implemented "sprintf" and we implemented "format". Probably not terribly useful here, but for the

Re: [HACKERS] psql metaqueries with \gexec

On Mon, Feb 22, 2016 at 10:08 AM, Daniel Verite wrote: > Corey Huinker wrote: > > > ...and query text visibility, and result visibility, and error handling, > > etc. In this case, we're leveraging the psql environment we'd already set > > up, and if there's an

Re: [HACKERS] checkpointer continuous flushing - V18

On 2016-02-22 11:05:20 -0500, Tom Lane wrote: > Andres Freund writes: > > Interesting. That doesn't reflect my own tests, even on rotating media, > > at all. I wonder if it's related to: > >

Re: [HACKERS] Typo in bufmgr.c that result in waste of memory

On Sat, Feb 20, 2016 at 01:55:55PM +0530, Robert Haas wrote: > On Fri, Feb 19, 2016 at 7:20 PM, Andres Freund wrote: > > On February 19, 2016 2:42:08 PM GMT+01:00, Tom Lane > > wrote: > >>> I think we should fix it, but not backpatch. > >> > >>I don't

Re: [HACKERS] checkpointer continuous flushing - V18

Andres Freund writes: > Interesting. That doesn't reflect my own tests, even on rotating media, > at all. I wonder if it's related to: > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=23d0127096cb91cb6d354bdc71bd88a7bae3a1d5 > If you use your 12.04

Re: [HACKERS] checkpointer continuous flushing - V18

On 2016-02-22 14:11:05 +0100, Fabien COELHO wrote: > > >I did a quick & small test with random updates on 16 tables with > >checkpoint_flush_after=16 checkpoint_timeout=30 > > Another run with more "normal" settings and over 1000 seconds, so less > "quick & small" that the previous one. > >

Re: [HACKERS] WIP: Failover Slots

Hi, > On 16 Feb 2016, at 09:11, Craig Ringer wrote: > > > > Revision attached. There was a file missing from the patch too. > All attached patches apply normally. I only took a look at first 2, but also tried to run the Patroni with the modified version to check

Re: [HACKERS] Writing new unit tests with PostgresNode

Craig Ringer wrote: > > +=pod > > + > > +=head2 Set up a node > > pod format... Do we really want that? Considering that those modules > > are only aimed at being dedicated for in-core testing, I would say no. > > If it's plain comments you have to scan through massive piles of verbose > Perl to

Re: [HACKERS] psql metaqueries with \gexec

Corey Huinker wrote: > ...and query text visibility, and result visibility, and error handling, > etc. In this case, we're leveraging the psql environment we'd already set > up, and if there's an error, \set ECHO queries shows us the errant SQL as > if we typed it ourselves.. BTW, about

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> ... However, this is one of the big problems that >> we'd have to have a solution for before we ever consider allowing >> read-write parallelism. > Having such a blocker for read-write parallelism would be

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

Robert Haas writes: > On Mon, Feb 22, 2016 at 11:02 AM, Thomas Munro > wrote: >> The postgres_fdw failure is a visibility-of-my-own-uncommitted-work >> problem. The first command in a transaction updates a row via an FDW, >> and then the

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

* Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > On Wed, Feb 17, 2016 at 9:48 PM, Tom Lane wrote: > >> I just had a rather disturbing thought to the effect that this entire > >> design --- ie, parallel workers taking out locks for

Re: [HACKERS] FDW: should GetFdwRoutine be called when drop table?

Robert Haas writes: > On Sat, Feb 20, 2016 at 1:43 AM, Tom Lane wrote: >> Yes, that's exactly the problem: you'd need some sort of atomic commit >> mechanism to make this work safely. >> >> It's possible we could give FDWs a bunch of hooks that would

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

Robert Haas writes: > On Mon, Feb 22, 2016 at 2:56 AM, Tom Lane wrote: > !held by the indicated process. False indicates that this process is > !currently waiting to acquire this lock, which implies that at > least one other > !process is

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

Robert Haas writes: > On Wed, Feb 17, 2016 at 9:48 PM, Tom Lane wrote: >> I just had a rather disturbing thought to the effect that this entire >> design --- ie, parallel workers taking out locks for themselves --- is >> fundamentally flawed. As far as

Re: [HACKERS] Support for N synchronous standby servers - take 2

On Tue, Feb 16, 2016 at 4:19 PM, Masahiko Sawada wrote: > On Mon, Feb 15, 2016 at 2:54 PM, Michael Paquier > wrote: >> On Mon, Feb 15, 2016 at 2:11 PM, Kyotaro HORIGUCHI wrote: >>> Surprizingly yes. The list is handled as an identifier list and

Re: [HACKERS] FDW: should GetFdwRoutine be called when drop table?

On Sat, Feb 20, 2016 at 1:43 AM, Tom Lane wrote: > Andres Freund writes: >> On 2016-02-19 14:18:19 -0500, Peter Eisentraut wrote: >>> On 2/19/16 12:21 PM, Feng Tian wrote: I have an fdw that each foreign table will acquire some persisted resource. >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

On Mon, Feb 22, 2016 at 2:56 AM, Tom Lane wrote: > I wrote: >> Robert Haas writes: >>> As for the patch itself, I'm having trouble grokking what it's trying >>> to do. I think it might be worth having a comment defining precisely >>> what we mean by "A

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

On Wed, Feb 17, 2016 at 9:48 PM, Tom Lane wrote: > I just had a rather disturbing thought to the effect that this entire > design --- ie, parallel workers taking out locks for themselves --- is > fundamentally flawed. As far as I can tell from README.parallel, > parallel

Re: [HACKERS] Writing new unit tests with PostgresNode

On 22 February 2016 at 20:21, Michael Paquier wrote: > On Mon, Feb 22, 2016 at 7:55 PM, Craig Ringer > wrote: > > Er, the patch is attached this time. > > top_builddir = ../.. > include $(top_builddir)/src/Makefile.global > > -SUBDIRS =

Re: [HACKERS] checkpointer continuous flushing - V18

I did a quick & small test with random updates on 16 tables with checkpoint_flush_after=16 checkpoint_timeout=30 Another run with more "normal" settings and over 1000 seconds, so less "quick & small" that the previous one. checkpoint_flush_after = 16 checkpoint_timeout = 5min # default

Re: [HACKERS] Writing new unit tests with PostgresNode

On Mon, Feb 22, 2016 at 7:55 PM, Craig Ringer wrote: > Er, the patch is attached this time. top_builddir = ../.. include $(top_builddir)/src/Makefile.global -SUBDIRS = regress isolation modules +SUBDIRS = regress isolation modules example_suite I have no problem with a

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

I did another round of review for the latest patch and well as performed the sanity test, and haven't found any functional issues. Found couple of issue, see in-line comments for the same. On Thu, Feb 18, 2016 at 3:15 PM, Etsuro Fujita wrote: > On 2016/02/12 21:19,

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

On Mon, Feb 22, 2016 at 11:02 AM, Thomas Munro wrote: > On Tue, Feb 16, 2016 at 12:12 PM, Noah Misch wrote: >> On Mon, Feb 15, 2016 at 06:07:48PM -0500, Tom Lane wrote: >>> Noah Misch writes: >>> > I configured a copy of

Re: [HACKERS] Writing new unit tests with PostgresNode

Er, the patch is attached this time. On 22 February 2016 at 18:54, Craig Ringer wrote: > On 22 February 2016 at 15:41, Michael Paquier > wrote: > > >> >> >> > Sound about right? I can tidy that up a bit and turn it into a README >> >> > and >>

Re: [HACKERS] Writing new unit tests with PostgresNode

On 22 February 2016 at 15:41, Michael Paquier wrote: > > >> > Sound about right? I can tidy that up a bit and turn it into a README > >> > and > >> > add a reference to that to the public tap docs to tell users where to > go > >> > if > >> > they want to write more

Re: [HACKERS] psql metaqueries with \gexec

> > FWIW, I also wish we had something better than format() for this stuff. I > did create [1] towards that end, but it currently depends on some C code, > which is cumbersome. For the most party, I'm pretty thrilled with format(), though: - I'll admit to being grumpy about the %1$s notation,

Re: [HACKERS] Relaxing SSL key permission checks

Re: Tom Lane 2016-02-22 <21507.1456099...@sss.pgh.pa.us> > Stephen Frost writes: > > Just to be clear, I'm not really against this patch as-is, but it > > shouldn't be a precedent or limit us from supporting more permissive > > permissions in other areas (or even here) if

Re: [HACKERS] checkpointer continuous flushing - V18

Hallo Andres, AFAICR I used a "flush context" for each table space in some version I submitted, because I do think that this whole writeback logic really does make sense *per table space*, which suggest that there should be as many write backs contexts as table spaces, otherwise the positive

Re: [HACKERS] WIP: SCRAM authentication

On Mon, Feb 15, 2016 at 11:05 AM, Michael Paquier wrote: > On Mon, Feb 15, 2016 at 10:51 AM, Stephen Frost wrote: >> * Tom Lane (t...@sss.pgh.pa.us) wrote: >>> Stephen Frost writes: >>> > Why do we need pg_shadow or pg_user or

Re: [HACKERS] psql metaqueries with \gexec

> > I like what you've proposed, though I am wondering if you considered doing > something server-side instead? It seems a shame to do all this work and > exclude all other tools. > I have, but my solutions closely mirror the one you mention in the next paragraph. > I frequently find myself