Re: [HACKERS] [POC] hash partitioning

On Mon, May 22, 2017 at 2:23 PM, amul sul wrote: > > Updated patch attached. Thanks a lot for review. > Minor fix in the document, PFA. Regards, Amul 0002-hash-partitioning_another_design-v12.patch Description: Binary data 0001-Cleanup_v4.patch Description: Binary data --

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

On Thu, Apr 6, 2017 at 4:37 PM, Tomas Vondra wrote: > Which brings me to the slightly suspicious bit. On 9.5, there's no > difference between GROUP and GROUP+LIKE cases - the estimates are exactly > the same in both cases. This is true too, but only without the

[HACKERS] Commit fests created for PG11 development

Hi all, Following the conclusions of the developer meeting whose minutes are here: https://wiki.postgresql.org/wiki/PgCon_2017_Developer_Meeting The commit fest schedule has been decided as the same as 2016-2017, with four commit fest entries, planned for one month each: - 2017-09 - 2017-11 -

[HACKERS] Server ignores contents of SASLInitialResponse

Hi all, I have noticed today that the server ignores completely the contents of SASLInitialResponse. For example with the patch attached called scram-trick-server: diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c index f4397afc64..8fe1c8edfb 100644 ---

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

On Thu, May 25, 2017 at 11:41:19AM +0900, Michael Paquier wrote: > On Thu, May 25, 2017 at 11:34 AM, Tsunakawa, Takayuki > wrote: > > From: pgsql-hackers-ow...@postgresql.org > >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Noah Misch > >> Ten feels

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

On Thu, May 25, 2017 at 11:34 AM, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Noah Misch >> Ten feels low to me. The value should be be low enough so users don't give >> up and

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Noah Misch > Ten feels low to me. The value should be be low enough so users don't give > up and assume a permanent hang, but there's little advantage to making it > lower. > I'd set it such that

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

On Wed, May 24, 2017 at 09:29:11AM -0400, Michael Paquier wrote: > On Tue, May 23, 2017 at 8:14 AM, Amit Kapila wrote: > > So it seems both you and Tom are leaning towards some sort of retry > > mechanism for shm reattach on Windows. I also think that is a viable > >

Re: [HACKERS] Create subscription with `create_slot=false` and incorrect slot name

On Wed, May 24, 2017 at 9:41 PM, Robert Haas wrote: > On Wed, May 24, 2017 at 7:31 PM, Peter Eisentraut > wrote: >> On 5/23/17 02:33, Kuntal Ghosh wrote: The command succeed even if slot_name is invalid. That's because slot_name

Re: [HACKERS] Get stuck when dropping a subscription during synchronizing table

On Wed, May 24, 2017 at 3:14 PM, Petr Jelinek wrote: > Hi, > > I finally had time to properly analyze this, and turns out we've been > all just trying to fix symptoms and the actual problems. Thank you for the patches! > All the locking works just fine the way it

Re: [HACKERS] Create subscription with `create_slot=false` and incorrect slot name

On Wed, May 24, 2017 at 7:31 PM, Peter Eisentraut wrote: > On 5/23/17 02:33, Kuntal Ghosh wrote: >>> The command succeed even if slot_name is invalid. That's because slot_name >>> isn't validated. ReplicationSlotValidateName() should be called in >>>

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tom Lane > I didn't look at exactly how you tried to do that, but GUCs whose values > depend on other GUCs generally don't work well at all. transaction_read_only and transaction_isolation depends

Re: [HACKERS] pg_dump ignoring information_schema tables which used in Create Publication.

On Wed, May 24, 2017 at 7:16 PM, Peter Eisentraut wrote: > On 5/22/17 07:42, Kuntal Ghosh wrote: >> pg_dump ignores anything created under object name "pg_*" or >> "information_schema". > > Publications have a slightly different definition of what tables to >

Re: [HACKERS] Improvement in log message of logical replication worker

On Wed, May 24, 2017 at 7:04 PM, Peter Eisentraut wrote: > On 5/17/17 23:36, Masahiko Sawada wrote: >> The patch looks good to me. >> There are some log messages saying just 'logical replication worker >> for subscription ...' in reread_subscription but should we

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

From: Robert Haas [mailto:robertmh...@gmail.com] > I've already stated my position on this, which is that: > > * target_session_attrs=read-only is a perfectly good new feature, but we're > past feature freeze, so it's material for v11. > > * I'm not opposed to adding a GUC_REPORT GUC of some

Re: [HACKERS] fix for table syncing with different column order

On 5/18/17 20:30, Peter Eisentraut wrote: > The issues with the different column orders in the regression test > database also revealed that logical replication table syncing was broken > for that case. Here is a fix and a test. committed -- Peter Eisentraut

Re: [HACKERS] Create subscription with `create_slot=false` and incorrect slot name

On 5/23/17 02:33, Kuntal Ghosh wrote: >> The command succeed even if slot_name is invalid. That's because slot_name >> isn't validated. ReplicationSlotValidateName() should be called in >> parse_subscription_options() to avoid a pilot error. IMHO we should prevent >> a future error (use of invalid

Re: [HACKERS] ALTER PUBLICATION materializing the list of tables

On 5/22/17 18:08, Jeff Janes wrote: > If I create a publication FOR ALL TABLES and then change my mind, the > only thing I can do is drop the publication and recreate it. > > Since "ALTER PUBLICATION name SET TABLE" allows you to replace the > entire table list, shouldn't it also let you change

Re: [HACKERS] ALTER PUBLICATION documentation

On 5/22/17 17:50, Jeff Janes wrote: > "The first variant of this command listed in the synopsis can change all > of the publication properties specified in CREATE PUBLICATION > ." > > That referenced first variant no longer

Re: [HACKERS] pg_dump ignoring information_schema tables which used in Create Publication.

On 5/22/17 07:42, Kuntal Ghosh wrote: > pg_dump ignores anything created under object name "pg_*" or > "information_schema". Publications have a slightly different definition of what tables to ignore/prohibit than pg_dump, partly because they have more built-in knowledge. I'm not sure whether

Re: [HACKERS] Improvement in log message of logical replication worker

On 5/20/17 00:58, Alvaro Herrera wrote: > Umm, just skimming here -- this patch shows some LOG messages using > elog() rather than ereport(), which seems bogus to me. Fixed that. > Also: > "logical replication table synchronization worker for subscription > \"%s\", table \"%s\" has

Re: [HACKERS] Improvement in log message of logical replication worker

On 5/17/17 23:36, Masahiko Sawada wrote: > The patch looks good to me. > There are some log messages saying just 'logical replication worker > for subscription ...' in reread_subscription but should we add 'apply' > to these messages in order for user to distinguish between apply > worker and

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

Robert Haas writes: > On Wed, May 24, 2017 at 3:16 AM, Tsunakawa, Takayuki > wrote: >> For this, I added a GUC_REPORT variable session_read_only which indicates >> the session's default read-only status. The characteristics are: >> >> *

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

On Wed, May 24, 2017 at 3:16 AM, Tsunakawa, Takayuki wrote: > I confirmed that the attached patch successfully provides: > > * target_session_attrs=read-only > * If the server is >= 10, avoid the round-trip for SHOW transaction_read_only. > > For this, I added a

Re: [HACKERS] unreachable code in partition.c

On Tue, May 23, 2017 at 2:06 PM, Jeevan Ladhe wrote: > Following code in function get_qual_for_list(partition.c) is not reachable. > > else > result = list_make1(opexpr); > > Attached is the patch that removes this dead code. Committed, but I added an Assert() as

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

On Wed, May 24, 2017 at 6:59 PM, Michael Paquier wrote: > On Tue, May 23, 2017 at 8:14 AM, Amit Kapila wrote: >> So it seems both you and Tom are leaning towards some sort of retry >> mechanism for shm reattach on Windows. I also think that is

Re: [HACKERS] pg_upgrade translation

Robert Haas writes: > On Mon, May 22, 2017 at 8:28 PM, Alvaro Herrera > wrote: >> Translatability was turned on for pg_upgrade on pg10, but it needs some >> work. The choices are we fix it now, or we revert the addition of NLS >> there and we do

Re: [HACKERS] pg_upgrade translation

On Mon, May 22, 2017 at 8:28 PM, Alvaro Herrera wrote: > Translatability was turned on for pg_upgrade on pg10, but it needs some > work. The choices are we fix it now, or we revert the addition of NLS > there and we do it for PG11. > > I think the ugliest one is to

Re: [HACKERS] ALTER SUBSCRIPTION ..SET PUBLICATION refresh is not throwing error.

On 24/05/17 21:28, Petr Jelinek wrote: > On 24/05/17 21:21, Petr Jelinek wrote: >> On 24/05/17 21:07, Euler Taveira wrote: >>> 2017-05-23 6:00 GMT-03:00 tushar >> >: >>> >>> >>> s=# alter subscription s1 set publication

Re: [HACKERS] Tightening isspace() tests where behavior should match SQL parser

I wrote: > Heikki Linnakangas writes: >> +1 for back-patching. If I understand correctly, it would change the >> behavior when you pass a string with non-ASCII leading or trailing >> whitespace characters to those functions. I suppose that can happen, but >> it's only pure

Re: [HACKERS] ALTER SUBSCRIPTION ..SET PUBLICATION refresh is not throwing error.

On 24/05/17 21:21, Petr Jelinek wrote: > On 24/05/17 21:07, Euler Taveira wrote: >> 2017-05-23 6:00 GMT-03:00 tushar > >: >> >> >> s=# alter subscription s1 set publication skip refresh ; >> NOTICE: removed

Re: [HACKERS] ALTER SUBSCRIPTION ..SET PUBLICATION refresh is not throwing error.

On 24/05/17 21:07, Euler Taveira wrote: > 2017-05-23 6:00 GMT-03:00 tushar >: > > > s=# alter subscription s1 set publication skip refresh ; > NOTICE: removed subscription for table public.t > NOTICE: removed

Re: [HACKERS] Get stuck when dropping a subscription during synchronizing table

Hi, I finally had time to properly analyze this, and turns out we've been all just trying to fix symptoms and the actual problems. All the locking works just fine the way it is in master. The issue with deadlock with apply comes from the wrong handling of the SIGTERM in the apply (we didn't set

Re: [HACKERS] ALTER SUBSCRIPTION ..SET PUBLICATION refresh is not throwing error.

2017-05-23 6:00 GMT-03:00 tushar : > > s=# alter subscription s1 set publication skip refresh ; > NOTICE: removed subscription for table public.t > NOTICE: removed subscription for table public.t1 > ALTER SUBSCRIPTION > s=# That's a design flaw. Since SKIP is

Re: [HACKERS] generate_series regression 9.6->10

Thanks Tom. This showed up in a regression test of ours that built the test data using generate_series, so it's not a critical production issue or anything, just a surprise change in behaviour. P. On Wed, May 24, 2017 at 10:28 AM, Tom Lane wrote: > Paul Ramsey

Re: [HACKERS] translatable string fixes

Alvaro Herrera writes: > It took me a very long time to figure out how to translate these 9.6-new > strings in the AM validate routines: > msgid "gin operator family \"%s\" contains support procedure %s with > cross-type registration" > ... > Maybe we can use this

Re: [HACKERS] generate_series regression 9.6->10

Paul Ramsey writes: > The behaviour of generate_series seems to have changed a little, at least > in conjunction w/ CTEs. What's changed is the behavior of multiple SRFs in a SELECT's targetlist, cf

Re: [HACKERS] generate_series regression 9.6->10

On 2017-05-24 10:09:19 -0700, Paul Ramsey wrote: > The behaviour of generate_series seems to have changed a little, at least > in conjunction w/ CTEs. Under 9.6 (and prior) this query returns 2127 rows, > with no nulls: > > with > ij as ( select i, j from generate_series(1, 10) i,

[HACKERS] generate_series regression 9.6->10

The behaviour of generate_series seems to have changed a little, at least in conjunction w/ CTEs. Under 9.6 (and prior) this query returns 2127 rows, with no nulls: with ij as ( select i, j from generate_series(1, 10) i, generate_series(1, 10) j), iijj as (select generate_series(1, i) as a,

Re: [HACKERS] 10beta1 sequence regression failure on sparc64

Re: To Andres Freund 2017-05-18 <20170518192924.jkrzevlencp3g...@msg.df7cb.de> > > If we had a typo or something in that code, the build farm should have > > caught it by now. > > > > I would try compiling with lower -O and see what happens. > > Trying -O0 now. Sorry for the late reply, I was

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

On Mon, May 22, 2017 at 7:51 PM, Mark Rofail wrote: > Cloned the git repo found @ https://github.com/postgres/postgres and > identified the main two files I will be concerned with. (I know I may need > to edit other files but these seem to where I will spend most of my

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

On Wed, May 24, 2017 at 10:32 AM, Andres Freund wrote: > Well, but then we should just remove minval/maxval if we can't rely on > it. That seems like a drastic overreaction to me. > I wonder if that's not actually very little new code, and I think we > might end up

Re: [HACKERS] UPDATE of partition key

On Wed, May 24, 2017 at 8:14 PM, Amit Kapila wrote: > On Wed, May 24, 2017 at 2:47 PM, Amit Khandekar > wrote: >> >> By now, majority of the opinions have shown that they do not favour >> two triggers getting fired on a single update. Amit, do

Re: [HACKERS] UPDATE of partition key

On Wed, May 24, 2017 at 2:47 PM, Amit Khandekar wrote: > On 18 May 2017 at 16:52, Amit Kapila wrote: >> On Wed, May 17, 2017 at 4:05 PM, Dilip Kumar wrote: >>> On Wed, May 17, 2017 at 3:15 PM, Amit Kapila

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

On 2017-05-24 10:24:19 -0400, Robert Haas wrote: > On Wed, May 24, 2017 at 9:04 AM, Andres Freund wrote: > > At the very least we'll have to error out. That's still not nice usability > > wise, but it sure beats returning flat out wrong values. > > I'm not sure. That seems

[HACKERS] Error log for psql (uploading backup) in PostgreSQL 9.3.17

Hi, I should say that I'm haviing this issue since 9.3.2 I though it would be resolved when I upgrade to 9.3.17 but this still appear. This is a very wierd issue.   I upload backups using pg_restore and psql -h SERVER -U USERNAME -f backup.sql -q -d databasename -pPORT 2>errors.txt >output.txt  

Re: [HACKERS] Error-like LOG when connecting with SSL for password authentication

On Tue, May 23, 2017 at 6:26 AM, Heikki Linnakangas wrote: > Yeah. The be_tls_read() change looks OK to me. > > Can SSL_ERROR_ZERO_RETURN also happen in a write? I suppose it can, but > returning 0 from secure_write() seems iffy. It seems to me that it could be the case, the man

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

On Wed, May 24, 2017 at 9:04 AM, Andres Freund wrote: > At the very least we'll have to error out. That's still not nice usability > wise, but it sure beats returning flat out wrong values. I'm not sure. That seems like it might often be worse. Now you need manual

Re: [HACKERS] Index created in BEFORE trigger not updated during INSERT

On 2017-05-24 08:26:24 -0400, Robert Haas wrote: > On Mon, May 22, 2017 at 7:05 AM, Albe Laurenz wrote: > > Not that it is a useful use case, but I believe that this is > > a bug that causes index corruption: > > > > CREATE TABLE mytable( > >id integer PRIMARY KEY, >

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

On Tue, May 23, 2017 at 8:14 AM, Amit Kapila wrote: > So it seems both you and Tom are leaning towards some sort of retry > mechanism for shm reattach on Windows. I also think that is a viable > option to negate the impact of ASLR. Attached patch does that. Note >

Re: [pgsql-translators] [HACKERS] translatable string fixes

On Tue, May 23, 2017 at 1:15 AM, Alvaro Herrera wrote: > It took me a very long time to figure out how to translate these 9.6-new > strings in the AM validate routines: > > msgid "gin operator family \"%s\" contains support procedure %s with > cross-type registration" >

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

On May 24, 2017 6:57:08 AM EDT, Robert Haas wrote: >On Tue, May 23, 2017 at 11:25 PM, Andres Freund >wrote: >> On 2017-05-23 22:47:07 -0400, Robert Haas wrote: >>> On Mon, May 22, 2017 at 11:42 AM, Andres Freund >wrote: >>> >

Re: [HACKERS] Is it possible to get query_string value in an event trigger?

:On Mon, May 22, 2017 at 2:18 PM, Jeremy Finzel wrote: > Hello. Is it possible to get the currently executing query in an event > trigger, for example, a create table event trigger function firing after > ddl_command_end, aside from checking pg_stat_activity for the current >

Re: [HACKERS] Broken hint bits (freeze)

On Wed, May 24, 2017 at 7:27 AM, Dmitriy Sarafannikov wrote: > It seems like replica did not replayed corresponding WAL records. > Any thoughts? heap_xlog_freeze_page() is a pretty simple function. It's not impossible that it could have a bug that causes it to

Re: [HACKERS] wal_level > WAL_LEVEL_LOGICAL

On Mon, May 22, 2017 at 9:08 AM, Neha Khatri wrote: > As per my understabding, current postgres server supports only three > values for wal_level i.e. 'minimal' , 'replica' or 'logical'. But > following error message brought to notice that there are various code > spots

Re: [HACKERS] Index created in BEFORE trigger not updated during INSERT

On Mon, May 22, 2017 at 7:05 AM, Albe Laurenz wrote: > Not that it is a useful use case, but I believe that this is > a bug that causes index corruption: > > CREATE TABLE mytable( >id integer PRIMARY KEY, >id2 integer NOT NULL > ); > > CREATE FUNCTION makeindex()

Re: [HACKERS] Proposal : For Auto-Prewarm.

On Wed, May 24, 2017 at 6:28 AM, Mithun Cy wrote: > On Tue, May 23, 2017 at 7:06 PM, Mithun Cy wrote: >> Thanks, Andres, >> >> I have tried to fix all of your comments. > > There was a typo issue in previous patch 07 where instead of == I

Re: [HACKERS] Broken hint bits (freeze)

We found that this problem appears also on shards with enabled checksums. This shard has 1st timeline, which means there was no switchover after upgrade to 9.6. xdb11f(master)=# select pg_current_xlog_location(), pg_xlogfile_name(pg_current_xlog_location()); pg_current_xlog_location |

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

On Tue, May 23, 2017 at 11:25 PM, Andres Freund wrote: > On 2017-05-23 22:47:07 -0400, Robert Haas wrote: >> On Mon, May 22, 2017 at 11:42 AM, Andres Freund wrote: >> > Ooops. >> > >> > Two issues: Firstly, we get a value smaller than seqmin, obviously >>

Re: [HACKERS] UPDATE of partition key

On 12 May 2017 at 09:27, Amit Kapila wrote: > > + is_partitioned_table = > + root_rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE; > + > + if (is_partitioned_table) > + ExecSetupPartitionTupleRouting( > + root_rel, > + /* Build WITH CHECK OPTION constraints for leaf

[HACKERS] server closed the connection message in the log file of standby while performing logical replication synchronization

Hi, I am getting server closed the connection message in the log file of standby after 'logical replication synchronization worker finished processing' LOG message 2017-05-24 08:58:04.451 BST [28496] LOG: logical replication sync for subscription s_5434, table t1039 started 2017-05-24

Re: [HACKERS] Proposal : For Auto-Prewarm.

On Tue, May 23, 2017 at 7:06 PM, Mithun Cy wrote: > Thanks, Andres, > > I have tried to fix all of your comments. There was a typo issue in previous patch 07 where instead of == I have used !=. And, a mistake in comments. I have corrected same now. -- Thanks and

[HACKERS] CentOS based openshift ready docker container

Hi I have successfully built postgresql containers, that work on openshift[1], available in my git repository[2]. These containers are currently building on CentOS Container Pipeline[3] and being delivered to registry.centos.org. I am proposing that they be added to the postgresql

[HACKERS] CentOS based openshift ready docker container

Hi I have successfully built postgresql containers, that work on openshift[1], available in my git repository[2]. These containers are currently building on CentOS Container Pipeline[3] and being delivered to registry.centos.org. I am proposing that they be added to the postgresql

Re: [HACKERS] UPDATE of partition key

On 18 May 2017 at 16:52, Amit Kapila wrote: > On Wed, May 17, 2017 at 4:05 PM, Dilip Kumar wrote: >> On Wed, May 17, 2017 at 3:15 PM, Amit Kapila wrote: Earlier I thought that option1 is better but later I think that

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Craig Ringer > On 13 April 2017 at 14:59, Tsunakawa, Takayuki > wrote: > > > 2. Make transaction_read_only GUC_REPORT This is to avoid the added > > round-trip by

Re: [HACKERS] pgbench more operators & functions

Hello Pavel, I am watching this patch - it looks so there are not problems. Great. I found only issue in documentation - new CASE expression is not documented. Hmmm. Actually there was a rather very discreet one in v10: + SQL CASE generic conditional expressions I've improved it

Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)

One solution that is striking me is 1. I'll create one background worker and will initialise DSA in it. 2. If there are any callbacks available for client open/close connections, i'll attach/detach to the DSA in those callbacks. But i'm not sure there are such callbacks available. If such

Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)

Hi, As Michael said, i'm creating DSA too early. Shared_Preload libraries are loading prior to memory related stuff. But i'm totally clueless how to solve my use case. Please help me with any work around. Thanks & Best Regards, - Mahi On Tue, May 23, 2017 at 5:52 PM, Mahi Gurram