Re: [HACKERS] Printing bitmap objects in the debugger

> > I'd suggest that this is parallel to nodeToString() and therefore > (a) should be placed beside it, Done. Added it after nodeToString(). > (b) should be named like it, > bmsToString() perhaps, bmsToString() is fine. Used that name. > and (c) should look more like it internally. > Done. I

Re: [HACKERS] Hash Indexes

On Thu, Sep 15, 2016 at 10:38 PM, Jesper Pedersen wrote: > On 09/15/2016 02:03 AM, Amit Kapila wrote: >>> >>> Same thing here - where the fields involving the hash index aren't >>> updated. >>> >> >> Do you mean that for such cases also you see 40-60% gain? >> > > No,

Re: [HACKERS] Stopping logical replication protocol

>Have you had a chance to look at adding the tests we discussed? Not yet. I plane do it on this weekends 2016-09-16 4:37 GMT+03:00 Craig Ringer : > On 9 September 2016 at 12:03, Craig Ringer wrote: > > > Setting "waiting on author" in CF per

Re: [HACKERS] patch: function xmltable

2016-09-16 1:44 GMT+02:00 Craig Ringer : > On 15 September 2016 at 19:31, Pavel Stehule > wrote: > > > b_expr enforces shift/reduce conflict :( > > No problem then. I just thought it'd be worth allowing more if it > worked to do so. > > > I found

Re: [HACKERS] Tackling JsonPath support

Hi 2016-09-15 18:05 GMT+02:00 Christian Convey : > On Mon, Sep 5, 2016 at 1:44 PM, Pavel Stehule > wrote: > ... > > > I wrote XMLTABLE function, and I am thinking about JSON_TABLE function. > But > > there is one blocker - missing JsonPath

Re: [HACKERS] Hash Indexes

On Thu, Sep 15, 2016 at 7:53 PM, Andres Freund wrote: > Hi, > > On 2016-05-10 17:39:22 +0530, Amit Kapila wrote: >> For making hash indexes usable in production systems, we need to improve >> its concurrency and make them crash-safe by WAL logging them. > > One earlier

Re: [HACKERS] Hash Indexes

On 16/09/16 18:35, Amit Kapila wrote: On Thu, Sep 15, 2016 at 7:53 PM, Andres Freund wrote: Hi, On 2016-05-10 17:39:22 +0530, Amit Kapila wrote: For making hash indexes usable in production systems, we need to improve its concurrency and make them crash-safe by WAL

[HACKERS] Why postgres take RowExclusiveLock on all partition

Hi Hackers, I checked if there is update transaction on master table involved in partition. Postgresql takes RowExclusiveLock on all partition tables. constraint exclusion is set to on. My question is why it locks on all partition tables instead only one partition tables where data is

Re: [HACKERS] standalone backend PANICs during recovery

--On 3. September 2016 um 02:05:00 -0400 Tom Lane wrote: >> Well, the "use case" was a crashed hot standby at a customer site (it >> kept PANICing after restarting), where i decided to have a look through >> the recovery process with gdb. The exact PANIC was > >>

Re: [HACKERS] Block level parallel vacuum WIP

On Thu, Sep 15, 2016 at 11:44 PM, Robert Haas wrote: > On Thu, Sep 15, 2016 at 7:21 AM, Masahiko Sawada > wrote: >> I'm implementing this patch but I need to resolve the problem >> regarding lock for extension by multiple parallel workers. >> In

[HACKERS] README of hash index

Currently README of hash module contain algorithms written in below form. The insertion algorithm is rather similar: pin meta page and take buffer content lock in shared mode loop: compute bucket number for target hash key release meta page buffer content lock if (correct bucket page is already

Re: [HACKERS] more parallel query documentation

On Thu, Apr 21, 2016 at 9:16 PM, Amit Langote wrote: > On 2016/04/15 12:02, Robert Haas wrote: >> As previously threatened, I have written some user documentation for >> parallel query. I put it up here: >> >> https://wiki.postgresql.org/wiki/Parallel_Query >> >>

[HACKERS] recovery_min_apply-delay and remote_apply

Current PostgreSQL Documentation on recovery.conf has this about recovery_min_apply_delay[1]: ---<--- This parameter is intended for use with streaming replication deployments; however, if the parameter is specified it will be honored in all cases. Synchronous replication is not affected by this

[HACKERS] Reminder: Call for Papers -- PGDay Austin 2016

Just a reminder / heads-up for those who don't follow -announce. Forwarded Message Subject:[ANNOUNCE] Call for Papers -- PGDay Austin 2016 Date: Wed, 7 Sep 2016 19:54:22 + From: Debra Cerda To: pgsql-annou...@postgresql.org

Re: [HACKERS] Reminder: Call for Papers -- PGDay Austin 2016

Hi, On 2016-09-16 16:22:39 -0500, Jim Nasby wrote: > Just a reminder / heads-up for those who don't follow -announce. Uh, I'm against making hackers a duplicate of announce. If people don't follow announce that's their choice. Andres -- Sent via pgsql-hackers mailing list

Re: [HACKERS] recovery_min_apply-delay and remote_apply

On Sat, Sep 17, 2016 at 8:45 AM, Bernd Helmle wrote: > Current PostgreSQL Documentation on recovery.conf has this about > recovery_min_apply_delay[1]: > > ---<--- > > This parameter is intended for use with streaming replication deployments; > however, if the parameter is

Re: [HACKERS] more parallel query documentation

Robert Haas wrote: > Hey, everybody: I intended to add this to the documentation before 9.6 > went out, but that didn't get done. Maybe it'll have to happen later > at this point, but can I get some advice on WHERE in the documentation > this stuff could be added? Assuming people agree it

Re: [HACKERS] Hash Indexes

On 2016-09-16 09:12:22 -0700, Jeff Janes wrote: > On Thu, Sep 15, 2016 at 7:23 AM, Andres Freund wrote: > > One earlier question about this is whether that is actually a worthwhile > > goal. Are the speed and space benefits big enough in the general case? > > Could those

Re: [HACKERS] Rename max_parallel_degree?

On 16/09/2016 20:24, Robert Haas wrote: > On Wed, Jun 29, 2016 at 10:46 PM, Amit Kapila wrote: >> Your patch looks good to me and is ready for a committer's look. >> >> Notes for committer - >> a. Verify if description of newly added Guc max_parallel_workers looks >> okay

Re: [HACKERS] WIP: About CMake v2

On Sat, Sep 17, 2016 at 1:40 AM, Yury Zhuravlev wrote: > Michael Paquier wrote: > I merged master to my branch and I spent time to porting all changes. I hope > send patch in the weekend without terrible flaws. By the way, I noticed that you did not register this

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 09/17/2016 05:23 AM, Amit Kapila wrote: On Sat, Sep 17, 2016 at 6:54 AM, Tomas Vondra wrote: On 09/14/2016 06:04 PM, Dilip Kumar wrote: ... (I've also ran it with 100M rows, called "large" in the results), and pgbench is running this transaction: \set

Re: [HACKERS] WIP: About CMake v2

Hi, On 2016-09-16 22:33:50 +0900, Michael Paquier wrote: > As supporting all platforms at once with cmake is going to be > uncommitable, we are going to need both methods able to live together > for a while. I very strongly disagree with this. It's way too likely that we end up releasing with

Re: [HACKERS] WIP: About CMake v2

On Sat, Sep 17, 2016 at 8:11 AM, Andres Freund wrote: > Hi, > > On 2016-09-16 22:33:50 +0900, Michael Paquier wrote: >> As supporting all platforms at once with cmake is going to be >> uncommitable, we are going to need both methods able to live together >> for a while. > > I

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Sat, Sep 17, 2016 at 6:54 AM, Tomas Vondra wrote: > On 09/14/2016 06:04 PM, Dilip Kumar wrote: >> >> On Wed, Sep 14, 2016 at 8:59 PM, Robert Haas >> wrote: >>> >>> Sure, but you're testing at *really* high client counts here. Almost >>>

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 09/14/2016 06:04 PM, Dilip Kumar wrote: On Wed, Sep 14, 2016 at 8:59 PM, Robert Haas wrote: Sure, but you're testing at *really* high client counts here. Almost nobody is going to benefit from a 5% improvement at 256 clients. I agree with your point, but here we

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 09/14/2016 05:29 PM, Robert Haas wrote: On Wed, Sep 14, 2016 at 12:55 AM, Dilip Kumar wrote: 2. Results ./pgbench -c $threads -j $threads -T 10 -M prepared postgres -f script.sql scale factor: 300 Clients head(tps)grouplock(tps) granular(tps)

Re: [HACKERS] Why postgres take RowExclusiveLock on all partition

On Fri, Sep 16, 2016 at 4:31 PM, Sachin Kotwal wrote: > Hi Hackers, > > > I checked if there is update transaction on master table involved in > partition. > Postgresql takes RowExclusiveLock on all partition tables. > > constraint exclusion is set to on. I checked this

Re: [HACKERS] Printing bitmap objects in the debugger

Ashutosh Bapat writes: >> I'd suggest that this is parallel to nodeToString() and therefore >> (a) should be placed beside it, > Done. Added it after nodeToString(). Pushed, thanks. regards, tom lane -- Sent via pgsql-hackers mailing

Re: [HACKERS] Parallel sec scan in plpgsql

On Fri, Sep 16, 2016 at 6:57 PM, Alex Ignatov wrote: > No it doesn't. > Paralleling neither sql function nor plpgsql: > Here is example : > > ipdr=> show max_worker_processes ; > max_worker_processes > -- > 128 > (1 row) > ipdr=> set

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Fri, Sep 16, 2016 at 9:47 AM, Pavan Deolasee wrote: > On Fri, Sep 16, 2016 at 7:03 PM, Robert Haas wrote: >> On Thu, Sep 15, 2016 at 11:39 PM, Pavan Deolasee >> wrote: >> > But I actually wonder if we are over

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

On Fri, Sep 16, 2016 at 9:45 AM, Jeevan Chalke wrote: > 6. Per my understanding, I think checking for just aggregate function is > enough as we are interested in whole aggregate result. +1. > Meanwhile I will > check whether we need to find and check shippability

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Fri, Sep 16, 2016 at 7:03 PM, Robert Haas wrote: > On Thu, Sep 15, 2016 at 11:39 PM, Pavan Deolasee > wrote: > > But I actually wonder if we are over engineering things and > overestimating > > cost of memmove etc. How about this simpler

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

On Mon, Sep 12, 2016 at 5:19 PM, Jeevan Chalke < jeevan.cha...@enterprisedb.com> wrote: > > > On Mon, Sep 12, 2016 at 12:20 PM, Prabhat Sahu < > prabhat.s...@enterprisedb.com> wrote: > >> Hi, >> >> While testing "Aggregate pushdown", i found the below error: >> -- GROUP BY alias showing different

Re: [HACKERS] Why postgres take RowExclusiveLock on all partition

Hi Tom, Thanks for reply. To take decision it should get locks for very small interval. Does it release locks after taking decision and then perform actual update operation on partition table? I feel update operation can take longer time than planner to examine and will not require lock in

Re: [HACKERS] WAL consistency check facility

On Fri, Sep 16, 2016 at 10:30 PM, Robert Haas wrote: > I don't think you have the right to tell Kuntal that he has to move > the patch to the next CommitFest because there are unspecified things > about the current version you don't like. If you don't have time to > review

Re: [HACKERS] Why postgres take RowExclusiveLock on all partition

Sachin Kotwal writes: > In another Terminal : > postgres=# select locktype, database::regclass , > relation::regclass,virtualtransaction, pid, mode , granted from pg_locks > where locktype='relation'; > locktype | database | relation | virtualtransaction | pid |

Re: [HACKERS] [PROPOSAL][PROTOTYPE] Individual options for each index column: Opclass options

On Tue, May 24, 2016 at 10:06 AM, Nikolay Shaplov wrote: > So adding options for opclass seems to be really good idea. > To see how it works you should do the following: > > # create extension intarray ; > # create table test (i int[]); > # create table test2 (i int[]);

Re: [HACKERS] Why postgres take RowExclusiveLock on all partition

Sachin Kotwal writes: > Does it release locks after taking decision and then perform actual update > operation on partition table? No, there's no attempt to do that, and we're unlikely to consider doing so because it would result in more lock-table thrashing. Why do you

Re: [HACKERS] Why postgres take RowExclusiveLock on all partition

Hi Ashutosh, Thanks for reply. Below are my findings: In 1 Terminal: postgres=# create table t1 (a int, b int); CREATE TABLE postgres=# show constraint_exclusion ; constraint_exclusion -- partition (1 row) postgres=# create table t1_p1() inherits (t1); CREATE TABLE

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

Re: To Heikki Linnakangas 2016-09-15 <20160915213406.2mjlhcg7px3sa...@msg.df7cb.de> > > Can you elaborate? Are you saying that Debian 9 (strect) will not ship > > OpenSSL 1.0.2 anymore, and will require using OpenSSL 1.1.0? > > I thought that was the plan, but upon asking on #debian-devel, it >

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Fri, Sep 9, 2016 at 3:17 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > Hi All, > > PFA the patch to support partition-wise joins for partitioned tables. The > patch > is based on the declarative parition support patches provided by Amit > Langote > on 26th August 2016. > I

Re: [HACKERS] README of hash index

On Fri, Sep 16, 2016 at 04:50:53PM +0530, Amit Kapila wrote: > Currently README of hash module contain algorithms written in below form. > > The insertion algorithm is rather similar: > > pin meta page and take buffer content lock in shared mode > loop: > compute bucket number for target hash

Re: [HACKERS] Parallel sec scan in plpgsql

No it doesn't. Paralleling neither sql function nor plpgsql: Here is example : ipdr=> show max_worker_processes ; max_worker_processes -- 128 (1 row) ipdr=> set max_parallel_workers_per_gather to 128; SET ipdr=> set force_parallel_mode=on; SET ipdr=> set

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Thu, Sep 15, 2016 at 11:39 PM, Pavan Deolasee wrote: > But I actually wonder if we are over engineering things and overestimating > cost of memmove etc. How about this simpler approach: Don't forget that you need to handle the case where maintenance_work_mem is quite

Re: [HACKERS] WAL consistency check facility

On Thu, Sep 15, 2016 at 9:23 PM, Michael Paquier wrote: > On Thu, Sep 15, 2016 at 7:30 PM, Kuntal Ghosh > wrote: >> Thoughts? > > There are still a couple of things that this patch makes me unhappy, > particularly the handling of the GUC

[HACKERS] Improvements in psql hooks for variables

Hi, Following the discussion on forbidding an AUTOCOMMIT off->on switch mid-transaction [1], attached is a patch that let the hooks return a boolean indicating whether a change is allowed. Using the hooks, bogus assignments to built-in variables can be dealt with more strictly. For example,

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

On Thu, Sep 15, 2016 at 8:52 PM, Craig Ringer wrote: > On 2 September 2016 at 23:29, Petr Jelinek wrote: > >> You could put it to txid.c where all the other txid stuff is in? > > Yeah, even though it's in adt/ I think it'll do. > > I thought I'd need

Re: [HACKERS] WIP: About CMake v2

On Fri, Sep 16, 2016 at 9:58 AM, Michael Paquier wrote: > Okay. That sounds good to me. I donas well a look around, and cmake really > looks like a robust alternative to ./configure. Now I am aware of the fact > that your patch recommends to build 't recall what your

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Sat, Sep 17, 2016 at 9:17 AM, Tomas Vondra wrote: > On 09/14/2016 05:29 PM, Robert Haas wrote: >> >> On Wed, Sep 14, 2016 at 12:55 AM, Dilip Kumar >> wrote: >>> >>> 2. Results >>> ./pgbench -c $threads -j $threads -T 10 -M prepared postgres

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Sat, Sep 17, 2016 at 6:54 AM, Tomas Vondra wrote: > Although most of the changes probably does not matter much for unlogged > tables (I planned to see how this affects regular tables, but as I see no > difference for unlogged ones, I haven't done that yet). > > So

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Sat, Sep 17, 2016 at 9:12 AM, Tomas Vondra wrote: > On 09/17/2016 05:23 AM, Amit Kapila wrote: >> >> The difference between these and tests performed by Dilip is that he >> has lesser savepoints. I think if you want to try it again, then can >> you once do it

Re: [HACKERS] Parallel sec scan in plpgsql

On 16.09.2016 16:50, Amit Kapila wrote: On Fri, Sep 16, 2016 at 6:57 PM, Alex Ignatov wrote: No it doesn't. Paralleling neither sql function nor plpgsql: Here is example : ipdr=> show max_worker_processes ; max_worker_processes -- 128 (1 row)

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

On Thu, Sep 15, 2016 at 04:24:07PM -0400, Tom Lane wrote: > Marco Pfatschbacher writes: > > the current implementation of PostmasterIsAlive() uses a pipe to > > monitor the existence of the postmaster process. > > One end of the pipe is held open in the postmaster,

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

On Thu, Sep 15, 2016 at 04:40:00PM -0400, Tom Lane wrote: > Thomas Munro writes: > > Very interesting. Perhaps that is why NetBSD shows a speedup with the > > kqueue patch[1] but FreeBSD doesn't. I guess that if I could get the > > kqueue patch to perform better

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

On Thu, Sep 15, 2016 at 12:26:16PM -0700, Andres Freund wrote: > Hi, > > On 2016-09-15 15:57:55 +0200, Marco Pfatschbacher wrote: > > the current implementation of PostmasterIsAlive() uses a pipe to > > monitor the existence of the postmaster process. > > One end of the pipe is held open in the

Re: [HACKERS] README of hash index

On Fri, Sep 16, 2016 at 4:20 AM, Amit Kapila wrote: > Currently README of hash module contain algorithms written in below form. > > The insertion algorithm is rather similar: > > pin meta page and take buffer content lock in shared mode > loop: > compute bucket number

Re: [HACKERS] Why postgres take RowExclusiveLock on all partition

Hi Tom, What I understood from this https://www.postgresql.org/docs/9.5/static/explicit-locking.html#TABLE-LOCK-COMPATIBILITY is : The RowExclusiveLock conflicts with queries want SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE ACCESS EXCLUSIVE locks. In one of our customer environment we want do some

Re: [HACKERS] Hash Indexes

On Thu, Sep 15, 2016 at 7:23 AM, Andres Freund wrote: > Hi, > > On 2016-05-10 17:39:22 +0530, Amit Kapila wrote: > > For making hash indexes usable in production systems, we need to improve > > its concurrency and make them crash-safe by WAL logging them. > > One earlier

Re: [HACKERS] Bug in to_timestamp().

On 25.08.2016 13:26, amul sul wrote: Thanks. I've created the entry in https://commitfest.postgresql.org/10/713/ . You can add yourself as a reviewer. Done, added myself as reviewer & changed status to "Ready for Committer". Thanks ! Regards, Amul Sul It seems there is no need to rebase

Re: [HACKERS] WIP: About CMake v2

Michael Paquier wrote: Your patch recommends to build with cmake after creating build/, now I would expect most users to run cmake from the root folder. However this causes all the Makefiles to get overwritten. As supporting all platforms at once with cmake is going to be uncommitable, we are

Re: [HACKERS] Hash Indexes

On 09/16/2016 03:18 AM, Amit Kapila wrote: Attached is a run with 1000 rows. I think 1000 is also less, you probably want to run it for 100,000 or more rows. I suspect that the reason why you are seeing the large difference between btree and hash index is that the range of values is narrow

Re: [HACKERS] Speedup twophase transactions

> On 07 Sep 2016, at 11:07, Stas Kelvich wrote: > >> On 07 Sep 2016, at 03:09, Michael Paquier wrote: >> On 06 Sep 2016, at 12:03, Michael Paquier wrote: On Tue, Sep 6, 2016 at 5:58 PM, Stas

Re: [HACKERS] Quorum commit for multiple synchronous replication.

On Fri, Sep 9, 2016 at 6:23 PM, Simon Riggs wrote: > On 8 September 2016 at 10:26, Masahiko Sawada wrote: > >> "k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward >> compatibility but most users would think "k(n1, n2, n3)" as quorum >>

[HACKERS] design for a partitioning feature (was: inheritance)

On Mon, May 23, 2016 at 6:31 PM, Alvaro Herrera wrote: > Tom Lane wrote: >> My feeling about it is that we need to provide a partitioning feature >> that doesn't rely on the current notion of inheritance at all. We've >> heard from multiple users who want to use large

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

Hi, On 2016-09-16 09:55:48 +0200, Marco Pfatschbacher wrote: > On Thu, Sep 15, 2016 at 12:26:16PM -0700, Andres Freund wrote: > > Yikes, that's a pretty absurd implementation. > > Not when you take into account that it's been written over 20 years ago ;) Well, that doesn't mean it can't be fixed

Re: [HACKERS] Rename max_parallel_degree?

On Wed, Jun 29, 2016 at 10:46 PM, Amit Kapila wrote: > Your patch looks good to me and is ready for a committer's look. > > Notes for committer - > a. Verify if description of newly added Guc max_parallel_workers looks > okay to you, me and Julien are not in 100%