Re: [HACKERS] Flexible configuration for full-text search

On Mon, 6 Nov 2017 18:05:23 +1300 Thomas Munro wrote: > On Sat, Oct 21, 2017 at 1:39 AM, Aleksandr Parfenov > wrote: > > In attachment updated patch with fixes of empty XML tags in > > documentation. > > Hi Aleksandr, > > I'm not

Re: [HACKERS] UPDATE of partition key

On 2017/11/07 14:40, Amit Khandekar wrote: > On 7 November 2017 at 00:33, Robert Haas wrote: > >> Also, +1 for Amit Langote's idea of trying to merge >> mt_perleaf_childparent_maps with mt_persubplan_childparent_maps. > > Currently I am trying to see if it simplifies

Re: [HACKERS] UPDATE of partition key

On 7 November 2017 at 00:33, Robert Haas wrote: > Also, +1 for Amit Langote's idea of trying to merge > mt_perleaf_childparent_maps with mt_persubplan_childparent_maps. Currently I am trying to see if it simplifies things if we do that. We will be merging these arrays

Re: [HACKERS] Planning counters in pg_stat_statements

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Thomas Munro > I have often wanted $SUBJECT and was happy to find that Fujii-san had posted > a patch five years ago[1]. The reception then seemed positive. > So here is a refurbished and

[HACKERS] Planning counters in pg_stat_statements

Hi hackers, I have often wanted $SUBJECT and was happy to find that Fujii-san had posted a patch five years ago[1]. The reception then seemed positive. So here is a refurbished and (hopefully) improved version of his patch with a new column for the replan count. Thoughts? Example output:

Re: [HACKERS] LDAP URI decoding bugs

On Fri, Nov 3, 2017 at 12:57 PM, Thomas Munro wrote: > 1. If you set up a pg_hba.conf with a URL that lacks a base DN or > hostname, hba.c will segfault on startup when it tries to pstrdup a > null pointer. Examples: ldapurl="ldap://localhost; and >

[HACKERS] Fix a typo in dsm_impl.c

Hi, Attached the patch for $subject. s/reamin/remain/ Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center fix_typo_in_dsm_impl_c.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] proposal - pg_dump: flag to suppress output of SQL comments

Malcolm Locke writes: > Would a patch to add a flag to pg_dump to suppress the output of SQL > comments be likely to be accepted? Not unless you can come up with a better rationale than this: > The SQL generated by pg_dump seems to be fairly constant between > Postgres

Re: [HACKERS] [PATCH] A hook for session start

On Sun, Nov 5, 2017 at 3:14 AM, Fabrízio de Royes Mello wrote: > On Sat, Nov 4, 2017 at 1:23 AM, Michael Paquier > wrote: >> On Fri, Nov 3, 2017 at 1:55 PM, Fabrízio de Royes Mello >> wrote: >> >> Passing the database

Re: [HACKERS] Parallel Plans and Cost of non-filter functions

On Mon, Nov 6, 2017 at 7:40 PM, Paul Ramsey wrote: > From my perspective, this is much much better. For sufficiently large > tables, I get parallel behaviour without jimmying with the defaults on > parallel_setup_cost and parallel_tuple_cost. *And*, the parallel

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Nov 6, 2017 at 7:05 PM, Robert Haas wrote: > On Mon, Nov 6, 2017 at 11:20 AM, Amit Kapila wrote: >> On Mon, Nov 6, 2017 at 3:51 AM, Robert Haas wrote: >>> This looks like it's on the right track to me. I hope Tom

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

On 2017/11/06 21:52, David Rowley wrote: > On 6 November 2017 at 23:01, Amit Langote > wrote: >> OK, I have gotten rid of the min/max partition index interface and instead >> adopted the bms_add_range() approach by including your patch to add the >> same in the

Re: [HACKERS] [PATCH] Assert that the correct locks are held when calling PageGetLSN()

On Tue, Nov 7, 2017 at 7:27 AM, Asim Praveen wrote: > On Mon, Oct 2, 2017 at 6:48 PM, Michael Paquier > wrote: >> Jacob, here are some ideas to make this thread move on. I would >> suggest to produce a set of patches that do things incrementally:

Re: [HACKERS] path toward faster partition pruning

On 7 November 2017 at 01:52, David Rowley wrote: > Thanks. I'll look over it all again starting my Tuesday morning. (UTC+13) I have a little more review to share: 1. Missing "in" in comment. Should be "mentioned in" * get_append_rel_partitions * Return the list

[HACKERS] proposal - pg_dump: flag to suppress output of SQL comments

Hello Hackers, Would a patch to add a flag to pg_dump to suppress the output of SQL comments be likely to be accepted? So for example `pg_dump`: -- -- Name: foos; Type: TABLE; Schema: public; Owner: - -- CREATE TABLE foos ( ... With `pg_dump --no-sql-comments` Would become:

Re: [HACKERS] Early locking option to parallel backup

Em 05/11/2017 21:09, Andres Freund escreveu: On 2017-11-05 17:38:39 -0500, Robert Haas wrote: On Sun, Nov 5, 2017 at 5:17 AM, Lucas wrote: The patch creates a "--lock-early" option which will make pg_dump to issue shared locks on all tables on the backup TOC on each

Re: [HACKERS] Small improvement to compactify_tuples

2017-11-07 1:14 GMT+03:00 Claudio Freire : > > On Mon, Nov 6, 2017 at 6:58 PM, Юрий Соколов wrote: > > > > 2017-11-06 17:55 GMT+03:00 Claudio Freire : > >> > >> On Mon, Nov 6, 2017 at 11:50 AM, Юрий Соколов

Re: [HACKERS] [PATCH] Assert that the correct locks are held when calling PageGetLSN()

Hi Michael On Mon, Oct 2, 2017 at 6:48 PM, Michael Paquier wrote: > > Jacob, here are some ideas to make this thread move on. I would > suggest to produce a set of patches that do things incrementally: > 1) One patch that changes the calls of PageGetLSN to >

Re: [HACKERS] Small improvement to compactify_tuples

On Mon, Nov 6, 2017 at 6:58 PM, Юрий Соколов wrote: > > 2017-11-06 17:55 GMT+03:00 Claudio Freire : >> >> On Mon, Nov 6, 2017 at 11:50 AM, Юрий Соколов >> wrote: >> >> Maybe leave a fallback to qsort if some corner case

Re: [HACKERS] Small improvement to compactify_tuples

2017-11-06 17:55 GMT+03:00 Claudio Freire : > > On Mon, Nov 6, 2017 at 11:50 AM, Юрий Соколов wrote: > >> Maybe leave a fallback to qsort if some corner case produces big buckets? > > > > For 8kb pages, each bucket is per 32 bytes. So, for heap

Re: [HACKERS] MERGE SQL Statement for PG11

Simon Riggs wrote: In step 3 we discover that an entry exists in the index for a committed row. Since we have a unique index we use it to locate the row we know exists and UPDATE that. We don't use a new MVCC snapshot, we do what EPQ does. EPQ is already violating MVCC

Re: [HACKERS] SQL procedures

On 31 October 2017 at 17:23, Peter Eisentraut wrote: > I've been working on SQL procedures. (Some might call them "stored > procedures", but I'm not aware of any procedures that are not stored, so > that's not a term that I'm using here.) Looks good >

Re: [HACKERS] MERGE SQL Statement for PG11

On 6 November 2017 at 18:35, Peter Geoghegan wrote: >> APPROACH2 (modified from my original proposal slightly) > > > This write-up actually begins to confront the issues that I've raised. > I'm glad to see this. > >> 1. Join... >> 2. Apply results for UPDATE, if present not visible

Re: [HACKERS] UPDATE of partition key

On Wed, Oct 25, 2017 at 11:40 AM, Amit Khandekar wrote: > Below I have addressed the remaining review comments : The changes to trigger.c still make me super-nervous. Hey THOMAS MUNRO, any chance you could review that part? + /* The caller must have already locked

Re: [HACKERS] MERGE SQL Statement for PG11

Simon Riggs wrote: APPROACH1 1. Join to produce results based upon snapshot at start of query 2. Apply results for INSERT, UPDATE or DELETE Such failures are of great concern in practice because the time between 1 and 2 could be very long for large statements, or for

Re: [HACKERS] [POC] Faster processing at Gather node

Hi -- pgsql-hackers-ow...@postgresql.org wrote on 11/06/2017 09:47:22 AM: > From: Andres Freund > > Hi, > > Please don't top-quote on postgresql lists. Sorry > > On 2017-11-06 09:44:24 -0600, Jim Van Fleet wrote: > > > >hammerdb, in this configuration, runs a variant of

Re: [HACKERS] MERGE SQL Statement for PG11

On 3 November 2017 at 16:35, Peter Geoghegan wrote: > Simon Riggs wrote: The *only* behavioural difference I have proposed would be the *lack* of an ERROR in (some) concurrent cases. >>> >>> >>> I think that's a big difference. Error vs.

Re: [HACKERS] pow support for pgbench

I don't want to go too deep into it, but you get stuff like this: Select pow(2.0, -3)::text = pow(2, -3)::text; Sure. It does so with any overloaded operator or function: fabien=# SELECT (2.0 + 3)::TEXT = (2 + 3)::TEXT; # f Patch applies, make check ok in pgbench, doc gen ok. ipow code

Re: [HACKERS] pow support for pgbench

Hi, Indeed, this is quite strange... I don't want to go too deep into it, but you get stuff like this: Select pow(2.0, -3)::text = pow(2, -3)::text; ?column? -- f (1 row) - you can simplify the ipow function by removing handling of y<0 case, >maybe add an assert to be sure

Re: [HACKERS] [pgsql-www] Schedule for migration to pglister

On Mon, Nov 6, 2017 at 5:00 PM, Tom Lane wrote: > Magnus Hagander writes: > > On Mon, Nov 6, 2017 at 4:46 PM, Tom Lane wrote: > >> Hm, around here it's no match -> spam bucket. But in any case, why > > > I think you're quite

Re: [HACKERS] [pgsql-www] Schedule for migration to pglister

Magnus Hagander writes: > On Mon, Nov 6, 2017 at 4:46 PM, Tom Lane wrote: >> Hm, around here it's no match -> spam bucket. But in any case, why > I think you're quite uncommon in that setup. Interesting, because "it's not addressed to me (or any list

Re: [HACKERS] [pgsql-www] Schedule for migration to pglister

On Mon, Nov 6, 2017 at 4:46 PM, Tom Lane wrote: > Magnus Hagander writes: > > On Mon, Nov 6, 2017 at 4:40 PM, Tom Lane wrote: > >> I suggest doing that the other way 'round. Otherwise, the email > >> about the change will inevitably

Re: [HACKERS] [POC] Faster processing at Gather node

Hi, Please don't top-quote on postgresql lists. On 2017-11-06 09:44:24 -0600, Jim Van Fleet wrote: > > >hammerdb, in this configuration, runs a variant of tpcc > > > > Hard to believe that any of the changes here are relevant in that > > case - this is parallelism specific stuff. Whereas tpcc

Re: [HACKERS] [pgsql-www] Schedule for migration to pglister

Magnus Hagander writes: > On Mon, Nov 6, 2017 at 4:40 PM, Tom Lane wrote: >> I suggest doing that the other way 'round. Otherwise, the email >> about the change will inevitably go into a lot of peoples' bit >> buckets if they haven't adjusted their mail

Re: [HACKERS] [POC] Faster processing at Gather node

correct > >hammerdb, in this configuration, runs a variant of tpcc > > Hard to believe that any of the changes here are relevant in that > case - this is parallelism specific stuff. Whereas tpcc is oltp, right? > > Andres > -- > Sent from my Android device with K-9 Mail. Please excuse my

Re: [HACKERS] [pgsql-www] Schedule for migration to pglister

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > Each list will receive an email with a link to the wiki about the > > migration after the list has been migrated. > > I suggest doing that the other way 'round. Otherwise, the email > about the change

Re: [HACKERS] [pgsql-www] Schedule for migration to pglister

On Mon, Nov 6, 2017 at 4:40 PM, Tom Lane wrote: > Stephen Frost writes: > > Each list will receive an email with a link to the wiki about the > > migration after the list has been migrated. > > I suggest doing that the other way 'round. Otherwise, the

Re: [HACKERS] [pgsql-www] Schedule for migration to pglister

Stephen Frost writes: > Each list will receive an email with a link to the wiki about the > migration after the list has been migrated. I suggest doing that the other way 'round. Otherwise, the email about the change will inevitably go into a lot of peoples' bit buckets if

[HACKERS] Schedule for migration to pglister

Greetings, The PostgreSQL Infrastructure team is working to migrate the project's mailing lists from the existing system (an ancient and unmaintained piece of software called "majordomo2") to a newly developed mailing list system (known as "PGLister"), which better addresses the needs of the

Re: [HACKERS] [POC] Faster processing at Gather node

On November 6, 2017 7:30:49 AM PST, Jim Van Fleet wrote: >Andres Freund wrote on 11/05/2017 03:40:15 PM: > >hammerdb, in this configuration, runs a variant of tpcc Hard to believe that any of the changes here are relevant in that case - this is

Re: [HACKERS] [POC] Faster processing at Gather node

Andres Freund wrote on 11/05/2017 03:40:15 PM: hammerdb, in this configuration, runs a variant of tpcc > > What query(s) did you measure? > > Andres > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. >

Re: [HACKERS] Fix performance degradation of contended LWLock on NUMA

On 2017-10-20 11:54, Sokolov Yura wrote: Hello, On 2017-10-19 19:46, Andres Freund wrote: On 2017-10-19 14:36:56 +0300, Sokolov Yura wrote: > > + init_local_spin_delay(); > > The way you moved this around has the disadvantage that we now do this - > a number of writes - even in the very

Re: [HACKERS] pow support for pgbench

Hello, Sorry for the confusion, I wasn't aware that SQL pow changed types depending on the input value. Indeed, this is quite strange... fabien=# SELECT i, POW(2, i) FROM generate_series(-2, 2) AS i; -2 | 0.25 -1 | 0.5 0 | 1 1 | 2 2 | 4 I've modified the function to

Re: [HACKERS] Small improvement to compactify_tuples

On Mon, Nov 6, 2017 at 11:50 AM, Юрий Соколов wrote: >> Maybe leave a fallback to qsort if some corner case produces big buckets? > > For 8kb pages, each bucket is per 32 bytes. So, for heap pages it is at > most 1 heap-tuple per bucket, and for index pages it is at most 2

Re: [HACKERS] Small improvement to compactify_tuples

2017-11-05 20:44 GMT+03:00 Claudio Freire : > > On Sat, Nov 4, 2017 at 8:07 PM, Юрий Соколов wrote: > > 2017-11-03 5:46 GMT+03:00 Tom Lane : > >> > >> Sokolov Yura writes: > >> > [

Re: [HACKERS] pow support for pgbench

Hi Fabien, Sorry for the confusion, I wasn't aware that SQL pow changed types depending on the input value. I've modified the function to match more closely the behaviour of SQL, except that 0^(negative) returns 'double inf'. Do you think there is any value in raising an error instead? On Mon,

Re: [HACKERS] Early locking option to parallel backup

Lucas, * Lucas (luca...@gmail.com) wrote: > pg_dump was taking more than 24 hours to complete in one of my databases. I > begin to research alternatives. Parallel backup reduced the backup time to > little less than a hour, but it failed almost every time because of > concurrent queries that

Re: [HACKERS] Parallel Plans and Cost of non-filter functions

>From my perspective, this is much much better. For sufficiently large tables, I get parallel behaviour without jimmying with the defaults on parallel_setup_cost and parallel_tuple_cost. *And*, the parallel behaviour *is* sensitive to the costs of functions in target lists, so reasonably chosen

Re: [HACKERS] Client Connection redirection support for PostgreSQL

On Thu, Nov 2, 2017 at 4:33 PM, Craig Ringer wrote: >> Add the ability to the PostgreSQL server instance to route the traffic to a >> different server instance based on the rules defined in server’s pg_bha.conf >> configuration file. At a high level this enables offloading

Re: [HACKERS] pow support for pgbench

Hi Fabien, Thanks for the review. I've fixed the documentation and added an ipow function that handles both positive and negative ints, having 0^0 == 1 and 0^(negative) == PG_INT64_MAX since that's what my glibc math.h pow() is returning. On Sat, Nov 4, 2017 at 12:34 PM, Fabien COELHO

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Nov 6, 2017 at 11:20 AM, Amit Kapila wrote: > On Mon, Nov 6, 2017 at 3:51 AM, Robert Haas wrote: >> This looks like it's on the right track to me. I hope Tom will look >> into it, but if he doesn't I may try to get it committed myself. >>

Re: [HACKERS] Early locking option to parallel backup

On 2017-11-05 22:43:34 -0500, Tom Lane wrote: > > IIUC the problem here is that even though a lock is already > > held by the main backend an independent locker's request will prevent > > the on-demand lock by the dump worker from being granted. It seems to > > me the correct fix here would be to

Re: [HACKERS] Early locking option to parallel backup

On Mon, Nov 6, 2017 at 4:43 AM, Tom Lane wrote: > I wonder if we couldn't somehow repurpose the work that was done for > parallel workers' locks. Lots of security-type issues to be handled > if we're to open that up to clients, but maybe it's solvable. For > instance, maybe

Re: [HACKERS] Restricting maximum keep segments by repslots

Hi, On 2017-10-31 18:43:10 +0900, Kyotaro HORIGUCHI wrote: > - distance: > how many bytes LSN can advance before the margin defined by > max_slot_wal_keep_size (and wal_keep_segments) is exhasuted, > or how many bytes this slot have lost xlog from restart_lsn. I don't think

Re: [HACKERS] Restricting maximum keep segments by repslots

On 2017-11-06 11:07:04 +0800, Craig Ringer wrote: > Would it make sense to teach xlogreader how to fetch from WAL archive, > too? That way if there's an archive, slots could continue to be used > even after we purge from local pg_xlog, albeit at a performance cost. > > I'm thinking of this mainly

Re: [HACKERS] Secondary index access optimizations

On 11/06/2017 04:27 AM, Thomas Munro wrote: On Fri, Sep 8, 2017 at 3:58 AM, Konstantin Knizhnik wrote: Updated version of the patch is attached to this mail. Also I added support of date type to operator_predicate_proof to be able to imply (logdate <= '2017-03-31')

Re: [HACKERS] pow support for pgbench

Hello Raúl, I've fixed the documentation and added an ipow function that handles both positive and negative ints, having 0^0 == 1 and 0^(negative) == PG_INT64_MAX since that's what my glibc math.h pow() is returning. From the comment: * For exp < 0 return 0 except when the base is 1 or

Re: [HACKERS] proposal - Default namespaces for XPath expressions (PostgreSQL 11)

Thank you for the new patch. - The latest patch is missing xpath_parser.h at least since ns-3. That of the first (not-numbered) version was still usable. - c29c578 conflicts on doc/src/sgml/func.sgml At Sun, 15 Oct 2017 12:06:11 +0200, Pavel Stehule wrote in

Re: [HACKERS] path toward faster partition pruning

On 6 November 2017 at 23:01, Amit Langote wrote: > OK, I have gotten rid of the min/max partition index interface and instead > adopted the bms_add_range() approach by including your patch to add the > same in the patch set (which is now 0002 in the whole set). I

Re: [HACKERS] Display number of heap accesses for index scans

On Mon, Nov 6, 2017 at 6:33 AM, Tom Lane wrote: > Peter Geoghegan writes: > > Andres Freund wrote: > >> The number of index lookups that failed to return anything can be a > >> critical performance factor in OLTP workloads. Therefore it

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

On Mon, Oct 30, 2017 at 3:17 PM, Masahiko Sawada wrote: > On Fri, Oct 27, 2017 at 12:03 AM, Robert Haas wrote: >> On Thu, Oct 26, 2017 at 12:36 PM, Masahiko Sawada >> wrote: >>> Since the previous patch conflicts with current

Re: [HACKERS] Statement-level rollback

On Wed, Nov 1, 2017 at 6:47 AM, MauMau wrote: > From: Simon Riggs > On 14 August 2017 at 23:58, Peter Eisentraut > wrote: >> On 2/28/17 02:39, Tsunakawa, Takayuki wrote: >>> The code for stored functions is not written yet, but I'd like your

Re: [HACKERS] Statement-level rollback

From: Thomas Munro With your v2 patch "make docs" fails. Here is a small patch to apply on top of yours to fix that and some small copy/paste errors, if I understood correctly. Ouch, thanks. I'd like to merge your fix when I submit the next revision of my patch. Regards MauMau -- Sent via

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Nov 6, 2017 at 3:51 AM, Robert Haas wrote: > On Sun, Nov 5, 2017 at 12:57 AM, Amit Kapila wrote: >> Thanks for the confirmation. Find rebased patch attached. > > This looks like it's on the right track to me. I hope Tom will look > into

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

On 2017/11/06 13:15, David Rowley wrote: > On 31 October 2017 at 21:43, Amit Langote > wrote: >> Attached updated version of the patches > > match_clauses_to_partkey() needs to allow for the way quals on Bool > columns are represented. > > create table pt (a bool

Re: [HACKERS] [PATCH] Overestimated filter cost and its mitigation

On Mon, Nov 6, 2017 at 10:01 AM, Thomas Munro wrote: > > This idea seems to makes intuitive sense. I see that you use > order_qual_clauses() to know what order they'll run in, so I'm > wondering if there is any reason we shouldn't do it up front and keep > it

[HACKERS] Removing useless DISTINCT clauses

In [1] we made a change to process the GROUP BY clause to remove any group by items that are functionally dependent on some other GROUP BY items. This really just checks if a table's PK columns are entirely present in the GROUP BY clause and removes anything else belonging to that table. All

[HACKERS] Race to build pg_isolation_regress in "make -j check-world"

I've been enjoying the speed of parallel check-world, but I get spurious failures from makefile race conditions. Commit c66b438 fixed the simple ones. More tricky is this problem of multiple "make" processes entering src/test/regress concurrently, which causes failures like these: gcc: error: