[HACKERS] Replacing lfirst() with lfirst_node() appropriately in planner.c

Hi, Happened to stumble across some instances of lfirst() which could use lfirst_node() in planner.c. Here's patch which replaces calls to lfirst() extracting node pointers by lfirst_node() in planner.c. I have covered all the occurences of lfirst() except 1. those extract generic pointers like

Re: [HACKERS] New partitioning - some feedback

On 2017/07/12 23:58, Alvaro Herrera wrote: > Amit Langote wrote: >> On 2017/07/11 13:34, Alvaro Herrera wrote: >>> Robert Haas wrote: On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote wrote: >>> > Actually, if \d had shown RELKIND_PARTITIONED_TABLE tables

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

On Wed, Jul 12, 2017 at 5:06 PM, Michael Paquier wrote: > On Tue, Jul 11, 2017 at 9:28 AM, Masahiko Sawada > wrote: >> Attached updated version patch. Please review it. > > Cool, thanks. Thank you for reviewing the patch. > > +useless. If

Re: [HACKERS] Typo in backend/storage/ipc/standby.c

At Wed, 12 Jul 2017 17:11:12 +0300, Heikki Linnakangas wrote in > On 07/11/2017 10:34 AM, Kyotaro HORIGUCHI wrote: > > I noticed that a comment above StandbyAcquireAccessExclusiveLock > > in backend/storage/ipc/standby.c using wrong

Re: [HACKERS] PostgreSQL10 beta2 with ICU - initdb fails on MacOS

I wrote: > I still think that this represents under-engineering by the ICU crew > and not anything we're doing wrong. After some more examination of their readme.html, I tried this: $ ./runConfigureICU MacOSX --prefix=/usr/local/icu-57.1 --enable-rpath and that gave me sane-looking libraries:

Re: [HACKERS] PostgreSQL10 beta2 with ICU - initdb fails on MacOS

On Thu, Jul 13, 2017 at 8:23 AM, Tom Lane wrote: > Sandeep Thakkar writes: > > On Thu, Jul 13, 2017 at 12:44 AM, Tom Lane wrote: > >> and this is evidently because the libraries themselves don't know where > >> they

Re: [HACKERS] building libpq.a static library

On 13 July 2017 at 10:58, Craig Ringer wrote: > On 12 July 2017 at 23:46, Jeroen Ooms wrote: > >> On Wed, Jul 12, 2017 at 5:11 PM, Tom Lane wrote: >> > Jeroen Ooms writes: >> >> I maintain static libraries

Re: [HACKERS] building libpq.a static library

On 12 July 2017 at 23:46, Jeroen Ooms wrote: > On Wed, Jul 12, 2017 at 5:11 PM, Tom Lane wrote: > > Jeroen Ooms writes: > >> I maintain static libraries for libpq for the R programming language > >> (we need static linking to ship

Re: [HACKERS] PostgreSQL10 beta2 with ICU - initdb fails on MacOS

Sandeep Thakkar writes: > On Thu, Jul 13, 2017 at 12:44 AM, Tom Lane wrote: >> and this is evidently because the libraries themselves don't know where >> they live: >> $ otool -D /usr/local/icu-57.1/lib/libicui18n.57.dylib >>

Re: [HACKERS] idea: custom log_line_prefix components besides application_name

On 07/12/17 08:38, Robert Haas wrote: > another protocol message. I feel like the usefulness of this for > connection pooling software is pretty obvious: it's a lot easier for > the pooler to disallow a certain protocol message than a certain SQL > command. I assume you mean easier than

Re: [HACKERS] PostgreSQL10 beta2 with ICU - initdb fails on MacOS

On Thu, Jul 13, 2017 at 12:44 AM, Tom Lane wrote: > Sandeep Thakkar writes: > > On Wed, Jul 12, 2017 at 8:13 PM, Tom Lane wrote: > >> Ugh. Please provide details about the hand-built ICU --- what version, > >> what

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes wrote: > On Tue, Jul 11, 2017 at 10:25 PM, Amit Kapila > wrote: >> >> On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes wrote: >> > On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar

Re: [HACKERS] New partitioning - some feedback

On 12 July 2017 at 23:23, Dean Rasheed wrote: > I also agree that there probably isn't much need for a list that > *only* includes partitions, but if someone comes up with a convincing > use case, then we could add another 2-letter command for that. > Actually, I just

Re: [HACKERS] building libpq.a static library

On 2017-07-12 23:55:56 +0100, Greg Stark wrote: > Fwiw I think the real problem is that building static libraries > "properly" requires different compiler options -- notably they're not > normally built with -fPIC. So that means building every object twice > which kind of breaks make's build model

Re: [HACKERS] building libpq.a static library

On 12 July 2017 at 16:11, Tom Lane wrote: > Jeroen Ooms writes: > >> This works but it's a bit of a pain to maintain. I was wondering if >> this hack could be merged so that the standard 'configure >> --enable-static' script would install a static library

Re: [HACKERS] More race conditions in logical replication

Petr Jelinek wrote: > So best idea I could come up with is to make use of the new condition > variable API. That lets us wait for variable which can be set per slot. Here's a cleaned up version of that patch, which I intend to get in the tree tomorrow. I verified that this allows the

Re: [HACKERS] New partitioning - some feedback

On 12 July 2017 at 15:58, Alvaro Herrera wrote: > Amit Langote wrote: >> On 2017/07/11 13:34, Alvaro Herrera wrote: >> > However, the "list tables" >> > command \dt should definitely IMO not list partitions. >> >> Do you mean never? Even if a modifier is specified? In

Re: [HACKERS] Causal reads take II

On Thu, Jul 13, 2017 at 2:51 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > Thank you. I started to play with it a little bit, since I think it's an > interesting idea. And there are already few notes: Thanks Dmitry. > * I don't see a CF item for that, where is it?

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

On Wed, Jul 12, 2017 at 2:17 PM, Peter Geoghegan wrote: > I'd be interested in seeing the difference it makes if Postgres is > built with the call to _bt_check_unique() commented out within > nbtinsert.c. Actually, I mean that I wonder how much of a difference it would make if this

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

On Wed, Jul 12, 2017 at 1:55 PM, Alvaro Herrera wrote: > Not to mention work done with a "buffer cleanup lock" held -- which is > compounded by the fact that acquiring such a lock is prone to starvation > if there are many scanners of that index. I've seen a case where

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

Peter Geoghegan wrote: > Now, that might not seem like that much of a difference, but if you > consider how duplicates are handled in the B-Tree code, and how unique > index enforcement works, I think it could be. It could lead to heavy > buffer lock contention, because we sometimes do a lot of

Re: [HACKERS] Adding support for Default partition in partitioning

Hi, I have tried to address your comments in the V22 set of patches[1]. Please find my feedback inlined on your comments. On Fri, Jun 16, 2017 at 1:46 PM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello, I'd like to review this but it doesn't fit the master, as > Robert

Re: [HACKERS] Adding support for Default partition in partitioning

Hi Ashutosh, I have tried to address your comments in the V22 set of patches[1]. Please find my feedback inlined on your comments. On Thu, Jun 15, 2017 at 10:24 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > Some more comments on the latest set of patches. > > In

Re: [HACKERS] Double shared memory allocation for SLRU LWLocks

On Wed, Jul 12, 2017 at 2:03 PM, Teodor Sigaev wrote: > It seems to me that we're allocating shared memory for SLRU lwlocks twice, >> unless I'm missing something. >> > I think you are right. > > Did you check previous versions? i.e. should it be applyed to previous >

Re: [HACKERS] Adding support for Default partition in partitioning

Hi Robert, I have tried to address your comments in the V22 set of patches[1]. Please find my feedback inlined on your comments. On Thu, Jun 15, 2017 at 1:21 AM, Robert Haas wrote: > > - Needs to be rebased over b08df9cab777427fdafe633ca7b8abf29817aa55. > Rebased on

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

On Wed, Jul 12, 2017 at 12:30 PM, Peter Geoghegan wrote: > On Wed, Jul 12, 2017 at 4:28 AM, Alik Khilazhev > wrote: >> I am attaching results of query that you sent. It shows that there is >> nothing have changed after executing tests. > > But something

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

On Wed, Jul 12, 2017 at 4:28 AM, Alik Khilazhev wrote: > I am attaching results of query that you sent. It shows that there is > nothing have changed after executing tests. But something did change! In the case where performance was good, all internal pages on the

Re: [HACKERS] Function Volatility and Views Unexpected Behavior

David Kohn writes: > I encountered some unexpected behavior when debugging a query that was > taking longer than expected, basically, a volatile function that makes a > column in a view is called even when that column is not selected in the > query, making it so that the

Re: [HACKERS] PostgreSQL10 beta2 with ICU - initdb fails on MacOS

Sandeep Thakkar writes: > On Wed, Jul 12, 2017 at 8:13 PM, Tom Lane wrote: >> Ugh. Please provide details about the hand-built ICU --- what version, >> what configure options did you use for it, etc. > I tried with ICU 53.1 and 57.1 and the

Re: [HACKERS] idea: custom log_line_prefix components besides application_name

On Wed, Jul 12, 2017 at 07:38:56AM -0500, Robert Haas wrote: > On Tue, May 9, 2017 at 9:43 PM, Chapman Flack wrote: > > That's where the appident.cookie() function comes in. You just > > query it once at session establishment and remember the cookie. > > That allows your

Re: [HACKERS] pl/perl extension fails on Windows

On 07/12/2017 11:49 AM, Dave Page wrote: > > > On Wed, Jul 12, 2017 at 4:35 PM, Tom Lane > wrote: > > Sandeep Thakkar > writes: > > I compiled PG 10

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Jul 11, 2017 at 10:25 PM, Amit Kapila wrote: > On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes wrote: > > On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar > wrote: > >> > >> So because of this high projection cost the

Re: Fwd: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Wed, Jul 12, 2017 at 1:08 PM, Claudio Freire wrote: > On Wed, Jul 12, 2017 at 11:48 AM, Alexey Chernyshov > wrote: >> Thank you for the patch and benchmark results, I have a couple remarks. >> Firstly, padding in DeadTuplesSegment >> >>

Re: Fwd: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Wed, Jul 12, 2017 at 11:48 AM, Alexey Chernyshov wrote: > Thank you for the patch and benchmark results, I have a couple remarks. > Firstly, padding in DeadTuplesSegment > > typedef struct DeadTuplesSegment > > { > > ItemPointerData last_dead_tuple;/* Copy

[HACKERS] Function Volatility and Views Unexpected Behavior

(Not sure whether I should be sending this to hackers or somewhere else, let me know if I should move, feel like I should say long time listener first time caller or something) I encountered some unexpected behavior when debugging a query that was taking longer than expected, basically, a

Re: [HACKERS] pl/perl extension fails on Windows

On Wed, Jul 12, 2017 at 4:35 PM, Tom Lane wrote: > Sandeep Thakkar writes: > > I compiled PG 10 beta1/beta2 with "--with-perl" option on Windows and the > > extension crashes the database. > > *src/pl/plperl/Util.c: loadable library and perl

Re: [HACKERS] building libpq.a static library

On Wed, Jul 12, 2017 at 5:11 PM, Tom Lane wrote: > Jeroen Ooms writes: >> I maintain static libraries for libpq for the R programming language >> (we need static linking to ship with the binary packages). > > How do you get that past vendor packaging

Re: [HACKERS] pl/perl extension fails on Windows

Sandeep Thakkar writes: > I compiled PG 10 beta1/beta2 with "--with-perl" option on Windows and the > extension crashes the database. > *src/pl/plperl/Util.c: loadable library and perl binaries are mismatched > (got handshake key 0A900080, needed 0AC80080)* >

Re: [HACKERS] PostgreSQL10 beta2 with ICU - initdb fails on MacOS

On Wed, Jul 12, 2017 at 8:13 PM, Tom Lane wrote: > Sandeep Thakkar writes: > > On MacOS, I configured PG10 beta2 sources with '--with-icu" option along > > with ICU_LIBS and ICU_CFLAGS env variables to define the location of the > > ICU libs

Re: [HACKERS] building libpq.a static library

Jeroen Ooms writes: > I maintain static libraries for libpq for the R programming language > (we need static linking to ship with the binary packages). How do you get that past vendor packaging policies? When I worked at Red Hat, there was a very strong policy against

Re: [HACKERS] Postgres process invoking exit resulting in sh-QUIT core

Hi Craig, Here is bt after installing all the missing debuginfo packages. (gdb) bt #0 0x00fff7682f18 in do_lookup_x (undef_name=undef_name@entry=0xfff75cece5 "_Jv_RegisterClasses", new_hash=new_hash@entry=2681263574, old_hash=old_hash@entry=0xa159b8, ref=0xfff75ceac8,

Re: [HACKERS] New partitioning - some feedback

Amit Langote wrote: > On 2017/07/11 13:34, Alvaro Herrera wrote: > > Robert Haas wrote: > >> On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote > >> wrote: > > > >>> Actually, if \d had shown RELKIND_PARTITIONED_TABLE tables as of Type > >>> "partitioned table", we

Re: [HACKERS] Causal reads take II

> On 23 June 2017 at 13:48, Thomas Munro wrote: > > Apologies for the extended delay. Here is the rebased patch, now with a > couple of improvements (see below). Thank you. I started to play with it a little bit, since I think it's an interesting idea. And there

Re: Fwd: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

Thank you for the patch and benchmark results, I have a couple remarks. Firstly, padding in DeadTuplesSegment typedef struct DeadTuplesSegment { ItemPointerData last_dead_tuple;/* Copy of the last dead tuple (unset * until the segment is

Re: [HACKERS] PostgreSQL10 beta2 with ICU - initdb fails on MacOS

Sandeep Thakkar writes: > On MacOS, I configured PG10 beta2 sources with '--with-icu" option along > with ICU_LIBS and ICU_CFLAGS env variables to define the location of the > ICU libs that I built. Once the staging is ready, I executed initdb but it > fails with

Re: [HACKERS] building libpq.a static library

On Wednesday, July 12, 2017 6:31:09 AM EDT Jeroen Ooms wrote: > I maintain static libraries for libpq for the R programming language > (we need static linking to ship with the binary packages). > Unfortunately currently the standard postgres makefile only generates > a shared library for libpq,

Re: [HACKERS] Typo in backend/storage/ipc/standby.c

On 07/11/2017 10:34 AM, Kyotaro HORIGUCHI wrote: Hello. I noticed that a comment above StandbyAcquireAccessExclusiveLock in backend/storage/ipc/standby.c using wrong names of a variable and a type. The attached patch fixes it. The same mistake is found in older versions back to 9.0.

Re: [HACKERS] DSA on 32 bit systems

On Wed, Jul 12, 2017 at 7:34 AM, Thomas Munro wrote: > That raises the question of whether we should go further and use a 64 > bit dsa_pointer even on 32 bit systems. In a 32 bit dsa_pointer > build, a single dsa_area is limited to 32 segments of up to 128MB in >

Re: [HACKERS] idea: custom log_line_prefix components besides application_name

On Tue, May 9, 2017 at 9:43 PM, Chapman Flack wrote: > That's where the appident.cookie() function comes in. You just > query it once at session establishment and remember the cookie. > That allows your code to say: > > SET SESSION ON BEHALF OF 'joe user' BECAUSE I HAVE

[HACKERS] DSA on 32 bit systems

Hi hackers, Commit e8fdbd58fe564a29977f4331cd26f9697d76fc40 introduced fallback atomic u64 support. PG_HAVE_ATOMIC_U64_SUPPORT now always finishes up defined so it is pointless to test for it in dsa.h. Here's a patch to remove the obsolete test and comment. That raises the question of whether

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

On Wed, Jul 12, 2017 at 12:53 AM, Alvaro Herrera wrote: > > We have one opclass for each type combination -- int4 to int2, int4 to > int4, int4 to int8, etc. You just need to add the new strategy to all > the opclasses. > Can you clarify this solution ? I think another

[HACKERS] pl/perl extension fails on Windows

Hi, I compiled PG 10 beta1/beta2 with "--with-perl" option on Windows and the extension crashes the database. -- postgres=# create extension plperl; server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. The

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

LastBeginState -> RetryState? I'm not sure why this state is a pointer in CState. Putting the struct would avoid malloc/free cycles. Index "-1" may be used to tell it is not set if necessary. Another detail I forgot about this point: there may be a memory leak on variables copies, ISTM that

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

Hello! I want to say that our company is already engaged in the search for the causes of the problem and their solution. And also we have few experimental patches that increases performance for 1000 clients by several times. In addition, I have fixed threadsafety issues and implemented

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

On 7 Jul 2017, at 21:53, Peter Geoghegan wrote:Is it possible for you to instrument the number of B-Tree pageaccesses using custom instrumentation for pgbench_accounts_pkey?If that seems like too much work, then it would still be interestingto see what the B-Tree keyspace looks like

Re: [HACKERS] Double shared memory allocation for SLRU LWLocks

It seems to me that we're allocating shared memory for SLRU lwlocks twice, unless I'm missing something. I think you are right. Did you check previous versions? i.e. should it be applyed to previous branches?? I suppose yes, to minimize code difference. Also I'd like an idea to add

[HACKERS] PostgreSQL10 beta2 with ICU - initdb fails on MacOS

Hi On MacOS, I configured PG10 beta2 sources with '--with-icu" option along with ICU_LIBS and ICU_CFLAGS env variables to define the location of the ICU libs that I built. Once the staging is ready, I executed initdb but it fails with the following error: -- $./initdb -D /tmp/data The files

[HACKERS] building libpq.a static library

I maintain static libraries for libpq for the R programming language (we need static linking to ship with the binary packages). Unfortunately currently the standard postgres makefile only generates a shared library for libpq, not a static one. In order to make a static library I always manually

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

Hello Marina, There's the second version of my patch for pgbench. Now transactions with serialization and deadlock failures are rolled back and retried until they end successfully or their number of attempts reaches maximum. In details: - You can set the maximum number of attempts by the

Re: [HACKERS] Multi column range partition table

On Wed, Jul 12, 2017 at 12:54 AM, Dean Rasheed wrote: > On 11 July 2017 at 13:29, Ashutosh Bapat > wrote: >> + >> + Also note that some element types, such as timestamp, >> + have a notion of "infinity", which is just

Re: [HACKERS] Minor style cleanup in tab-complete.c

On 07/12/2017 07:39 AM, Michael Paquier wrote: On Wed, Jul 12, 2017 at 11:19 AM, Thomas Munro wrote: From the triviality department: I noticed some branches in tab-complete.c's gargantuan if statement, mostly brand new, that break from the established brace

Re: [HACKERS] New partitioning - some feedback

On Wed, Jul 12, 2017 at 9:39 AM, Amit Langote wrote: > On 2017/07/12 12:47, Ashutosh Bapat wrote: >> On Wed, Jul 12, 2017 at 8:23 AM, Amit Langote >> wrote: >>> On 2017/07/11 18:57, Ashutosh Bapat wrote: On Tue, Jul 11, 2017 at

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Jul 12, 2017 at 10:55 AM, Amit Kapila wrote: > On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes wrote: >> On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar wrote: >>> >>> So because of this high projection cost the seqpath and

Re: [HACKERS] New partitioning - some feedback

On 2017/07/12 13:09, Amit Langote wrote: > On 2017/07/12 12:47, Ashutosh Bapat wrote: >> Do you see that those patches can be used in current discussion in any way? > > It wouldn't really be a bad idea to put that patch here, because there's > no special reason for it to be in the CF for PG 11,

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

On Tue, Jul 11, 2017 at 9:28 AM, Masahiko Sawada wrote: > Attached updated version patch. Please review it. Cool, thanks. +useless. If the second parameter wait_for_archive is true and +the backup is taken on a standby, pg_stop_backup waits for WAL +to