Re: [HACKERS] Slowness of extended protocol

2016-08-18 Thread Tatsuo Ishii
> Tatsuo>understanding it always uses unnamed portal even if the SQL is like > "BEGIN" or "COMMIT" (no parameters). They are too often used. Why not > doing like this? > > Does it actually work? > > The documentation says named portals last till the end of the transaction: > > https://www.postgr

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-08-18 Thread Michael Paquier
On Fri, Aug 19, 2016 at 1:51 AM, Heikki Linnakangas wrote: > On 08/18/2016 03:45 PM, Michael Paquier wrote: >> >> On Thu, Aug 18, 2016 at 9:28 PM, Heikki Linnakangas >> wrote: >> For the current ip.c, I don't have a better idea than putting in >> src/common/ip.c the set of routines used by both t

Re: [HACKERS] Slowness of extended protocol

2016-08-18 Thread Vladimir Sitnikov
Tatsuo>understanding it always uses unnamed portal even if the SQL is like "BEGIN" or "COMMIT" (no parameters). They are too often used. Why not doing like this? Does it actually work? The documentation says named portals last till the end of the transaction: https://www.postgresql.org/docs/9.5/

[HACKERS] Exporting more function in libpq

2016-08-18 Thread Tatsuo Ishii
I would like to proppse to export these functions in libpq. pqPutMsgStart pqPutMsgEnd pqPutc pqPuts pqPutInt pqPutnchar pqFlush pqHandleSendFailure I think this would be useful to create a tool/library which needs to handle frontend/backend protocol messages in detail. Best regards, -- Tatsuo Is

[HACKERS] Should we cacheline align PGXACT?

2016-08-18 Thread Alexander Korotkov
Hackers, originally this idea was proposed by Andres Freund while experimenting with lockfree Pin/UnpinBuffer [1]. The patch is attached as well as results of pgbench -S on 72-cores machine. As before it shows huge benefit in this case. For sure, we should validate that it doesn't cause performan

Re: [HACKERS] Slowness of extended protocol

2016-08-18 Thread Tatsuo Ishii
BTW, there seem to be a room to enhance JDBC driver performance. In my understanding it always uses unnamed portal even if the SQL is like "BEGIN" or "COMMIT" (no parameters). They are too often used. Why not doing like this? Prepare(stmt=S1,query="BEGIN") Bind(stmt=S1,portal=P1) Execute(portal=P1

Re: [HACKERS] pg_basebackup wish list

2016-08-18 Thread Masahiko Sawada
On Fri, Aug 19, 2016 at 7:06 AM, Peter Eisentraut wrote: > On 7/12/16 9:55 PM, Masahiko Sawada wrote: >> And what I think is pg_baseback never remove the directory specified >> by -D option even if execution is failed. initdb command behaves so. >> I think it's helpful for backup operation. > > Th

Re: [HACKERS] Bug in to_timestamp().

2016-08-18 Thread amul sul
On Friday, August 19, 2016 12:42 AM, Robert Haas wrote: >On Wed, Aug 17, 2016 at 10:35 AM, amul sul wrote: > > >> Hmm. I haven't really looked into the code, but with applying both patches >> it looks precisely imitate Oracle's behaviour. Thanks. > > >This is good to hear, but for us to conside

Re: [HACKERS] errno clobbering in reorderbuffer

2016-08-18 Thread Andres Freund
On August 18, 2016 7:21:03 PM PDT, Alvaro Herrera wrote: >Andres Freund wrote: >> On 2016-08-18 19:06:02 -0300, Alvaro Herrera wrote: >> > While researching a customer issue with BDR I noticed that one >ereport() >> > call happens after clobbering errno, leading to the wrong strerror >being >>

Re: [HACKERS] errno clobbering in reorderbuffer

2016-08-18 Thread Alvaro Herrera
Andres Freund wrote: > On 2016-08-18 19:06:02 -0300, Alvaro Herrera wrote: > > While researching a customer issue with BDR I noticed that one ereport() > > call happens after clobbering errno, leading to the wrong strerror being > > reported. This patch fixes it by saving before calling > > CloseT

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-08-18 Thread Michael Paquier
On Thu, Aug 18, 2016 at 6:12 PM, Heikki Linnakangas wrote: > On 06/22/2016 04:41 AM, Michael Paquier wrote: >> make s >> OK, there is not much that we can do here then. What about the rest? >> Those seem like legit concerns to me. > > > There's also a realloc() and an strdup() call in refint.c. Bu

Re: [HACKERS] [PATCH] bigint txids vs 'xid' type, new txid_recent(bigint) => xid

2016-08-18 Thread Craig Ringer
On 19 August 2016 at 02:35, Jim Nasby wrote: > On 8/18/16 5:46 AM, Amit Kapila wrote: > >> I think there is a value in exposing such a variant which takes bigint >> and internally converts it to xid. I am not sure the semantics for >> > > I think that's a bad idea because you have the exact same

Re: [HACKERS] Most efficient way for libPQ .. PGresult serialization

2016-08-18 Thread Craig Ringer
On 19 August 2016 at 03:08, Joshua Bay wrote: > Thanks, > But I don't think my question was clear enough. > > I already managed the connection pooling, and what I need is to serialize > the result. > > If PGresult was a contiguous block, I could have just create buffer and > call memcpy for seria

Re: [HACKERS] Fix comment in ATExecValidateConstraint

2016-08-18 Thread Amit Langote
On 2016/08/19 5:35, Robert Haas wrote: > On Thu, Aug 18, 2016 at 5:15 AM, Amit Langote > wrote: >> On 2016/07/25 17:18, Amit Langote wrote: >>> The comment seems to have been copied from ATExecAddColumn, which says: >>> >>> /* >>> * If we are told not to recurse, there had better not be any >>>

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-08-18 Thread Peter Geoghegan
On Thu, Aug 18, 2016 at 5:35 PM, Michael Paquier wrote: > This would be packaged from source in my case, but that's no big deal > :) At least I can see that it is added in the next CF, and that's > marked as ready for committer for a couple of months now... If you consider how the code is written

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Michael Paquier
On Fri, Aug 19, 2016 at 4:25 AM, Tom Lane wrote: > BTW, I just noticed that cmake doesn't seem to be supplied as part of > Apple's dev tools, at least not up to current (El Capitan) releases. > That's going to be a rather large minus to be taken into account > whenever we make the go/no-go decisio

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-18 Thread Peter Geoghegan
On Thu, Aug 18, 2016 at 5:42 PM, Andres Freund wrote: > How large was the index & table in question? I mean this really only > comes into effect at 100+ segments. Not that big, but I see no reason to take the chance, I suppose. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-18 Thread Andres Freund
On 2016-08-18 17:35:47 -0700, Peter Geoghegan wrote: > On Thu, Aug 18, 2016 at 5:28 PM, Andres Freund wrote: > >> I can review this next week. > > > > Thanks > > Given the time frame that you have in mind, I won't revisit the > question the parallel CLUSTER CPU bottleneck issue until this is > co

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-18 Thread Peter Geoghegan
On Thu, Aug 18, 2016 at 5:28 PM, Andres Freund wrote: >> I can review this next week. > > Thanks Given the time frame that you have in mind, I won't revisit the question the parallel CLUSTER CPU bottleneck issue until this is committed. The patch might change things enough that that would be a wa

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-08-18 Thread Michael Paquier
On Fri, Aug 19, 2016 at 9:14 AM, Peter Geoghegan wrote: > On Thu, Aug 18, 2016 at 5:06 PM, Michael Paquier > wrote: >> Cool. I have been honestly wondering about deploying this tool as well >> to allow some of the QE tests to perform live checks of btree indexes >> as we use a bunch of them. > >

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-18 Thread Andres Freund
On 2016-08-18 17:27:59 -0700, Peter Geoghegan wrote: > On Thu, Aug 18, 2016 at 5:26 PM, Andres Freund wrote: > > Rebased version attached. A review would be welcome. Plan to push this > > forward otherwise in the not too far away future. > > I can review this next week. Thanks -- Sent via pgs

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-18 Thread Peter Geoghegan
On Thu, Aug 18, 2016 at 5:26 PM, Andres Freund wrote: > Rebased version attached. A review would be welcome. Plan to push this > forward otherwise in the not too far away future. I can review this next week. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-18 Thread Andres Freund
On 2016-06-30 18:14:15 -0700, Peter Geoghegan wrote: > On Tue, Dec 15, 2015 at 10:04 AM, Andres Freund wrote: > > Took a while. But here we go. The attached version is a significantly > > revised version of my earlier patch. Notably I've pretty much entirely > > revised the code in _mdfd_getseg()

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-08-18 Thread Peter Geoghegan
On Thu, Aug 18, 2016 at 5:06 PM, Michael Paquier wrote: > Cool. I have been honestly wondering about deploying this tool as well > to allow some of the QE tests to perform live checks of btree indexes > as we use a bunch of them. I'd certainly welcome that. There are Debian packages available fro

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-08-18 Thread Michael Paquier
On Fri, Aug 19, 2016 at 2:40 AM, Peter Geoghegan wrote: > Heroku began a selective roll-out of amcheck yesterday. amcheck > already found a bug in the PostGiS Geography B-Tree opclass: > [...] > I'll go report this to the PostGiS people. Cool. I have been honestly wondering about deploying this t

Re: [HACKERS] anyelement -> anyrange

2016-08-18 Thread Corey Huinker
I'd be happy to roll your code into the extension, and make it marked more stable. On Thu, Aug 18, 2016 at 2:49 PM, Jim Nasby wrote: > On 8/18/16 1:06 PM, Corey Huinker wrote: > >> You might also find some gleanable gems in: >> https://github.com/moat/range_type_functions/blob/master/doc >> /ran

Re: [HACKERS] errno clobbering in reorderbuffer

2016-08-18 Thread Andres Freund
Hi, On 2016-08-18 19:06:02 -0300, Alvaro Herrera wrote: > if (write(fd, rb->outbuf, ondisk->size) != ondisk->size) > { > + int save_errno = errno; > + > CloseTransientFile(fd); > + errno = save_errno; > ereport(ERROR, >

Re: [HACKERS] errno clobbering in reorderbuffer

2016-08-18 Thread Tom Lane
Alvaro Herrera writes: > While researching a customer issue with BDR I noticed that one ereport() > call happens after clobbering errno, leading to the wrong strerror being > reported. This patch fixes it by saving before calling > CloseTransientFile and restoring afterwards. > I also threw in a

Re: [HACKERS] errno clobbering in reorderbuffer

2016-08-18 Thread Andres Freund
On 2016-08-18 19:06:02 -0300, Alvaro Herrera wrote: > While researching a customer issue with BDR I noticed that one ereport() > call happens after clobbering errno, leading to the wrong strerror being > reported. This patch fixes it by saving before calling > CloseTransientFile and restoring afte

Re: [HACKERS] drop src/backend/port/darwin/system.c ?

2016-08-18 Thread Peter Eisentraut
On 8/17/16 12:29 PM, Tom Lane wrote: > Also, the early releases of OS X were rough enough that it's pretty hard > to believe anyone is still using them anywhere (certainly the buildfarm > isn't). So the odds of anyone caring if we remove this file seem > negligible. Let's nuke it. done -- Pete

Re: [HACKERS] pg_basebackup wish list

2016-08-18 Thread Peter Eisentraut
On 7/12/16 9:55 PM, Masahiko Sawada wrote: > And what I think is pg_baseback never remove the directory specified > by -D option even if execution is failed. initdb command behaves so. > I think it's helpful for backup operation. This has been bothering me as well. How about the attached patch as

[HACKERS] errno clobbering in reorderbuffer

2016-08-18 Thread Alvaro Herrera
While researching a customer issue with BDR I noticed that one ereport() call happens after clobbering errno, leading to the wrong strerror being reported. This patch fixes it by saving before calling CloseTransientFile and restoring afterwards. I also threw in a missing errcode I noticed while l

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Claudio Freire
On Thu, Aug 18, 2016 at 6:38 PM, Peter Geoghegan wrote: > On Thu, Aug 18, 2016 at 2:26 PM, Claudio Freire > wrote: >> I see that. I could try to measure average depth to measure the impact >> this had on fan-in. >> >> While it should cut it in half for narrow indexes, half of very high >> is sti

Re: [HACKERS] [PATCH] add option to pg_dumpall to exclude tables from the dump

2016-08-18 Thread Tom Lane
Jim Nasby writes: > On 8/18/16 2:40 PM, Tom Lane wrote: >> This seems pretty dubious to me, in particular that the identical -T >> option will be passed willy-nilly into the pg_dump runs for every >> database. That seems more likely to be a foot-gun than something useful. > I agree, but I think

Re: [HACKERS] Curing plpgsql's memory leaks for statement-lifespan values

2016-08-18 Thread Jim Nasby
On 7/25/16 1:50 PM, Tom Lane wrote: There's a glibc-dependent hack in aset.c that reports any plpgsql-driven palloc or pfree against a context named "SPI Proc", as well as changes in pl_comp.c so that transient junk created during initial parsing of a plpgsql function body doesn't end up in the S

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Peter Geoghegan
On Thu, Aug 18, 2016 at 2:26 PM, Claudio Freire wrote: > I see that. I could try to measure average depth to measure the impact > this had on fan-in. > > While it should cut it in half for narrow indexes, half of very high > is still high. Wide indexes, which are are the ones that would suffer > f

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Claudio Freire
On Thu, Aug 18, 2016 at 6:27 PM, Tom Lane wrote: > Claudio Freire writes: >> On Thu, Aug 18, 2016 at 6:04 PM, Kevin Grittner wrote: >>> Speaking of performance side effects, does this avoid O(N^2) >>> performance on index tuple insertion with duplicate values, for all >>> insertion orderings? F

Re: [HACKERS] [PATCH] add option to pg_dumpall to exclude tables from the dump

2016-08-18 Thread Jim Nasby
On 8/18/16 2:40 PM, Tom Lane wrote: This seems pretty dubious to me, in particular that the identical -T option will be passed willy-nilly into the pg_dump runs for every database. That seems more likely to be a foot-gun than something useful. I agree, but I think mandating a database name (wh

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Tom Lane
Claudio Freire writes: > On Thu, Aug 18, 2016 at 6:04 PM, Kevin Grittner wrote: >> Speaking of performance side effects, does this avoid O(N^2) >> performance on index tuple insertion with duplicate values, for all >> insertion orderings? For example, does it descend directly to the >> right lea

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Claudio Freire
On Thu, Aug 18, 2016 at 6:15 PM, Peter Geoghegan wrote: > On Thu, Aug 18, 2016 at 1:41 PM, Claudio Freire > wrote: >> In fact, that's why non-leaf index tuples need a different format, >> because while leaf index tuples contain the heap pointer already, >> non-leaf ones contain only the downlink

Re: [HACKERS] distinct estimate of a hard-coded VALUES list

2016-08-18 Thread Tom Lane
Jeff Janes writes: > So even though it knows that 6952 values have been shoved in the bottom, it > thinks only 200 are going to come out of the aggregation. This seems like > a really lousy estimate. In more complex queries than the example one > given it leads to poor planning choices. > Is th

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Alvaro Herrera
Claudio Freire wrote: > Unique indexes still need to scan all duplicates to check visibility > and will become O(N^2) there. That scenario doesn't matter, because on unique indexes there aren't many duplicate values anyway -- only one can be a live tuple. -- Álvaro Herrerahttp:/

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Peter Geoghegan
On Thu, Aug 18, 2016 at 1:41 PM, Claudio Freire wrote: > In fact, that's why non-leaf index tuples need a different format, > because while leaf index tuples contain the heap pointer already, > non-leaf ones contain only the downlink, not the pointer into the > heap. To be able to do comparisons a

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Claudio Freire
On Thu, Aug 18, 2016 at 6:04 PM, Kevin Grittner wrote: > On Thu, Aug 18, 2016 at 3:41 PM, Claudio Freire > wrote: > >> It also makes index scans of the form >> >> SELECT * FROM t WHERE some_col = some_const; >> >> Scan the heap in sequential order, even if some_col has low >> cardinality (ie: th

Re: [HACKERS] Improving planner's checks for parallel-unsafety

2016-08-18 Thread Tom Lane
Robert Haas writes: > I have reviewed this and it looks good to me. My only comment is that > this comment is slightly confusing: > ! * Returns the first of PROPARALLEL_UNSAFE, PROPARALLEL_RESTRICTED, or > ! * PROPARALLEL_SAFE that can be found in the given parsetree. We use this > "First" m

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Kevin Grittner
On Thu, Aug 18, 2016 at 3:41 PM, Claudio Freire wrote: > It also makes index scans of the form > > SELECT * FROM t WHERE some_col = some_const; > > Scan the heap in sequential order, even if some_col has low > cardinality (ie: the query returns lots of tuples), which is a nice > performance side

[HACKERS] distinct estimate of a hard-coded VALUES list

2016-08-18 Thread Jeff Janes
I have a query which contains a where clause like: aid =ANY(VALUES (1),(45),(87), <6948 more>, (447)) for example: perl -le 'print "explain (analyze) select sum(abalance) from pgbench_accounts where aid=ANY(VALUES "; print join ",", map "($_)", sort {$a<=>$b} map int(rand(500)), 1..6952

Re: [HACKERS] Improving planner's checks for parallel-unsafety

2016-08-18 Thread Robert Haas
On Thu, Aug 18, 2016 at 12:39 PM, Tom Lane wrote: > Attached is a patch I'd fooled around with back in July but not submitted. > The idea is that, if our initial scan of the query tree found only > parallel-safe functions, there is no need to rescan subsets of the tree > looking for parallel-restr

Re: [HACKERS] Patch: initdb: "'" for QUOTE_PATH (non-windows)

2016-08-18 Thread Ryan Murphy
On Thu, Aug 18, 2016 at 3:44 PM, Tom Lane wrote: > Ryan Murphy writes: > > On Thu, Aug 18, 2016 at 3:22 PM, Robert Haas > wrote: > +{ /* pg_ctl command w path, properly quoted */ > +PQExpBuffer pg_ctl_path = createPQExpBuffer(); > +printfPQExpBuffe

Re: [HACKERS] Making pg_hba.conf case-insensitive

2016-08-18 Thread Bruce Momjian
On Thu, Aug 18, 2016 at 03:01:48PM -0400, Peter Eisentraut wrote: > On 8/18/16 1:59 PM, Bruce Momjian wrote: > > o compares words in columns that can only support keywords as > > case-insensitive, double-quoted or not > > > > o compares words in columns that can contain user/db names or keywords

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Peter Eisentraut
On 8/18/16 4:23 PM, Christian Convey wrote: > What standard would you suggest for those platforms which don't have > an obvious default version of CMake? In the olden days, when many platforms we supported didn't come with GNU make or other GNU tools or even a compiler, we collected a lot of pract

Re: [HACKERS] Patch: initdb: "'" for QUOTE_PATH (non-windows)

2016-08-18 Thread Tom Lane
Ryan Murphy writes: > On Thu, Aug 18, 2016 at 3:22 PM, Robert Haas wrote: +{ /* pg_ctl command w path, properly quoted */ +PQExpBuffer pg_ctl_path = createPQExpBuffer(); +printfPQExpBuffer(pg_ctl_path, "%s%spg_ctl", >> I think it's worth reducing th

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Claudio Freire
On Thu, Aug 18, 2016 at 5:05 PM, Robert Haas wrote: > On Wed, Aug 17, 2016 at 10:54 PM, Claudio Freire > wrote: >> The attached patch tries to maintain the initial status of B-Tree >> indexes, which are created with equal-key runs in physical order, >> during the whole life of the B-Tree, and ma

Re: [HACKERS] Fix comment in ATExecValidateConstraint

2016-08-18 Thread Robert Haas
On Thu, Aug 18, 2016 at 5:15 AM, Amit Langote wrote: > On 2016/07/25 17:18, Amit Langote wrote: >> The comment seems to have been copied from ATExecAddColumn, which says: >> >> /* >> * If we are told not to recurse, there had better not be any >> - * child tables; else the addition would put th

Re: [HACKERS] Patch: initdb: "'" for QUOTE_PATH (non-windows)

2016-08-18 Thread Ryan Murphy
On Thu, Aug 18, 2016 at 3:22 PM, Robert Haas wrote: > On Thu, Aug 18, 2016 at 4:15 PM, Andres Freund wrote: > > On 2016-08-18 16:11:27 -0400, Robert Haas wrote: > >> On Wed, Aug 17, 2016 at 11:18 PM, Andres Freund > wrote: > >> > On August 17, 2016 8:15:56 PM PDT, Michael Paquier < > michael.pa

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Tom Lane
Christian Convey writes: >> well I personally think the level to meet would be that all the systems >> on the buildfarm that can build -HEAD at the time the patch is proposed >> for a commit should be able to build using the new system with whatever >> cmake version is available in those by defaul

Re: [HACKERS] synchronous_commit = remote_flush

2016-08-18 Thread Robert Haas
On Thu, Aug 18, 2016 at 12:22 AM, Thomas Munro wrote: > To do something about the confusion I keep seeing about what exactly > "on" means, I've often wished we had "remote_flush". But it's not > obvious how the backwards compatibility could work, ie how to keep the > people happy who use "local"

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Christian Convey
Hi Stefan, >> I ask because I'm curious if/how someone in Yury's situation could >> predict which minimum version of CMake must be supported in order for >> his patch to be accepted. (And if he accepts my offer to pitch in, >> I'll actually need that particular detail.) > > well I personally thin

Re: [HACKERS] Patch: initdb: "'" for QUOTE_PATH (non-windows)

2016-08-18 Thread Robert Haas
On Thu, Aug 18, 2016 at 4:15 PM, Andres Freund wrote: > On 2016-08-18 16:11:27 -0400, Robert Haas wrote: >> On Wed, Aug 17, 2016 at 11:18 PM, Andres Freund wrote: >> > On August 17, 2016 8:15:56 PM PDT, Michael Paquier >> > wrote: >> > >> >>+{ /* pg_ctl command w path, properly quoted *

Re: [HACKERS] Patch: initdb: "'" for QUOTE_PATH (non-windows)

2016-08-18 Thread Andres Freund
On 2016-08-18 16:11:27 -0400, Robert Haas wrote: > On Wed, Aug 17, 2016 at 11:18 PM, Andres Freund wrote: > > On August 17, 2016 8:15:56 PM PDT, Michael Paquier > > wrote: > > > >>+{ /* pg_ctl command w path, properly quoted */ > >>+PQExpBuffer pg_ctl_path = createPQExpBuffer

Re: [HACKERS] Patch: initdb: "'" for QUOTE_PATH (non-windows)

2016-08-18 Thread Robert Haas
On Wed, Aug 17, 2016 at 11:18 PM, Andres Freund wrote: > On August 17, 2016 8:15:56 PM PDT, Michael Paquier > wrote: > >>+{ /* pg_ctl command w path, properly quoted */ >>+PQExpBuffer pg_ctl_path = createPQExpBuffer(); >>+printfPQExpBuffer(pg_ctl_path, "%s%spg_ctl

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Andres Freund
On 2016-08-18 15:55:20 -0400, Christian Convey wrote: > * Allow the CMake-based build system to assume a fairly modern version > of CMake. (Maybe 2.8.12, or 3.0.) > > * For systems where the minimum CMake version isn't readily available, > have an alternative build system which is just a simplist

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Alvaro Herrera
Christian Convey wrote: > I see. In other projects I've worked on, the configuration of a build > farm has been driven by some list of platforms that were considered > important to support. Build farm members are systems that somebody has seen as interesting enough that they deserve enough effor

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-18 Thread Robert Haas
On Wed, Aug 17, 2016 at 10:54 PM, Claudio Freire wrote: > The attached patch tries to maintain the initial status of B-Tree > indexes, which are created with equal-key runs in physical order, > during the whole life of the B-Tree, and make key-tid pairs > efficiently searchable in the process. I

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Christian Convey
> > I don't think we're interested in maintaining more build systems than we > already are. If cmake can replace the current MSVC tooling and > autoconf, all in one, my impression is that we're in. If we're > replacing two tools we've hammered pretty well over the years with two > tools that we h

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Alvaro Herrera
Christian Convey wrote: > * Allow the CMake-based build system to assume a fairly modern version > of CMake. (Maybe 2.8.12, or 3.0.) > > * For systems where the minimum CMake version isn't readily available, > have an alternative build system which is just a simplistic Bash > script that naively

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Christian Convey
Hi Tom, >> I ask because I'm curious if/how someone in Yury's situation could >> predict which minimum version of CMake must be supported in order for >> his patch to be accepted. (And if he accepts my offer to pitch in, >> I'll actually need that particular detail.) > > well I personally think t

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Stefan Kaltenbrunner
On 08/18/2016 09:52 PM, Alvaro Herrera wrote: > Stefan Kaltenbrunner wrote: >> On 08/18/2016 09:30 PM, Christian Convey wrote: > >>> Yury: Would it make sense to add a call to "cmake_minimum_required" in >>> one or more of your CMakeLists.txt files? >> >> it would make sense nevertheless but I don

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Christian Convey
Hi Stefan, >> Yury: Would it make sense to add a call to "cmake_minimum_required" in >> one or more of your CMakeLists.txt files? > > it would make sense nevertheless but I dont think that 2.8.11 is old > enough - looking at the release information and the feature compatibily > matrix it would see

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Stefan Kaltenbrunner
On 08/18/2016 09:42 PM, Christian Convey wrote: > Hi Tom, > > Thanks for that information. > > Is there some document I can read that explains which platform > versions (e.g., OpenBSD 5.3) are considered strongly supported? well not sure we have very clear document on that - I would say that the

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Alvaro Herrera
Stefan Kaltenbrunner wrote: > On 08/18/2016 09:30 PM, Christian Convey wrote: > > Yury: Would it make sense to add a call to "cmake_minimum_required" in > > one or more of your CMakeLists.txt files? > > it would make sense nevertheless but I dont think that 2.8.11 is old > enough - looking at the

Re: [HACKERS] anyelement -> anyrange

2016-08-18 Thread David E. Wheeler
On Aug 18, 2016, at 11:49 AM, Jim Nasby wrote: > Well crap, I searched for range stuff on PGXN before creating > http://pgxn.org/dist/range_tools/ and the only thing that came up was your > range_partitioning stuff, which AFAICT is unrelated. > http://pgxn.org/dist/range_type_functions/still d

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Christian Convey
Hi Tom, Thanks for that information. Is there some document I can read that explains which platform versions (e.g., OpenBSD 5.3) are considered strongly supported? I ask because I'm curious if/how someone in Yury's situation could predict which minimum version of CMake must be supported in order

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Stefan Kaltenbrunner
On 08/18/2016 09:30 PM, Christian Convey wrote: > Hi Karl, > > I'll need to let Yury answer your original question regarding the best > way to report CMake-related bugs. > > Regarding the errors you're getting... I just looked at CMake's > online documentation regarding your "target_compile_defi

Re: [HACKERS] [PATCH] add option to pg_dumpall to exclude tables from the dump

2016-08-18 Thread Tom Lane
Juergen Hannappel writes: > A new option -T --exlude-table for pg_dumpall. This option is then > passed through to the pg_dump which really does the work. > This feature can be used to exclude large tables that are known not > to change from a database backup dump so that only the changing parts >

[HACKERS] Re: [COMMITTERS] pgsql: Fix deletion of speculatively inserted TOAST on conflict

2016-08-18 Thread Alvaro Herrera
Andres Freund wrote: > This commit also adds a isolationtester spec test, exercising the > relevant code path. Unfortunately 9.5 cannot handle two waiting > sessions, and thus cannot execute this test. This test seems to fail randomly, depending on which session is awakened first. Annoying ...

Re: [HACKERS] Re: PROPOSAL: make PostgreSQL sanitizers-friendly (and prevent information disclosure)

2016-08-18 Thread Peter Eisentraut
On 3/22/16 9:27 AM, Aleksander Alekseev wrote: > diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c > index 28f9fb5..45aa802 100644 > --- a/src/backend/libpq/hba.c > +++ b/src/backend/libpq/hba.c > @@ -1008,14 +1008,9 @@ parse_hba_line(List *line, int line_num, char > *raw_line) >

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Christian Convey
Hi Karl, I'll need to let Yury answer your original question regarding the best way to report CMake-related bugs. Regarding the errors you're getting... I just looked at CMake's online documentation regarding your "target_compile_definitions" error. >From what I can tell, the "target_compile_de

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Tom Lane
Stefan Kaltenbrunner writes: > On 08/18/2016 08:57 PM, Christian Convey wrote: >> Which version of CMake gave you those errors? (Sorry if you provided >> that detail and I'm just missing it.) > % cmake --version > cmake version 2.8.10.2 > a quick look in the docs does not seem to reveal any kin

Re: [HACKERS] Use pread and pwrite instead of lseek + write and read

2016-08-18 Thread Tom Lane
Robert Haas writes: > Well, I think you're pointing out some things that need to be figured > out, but I hardly think that's a good enough reason to pour cold water > on the whole approach. If somebody feels like doing the legwork to find out if those performance hazards are real (which I freely

Re: [HACKERS] Most efficient way for libPQ .. PGresult serialization

2016-08-18 Thread Joshua Bay
Thanks, But I don't think my question was clear enough. I already managed the connection pooling, and what I need is to serialize the result. If PGresult was a contiguous block, I could have just create buffer and call memcpy for serialization, but structure of result seems much more complicated.

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Stefan Kaltenbrunner
On 08/18/2016 08:57 PM, Christian Convey wrote: > Hi Stefan, > > I think I've seen similar errors when a project's CMake files assumed > a newer version of CMake than the one being run. > > Which version of CMake gave you those errors? (Sorry if you provided > that detail and I'm just missing it

Re: [HACKERS] Making pg_hba.conf case-insensitive

2016-08-18 Thread Peter Eisentraut
On 8/18/16 1:59 PM, Bruce Momjian wrote: > o compares words in columns that can only support keywords as > case-insensitive, double-quoted or not > > o compares words in columns that can contain user/db names or keywords > as case-sensitive if double-quoted, case-insensitive if not I can maybe

Re: [HACKERS] Making pg_hba.conf case-insensitive

2016-08-18 Thread Tom Lane
Bruce Momjian writes: > On Thu, Aug 18, 2016 at 02:06:39PM -0400, Tom Lane wrote: >> Is there really enough demand for this to justify subtle breakage >> of existing pg_hba.conf entries? It'd probably have been fine if >> we did it like that originally, but I think it's a bit late now. > Well, i

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Christian Convey
Hi Stefan, I think I've seen similar errors when a project's CMake files assumed a newer version of CMake than the one being run. Which version of CMake gave you those errors? (Sorry if you provided that detail and I'm just missing it.) Kind regards, Christian On Thu, Aug 18, 2016 at 2:45 PM,

Re: [HACKERS] anyelement -> anyrange

2016-08-18 Thread Jim Nasby
On 8/18/16 1:06 PM, Corey Huinker wrote: You might also find some gleanable gems in: https://github.com/moat/range_type_functions/blob/master/doc/range_type_functions.md Well crap, I searched for range stuff on PGXN before creating http://pgxn.org/dist/range_tools/ and the only thing that came

Re: [HACKERS] Add -c to rsync commands on SR tutorial wiki page

2016-08-18 Thread Jim Nasby
On 8/18/16 1:31 PM, Stephen Frost wrote: To have proper incremental backups done requires a lot more than just throwing "-c" into the rsyncs. For my 2c, I'm at the point where I'd prefer we discourage people from using rsync, cp, or generally try to set up their own hand-rolled backup system wit

Re: [HACKERS] Making pg_hba.conf case-insensitive

2016-08-18 Thread Bruce Momjian
On Thu, Aug 18, 2016 at 02:06:39PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I was looking at this TODO item from 2009: > > https://www.postgresql.org/message-id/4AA7B197.70002%40usit.uio.no > > I have implemented this in the attached patch. It does two things: > > Is there really e

Re: [HACKERS] WIP: About CMake v2

2016-08-18 Thread Stefan Kaltenbrunner
On 06/29/2016 06:23 PM, Yury Zhuravlev wrote: > Hello Hackers. > > I decided to talk about the current state of the project: > 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, plsql > all work correctly (all tests pass). > 3. Works done for all contrib modules. 4. You can use gett

Re: [HACKERS] [Patch] New psql prompt substitution %r (m = master, r = replica)

2016-08-18 Thread Jim Nasby
On 8/18/16 10:05 AM, Alvaro Herrera wrote: Bash has PROMPT_COMMAND, which is executed prior to issuing each prompt. What about introducing such a feature into psql? Then the \gset command you had in your first post could be used to set the variable correctly just before each prompt. As someone

Re: [HACKERS] Bug in to_timestamp().

2016-08-18 Thread Robert Haas
On Wed, Aug 17, 2016 at 10:35 AM, amul sul wrote: > Hmm. I haven't really looked into the code, but with applying both patches it > looks precisely imitate Oracle's behaviour. Thanks. This is good to hear, but for us to consider applying something like this, somebody would need to look into the

Re: [HACKERS] Use pread and pwrite instead of lseek + write and read

2016-08-18 Thread Robert Haas
On Wed, Aug 17, 2016 at 3:11 PM, Tom Lane wrote: > Robert Haas writes: >> I don't understand why you think this would create non-trivial >> portability issues. > > The patch as submitted breaks entirely on platforms without pread/pwrite. > Yes, we can add a configure test and some shim functions

Re: [HACKERS] [PATCH] bigint txids vs 'xid' type, new txid_recent(bigint) => xid

2016-08-18 Thread Jim Nasby
On 8/18/16 5:46 AM, Amit Kapila wrote: I think there is a value in exposing such a variant which takes bigint and internally converts it to xid. I am not sure the semantics for I think that's a bad idea because you have the exact same problems we have now: bigint is signed, epoch is not. t

Re: [HACKERS] Add -c to rsync commands on SR tutorial wiki page

2016-08-18 Thread Stephen Frost
* Jim Nasby (jim.na...@bluetreble.com) wrote: > On 8/17/16 9:46 PM, Stephen Frost wrote: > >* Jim Nasby (jim.na...@bluetreble.com) wrote: > >>> https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does > >>> not specify -c for any of the rsync commands. That's maybe safe for > >>> WAL, but

Re: [HACKERS] synchronous_commit = remote_flush

2016-08-18 Thread Jim Nasby
On 8/17/16 11:22 PM, Thomas Munro wrote: Hi hackers, To do something about the confusion I keep seeing about what exactly "on" means, I've often wished we had "remote_flush". But it's not obvious how the backwards compatibility could work, ie how to keep the people happy who use "local" vs "on"

Re: [HACKERS] Add -c to rsync commands on SR tutorial wiki page

2016-08-18 Thread Jim Nasby
On 8/17/16 9:46 PM, Stephen Frost wrote: * Jim Nasby (jim.na...@bluetreble.com) wrote: > https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does > not specify -c for any of the rsync commands. That's maybe safe for > WAL, but I don't think it's safe for any of the other uses, right? >

Re: [HACKERS] anyelement -> anyrange

2016-08-18 Thread Corey Huinker
On Tue, Aug 16, 2016 at 9:29 PM, Tom Lane wrote: > Jim Nasby writes: > > I can't think of any reason you'd want two different range types on a > > single element type. > > We would not have built it that way if there were not clear use-cases. > An easy example is you might want both a continuous

Re: [HACKERS] Making pg_hba.conf case-insensitive

2016-08-18 Thread Tom Lane
Bruce Momjian writes: > I was looking at this TODO item from 2009: > https://www.postgresql.org/message-id/4AA7B197.70002%40usit.uio.no > I have implemented this in the attached patch. It does two things: Is there really enough demand for this to justify subtle breakage of existing pg_hba.

[HACKERS] Making pg_hba.conf case-insensitive

2016-08-18 Thread Bruce Momjian
I was looking at this TODO item from 2009: https://www.postgresql.org/message-id/4AA7B197.70002%40usit.uio.no I have implemented this in the attached patch. It does two things: o compares words in columns that can only support keywords as case-insensitive, double-quoted or not o comp

[HACKERS] Re: PROPOSAL: make PostgreSQL sanitizers-friendly (and prevent information disclosure)

2016-08-18 Thread Heikki Linnakangas
On 03/22/2016 03:27 PM, Aleksander Alekseev wrote: diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c index 1ff5728..a10c078 100644 --- a/src/backend/commands/tablespace.c +++ b/src/backend/commands/tablespace.c @@ -669,6 +669,11 @@ destroy_tablespace_directories(

  1   2   >