Re: WIP: Avoid creation of the free space map for small tables

2018-11-02 Thread Amit Kapila
On Fri, Nov 2, 2018 at 7:37 PM Tom Lane wrote: > > Robert Haas writes: > > That's not what I'm saying. If we don't have the FSM, we have to > > check every page of the table. If there's a workload where that > > happens a lot on a table that is just under the size threshold for > > creating

Re: WIP: Avoid creation of the free space map for small tables

2018-11-02 Thread Amit Kapila
On Fri, Nov 2, 2018 at 7:29 PM Robert Haas wrote: > > On Fri, Nov 2, 2018 at 7:23 AM Amit Kapila wrote: > > > That doesn't seem like an unreasonable argument. I'm not sure whether > > > the right threshold is 4 or something a little bigger, but I bet it's > > > not very large. It seems

Re: zheap: a new storage format for PostgreSQL

2018-11-02 Thread Amit Kapila
On Fri, Nov 2, 2018 at 6:41 PM Tomas Vondra wrote: > > On 11/02/2018 12:12 PM, Amit Kapila wrote: > > On Thu, Nov 1, 2018 at 7:26 PM Tomas Vondra > > wrote: > >> > >> On 11/01/2018 07:43 AM, Amit Kapila wrote: > >>> > >>> You can find the latest code at https://github.com/EnterpriseDB/zheap >

Re: pgbench -M option can be specified more than once

2018-11-02 Thread Tom Lane
Andres Freund writes: > On 2018-11-03 10:12:14 +0900, Tatsuo Ishii wrote: >> One of my colleagues actually believed that if both "-M extended" and >> "-M prepared" were specified, pgbench runs in mixture of those >> modes. So I felt avoiding such misunderstanding is more important. > I regularly

Re: COPY FROM WHEN condition

2018-11-02 Thread Corey Huinker
> > > > SELECT x.a, sum(x.b) > > FROM ( COPY INLINE '/path/to/foo.txt' FORMAT CSV ) as x( a integer, b > numeric, c text, d date, e json) ) > > Apologies for bike-shedding, but wouldn't the following be a better > fit with the current COPY? > > COPY t(a integer, b numeric, c text, d date, e

Re: pgbench -M option can be specified more than once

2018-11-02 Thread Andres Freund
Hi, On 2018-11-03 10:12:14 +0900, Tatsuo Ishii wrote: > > Tatsuo Ishii writes: > >> While playing with pgbench, I found multiple "-M query_mode" can be > >> set more than once. For example, > > > > I think that's true of just about every option in all of our programs. > > Why is this one

Re: pgbench -M option can be specified more than once

2018-11-02 Thread Isaac Morland
In many cases, it is handy to be able to specify an option in an alias, but still be able to override on the actual command line. I can't say whether that is useful with this specific option, but it seems the same debate could be had about almost any option. I'm pretty sure the existing behaviour

Re: pgbench -M option can be specified more than once

2018-11-02 Thread Tatsuo Ishii
> Tatsuo Ishii writes: >> While playing with pgbench, I found multiple "-M query_mode" can be >> set more than once. For example, > > I think that's true of just about every option in all of our programs. > Why is this one instance so much worse than others that it deserves > to be handled

Re: pgbench -M option can be specified more than once

2018-11-02 Thread Tom Lane
Tatsuo Ishii writes: > While playing with pgbench, I found multiple "-M query_mode" can be > set more than once. For example, I think that's true of just about every option in all of our programs. Why is this one instance so much worse than others that it deserves to be handled differently?

pgbench -M option can be specified more than once

2018-11-02 Thread Tatsuo Ishii
While playing with pgbench, I found multiple "-M query_mode" can be set more than once. For example, $ pgbench -p 11002 -M extended -S -M prepared test starting vacuum...end. transaction type: scaling factor: 1 query mode: prepared number of clients: 1 number of threads: 1 number of transactions

First-draft release notes for back-branch releases

2018-11-02 Thread Tom Lane
I've made a pass over the commit log up to now, and prepared draft release note entries for everything that seemed worth documenting; see https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=65a82a7649860f8010db581a0d1f12aa92f5969b As I usually do, I dropped all of these into a

Re: pgbench doc fix

2018-11-02 Thread Tatsuo Ishii
> So I do not think a more precise wording harms. Maybe: "prepared: use > extended query protocol with REUSED named prepared statements" would > be even less slightly ambiguous. I like this. But maybe we can remove "named"? "prepared: use extended query protocol with reused prepared statements"

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-11-02 Thread Peter Geoghegan
On Fri, Nov 2, 2018 at 3:06 AM Andrey Lepikhov wrote: > Documentation is full and clear. All non-trivial logic is commented > accurately. Glad you think so. I had the opportunity to discuss this patch at length with Heikki during pgConf.EU. I don't want to speak on his behalf, but I will say

Re: partitioned tables referenced by FKs

2018-11-02 Thread Alvaro Herrera
Oh, I forgot to mention one thing. When creating a constraint, an index OID is normally given. I'm not sure what is this for. In the patch it's a little convoluted to get the correct index OID, so I'm just passing InvalidOid. Things work nonetheless. I wonder if we shouldn't just do away with

partitioned tables referenced by FKs

2018-11-02 Thread Alvaro Herrera
Here's a patch to allow partitioned tables to be referenced by foreign keys. Current state is WIP, but everything should work; see below for the expected exception. The design is very simple: have one pg_constraint row for each partition on each side, each row pointing to the topmost table on

Re: INSTALL file

2018-11-02 Thread Michael Paquier
On Fri, Nov 02, 2018 at 06:47:19AM -0400, Stephen Frost wrote: > As for what's in the README on the master branch, I was saying that it > *should* point to the development documentation, since that should be > current with whatever is actually in the git repo (or only a day behind > or such).

Re: INSTALL file

2018-11-02 Thread Andreas 'ads' Scherbaum
On 02.11.18 01:38, Stephen Frost wrote: Greetings, * Andreas 'ads' Scherbaum (a...@pgug.de) wrote: How about the attached one? Picked up your draft, and cleaned it up a bit. (unsurprisingly) this is looking pretty good to me. A few additional notes: Incorporated. See the attached. If that

Re: partitioned indexes and tablespaces

2018-11-02 Thread Alvaro Herrera
On 2018-Nov-03, Michael Paquier wrote: > On Fri, Nov 02, 2018 at 03:53:51PM -0300, Alvaro Herrera wrote: > > In this thread I'm not proposing to change the behavior for tables, only > > for indexes. If people want to change behavior for tables (and I agree > > with doing so), they can start

Re: pg_promote not marked as parallel-restricted in pg_proc.dat

2018-11-02 Thread Michael Paquier
On Fri, Nov 02, 2018 at 09:27:39AM +0900, Michael Paquier wrote: > You actually do a lot, moving just one person with MP as initials to > consider moving the function as being parallel-safe. Thanks for the > points you raised, what needs to be done looks clear now. So anybody has an objection

Re: partitioned indexes and tablespaces

2018-11-02 Thread Michael Paquier
On Fri, Nov 02, 2018 at 03:53:51PM -0300, Alvaro Herrera wrote: > In this thread I'm not proposing to change the behavior for tables, only > for indexes. If people want to change behavior for tables (and I agree > with doing so), they can start their own threads. Changing this behavior on

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Andres Freund
On 2018-11-02 23:24:35 +0100, David Fetter wrote: > On Wed, Oct 31, 2018 at 11:18:46AM -0400, Stephen Frost wrote: > > Greetings, > > > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > > Stephen Frost writes: > > > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > > >> I dunno, I do not think it's a

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread David Fetter
On Wed, Oct 31, 2018 at 11:18:46AM -0400, Stephen Frost wrote: > Greetings, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Stephen Frost writes: > > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > >> I dunno, I do not think it's a great idea to expose jsonb's internal > > >> format to the world.

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Andrew Dunstan
On 11/02/2018 05:20 PM, Andres Freund wrote: Hi, On 2018-11-02 17:02:24 -0400, Andrew Dunstan wrote: On 11/02/2018 11:34 AM, Merlin Moncure wrote: Binary format consuming applications already have to deal with these kinds of issues. We already expose internal structures in the other

Special role for subscriptions

2018-11-02 Thread Evgeniy Efimkin
Hi hackers! In postgresql 10 and 11 only superuser can create/alter subscriptions. If there was a special role (like pg_monitor), it would be more easy to grant control on subscriptions. I can make a patch if there are no objections against it.

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Andres Freund
Hi, On 2018-11-02 17:02:24 -0400, Andrew Dunstan wrote: > On 11/02/2018 11:34 AM, Merlin Moncure wrote: > > > > Binary format consuming applications already have to deal with these > > kinds of issues. We already expose internal structures in the other > > functions -- not sure why jsonb is held

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Andres Freund
Hi, On 2018-11-02 11:52:59 -0400, Tom Lane wrote: > Andres' point about alignment is a pretty good one as well, if it applies > here --- I don't recall just what internal alignment requirements jsonb > has. We have not historically expected clients to have to deal with that. Certainly looks

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Andres Freund
On 2018-11-02 10:34:07 -0500, Merlin Moncure wrote: > On Wed, Oct 31, 2018 at 10:23 AM Andres Freund wrote: > > > > Hi, > > > > On 2018-10-31 11:13:13 -0400, Andrew Dunstan wrote: > > > I agree that just sending a blob of the internal format isn't a great > > > idea. > > > > It's entirely

Re: Should pg 11 use a lot more memory building an spgist index?

2018-11-02 Thread Bruno Wolff III
I see that a fix got committed. Thanks! I'll double check it after the point release comes out (which looks like it will be next week) and let you know if there is still a problem.

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Andrew Dunstan
On 11/02/2018 11:34 AM, Merlin Moncure wrote: Binary format consuming applications already have to deal with these kinds of issues. We already expose internal structures in the other functions -- not sure why jsonb is held to a different standard. For other data types where format changes

Re: [PATCH][PROPOSAL] Add enum releation option type

2018-11-02 Thread Nikolay Shaplov
В письме от 1 ноября 2018 18:26:20 пользователь Nikolay Shaplov написал: > In this case the only solution I can see is > > DETAIL: Valid values are: "value1", "value2", "value3". > > Where list '"value1", "value2", "value3"' is built in runtime but have no > any bindnings to any specific

Re: CF app feature request

2018-11-02 Thread Dmitry Dolgov
On Fri, 2 Nov 2018 at 10:24, Fabien COELHO wrote: > > > Bonjour Michaël, > > >> Because the same patch submission is already counted? It is a rare > >> occurence, so just a "Withdrawn" state could be enough, and slightly false > >> CF stats are no big deal. > > > > Or as we are dealing with

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Merlin Moncure
On Fri, Nov 2, 2018 at 2:34 PM Stephen Frost wrote: > * Merlin Moncure (mmonc...@gmail.com) wrote: > As for what language it's written in- I don't think that matters much. > I'd very much expect it to be more performant to use binary if you're > working in C, of course, but there's no point

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Stephen Frost
Greetings, * Merlin Moncure (mmonc...@gmail.com) wrote: > On Fri, Nov 2, 2018 at 11:15 AM Stephen Frost wrote: > > * Merlin Moncure (mmonc...@gmail.com) wrote: > > > I'll still stand other point I made though; I'd > > > really want to see some benchmarks demonstrating benefit over > > >

Re: partitioned indexes and tablespaces

2018-11-02 Thread Alvaro Herrera
On 2018-Nov-02, Robert Haas wrote: > On Fri, Nov 2, 2018 at 12:05 PM Alvaro Herrera > wrote: > > On 2018-Nov-02, Robert Haas wrote: > > > I strongly object to inserting behavior changes into released branches > > > on the grounds that the behavior wasn't considered carefully enough > > > before

Re: partitioned indexes and tablespaces

2018-11-02 Thread Robert Haas
On Fri, Nov 2, 2018 at 12:05 PM Alvaro Herrera wrote: > On 2018-Nov-02, Robert Haas wrote: > > I strongly object to inserting behavior changes into released branches > > on the grounds that the behavior wasn't considered carefully enough > > before feature freeze. > > I'm not proposing to change

Re: Constraint documentation

2018-11-02 Thread Patrick Francelle
Thanks for your remarks and advices, and of course for your help to rewrite the text. So, it is now included in the new version attached. I hope it will be ok this time. Patrick Francelle On 10/30/18 17:14, David G. Johnston wrote: > The product name, when used in the documentation, is

Re: Compressed TOAST Slicing

2018-11-02 Thread Paul Ramsey
As threatened, I have also added a patch to left() to also use sliced access. compressed-datum-slicing-20190102a.patch Description: Binary data compressed-datum-slicing-left-20190102a.patch Description: Binary data

Re: Compressed TOAST Slicing

2018-11-02 Thread Paul Ramsey
On Thu, Nov 1, 2018 at 4:02 PM Tom Lane wrote: > Paul Ramsey writes: > > On Thu, Nov 1, 2018 at 2:29 PM Stephen Frost wrote: > >> and secondly, why we wouldn't consider > >> handling a non-zero offset. A non-zero offset would, of course, still > >> require decompressing from the start and

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Merlin Moncure
On Fri, Nov 2, 2018 at 11:15 AM Stephen Frost wrote: > > Greetings, > > * Merlin Moncure (mmonc...@gmail.com) wrote: > > On Fri, Nov 2, 2018 at 10:53 AM Tom Lane wrote: > > > Andres' point about alignment is a pretty good one as well, if it applies > > > here --- I don't recall just what

Re: COPY FROM WHEN condition

2018-11-02 Thread David Fetter
On Fri, Nov 02, 2018 at 12:58:12PM +0100, Daniel Verite wrote: > Pavel Stehule wrote: > > > > SELECT x.a, sum(x.b) > > > FROM ( COPY INLINE '/path/to/foo.txt' FORMAT CSV ) as x( a integer, b > > > numeric, c text, d date, e json) ) > > > WHERE x.d >= '2018-11-01' > > > > > > > > Without

Re: WIP: Avoid creation of the free space map for small tables

2018-11-02 Thread John Naylor
On 11/2/18, Robert Haas wrote: > On Fri, Nov 2, 2018 at 10:07 AM Tom Lane wrote: >> Robert Haas writes: >> > That's not what I'm saying. If we don't have the FSM, we have to >> > check every page of the table. If there's a workload where that >> > happens a lot on a table that is just under

Re: partitioned indexes and tablespaces

2018-11-02 Thread Alvaro Herrera
On 2018-Nov-02, Robert Haas wrote: > On Fri, Nov 2, 2018 at 11:02 AM Alvaro Herrera > wrote: > > > By the way, if we decide to do something about this, I think we do the > > > same for partitioned tables. > > > > I'm up for changing the behavior of partitioned tables in pg12 (please > > send a

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Stephen Frost
Greetings, * Merlin Moncure (mmonc...@gmail.com) wrote: > On Fri, Nov 2, 2018 at 10:53 AM Tom Lane wrote: > > Andres' point about alignment is a pretty good one as well, if it applies > > here --- I don't recall just what internal alignment requirements jsonb > > has. We have not historically

Re: PG vs macOS Mojave

2018-11-02 Thread Tom Lane
Jakob Egger writes: > Then I upgraded the "Comand Line Tools for Xcode 10.1" using Software Update, > and suddenly the "-isysroot" flag disappeared (for both Xcode 10 and Xcode > 10.1). Interesting. I'd not tested the intermediate state without the CLT update. > In any case, it looks like we

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Merlin Moncure
On Fri, Nov 2, 2018 at 10:53 AM Tom Lane wrote: > Merlin Moncure writes: > > On Wed, Oct 31, 2018 at 10:23 AM Andres Freund wrote: > >> It's entirely unacceptable afaict. Besides the whole "exposing > >> internals" issue, it's also at least not endianess safe, depends on the > >> local

Re: partitioned indexes and tablespaces

2018-11-02 Thread Alvaro Herrera
On 2018-Nov-02, Robert Haas wrote: > I strongly object to inserting behavior changes into released branches > on the grounds that the behavior wasn't considered carefully enough > before feature freeze. I'm not proposing to change any stable behavior. The thing I'm proposing to change clearly

Re: partitioned indexes and tablespaces

2018-11-02 Thread Robert Haas
On Fri, Nov 2, 2018 at 11:02 AM Alvaro Herrera wrote: > > By the way, if we decide to do something about this, I think we do the > > same for partitioned tables. > > I'm up for changing the behavior of partitioned tables in pg12 (please > send a patch), but I'm up for changing the behavior of

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Tom Lane
Merlin Moncure writes: > On Wed, Oct 31, 2018 at 10:23 AM Andres Freund wrote: >> It's entirely unacceptable afaict. Besides the whole "exposing >> internals" issue, it's also at least not endianess safe, depends on the >> local alignment requirements (which differ both between platforms and >>

Re: PG vs macOS Mojave

2018-11-02 Thread Jakob Egger
> Am 1. Nov. 2018 um 04:17 schrieb Tom Lane : > > You can verify the thing's built-in settings with something like > > $ echo >test.c > $ clang -v -E test.c > > and on Xcode 10.0 I get > > "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" > -cc1

Re: Doubts about pushing LIMIT to MergeAppendPath

2018-11-02 Thread Antonin Houska
Tomas Vondra wrote: > On 11/02/2018 08:16 AM, Antonin Houska wrote: > > Tomas Vondra wrote: > > > >> OK, so the reason is that when building child paths, we don't keep > >> the pathkeys unless it matches the "interesting" pathkeys. > >> > >> So for example we may have an IndexPath, but with

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Merlin Moncure
On Wed, Oct 31, 2018 at 10:23 AM Andres Freund wrote: > > Hi, > > On 2018-10-31 11:13:13 -0400, Andrew Dunstan wrote: > > I agree that just sending a blob of the internal format isn't a great idea. > > It's entirely unacceptable afaict. Besides the whole "exposing > internals" issue, it's also at

Re: PG vs macOS Mojave

2018-11-02 Thread Tom Lane
Daniel Gustafsson writes: >> On 2 Nov 2018, at 15:42, Tom Lane wrote: >> Anybody have an opinion about which approach to use? We need to choose >> one, and PDQ too, if we want full buildfarm coverage on it before Monday's >> wrap. > #1 is the option that appeals to me the most, mostly because

Re: partitioned indexes and tablespaces

2018-11-02 Thread Alvaro Herrera
On 2018-Nov-02, Amit Langote wrote: > On 2018/11/02 10:27, Michael Paquier wrote: > > It seems to me that the current behavior is wanted in this case, because > > partitioned tables and partitioned indexes have no physical storage. > > Keith Fiske complained about this behavior for partitioned

Re: PG vs macOS Mojave

2018-11-02 Thread Daniel Gustafsson
> On 2 Nov 2018, at 15:42, Tom Lane wrote: > > I wrote: >> Here's a lightly-tested patch for that approach. > > Anybody have an opinion about which approach to use? We need to choose > one, and PDQ too, if we want full buildfarm coverage on it before Monday's > wrap. Sorry for being slow to

Re: partitioned indexes and tablespaces

2018-11-02 Thread Alvaro Herrera
On 2018-Nov-02, Michael Paquier wrote: > On Thu, Nov 01, 2018 at 09:31:38PM -0300, Alvaro Herrera wrote: > > 1. When a CREATE INDEX specifies a tablespace, existing partitions get > > the index in the correct tablespace; however, the parent index itself > > does not record the tablespace. So

Re: replication_slots usability issue

2018-11-02 Thread Petr Jelinek
On 01/11/2018 18:54, Andres Freund wrote:> >> Also, from 691d79a which you just committed: >> + ereport(FATAL, >> + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), >> +errmsg("logical replication slots \"%s\" exists, but >> wal_level < logical", >> +

Re: Use durable_unlink for .ready and .done files for WAL segment removal

2018-11-02 Thread Nathan Bossart
One argument for instead checking WAL file existence before calling archive_command might be to avoid the increased startup time. Granted, any added delay from this patch is unlikely to be noticeable unless your archiver is way behind and archive_status has a huge number of files. However, I have

Re: PG vs macOS Mojave

2018-11-02 Thread Tom Lane
I wrote: > Here's a lightly-tested patch for that approach. Anybody have an opinion about which approach to use? We need to choose one, and PDQ too, if we want full buildfarm coverage on it before Monday's wrap. The main argument in favor of #1 (restore use of -isysroot) is fear that Apple's

Re: WIP: Avoid creation of the free space map for small tables

2018-11-02 Thread Robert Haas
On Fri, Nov 2, 2018 at 10:07 AM Tom Lane wrote: > Robert Haas writes: > > That's not what I'm saying. If we don't have the FSM, we have to > > check every page of the table. If there's a workload where that > > happens a lot on a table that is just under the size threshold for > > creating the

Re: Vacuum Full does not release the disk size space after delete from table

2018-11-02 Thread Tom Lane
Haozhou Wang writes: > We meet a corner case that related to the behavior of Vacuum Full. > ... > If we run both sql scripts on same database in parallel, the "VACUUM FULL > a;" will not release the disk space. I think what's happening is that the delete in script 1 happens after the "pg_sleep"

Re: Hash Joins vs. Bloom Filters / take 2

2018-11-02 Thread Robert Haas
On Thu, Nov 1, 2018 at 5:07 PM Thomas Munro wrote: > Would you compute the hash for the outer tuples in the scan, and then > again in the Hash Join when probing, or would you want to (somehow) > attach the hash to emitted tuples for later reuse by the higher node? I'm interested in what Jim has

Re: COPY FROM WHEN condition

2018-11-02 Thread David Fetter
On Thu, Nov 01, 2018 at 10:57:25PM -0400, Corey Huinker wrote: > > > > > Are you thinking something like having a COPY command that provides > > > results in such a way that they could be referenced in a FROM clause > > > (perhaps a COPY that defines a cursor…)? > > > > That would also be nice,

Re: WIP: Avoid creation of the free space map for small tables

2018-11-02 Thread Tom Lane
Robert Haas writes: > That's not what I'm saying. If we don't have the FSM, we have to > check every page of the table. If there's a workload where that > happens a lot on a table that is just under the size threshold for > creating the FSM, then it's likely to be a worst case for this patch.

Re: WIP: Avoid creation of the free space map for small tables

2018-11-02 Thread Robert Haas
On Fri, Nov 2, 2018 at 7:23 AM Amit Kapila wrote: > > That doesn't seem like an unreasonable argument. I'm not sure whether > > the right threshold is 4 or something a little bigger, but I bet it's > > not very large. It seems important to me that before anybody thinks > > about committing

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Tomas Vondra
On 11/02/2018 01:42 PM, Daniel Gustafsson wrote: >> On 2 Nov 2018, at 04:21, Tom Lane wrote: > >> (In short, I remain unconvinced that we'd not be better off spending >> our effort on making json_out faster...) > > Shooting wildly from the hip, isn't this a case where we can > potentially

Re: COPY FROM WHEN condition

2018-11-02 Thread Tomas Vondra
On 11/02/2018 03:57 AM, Corey Huinker wrote: > > Are you thinking something like having a COPY command that provides > > results in such a way that they could be referenced in a FROM clause > > (perhaps a COPY that defines a cursor…)? > > That would also be nice, but what I was

Re: zheap: a new storage format for PostgreSQL

2018-11-02 Thread Tomas Vondra
On 11/02/2018 12:12 PM, Amit Kapila wrote: > On Thu, Nov 1, 2018 at 7:26 PM Tomas Vondra > wrote: >> >> On 11/01/2018 07:43 AM, Amit Kapila wrote: >>> >>> You can find the latest code at https://github.com/EnterpriseDB/zheap >>> >> >> Seems valgrind complains about a couple of places in the

[PATCH] Improvements to "Getting started" tutorial for Google Code-in task

2018-11-02 Thread LAM JUN RONG
Hi, I’m a student taking part in Google Code-in 2018. The task I am currently working on, https://codein.withgoogle.com/dashboard/task-instances/6406170207059968/, requires that I review and improve the “Getting Started” tutorial in the PostgreSQL docs, and submit

Re: pread() and pwrite()

2018-11-02 Thread Jesper Pedersen
Hi Thomas, On 10/9/18 4:56 PM, Thomas Munro wrote: Thanks, much nicer. Rebased. This still applies, and passes make check-world. I wonder what the commit policy is on this, if the Windows part isn't included. I read Heikki's comment [1] as it would be ok to commit benefiting all

Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

2018-11-02 Thread Etsuro Fujita
(2018/10/29 15:58), Kyotaro HORIGUCHI wrote: At Tue, 23 Oct 2018 13:21:31 +0100, Tom Lane wrote in<18397.1540297...@sss.pgh.pa.us> After a bit of thought, the problem here is blindingly obvious: we generally run the backend with SIGPIPE handing set to SIG_IGN, and evidently popen() allows the

Re: Fix various typos around the tree

2018-11-02 Thread Magnus Hagander
On Fri, Nov 2, 2018 at 1:37 PM Daniel Gustafsson wrote: > Attached patch fixes a set of typos found here and there around the tree. > Some > are in the documentation which may be of interest for backporting, but I’ve > included all in the same patch. > Pushed, with back-patching of the docs

Re: Doubts about pushing LIMIT to MergeAppendPath

2018-11-02 Thread Tomas Vondra
On 11/02/2018 08:16 AM, Antonin Houska wrote: > Tomas Vondra wrote: > >> OK, so the reason is that when building child paths, we don't keep >> the pathkeys unless it matches the "interesting" pathkeys. >> >> So for example we may have an IndexPath, but with pathkeys=NIL if >> the index does not

Logical decoding on a standby node - any progress?

2018-11-02 Thread Josef Machytka
Hello! On stackoverflow I got a hint to ask on this mailing list... I read this question on stackoverflow: Logical decoding on a standby node and checked PostgreSQL documentation and PG mail lists but I did not

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Daniel Gustafsson
> On 2 Nov 2018, at 04:21, Tom Lane wrote: > (In short, I remain unconvinced that we'd not be better off spending > our effort on making json_out faster...) Shooting wildly from the hip, isn't this a case where we can potentially utilize the JIT infrastructure to speed it up? cheers ./daniel

Fix various typos around the tree

2018-11-02 Thread Daniel Gustafsson
Attached patch fixes a set of typos found here and there around the tree. Some are in the documentation which may be of interest for backporting, but I’ve included all in the same patch. cheers ./daniel typos-various.patch Description: Binary data

Re: csv format for psql

2018-11-02 Thread Daniel Verite
Michael Paquier wrote: > If you can produce a new version, please feel free to post it. Here's a rebased version with a couple regression tests added per the discussions during the previous CF. Now at https://commitfest.postgresql.org/20/1861/ Best regards, -- Daniel Vérité

Re: COPY FROM WHEN condition

2018-11-02 Thread Daniel Verite
Pavel Stehule wrote: > > SELECT x.a, sum(x.b) > > FROM ( COPY INLINE '/path/to/foo.txt' FORMAT CSV ) as x( a integer, b > > numeric, c text, d date, e json) ) > > WHERE x.d >= '2018-11-01' > > > > > Without some special feature this example is not extra useful. It is based > on copy on

Re: COPY FROM WHEN condition

2018-11-02 Thread Surafel Temesgen
hi, On Wed, Oct 31, 2018 at 10:54 AM Masahiko Sawada wrote: > On Tue, Oct 30, 2018 at 11:47 PM Surafel Temesgen > wrote: > > I've looked at this patch and tested. > > When I use a function returning string in WHEN clause I got the following > error: > > =# copy test from '/tmp/aaa.csv' (format

Re: WIP: Avoid creation of the free space map for small tables

2018-11-02 Thread Amit Kapila
On Wed, Oct 31, 2018 at 10:29 PM Robert Haas wrote: > > On Tue, Oct 23, 2018 at 9:42 AM John Naylor wrote: > > A case could be made for setting the threshold to 4, since not having > > 3 blocks of FSM in shared buffers exactly makes up for the 3 other > > blocks of heap that are checked when

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-11-02 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Christian Ohler writes: > > On Wed, Oct 31, 2018 at 7:22 AM Tom Lane wrote: > >> If we're going to expose the > >> internal format, let's just change the definition of the type's binary > >> I/O format, thereby getting a win for purposes like

Re: zheap: a new storage format for PostgreSQL

2018-11-02 Thread Amit Kapila
On Thu, Nov 1, 2018 at 7:26 PM Tomas Vondra wrote: > > On 11/01/2018 07:43 AM, Amit Kapila wrote: > > > > You can find the latest code at https://github.com/EnterpriseDB/zheap > > > > Seems valgrind complains about a couple of places in the code - nothing > major, might be noise, but probably

Re: [PATCH] Change simple_heap_insert() to a macro

2018-11-02 Thread Peter Eisentraut
On 12/10/2018 12:09, Andrey Klychkov wrote: > I don't see any reasons against to use the proposed macro instead of > this function. Macros are weird and should be avoided if possible. If we were to do this, it should be an inline function, I think. But I think it's not useful here. I think

Re: INSTALL file

2018-11-02 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Thu, Nov 01, 2018 at 08:42:34PM -0400, Stephen Frost wrote: > > If we go down this route, the master branch should probably link to the > > regularly built devel documentation, so that if/when we do make such a > > change, we'll point

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-11-02 Thread Andrey Lepikhov
I do the code review. Now, it is first patch - v6-0001... dedicated to a logical duplicates ordering. Documentation is full and clear. All non-trivial logic is commented accurately. Patch applies cleanly on top of current master. Regression tests passed and my "Retail Indextuple deletion"

Re: Resetting PGPROC atomics in ProcessInit()

2018-11-02 Thread Amit Kapila
On Sat, Oct 27, 2018 at 9:42 PM Amit Kapila wrote: > > On Sat, Oct 27, 2018 at 8:52 PM Andres Freund wrote: > > On October 27, 2018 3:36:45 PM GMT+01:00, Amit Kapila > > wrote: > > >On Sat, Oct 27, 2018 at 4:11 PM Andres Freund > > >wrote: > > >> > > >> Hi, > > >> > > >> I just noticed, while

Re: [HACKERS] Block level parallel vacuum

2018-11-02 Thread Masahiko Sawada
Hi, On Thu, Nov 1, 2018 at 2:28 PM Yura Sokolov wrote: > > Excuse me for being noisy. > > Increasing vacuum's ring buffer improves vacuum upto 6 times. > https://www.postgresql.org/message-id/flat/20170720190405.GM1769%40tamriel.snowman.net > This is one-line change. > > How much improvement

pg_ls_dir_files issue with concurrent file unlink

2018-11-02 Thread Sergei Kornilov
Hello We have some functions for directory listing, e.g. pg_ls_dir, pg_ls_logdir, pg_ls_waldir. All of them call pg_ls_dir_files function (src/backend/utils/adt/genfile.c) and i have question about per file error in this function. > /* Get the file info */ >

Re: CF app feature request

2018-11-02 Thread Fabien COELHO
Bonjour Michaël, Because the same patch submission is already counted? It is a rare occurence, so just a "Withdrawn" state could be enough, and slightly false CF stats are no big deal. Or as we are dealing with duplicated entries, perhaps we could just delete the entry not wanted, which

Vacuum Full does not release the disk size space after delete from table

2018-11-02 Thread Haozhou Wang
Hi hackers, We meet a corner case that related to the behavior of Vacuum Full. There are two SQL scripts SQL1: - -- Test vacuum full create schema s2; set search_path to s2; create table a (i int); create table b (i int); insert

Re: bugfix: BUG #15477: Procedure call with named inout refcursor parameter - "invalid input syntax for type boolean"

2018-11-02 Thread Pavel Stehule
pá 2. 11. 2018 v 9:02 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 02/11/2018 06:04, Pavel Stehule wrote: > > čt 1. 11. 2018 v 14:31 odesílatel Pavel Stehule > > napsal: > > > > Cleaned patch with regress tests > > > > > > minor

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

2018-11-02 Thread Amit Langote
On 2018/11/01 20:34, Dilip Kumar wrote: > On Mon, Oct 29, 2018 at 2:53 PM Amit Langote wrote: >> Anyway, why don't we just use the child table's AppendRelInfo to get the >> parent's version of varattno instead of creating a new function? It can >> be done as shown in the attached revised version

Re: bugfix: BUG #15477: Procedure call with named inout refcursor parameter - "invalid input syntax for type boolean"

2018-11-02 Thread Peter Eisentraut
On 02/11/2018 06:04, Pavel Stehule wrote: > čt 1. 11. 2018 v 14:31 odesílatel Pavel Stehule > napsal: > > Cleaned patch with regress tests > > > minor cleaning Could you explain your analysis of the problem and how this patch proposes to fix it? About the

Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-11-02 Thread Amit Langote
Hi, On 2018/11/02 14:27, Michael Paquier wrote: > On Fri, Nov 02, 2018 at 02:18:04PM +0900, Michael Paquier wrote: >> This case is funky. Interesting indeed. >> The parent gets dropped at commit time, but it does >> not know that it should drop the child as well per their dependencies. >> This

Re: CF app feature request

2018-11-02 Thread Michael Paquier
On Fri, Nov 02, 2018 at 08:17:51AM +0100, Fabien COELHO wrote: > Because the same patch submission is already counted? It is a rare > occurence, so just a "Withdrawn" state could be enough, and slightly false > CF stats are no big deal. Or as we are dealing with duplicated entries, perhaps we

Re: wal_dump output on CREATE DATABASE

2018-11-02 Thread Peter Eisentraut
On 26/10/2018 15:53, Jean-Christophe Arnu wrote: > Exemple on CREATE DATABASE (without defining a template database) : > rmgr: Database    len (rec/tot): 42/    42, tx:    568, lsn: > 0/01865790, prev 0/01865720, desc: CREATE copy dir 1/1663 to 16384/1663 > > It comes out (to me) it may

Re: pgbench doc fix

2018-11-02 Thread Fabien COELHO
Robert, Yes, you need to send params (thus send bind message) anyway. Regarding re-parsing, maybe you mixed up parse-analythis with planning? Re-parse-analythis can only be avoided if you can reuse named (or unnamed) parepared statements. So given this, I'm struggling to see anything wrong

Re: Small run-time pruning doc fix

2018-11-02 Thread Peter Eisentraut
On 08/10/2018 10:22, David Rowley wrote: > Before 5220bb7533f a note in ddl.sgml used to mention that run-time > pruning was only implemented for Append. When we got MergeAppend > support the commit updated this to mention MergeAppend is supported > too. This is slightly weird as it's not all that

Re: CF app feature request

2018-11-02 Thread Fabien COELHO
I don't know enough about the app internals to comment. But it probably shouldn't appear in the stats, or else should have its own category in the stats. Or that's closer to "Rejected by the author himself"? "Withdrawn" sounds like a good term for that, we surely don't want to simply remove

Re: Doubts about pushing LIMIT to MergeAppendPath

2018-11-02 Thread Antonin Houska
Tomas Vondra wrote: > OK, so the reason is that when building child paths, we don't keep the > pathkeys unless it matches the "interesting" pathkeys. > > So for example we may have an IndexPath, but with pathkeys=NIL if the > index does not match the ORDER BY we need. I don't agree that

Re: row filtering for logical replication

2018-11-02 Thread Erik Rijkers
On 2018-11-02 02:59, Euler Taveira wrote: Em qui, 1 de nov de 2018 às 05:30, Erik Rijkers escreveu: > I ran pgbench-over-logical-replication with a WHERE-clause and could > not get this to do a correct replication. Below is the output of the > attached test program. > > > $

  1   2   >