Re: [HACKERS] Partitioned tables and relfilenode

2017-02-23 Thread Amit Langote
On 2017/02/23 16:48, Ashutosh Bapat wrote: > On Thu, Feb 23, 2017 at 1:08 PM, Amit Langote wrote: >> @@ -996,10 +996,20 @@ inheritance_planner(PlannerInfo *root) >> /* >> +* Partitioned tables do not have storage for themselves and should >> not be >> +* scanned. >> >> @@ -

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-23 Thread Ashutosh Bapat
On Thu, Feb 23, 2017 at 1:38 PM, Amit Langote wrote: > On 2017/02/23 16:48, Ashutosh Bapat wrote: >> On Thu, Feb 23, 2017 at 1:08 PM, Amit Langote wrote: >>> @@ -996,10 +996,20 @@ inheritance_planner(PlannerInfo *root) >>> /* >>> +* Partitioned tables do not have storage for themse

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-23 Thread Bernd Helmle
Am Mittwoch, den 22.02.2017, 22:17 -0500 schrieb Tom Lane: > [ shrug... ]  Well, I won't resist this hard as long as it's done > competently, which to me means "the subquery name doesn't conflict > with > anything else".  Not "it doesn't conflict unless you're unlucky > enough > to have used the sa

[HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-02-23 Thread Jan Michálek
Have a nice day I`m using \pset format html for generating tables for some documentation, training, reports, blogs... Mainly sample of data for SQL queries, or explaining structure etc. But in some software like sphinx, wiki on redmine, or markdown on github issues I`m not able to insert table in h

Re: [HACKERS] UPDATE of partition key

2017-02-23 Thread Amit Langote
Hi Amit, Thanks for working on this. On 2017/02/13 21:01, Amit Khandekar wrote: > Currently, an update of a partition key of a partition is not allowed, > since it requires to move the row(s) into the applicable partition. > > Attached is a WIP patch (update-partition-key.patch) that removes thi

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-23 Thread Pantelis Theodosiou
On Thu, Feb 23, 2017 at 3:17 AM, Tom Lane wrote: > Robert Haas writes: > > On Wed, Feb 22, 2017 at 10:33 PM, Nico Williams > wrote: > >> I suspect most users, like me, just roll their eyes, grumble, and put up > >> with it rather than complain. It's a pain point, but tolerable enough > >> that

[HACKERS] proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-02-23 Thread Pavel Stehule
Hi Currently is not possible to control sort columns for \d* commands. Usually schema and table name is used. Really often task is collect the most big objects in database. "\dt+, \di+" shows necessary information, but not in practical order. Instead introduction some additional flags to backslas

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-23 Thread Masahiko Sawada
On Wed, Feb 22, 2017 at 12:01 AM, Amit Kapila wrote: > On Tue, Feb 21, 2017 at 1:09 AM, Simon Riggs wrote: >> On 20 February 2017 at 10:27, Amit Kapila wrote: >>> On Mon, Feb 20, 2017 at 3:01 PM, Simon Riggs wrote: On 20 February 2017 at 09:15, Amit Kapila wrote: > On Mon, Feb 20, 201

Re: [HACKERS] asynchronous execution

2017-02-23 Thread Kyotaro HORIGUCHI
Hello, I totally reorganized the patch set to four pathces on the current master (9e43e87). At Wed, 22 Feb 2017 17:39:45 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170222.173945.262776579.horiguchi.kyot...@lab.ntt.co.jp> > Finally, I couldn't see the crash for the (maybe) same c

[HACKERS] A typo in mcxt.c

2017-02-23 Thread Kyotaro HORIGUCHI
Hello, while I investigated a bug of async-exec patch. I found a (maybe) typo in mcxt.c. | * It's not entirely clear whether 'tis better to do this before or after | * delinking the context; but an error in a callback will likely result in I'm not so confident, but the "'tis" seems to me to be

Re: [HACKERS] A typo in mcxt.c

2017-02-23 Thread Thomas Munro
On Thu, Feb 23, 2017 at 5:40 PM, Kyotaro HORIGUCHI wrote: > Hello, while I investigated a bug of async-exec patch. I found a > (maybe) typo in mcxt.c. > > | * It's not entirely clear whether 'tis better to do this before or after > | * delinking the context; but an error in a callback will likel

Re: [HACKERS] Declarative partitioning - another take

2017-02-23 Thread Yugo Nagata
Hi Amit, On Thu, 23 Feb 2017 16:29:32 +0900 Amit Langote wrote: > Thanks for taking care of that. > > + * PartitionRoot relation descriptor for parent > relation > > Maybe: relation descriptor for root parent relation This seems better. Patch is updated. Thanks,

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-23 Thread Greg Stark
On 22 February 2017 at 15:08, Tom Lane wrote: > Indeed. When I wrote the comment you're referring to, quite a few years > ago now, I thought that popular demand might force us to allow omitted > aliases. But the demand never materialized. At this point it seems > clear to me that there isn't re

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-23 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 20, 2017 at 3:06 PM, Joel Jacobson wrote: >> On Mon, Feb 20, 2017 at 1:45 AM, Tom Lane wrote: >>> The versions of autocommit that have actually stood the test of time were >>> implemented on the client side (in psql and JDBC, and I think ODBC as >>> well), where

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-23 Thread Greg Stark
On 23 February 2017 at 13:27, Greg Stark wrote: > => SELECT "?column"? FROM (select 1+1 as "?column?", 1+1) AS x; Oops, I missed the typo there: => SELECT "?column?" FROM (select 1+1 AS "?column?", 1+1) AS x; ERROR: 42702: column reference "?column?" is ambiguous LINE 2: SELECT "?column?" FROM

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-02-23 Thread Michael Paquier
On Thu, Feb 23, 2017 at 7:29 PM, Jan Michálek wrote: > Is there possibility, that support for some others formats will be added in > the future? Or is it possible to create some of this formats by manipulating > with pset? Addition of new formats is definitely possible, the latest one added was a

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-23 Thread Tom Lane
Greg Stark writes: > On 22 February 2017 at 15:08, Tom Lane wrote: >> Indeed. When I wrote the comment you're referring to, quite a few years >> ago now, I thought that popular demand might force us to allow omitted >> aliases. But the demand never materialized. At this point it seems >> clear

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-23 Thread Stephen Frost
Micahel, * Michael Paquier (michael.paqu...@gmail.com) wrote: > Is there any interest for a feature like that? I have a non-polished > patch at hand but I can work on that for the upcoming CF if there are > voices in favor of such a feature. The feature could be simply > activated with a dedicated

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-23 Thread Tom Lane
Bernd Helmle writes: > Right, i'm going to give it a try then. Currently i see these options: > * Validate any generated alias against a list of explicit alias names. > This means we have to collect explicit alias names in, say a hashtable, > and validate a generated name against potential colli

[HACKERS] user mapping messages

2017-02-23 Thread Andrew Dunstan
While reviewing the IF NOT EXISTS patch for CREATE USER MAPPING I noticed that in several places we treat the user name as the name of the user mapping. Strictly ISTM that user mappings are really anonymous objects, so instead of something like user "mapping \"%s\" does not exist for the server" w

Re: [HACKERS] Keep ECPG comment for log_min_duration_statement

2017-02-23 Thread Dr. Michael Meskes
Hi,     > So, I'd like to modify ecpg command not to remove some specific > comments. > ... > [Interface] > add a new option "--enable-parse-comment" to ecpg command. >   >   ecpg --enable-parse-comment ,..., prog.pgc >   > This option enables ecpg command to pass on block comments (/* 〜 */) > to c

Re: [HACKERS] user mapping messages

2017-02-23 Thread Alvaro Herrera
Andrew Dunstan wrote: > > While reviewing the IF NOT EXISTS patch for CREATE USER MAPPING I > noticed that in several places we treat the user name as the name of the > user mapping. Strictly ISTM that user mappings are really anonymous > objects, so instead of something like user "mapping \"%s\"

Re: [HACKERS] FYI: git worktrees as replacement for "rsync the CVSROOT"

2017-02-23 Thread Bruce Momjian
On Sun, Jan 15, 2017 at 03:01:47PM -0600, Jim Nasby wrote: > Not sure how many people still use [1], as referenced by our git wiki[2], > but it appears git worktrees are a viable replacement for that technique. In > short, if you're already in your checkout: > > git worktree add ../9.6 REL9_6_STAB

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-23 Thread Corey Huinker
> > > Files "conditional.h" and "conditional.c" are missing from the patch. > > Also, is there a particular reason why tap test have been removed? That would be because I diffed against my last commit, not the master branch, sigh. v16 is everything v15 promised to be. diff --git a/doc/src/sgml/r

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-02-23 Thread Alvaro Herrera
Jan Michálek wrote: > Is there posibility, that support for some others formats will be added in > the future? Or id posible to create some of this formats by manipulating > with pset? Or I need use xml for export and use some xslt templates, or > lxml etc. to create format that I will be able to

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-02-23 Thread Dilip Kumar
On Thu, Feb 23, 2017 at 12:11 PM, Rafia Sabih wrote: > Yes, it can be simplified to > if (dest->mydest == DestIntoRel || (numberTuples && (dest->mydest != > DestSPI && dest->mydest ! DestSQLFunction))) > Thanks. Okay, this looks cleaner. Few more comments. 1.I don't see any check in the code whi

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-23 Thread Daniel Verite
Tom Lane wrote: > Ah, I see why *that* wants to know about it ... I think. I suppose you're > arguing that variable expansion shouldn't be able to insert, say, an \else > in a non-active branch? Maybe, but if it can insert an \else in an active > branch, then why not non-active too? See

Re: [HACKERS] pg_upgrade loses security lables and COMMENTs on blobs

2017-02-23 Thread Stephen Frost
All, * Stephen Frost (sfr...@snowman.net) wrote: > Just wanted to get a note out to -hackers about the issue, I'll see > about getting a fix written up for it soon. Attached is a patch which addresses this issue. I'm not terribly pleased with it, but I also haven't got any great ideas of what el

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-02-23 Thread Dilip Kumar
On Thu, Feb 23, 2017 at 8:58 PM, Dilip Kumar wrote: > Few more comments. > 1.I don't see any check in the code which will prevent the parallel > execution of the query inside a function if its called from a DML > statement. > e.g. If we use a function in the update statement's which has the > sele

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Tue, Jan 31, 2017 at 04:52:39PM +0530, Pavan Deolasee wrote: > The other critical bug I found, which unfortunately exists in the master too, > is the index corruption during CIC. The patch includes the same fix that I've > proposed on the other thread. With these changes, WARM stress is running

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-02-23 Thread Leknín Řepánek
I was mainly using tables in rst, i think, that this format is very similar to "grid tables" in markdown. And it is really similar to default borders used in psql with border 2. On Thu, Feb 23, 2017 at 12:06:36PM -0300, Alvaro Herrera wrote: > Jan Michálek wrote: > > > Is there posibility, that

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Simon Riggs
On 22 February 2017 at 19:57, Peter Geoghegan wrote: > FWIW, I agree that some of what has been claimed about what > contributors failed to do with this patch is exaggerated, and not in a > way that I'd understand as hyperbole that drives home a deeper point. What claims are you talking about? W

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-23 Thread Magnus Hagander
On Thu, Feb 23, 2017 at 7:36 AM, Michael Paquier wrote: > Hi all, > > When storing WAL segments on a dedicated partition with > pg_receivexlog, for some deployments, the removal of past WAL segments > depends on the frequency of base backups happening on the server. In > short, once a new base ba

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Simon Riggs
On 22 February 2017 at 02:14, Robert Haas wrote: > On Tue, Feb 21, 2017 at 2:03 AM, Bruce Momjian wrote: >> I have to admit my reaction was similar to Simon's, meaning that the >> lack of docs is a problem, and that the limitations are kind of a >> surprise, and I wonder what other surprises ther

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-23 Thread Nico Williams
On Thu, Feb 23, 2017 at 10:37:16AM +0100, Bernd Helmle wrote: > Am Mittwoch, den 22.02.2017, 22:17 -0500 schrieb Tom Lane: > > [ shrug... ]  Well, I won't resist this hard as long as it's done > > competently, which to me means "the subquery name doesn't conflict > > with > > anything else".  Not "

[HACKERS] Report the number of skipped frozen pages by manual VACUUM

2017-02-23 Thread Masahiko Sawada
Hi all, The autovacuum reports the number of skipped frozen pages to the VACUUM output. But these information is not appeared by manual VACUUM. This information is useful for the user to check efficiency of VACUUM. Attached patch add this information to VACUUM output. * Example =# VACUUM VERBOSE

Re: [HACKERS] Logical Replication and Character encoding

2017-02-23 Thread Peter Eisentraut
On 2/17/17 10:14, Peter Eisentraut wrote: > Well, it is sort of a libpq connection, and a proper libpq client should > set the client encoding, and a proper libpq server should do encoding > conversion accordingly. If we just play along with this, it all works > correctly. > > Other output plugin

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-02-23 Thread Simon Riggs
On 23 February 2017 at 06:40, Ashutosh Bapat wrote: > I think this is ready for committer. Thanks for writing and reviewing this. I'll be happy to review and commit. Please add to CF. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Tr

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-02-23 Thread Alvaro Herrera
Leknín Řepánek wrote: > I was mainly using tables in rst, i think, that this format is very > similar to "grid tables" in markdown. And it is really similar to default > borders used > in psql with border 2. Great. So are you writing the code now? :-) -- Álvaro Herrerahttps://

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Peter Geoghegan
On Thu, Feb 23, 2017 at 8:08 AM, Simon Riggs wrote: > What claims are you talking about? Which things have been exaggerated, > and by whom? * The specious argument that we should "just" have CREATE INDEX create equivalent indexes across partitions, to save all those people from writing all those

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Joshua D. Drake
On 02/23/2017 09:27 AM, Peter Geoghegan wrote: On Thu, Feb 23, 2017 at 8:08 AM, Simon Riggs wrote: * "Good work so far, but there is still a very significant amount of work to do." There is always more work to do, so why say so? I think that the implication is that this isn't complete as a f

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-23 Thread Fabien COELHO
Hello Daniel, Ah, I see why *that* wants to know about it ... I think. I suppose you're arguing that variable expansion shouldn't be able to insert, say, an \else in a non-active branch? Maybe, but if it can insert an \else in an active branch, then why not non-active too? Seems a bit incons

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Wed, Feb 1, 2017 at 10:46:45AM +0530, Pavan Deolasee wrote: > > contains a WARM tuple. Alternate ideas/suggestions and review of the > design > > are welcome! > > t_infomask2 contains one last unused bit, > > > Umm, WARM is using 2 unused bits from t_infomask2. You mean there

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Alvaro Herrera
Bruce Momjian wrote: > As I remember, WARM only allows > a single index-column change in the chain. Why are you seeing such a > large performance improvement? I would have thought it would be that > high if we allowed an unlimited number of index changes in the chain. The second update in a cha

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-02-23 Thread Jan Michálek
I can try it, doesn`t look dificult, but I`m worry, that I`m not able to write clean, pretty code. 2017-02-23 18:03 GMT+01:00 Alvaro Herrera : > Leknín Řepánek wrote: > > I was mainly using tables in rst, i think, that this format is very > > similar to "grid tables" in markdown. And it is really

Re: [HACKERS] FYI: git worktrees as replacement for "rsync the CVSROOT"

2017-02-23 Thread Dagfinn Ilmari Mannsåker
Bruce Momjian writes: > On Sun, Jan 15, 2017 at 03:01:47PM -0600, Jim Nasby wrote: >> Not sure how many people still use [1], as referenced by our git wiki[2], >> but it appears git worktrees are a viable replacement for that technique. In >> short, if you're already in your checkout: >> >> git

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-23 Thread Andrew Dunstan
On 11/05/2016 03:11 PM, Andrew Dunstan wrote: > > > On 11/05/2016 01:13 PM, Tom Lane wrote: >> Andrew Dunstan writes: >>> On 11/05/2016 11:46 AM, Tom Lane wrote: That may be a good fix for robustness purposes, but it seems pretty horrid from an efficiency standpoint. Where is thi

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 03:03:39PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > As I remember, WARM only allows > > a single index-column change in the chain. Why are you seeing such a > > large performance improvement? I would have thought it would be that > > high if we allowed an

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Alvaro Herrera
Bruce Momjian wrote: > On Thu, Feb 23, 2017 at 03:03:39PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > As I remember, WARM only allows > > > a single index-column change in the chain. Why are you seeing such a > > > large performance improvement? I would have thought it would

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 03:26:09PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 03:03:39PM -0300, Alvaro Herrera wrote: > > > Bruce Momjian wrote: > > > > > > > As I remember, WARM only allows > > > > a single index-column change in the chain. Why are you seeing

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Alvaro Herrera
Bruce Momjian wrote: > Well, let's walk through this. Let's suppose you have three updates > that stay on the same page and don't update any indexed columns --- that > would produce a HOT chain of four tuples. If you then do an update that > changes an indexed column, prior to this patch, you ge

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 03:45:24PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Well, let's walk through this. Let's suppose you have three updates > > that stay on the same page and don't update any indexed columns --- that > > would produce a HOT chain of four tuples. If you then

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Alvaro Herrera
Bruce Momjian wrote: > On Thu, Feb 23, 2017 at 03:45:24PM -0300, Alvaro Herrera wrote: > > > and potentially trim the first HOT chain as those tuples become > > > invisible. > > > > That can already happen even without WARM, no? > > Uh, the point is that with WARM those four early tuples can be

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 03:58:59PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 03:45:24PM -0300, Alvaro Herrera wrote: > > > > > and potentially trim the first HOT chain as those tuples become > > > > invisible. > > > > > > That can already happen even without

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Simon Riggs
On 23 February 2017 at 17:27, Peter Geoghegan wrote: > On Thu, Feb 23, 2017 at 8:08 AM, Simon Riggs wrote: >> What claims are you talking about? Which things have been exaggerated, >> and by whom? > > * The specious argument that we should "just" have CREATE INDEX create > equivalent indexes acro

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Peter Geoghegan
On Thu, Feb 23, 2017 at 11:13 AM, Simon Riggs wrote: > My argument was that CREATE INDEX is expected to just work on tables > at present, so should also just work on partitioned tables. Without > that, the reality is people will need to write scripts. Really? What about postgres_fdw? Even if tha

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-02-23 Thread Pavel Stehule
2017-02-23 19:09 GMT+01:00 Jan Michálek : > I can try it, doesn`t look dificult, but I`m worry, that I`m not able to > write clean, pretty code. > It is not a problem, there is lot of iteration for code cleaning Regards Pavel > > 2017-02-23 18:03 GMT+01:00 Alvaro Herrera : > >> Leknín Řepánek

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-23 Thread Pavel Stehule
2017-02-23 18:52 GMT+01:00 Fabien COELHO : > > Hello Daniel, > > Ah, I see why *that* wants to know about it ... I think. I suppose you're >>> arguing that variable expansion shouldn't be able to insert, say, an >>> \else >>> in a non-active branch? Maybe, but if it can insert an \else in an >>>

[HACKERS] bytea_output output of base64

2017-02-23 Thread Bruce Momjian
Currently bytea_output supports values of 'hex' (the default) and 'escape' (octal). 'hex' uses two characters per byte, while escape uses three (ignoring the prefix overhead of \x or \[0-9].) It is my understanding that base64 uses 1.37 characters per byte: https://en.wikipedia.org/wiki/

Re: [HACKERS] Measuring replay lag

2017-02-23 Thread Simon Riggs
On 21 February 2017 at 21:38, Thomas Munro wrote: > On Tue, Feb 21, 2017 at 6:21 PM, Simon Riggs wrote: >> And happier again, leading me to move to the next stage of review, >> focusing on the behaviour emerging from the design. >> >> So my current understanding is that this doesn't rely upon LSN

Re: [HACKERS] A typo in mcxt.c

2017-02-23 Thread Jim Nasby
On 2/23/17 6:38 AM, Thomas Munro wrote: I'm not so confident, but the "'tis" seems to me to be a typo of "it's". That is an archaic way of contracting the same words differently: Given the number of non-native English speakers we have, it's probably worth changing it... -- Jim Nasby, Data Ar

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-23 Thread Jim Nasby
On 2/23/17 10:10 AM, Magnus Hagander wrote: Wouldn't this one, along with some other scenarios, be better provided by the "run command at end of segment" function that we've talked about before? And then that external command could implement whatever aging logic would be appropriate for the envir

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

2017-02-23 Thread Jim Nasby
On 2/19/17 5:27 AM, Robert Haas wrote: (1) a multi-batch hash join, (2) a nested loop, and (3) a merge join. (2) is easy to implement but will generate a ton of random I/O if the table is not resident in RAM. (3) is most suitable for very large tables but takes more work to code, and is also li

Re: [HACKERS] Add checklist item for psql completion to commitfest review

2017-02-23 Thread Jim Nasby
On 2/15/17 9:49 AM, Robert Haas wrote: On Tue, Feb 14, 2017 at 1:13 PM, Jim Nasby wrote: After seeing Yet Another Missing Psql Tab Completion it occurred to me... why not add a checklist item to the commitfest review page? I realize most regular contributors don't use the form, but a fair numbe

Re: [HACKERS] bytea_output output of base64

2017-02-23 Thread Tom Lane
Bruce Momjian writes: > Is there a reason we don't support base64 as a bytea_output output > option, except that no one has implemented it? How about "we already have one too many bytea output formats"? I don't think forcing code to try to support still another one is a great thing ... especially

Re: [HACKERS] Idea on how to simplify comparing two sets

2017-02-23 Thread Jim Nasby
On 2/7/17 12:03 PM, Tom Lane wrote: That said I'm not sure how much we want to go down this road on our own. It'd be nice to have when its needed but its not something that gets much visibility on these lists to suggest a large pent-up demand. Yeah, if this isn't in the standard and not in other

Re: [HACKERS] PinBuffer() no longer makes use of strategy

2017-02-23 Thread Jim Nasby
On 2/4/17 1:47 PM, Alexander Korotkov wrote: I'm tempted to put the old logic back, but more because this likely was unintentional, not because I think it's clearly better. +1 Yes, it was unintentional change. So we should put old logic back unless we have proof that this change make i

Re: [HACKERS] Idea on how to simplify comparing two sets

2017-02-23 Thread Nico Williams
On Tue, Feb 07, 2017 at 10:58:41AM -0500, Tom Lane wrote: > Joel Jacobson writes: > > Currently there is no simple way to check if two sets are equal. > > Uh ... maybe check whether SELECT set1 EXCEPT SELECT set2 > and SELECT set2 EXCEPT SELECT set1 are both empty? Even better, NATURAL(*) FULL O

Re: [HACKERS] Idea on how to simplify comparing two sets

2017-02-23 Thread Corey Huinker
> > FWIW I've found myself needing the precursor to this this (give me the > full diff) at least a couple times, and it's quite a PITA on anything but a > trivial relation. > > It's also not possible to make this easier via an SRF because you don't > know in advance what the result set looks like.

Re: [HACKERS] Checksums by default?

2017-02-23 Thread Bruce Momjian
On Sat, Jan 21, 2017 at 12:46:05PM -0500, Stephen Frost wrote: > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: > > As we don't know the performance impact is (there was no benchmark done > > on reasonably current code base) I really don't understand how you can > > judge if it's worth it or

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-23 Thread Tom Lane
Andrew Dunstan writes: > I'm not entirely sure how I should replace those DirectFunctionCall2 calls. Basically what we want is for the called function to be able to use the fcinfo->flinfo->fn_extra and fcinfo->flinfo->fn_mcxt fields of the FmgrInfo struct that GIN has set up for the btree_gin sup

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-23 Thread Andrew Dunstan
On 02/23/2017 04:41 PM, Tom Lane wrote: > BTW, while I'm looking at this ... isn't gin_enum_cmp broken on its face? > It's using DirectFunctionCall2 to call enum_cmp, and that's wrong because > DirectFunctionCall2 does not pass through a flinfo but enum_cmp needs to > have one. I've not tested,

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-23 Thread Fabien COELHO
Hello Corey, v16 is everything v15 promised to be. My 0.02€: Patch applies, make check ok, psql make check ok as well. Welcome to v15, highlights: - all conditional data structure management moved to conditional.h and conditional.c Indeed. I cannot say that I find it better, but (1) T

Re: [HACKERS] Idea on how to simplify comparing two sets

2017-02-23 Thread Nico Williams
On Tue, Feb 07, 2017 at 01:03:14PM -0500, Tom Lane wrote: > "David G. Johnston" writes: > > Actually ... now that you mention full join, I believe this works: > > select * from (select ...) s1 full join (select ...) s2 > on ((s1.*)=(s2.*)) where s1.* is distinct from s2.*; You can drop the .*

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-02-23 Thread Jim Nasby
On 1/23/17 9:23 PM, Jim Nasby wrote: I think the last step here is to figure out how to support switching between the current behavior and the "columnar" behavior of a dict of lists. I've thought more about this... instead of trying to switch from the current situation of 1 choice of how resul

[HACKERS] Poor memory context performance in large hash joins

2017-02-23 Thread Jeff Janes
When doing a hash join with large work_mem, you can have a large number of chunks. Then if ExecHashIncreaseNumBatches gets called, those chunks are walked through, moving the tuples to new chunks (or to disk, if they no longer match the batch's bitmask), and freeing the old chunks. The number of

Re: [HACKERS] Poor memory context performance in large hash joins

2017-02-23 Thread Peter Geoghegan
On Thu, Feb 23, 2017 at 2:13 PM, Jeff Janes wrote: > Is there a good solution to this? Could the new chunks be put in a > different memory context, and then destroy the old context and install the > new one at the end of ExecHashIncreaseNumBatches? I couldn't find a destroy > method for memory co

Re: [HACKERS] Poor memory context performance in large hash joins

2017-02-23 Thread Tom Lane
Jeff Janes writes: > The number of new chunks can be almost as as large as the number of old > chunks, especially if there is a very popular value. The problem is that > every time an old chunk is freed, the code in aset.c around line 968 has to > walk over all the newly allocated chunks in the l

Re: [HACKERS] bytea_output output of base64

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > Is there a reason we don't support base64 as a bytea_output output > > option, except that no one has implemented it? > > How about "we already have one too many bytea output formats"? > I don't think forcing co

Re: [HACKERS] Range Partitioning behaviour - query

2017-02-23 Thread Venkata B Nagothi
On Thu, Feb 23, 2017 at 3:14 PM, Amit Langote wrote: > Hi, > > On 2017/02/23 11:55, Venkata B Nagothi wrote: > > Hi Hackers, > > > > I have noticed the following behaviour in range partitioning which i felt > > is not quite correct (i missed reporting this) - > > > > I have tested by creating a d

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-02-23 Thread Michael Paquier
On Fri, Feb 24, 2017 at 3:09 AM, Jan Michálek wrote: > I can try it, doesn`t look dificult, but I`m worry, that I`m not able to > write clean, pretty code. If you want to have something available in Postgres 10, you had better be quick. The last commit fest of the development cycle of Postgres 10

Re: [HACKERS] bytea_output output of base64

2017-02-23 Thread David Fetter
On Thu, Feb 23, 2017 at 05:55:37PM -0500, Bruce Momjian wrote: > On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: > > Bruce Momjian writes: > > > Is there a reason we don't support base64 as a bytea_output output > > > option, except that no one has implemented it? > > > > How about "we

Re: [HACKERS] bytea_output output of base64

2017-02-23 Thread Tom Lane
David Fetter writes: > On Thu, Feb 23, 2017 at 05:55:37PM -0500, Bruce Momjian wrote: >> Is there a reason we chose hex over base64? > Whether there was or not, there's not a compelling reason now to break > people's software. When people want compression, methods a LOT more > effective than bas

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-23 Thread Corey Huinker
> > I'm not sure that '@' is the best choice, but this is just one char. > I noticed that it takes precedence over '!'. Why not. ISTM that orthogonal > features are not shown independently, but this is a preexisting state, and > it allows to have a shorter prompt, so why not. > My reasoning was

Re: [HACKERS] bytea_output output of base64

2017-02-23 Thread Andrew Dunstan
On 02/23/2017 06:52 PM, David Fetter wrote: > On Thu, Feb 23, 2017 at 05:55:37PM -0500, Bruce Momjian wrote: >> On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: >>> Bruce Momjian writes: Is there a reason we don't support base64 as a bytea_output output option, except that no

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-23 Thread Andrew Dunstan
On 02/23/2017 04:41 PM, Tom Lane wrote: > Andrew Dunstan writes: >> I'm not entirely sure how I should replace those DirectFunctionCall2 calls. > Basically what we want is for the called function to be able to use the > fcinfo->flinfo->fn_extra and fcinfo->flinfo->fn_mcxt fields of the > FmgrInf

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-23 Thread Michael Paquier
On Thu, Feb 23, 2017 at 10:54 PM, Stephen Frost wrote: > Micahel, > > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> Is there any interest for a feature like that? I have a non-polished >> patch at hand but I can work on that for the upcoming CF if there are >> voices in favor of such a f

Re: [HACKERS] Range Partitioning behaviour - query

2017-02-23 Thread Amit Langote
On 2017/02/24 8:38, Venkata B Nagothi wrote: > On Thu, Feb 23, 2017 at 3:14 PM, Amit Langote wrote: >> Upper bound of a range partition is an exclusive bound. A note was added >> recently to the CREATE TABLE page to make this clear. >> >> https://www.postgresql.org/docs/devel/static/sql-createtabl

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-23 Thread Tom Lane
Andrew Dunstan writes: > On 02/23/2017 04:41 PM, Tom Lane wrote: >> BTW, while I'm looking at this ... isn't gin_enum_cmp broken on its face? > Yes, that's what I'm trying to fix. I'd forgotten where this thread started. For a minute there I thought we had a live bug, rather than a deficiency i

Re: [HACKERS] Range Partitioning behaviour - query

2017-02-23 Thread David G. Johnston
On Thu, Feb 23, 2017 at 6:17 PM, Amit Langote wrote: > On 2017/02/24 8:38, Venkata B Nagothi wrote: > > On Thu, Feb 23, 2017 at 3:14 PM, Amit Langote wrote: > >> Upper bound of a range partition is an exclusive bound. A note was > added > >> recently to the CREATE TABLE page to make this clear.

Re: [HACKERS] Range Partitioning behaviour - query

2017-02-23 Thread Amit Langote
On 2017/02/24 10:38, David G. Johnston wrote: > On Thu, Feb 23, 2017 at 6:17 PM, Amit Langote > wrote: > >> On 2017/02/24 8:38, Venkata B Nagothi wrote: >>> On Thu, Feb 23, 2017 at 3:14 PM, Amit Langote wrote: Upper bound of a range partition is an exclusive bound. A note was >> added r

Re: [HACKERS] bytea_output output of base64

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 07:09:57PM -0500, Andrew Dunstan wrote: > > > On 02/23/2017 06:52 PM, David Fetter wrote: > > On Thu, Feb 23, 2017 at 05:55:37PM -0500, Bruce Momjian wrote: > >> On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: > >>> Bruce Momjian writes: > Is there a reason

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-02-23 Thread Amit Langote
On 2017/02/24 1:33, Simon Riggs wrote: > On 23 February 2017 at 06:40, Ashutosh Bapat > wrote: > >> I think this is ready for committer. > > Thanks for writing and reviewing this. I'll be happy to review and > commit. Please add to CF. Thanks. I've added it to CF: https://commitfest.postgresql

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Robert Haas
On Thu, Feb 23, 2017 at 10:00 PM, Simon Riggs wrote: > You seem a little defensive about some reasonable review comments. I am prone to that from time to time, and this may be an instance of it. > While its true that the patch had syntax documentation, there was no > user design documentation wh

Re: [HACKERS] Logical Replication and Character encoding

2017-02-23 Thread Shinoda, Noriyoshi
>From: Peter Eisentraut [mailto:peter.eisentr...@2ndquadrant.com] >Sent: Friday, February 24, 2017 1:32 AM >To: Petr Jelinek ; Kyotaro HORIGUCHI > >Cc: cr...@2ndquadrant.com; Shinoda, Noriyoshi ; >pgsql-hackers@postgresql.org >Subject: Re: [HACKERS] Logical Replication and Character encoding > >

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Jim Nasby
On 2/23/17 8:36 PM, Robert Haas wrote: We're 4 or 5 days away from the start of the last CommitFest. We have time to fix bugs and improve documentation and maybe tweak things here and there, but 3 and 4 are significant development projects. There isn't time to do that stuff right now and get it

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-23 Thread Michael Paquier
On Fri, Feb 24, 2017 at 5:37 AM, Jim Nasby wrote: > ISTM what's really needed is a good way for users to handle retention for > both WAL as well as base backups. A tool that did that would need to > understand what WAL is required to safely restore a base backup. It should > be possible for users

Re: [HACKERS] bytea_output output of base64

2017-02-23 Thread Jim Nasby
On 2/23/17 8:22 PM, Bruce Momjian wrote: I was just curious because it seems more compact than hex and many exchange formats use it, like SSL certificates and keys. I know you can encode() but I thought it might help make pg_dump output smaller. It undoubtedly would make pg_dump smaller, thoug

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Robert Haas
On Fri, Feb 24, 2017 at 8:06 AM, Robert Haas wrote: > Simon, this is ridiculous. We're 4 or 5 days away from the start of > the last CommitFest. We have time to fix bugs and improve > documentation and maybe tweak things here and there, but 3 and 4 are > significant development projects. There

Re: [HACKERS] Automatic cleanup of oldest WAL segments with pg_receivexlog

2017-02-23 Thread Michael Paquier
On Fri, Feb 24, 2017 at 1:10 AM, Magnus Hagander wrote: > I'm not sure this logic belongs in pg_receivexlog. If we put the decision > making there, then we lock ourselves into one "type of policy". > > Wouldn't this one, along with some other scenarios, be better provided by > the "run command at

  1   2   >