Re: TupleTableSlot abstraction

2018-03-13 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > I've recently been discussing with Robert how to abstract > TupleTableSlots in the light of upcoming developments around abstracting > storage away. Besides that aspect I think we also need to make them > more abstract to later deal with vectorized excution, but

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Chapman Flack
On 03/13/2018 01:50 PM, Stephen Frost wrote: > Yes, that's true, but PG has never paid attention to POSIX ACLs or Linux > capabilities. Changing it to do so is quite a bit beyond the scope... I think we're largely in agreement here, as my aim was not to advocate that PG should work harder to

Re: TupleTableSlot abstraction

2018-03-13 Thread Andres Freund
On 2018-03-13 15:18:34 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Hi, > > > > I've recently been discussing with Robert how to abstract > > TupleTableSlots in the light of upcoming developments around abstracting > > storage away. Besides that aspect I think we also need to make

Re: JIT compiling with LLVM v11

2018-03-13 Thread Andres Freund
On 2018-03-13 10:25:49 -0400, Peter Eisentraut wrote: > On 3/12/18 17:04, Andres Freund wrote: > > │ JIT: > > │ > > │ Functions: 4 >

Re: JIT compiling with LLVM v11

2018-03-13 Thread Andres Freund
Hi, On 2018-03-13 14:36:44 -0400, Robert Haas wrote: > On Mon, Mar 12, 2018 at 5:04 PM, Andres Freund wrote: > > Currently a handful of explain outputs in the regression tests change > > output when compiled with JITing. Therefore I'm thinking of adding > > JITINFO or such

Re: INOUT parameters in procedures

2018-03-13 Thread Pavel Stehule
2018-03-13 14:14 GMT+01:00 Peter Eisentraut < peter.eisentr...@2ndquadrant.com>: > On 3/8/18 02:25, Pavel Stehule wrote: > > It looks like some error in this concept. The rules for enabling > > overwriting procedures should modified, so this collision should not be > > done. > > > > When I using

Re: [HACKERS] why not parallel seq scan for slow functions

2018-03-13 Thread Robert Haas
On Wed, Feb 14, 2018 at 5:37 AM, Amit Kapila wrote: > Your concern is valid, but isn't the same problem exists in another > approach as well, because in that also we can call > adjust_paths_for_srfs after generating gather path which means that we > might lose some

Re: JIT compiling with LLVM v11

2018-03-13 Thread Robert Haas
On Mon, Mar 12, 2018 at 5:04 PM, Andres Freund wrote: > Currently a handful of explain outputs in the regression tests change > output when compiled with JITing. Therefore I'm thinking of adding > JITINFO or such option, which can be set to false for those tests? Can we spell

Re: proposal: schema variables

2018-03-13 Thread Pavel Stehule
2018-03-13 10:54 GMT+01:00 Pavel Luzanov : > Pavel Stehule wrote > > Now, there is one important question - storage - Postgres stores all > > objects to files - only memory storage is not designed yet. This is part, > > where I need a help. > > O, I do not feel confident

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Tom Lane
Chapman Flack writes: > On 03/13/2018 01:50 PM, Stephen Frost wrote: >> I'll point out that PG does run on quite a few other OS's beyond Linux. > I'll second that, as I think it is making my argument. When I can > supply three or four examples of semantic subtleties that

Re: Fix error in ECPG while connection handling

2018-03-13 Thread Jeevan Ladhe
> Thanks for spotting and fixing. I will push the patch as soon as I'm > online again. > Thanks Michael for taking care of this. Regards, Jeevan Ladhe.

neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Jeff Janes
The following commit has caused a devastating performance regression in concurrent refresh of MV: commit 7ca25b7de6aefa5537e0dbe56541bc41c0464f97 Author: Tom Lane Date: Wed Nov 29 22:00:29 2017 -0500 Fix neqjoinsel's behavior for semi/anti join cases. The below

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2018-03-13 Thread Robert Haas
On Mon, Feb 19, 2018 at 4:02 AM, David Rowley wrote: > On 19 February 2018 at 18:01, David Rowley > wrote: >> On 19 February 2018 at 15:11, Tomas Vondra >> wrote: >>> and perhaps we should do

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Chapman Flack
On 03/13/2018 02:47 PM, Tom Lane wrote: > Well, to be blunt, what we target is POSIX-compatible systems. If > you're telling us to worry about non-POSIX filesystem semantics, > and your name is not Microsoft, it's going to be a hard sell. > We have enough to do just keeping up with that scope of

Re: parallel append vs. simple UNION ALL

2018-03-13 Thread Robert Haas
On Tue, Mar 13, 2018 at 12:35 AM, Ashutosh Bapat wrote: > It looks like it was not changed in all the places. make falied. I > have fixed all the instances of these two functions in the attached > patchset (only 0003 changes). Please check. Oops. Thanks. I'm

Re: Problem while setting the fpw with SIGHUP

2018-03-13 Thread Michael Paquier
On Fri, Mar 09, 2018 at 01:42:04PM +0530, Dilip Kumar wrote: > While setting the full_page_write with SIGHUP I hit an assert in checkpoint > process. And, that is because inside a CRITICAL section we are calling > RecoveryInProgress which intern allocates memory. So I have moved >

Re: PATCH: Unlogged tables re-initialization tests

2018-03-13 Thread Michael Paquier
On Mon, Mar 12, 2018 at 02:33:18PM -0400, David Steele wrote: > On 3/12/18 11:59 AM, Dagfinn Ilmari Mannsåker wrote: >> However, that is still wrong, because die() and BAIL_OUT() mean >> different things: die() aborts the current test script, while BAIL_OUT() >> aborts the entire 'prove' run, i.e.

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread David Gould
On Mon, 12 Mar 2018 12:21:34 -0400 Tom Lane wrote: > I wrote: > > Maybe this type of situation is an argument for trusting an ANALYZE-based > > estimate more than the VACUUM-based estimate. I remain uncomfortable with > > that in cases where VACUUM looked at much more of the

Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Ashutosh Bapat
On Mon, Mar 12, 2018 at 10:02 PM, Narendra Pradeep U U wrote: > Hi , > > Recently I came across a case where the planner choose larger table on > hash side. I am not sure whether it is an intended behavior or we are > missing something. > > I have two

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-13 Thread Andrey Borodin
Tom, thanks for inspecting the patch. There's so many problems that slipped from my attention... But one thing that bothers me most is the problem with predicate locking > 13 марта 2018 г., в 0:55, Tom Lane написал(а): > > The PredicateLockPage call also troubles me quite a

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Michael Paquier
On Mon, Mar 12, 2018 at 03:14:13PM -0400, Stephen Frost wrote: > We already had a discussion about having a GUC for this and concluded, > rightly in my view, that it's not sensible to have since we don't want > all of the various tools having to read and parse out postgresql.conf. If the problem

RE: Temporary tables prevent autovacuum, leading to XID wraparound

2018-03-13 Thread Tsunakawa, Takayuki
From: Tom Lane [mailto:t...@sss.pgh.pa.us] > So the correct fix is to improve autovacuum's check to discover whether > an old temp table is orphaned, so that it isn't fooled by putative owner > processes that are connected to some other DB. Step 2 of the proposed patch > tries to do that, but

Re: proposal: schema variables

2018-03-13 Thread Pavel Luzanov
Pavel Stehule wrote > Now, there is one important question - storage - Postgres stores all > objects to files - only memory storage is not designed yet. This is part, > where I need a help. O, I do not feel confident in such questions. May be some ideas you can get from extension with similar

Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Narendra Pradeep U U
Hi, Thanks everyone for your suggestions. I would like to add explain analyze of both the plans so that we can have broader picture. I have a work_mem of 1000 MB. The Plan which we get regularly with table being analyzed . tpch=# explain analyze select b from tab2 left join tab1

Re: SQL/JSON: functions

2018-03-13 Thread Simon Riggs
On 7 March 2018 at 14:34, Nikita Glukhov wrote: > Attached 12th version of SQL/JSON patches rebased onto the latest master. Please write some docs or notes to go with this. If you drop a big pile of code with no explanation it will just be ignored. I think many people

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Aleksander Alekseev
Hello everyone, > > I am mostly interested in anything that requires C/C++ implementation and > > AlgoDS. > > > > For that reason I would love to work in any of the following (in that > > order of preference): > > > >1. Sorting algorithms benchmark and implementation > >2. Enhancing

Re: Using base backup exclusion filters to reduce data transferred with pg_rewind

2018-03-13 Thread Anastasia Lubennikova
05.02.2018 10:10, Michael Paquier: So the patch set attached is made of the following: - 0001, which refactors all hardcoded system paths into pg_paths.h. This modifies only initdb.c and basebackup.c to ease reviews. - 0002 spreads the path changes and the use of pg_paths.h across the core code.

Re: Additional Statistics Hooks

2018-03-13 Thread David Rowley
On 13 March 2018 at 11:44, Tom Lane wrote: > While it would certainly be nice to have better behavior for that, > "add a hook so users who can write C can fix it by hand" doesn't seem > like a great solution. On top of the sheer difficulty of writing a > hook function, you'd

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-13 Thread Alexander Korotkov
On Mon, Mar 12, 2018 at 9:47 AM, Andrey Borodin wrote: > > 12 марта 2018 г., в 1:54, Alexander Korotkov > написал(а): > > > > On Wed, Mar 7, 2018 at 8:30 PM, Alvaro Herrera > wrote: > > I suggest to create a new

Re: MCV lists for highly skewed distributions

2018-03-13 Thread John Naylor
Hi Dean, I've looked over your patch briefly, but not tested it yet. > [construction of a pathological data set] > So the table has around 31 million rows, and the stats target is maxed > out -- we're sampling around 10% of the table, and it's not possible > to sample more. Looking at the value

Re: Additional Statistics Hooks

2018-03-13 Thread Ashutosh Bapat
On Tue, Mar 13, 2018 at 4:14 AM, Tom Lane wrote: > Mat Arye writes: >> So the use-case is an analytical query like > >> SELECT date_trunc('hour', time) AS MetricMinuteTs, AVG(value) as avg >> FROM hyper >> WHERE time >= '2001-01-04T00:00:00' AND time <=

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Christos Maris
Hi all, At first, I appreciate your insights Craig, but I think I will stick with AlgoDS ;) Aleksander, I am mostly interested on the sorting algos benchmark and implementation one. I will start writing a proposal soon enough. Do you have any project related insights as to what I should put in

Re: Re: [HACKERS] make async slave to wait for lsn to be replayed

2018-03-13 Thread David Steele
Hi Ivan, On 3/6/18 9:25 PM, Michael Paquier wrote: > On Tue, Mar 06, 2018 at 02:24:24PM +0300, Ivan Kartyshov wrote: >> Hello, I now is preparing the patch over syntax that Simon offered. And in >> few day I will update the patch. >> Thank you for your interest in thread. > > It has been more

Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2018-03-13 Thread Amit Kapila
On Mon, Mar 12, 2018 at 7:18 PM, Amit Kapila wrote: > On Mon, Mar 12, 2018 at 12:18 AM, Alexander Korotkov > wrote: >> On Sat, Mar 3, 2018 at 2:53 PM, Amit Kapila wrote: >>> >>> On Fri, Mar 2, 2018 at 9:27 AM, Thomas

Re: PATCH: Unlogged tables re-initialization tests

2018-03-13 Thread David Steele
Thanks for reviewing, Peter. On 3/9/18 5:23 PM, Peter Eisentraut wrote: > This seems like a useful test. > > On 3/5/18 12:35, David Steele wrote: >> +mkdir($tablespaceDir) >> +or die "unable to mkdir \"$tablespaceDir\""; > > Use BAIL_OUT instead of die in tests. Done. >> +

Re: Fix error in ECPG while connection handling

2018-03-13 Thread Michael Meskes
> Now, ideally the connection would have been null here, but, as the > 'ClosePortalStmt' > rule freed the connection but did not set it to NULL, it still sees > that there > is a connection(which is actually having garbage in it) and throws an > error. Thanks for spotting and fixing. I will push

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-13 Thread Alvaro Herrera
Alexander Korotkov wrote: > And what happen if somebody concurrently set (fastupdate = on)? > Can we miss conflicts because of that? I think it'd be better to have that option require AccessExclusive lock, so that it can never be changed concurrently with readers. Seems to me that penalizing

Re: INOUT parameters in procedures

2018-03-13 Thread Peter Eisentraut
On 3/8/18 02:25, Pavel Stehule wrote: > It looks like some error in this concept. The rules for enabling > overwriting procedures should modified, so this collision should not be > done. > > When I using procedure from PL/pgSQL, then it is clear, so I place on > *OUT position variables. But when

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Aleksander Alekseev
Hello Christos, > Do you have any project related insights as to what I should put in > there? Nope :) I believe Andrey Borodin and Atri Sharma have (added to CC). -- Best regards, Aleksander Alekseev signature.asc Description: PGP signature

Re: Using base backup exclusion filters to reduce data transferred with pg_rewind

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 01:16:27PM +0300, Anastasia Lubennikova wrote: > Since these patches contain mostly cosmetic changes and do not break > anything, I think it's fine to mark this thread as Ready For Committer > without long discussion. Thanks Anastasia for the review. The refactoring is

Re: SQL/JSON: functions

2018-03-13 Thread Oleg Bartunov
On Tue, Mar 13, 2018 at 2:04 PM, Simon Riggs wrote: > On 7 March 2018 at 14:34, Nikita Glukhov wrote: > >> Attached 12th version of SQL/JSON patches rebased onto the latest master. > > Please write some docs or notes to go with this. > > If you

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-13 Thread Jeevan Chalke
Hi, I have resolved all the comments/issues reported in this new patch-set. Changes done by Ashutosh Bapat for splitting out create_grouping_paths() into separate functions so that partitionwise aggregate code will use them were based on my partitionwise aggregate changes. Those were like

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-13 Thread Andrey Borodin
> 13 марта 2018 г., в 17:02, Alexander Korotkov > написал(а): > > BTW to BTW. I think we should check pending list size with > GinGetPendingListCleanupSize() here > + > + /* > +* If fast update is enabled, we acquire a predicate lock on the > entire

Re: JIT compiling with LLVM v11

2018-03-13 Thread Peter Eisentraut
On 3/12/18 17:04, Andres Freund wrote: > │ JIT: >│ > │ Functions: 4 >│ > │ Inlining: false

Re: PATCH: Configurable file mode mask

2018-03-13 Thread David Steele
On 3/13/18 2:46 AM, Michael Paquier wrote: > On Mon, Mar 12, 2018 at 03:14:13PM -0400, Stephen Frost wrote: >> We already had a discussion about having a GUC for this and concluded, >> rightly in my view, that it's not sensible to have since we don't want >> all of the various tools having to read

Re: Transform for pl/perl

2018-03-13 Thread Nikita Glukhov
Hi. I have reviewed this patch too. Attached new version with v8-v9 delta-patch. Here is my changes: * HV_ToJsonbValue(): - addded missing hv_iterinit() - used hv_iternextsv() instead of hv_iternext(), HeSVKEY_force(), HeVAL() * SV_ToJsonbValue(): - added recursive

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Tom Lane
Stephen Frost writes: > * Michael Paquier (mich...@paquier.xyz) wrote: >> If the problem is parsing, it could as well be more portable to put that >> in the control file, no? > Then we'd need a tool to allow changing it for people who do want to > change it. There's no

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Stephen Frost
Greetings, * Aleksander Alekseev (a.aleks...@postgrespro.ru) wrote: > Craig, I believe you probably did something wrong if you had to work > with some library directly. Actually you generate classes from text > description and just use them. I worked with Thrift some time ago, in > 2015 [1]. I

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Michael, * Michael Paquier (mich...@paquier.xyz) wrote: > On Mon, Mar 12, 2018 at 03:14:13PM -0400, Stephen Frost wrote: > > We already had a discussion about having a GUC for this and concluded, > > rightly in my view, that it's not sensible to have since we don't want > > all of the various

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-13 Thread Stephen Frost
Greetings, * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: > On Mon, Mar 12, 2018 at 1:25 PM, Etsuro Fujita > wrote: > > (2018/03/09 20:55), Etsuro Fujita wrote: > >> (2018/03/08 14:24), Ashutosh Bapat wrote: > >>> For local constraints to be enforced, we

Re: [HACKERS] Proposal: generic WAL compression

2018-03-13 Thread Oleg Ivanov
On 02/07/2018 09:02 PM, Markus Nullmeier wrote: One general comment I can already make is that enabling compression should be made optional, which appears to be a small and easy addition to the generic WAL API. The new version of the patch is attached. In order to control generic WAL

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Andrey Borodin
Thanks, Aleksander!SP- > 13 марта 2018 г., в 19:03, Aleksander Alekseev > написал(а): > >> Do you have any project related insights as to what I should put in >> there? > Christos, as far as I remember, good proposal must have schedule, implementation details and

Re: committing inside cursor loop

2018-03-13 Thread Peter Eisentraut
On 3/6/18 07:48, Ildus Kurbangaliev wrote: > Although as was discussed before it seems inconsistent without ROLLBACK > support. There was a little discussion about it, but no replies. Maybe > the status of the patch should be changed to 'Waiting on author' until > the end of discussion. I'm

Re: PATCH: Configurable file mode mask

2018-03-13 Thread David Steele
On 3/13/18 11:00 AM, Tom Lane wrote: > Stephen Frost writes: >> * Michael Paquier (mich...@paquier.xyz) wrote: >>> If the problem is parsing, it could as well be more portable to put that >>> in the control file, no? > >> Then we'd need a tool to allow changing it for people

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread Tom Lane
David Gould writes: > I also thought about the theory and am confident that there really is no way > to trick it. Basically if there are enough pages that are different to affect > the overall density, say 10% empty or so, there is no way a random sample > larger than a few

Re: Additional Statistics Hooks

2018-03-13 Thread Mat Arye
On Tue, Mar 13, 2018 at 6:56 AM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Tue, Mar 13, 2018 at 4:14 AM, Tom Lane wrote: > > Mat Arye writes: > >> So the use-case is an analytical query like > > > >> SELECT date_trunc('hour', time) AS

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread Tom Lane
David Gould writes: > I have attached the patch we are currently using. It applies to 9.6.8. I > have versions for older releases in 9.4, 9.5, 9.6. I fails to apply to 10, > and presumably head but I can update it if there is any interest. > The patch has three main features: >

Re: Additional Statistics Hooks

2018-03-13 Thread Mat Arye
On Tue, Mar 13, 2018 at 6:31 AM, David Rowley wrote: > On 13 March 2018 at 11:44, Tom Lane wrote: > > While it would certainly be nice to have better behavior for that, > > "add a hook so users who can write C can fix it by hand" doesn't seem >

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-13 Thread Arthur Zakirov
On Mon, Mar 12, 2018 at 02:11:42PM +0100, Julian Markwort wrote: > > In same manner pg_stat_statements_good_plan_reset() and > > pg_stat_statements_bad_plan_reset() functions can be combined in > > function: > > > pg_stat_statements_plan_reset(in queryid bigint, in good boolean, in bad > >

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-13 Thread Michail Nikolaev
Hello. Tom, thanks a lot for your thorough review. > What you've done to > IndexNext() is a complete disaster from a modularity standpoint: it now > knows all about the interactions between index_getnext, index_getnext_tid, > and index_fetch_heap. I was looking into the current IndexOnlyNext

Re: PATCH: Unlogged tables re-initialization tests

2018-03-13 Thread Alvaro Herrera
Dagfinn Ilmari Mannsåker wrote: > $SIG{__DIE__} gets called even for exceptions that would be caught by a > surrunding eval block, so this should at the very least be: > > $SIG{__DIE__} = sub { BAIL_OUT(@_) unless $^S }; > > However, that is still wrong, because die() and BAIL_OUT() mean >

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Greetings, * David Steele (da...@pgmasters.net) wrote: > On 3/13/18 11:00 AM, Tom Lane wrote: > > Stephen Frost writes: > >> * Michael Paquier (mich...@paquier.xyz) wrote: > >>> If the problem is parsing, it could as well be more portable to put that > >>> in the control

Re: PATCH: Exclude temp relations from base backup

2018-03-13 Thread David Steele
Hi, On 2/28/18 10:55 AM, David Steele wrote: > This is a follow-up patch from the exclude unlogged relations discussion > [1]. > > The patch excludes temporary relations during a base backup using the > existing looks_like_temp_rel_name() function for identification. > > It shares code to

Re: PATCH: Configurable file mode mask

2018-03-13 Thread David Steele
Hi Michael, On 3/12/18 3:28 AM, Michael Paquier wrote: > On Fri, Mar 09, 2018 at 01:51:14PM -0500, David Steele wrote: >> How about a GUC that enforces one mode or the other on startup? Default >> would be 700. The GUC can be set automatically by initdb based on the >> -g option. We had this

Re: [submit code] I develop a tool for pgsql, how can I submit it

2018-03-13 Thread Euler Taveira
2018-03-13 12:19 GMT-03:00 leap : > I develop a tool for pgsql, I want to submit it on pgsql. > how can I submit it? > As Laurenz said a tool doesn't necessarily have to be part of PostgreSQL. Sometimes a lot of people ask for a tool, someone write it and the community decide to

Re: ALTER TABLE ADD COLUMN fast default

2018-03-13 Thread Andrew Dunstan
On Tue, Mar 13, 2018 at 10:58 PM, Andrew Dunstan wrote: > On Tue, Mar 13, 2018 at 2:40 PM, Andrew Dunstan > wrote: > >>> >>> Going by the commitfest app, the patch still does appear to be waiting >>> on Author. Never-the-less, I've

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread Alvaro Herrera
Amit Langote wrote: > I will continue working on improving the comments / cleaning things up and > post a revised version soon, but until then please look at the attached. I tried to give this a read. It looks pretty neat stuff -- as far as I can tell, it follows Robert's sketch for how this

Re: SQL/JSON: functions

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 04:08:01PM +0300, Oleg Bartunov wrote: > The docs are here > https://github.com/obartunov/sqljsondoc/blob/master/README.jsonpath.md > > It's not easy to write docs for SQL/JSON in xml, so I decided to write in more > friendly way. We'll have time to convert it to postgres

Re: SQL/JSON: functions

2018-03-13 Thread Andres Freund
On 2018-03-14 07:54:35 +0900, Michael Paquier wrote: > On Tue, Mar 13, 2018 at 04:08:01PM +0300, Oleg Bartunov wrote: > > The docs are here > > https://github.com/obartunov/sqljsondoc/blob/master/README.jsonpath.md > > > > It's not easy to write docs for SQL/JSON in xml, so I decided to write in

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Tom Lane
Thomas Munro writes: > There is a fundamental and complicated estimation problem lurking here > of course and I'm not sure what to think about that yet. Maybe there > is a very simple fix for this particular problem: Ah, I see you thought of the same hack I did.

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread Alvaro Herrera
By the way, I checked whether patch 0002 (additional tests) had an effect on coverage, and couldn't detect any changes in terms of lines/functions. Were you able to find any bugs in your code thanks to the new tests that would not have been covered by existing tests? -- Álvaro Herrera

Re: JIT compiling with LLVM v11

2018-03-13 Thread Andres Freund
Hi, On 2018-03-14 10:32:40 +1300, Thomas Munro wrote: > I decided to try this on a CentOS 7.2 box. It has LLVM 3.9 in the > 'epel' package repo, but unfortunately it only has clang 3.4. That's a bit odd, given llvm and clang really live in the same repo... > clang: error: unknown argument:

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Thomas Munro
On Wed, Mar 14, 2018 at 11:34 AM, Thomas Munro wrote: > LOG: duration: 26101.612 ms plan: > Query Text: SELECT newdata FROM pg_temp_3.pg_temp_16452 newdata WHERE > newdata IS NOT NULL AND EXISTS (SELECT * FROM pg_temp_3.pg_temp_16452 > newdata2 WHERE newdata2 IS

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Tom Lane
Thomas Munro writes: > This looks like an invisible correlation problem. Yeah --- the planner has no idea that the join rows satisfying newdata.* *= newdata2.* are likely to be exactly the ones not satisfying newdata.ctid <> newdata2.ctid. It's very accidental

JIT compiling with LLVM v12

2018-03-13 Thread Andres Freund
Hi, I've pushed a revised and rebased version of my JIT patchset. The git tree is at https://git.postgresql.org/git/users/andresfreund/postgres.git in the jit branch https://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=shortlog;h=refs/heads/jit There's nothing hugely

Re: JIT compiling with LLVM v11

2018-03-13 Thread Thomas Munro
On Thu, Mar 1, 2018 at 9:02 PM, Andres Freund wrote: > Biggest changes: > - LLVM 3.9 - master are now supported. This includes a good chunk of > work by Pierre Ducroquet. I decided to try this on a CentOS 7.2 box. It has LLVM 3.9 in the 'epel' package repo, but

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Thomas Munro
On Wed, Mar 14, 2018 at 8:07 AM, Jeff Janes wrote: > The following commit has caused a devastating performance regression > in concurrent refresh of MV: > > commit 7ca25b7de6aefa5537e0dbe56541bc41c0464f97 > Author: Tom Lane > Date: Wed Nov 29 22:00:29

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Chapman Flack
On 03/13/2018 10:40 AM, Stephen Frost wrote: > * Michael Paquier (mich...@paquier.xyz) wrote: >> Well, one thing is that the current checks in the postmaster make sure >> that a data folder is never using anything else than 0700. From a >> security point of view, making it possible to allow a

Re: Additional Statistics Hooks

2018-03-13 Thread Tom Lane
Mat Arye writes: > An example, of a case a protransform type system would not be able to > optimize is mathematical operator expressions like bucketing integers by > decile --- (integer / 10) * 10. Uh, why not? An estimation function that is specific to integer divide

Inquiry regarding the projects under GSOC

2018-03-13 Thread Dr K.G Yadav
Hello Postgres Team I am Ankit 3rd year B.tech student from India I am really interested to work along with your organisation as an intern to rectify the issues and bugs as I love challenges and stated in the docs that some features might be not possible to implement so I would love to try. My

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Greetings Chapman, * Chapman Flack (c...@anastigmatix.net) wrote: > On 03/13/2018 10:40 AM, Stephen Frost wrote: > > ... Ultimately, the default which makes sense here isn't a > > one-size-fits-all but is system dependent and the administrator should > > be able to choose what permissions they

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread Jesper Pedersen
Hi Amit, On 03/13/2018 07:37 AM, Amit Langote wrote: I will continue working on improving the comments / cleaning things up and post a revised version soon, but until then please look at the attached. Passes check-world. Some minor comments: 0001: Ok 0002: Ok 0003: * Trailing white space

[submit code] I develop a tool for pgsql, how can I submit it

2018-03-13 Thread leap
hello! I develop a tool for pgsql, I want to submit it on pgsql. how can I submit it? address: https://github.com/leapking/pgcheck

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Tom Lane
David Steele writes: > On 3/12/18 3:28 AM, Michael Paquier wrote: >> In pg_rewind and pg_resetwal, isn't that also a portion which is not >> necessary without the group access feature? > These seem like a good idea to me with or without patch 03. Some of our > front-end

Re: add queryEnv to ExplainOneQuery_hook

2018-03-13 Thread Michael Paquier
On Wed, Mar 14, 2018 at 05:36:26PM +1300, Thomas Munro wrote: > Hmm. I suppose we could have invented a new extended hook with a > different name and back-patched it so that PG10 would support both. > Then binary compatibility with existing compiled extensions wouldn't > be affected AFAICS, but

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-13 Thread Michael Paquier
On Fri, Mar 09, 2018 at 04:42:30PM -0500, Peter Eisentraut wrote: > It seems, however, that PGhost() has always been broken for hostaddr > use. In 9.6 (before the multiple-hosts stuff was introduced), when > connecting to "hostaddr=127.0.0.1", PGhost() returns "/tmp". Urgh. > > I think we

Re: User defined data types in Logical Replication

2018-03-13 Thread Masahiko Sawada
On Wed, Mar 7, 2018 at 9:19 PM, Masahiko Sawada wrote: > On Wed, Mar 7, 2018 at 2:52 AM, Alvaro Herrera > wrote: >> Masahiko Sawada wrote: >>> On Tue, Mar 6, 2018 at 8:35 AM, Alvaro Herrera >>> wrote: >> >>> > Therefore,

planner bug regarding lateral and subquery?

2018-03-13 Thread Tatsuro Yamada
Hi Hackers, I found a bug, maybe. If it is able to get an explain command result from below query successfully, I think that it means the query is executable. However, I got an error by executing the query without an explain command. I guess that planner makes a wrong plan. I share a

Re: planner bug regarding lateral and subquery?

2018-03-13 Thread David G. Johnston
On Tuesday, March 13, 2018, Tatsuro Yamada wrote: > Hi Hackers, > > I found a bug, maybe. > If it is able to get an explain command result from below query > successfully, > I think that it means the query is executable. > There is a difference between executable,

Re: Faster inserts with mostly-monotonically increasing values

2018-03-13 Thread Pavan Deolasee
On Sun, Mar 11, 2018 at 9:18 PM, Claudio Freire wrote: > On Sun, Mar 11, 2018 at 2:27 AM, Pavan Deolasee > > > > > Yes, I will try that next - it seems like a good idea. So the idea would > be: > > check if the block is still the rightmost block and the insertion-key is >

Re: inserts into partitioned table may cause crash

2018-03-13 Thread Amit Langote
Fujita-san, Thanks for the updates and sorry I couldn't reply sooner. On 2018/03/06 21:26, Etsuro Fujita wrote: > One thing I notice while working on this is this in ExecInsert/CopyFrom, > which I moved to ExecPrepareTupleRouting as-is for the former: > > /* > * If we're capturing

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-03-13 Thread Masahiko Sawada
On Sat, Mar 10, 2018 at 3:40 AM, Alexander Korotkov wrote: > On Fri, Mar 9, 2018 at 3:12 PM, Masahiko Sawada > wrote: >> >> On Fri, Mar 9, 2018 at 8:43 AM, Alexander Korotkov >> wrote: >> > 2) These parameters are

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread David Rowley
On 14 March 2018 at 06:54, Jesper Pedersen wrote: > * "Add more expression types..." -- Are you planning to add more of > these ? Otherwise change the comment Run-time pruning will deal with Param types here. The comment there might be just to remind us all that

Re:Re: Re: [GSOC 18] Performance Farm Project——Initialization Project

2018-03-13 Thread Hongyuan Ma
Hi Dave, I am willing to use React to re-create the front-end application. Since I plan to use separate front-end and back-end development methods, this means that there will be no html files in Django applications. Front-end and back-end applications will interact via restful api. I will use

Re: Faster inserts with mostly-monotonically increasing values

2018-03-13 Thread Claudio Freire
On Wed, Mar 14, 2018 at 1:36 AM, Pavan Deolasee wrote: > > > On Sun, Mar 11, 2018 at 9:18 PM, Claudio Freire > wrote: >> >> On Sun, Mar 11, 2018 at 2:27 AM, Pavan Deolasee >> >> > >> > Yes, I will try that next - it seems like a good idea. So the

Re: Faster inserts with mostly-monotonically increasing values

2018-03-13 Thread Pavan Deolasee
On Wed, Mar 14, 2018 at 10:58 AM, Claudio Freire wrote: > > > I'm thinking there could be contention on some lock somewhere. > > Can you attach the benchmark script you're using so I can try to reproduce > it? > I am using a very ad-hoc script.. But here it is.. It

Re: add queryEnv to ExplainOneQuery_hook

2018-03-13 Thread Tom Lane
Michael Paquier writes: > On Wed, Mar 14, 2018 at 05:36:26PM +1300, Thomas Munro wrote: >> Hmm. I suppose we could have invented a new extended hook with a >> different name and back-patched it so that PG10 would support both. >> Then binary compatibility with existing

Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Ashutosh Bapat
On Tue, Mar 13, 2018 at 4:32 PM, Narendra Pradeep U U wrote: > Hi, > Thanks everyone for your suggestions. I would like to add explain > analyze of both the plans so that we can have broader picture. > > I have a work_mem of 1000 MB. > > The Plan which we get

Re: Changing the autovacuum launcher scheduling; oldest table first algorithm

2018-03-13 Thread Masahiko Sawada
On Tue, Mar 6, 2018 at 11:27 PM, Alvaro Herrera wrote: > Hello > > I haven't read your respective patches yet, but both these threads > brought to memory a patch I proposed a few years ago that I never > completed: > >

Re: [bug fix] Cascaded standby cannot start after a clean shutdown

2018-03-13 Thread Michael Paquier
On Tue, Feb 27, 2018 at 05:15:29AM +, Tsunakawa, Takayuki wrote: > That was my first thought, and I gave it up. As you say, > XLogReadRecord() could allocate up to 1 GB of memory for a garbage. > That allocation can fail due to memory shortage, which prevents the > recovery from proceeding.

  1   2   >