Re: [HACKERS] bytea_output output of base64

2017-02-24 Thread Fabien COELHO
It undoubtedly would make pg_dump smaller, though I'm not sure how much that's worth since if you care at all about that you'll gzip it. But, the other thing it might do is speed up COPY, especially on input. Some performance tests of that might be interesting. For what it is worth: Ascii8

Re: [HACKERS] UPDATE of partition key

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 1:18 PM, David G. Johnston wrote: > For my own sanity - the move update would complete successfully and break > every ctid chain that it touches. Any update lined up behind it in the lock > queue would discover their target record has been deleted and would > experience wh

Re: [HACKERS] Add checklist item for psql completion to commitfest review

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 2:28 AM, Jim Nasby wrote: > I've never messed with completion so I don't know how hard it is, but my > impression is that it gets added after the fact not because of any > intentional decisions but because people simply forget about it. ISTM it > would be more efficient of

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Pavan Deolasee
On Thu, Feb 23, 2017 at 11:30 PM, Bruce Momjian wrote: > On Wed, Feb 1, 2017 at 10:46:45AM +0530, Pavan Deolasee wrote: > > > contains a WARM tuple. Alternate ideas/suggestions and review of > the > > design > > > are welcome! > > > > t_infomask2 contains one last unused bit, > >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Robert Haas
On Thu, Feb 23, 2017 at 9:21 PM, Bruce Momjian wrote: > On Tue, Jan 31, 2017 at 04:52:39PM +0530, Pavan Deolasee wrote: >> The other critical bug I found, which unfortunately exists in the master too, >> is the index corruption during CIC. The patch includes the same fix that I've >> proposed on t

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Pavan Deolasee
On Thu, Feb 23, 2017 at 11:53 PM, Bruce Momjian wrote: > On Thu, Feb 23, 2017 at 03:03:39PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > As I remember, WARM only allows > > > a single index-column change in the chain. Why are you seeing such a > > > large performance improveme

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Pavan Deolasee
On Fri, Feb 24, 2017 at 2:13 PM, Robert Haas wrote: > On Thu, Feb 23, 2017 at 9:21 PM, Bruce Momjian wrote: > > > > > I have what might be a supid question. As I remember, WARM only allows > > a single index-column change in the chain. Why are you seeing such a > > large performance improvemen

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Pavan Deolasee
On Fri, Feb 24, 2017 at 12:28 AM, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 03:45:24PM -0300, Alvaro Herrera wrote: > > > > > and potentially trim the first HOT chain as those tuples become > > > > invisible. > > > > > > That can already happen even without WARM, no

Re: [HACKERS] Keep ECPG comment for log_min_duration_statement

2017-02-24 Thread Okano, Naoki
Hi, Michael wrote: > The reason for not keeping the comments in the statement was simply to > make the parser simpler. If you added this feature, do we still see a > reason for keeping the old version? Or in other words, shouldn't we > make the "enable-parse-comment" version the default without a

Re: [HACKERS] Proposal : Parallel Merge Join

2017-02-24 Thread Amit Kapila
On Tue, Feb 14, 2017 at 5:22 PM, Dilip Kumar wrote: > On Tue, Feb 14, 2017 at 12:25 PM, Amit Kapila wrote: > Apart from this, there was one small problem in an earlier version > which I have corrected in this. > > + /* Consider only parallel safe inner path */ > + if (innerpath != NULL && > + inn

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 2:42 PM, Pavan Deolasee wrote: > Let's take an example. Say, we have a table (a int, b int, c text) and two > indexes on first two columns. > >H W > H > (1, 100, 'foo') -> (1, 100, 'bar') --> (1, 200, 'bar') -

Re: [HACKERS] UPDATE of partition key

2017-02-24 Thread Simon Riggs
On 24 February 2017 at 07:02, Robert Haas wrote: > On Mon, Feb 20, 2017 at 2:58 PM, Amit Khandekar > wrote: >> I am inclined to at least have some option for the user to decide the >> behaviour. In the future we can even consider support for walking >> through the ctid chain across multiple relf

[HACKERS] Making clausesel.c Smarter

2017-02-24 Thread David Rowley
I've stumbled over an interesting query out in the wild where the query was testing a nullable timestamptz column was earlier than some point in time, and also checking the column IS NOT NULL. The use pattern here was that records which required processing had this timestamp set to something other

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Pavan Deolasee
On Fri, Feb 24, 2017 at 3:23 PM, Robert Haas wrote: > > I don't immediately see how this will work with index-only scans. If > the tuple is HOT updated several times, HOT-pruned back to a single > version, and then the page is all-visible, the index entries are > guaranteed to agree with the rem

Re: [HACKERS] Proposal : Parallel Merge Join

2017-02-24 Thread Dilip Kumar
On Fri, Feb 24, 2017 at 3:04 PM, Amit Kapila wrote: Thanks for the comments. > What advantage do you see for considering such a path when the cost of > innerpath is more than cheapest_total_inner? Remember the more paths > we try to consider, the more time we spend in the planner. By any > cha

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 3:31 PM, Pavan Deolasee wrote: > On Fri, Feb 24, 2017 at 3:23 PM, Robert Haas wrote: >> I don't immediately see how this will work with index-only scans. If >> the tuple is HOT updated several times, HOT-pruned back to a single >> version, and then the page is all-visible

[HACKERS] Monitoring roles patch

2017-02-24 Thread Dave Page
Per the discussion at https://www.postgresql.org/message-id/CA%2BOCxoyYxO%2BJmzv2Micj4uAaQdAi6nq0w25BPQgLLxsrvTmREw%40mail.gmail.com, attached is a patch that implements the following: - Adds a default role called pg_monitor - Gives members of the pg_monitor role full access to: pg_ls_logdir()

Re: [HACKERS] UPDATE of partition key

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 3:24 PM, Simon Riggs wrote: > I'd give the view that we cannot silently ignore this issue, bearing > in mind the point that we're expecting partitioned tables to behave > exactly like normal tables. At the risk of repeating myself, I don't expect that, and I don't think it

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 1:04 PM, Craig Ringer wrote: > On 23 February 2017 at 22:20, Tom Lane wrote: >>> * Don't force/generate an alias at all. >> >>> I've no idea for this yet and Tom already was concerned what this might >>> break. There are several places in the transform phase where the >>>

Re: [HACKERS] Proposal : Parallel Merge Join

2017-02-24 Thread Amit Kapila
On Fri, Feb 24, 2017 at 3:42 PM, Dilip Kumar wrote: > On Fri, Feb 24, 2017 at 3:04 PM, Amit Kapila wrote: > > Thanks for the comments. > >> What advantage do you see for considering such a path when the cost of >> innerpath is more than cheapest_total_inner? Remember the more paths >> we try to

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Pavan Deolasee
On Fri, Feb 24, 2017 at 3:42 PM, Robert Haas wrote: > > > Wow, OK. In my view, that makes the chain conversion code pretty much > essential, because if you had WARM without chain conversion then the > visibility map gets more or less irrevocably less effective over time, > which sounds terrible.

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-02-24 Thread Simon Riggs
On 12 January 2017 at 13:34, Peter Eisentraut wrote: > On 1/11/17 5:27 AM, Simon Riggs wrote: >> The main area of "design doubt" remains the implementation of the >> recovery_target parameter set. Are we happy with the user interface >> choices in the patch, given the understanding that the situat

Re: [HACKERS] Proposal : Parallel Merge Join

2017-02-24 Thread Dilip Kumar
On Fri, Feb 24, 2017 at 3:04 PM, Amit Kapila wrote: > What advantage do you see for considering such a path when the cost of > innerpath is more than cheapest_total_inner? Remember the more paths > we try to consider, the more time we spend in the planner. By any > chance are you able to generat

Re: [HACKERS] Keep ECPG comment for log_min_duration_statement

2017-02-24 Thread Michael Meskes
> As I said in the first e-mail, there are some restrictions of comment > position in my implementation. I am concerned that an error will > occur in .pgc files users made in old version. > So, this feature should be a new option. I see, this makes sense. > When the pre-compiler(ECPG) converts C

Re: [HACKERS] increasing the default WAL segment size

2017-02-24 Thread Kuntal Ghosh
On Fri, Feb 24, 2017 at 12:47 PM, Beena Emerson wrote: > > Hello, > > The recent commit c29aff959dc64f7321062e7f33d8c6ec23db53d has again changed > the code and the second patch cannot be applied cleanly. Please find > attached the rebased 02 patch. 01 patch is the same . > I've done an initial r

Re: [HACKERS] safer node casting

2017-02-24 Thread Andres Freund
Hi, I was about to add a few more uses of castNode, which made me think. You proposed replacing: On 2016-12-31 12:08:22 -0500, Peter Eisentraut wrote: > There is a common coding pattern that goes like this: > > RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc); > Assert(IsA(rinfo, Restr

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-02-24 Thread Peter Moser
2017-02-22 19:43 GMT+01:00 Peter Eisentraut : > On 2/16/17 07:41, Robert Haas wrote: >> Also, it sounds like all of this is intended to work with ranges that >> are stored in different columns rather than with PostgreSQL's built-in >> range types. > > Yeah, that should certainly be changed. Our sy

Re: [HACKERS] FYI: git worktrees as replacement for "rsync the CVSROOT"

2017-02-24 Thread Andrew Dunstan
On 02/24/2017 02:36 AM, Craig Ringer wrote: > On 16 January 2017 at 05:01, Jim Nasby wrote: >> Not sure how many people still use [1], as referenced by our git wiki[2], >> but it appears git worktrees are a viable replacement for that technique. In >> short, if you're already in your checkout: >

Re: [HACKERS] bytea_output output of base64

2017-02-24 Thread Kenneth Marshall
On Thu, Feb 23, 2017 at 03:52:46PM -0800, David Fetter wrote: > On Thu, Feb 23, 2017 at 05:55:37PM -0500, Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: > > > Bruce Momjian writes: > > > > Is there a reason we don't support base64 as a bytea_output output > > >

Re: ParallelFinish-hook of FDW/CSP (Re: [HACKERS] Steps inside ExecEndGather)

2017-02-24 Thread Kohei KaiGai
Hello, This attached patch re-designed the previous v2 according to Robert's comment. All I had to do is putting entrypoint for ForeignScan/CustomScan at ExecShutdownNode, because it is already modified to call child node first, earlier than ExecShutdownGather which releases DSM segment. Thanks,

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Magnus Hagander
On Thu, Feb 23, 2017 at 10:37 PM, Bruce Momjian wrote: > On Sat, Jan 21, 2017 at 12:46:05PM -0500, Stephen Frost wrote: > > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: > > > As we don't know the performance impact is (there was no benchmark done > > > on reasonably current code base) I r

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-24 Thread Andrew Dunstan
On 02/23/2017 04:41 PM, Tom Lane wrote: > Andrew Dunstan writes: >> I'm not entirely sure how I should replace those DirectFunctionCall2 calls. > Basically what we want is for the called function to be able to use the > fcinfo->flinfo->fn_extra and fcinfo->flinfo->fn_mcxt fields of the > FmgrInf

Re: [HACKERS] UPDATE of partition key

2017-02-24 Thread David G. Johnston
On Friday, February 24, 2017, Simon Riggs wrote: > > 2. I know that DB2 handles this by having the user specify WITH ROW > MOVEMENT to explicitly indicate they accept the issue and want update > to work even with that. We could have an explicit option to allow > that. This appears to be the only w

Re: [HACKERS] utility commands benefiting from parallel plan

2017-02-24 Thread Dilip Kumar
On Fri, Feb 24, 2017 at 11:43 AM, Haribabu Kommi wrote: > Here I attached an implementation patch that allows > utility statements that have queries underneath such as > CREATE TABLE AS, CREATE MATERIALIZED VIEW > and REFRESH commands to benefit from parallel plan. > > These write operations not p

Re: [HACKERS] safer node casting

2017-02-24 Thread Tom Lane
Andres Freund writes: > Those aren't actually equivalent, because of the !nodeptr. IsA() crashes > for NULL pointers, but the new code won't. Which means 9ba8a9ce4548b et > al actually weakened some asserts. > Should we perhaps have one NULL accepting version (castNodeNull?) and > one that separa

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-24 Thread Tom Lane
Andrew Dunstan writes: > On 02/23/2017 04:41 PM, Tom Lane wrote: >> The reason this is kind of scary is that it's just blithely assuming >> that the function won't look at the *other* fields of the FmgrInfo. >> If it did, it would likely get very confused, since those fields >> would be describing

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 4:06 PM, Pavan Deolasee wrote: >> Wow, OK. In my view, that makes the chain conversion code pretty much >> essential, because if you had WARM without chain conversion then the >> visibility map gets more or less irrevocably less effective over time, >> which sounds terribl

Re: [HACKERS] safer node casting

2017-02-24 Thread Peter Eisentraut
On 2/24/17 10:54, Tom Lane wrote: > Andres Freund writes: >> Those aren't actually equivalent, because of the !nodeptr. IsA() crashes >> for NULL pointers, but the new code won't. Which means 9ba8a9ce4548b et >> al actually weakened some asserts. > >> Should we perhaps have one NULL accepting ver

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-24 Thread Tom Lane
Craig Ringer writes: > Personally I think we need to generate one, if nothing else for error > messages where we try to emit qualified names of columns. Also for EXPLAIN, where there has to be a way to name everything. > But I don't see that the name needs to be anything we can refer to > elsewh

Re: [HACKERS] utility commands benefiting from parallel plan

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 11:43 AM, Haribabu Kommi wrote: > Here I attached an implementation patch that allows > utility statements that have queries underneath such as > CREATE TABLE AS, CREATE MATERIALIZED VIEW > and REFRESH commands to benefit from parallel plan. > > These write operations not p

Re: ParallelFinish-hook of FDW/CSP (Re: [HACKERS] Steps inside ExecEndGather)

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 7:29 PM, Kohei KaiGai wrote: > This attached patch re-designed the previous v2 according to Robert's comment. > All I had to do is putting entrypoint for ForeignScan/CustomScan at > ExecShutdownNode, > because it is already modified to call child node first, earlier than >

Re: [HACKERS] FYI: git worktrees as replacement for "rsync the CVSROOT"

2017-02-24 Thread Tom Lane
Andrew Dunstan writes: > On 02/24/2017 02:36 AM, Craig Ringer wrote: >> On 16 January 2017 at 05:01, Jim Nasby wrote: >>> git worktree add ../9.6 REL9_6_STABLE > Does this do anythng different from the git contrib script > git-new-workdir that I have been using for quite a long while? I think i

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-24 Thread David Fetter
On Thu, Feb 23, 2017 at 01:27:29PM +, Greg Stark wrote: > On 22 February 2017 at 15:08, Tom Lane wrote: > > Indeed. When I wrote the comment you're referring to, quite a few years > > ago now, I thought that popular demand might force us to allow omitted > > aliases. But the demand never mat

Re: ParallelFinish-hook of FDW/CSP (Re: [HACKERS] Steps inside ExecEndGather)

2017-02-24 Thread Claudio Freire
On Fri, Feb 24, 2017 at 1:24 PM, Robert Haas wrote: > On Fri, Feb 24, 2017 at 7:29 PM, Kohei KaiGai wrote: >> This attached patch re-designed the previous v2 according to Robert's >> comment. >> All I had to do is putting entrypoint for ForeignScan/CustomScan at >> ExecShutdownNode, >> because i

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Bruce Momjian
On Fri, Feb 24, 2017 at 02:14:23PM +0530, Pavan Deolasee wrote: > > > On Thu, Feb 23, 2017 at 11:53 PM, Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 03:03:39PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > As I remember, WARM only allows > > > a single

Re: [HACKERS] Hash support for grouping sets

2017-02-24 Thread Andrew Gierth
> "Thom" == Thom Brown writes: Thom> This doesn't apply cleanly to latest master. Could you please Thom> post a rebased patch? Sure. -- Andrew (irc:RhodiumToad) diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index c9e0a3e..480a07e 100644 --- a/src/backend/

Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-24 Thread David G. Johnston
On Fri, Feb 24, 2017 at 9:35 AM, David Fetter wrote: > > > => SELECT "?column"? FROM (select 1+1 as "?column?", 1+1) AS x; > > ERROR: 42703: column "?column" does not exist > > LINE 2: SELECT "?column"? FROM (select 1+1 as "?column?", 1+1) AS x; > >^ > > HINT: Perhaps you meant

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 8:49 AM, Amit Kapila wrote: >> IIUC, I think that we need to have the number of half-dead pages in meta >> page. > > Don't you think we need to consider backward compatibility if we want > to do that? Yeah, that would be an on-disk format break. I think this thread is pr

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-24 Thread Joel Jacobson
On Thu, Feb 23, 2017 at 8:04 AM, Robert Haas wrote: > > It doesn't sound like a good solution to me, because there can be SQL > code inside stored procedures that clients never see. In our code base, we use CamelCase in all PL/pgSQL functions, both for columns and variables, e.g. SELECT UserID IN

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Bruce Momjian
On Sat, Jan 21, 2017 at 09:02:25PM +0200, Ants Aasma wrote: > > It might be worth looking into using the CRC CPU instruction to reduce this > > overhead, like we do for the WAL checksums. Since that is a different > > algorithm it would be a compatibility break and we would need to support the > >

Re: [HACKERS] Poor memory context performance in large hash joins

2017-02-24 Thread Jeff Janes
On Thu, Feb 23, 2017 at 2:28 PM, Tom Lane wrote: > Jeff Janes writes: > > The number of new chunks can be almost as as large as the number of old > > chunks, especially if there is a very popular value. The problem is that > > every time an old chunk is freed, the code in aset.c around line 968

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Tomas Vondra
On 02/24/2017 06:47 PM, Bruce Momjian wrote: On Sat, Jan 21, 2017 at 09:02:25PM +0200, Ants Aasma wrote: It might be worth looking into using the CRC CPU instruction to reduce this overhead, like we do for the WAL checksums. Since that is a different algorithm it would be a compatibility break a

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Ants Aasma
On Fri, Feb 24, 2017 at 7:47 PM, Bruce Momjian wrote: > On Sat, Jan 21, 2017 at 09:02:25PM +0200, Ants Aasma wrote: >> > It might be worth looking into using the CRC CPU instruction to reduce this >> > overhead, like we do for the WAL checksums. Since that is a different >> > algorithm it would be

Re: [HACKERS] Poor memory context performance in large hash joins

2017-02-24 Thread Jeff Janes
On Thu, Feb 23, 2017 at 10:47 PM, Andres Freund wrote: > On 2017-02-23 17:28:26 -0500, Tom Lane wrote: > > Jeff Janes writes: > > > The number of new chunks can be almost as as large as the number of old > > > chunks, especially if there is a very popular value. The problem is > that > > > ever

Re: [HACKERS] Poor memory context performance in large hash joins

2017-02-24 Thread Tom Lane
Jeff Janes writes: > On Thu, Feb 23, 2017 at 2:28 PM, Tom Lane wrote: >> Maybe it's time to convert that to a doubly-linked list. > I don't think that would help. You would need some heuristic to guess > whether the chunk you are looking for is near the front, or near the end. Uh, what? In a

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Bruce Momjian
On Fri, Feb 24, 2017 at 08:31:09PM +0200, Ants Aasma wrote: > >> We looked at that when picking the algorithm. At that point it seemed > >> that CRC CPU instructions were not universal enough to rely on them. > >> The algorithm we ended up on was designed to be fast on SIMD hardware. > >> Unfortuna

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Jim Nasby
On 2/24/17 12:30 PM, Tomas Vondra wrote: In any case, we can't just build x86-64 packages with compile-time SSE4.1 checks. Dumb question... since we're already discussing llvm for the executor, would that potentially be an option here? AIUI that also opens the possibility of using the GPU as

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-24 Thread Andrew Dunstan
On 02/24/2017 11:02 AM, Tom Lane wrote: > Andrew Dunstan writes: >> On 02/23/2017 04:41 PM, Tom Lane wrote: >>> The reason this is kind of scary is that it's just blithely assuming >>> that the function won't look at the *other* fields of the FmgrInfo. >>> If it did, it would likely get very con

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-24 Thread Jim Nasby
On 2/24/17 11:34 AM, Joel Jacobson wrote: SELECT SomeCol, OtherCol, FooCol, BarCol, MyCol, ExtraCol, LastCol INTO _SomeCol, _OtherCol, _FooCol, _BarCol, _MyCol, _ExtraCol, _LastCol FROM Foo WHERE Bar = 'Baz'; This is to avoid typos that are then visually easy to spot, thanks to a

Re: [HACKERS] Poor memory context performance in large hash joins

2017-02-24 Thread Jeff Janes
On Fri, Feb 24, 2017 at 10:59 AM, Tom Lane wrote: > Jeff Janes writes: > > On Thu, Feb 23, 2017 at 2:28 PM, Tom Lane wrote: > >> Maybe it's time to convert that to a doubly-linked list. > > > I don't think that would help. You would need some heuristic to guess > > whether the chunk you are lo

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Bruce Momjian
On Fri, Feb 24, 2017 at 01:49:07PM -0600, Jim Nasby wrote: > On 2/24/17 12:30 PM, Tomas Vondra wrote: > >In any case, we can't just build x86-64 packages with compile-time > >SSE4.1 checks. > > Dumb question... since we're already discussing llvm for the executor, would > that potentially be an op

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-24 Thread Jim Nasby
On 2/24/17 12:28 AM, Robert Haas wrote: On Thu, Feb 23, 2017 at 6:59 PM, Tom Lane wrote: I think these are straw-man arguments, really. Consider the actual use case for such a feature: it's for porting some application that was not written against Postgres to begin with. I'm not sure that's t

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Ants Aasma
On Fri, Feb 24, 2017 at 10:02 PM, Bruce Momjian wrote: > Uh, as far as I know, the best you are going to get from llvm is > standard assembly, while the SSE4.1 instructions use special assembly > instructions, so they would be faster, and in a way they are a GPU built > into CPUs. Both LLVM and G

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Ants Aasma
On Fri, Feb 24, 2017 at 9:37 PM, Bruce Momjian wrote: > Oh, that's why we will hopefully eventually change the page checksum > algorithm to use the special CRC32 instruction, and set a new checksum > version --- got it. I assume there is currently no compile-time way to > do this. Using CRC32 as

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-24 Thread Jim Nasby
On 2/24/17 11:26 AM, Robert Haas wrote: I think we need to come up with some set of tests to figure out what actually works well in practice here. Theories are a good starting point, but good vacuum behavior is really important, and a patch that changes it ought to be backed up by at least some

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Ants Aasma
On Fri, Feb 24, 2017 at 9:49 PM, Jim Nasby wrote: > On 2/24/17 12:30 PM, Tomas Vondra wrote: >> >> In any case, we can't just build x86-64 packages with compile-time >> SSE4.1 checks. > > > Dumb question... since we're already discussing llvm for the executor, would > that potentially be an option

Re: [HACKERS] FYI: git worktrees as replacement for "rsync the CVSROOT"

2017-02-24 Thread Jim Nasby
On 2/24/17 10:24 AM, Tom Lane wrote: Andrew Dunstan writes: On 02/24/2017 02:36 AM, Craig Ringer wrote: On 16 January 2017 at 05:01, Jim Nasby wrote: git worktree add ../9.6 REL9_6_STABLE Does this do anythng different from the git contrib script git-new-workdir that I have been using for

Re: [HACKERS] bytea_output output of base64

2017-02-24 Thread Jim Nasby
On 2/24/17 7:44 AM, Kenneth Marshall wrote: Like David suggests, if you want compact, run it through lz4/gzip/lzop...for a much better size return. Speaking of which; any bytea where you care about this is likely to live in an already compressed state in toast. ISTM it would be valuable if we

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-02-24 Thread Jim Nasby
On 2/24/17 6:40 AM, Peter Moser wrote: Do you think it is better to remove the syntax for ranges expressed in different columns? It's not that hard to construct a range type on-the-fly from 2 columns, so (without having looked at the patch or really followed the thread) I would think the answ

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-24 Thread Gavin Flower
On 25/02/17 09:02, Jim Nasby wrote: On 2/24/17 12:28 AM, Robert Haas wrote: On Thu, Feb 23, 2017 at 6:59 PM, Tom Lane wrote: I think these are straw-man arguments, really. Consider the actual use case for such a feature: it's for porting some application that was not written against Postgres

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Bruce Momjian
On Fri, Feb 24, 2017 at 10:09:50PM +0200, Ants Aasma wrote: > On Fri, Feb 24, 2017 at 9:37 PM, Bruce Momjian wrote: > > Oh, that's why we will hopefully eventually change the page checksum > > algorithm to use the special CRC32 instruction, and set a new checksum > > version --- got it. I assume

Re: [HACKERS] WIP: About CMake v2

2017-02-24 Thread Bruce Momjian
On Wed, Feb 8, 2017 at 04:52:19PM -0500, Tom Lane wrote: > Peter Eisentraut writes: > > On 2/8/17 6:21 AM, Yuriy Zhuravlev wrote: > >> Support two build systems it's not big deal really. I have been working > >> on this past year without any big troubles. > >> Also we have second perl build syst

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Ants Aasma
On Fri, Feb 24, 2017 at 10:30 PM, Bruce Momjian wrote: > On Fri, Feb 24, 2017 at 10:09:50PM +0200, Ants Aasma wrote: >> On Fri, Feb 24, 2017 at 9:37 PM, Bruce Momjian wrote: >> > Oh, that's why we will hopefully eventually change the page checksum >> > algorithm to use the special CRC32 instructi

[HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-02-24 Thread Pavel Stehule
Hi 2017-02-23 12:17 GMT+01:00 Pavel Stehule : > Hi > > Currently is not possible to control sort columns for \d* commands. > Usually schema and table name is used. Really often task is collect the > most big objects in database. "\dt+, \di+" shows necessary information, but > not in practical ord

Re: [HACKERS] snapbuild woes

2017-02-24 Thread Petr Jelinek
On 22/02/17 03:05, Petr Jelinek wrote: > On 13/12/16 00:38, Petr Jelinek wrote: >> On 12/12/16 23:33, Andres Freund wrote: >>> On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: On 12/12/16 22:42, Andres Freund wrote: > Hi, > > On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: >

Re: [HACKERS] PATCH: two slab-like memory allocators

2017-02-24 Thread Andres Freund
Hi, On 2017-02-21 01:43:46 +0100, Tomas Vondra wrote: > Attached is v9 of this patch series. This addresses most of the points > raised in the review, namely: Cool, thanks. > 3) get rid of the block-level bitmap tracking free chunks > > Instead of the bitmap, I've used a simple singly-linked li

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-24 Thread Peter Geoghegan
On Fri, Feb 24, 2017 at 9:26 AM, Robert Haas wrote: > I think this thread is pretty short on evidence that would let us make > a smart decision about what to do here. I see three possibilities. > The first is that this patch is a good idea whether we do something > about the issue of half-dead pa

Re: [HACKERS] btree_gin and btree_gist for enums

2017-02-24 Thread Andrew Dunstan
On 02/24/2017 02:55 PM, Andrew Dunstan wrote: > > On 02/24/2017 11:02 AM, Tom Lane wrote: >>> I don't know what to call it either. In my test I used >>> CallerContextFunctionCall2 - not sure if that's quite right, but should >>> be close. >> CallerInfo? CallerFInfo? Or we could spell out Caller

Re: [HACKERS] Poor memory context performance in large hash joins

2017-02-24 Thread Tom Lane
Jeff Janes writes: > On Fri, Feb 24, 2017 at 10:59 AM, Tom Lane wrote: >> Uh, what? In a doubly-linked list, you can remove an element in O(1) >> time, you don't need any searching. > Currently it is walking the chain to identify which block holds the chunk > in the first place, not just to get

Re: [HACKERS] Logical replication existing data copy

2017-02-24 Thread Erik Rijkers
On 2017-02-24 22:58, Petr Jelinek wrote: On 23/02/17 01:41, Petr Jelinek wrote: On 23/02/17 01:02, Erik Rijkers wrote: On 2017-02-22 18:13, Erik Rijkers wrote: On 2017-02-22 14:48, Erik Rijkers wrote: On 2017-02-22 13:03, Petr Jelinek wrote: 0001-Skip-unnecessary-snapshot-builds.patch 0002-

Re: [HACKERS] Logical replication existing data copy

2017-02-24 Thread Petr Jelinek
On 25/02/17 00:05, Erik Rijkers wrote: > On 2017-02-24 22:58, Petr Jelinek wrote: >> On 23/02/17 01:41, Petr Jelinek wrote: >>> On 23/02/17 01:02, Erik Rijkers wrote: On 2017-02-22 18:13, Erik Rijkers wrote: > On 2017-02-22 14:48, Erik Rijkers wrote: >> On 2017-02-22 13:03, Petr Jeline

Re: [HACKERS] Poor memory context performance in large hash joins

2017-02-24 Thread Andres Freund
On 2017-02-24 18:04:18 -0500, Tom Lane wrote: > Concretely, something like the attached. This passes regression tests > but I've not pushed on it any harder than that. Heh, I'd just gotten something that didn't immediately crash anymore ;) Running your patch against Jeff's test-case, verified be

Re: [HACKERS] Poor memory context performance in large hash joins

2017-02-24 Thread Andres Freund
On 2017-02-24 15:12:37 -0800, Andres Freund wrote: > On 2017-02-24 18:04:18 -0500, Tom Lane wrote: > > Concretely, something like the attached. This passes regression tests > > but I've not pushed on it any harder than that. > > Heh, I'd just gotten something that didn't immediately crash anymore

Re: [HACKERS] Logical replication existing data copy

2017-02-24 Thread Erik Rijkers
On 2017-02-25 00:08, Petr Jelinek wrote: There is now a lot of fixes for existing code that this patch depends on. Hopefully some of the fixes get committed soonish. Indeed - could you look over the below list of 8 patches; is it correct and in the right (apply) order? 0001-Use-asynchronous

Re: [HACKERS] Logical replication existing data copy

2017-02-24 Thread Petr Jelinek
On 25/02/17 00:39, Erik Rijkers wrote: > On 2017-02-25 00:08, Petr Jelinek wrote: >> >> There is now a lot of fixes for existing code that this patch depends >> on. Hopefully some of the fixes get committed soonish. > > Indeed - could you look over the below list of 8 patches; is it correct > and

[HACKERS] Oblivious nested loop join algorithm

2017-02-24 Thread Amin Fallahi
Hi all Is the current implementation of nested loop join in postgres (memory-trace) oblivious? Why or why not?

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-24 Thread Amit Kapila
On Sat, Feb 25, 2017 at 3:40 AM, Peter Geoghegan wrote: > On Fri, Feb 24, 2017 at 9:26 AM, Robert Haas wrote: >> I think this thread is pretty short on evidence that would let us make >> a smart decision about what to do here. I see three possibilities. >> The first is that this patch is a good

Re: ParallelFinish-hook of FDW/CSP (Re: [HACKERS] Steps inside ExecEndGather)

2017-02-24 Thread Kohei KaiGai
2017-02-25 1:40 GMT+09:00 Claudio Freire : > On Fri, Feb 24, 2017 at 1:24 PM, Robert Haas wrote: >> On Fri, Feb 24, 2017 at 7:29 PM, Kohei KaiGai wrote: >>> This attached patch re-designed the previous v2 according to Robert's >>> comment. >>> All I had to do is putting entrypoint for ForeignSca

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Pavan Deolasee
On Fri, Feb 24, 2017 at 9:47 PM, Robert Haas wrote: > > And there are no bugs, right? :-) Yeah yeah absolutely nothing. Just like any other feature committed to Postgres so far ;-) I need to polish the chain conversion patch a bit and also add missing support for redo, hash indexes etc. Suppo

Re: [HACKERS] Proposal : Parallel Merge Join

2017-02-24 Thread Amit Kapila
On Fri, Feb 24, 2017 at 4:49 PM, Dilip Kumar wrote: > On Fri, Feb 24, 2017 at 3:04 PM, Amit Kapila wrote: >> What advantage do you see for considering such a path when the cost of >> innerpath is more than cheapest_total_inner? Remember the more paths >> we try to consider, the more time we spen