Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-03-11 Thread Peter Geoghegan
th the left-right keyspace checks, which don't have relation ExclusiveLock protection (IOW, I think that that might be buggy). I need to think about that some more, but current thinking is that it would hardly matter if we used the highkey from right page rather than the first data item,

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-03-11 Thread Peter Geoghegan
On Fri, Mar 11, 2016 at 1:09 PM, Peter Geoghegan wrote: > Or, you could add code like this to comparetup_index_btree(), to > simulate a broken opclass: > > diff --git a/src/backend/utils/sort/tuplesort.c > b/src/backend/utils/sort/tuplesort.c > index 67d86ed..23712ff 100644 &

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-03-11 Thread Peter Geoghegan
return compare; There are many options when you want to produce a corrupt B-Tree index! -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] checkpointer continuous flushing - V18

2016-03-10 Thread Peter Geoghegan
e documentation. I don't know what can be done about that. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-03-10 Thread Peter Geoghegan
ue *after* an error happens, in SSLerrmessage(). """ The problem with this, as Heikki goes on to say, it that we might not get to that point in SSLerrmessage(); we may not be able to pop the queue/call ERR_get_error(), more or less by accident (e.g. I noticed an OOM could do

Re: [HACKERS] Using quicksort for every external sort run

2016-03-10 Thread Peter Geoghegan
On Sun, Feb 14, 2016 at 8:01 PM, Peter Geoghegan wrote: > The query I'm testing is: "reindex index pgbench_accounts_pkey;" > > Now, with a maintenance_work_mem of 5MB, the most recent revision of > my patch takes about 54.2 seconds to complete this, as compared to

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-03-10 Thread Peter Geoghegan
for the benefit of others. Is this informed by a concern about some specific downside to taking that extra precaution? Thanks -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-03-10 Thread Peter Geoghegan
On Thu, Mar 10, 2016 at 3:09 PM, Peter Geoghegan wrote: > Getting to it very soon. Just really busy right this moment. That said, I agree with Peter's remarks about doing this frontend and backend. So, while I'm not sure, I think we're in agreement on all issues. I would ha

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-03-10 Thread Peter Geoghegan
urned with Feedback in the CF and move on, which seems a bit of a > shame since this appears to be a bona fide bug, but if nobody's > willing to work on it, it ain't gettin' fixed. Getting to it very soon. Just really busy right this moment. -- Peter Geoghegan -- Sent vi

Re: [HACKERS] WIP: Upper planner pathification

2016-03-10 Thread Peter Geoghegan
y-paste coding. > > +1. Frustrating Citus's attempt to open-source their stuff is not in > the project's interest. I agree. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Using quicksort for every external sort run

2016-03-10 Thread Peter Geoghegan
sn't really a concern anymore. I think we may be able to comprehensively address Robert's concerns about regressions with very little work_mem and lots of data by fixing a problem with polyphase merge. More to come soon. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] Using quicksort for every external sort run

2016-03-10 Thread Peter Geoghegan
update coming. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Using quicksort for every external sort run

2016-03-07 Thread Peter Geoghegan
hin ~5% of an *internal* sort with the patch series applied, even though ~80% of time is spent copying and sorting SortTuples overall in the internal sort case (the internal case cannot overlap sorting and aggregate processing, since it has no final merge step). This is a nice demonstration of ho

Re: [HACKERS] Freeze avoidance of very large table.

2016-03-07 Thread Peter Geoghegan
ls apart for some reason, I'll update this. I > plan to push this RSN if nobody objects. Thanks for making the effort to make the tool generally available. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

[HACKERS] Minor bug affecting ON CONFLICT lock wait log messages

2016-03-07 Thread Peter Geoghegan
. -- Peter Geoghegan From bc481af77994057cb1ffe4a0e471b38bb00dc228 Mon Sep 17 00:00:00 2001 From: Peter Geoghegan Date: Mon, 7 Mar 2016 13:16:24 -0800 Subject: [PATCH] Avoid incorrectly indicating exclusion constraint wait INSERT ... ON CONFLICT's precheck may have to wait on the outcome of an

Re: [HACKERS] ExecGather() + nworkers

2016-03-07 Thread Peter Geoghegan
w useful is it in the general case? I'm not suggesting that it isn't, but I'm not sure. How common is it for the leader process to do anything other than coordinate and consume from worker processes? -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] WIP: Upper planner pathification

2016-03-06 Thread Peter Geoghegan
vrkkasy1xbshgzxkd6-hnxx3gq7x-p-dz0zt...@mail.gmail.com In summary, I think it's surprising that a max_parallel_degree of 1 doesn't disable parallel workers entirely. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] The plan for FDW-based sharding

2016-03-06 Thread Peter Geoghegan
o get agreement about something abstract, like the general idea of a distributed transaction manager. I fear that any particular detailed interpretation of what that phrase means will be very hard to get accepted into PostgreSQL. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Freeze avoidance of very large table.

2016-03-02 Thread Peter Geoghegan
do is restrict the functions to throw an error when called by non-superusers, out of caution. It's a problem that modules like pg_stat_statements and pg_freespacemap are currently lumped together in the documentation, but we all know that. -- Peter Geoghegan -- Sent via pgsql-hackers mailin

Re: Commitfest Bug (was: [HACKERS] Re: Reusing abbreviated keys during second pass of ordered [set] aggregates)

2016-03-02 Thread Peter Geoghegan
On Wed, Mar 2, 2016 at 5:41 AM, Magnus Hagander wrote: > Ok, I've pushed a code that does that. Thank you. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Commitfest Bug (was: [HACKERS] Re: Reusing abbreviated keys during second pass of ordered [set] aggregates)

2016-03-01 Thread Peter Geoghegan
been on both sides of this, as a patch author and patch reviewer. If the patch was left as "Waiting on Author", as my review of Alexander's patch was, then it ought to not change to "Needs Review" silently. That makes absolutely no sense. -- Peter Geoghegan -- Sent vi

[HACKERS] amcheck (B-Tree integrity checking tool)

2016-02-29 Thread Peter Geoghegan
ss time than having to REINDEX the index, and doesn't need too much memory. I think that in practice most problems that can be detected by the B-Tree checker functions will be detected with the lighter variant. -- Peter Geoghegan From 573810d8d3c994ce1a16ecffb2f5d208c0ff93e3 Mon Sep 17 00:00:00

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

2016-02-19 Thread Peter Geoghegan
new > index pages, but I don't think that's an interesting case.) Sure. I'm talking about a narrow issue around how things are represented in pgstatindex() only. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to yo

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

2016-02-19 Thread Peter Geoghegan
uld happen. I meant where pages could accumulate without being recycled. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

2016-02-18 Thread Peter Geoghegan
t-link/left-link chains, and there are usage patterns where half-dead pages might accumulate. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

2016-02-18 Thread Peter Geoghegan
ngs like this because I care about making sure these tools are useful for teaching novice hackers about Postgres internals. I've made my point and will leave it at that, though. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Remove or weaken hints about "effective resolution of sleep delays is 10 ms"?

2016-02-18 Thread Peter Geoghegan
dance for commit_delay's effective resolution from "29.4. WAL Configuration" to where commit_delay is introduced, "18.5. Write Ahead Log". -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

2016-02-18 Thread Peter Geoghegan
r example by updating the descriptions of the fields avg_leaf_density and leaf_fragmentation in the docs. Just a thought. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

2016-02-18 Thread Peter Geoghegan
I vote back patch. Subtle differences between the branches should be avoided. -- Peter Geoghegan

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

2016-02-18 Thread Peter Geoghegan
t; possible to see zero or multiple roots because the root's location > changes. That's already mentioned in the documentation, if somewhat > obliquely. Ah, yes. Another consequence of going in physical order. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

2016-02-18 Thread Peter Geoghegan
─┼┼┼─── 206 │81 │ 3 │160 │42 (1 row) BTW, I am actively working on the amcheck B-Tree checker tool, and hope to post it soon. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] [DOCS] The number of bytes is stored in index_size of pgstatindex() ?

2016-02-18 Thread Peter Geoghegan
attuple and pageinspect going into this kind of detail in their documentation generally. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-02-17 Thread Peter Geoghegan
. Obviously that's the really interesting baseline. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: Reusing abbreviated keys during second pass of ordered [set] aggregates

2016-02-17 Thread Peter Geoghegan
On Wed, Feb 17, 2016 at 2:12 AM, Robert Haas wrote: > Committed, except I left out one comment hunk that I wasn't convinced about. Thank you. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscripti

Re: [HACKERS] Bug with int2

2016-02-16 Thread Peter Geoghegan
,768 (1 row) -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: Reusing abbreviated keys during second pass of ordered [set] aggregates

2016-02-15 Thread Peter Geoghegan
st time in a long time that the CF wasn't closed because the next CF began. :-) -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PoC: Partial sort

2016-02-15 Thread Peter Geoghegan
fixed soon, so moving a patch to the next commitfest no longer clobbers its state. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Remove or weaken hints about "effective resolution of sleep delays is 10 ms"?

2016-02-15 Thread Peter Geoghegan
On Mon, Feb 15, 2016 at 11:23 AM, Peter Geoghegan wrote: > commit_delay doesn't have any guidance like this, where it could > certainly matter, because optimal settings are rarely greater than 10 > milliseconds. Actually, it does, but it's in "29.4. WAL Configu

Re: [HACKERS] Remove or weaken hints about "effective resolution of sleep delays is 10 ms"?

2016-02-15 Thread Peter Geoghegan
ould certainly matter, because optimal settings are rarely greater than 10 milliseconds. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: Reusing abbreviated keys during second pass of ordered [set] aggregates

2016-02-15 Thread Peter Geoghegan
On Tue, Dec 22, 2015 at 10:49 PM, Peter Geoghegan wrote: > I attach a rebased patch for 9.6 only. I marked the patch -- my own patch -- "Ready for Committer". I'm the third person to have marked the patch "Ready for Committer", even though no committer bounced the

Re: [HACKERS] Using quicksort for every external sort run

2016-02-14 Thread Peter Geoghegan
On Sun, Feb 7, 2016 at 4:50 PM, Peter Geoghegan wrote: >> I'm not even sure this is necessary. The idea of missing out on >> producing a single sorted run sounds bad but in practice since we >> normally do the final merge on the fly there doesn't seem like there's

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Peter Geoghegan
ssues you noticed'. I would really like for there to be a way to do that more often. It would be a significant time saver, because it removes problems with minor bitrot. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subsc

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Peter Geoghegan
y try to do that. That's all. Thank you (and Amit) for working really hard on parallelism. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] enable parallel query by default?

2016-02-08 Thread Peter Geoghegan
(it was eventually replaced with a #define before release, though). -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Peter Geoghegan
shout at me. If I persisted, you wouldn't commit my patch, and for me that probably means it's DOA. I don't think I'm asking for much here. [1] http://www.postgresql.org/message-id/CAM3SWZSefE4uQk3r_3gwpfDWWtT3P51SceVsL4=g8v_me2a...@mail.gmail.com [2] http://www.postgresql.org/

Re: [HACKERS] Recently added typedef "string" is a horrid idea

2016-02-08 Thread Peter Geoghegan
On Sun, Feb 7, 2016 at 7:47 AM, Tom Lane wrote: > Works for me. Attached patch is what I came up with. It required only minimal additional changes for consistency. -- Peter Geoghegan From 01d8cb278cecb995ecc30cda0125d10c98f4d05c Mon Sep 17 00:00:00 2001 From: Peter Geoghegan Date: Sun, 7

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Peter Geoghegan
atch patch series for complicated stuff these days, but it's important to make it clear how everything fits together. That's actually what I was thinking about when I said we need to be clear on how things fit together from the CF app patch page, because there doesn't seem to be a cult

Re: [HACKERS] Using quicksort for every external sort run

2016-02-07 Thread Peter Geoghegan
On Sun, Feb 7, 2016 at 4:50 PM, Peter Geoghegan wrote: >> I'm not even sure this is necessary. The idea of missing out on >> producing a single sorted run sounds bad but in practice since we >> normally do the final merge on the fly there doesn't seem like there's

Re: [HACKERS] Using quicksort for every external sort run

2016-02-07 Thread Peter Geoghegan
ee > tapes when outputing the final results. There will be the same amount > of I/O happening and a 2-way or 3-way merge for most data types should > be basically free. I basically agree with you, but it seems possible to fix the regression (generally misguided though those regressed cases ar

Re: [HACKERS] Using quicksort for every external sort run

2016-02-07 Thread Peter Geoghegan
sn't matter. "Juggling capacity" has nothing to do with modern hardware characteristics, except that modern machines are where the cost of excessive "juggling capacity" really hurts, so this is simple. It is simple *especially* because we can throw out the idea of a cost model that

Re: [HACKERS] Recently added typedef "string" is a horrid idea

2016-02-07 Thread Peter Geoghegan
able types generally. So, a rename to "VarString" probably makes sense, independent of your pgindent concern. If this sounds like a good idea, I'll produce a patch soon. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Generalizing SortSupport for text to work with char(n), bytea, and alternative opclasses

2016-02-05 Thread Peter Geoghegan
On Wed, Feb 3, 2016 at 11:31 AM, Robert Haas wrote: > Thanks for the review. I fixed the OID conflict, tweaked a few > comments, and committed this. Thanks. I noticed a tiny, preexisting typo in the string abbreviated key code. The attached patch fixes it. -- Peter Geoghegan diff --git

[HACKERS] Fix for OpenSSL error queue bug

2016-02-05 Thread Peter Geoghegan
.@wrigleys.postgresql.org#20150224030956.2529.83...@wrigleys.postgresql.org -- Peter Geoghegan From d5433bc562f792afd75ef8664729db9e6a60ee62 Mon Sep 17 00:00:00 2001 From: Peter Geoghegan Date: Tue, 26 Jan 2016 15:11:15 -0800 Subject: [PATCH] Distrust external OpenSSL clients; clear err queue OpenSSL has an unfor

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2016-02-04 Thread Peter Geoghegan
y, this is really cool work; I have often thought that we don't have nearly enough people thinking about how to optimize B-Tree indexing. It is hard, but so is anything worthwhile. That's all I have for now. Just a quick review focused on code and correctness (and not on the benefits).

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2016-02-04 Thread Peter Geoghegan
nch_accounts and reviewing the size of that. Right. Because, apart from everything else, unique indexes are not currently supported. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2016-02-04 Thread Peter Geoghegan
992 │0.000 │ 99.643 % (3 rows) But this looks healthy -- I see the same with master. And since the accounts table is listed as 1281 MB, this looks like a plausible ratio in the size of the table to its primary index (which I would not say is true of an 87MB primary key index). Are you sure

Re: [HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

2016-02-04 Thread Peter Geoghegan
hat wasn't very important. But that's a problem that I'm closer to, and I won't express an opinion on this patch. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Raising the checkpoint_timeout limit

2016-02-04 Thread Peter Geoghegan
de a bad decision, there will usually be a chain of faulty but not outrageous reasoning behind it. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Using quicksort for every external sort run

2016-02-04 Thread Peter Geoghegan
On Thu, Feb 4, 2016 at 1:46 AM, Peter Geoghegan wrote: > It wasn't my original insight that replacement selection has become > all but obsolete. It took me a while to come around to that point of > view. Nyberg et al may have said it best in 1994, in the Alphasort Paper [1]:

Re: [HACKERS] Using quicksort for every external sort run

2016-02-04 Thread Peter Geoghegan
t adds no reliance on any estimate; it's fully self-tuning. Fine, but the point of "quicksort with spillover" is that it avoids I/O entirely. I'm not promoting it as useful for any of the reasons that replacement selection was traditionally useful (on 1970s hardware). So, we aren't much closer to working out a better cost model for "quicksort with spillover" (I guess you weren't really talking about that, though), an annoying sticking point (as already mentioned). -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Generalizing SortSupport for text to work with char(n), bytea, and alternative opclasses

2016-02-04 Thread Peter Geoghegan
On Sun, Jan 31, 2016 at 7:59 PM, Andreas Karlsson wrote: > Nice work, I like your sorting patches. Thanks. I like your reviews of my sorting patches. :-) -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

Re: [HACKERS] Using quicksort for every external sort run

2016-01-29 Thread Peter Geoghegan
hase merges. I'm not sure that that's practical. Currently, tuplesort decides on a number of tapes ahead of time. When we're constrained on those, the stretch multiplier would apply, but I think that that could be invasive because the number of tapes ("merge order" + 1) was a function of non-stretched work_mem. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Using quicksort for every external sort run

2016-01-29 Thread Peter Geoghegan
ed at 50% (bad for "quicksort with spillover"), 25% (better), and 5% (good). An alternative approach that might be acceptable is to add a generic, conservative 90% threshold (so 10% of tuples sorted by heap). -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] Using quicksort for every external sort run

2016-01-29 Thread Peter Geoghegan
ult to interpret. > I think for long keys both old (RS) and new (Qsort) sort method has its own > characteristics > based on data distribution. I think work_mem is the key If properly set new > method(Qsort) will > be able to fit most of the cases. If work_mem is not tune

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2016-01-28 Thread Peter Geoghegan
e all the > values are the same, but perhaps that's due to my lack of understanding > regarding the way they're being stored. I think that you'd need an I/O bound workload to see significant benefits. That seems unsurprising. I believe that random I/O from index writes is a bi

Re: [HACKERS] New committer

2016-01-28 Thread Peter Geoghegan
On Thu, Jan 28, 2016 at 6:37 AM, Magnus Hagander wrote: > The PostgreSQL core team would like to welcome Dean Rasheed as a new > committer for the PostgreSQL project. Congratulations, Dean. Well deserved. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] Using quicksort for every external sort run

2016-01-27 Thread Peter Geoghegan
icksort with spillover" cost model. Hopefully this can be cleared up soon. As I've said, I am very receptive to other people's suggestions about how that should work. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PoC: Partial sort

2016-01-23 Thread Peter Geoghegan
nterested in seeing Alexander get back to this because it's an important project on its own. We should really have this. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Releasing in September

2016-01-20 Thread Peter Geoghegan
reason why introducing it should be attached to sqlsmith's inclusion in core. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Releasing in September

2016-01-20 Thread Peter Geoghegan
that you mentioned. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Releasing in September

2016-01-20 Thread Peter Geoghegan
o you really think that people need to be able to commit something before seriously working on it? Rules that are expressly coercive are a *bad* idea. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postg

Re: [HACKERS] Releasing in September

2016-01-20 Thread Peter Geoghegan
cking WAL replay by comparing master and slave disk contents. We > should make an effort to get that into a state where anyone can use it. I think that that was initially Jeff Janes. Jeff also provided great testing infrastructure for UPSERT, something that I was very grateful for. -- Peter Geoghe

Re: [HACKERS] Releasing in September

2016-01-20 Thread Peter Geoghegan
se, to the surprise of many, the number of bugs in each point release (or their severity) did not really taper off. There was a particular qualitative nature to the bugs. This seems to be because the design was excessively complicated. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Releasing in September

2016-01-20 Thread Peter Geoghegan
.3. I don't think that one of those lessons was "take more time to release". There is reason to doubt that that would have changed matters one bit with 9.3. It might be a good idea to formally state what those lessons are. -- Peter Geoghegan -- Sent via pgsql-hackers maili

Re: [HACKERS] Releasing in September

2016-01-20 Thread Peter Geoghegan
developer meeting. I am not on the release list. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Extending the HyperLogLog API a bit

2016-01-19 Thread Peter Geoghegan
On Tue, Jan 19, 2016 at 2:22 PM, Peter Geoghegan wrote: > I don't have time right now to do a full analysis of whether or not it's > possible to just fix the bug without doing all that, but I think it > might not be. IOW: I think that Hideaki's bug fix might itself be wr

Re: [HACKERS] PATCH: Extending the HyperLogLog API a bit

2016-01-19 Thread Peter Geoghegan
t complete implementations seem to switch from sparse to full at a certain point). So, ripping out mergeHyperLogLog() doesn't really make that effort any more difficult. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Extending the HyperLogLog API a bit

2016-01-19 Thread Peter Geoghegan
esentation. The distinction is explained in the README for postgresql-hll: https://github.com/aggregateknowledge/postgresql-hll postgresql-hll has no support for merging HLLs that are sparse: https://github.com/aggregateknowledge/postgresql-hll/blob/master/hll.c#L1888 Can't we just tear

Re: [HACKERS] [PATCH] Improve spinlock inline assembly for x86.

2016-01-18 Thread Peter Geoghegan
e, let's let it drop. If we don't want to apply this, then I think that a sensible compromise would be to add a code comment that says that we don't believe the LOCK prefix matters. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] [PATCH] Improve spinlock inline assembly for x86.

2016-01-18 Thread Peter Geoghegan
emoving superfluous code; doing so sometimes clarifies intent and understanding. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Interesting read on SCM upending software and hardware architecture

2016-01-18 Thread Peter Geoghegan
15/ I think it's much more true that main memory scaling (in particular, main memory capacity) has had a huge impact, but that trend appears to now be stalling. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-01-18 Thread Peter Geoghegan
; I don't -- I just think you could have sent a patch that addressed all > the other points, leave this one as initially submitted, and note that > the new submission left it unaddressed because you disagreed. I'll try to do that soon. I've got a very busy schedule over the

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-01-18 Thread Peter Geoghegan
other committer wants to engage with me on this, then I will of course work with them. But that will not be predicated on my first revising the patch in a way that this other committer does not understand. That would be profoundly unfair. -- Peter Geoghegan -- Sent via pgsql-hackers mailin

Re: [HACKERS] Proposal: speeding up GIN build with parallel workers

2016-01-17 Thread Peter Geoghegan
and nbtree at some point, though. I think that there are some useful general consequences for performance that come from consolidation. For example, my ongoing work on external sorting makes it use much of the same infrastructure as internal sorting. Now, external sorts automatically benefit from optimi

Re: [HACKERS] Proposal: speeding up GIN build with parallel workers

2016-01-15 Thread Peter Geoghegan
aving a clear path forward to performing the build in parallel. I understand that a long term ambition for the gin am is to merge it with nbtree, to almost automatically benefit from enhancements, and to reduce the maintenance burden of each. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list

Re: [HACKERS] pgindent-polluted commits

2016-01-13 Thread Peter Geoghegan
s approach seems to work nicely, without any downside that I've noticed. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ExecGather() + nworkers

2016-01-11 Thread Peter Geoghegan
gt;> happen seems messy -- you actually have the postmaster launch >> processes, but they must immediately get permission to do anything. >> >> It wouldn't be that hard to offer a general way of doing this, so why not? > > Well, if these things become actual problems,

Re: [HACKERS] Spelling corrections

2016-01-11 Thread Peter Geoghegan
On Sun, Jan 10, 2016 at 2:17 AM, David Rowley wrote: > Snap! > http://www.postgresql.org/message-id/CAKJS1f-AGgQaurTwhY=wkJjspgDcmDUE8Yx03XTXCDz=kae...@mail.gmail.com Great minds think alike. :-) -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] 2016-01 Commitfest

2016-01-11 Thread Peter Geoghegan
On Mon, Jan 11, 2016 at 6:38 AM, Alvaro Herrera wrote: > * Partial sort That shouldn't have been in "needs review" state. Fixed. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postg

Re: [HACKERS] ExecGather() + nworkers

2016-01-10 Thread Peter Geoghegan
On Sun, Jan 10, 2016 at 1:44 PM, Peter Geoghegan wrote: > With parallel sequential scan, a max_parallel_degree of 8 could result > in 16 processes scanning in parallel. I meant a max_worker_processes setting, which of course is different. Nevertheless, I find it surprisin

Re: [HACKERS] ExecGather() + nworkers

2016-01-10 Thread Peter Geoghegan
to have the workers just exit if you decide you don't want them, > and I don't really want to make the signaling here more complicated > than it really needs to be. I worry about the additional overhead of constantly starting and stopping a single worker in some cases (not so much with pa

[HACKERS] Spelling corrections

2016-01-09 Thread Peter Geoghegan
Attached patch fixes a couple of misspellings. -- Peter Geoghegan From 1c6149c0ed347d7514104a11bc3b47e1a9a5724a Mon Sep 17 00:00:00 2001 From: Peter Geoghegan Date: Sat, 9 Jan 2016 22:19:49 -0800 Subject: [PATCH] Fix a couple of misspellings --- src/backend/executor/nodeGather.c | 2 +- src

[HACKERS] ExecGather() + nworkers

2016-01-09 Thread Peter Geoghegan
way, without the postmaster having actually launched anything, and without custom "call it all off" code for parallel index builds. This would probably involve introducing a distinction between a BackgroundWorkerSlot being "reserved" rather than "in_use", lest the postm

Re: [HACKERS] WIP: bloom filter in Hash Joins with batches

2016-01-09 Thread Peter Geoghegan
ings, like a hash join outer relation's attribute. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: bloom filter in Hash Joins with batches

2016-01-09 Thread Peter Geoghegan
On Sat, Jan 9, 2016 at 4:08 PM, Peter Geoghegan wrote: > Also, have you considered Hash join conditions with multiple > attributes as a special case? I'm thinking of cases like this: Sorry, accidentally fat-fingered my enter key before I was finished drafting that mail. That example i

Re: [HACKERS] WIP: bloom filter in Hash Joins with batches

2016-01-09 Thread Peter Geoghegan
e is a bit faster on my machine). -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fwd: Re: [DOCS] Document Upper Limit for NAMEDATELEN in pgsql 9.5+

2016-01-08 Thread Peter Geoghegan
complexity.) */ if (m > MAX_LEVENSHTEIN_STRLEN || n > MAX_LEVENSHTEIN_STRLEN) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("argument exceeds the maximum length of %d bytes", MAX_LEVENSHTEIN_STR

Re: [HACKERS] Generalizing SortSupport for text to work with char(n), bytea, and alternative opclasses

2016-01-07 Thread Peter Geoghegan
;t feel too strongly about holding on to the "churn" in this patch. I attach a new revision, which has the changes I outlined to code comments. I haven't minimized the differences in the way you suggest just yet. -- Peter Geoghegan From b18d45437e99657fed990edf718c21ad6d212970 Mon Sep 17

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-06 Thread Peter Geoghegan
work on myself, which excludes most items on the TODO list. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Broken lock management in policy.c.

2016-01-04 Thread Peter Geoghegan
pushed for a worked out example myself, but there are only so many hours in the day. I've reviewed your changes, and have nothing further to add. Thanks -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.

<    5   6   7   8   9   10   11   12   13   14   >