Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-02 Thread Peter Geoghegan
mited to hitting this bug (when Valgrind memcheck is used)? Are you satisfied that I have adequately described steps to reproduce? -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
it for Robert to comment on what I've said before acting. We should be more ambitious about adding test coverage to tuplesort.c. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
scussion with Tom. > I don't know, either. You read both Robert and me indicating that this bug > fix will be better with a test, and nobody has said that a test-free fix is > optimal. Here's your chance to obliterate that no-tests precedent. I'm happy that I can do at least

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
x27;t add tests! There is scarcely any example of anyone deliberately adding test coverage to that file. I don't understand why it is so. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] _mdfd_getseg can be expensive

2016-07-01 Thread Peter Geoghegan
ck). Those two cases were not exactly comparable in perhaps several other ways, but even still my sense is that that this can be at least partially explained by md.c bottlenecks. This is something that we'll need to confirm through profiling. Hopefully it's just this one bottleneck.

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
On Fri, Jul 1, 2016 at 12:30 PM, Peter Geoghegan wrote: > Checkout my gensort tool from github. Build the C tool with "make". > Then, from the working directory: > > ./postgres_load.py -m 250 --skew --logged > psql -c "CREATE INDEX segfaults on sort_test_skew

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
ess needs to occur following any movement of the tuple. Whereas, in all other cases there is no need to do something similar, as it happens. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
mory. Clearly that needs to be updated if the tuple is moved, since it always points to an offset into the same tuple. I thought that that explanation was simple. [1] https://github.com/petergeoghegan/gensort -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] _mdfd_getseg can be expensive

2016-06-30 Thread Peter Geoghegan
involved or difficult question, then no need to answer now. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] _mdfd_getseg can be expensive

2016-06-30 Thread Peter Geoghegan
master. Also some comment policing. Are you planning to pick this up again, Andres? -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-06-26 Thread Peter Geoghegan
tuples using memmove()), because their particular variety of caller tuple happens to itself contain a pointer to palloc()'d memory. Attached patch fixes this use-after-free bug. -- Peter Geoghegan From aa68ed9e3f952bc19aa3bc8e31d0e216e13e0fae Mon Sep 17 00:00:00 2001 From: Peter Geoghegan

Re: [HACKERS] parallel workers and client encoding

2016-06-26 Thread Peter Geoghegan
is to be least important (except for the > documentation update, but that's pending further from Peter Geoghegan > about exactly what he thinks needs to be changed). I got sidetracked on that, in part due to investigating a bug in the 9.6 external sort work. I'll have more informa

Re: [HACKERS] Improving executor performance

2016-06-24 Thread Peter Geoghegan
27;s really hard to prove. It may be that it doesn't matter there, because working on fixing that yields obvious benefits. But, as you suggest, we should consider that that might not always be true. That's really tricky, and has historically been the kind of thing we've managed very badly. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] tuplesort.c's copytup_index() is dead code

2016-06-24 Thread Peter Geoghegan
t_putdatum(), and now tuplesort_putindextuplevalues()). I believe that this has value. All the extra boilerplate code misleads. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] tuplesort.c's copytup_index() is dead code

2016-06-24 Thread Peter Geoghegan
rtainly not opposed to that. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] tuplesort.c's copytup_index() is dead code

2016-06-23 Thread Peter Geoghegan
#x27;m well aware of that issue. This is the same reason why we don't have any regression test coverage of external sorts. I don't think that that's good enough. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] tuplesort.c's copytup_index() is dead code

2016-06-23 Thread Peter Geoghegan
indexes. I broke hash index tuplesort builds in a similar way at one point, too. The slightest bit of regression test coverage would have caught either bug, I believe. I think that some minimal regression tests should be added, because evidently they are needed. -- Peter Geoghegan -- Sent via pgsql-

[HACKERS] tuplesort.c's copytup_index() is dead code

2016-06-23 Thread Peter Geoghegan
milar considerations around memory management). -- Peter Geoghegan From 8b2c0d3b1844caaf144971f488201e806c7fcc5e Mon Sep 17 00:00:00 2001 From: Peter Geoghegan Date: Thu, 23 Jun 2016 18:51:37 -0700 Subject: [PATCH] Remove dead COPYTUP routine in tuplesort.c Commit 9f03ca915 made copytup_index() dead

Re: [HACKERS] Gin index on array of uuid

2016-06-21 Thread Peter Geoghegan
nd for this. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-20 Thread Peter Geoghegan
like work_mem and temp file space. This obviously > isn't ideal, but it's what could be done by the ship date. Where would that be documented, though? Would it need to be noted in the case of each such GUC? -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] Rename max_parallel_degree?

2016-06-09 Thread Peter Geoghegan
lity statements. It's just clear that it's not appropriate to use max_parallel_workers_per_gather within utility statements, even though something like that will be needed. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

Re: [HACKERS] Rename max_parallel_degree?

2016-06-09 Thread Peter Geoghegan
9.6. I am in favor of having something similar to max_parallel_workers_per_gather for utility statements like CREATE INDEX. That will need a cost model, at least where the DBA isn't explicit about the number of workers to use. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-07 Thread Peter Geoghegan
xamine the issue more closely. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reviewing freeze map code

2016-06-06 Thread Peter Geoghegan
ability is a design goal that I'd be willing to give up other things for. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-05 Thread Peter Geoghegan
On Wed, May 18, 2016 at 12:01 PM, Peter Geoghegan wrote: >> I think for 9.6 we just have to document this issue. In the next >> release, we could (and might well want to) try to do something more >> clever. > > Works for me. You may wish to update comments within fd.c

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
so disregard max_parallel_degree if you so choose. My guess is that DBAs will frequently want to do so. I'm not attached to any of this, but that's what I've come up with to date. Possibly the index storage parameter concept has baggage elsewhere, which comes up when we later go to pa

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
; available slots. That never occurred to me. I'd say it could be a serious problem, that should be fixed promptly. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
cores to I use? Well, it depends on the planner, of course. But when constrained only by the availability of worker processes, then your example could use 3 cores -- the 2 remaining parallel workers, plus the leader itself. > Presumably the leader isn't counted towards max_parallel_workers

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
actually corresponds to > leader + (N-1) workers internally. > > I think we can hope that developers are going to be less confused about that > than users. +1. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 11:09 AM, Peter Geoghegan wrote: > The only reason I favor defining parallel_degree = 1 I meant "redefining max_parallel_degree =1 to effectively disable parallel query", of course. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
and Oracle (as long as we match those semantics). Also, I think that number of cores used is a more important consideration for users than the number of workers used. Users will definitely be confused about workers used vs. cores used, but I don't think that any proposal fixes that. --

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
ut the number of cores being the primary consideration. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
. Similarly, parallel aggregate does not really have much for the leader process to do other than act as a worker. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
ate in parallel query does need to be user-visible. Worker processes are a commodity (i.e. the user must consider max_worker_processes). -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ORDER/GROUP BY expression not found in targetlist

2016-05-25 Thread Peter Geoghegan
; which appears to be a 9.6 regression, presumable fallout from the path > restructuring. It's surprising that SQL Smith didn't catch something with such simple steps to reproduce. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Is the unfair lwlock behavior intended?

2016-05-24 Thread Peter Geoghegan
ition, was essential for something like LWLocks. The paper made me aware that it's perfectly acceptable practice to make informed trade-offs around fairness for what it calls "latches" (what we'd call LWLocks). Maybe that was obvious to you, but it wasn't to

Re: [HACKERS] Is the unfair lwlock behavior intended?

2016-05-24 Thread Peter Geoghegan
/gray/papers/Convoy%20Phenomenon%20RJ%202516.pdf I could believe that there's a case to be made for per-LWLock fairness characteristics, which may be roughly what Andres meant. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] BTREE_BUILD_STATS build is broken

2016-05-23 Thread Peter Geoghegan
are initialized there) seems a little contrived. A single entry point for nbtsort.c works better. Short term, I guess the best solution is to just have a pgrminclude directive in both files. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] BTREE_BUILD_STATS build is broken

2016-05-23 Thread Peter Geoghegan
On Mon, May 23, 2016 at 4:41 PM, Tom Lane wrote: > Pushed, thanks. Thanks. If BTREE_BUILD_STATS needs a "tcopprot.h pgrminclude ignore" within nbtree.c, then ISTM that the similar include directive within nbtsort.c ought to receive the same treatment. -- Peter Geoghegan -- S

Re: [HACKERS] Re: [BUGS] BUG #14153: Unrecognized node type error when upsert is present in recursive CTE

2016-05-23 Thread Peter Geoghegan
; The important thing is to have at least one buildfarm animal using it, > which there soon will be. I manually enable it sometimes. And, I've pointed out bugs that that would have caught to other hackers during review a couple of times. That's my view. I'm not going to make a fuss

Re: [HACKERS] Re: [BUGS] BUG #14153: Unrecognized node type error when upsert is present in recursive CTE

2016-05-23 Thread Peter Geoghegan
dfarm animal or two to add the new #define, which is not exactly > a huge cost. You also have to be aware of the new thing, which a lot of hackers will not be, if awareness of COPY_PARSE_PLAN_TREES is anything to go by. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hacker

[HACKERS] BTREE_BUILD_STATS build is broken

2016-05-23 Thread Peter Geoghegan
The attached patch fixes the BTREE_BUILD_STATS build. Looks like 65c5fcd353a859da9e61bfb2b92a99f12937de3b broke this. That commit was made back in January, so no backpatch is required. -- Peter Geoghegan From 06d2150ff20dfa3e6fbc579a9e41ff3f6487 Mon Sep 17 00:00:00 2001 From: Peter Geoghegan

Re: [HACKERS] Re: [BUGS] BUG #14153: Unrecognized node type error when upsert is present in recursive CTE

2016-05-23 Thread Peter Geoghegan
is what > to use to enable the debug test. I propose "#ifdef COPY_PARSE_PLAN_TREES" > since that enables similar sanity checking for other parts of > backend/nodes/, and we do have at least one buildfarm member using it. That's what I was thinking, too. No need to keep it separate. -- Pe

Re: [HACKERS] Re: [BUGS] BUG #14153: Unrecognized node type error when upsert is present in recursive CTE

2016-05-23 Thread Peter Geoghegan
x27;t think it would be difficult to find a choke point for that. Perhaps there is some subtlety I've missed, since what I propose seems too obvious. FWIW, I don't think it would much matter if this debug code was not reliably executed for every possible SelectStmt. Just limiting it to top

Re: [HACKERS] Re: [BUGS] BUG #14153: Unrecognized node type error when upsert is present in recursive CTE

2016-05-23 Thread Peter Geoghegan
cycles. A noticeable omission has a cost: it gets noticed. Given this code path is likely to hardly ever be hit, this mechanical approach seemed best. That's all. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subs

[HACKERS] Re: [BUGS] BUG #14153: Unrecognized node type error when upsert is present in recursive CTE

2016-05-23 Thread Peter Geoghegan
On Sat, May 21, 2016 at 5:03 PM, Peter Geoghegan wrote: > On Sat, May 21, 2016 at 4:28 PM, wrote: >> ERROR: XX000: unrecognized node type: 920 >> LOCATION: raw_expression_tree_walker, nodeFuncs.c:3410 >> >> I expected the query run successfully and return one row wi

Re: [HACKERS] memory layouts for binary search in nbtree

2016-05-19 Thread Peter Geoghegan
ox. Obviously this will be slower than an equivalent plain pgbench SELECT, but the difference may be smaller than you expect. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] memory layouts for binary search in nbtree

2016-05-19 Thread Peter Geoghegan
will be very difficult without better testing. This idea also enables complementary techniques, like interpolation search that can degrade to binary search. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.

Re: [HACKERS] [PATCH] Add EXPLAIN (ALL) shorthand

2016-05-19 Thread Peter Geoghegan
Non-boolean options would be problematic as well. While the spelling needs work, I like the general idea. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reviewing freeze map code

2016-05-18 Thread Peter Geoghegan
erification would be a somewhat generic operation, with AM-specific code invoked as appropriate. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reviewing freeze map code

2016-05-18 Thread Peter Geoghegan
On Wed, May 18, 2016 at 8:52 AM, Jeff Janes wrote: > How about going with something that says more about why we are doing > it, rather than trying to describe in one or two words what it is > doing? > > VACUUM (FORENSIC) > > VACUUM (DEBUG) > > VACUUM (LINT) +1 --

Re: [HACKERS] Parallel query and temp_file_limit

2016-05-18 Thread Peter Geoghegan
; clever. Works for me. You may wish to update comments within fd.c at the same time. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel query and temp_file_limit

2016-05-17 Thread Peter Geoghegan
On Tue, May 17, 2016 at 3:33 PM, Peter Geoghegan wrote: > Fundamentally, since temporary_files_size enforcement simply > piggy-backs on low-level fd.c file management, without any > consideration of what the temp files contain, it'll be hard to be sure > that parallel workers wil

Re: [HACKERS] Parallel query and temp_file_limit

2016-05-17 Thread Peter Geoghegan
piggy-backs on low-level fd.c file management, without any consideration of what the temp files contain, it'll be hard to be sure that parallel workers will not have issues. I think it'll be far easier to fix the problem then it would be to figure out if it's possible to get away

[HACKERS] Parallel query and temp_file_limit

2016-05-17 Thread Peter Geoghegan
ives no special treatment for parallel query. Enforcement of temp_file_limit will therefore not work *per-session*. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-05-06 Thread Peter Geoghegan
On Fri, May 6, 2016 at 9:12 PM, Peter Eisentraut wrote: > All committed. Thanks! This should no longer be referenced in the 9.6 release notes. It should just appear in the next batch of point releases. Tom has an sgml comment in the draft 9.6 release notes to that effect. -- Peter Geoghe

[HACKERS] "Allow usage of huge maintenance_work_mem for GIN build" patch

2016-05-06 Thread Peter Geoghegan
ay let allocatedMemory go negative for a period, which would now be broken. If you did make this exact error, you would not be the first. If it isn't actually broken, perhaps you should still make this change, simply on general principle. I'd like to hear other opinions on that,

Re: [HACKERS] Reviewing freeze map code

2016-05-06 Thread Peter Geoghegan
uld be even better if Jeff's approach to testing was followed as an example by other people, but I wouldn't bet on it ever happening. It requires real persistence and deep understanding to do well. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Initial release notes created for 9.6

2016-05-05 Thread Peter Geoghegan
On Thu, May 5, 2016 at 7:42 PM, Tom Lane wrote: > Peter Geoghegan writes: >> I think that there could stand to be some consolidation among the >> items that I authored. > > After thinking a bit, I merged all the abbreviated-keys stuff including > the ordered-set-aggrega

Re: [HACKERS] Initial release notes created for 9.6

2016-05-05 Thread Peter Geoghegan
ature as follows: """ Improve the speed of sorting of varchar, text, and numeric fields via "abbreviated" keys (Peter Geoghegan, Andrew Gierth, Robert Haas) """ Users only cared that there was a nice optimization added to sorting that affects only the type

Re: [HACKERS] Initial release notes created for 9.6

2016-05-05 Thread Peter Geoghegan
seems to be a lack of urgency about it, and given that it's moderately complicated, that tends to mean it will keep getting put off. I did notice that you have an sgml comment about it, but the wording isn't optimistic. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] what to revert

2016-05-04 Thread Peter Geoghegan
can shed some light. I focused exclusively on the correctness of the core mechanism. It simply didn't occur to me to check for problems of the type you describe, that affect the system even when the feature is disabled. I didn't check because Kevin is a committer, that I assumed wouldn&#

Re: [HACKERS] what to revert

2016-05-03 Thread Peter Geoghegan
this attitude to reverting patches. It should not be seen as a big personal failure, because it isn't. Stigmatizing reverts incentivizes behavior that leads to bad outcomes. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] New 9.6 external sort guidance around temp_tablespaces and maintenance_work_mem

2016-04-30 Thread Peter Geoghegan
On Sat, Apr 30, 2016 at 4:26 PM, Bruce Momjian wrote: > On Sat, Apr 30, 2016 at 04:23:00PM -0700, Peter Geoghegan wrote: >> Maybe there is a place to emphasize this change in the release notes. >> I don't really want to make it about the external sort feature, >> thoug

Re: [HACKERS] New 9.6 external sort guidance around temp_tablespaces and maintenance_work_mem

2016-04-30 Thread Peter Geoghegan
does not disadvantage external sorts is as much about enabling HashAggregates as it is about enabling internal sorts. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New 9.6 external sort guidance around temp_tablespaces and maintenance_work_mem

2016-04-30 Thread Peter Geoghegan
g the full benefit of the hardware). Getting this right could allow unsharded Postgres to go a lot further (parallelism is the final part of this), which could make all the difference for some users. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: Covering + unique indexes.

2016-04-27 Thread Peter Geoghegan
changing that, since I think that you have commensurate skill. But, in order to be successful in these projects, patience and resolve are very important. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Removing faulty hyperLogLog merge function

2016-04-26 Thread Peter Geoghegan
gt; instead of removing on your say-so. I don't think that that's an efficient use of anyone's time. If somebody proposes a patch with functionality that needs to merge HLL states, then they can add an implementation at that time. -- Peter Geoghegan -- Sent via pgsql-hackers maili

Re: [HACKERS] Removing faulty hyperLogLog merge function

2016-04-26 Thread Peter Geoghegan
On Tue, Apr 26, 2016 at 4:41 PM, Robert Haas wrote: > I'm not prepared to commit this over the objection offered by Tomas > Vondra on that thread. You don't want to remove buggy code that is currently unused simply because it might be useful to have that functionality in the fu

[HACKERS] Removing faulty hyperLogLog merge function

2016-04-25 Thread Peter Geoghegan
The function hyperLogLogMerge() is faulty [1]. It has no current callers, though. I propose that we rip it out, as in the attached patch. [1] http://www.postgresql.org/message-id/CAM3SWZT-i6R9JU5YXa8MJUou2_r3LfGJZpQ9tYa1BYxfkj0=c...@mail.gmail.com -- Peter Geoghegan From

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-04-25 Thread Peter Geoghegan
xec-over-ssl-connection-can-fail-on There are numerous problem reports like this that are easily found using Google. I think that there are probably a variety of unpleasant interactions and symptoms. Crashes are one rarer symptom, seen in certain scenarios only (crashes are not described in the link ab

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-04-25 Thread Peter Geoghegan
On Fri, Apr 8, 2016 at 11:43 AM, Peter Geoghegan wrote: > I'll do so soon. I was waiting on Peter E to take me up on the offer. Attached is a series of patches for each supported release branch. As discussed, I would like to target every released version of Postgres with this bugfix.

Re: [HACKERS] Rename max_parallel_degree?

2016-04-25 Thread Peter Geoghegan
my complaint is entirely about the baggage that the term "degree of parallelism" happens to have, and how effectively that has been managed. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rename max_parallel_degree?

2016-04-25 Thread Peter Geoghegan
istent with that of other major systems. Since the GUC's behavior isn't going to change now, the terminology should change. I'm fine with that. I'm not particularly concerned with the specifics of some new terminology, as long as it's consistent with the idea of auxiliary

Re: [HACKERS] Ordering in guc.c vs. config.sgml vs. postgresql.sample.conf

2016-04-25 Thread Peter Geoghegan
In short, the existing ordering isn't really so arbitrary. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rename max_parallel_degree?

2016-04-23 Thread Peter Geoghegan
And so, parallelism *will* still be used on Postgres. That's a pretty significant difference. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_dump dump catalog ACLs

2016-04-22 Thread Peter Geoghegan
ll keep things tolerable for the 1000-database > case, but the performance regression remains jarring. I think we should not > release 9.6 with pg_dump performance as it stands today. As someone that is responsible for many such clusters, I strongly agree. -- Peter Geoghegan --

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-22 Thread Peter Geoghegan
t it again with fresh eyes, but offhand having no safe interlock for the well established RecordFreeIndexPage() call for GIN seems like an implausibly obvious bug. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] New 9.6 external sort guidance around temp_tablespaces and maintenance_work_mem

2016-04-21 Thread Peter Geoghegan
ing work_mem/maintenance_work_mem may fail to improve performance only because sorts then become more I/O bound. When in doubt, testing is advised. A balance may need to be found, if only to avoid wasting memory. Thoughts? [1] https://queue.acm.org/detail.cfm?id=1563874 -- Peter Geogh

Re: [HACKERS] Avoid parallel full and right join paths.

2016-04-20 Thread Peter Geoghegan
get. Those went into amcheck's regression tests, and so have yet to be committed. I think that your approach there has plenty to recommend it, at least where requirements are more complicated. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Avoid parallel full and right join paths.

2016-04-20 Thread Peter Geoghegan
On Wed, Apr 20, 2016 at 2:49 PM, Robert Haas wrote: > Committed. But I think the regression test needs more thought, so I > left that out. It would be nice if there was a fuzz testing infrastructure that verified that parallel plans produce the same answer as serial plans. -- Peter Geo

Re: [HACKERS] Postgres 9.6 scariest patch tournament

2016-04-19 Thread Peter Geoghegan
not critically important if it gets to that). Rarely, the message will be "I'm deeply concerned about the *lasting* repercussions of having merged this patch". And so, yes, I think that we might want to be clearer about looking for nuances like that. [1] http://www.scottaaronson.co

Re: [HACKERS] Postgres 9.6 scariest patch tournament

2016-04-19 Thread Peter Geoghegan
ident I am in my prediction, and how scary in absolute terms I consider the scariest patches to be. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Draft release notes for next week's releases

2016-04-14 Thread Peter Geoghegan
On Thu, Apr 14, 2016 at 4:42 PM, Peter Geoghegan wrote: > * We should look into using the ucol_nextSortKeyPart() API: > > http://userguide.icu-project.org/collation/architecture#TOC-Partial-sort-keys Another more rich API we could immediately put to good use is the ICU strcoll() var

Re: [HACKERS] Draft release notes for next week's releases

2016-04-14 Thread Peter Geoghegan
pecial strxfrm() (which only actually needs 8 bytes for abbreviated keys) a lot faster. I'd be interested to see how your Russian text example does with this extra optimization. We should not be surprised that this kind of support exists within ICU, because abbreviated keys are actually quite an o

[HACKERS] Obsolete comment within fmgr.c

2016-04-14 Thread Peter Geoghegan
Attached patch removes obsolete comment from fmgr.c. The code the comment refers to was already removed by commit 5ea86e6e65dd2da3e9a3464484985d48328e7fe3. -- Peter Geoghegan remove_obsolete_comment.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] SEGFAULT in CREATE EXTENSION related pg_init_privs

2016-04-14 Thread Peter Geoghegan
aptuple.c:881 The memory appears to be freed -- CLOBBER_FREED_MEMORY stuff does this. You might try using Valgrind, with --track-origins, which will show where the memory was first allocated. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

2016-04-13 Thread Peter Geoghegan
bout that being "small enough to fit in L2 cache". I'm pretty sure that that's still true at least one time over with the latest Raspberry Pi model, so it should be revisited. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Incomplete startup packet errors

2016-04-13 Thread Peter Geoghegan
On Wed, Apr 13, 2016 at 1:02 AM, Magnus Hagander wrote: > It's fairly common to see a lot of "Incomplete startup packet" in the > logfiles caused by monitoring or healthcheck connections. I've also seen it caused by port scanning. -- Peter Geoghegan -- Sent via

Re: [HACKERS] Removing the TRACE_SORT macro

2016-04-11 Thread Peter Geoghegan
of the TRACE_SORT crud. That's all. I'll put it on my long-term TODO list. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Removing the TRACE_SORT macro

2016-04-11 Thread Peter Geoghegan
to log_executor_stats, which is categorized as "Statistics / Monitoring". I guess it isn't worth changing now, though. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Removing the TRACE_SORT macro

2016-04-11 Thread Peter Geoghegan
the associated crud. Just having that as an option suggests that there is some possible upside to disabling trace_sort, which is clearly not true. I will write a patch doing this if there are no objections. I think this is justifiable as clean-up for 9.6. -- Peter Geoghegan -- Sent via pgsql-ha

Re: [HACKERS] [COMMITTERS] pgsql: Bloom index contrib module

2016-04-09 Thread Peter Geoghegan
modules > have rather little test coverage, so they're poor as benchmarks. +1. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] snapshot too old, configured by time

2016-04-08 Thread Peter Geoghegan
t from this patch. I am reminded of this blog post, written by a friend and former co-worker: https://brandur.org/postgres-queues -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PoC: Partial sort

2016-04-08 Thread Peter Geoghegan
should be moved to the next CF, unless a committer wants to pick it up today. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2016 at 11:42 AM, Tom Lane wrote: > Seems like a reasonable thing to do, but somebody would have to do the > legwork to produce back-branch patches. I'll do so soon. I was waiting on Peter E to take me up on the offer. -- Peter Geoghegan -- Sent via pgsql-hackers m

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2016 at 11:20 AM, Peter Geoghegan wrote: > That seems reasonable. I'm glad we finally got this done. Thanks. Are we going to backpatch this? -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscripti

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-04-08 Thread Peter Geoghegan
djusted this so that we check for n<0 after SSL_read and > SSL_write, as you had in the frontend code but not in the backend code. That seems reasonable. I'm glad we finally got this done. Thanks. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] Using quicksort for every external sort run

2016-04-07 Thread Peter Geoghegan
ers to put all this together in more detail, which I believe has value. It suggests that increasing work_mem or maintenance_work_mem can have almost no downside provided you don't need that memory for anything else, which is true. I will be glad to see this through. Thanks for your help with this, Rober

Re: [HACKERS] Fix for OpenSSL error queue bug

2016-04-07 Thread Peter Geoghegan
tead of just 0? Because the variable is declared as a raw unsigned long, IIRC. It hardly matters. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

<    3   4   5   6   7   8   9   10   11   12   >