Re: [HACKERS] Parallel Hash take II

2017-11-15 Thread Andres Freund
ads are very complicated, and always having users target the > worst case leaves a lot to be desired. Obviously that's nice and worthwhile goal, but it seems more than a bit out of reach for this patchset. Greetings, Andres Freund

Re: [HACKERS] Parallel Hash take II

2017-11-15 Thread Andres Freund
On 2017-11-15 08:37:11 -0500, Robert Haas wrote: > On Tue, Nov 14, 2017 at 4:24 PM, Andres Freund <and...@anarazel.de> wrote: > >> I agree, and I am interested in that subject. In the meantime, I > >> think it'd be pretty unfair if parallel-oblivious hash join and >

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-15 Thread Andres Freund
if the same result would present if the benchmarks were run with parallel_leader_participation. The theory being what were seing is just that the leader doesn't accept any tuples, and the large queue size just helps because workers can run for longer. Greetings, Andres Freund

Re: pgindent run?

2017-11-28 Thread Andres Freund
Hi, On 2017-11-28 14:51:06 -0500, Robert Haas wrote: > If nobody minds too much, I'd like to update typedefs.list and > pgindent the whole tree again. +1. Greetings, Andres Freund

Re: [HACKERS] Small improvement to compactify_tuples

2017-11-28 Thread Andres Freund
till want to have this after the change? These should be just specializations of the template version imo. Greetings, Andres Freund

Re: Combine function returning NULL unhandled?

2017-11-23 Thread Andres Freund
On 2017-11-21 15:51:59 -0500, Robert Haas wrote: > On Mon, Nov 20, 2017 at 10:36 PM, Andres Freund <and...@anarazel.de> wrote: > > The plain transition case contains: > > if (pergroupst

Re: [PATCH] using arc4random for strong randomness matters.

2017-11-22 Thread Andres Freund
* family, RC4 has > predictability issues), there is no need of a file descriptor. I don't really see much benefit in those properties for postgres specifically. Not needing an fd is nice for cases where you're not guaranteed to have access to a filesystem, but postgres isn't going to work in those cases anyway. Greetings, Andres Freund

Re: [PATCH] using arc4random for strong randomness matters.

2017-11-22 Thread Andres Freund
On November 22, 2017 8:51:07 AM PST, ilm...@ilmari.org wrote: >If what is wanted is something more like /dev/urandom, one can call >getentropy(2) (or on Linux, getrandom(2)) directly, which avoids having >to open the device file each time. What does that buy us for our usages? Andres -- Sent

Expression based grouping equality implementation

2017-11-29 Thread Andres Freund
from [1] here as the patches do conflict. Greetings, Andres Freund [1] https://www.postgresql.org/message-id/20171128003121.nmxbm2ounxzb6...@alap3.anarazel.de >From 3ff1371aae4615c15f6bcdc232170a3a295e8021 Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Fri, 24 No

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Andres Freund
On November 29, 2017 8:50:31 AM PST, Stephen Frost wrote: >As for conflicting snapshots, isn't the lock we're taking already >AccessExclusive..? Doesn't help if e.g. the current xact is repeatable read or if your own xact deleted things (other xacts with snapshots could

Re: [PATCH] Atomic pgrename on Windows

2017-11-28 Thread Andres Freund
to extend WAL to include any temp file names, > etc, needed to allow us to recover from an interrupted operation during > redo. Frankly that may be the safest option; on Windows presume that there > is no atomic rename we can trust for critical files, and do it in multiple > steps. I'd rather remove windows support than go there. Greetings, Andres Freund

Re: [HACKERS] static assertions in C++

2017-11-29 Thread Andres Freund
On 2017-11-29 16:39:14 -0500, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-11-29 09:41:15 -0500, Robert Haas wrote: > >> +/* not worth providing a workaround */ > > > FWIW, I think that's a perfectly reasonable choice. Adding c

Re: [HACKERS] Issues with logical replication

2017-11-29 Thread Andres Freund
implementing the > XactLockTableInsert() instead of calling it. Right. But we fairly trivially can change that. I'm remarking on it because other people's, not yours, suggestions aimed at making this bulletproof. I just wanted to make clear that I don't think that's necessary at all. Greetings, Andres Freund

Re: [HACKERS] Issues with logical replication

2017-11-29 Thread Andres Freund
og a new xl_running_xacts record where it'd be helpful, without having * to write for bgwriter or checkpointer. Greetings, Andres Freund

Re: [HACKERS] Parallel Hash take II

2017-11-29 Thread Andres Freund
On 2017-11-30 14:17:51 +1300, Thomas Munro wrote: > On Mon, Nov 27, 2017 at 10:25 PM, Thomas Munro > wrote: > > On Thu, Nov 23, 2017 at 12:36 AM, Thomas Munro > > wrote: > >> Here's a new patch set with responses to the last batch of

Re: [PATCH] Fix crash in int8_avg_combine().

2017-11-26 Thread Andres Freund
ew callers should be interested in this, but tuplesort/tuplestore need > * to know it. > */ > -#define ALLOCSET_SEPARATE_THRESHOLD 8192 > +#define ALLOCSET_SEPARATE_THRESHOLD 16384 Huh, what's that about in this context? Greetings, Andres Freund

Expression based aggregate transition / combine function invocation

2017-11-27 Thread Andres Freund
. Greetings, Andres Freund >From 3ff1371aae4615c15f6bcdc232170a3a295e8021 Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Fri, 24 Nov 2017 14:14:35 -0800 Subject: [PATCH 1/3] Simplify representation of aggregate transition values a bit. Previously aggregate transiti

Re: [PATCH] Atomic pgrename on Windows

2017-11-27 Thread Andres Freund
all files FILE_SHARE_DELETE take care of this? See https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx "Note Delete access allows both delete and rename operations." Is there an external process active that doesn't set that flag? Are we missing setting it somewhere? Greetings, Andres Freund

Re: Would a BGW need shmem_access or database_connection to enumerate databases?

2017-11-29 Thread Andres Freund
On 2017-11-29 18:23:40 -0500, Chapman Flack wrote: > On 11/29/2017 05:54 PM, Michael Paquier wrote: > > > Yes. That's actually what the autovacuum launcher does. It connects > > using InitPostgres(NULL, InvalidOid, NULL, NULL), and then scans > > pg_database to fetch a list (see

Re: simplehash: tb->sizemask = 0

2017-11-29 Thread Andres Freund
sly wrong. Not sure how that happened. I might have had it as a shift at first? > (Why are those assertions, anyway, and not test-and-elog? > I do not think an assertion failure is a suitable way to > report "hash table full".) There's a test and elog during insert. Adding actual branches into SH_NEXT/SH_PREV seems like a bad idea. Will test a fix. Greetings, Andres Freund

Re: [HACKERS] static assertions in C++

2017-11-29 Thread Andres Freund
nor I seem to be convinced about: > > +/* not worth providing a workaround */ FWIW, I think that's a perfectly reasonable choice. Adding complications in making static assertions work for random archaic compilers when compiling with c++ just doesn't seem worth more than a few mins of thought. Greetings, Andres Freund

Re: [HACKERS] Parallel Hash take II

2017-12-01 Thread Andres Freund
small and there's normally not goign to be that many huge tuples (otherwise things are slow anyway). - why are we using a separate hardcoded 32 for sts names? Why not just go for NAMEDATALEN or such? - I'd replace most of the "should's" in comments with "need". Greetings, Andres Freund

Re: Protect syscache from bloating with negative cache entries

2017-12-16 Thread Andres Freund
by that problem? By bumping the generation on hash resize, we have recency information we can take into account. Greetings, Andres Freund

Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

2017-12-13 Thread Andres Freund
m. It'd be way less work to package abstime as an extension if you want to do so. Greetings, Andres Freund

Re: Reproducible builds: genbki.pl vs schemapg.h

2017-12-15 Thread Andres Freund
d what's proposed here is really simple to make PG more compliant > with this purpose in mind. It's not like $0 instead of a hardcoded name in the header actually buys us anything afaict. Greetings, Andres Freund

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-12 Thread Andres Freund
On 2017-12-12 14:16:59 -0500, Robert Haas wrote: > On Tue, Dec 12, 2017 at 2:12 PM, Andres Freund <and...@anarazel.de> wrote: > > I agree that it's more reliable - I hope there's no meaningful safety > > difference. I think you overestimate users a bit however - far

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-12 Thread Andres Freund
g and then have the requestor dig that out doesn't seem particularly user friendly. Greetings, Andres Freund

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Andres Freund
a suggest we ought to rip that code out in the not too far away future. The background is that I was working on committing the faster (& correct) overflow checks, and wanted to compile postgres with -ftrapv. Some rudimentary cash (as well as pgbench, rel/abstime) fixes were required to make the tests succeed... Greetings, Andres Freund

Re: pgbench's expression parsing & negative numbers

2017-12-12 Thread Andres Freund
tests are pretty hard to parse > > for humans. > > Probably. > > Perl is pretty hard to humans to start with:-) Meh. > There is a lot of code factorization to cram many tests together, so that > one test is more or less one line and there are hundreds of them. Not sure > it would help if it was expanded. I don't think expanding it is really a problem, I think they're just largely not well documented, formatted. With some effort the tests could be a lot easier to read. Greetings, Andres Freund

Re: pgsql: Provide overflow safe integer math inline functions.

2017-12-14 Thread Andres Freund
Hi, On 2017-12-14 09:28:08 +0100, Christoph Berg wrote: > Re: Andres Freund 2017-12-13 > <20171213173524.rjs7b3ahsong5...@alap3.anarazel.de> > > After staring at it for a while, I seem to have partially mis-copied the > > note for addition to the subtraction ope

Re: pgbench's expression parsing & negative numbers

2017-12-14 Thread Andres Freund
Hi, On 2017-12-14 10:41:04 +0100, Fabien COELHO wrote: > - I do not think that updating pgbench arithmetic for managing integer >overflows is worth Andres Freund time. My guess is that most >script would not trigger client-side overflows, so the change would >be a no-op

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-14 Thread Andres Freund
ROR, > + (errcode(ERRCODE_DATA_CORRUPTED), > +errmsg_internal("uncommitted Xmin %u from > before xid cutoff %u needs to be frozen", > +xid, cutoff_xid))); > "Xmin" I have never seen, but "xmin" I did. Changed... Greetings, Andres Freund

Re: Protect syscache from bloating with negative cache entries

2017-12-18 Thread Andres Freund
On 2017-12-17 19:23:45 -0500, Robert Haas wrote: > On Sat, Dec 16, 2017 at 11:42 PM, Andres Freund <and...@anarazel.de> wrote: > >> I'm not sure we should regard very quick bloating as a problem in need > >> of solving. Doesn't that just mean we need the cache

Re: [HACKERS] Parallel Hash take II

2017-12-18 Thread Andres Freund
On 2017-12-14 21:06:34 +1300, Thomas Munro wrote: > On Thu, Dec 14, 2017 at 11:45 AM, Andres Freund <and...@anarazel.de> wrote: > > + if (accessor->write_pointer + > > accessor->sts->meta_data_size >= > > +

Re: Logical replication without a Primary Key

2017-12-18 Thread Andres Freund
FWIW, I wonder if we need to add a warning somewhere about FULL replication, given it's essentially O(#changes * #rows) -> O(n^2) for updating the whole table. Greetings, Andres Freund

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-14 Thread Andres Freund
On 2017-12-14 17:00:29 -0800, Andres Freund wrote: > Hi, > > On 2017-11-13 19:03:41 -0800, Andres Freund wrote: > > diff --git a/src/backend/access/heap/rewriteheap.c > > b/src/backend/access/heap/rewriteheap.c > > index f93c194e182..7d163c91379 100644 > &

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-14 Thread Andres Freund
Hi, On 2017-11-13 19:03:41 -0800, Andres Freund wrote: > diff --git a/src/backend/access/heap/rewriteheap.c > b/src/backend/access/heap/rewriteheap.c > index f93c194e182..7d163c91379 100644 > --- a/src/backend/access/heap/rewriteheap.c > +++ b/src/backend/access/heap/rewritehea

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-15 Thread Andres Freund
On 2017-12-15 10:46:05 -0800, Peter Geoghegan wrote: > On Thu, Dec 14, 2017 at 6:30 PM, Andres Freund <and...@anarazel.de> wrote: > > Pushed this way. Moved some more relfrozenxid/relminmxid tests outside > > of the cutoff changes, polished some error messages. > > >

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-15 Thread Andres Freund
On 2017-12-15 20:25:22 +0900, Michael Paquier wrote: > On Fri, Dec 15, 2017 at 11:30 AM, Andres Freund <and...@anarazel.de> wrote: > > Alvaro, Michael, Peter, and everyone else I'd greatly appreciate if you > > could have a look at the backported version, just about every

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-15 Thread Andres Freund
; > there, but they were kept in sync. > > BTW, if you're going to do it, I would target the similar error within > validate_index_heapscan(), too. That was also added by d70cf811. Please send a patch for master on a *new* thread. Greetings, Andres Freund

Re: [HACKERS] Parallel Hash take II

2017-12-14 Thread Andres Freund
Hi, Looking at the main patch (v28). First off: This looks pretty good, the code's quite readable now (especially compared to earlier versions), the comments are good. Really like the nodeHash split, and the always inline hackery in nodeHashjoin. Think we're getting really really close. *

pg_ctl on windows can't open postmaster.pid: Permission denied

2017-12-12 Thread Andres Freund
open which backs open and fopen in backend code. I've not seen this error previously, so maybe this is just a random one-off. Greetings, Andres Freund

Re: Rethinking MemoryContext creation

2017-12-12 Thread Andres Freund
The best thing to do would probably to do two walks during executor initialization, one to compute memory sizes, and a second to initialize pre-requested memory that's laid out serially... But obviously that's not a small change. Greetings, Andres Freund

vacuum vs heap_update_tuple() and multixactids

2017-12-19 Thread Andres Freund
tuple's xmax for a new tuple version, we need to prune dead multixact members. I think we can do so unconditionally and rely on multixact id caching layer to avoid unnecesarily creating multis when all members are the same. Greetings, Andres Freund [1] http://archives.postgresql.org/message-id/20

Re: vacuum vs heap_update_tuple() and multixactids

2017-12-19 Thread Andres Freund
On 2017-12-19 15:35:12 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > I think the bugfix is going to have to essentially be something similar > > to FreezeMultiXactId(). I.e. when reusing an old tuple's xmax for a new > > tuple version, we need to prune dead mu

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-19 Thread Andres Freund
sense to expand that API and leave MemoryContextStats() alone. I don't think there's a need for a third variant? Greetings, Andres Freund

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-12 Thread Andres Freund
On 2017-12-12 14:04:55 -0500, Robert Haas wrote: > On Tue, Dec 12, 2017 at 1:50 PM, Andres Freund <and...@anarazel.de> wrote: > > I do wonder if the right thing here wouldn't be to put the result into a > > dsm segment, and then return that to the UDF on the request

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Andres Freund
to fix those up. > (OTOH, I bet we could drop reltime/abstime without too many complaints. > Y2038 is coming.) I'm actually about to send a patch doing so, that code is one mess WRT overflow handling. Greetings, Andres Freund

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread Andres Freund
Hi, On 2017-12-12 17:49:54 -0500, David Steele wrote: > Including unlogged relations in base backups takes up space and is wasteful > since they are truncated during backup recovery. > > The attached patches exclude unlogged relations from base backups except for > the init fork, which is

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread Andres Freund
Hi, On 2017-12-12 18:04:44 -0500, David Steele wrote: > On 12/12/17 5:52 PM, Andres Freund wrote: > > On 2017-12-12 17:49:54 -0500, David Steele wrote: > > > Including unlogged relations in base backups takes up space and is > > > wasteful > > > since they

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread Andres Freund
On 2017-12-12 18:18:09 -0500, David Steele wrote: > On 12/12/17 6:07 PM, Andres Freund wrote: > > > > > > I don't see this as any different than what happens during recovery. The > > > unlogged forks are cleaned / re-inited before replay starts which is the >

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Andres Freund
On 2017-12-13 08:27:42 +0900, Michael Paquier wrote: > On Wed, Dec 13, 2017 at 6:50 AM, Andres Freund <and...@anarazel.de> wrote: > > On 2017-12-12 16:47:17 -0500, Tom Lane wrote: > >> Really? We've got test cases that intentionally exercise overflow > >> in the m

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread Andres Freund
unacceptable to me. On a machine with lots of toasting etc going on an oid wraparound doesn't take a long time. We've only one oid counter for all tables, and relfilenodes are inferred from that Greetings, Andres Freund

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-12-12 Thread Andres Freund
e require the filesystem to perform metadata operations, which in turn requires journaling on most FSs. Which'll often result in synchronous disk writes. Greetings, Andres Freund

Re: [HACKERS] Parallel Hash take II

2017-12-13 Thread Andres Freund
+ /* If this is an overflow chunk, we skip it. */ + if (chunk_header.overflow) + continue; + + accessor->read_ntuples = 0; + accessor->read_ntuples_available = chunk_header.ntuples; + accessor->read_bytes = offsetof(SharedTuplestoreChunk, data); Perhaps somewhere around here comment that we'll just loop around and call sts_read_tuple() in the next loop iteration? Greetings, Andres Freund

Re: Inlining functions with "expensive" parameters

2017-11-17 Thread Andres Freund
On November 17, 2017 5:15:57 AM PST, Paul Ramsey <pram...@cleverelephant.ca> wrote: >On Thu, Nov 16, 2017 at 12:37 PM, Andres Freund <and...@anarazel.de> >wrote: > >> Hi, >> >> On 2017-11-16 15:27:59 -0500, Tom Lane wrote: >> > Andres Freund &l

Re: [HACKERS] Parallel Hash take II

2017-11-17 Thread Andres Freund
d to document somehwere that the temp_file_limit in a shared file set applies independently for each participant that's writing something. We also should discuss whether that's actually sane behaviour. Greetings, Andres Freund

Re: FP16 Support?

2017-11-13 Thread Andres Freund
r priorities reversed. Agree that there's no performance argument. I think you could kinda sorta make an argument for higher storage density in cases where a lot of floats are stored in the database. I'd personally still consider that not worthwhile to invest time in, but ... Greetings, Andres Freund

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-13 Thread Andres Freund
Hi, On 2017-11-03 07:53:30 -0700, Andres Freund wrote: > Here's that patch. I've stared at this some, and Robert did too. Robert > mentioned that the commit message might need some polish and I'm not > 100% sure about the error message texts yet. > > I'm not yet convinced th

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-14 Thread Andres Freund
On 2017-11-13 19:03:41 -0800, Andres Freund wrote: > > Staring at the vacuumlazy hunk I think I might have found a related bug: > > heap_update_tuple() just copies the old xmax to the new tuple's xmax if > > a multixact and still running. It does so without verifying livelin

Re: [PATCH]pg_buffercache add a buffer state column, Add fuction to decode buffer state

2017-11-14 Thread Andres Freund
around that would be to create a buffer_state type that's returned by pg_buffercache and then only decoded when outputting. Doing that + having a cast to an array seems like it'd provide most of the needed functionality? Greetings, Andres Freund

Re: Add PGDLLIMPORT lines to some variables

2017-11-20 Thread Andres Freund
Hi, Please quote properly on postgres mailing lists... We're old school [tm]. On 2017-11-20 11:58:44 -0800, Brian Cloutier wrote: > > please, append session_timezone to your list > > Here's a v2 patch which also includes session_timezone. > > Separately, is this the kind of thing which is

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-20 Thread Andres Freund
Hi, On 2017-11-13 19:03:41 -0800, Andres Freund wrote: > Hi, > > On 2017-11-03 07:53:30 -0700, Andres Freund wrote: > > Here's that patch. I've stared at this some, and Robert did too. Robert > > mentioned that the commit message might need some polish and I'm not > >

Combine function returning NULL unhandled?

2017-11-20 Thread Andres Freund
ee https://www.postgresql.org/docs/devel/static/sql-createaggregate.html spell out a requirement that that not be the case. Greetings, Andres Freund

Re: Inlining functions with "expensive" parameters

2017-11-21 Thread Andres Freund
Hi, On 2017-11-21 10:16:58 -0500, Tom Lane wrote: > I'm also wondering about folding CaseTestExpr and CoerceToDomainValue > into the same mechanism. It's not very hard to see those cases as > being the same as a function-based lambda. Yea, that'd be good. The current mechanisms is uh,

Re: Inlining functions with "expensive" parameters

2017-11-21 Thread Andres Freund
On 2017-11-21 09:59:00 -0500, Robert Haas wrote: > On Thu, Nov 16, 2017 at 2:51 PM, Andres Freund <and...@anarazel.de> wrote: > > Right, but it doesn't sound that hard to introduce. Basically there'd need > > to be a WithParamValue node, that first evaluates parameters a

Re: Inlining functions with "expensive" parameters

2017-11-16 Thread Andres Freund
On November 16, 2017 11:44:52 AM PST, Tom Lane <t...@sss.pgh.pa.us> wrote: >Andres Freund <and...@anarazel.de> writes: >> Well, it's not a question of cost of the function now? Imagine >> SELECT inlineable(something()); >> if you have 10 references fo

Re: Inlining functions with "expensive" parameters

2017-11-16 Thread Andres Freund
ow out the cost checks and be done with it. But what I *am* wondering about, is why we're not handling the parameters in a different way. Instead of replacing the all parameter references with the parameter, it shouldn't be too hard to instead replace them with a new PARAM_EXEC like Param. Besides reducing the cost barrier for inlining, that should also quite drastically expand the set of inlinable cases because volatile functions etc aren't a problem anymore. Presumably we'd want either to continue replacing single references to parameters like we currently are. But maybe we could store enough information and make that just part of expression simplification? Greetings, Andres Freund

Re: [HACKERS] Parallel Hash take II

2017-11-16 Thread Andres Freund
Hi, On 2017-11-15 14:09:13 -0500, Robert Haas wrote: > On Wed, Nov 15, 2017 at 1:35 PM, Andres Freund <and...@anarazel.de> wrote: > > But this does bug me, and I think it's what made me pause here to make a > > bad joke. The way that parallelism treats work_mem makes it

Re: [HACKERS] Issues with logical replication

2017-11-16 Thread Andres Freund
ckTableWait() or the procarray, which could affect an > awful lot of other things. Hm. Thinking about this. Greetings, Andres Freund

Re: Inlining functions with "expensive" parameters

2017-11-16 Thread Andres Freund
Hi, On 2017-11-16 15:27:59 -0500, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On November 16, 2017 11:44:52 AM PST, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> Yeah, there's no mechanism like that now, but there could be. > > >

Re: [HACKERS] pg_upgrade to clusters with a different WAL segment size

2017-11-13 Thread Andres Freund
On 2017-11-14 07:26:22 +0900, Michael Paquier wrote: > On Tue, Nov 14, 2017 at 6:11 AM, Andres Freund <and...@anarazel.de> wrote: > > Hm. I'm not really on-board with doing this in pg_upgrade. A more > > logical place seems to be pg_resetwal or something - there's no need to &

Re: [HACKERS] Parallel Hash take II

2017-11-17 Thread Andres Freund
Hi, On 2017-11-14 01:30:30 +1300, Thomas Munro wrote: > New patch attached. 0002-Add-a-barrier-primitive-for-synchronizing-backends.patch - Intro starts with: + * + * This implementation of barriers allows for static sets of participants + * known up front, or dynamic sets of participants which

Re: [HACKERS] Parallel Hash take II

2017-12-07 Thread Andres Freund
r investigating crashes. I don't want to hold up this patch with a discussion of that however, so I'm ok with your fix. Greetings, Andres Freund

Re: Usage of epoch in txid_current

2017-12-05 Thread Andres Freund
f xid epochs. I don't really how we got there, but it doesn't make terribly much sense. Don't think we need additional WAL logging though - we should be able to piggyback this onto the already existing clog logging. I kinda wonder if we shouldn't just track nextXid as a 64bit integer internally, instead of bothering with tracking the epoch separately. Then we can "just" truncate it in the cases where it's stored in space constrained places etc. Greetings, Andres Freund

Re: Usage of epoch in txid_current

2017-12-05 Thread Andres Freund
On December 5, 2017 10:01:43 AM PST, Stephen Frost <sfr...@snowman.net> wrote: >Andres, > >* Andres Freund (and...@anarazel.de) wrote: >> I think it makes a fair bit of sense to not do the current crufty >> tracking of xid epochs. I don't really how we got there, but i

Re: Is it possible and worthy to optimize scanRTEForColumn()?

2017-12-08 Thread Andres Freund
the complexity is essentially O(#available-columns * #looked-up-columns). However I don't think a microoptimization, even if it were correct, as breaking earlier out of the loop would help meaningfully. We'd need a different datastructure. Greetings, Andres Freund

Re: Is it possible and worthy to optimize scanRTEForColumn()?

2017-12-08 Thread Andres Freund
he planner, but that's a bigger piece of work than I can take on right now. Greetings, Andres Freund

Re: Postgres with pthread

2017-12-07 Thread Andres Freund
imo. But I'm unclear what you'd use SPI for that really necessitates that. We don't suspend execution it the middle of function execution... Greetings, Andres Freund

Faster str to int conversion (was Table with large number of int columns, very slow COPY FROM)

2017-12-08 Thread Andres Freund
Hi, On 2017-12-08 10:17:34 -0800, Andres Freund wrote: > the strtoll is libc functionality triggered by pg_atoi(), something I've > seen show up in numerous profiles. I think it's probably time to have > our own optimized version of it rather than relying on libcs. Attached is a ha

Re: Partition pruning for Star Schema

2017-12-02 Thread Andres Freund
and* linked to the last time you sent something it'd be different. Greetings, Andres Freund

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-07 Thread Andres Freund
ase and patch. Yea, you're right. I was writing it with 14h of jetlag, apparently that does something to your brain... Greetings, Andres Freund

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-07 Thread Andres Freund
RelationGetRelationName(info->relation)); > +} Hm, perhaps rephrase so both messages refer to vacuuming? E.g. just by replacing scanning with vacuuming? Greetings, Andres Freund

Re: Signals in a BGW

2017-12-07 Thread Andres Freund
da ok, everything else not. But it definitely has caused issues in the ast, mostly due to malloc being entered while already in malloc. Greetings, Andres Freund

Re: Postgres with pthread

2017-12-07 Thread Andres Freund
On 2017-12-07 11:26:07 +0800, Craig Ringer wrote: > PostgreSQL's architecture conflates "connection", "session" and "executor" > into one somewhat muddled mess. How is the executor entangled in the other two? Greetings, Andres Freund

Re: Logical replication without a Primary Key

2017-12-07 Thread Andres Freund
update didn't actually change anything. In which case the same row gets updated twice, but because the column values didn't change, that doesn't matter. Greetings, Andres Freund

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-07 Thread Andres Freund
> which Andres has reduced to a subset of permutations. Peter G. also > complained about what is visibly the same bug we are discussing here > but without a test case.) The previous versions of the test case didn't actually hit the real issues, so I don't think that matter much. Greetings, Andres Freund

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-07 Thread Andres Freund
On 2017-12-07 17:41:56 -0300, Alvaro Herrera wrote: > > > Looking at 0002: I agree with the stuff being done here. I think a > > > couple of these checks could be moved one block outerwards in term of > > > scope; I don't see any reason why the check should not apply in that > > > case. I didn't

Re: Rethinking MemoryContext creation

2017-12-11 Thread Andres Freund
On 2017-12-11 13:09:42 -0500, Tom Lane wrote: > Tomas Vondra writes: > > On 12/10/2017 04:42 PM, Tom Lane wrote: > >> Peter Geoghegan writes: > >>> On Sat, Dec 9, 2017 at 5:53 PM, Tom Lane wrote: > Overall I'm seeing about a

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-12-11 Thread Andres Freund
t's imo perfectly fine to say that there's only a limited number of file extension locks, but that there's a far from neglegible chance of conflict even without the array being full doesn't seem nice. Think this needs use some open addressing like conflict handling or something alike. Greetings, Andres Freund

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-12-11 Thread Andres Freund
On 2017-12-11 15:55:42 -0500, Robert Haas wrote: > On Mon, Dec 11, 2017 at 3:25 PM, Andres Freund <and...@anarazel.de> wrote: > > For me "very short periods of time" and journaled metadatachanging > > filesystem operations don't quite mesh. Language lawyering asi

Re: Postgres with pthread

2017-12-06 Thread Andres Freund
parallelism if any loaded shared library doesn't signal parallelism support. We also don't necessarily want to require threads on all platforms at the same time. I think the biggest problem with doing this for real is that it's a huge project, and that it'll take a long time. Thanks for working on this! Andres Freund

fixed tuple descs (was JIT compiling expressions/deform)

2017-12-06 Thread Andres Freund
submitted patches, but it'd just be an hour's work or so to reorder that. Comments about either the outline above or the patch? Regards, Andres >From cd04258d92bed57d7e8a6bbe0408c7fb31f1a182 Mon Sep 17 00:00:00 2001 From: Andres Freund <and...@anarazel.de> Date: Tue, 3 Oct 2017 23:45:44 -0

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-12-06 Thread Andres Freund
On 2017-11-20 11:18:45 -0800, Andres Freund wrote: > Hi, > > On 2017-11-13 19:03:41 -0800, Andres Freund wrote: > > Hi, > > > > On 2017-11-03 07:53:30 -0700, Andres Freund wrote: > > > Here's that patch. I've stared at this some, and Robert did too. Ro

Re: Postgres with pthread

2017-12-06 Thread Andres Freund
long term complexity and feature viability in control. Greetings, Andres Freund

Re: Usage of epoch in txid_current

2017-12-06 Thread Andres Freund
increments only at > wraparound seems fairly straightforward as described in my previous > email. I think it should be fairly simple if simply add the 64bit xid to the existing clog extension WAL records. Greetings, Andres Freund

Re: Postgres with pthread

2017-12-06 Thread Andres Freund
Hi, On 2017-12-06 12:28:29 -0500, Robert Haas wrote: > > Possibly we even want to continue having various > > processes around besides that, the most interesting cases involving > > threads are around intra-query parallelism, and pooling, and for both a > > hybrid model could be beneficial. > >

money type's overflow handling is woefully incomplete

2017-12-11 Thread Andres Freund
1]=# SELECT '92233720368547758.07'::money+'0.1'; ┌─┐ │ ?column? │ ├─┤ │ -$92,233,720,368,547,757.99 │ └─────────┘ Greetings, Andres Freund

pgbench's expression parsing & negative numbers

2017-12-11 Thread Andres Freund
rse for humans. Greetings, Andres Freund

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-11 Thread Andres Freund
Not sure if that's still good enough for you? Another question is whether printing to stderr, bypassing proper logging!, is good enough for something like this. Greetings, Andres Freund

  1   2   3   4   5   6   7   8   9   >