ads are very complicated, and always having users target the
> worst case leaves a lot to be desired.
Obviously that's nice and worthwhile goal, but it seems more than a bit
out of reach for this patchset.
Greetings,
Andres Freund
On 2017-11-15 08:37:11 -0500, Robert Haas wrote:
> On Tue, Nov 14, 2017 at 4:24 PM, Andres Freund <and...@anarazel.de> wrote:
> >> I agree, and I am interested in that subject. In the meantime, I
> >> think it'd be pretty unfair if parallel-oblivious hash join and
>
if the same result would present if the benchmarks
were run with parallel_leader_participation. The theory being what were
seing is just that the leader doesn't accept any tuples, and the large
queue size just helps because workers can run for longer.
Greetings,
Andres Freund
Hi,
On 2017-11-28 14:51:06 -0500, Robert Haas wrote:
> If nobody minds too much, I'd like to update typedefs.list and
> pgindent the whole tree again.
+1.
Greetings,
Andres Freund
On 2017-11-21 15:51:59 -0500, Robert Haas wrote:
> On Mon, Nov 20, 2017 at 10:36 PM, Andres Freund <and...@anarazel.de> wrote:
> > The plain transition case contains:
> > if (pergroupst
* family, RC4 has
> predictability issues), there is no need of a file descriptor.
I don't really see much benefit in those properties for postgres
specifically. Not needing an fd is nice for cases where you're not
guaranteed to have access to a filesystem, but postgres isn't going to
work in those cases anyway.
Greetings,
Andres Freund
On November 22, 2017 8:51:07 AM PST, ilm...@ilmari.org wrote:
>If what is wanted is something more like /dev/urandom, one can call
>getentropy(2) (or on Linux, getrandom(2)) directly, which avoids having
>to open the device file each time.
What does that buy us for our usages?
Andres
--
Sent
On November 29, 2017 8:50:31 AM PST, Stephen Frost wrote:
>As for conflicting snapshots, isn't the lock we're taking already
>AccessExclusive..?
Doesn't help if e.g. the current xact is repeatable read or if your own xact
deleted things (other xacts with snapshots could
implementing the
> XactLockTableInsert() instead of calling it.
Right. But we fairly trivially can change that. I'm remarking on it
because other people's, not yours, suggestions aimed at making this
bulletproof. I just wanted to make clear that I don't think that's
necessary at all.
Greetings,
Andres Freund
og a new xl_running_xacts record where it'd be helpful, without having
* to write for bgwriter or checkpointer.
Greetings,
Andres Freund
On 2017-11-30 14:17:51 +1300, Thomas Munro wrote:
> On Mon, Nov 27, 2017 at 10:25 PM, Thomas Munro
> wrote:
> > On Thu, Nov 23, 2017 at 12:36 AM, Thomas Munro
> > wrote:
> >> Here's a new patch set with responses to the last batch of
ew callers should be interested in this, but tuplesort/tuplestore need
> * to know it.
> */
> -#define ALLOCSET_SEPARATE_THRESHOLD 8192
> +#define ALLOCSET_SEPARATE_THRESHOLD 16384
Huh, what's that about in this context?
Greetings,
Andres Freund
.
Greetings,
Andres Freund
>From 3ff1371aae4615c15f6bcdc232170a3a295e8021 Mon Sep 17 00:00:00 2001
From: Andres Freund <and...@anarazel.de>
Date: Fri, 24 Nov 2017 14:14:35 -0800
Subject: [PATCH 1/3] Simplify representation of aggregate transition values a
bit.
Previously aggregate transiti
all files FILE_SHARE_DELETE take care of this? See
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx
"Note Delete access allows both delete and rename operations."
Is there an external process active that doesn't set that flag? Are we
missing setting it somewhere?
Greetings,
Andres Freund
On 2017-11-29 18:23:40 -0500, Chapman Flack wrote:
> On 11/29/2017 05:54 PM, Michael Paquier wrote:
>
> > Yes. That's actually what the autovacuum launcher does. It connects
> > using InitPostgres(NULL, InvalidOid, NULL, NULL), and then scans
> > pg_database to fetch a list (see
nor I seem to be convinced about:
>
> +/* not worth providing a workaround */
FWIW, I think that's a perfectly reasonable choice. Adding complications
in making static assertions work for random archaic compilers when
compiling with c++ just doesn't seem worth more than a few mins of
thought.
Greetings,
Andres Freund
small and there's
normally not goign to be that many huge tuples (otherwise things are
slow anyway).
- why are we using a separate hardcoded 32 for sts names? Why not just
go for NAMEDATALEN or such?
- I'd replace most of the "should's" in comments with "need".
Greetings,
Andres Freund
by that problem? By bumping the generation on hash resize, we
have recency information we can take into account.
Greetings,
Andres Freund
m.
It'd be way less work to package abstime as an extension if you want to
do so.
Greetings,
Andres Freund
d what's proposed here is really simple to make PG more compliant
> with this purpose in mind.
It's not like $0 instead of a hardcoded name in the header actually buys
us anything afaict.
Greetings,
Andres Freund
On 2017-12-12 14:16:59 -0500, Robert Haas wrote:
> On Tue, Dec 12, 2017 at 2:12 PM, Andres Freund <and...@anarazel.de> wrote:
> > I agree that it's more reliable - I hope there's no meaningful safety
> > difference. I think you overestimate users a bit however - far
g and then have the requestor dig that out
doesn't seem particularly user friendly.
Greetings,
Andres Freund
a suggest we ought to rip
that code out in the not too far away future.
The background is that I was working on committing the faster (&
correct) overflow checks, and wanted to compile postgres with -ftrapv.
Some rudimentary cash (as well as pgbench, rel/abstime) fixes were
required to make the tests succeed...
Greetings,
Andres Freund
tests are pretty hard to parse
> > for humans.
>
> Probably.
>
> Perl is pretty hard to humans to start with:-)
Meh.
> There is a lot of code factorization to cram many tests together, so that
> one test is more or less one line and there are hundreds of them. Not sure
> it would help if it was expanded.
I don't think expanding it is really a problem, I think they're just
largely not well documented, formatted. With some effort the tests could
be a lot easier to read.
Greetings,
Andres Freund
Hi,
On 2017-12-14 09:28:08 +0100, Christoph Berg wrote:
> Re: Andres Freund 2017-12-13
> <20171213173524.rjs7b3ahsong5...@alap3.anarazel.de>
> > After staring at it for a while, I seem to have partially mis-copied the
> > note for addition to the subtraction ope
Hi,
On 2017-12-14 10:41:04 +0100, Fabien COELHO wrote:
> - I do not think that updating pgbench arithmetic for managing integer
>overflows is worth Andres Freund time. My guess is that most
>script would not trigger client-side overflows, so the change would
>be a no-op
ROR,
> + (errcode(ERRCODE_DATA_CORRUPTED),
> +errmsg_internal("uncommitted Xmin %u from
> before xid cutoff %u needs to be frozen",
> +xid, cutoff_xid)));
> "Xmin" I have never seen, but "xmin" I did.
Changed...
Greetings,
Andres Freund
On 2017-12-17 19:23:45 -0500, Robert Haas wrote:
> On Sat, Dec 16, 2017 at 11:42 PM, Andres Freund <and...@anarazel.de> wrote:
> >> I'm not sure we should regard very quick bloating as a problem in need
> >> of solving. Doesn't that just mean we need the cache
FWIW, I wonder if we need to add a warning somewhere about FULL
replication, given it's essentially O(#changes * #rows) -> O(n^2) for
updating the whole table.
Greetings,
Andres Freund
On 2017-12-14 17:00:29 -0800, Andres Freund wrote:
> Hi,
>
> On 2017-11-13 19:03:41 -0800, Andres Freund wrote:
> > diff --git a/src/backend/access/heap/rewriteheap.c
> > b/src/backend/access/heap/rewriteheap.c
> > index f93c194e182..7d163c91379 100644
> &
Hi,
On 2017-11-13 19:03:41 -0800, Andres Freund wrote:
> diff --git a/src/backend/access/heap/rewriteheap.c
> b/src/backend/access/heap/rewriteheap.c
> index f93c194e182..7d163c91379 100644
> --- a/src/backend/access/heap/rewriteheap.c
> +++ b/src/backend/access/heap/rewritehea
On 2017-12-15 10:46:05 -0800, Peter Geoghegan wrote:
> On Thu, Dec 14, 2017 at 6:30 PM, Andres Freund <and...@anarazel.de> wrote:
> > Pushed this way. Moved some more relfrozenxid/relminmxid tests outside
> > of the cutoff changes, polished some error messages.
> >
>
On 2017-12-15 20:25:22 +0900, Michael Paquier wrote:
> On Fri, Dec 15, 2017 at 11:30 AM, Andres Freund <and...@anarazel.de> wrote:
> > Alvaro, Michael, Peter, and everyone else I'd greatly appreciate if you
> > could have a look at the backported version, just about every
; > there, but they were kept in sync.
>
> BTW, if you're going to do it, I would target the similar error within
> validate_index_heapscan(), too. That was also added by d70cf811.
Please send a patch for master on a *new* thread.
Greetings,
Andres Freund
Hi,
Looking at the main patch (v28).
First off: This looks pretty good, the code's quite readable now
(especially compared to earlier versions), the comments are good. Really
like the nodeHash split, and the always inline hackery in nodeHashjoin.
Think we're getting really really close.
*
open which backs open and
fopen in backend code.
I've not seen this error previously, so maybe this is just a random
one-off.
Greetings,
Andres Freund
The best thing to do would probably to do two
walks during executor initialization, one to compute memory sizes, and a
second to initialize pre-requested memory that's laid out
serially... But obviously that's not a small change.
Greetings,
Andres Freund
tuple's xmax for a new
tuple version, we need to prune dead multixact members. I think we can
do so unconditionally and rely on multixact id caching layer to avoid
unnecesarily creating multis when all members are the same.
Greetings,
Andres Freund
[1]
http://archives.postgresql.org/message-id/20
On 2017-12-19 15:35:12 -0300, Alvaro Herrera wrote:
> Andres Freund wrote:
>
> > I think the bugfix is going to have to essentially be something similar
> > to FreezeMultiXactId(). I.e. when reusing an old tuple's xmax for a new
> > tuple version, we need to prune dead mu
sense to
expand that API and leave MemoryContextStats() alone. I don't think
there's a need for a third variant?
Greetings,
Andres Freund
On 2017-12-12 14:04:55 -0500, Robert Haas wrote:
> On Tue, Dec 12, 2017 at 1:50 PM, Andres Freund <and...@anarazel.de> wrote:
> > I do wonder if the right thing here wouldn't be to put the result into a
> > dsm segment, and then return that to the UDF on the request
to fix
those up.
> (OTOH, I bet we could drop reltime/abstime without too many complaints.
> Y2038 is coming.)
I'm actually about to send a patch doing so, that code is one mess WRT
overflow handling.
Greetings,
Andres Freund
Hi,
On 2017-12-12 17:49:54 -0500, David Steele wrote:
> Including unlogged relations in base backups takes up space and is wasteful
> since they are truncated during backup recovery.
>
> The attached patches exclude unlogged relations from base backups except for
> the init fork, which is
Hi,
On 2017-12-12 18:04:44 -0500, David Steele wrote:
> On 12/12/17 5:52 PM, Andres Freund wrote:
> > On 2017-12-12 17:49:54 -0500, David Steele wrote:
> > > Including unlogged relations in base backups takes up space and is
> > > wasteful
> > > since they
On 2017-12-12 18:18:09 -0500, David Steele wrote:
> On 12/12/17 6:07 PM, Andres Freund wrote:
> > >
> > > I don't see this as any different than what happens during recovery. The
> > > unlogged forks are cleaned / re-inited before replay starts which is the
>
On 2017-12-13 08:27:42 +0900, Michael Paquier wrote:
> On Wed, Dec 13, 2017 at 6:50 AM, Andres Freund <and...@anarazel.de> wrote:
> > On 2017-12-12 16:47:17 -0500, Tom Lane wrote:
> >> Really? We've got test cases that intentionally exercise overflow
> >> in the m
unacceptable to me. On a machine with lots of
toasting etc going on an oid wraparound doesn't take a long time. We've
only one oid counter for all tables, and relfilenodes are inferred from
that
Greetings,
Andres Freund
e require the filesystem to perform
metadata operations, which in turn requires journaling on most
FSs. Which'll often result in synchronous disk writes.
Greetings,
Andres Freund
+ /* If this is an overflow chunk, we skip it. */
+ if (chunk_header.overflow)
+ continue;
+
+ accessor->read_ntuples = 0;
+ accessor->read_ntuples_available = chunk_header.ntuples;
+ accessor->read_bytes = offsetof(SharedTuplestoreChunk,
data);
Perhaps somewhere around here comment that we'll just loop around and
call sts_read_tuple() in the next loop iteration?
Greetings,
Andres Freund
On November 17, 2017 5:15:57 AM PST, Paul Ramsey <pram...@cleverelephant.ca>
wrote:
>On Thu, Nov 16, 2017 at 12:37 PM, Andres Freund <and...@anarazel.de>
>wrote:
>
>> Hi,
>>
>> On 2017-11-16 15:27:59 -0500, Tom Lane wrote:
>> > Andres Freund &l
d to document somehwere that the temp_file_limit in a
shared file set applies independently for each participant that's
writing something. We also should discuss whether that's actually
sane behaviour.
Greetings,
Andres Freund
r priorities reversed.
Agree that there's no performance argument. I think you could kinda
sorta make an argument for higher storage density in cases where a lot
of floats are stored in the database. I'd personally still consider
that not worthwhile to invest time in, but ...
Greetings,
Andres Freund
Hi,
On 2017-11-03 07:53:30 -0700, Andres Freund wrote:
> Here's that patch. I've stared at this some, and Robert did too. Robert
> mentioned that the commit message might need some polish and I'm not
> 100% sure about the error message texts yet.
>
> I'm not yet convinced th
around that would be to create a buffer_state type that's
returned by pg_buffercache and then only decoded when outputting. Doing
that + having a cast to an array seems like it'd provide most of the
needed functionality?
Greetings,
Andres Freund
Hi,
On 2017-11-13 19:03:41 -0800, Andres Freund wrote:
> Hi,
>
> On 2017-11-03 07:53:30 -0700, Andres Freund wrote:
> > Here's that patch. I've stared at this some, and Robert did too. Robert
> > mentioned that the commit message might need some polish and I'm not
> >
ow out the cost checks and be done
with it.
But what I *am* wondering about, is why we're not handling the
parameters in a different way. Instead of replacing the all parameter
references with the parameter, it shouldn't be too hard to instead
replace them with a new PARAM_EXEC like Param.
Besides reducing the cost barrier for inlining, that should also quite
drastically expand the set of inlinable cases because volatile functions
etc aren't a problem anymore.
Presumably we'd want either to continue replacing single references to
parameters like we currently are. But maybe we could store enough
information and make that just part of expression simplification?
Greetings,
Andres Freund
Hi,
On 2017-11-15 14:09:13 -0500, Robert Haas wrote:
> On Wed, Nov 15, 2017 at 1:35 PM, Andres Freund <and...@anarazel.de> wrote:
> > But this does bug me, and I think it's what made me pause here to make a
> > bad joke. The way that parallelism treats work_mem makes it
Hi,
On 2017-11-16 15:27:59 -0500, Tom Lane wrote:
> Andres Freund <and...@anarazel.de> writes:
> > On November 16, 2017 11:44:52 AM PST, Tom Lane <t...@sss.pgh.pa.us> wrote:
> >> Yeah, there's no mechanism like that now, but there could be.
>
> >
Hi,
On 2017-11-14 01:30:30 +1300, Thomas Munro wrote:
> New patch attached.
0002-Add-a-barrier-primitive-for-synchronizing-backends.patch
- Intro starts with:
+ *
+ * This implementation of barriers allows for static sets of participants
+ * known up front, or dynamic sets of participants which
r investigating crashes. I don't want to hold up this patch with a
discussion of that however, so I'm ok with your fix.
Greetings,
Andres Freund
the complexity is essentially
O(#available-columns * #looked-up-columns).
However I don't think a microoptimization, even if it were correct, as
breaking earlier out of the loop would help meaningfully. We'd need a
different datastructure.
Greetings,
Andres Freund
he
planner, but that's a bigger piece of work than I can take on right now.
Greetings,
Andres Freund
imo. But I'm unclear what you'd use SPI for that really
necessitates that. We don't suspend execution it the middle of function
execution...
Greetings,
Andres Freund
Hi,
On 2017-12-08 10:17:34 -0800, Andres Freund wrote:
> the strtoll is libc functionality triggered by pg_atoi(), something I've
> seen show up in numerous profiles. I think it's probably time to have
> our own optimized version of it rather than relying on libcs.
Attached is a ha
and* linked to the last time you sent something it'd be
different.
Greetings,
Andres Freund
ase and patch.
Yea, you're right. I was writing it with 14h of jetlag, apparently that
does something to your brain...
Greetings,
Andres Freund
RelationGetRelationName(info->relation));
> +}
Hm, perhaps rephrase so both messages refer to vacuuming? E.g. just by
replacing scanning with vacuuming?
Greetings,
Andres Freund
da ok, everything else not. But it
definitely has caused issues in the ast, mostly due to malloc being
entered while already in malloc.
Greetings,
Andres Freund
On 2017-12-07 11:26:07 +0800, Craig Ringer wrote:
> PostgreSQL's architecture conflates "connection", "session" and "executor"
> into one somewhat muddled mess.
How is the executor entangled in the other two?
Greetings,
Andres Freund
update didn't actually change anything. In which case the same
row gets updated twice, but because the column values didn't change,
that doesn't matter.
Greetings,
Andres Freund
> which Andres has reduced to a subset of permutations. Peter G. also
> complained about what is visibly the same bug we are discussing here
> but without a test case.)
The previous versions of the test case didn't actually hit the real
issues, so I don't think that matter much.
Greetings,
Andres Freund
On 2017-12-07 17:41:56 -0300, Alvaro Herrera wrote:
> > > Looking at 0002: I agree with the stuff being done here. I think a
> > > couple of these checks could be moved one block outerwards in term of
> > > scope; I don't see any reason why the check should not apply in that
> > > case. I didn't
On 2017-12-11 13:09:42 -0500, Tom Lane wrote:
> Tomas Vondra writes:
> > On 12/10/2017 04:42 PM, Tom Lane wrote:
> >> Peter Geoghegan writes:
> >>> On Sat, Dec 9, 2017 at 5:53 PM, Tom Lane wrote:
> Overall I'm seeing about a
t's imo perfectly fine to say that there's only a limited number of
file extension locks, but that there's a far from neglegible chance of
conflict even without the array being full doesn't seem nice. Think this
needs use some open addressing like conflict handling or something
alike.
Greetings,
Andres Freund
On 2017-12-11 15:55:42 -0500, Robert Haas wrote:
> On Mon, Dec 11, 2017 at 3:25 PM, Andres Freund <and...@anarazel.de> wrote:
> > For me "very short periods of time" and journaled metadatachanging
> > filesystem operations don't quite mesh. Language lawyering asi
parallelism if any loaded shared library doesn't signal parallelism
support. We also don't necessarily want to require threads on all
platforms at the same time.
I think the biggest problem with doing this for real is that it's a huge
project, and that it'll take a long time.
Thanks for working on this!
Andres Freund
submitted patches, but it'd just be an hour's work or so to reorder
that.
Comments about either the outline above or the patch?
Regards,
Andres
>From cd04258d92bed57d7e8a6bbe0408c7fb31f1a182 Mon Sep 17 00:00:00 2001
From: Andres Freund <and...@anarazel.de>
Date: Tue, 3 Oct 2017 23:45:44 -0
On 2017-11-20 11:18:45 -0800, Andres Freund wrote:
> Hi,
>
> On 2017-11-13 19:03:41 -0800, Andres Freund wrote:
> > Hi,
> >
> > On 2017-11-03 07:53:30 -0700, Andres Freund wrote:
> > > Here's that patch. I've stared at this some, and Robert did too. Ro
long
term complexity and feature viability in control.
Greetings,
Andres Freund
increments only at
> wraparound seems fairly straightforward as described in my previous
> email.
I think it should be fairly simple if simply add the 64bit xid to the
existing clog extension WAL records.
Greetings,
Andres Freund
Hi,
On 2017-12-06 12:28:29 -0500, Robert Haas wrote:
> > Possibly we even want to continue having various
> > processes around besides that, the most interesting cases involving
> > threads are around intra-query parallelism, and pooling, and for both a
> > hybrid model could be beneficial.
>
>
1]=# SELECT '92233720368547758.07'::money+'0.1';
┌─┐
│ ?column? │
├─┤
│ -$92,233,720,368,547,757.99 │
└─────────┘
Greetings,
Andres Freund
rse
for humans.
Greetings,
Andres Freund
Not sure if that's still good enough
for you?
Another question is whether printing to stderr, bypassing proper
logging!, is good enough for something like this.
Greetings,
Andres Freund
a lot faster, and I've seen way way to many workloads with
50%+ time spent in pglz.
Greetings,
Andres Freund
On 2017-12-01 17:03:28 -0500, Tom Lane wrote:
> Andres Freund <and...@anarazel.de> writes:
> > On 2017-12-01 16:40:23 -0500, Tom Lane wrote:
> >> I have no faith in either of these proposals, because they both assume
> >> that the problem only arise
n X cycles get pruned.
Greetings,
Andres Freund
On 2017-12-01 16:40:23 -0500, Tom Lane wrote:
> Andres Freund <and...@anarazel.de> writes:
> > On 2017-12-01 16:20:44 -0500, Robert Haas wrote:
> >> Well, yeah, that would be insane. But I think even something very
> >> rough could work well enough. I thin
-core *and* then allow
extensibility, that's one thing, but keeping the bad and tell forks
"please take our users with this code we give you" is ...
Greetings,
Andres Freund
On 2017-12-02 15:54:29 +1300, Thomas Munro wrote:
> On Sat, Dec 2, 2017 at 1:55 PM, Andres Freund <and...@anarazel.de> wrote:
> > - Right now RemovePgTempFilesInDir() will recurse into appropriately
> > named directories, and when it recurses it doesn't require the same
>
ore paranoid and prevent recursing into symlinked
subdirectories.
- As we don't clean temp files after crash-restarts it isn't impossible
to have a bunch of crash-restarts and end up with pids *and* per-pid
shared file set counters reused. Which'd lead to conflicts. Do we care?
Greetings,
And
the query that might otherwise continue to work,
so that seems no excuse.
Greetings,
Andres Freund
Hi,
On 2017-12-19 15:01:03 -0500, Robert Haas wrote:
> On Tue, Dec 19, 2017 at 1:31 PM, Andres Freund <and...@anarazel.de> wrote:
> > Could I perhaps convince somebody to add that as a feature to
> > isolationtester? I'm willing to work on a bugfix for the bug itself, but
&
Hi,
On 2017-12-20 12:12:48 -0500, Tom Lane wrote:
> Andres Freund <and...@anarazel.de> writes:
> > What's the workload you're testing? I'm mildly surprised to see
> > ExecEvalParamExtern() show up, rather than just plpgsql_param_fetch() &
> > exec_eval_da
Hi,
On 2017-12-21 02:42:25 -0800, Andres Freund wrote:
> Trying to debug this I found another issue. I'd placed a sleep(10) in
> ExecParallelHashCloseBatchAccessors() and then ctrl-c'ed the server for
> some reason. Segfault time:
>
> #0 0x55bfbac42539 in tas (lock=0x7fcd8
,
Andres Freund
Hi,m
On 2018-05-04 10:58:22 -0400, Tom Lane wrote:
> On balance I'm leaning towards option #2, ie go with the new data but
> wait till just after the minor releases. Thoughts?
+1. Should have a beta in that timeframe as well, giving more people a
chance to notice.
Greetings,
Andres Freund
On 2018-05-04 11:42:35 -0700, Peter Geoghegan wrote:
> On Fri, May 4, 2018 at 11:41 AM, Andres Freund <and...@anarazel.de> wrote:
> > Hi,
> >
> > Currently there's no convenient way to get a corrupted page (be it a
> > checksum failure, corrupted pd_upper/lo
functions, instead of a parameter.
It's unfortunately not entirely trivial to access specific blocks with
pg_read_binary_file() - one needs a query that deals with block sizes,
segment numbers, segment sizes etc...
Greetings,
Andres Freund
On 2018-05-04 11:53:25 -0700, Peter Geoghegan wrote:
> On Fri, May 4, 2018 at 11:46 AM, Andres Freund <and...@anarazel.de> wrote:
> > Could you expand on that? Are you envisioning an option to
> > ReadBufferExtended()? Because that's certainly not what I'm thinking of
>
1 - 100 of 8656 matches
Mail list logo