Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Jeff Davis
that a prime number can be a good modulus for Fletcher-32. Perhaps we could use 251 instead of 255? That would make it less likely to miss a common form of hardware failure, although it would also reduce the number of possible checksums slightly (about 4% fewer than 2^16). I'm leaning toward this o

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Jeff Davis
is very cheap and works on large word sizes, but I'm not sure. > On another note, I think I found a bug with the current latest patch. Ugh. Great catch, thank you! Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to y

Re: [HACKERS] Enabling Checksums

2013-03-21 Thread Jeff Davis
ing pattern until we settle on something. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-03-22 Thread Jeff Davis
edia.org/wiki/Fowler%E2%80%93Noll%E2% 80%93Vo_hash_function explicitly mentions adapting FNV to a smaller size. That gives me a little more confidence. Do you have other links we should read about this approach, or possible weaknesses? Regards, Jeff Davis -- Sent via pgsql-hackers mailin

Re: [HACKERS] Enabling Checksums

2013-03-22 Thread Jeff Davis
we want to offer a transition path, since we no longer use header bits. Avoiding zero might help us there. Hopefully not necessary, but something we might find useful. Also, it would help us identify situations where the checksum is never set. Regards, Jeff Davis -- Sent via pgsql-ha

Re: [HACKERS] Default connection parameters for postgres_fdw and dblink

2013-03-22 Thread Jeff Davis
eplaced by the current user or something (similar to search_path). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Limiting setting of hint bits by read-only queries; vacuum_delay

2013-03-25 Thread Jeff Davis
ior, unless we provide an additional GUC to say whether SELECT causes a page to be marked soft-dirty or dirty. At least that would be a boolean though, and the finer-grained control can be over VACUUM. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] regression test failed when enabling checksum

2013-03-26 Thread Jeff Davis
will look more closely later. Regards, Jeff Davis *** a/src/backend/access/heap/rewriteheap.c --- b/src/backend/access/heap/rewriteheap.c *** *** 273,286 end_heap_rewrite(RewriteState state) /* Write the last page, if any */ if (state-&

Re: [HACKERS] Hash Join cost estimates

2013-03-29 Thread Jeff Davis
f qual evaluations? The only reason I can think of is to model the cost of comparing the hashes themselves. Also, searching the archives turns up at least one other, but I think I've seen more: http://www.postgresql.org/message-id/a82128a6-4e3b-43bd-858d-21b129f7b...@richrelevance.com Regar

Re: [HACKERS] Hash Join cost estimates

2013-03-29 Thread Jeff Davis
On Fri, 2013-03-29 at 16:37 -0400, Tom Lane wrote: > Jeff Davis writes: > > Yes, I have run into this issue (or something very similar). I don't > > understand why the bucketsize even matters much -- assuming few hash > > collisions, we are not actually evaluating the

Re: [HACKERS] Hash Join cost estimates

2013-03-31 Thread Jeff Davis
etsize numbers is estimating > peak memory consumption to gate whether we'll accept the plan at all, > rather than adding terms to the cost estimate. Sounds reasonable. Ideally, we'd have a way to continue executing even in that case; but that's a project by itself, and would m

Re: [HACKERS] regression test failed when enabling checksum

2013-04-01 Thread Jeff Davis
acuum may kick in sometimes? Thank you for testing! I will try to reproduce it, as well. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] regression test failed when enabling checksum

2013-04-03 Thread Jeff Davis
when the file is extended but the page is not written yet. So empty pages aren't necessarily an indication of corruption (though I'd like to fix that eventually, because sometimes zeroing is corruption). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] regression test failed when enabling checksum

2013-04-03 Thread Jeff Davis
could still be caused by the checksums patch, but it seems a little less likely now. The reason I say that is because it's a new page with tuples on it, so that means something in the insert/update path ended up not writing the FPI before writing the page. Regards, Jeff Davis --

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-03 Thread Jeff Davis
led sometimes before the heap page is marked dirty. I will examine a little more closely, but I expect that to require another patch. Not sure if that could explain this problem or not. Regards, Jeff Davis *** a/src/backend/access/heap/pruneheap.c --- b/src/backend/access/heap/pruneh

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-04 Thread Jeff Davis
er and re-acquire it in exclusive mode after we find out we'll need to do XLogInsert. It means that MarkBufferDirtyHint may end up blocking for longer, but would avoid the memcpy. I haven't really thought through the details. Regards, Jeff Davis -- Sent via pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-04 Thread Jeff Davis
c commits. It would only need to store a few bytes. Streaming replication mitigates the problem somewhat, by being a second place to write WAL. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-04 Thread Jeff Davis
uess the simpler > code ends up faster. I was thinking more about the WAL size, but I don't have a strong opinion. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-04 Thread Jeff Davis
he future. I'm not advocating for this particular solution, but I do concur with Jeff Janes that it's a little bit scary and that we can entertain some ideas about how to mitigate it. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Enabling Checksums

2013-04-05 Thread Jeff Davis
x27;t matter. Are you still looking into SIMD? Right now, it's using the existing CRC implementation. Obviously we can't change it after it ships. Or is it too late to change it already? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] Enabling Checksums

2013-04-05 Thread Jeff Davis
ry errors anyway. Of course it would be nice, but I would rather have more people using a slightly flawed algorithm than fewer using it because it has too great a performance impact. > Unless somebody tells me not to waste my time I'll go ahead and come > up with a workable patch by

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Jeff Davis
tests and walk through it with gdb. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Jeff Davis
we'd need to be a little careful when written-but-not-flushed WAL data makes it to disk, which could cause a false positive and may be a fairly common case. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-05 Thread Jeff Davis
eparate issue. Simon, should that patch be applied or are you waiting on the issue Jeff Janes raised to be resolved, as well? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-09 Thread Jeff Davis
and > codewise. I don't even see why we allow changing that while the server is on. Either the I/O system requires it for safety or not, right? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-09 Thread Jeff Davis
sets a hint on a non-standard buffer might easily miss the assumption that we have a standard buffer. Regards, Jeff Davis *** a/src/backend/access/hash/hash.c --- b/src/backend/access/hash/hash.c *** *** 287,293 hashgettuple(PG_FUNCTION_ARGS) /* * Since this can

Re: [HACKERS] Enabling Checksums

2013-04-09 Thread Jeff Davis
them or something). Does that mean that this is not suitable for WAL at all? Using SIMD for WAL is not a requirement at all; I just thought it might be a nice benefit for non-checksum-enabled users in some later release. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] Enabling Checksums

2013-04-10 Thread Jeff Davis
bly to be in a separate file (if the specialization makes it in 9.3). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-04-10 Thread Jeff Davis
the backuo block. I like this, but it sounds like there is some room for discussion on some of these points. I assume changes to the WAL checksums are 9.4 material? I'm satisfied with SIMD data checksums in 9.3 and that we have a plan for using SIMD for WAL checksums later. Regards,

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Jeff Davis
that the hole is empty; otherwise the checksum will fail when we re-expand it. It might be worth a memset beforehand just to be sure. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Jeff Davis
g zeros for the page hole later, then the page will not match the checksum any more. So, we need to be sure the original page hole is all-zero when we calculate the checksum. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To ma

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Jeff Davis
nyway, I would be perfectly happy if we just got the SIMD algorithm in for data pages. The support for changing the WAL checksums seems lukewarm, and there might be quite a few alternatives (e.g. optimizing the CRC for backup blocks as Heikki suggested) to achieve that performance goal. Regards,

Re: [HACKERS] Enabling Checksums

2013-04-12 Thread Jeff Davis
his right now. Personally, I'm satisfied having SIMD checksums on data pages now and leaving WAL optimization until later. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Jeff Davis
grade certainly doesn't need it. When we get to 9.4, if someone has checksums enabled and wants to disable it, why is pg_upgrade the right time to do that? Wouldn't it make more sense to allow them to do that at any time? Regards, Jeff Davis -- Sent via pgsql-hackers mailing

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Jeff Davis
d get that would be fast enough that many people would be able to use it. I expect it to be useful until we do decide to break the page format. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-04-18 Thread Jeff Davis
y using a different modulus: 251 vs 255. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-04-18 Thread Jeff Davis
On Thu, 2013-04-18 at 19:05 +0200, Florian Pflug wrote: > On Apr18, 2013, at 19:04 , Jeff Davis wrote: > > On Wed, 2013-04-17 at 20:21 -0400, Greg Smith wrote: > >> -Original checksum feature used Fletcher checksums. Its main problems, > >> to quote wikiped

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Jeff Davis
ll be missing out on a lot of potential checksum users for whom the existing CRC algorithm is just too slow. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Jeff Davis
bility). I'm not worried about the performance based on your analysis; nor am I worried about the error detection rate. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Jeff Davis
ecksum/checksum_fnv.c? Is there a clean way to override the compiler flags for a single file so we don't need to put it in its own directory? Regards, Jeff Davis *** a/src/backend/storage/page/bufpage.c --- b/src/backend/storage/page/bufpage.c *** *** 23,28 static cha

[HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-23 Thread Jeff Davis
README why there is a problem with plain FNV-1a, and why this modification is both effective and safe. I'd lean toward simplicity and closer adherence to the published version of the algorithm rather than detecting a few more obscure error patterns. It looks like the modification slows down the algo

Re: [HACKERS] Enabling Checksums

2013-04-23 Thread Jeff Davis
code -- e.g. 0 for no checksum, 1 for FNV-1a-SR3, etc. That would allow us to release 9.4 with a new algorithm without forcing existing users to change. initdb would have to take the code as an option, probably in string form. What do you think? Regards, Jeff Davis -- Sent via pg

Re: [HACKERS] Enabling Checksums

2013-04-24 Thread Jeff Davis
On Wed, 2013-04-24 at 08:20 +0100, Simon Riggs wrote: > On 24 April 2013 01:10, Jeff Davis wrote: > > I'd prefer that it was some kind of a checksum ID code -- e.g. 0 for no > > checksum, 1 for FNV-1a-SR3, etc. That would allow us to release 9.4 with > > a new algorith

Re: [HACKERS] Enabling Checksums

2013-04-24 Thread Jeff Davis
On Wed, 2013-04-24 at 21:09 +0100, Simon Riggs wrote: > On 24 April 2013 21:06, Jeff Davis wrote: > > > What goal are you trying to accomplish with this patch? > > That we might need to patch the checksum version on a production release. Oh, I see. I don't think we need

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-25 Thread Jeff Davis
com I'm not sure if others are waiting on me for a new patch or not. I can give the documentation issues a try, but I was hesitant to do so because you've done the research. The problems that I can correct are fairly trivial. Regards, Jeff Davis -- Sent via pgsql-hackers mailin

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Jeff Davis
r use, -msse4.1, but that can be specified by the user by configuring with CFLAGS_EXTRA="-msse4.1". I don't know of any more required changes, aside from documentation improvements. Regards, Jeff Davis fnv-jeff-20130426.patch Description: Binary data fnv-jeff-20130426-cfl

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Jeff Davis
and should be considered separately from checksums. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Jeff Davis
he FNV checksum algorithm is unchanged). Regards, Jeff Davis *** a/config/c-compiler.m4 --- b/config/c-compiler.m4 *** *** 242,247 undefine([Ac_cachevar])dnl --- 242,272 + # PGAC_PROG_CC_VAR_OPT + # --- + # Given a variable name and a strin

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix collation assignment for aggregates with ORDER BY.

2013-04-26 Thread Jeff Davis
the bug was introduced. It was explained in the commit message why it was not backported. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-30 Thread Jeff Davis
xtremely rare). Am I missing something? The reason I removed that special case was just code complexity/readability. I tried preserving the previous behavior, and it's not so bad, but it seemed unnecessarily ugly for the benefit of a rare case. Regards, Jeff Davis -- Sent via pgsql

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Jeff Davis
on how we'd lose so many VM bits at once, because they are logged. I see how we can lose one if the heap page makes it to disk before the VM bit's WAL is flushed, but that seems to be bounded to me. Regardless, you have a reasonable claim that my patch had effects that were not necessary

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Jeff Davis
not). I also explored the idea of passing more options to MarkBufferDirty so that we could force it to do what we want in each case, but decided against it: http://www.postgresql.org/message-id/1357798000.5162.38.camel@jdavis-laptop Regards, Jeff Davis -- Sent via pgsql-hackers mai

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-01 Thread Jeff Davis
es for the heap pages. I think we have to special-case that somehow, and I believe that's the conditional logic that Robert is referring to. That being said, there may be a simpler way to achieve the same result, and that may be what you have in mind. Regards, Jeff Davis -- Sent

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-03 Thread Jeff Davis
On Fri, 2013-05-03 at 19:52 +0100, Simon Riggs wrote: > On 1 May 2013 20:40, Jeff Davis wrote: > > >> Looks easy. There is no additional logic for checksums, so there's no > >> third complexity. > >> > >> So we either have > >> * cleanu

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-06 Thread Jeff Davis
On Mon, 2013-05-06 at 15:31 -0400, Robert Haas wrote: > On Wed, May 1, 2013 at 3:04 PM, Jeff Davis wrote: > > Regardless, you have a reasonable claim that my patch had effects that > > were not necessary. I have attached a draft patch to remedy that. Only > > rudimen

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-07 Thread Jeff Davis
posted a patch was not meant to advocate either way; merely to present the options. Not sure exactly what you mean about the code simplification, but I agree that anything more substantial than this patch should be left for 9.4. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-08 Thread Jeff Davis
ght hardware; but if we just write to disk that would be pretty bad. Ideas welcome. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Jeff Davis
orrupt, and it starts up too early, then that should be obvious when you try to reconnect streaming replication. I haven't tried it, but I'm assuming that it gives a useful error message. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Jeff Davis
7;t so bad; and combined with periodic fsync()s of the auxiliary file, should offer a lot of coverage against problems. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-10 Thread Jeff Davis
agree that it sounds unlikely that blocks 100 and 102 would be written, but not 101. But perhaps that's more likely in systems like ZFS where the physical blocks might be in very different places. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] Using pg_upgrade on log-shipping standby servers

2012-07-26 Thread Jeff Davis
x27;t help because the two systems would be diverged, and you'd need a new base backup anyway. And if they have played exactly the same WAL, what does this accomplish? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Covering Indexes

2012-07-27 Thread Jeff Davis
e will want to do similar things for exclusion constraints anyway, so it just seems duplicative. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Covering Indexes

2012-07-28 Thread Jeff Davis
more use cases where people want the flexibility of exclusion constraints and the speed of UNIQUE, we should look into it. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Covering Indexes

2012-07-31 Thread Jeff Davis
e will want to do similar things for exclusion constraints anyway, so it just seems duplicative. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 16-bit page checksums for 9.2

2012-08-10 Thread Jeff Davis
and maybe the OID. Regards, Jeff Davis - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-08-19 Thread Jeff Davis
r tests and they seemed to work. I haven't reviewed the spgist code changes in detail though. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-08-19 Thread Jeff Davis
at was just committed for Range Types SP-GiST is already using index-only scans. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] temporal support patch

2012-08-19 Thread Jeff Davis
M table. If there is some syntax that offers a convenient shorthand for WHERE, that's fine with me. Or using two tables, one called foo and one called foo_history, is also fine. But I don't want the DML syntax to introduce new mechanisms that aren't available without the fancy sy

Re: [HACKERS] gistchoose vs. bloat

2012-08-19 Thread Jeff Davis
assign 2 to the 'which' variable in the first iteration of the inner loop, before it realizes that it2 actually has a higher penalty. I think you need to finish out the inner loop and have a flag that indicates that all attributes are equal before you do the probabilistic replacement. Al

Re: [HACKERS] NOT NULL constraints in foreign tables

2012-08-20 Thread Jeff Davis
ce, and silently produce wrong results if it's false. Which are you proposing, and how will the user know which kind of constraint they've got? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] temporal support patch

2012-08-20 Thread Jeff Davis
On Mon, 2012-08-20 at 17:04 -0400, Robert Haas wrote: > On Sun, Aug 19, 2012 at 6:28 PM, Jeff Davis wrote: > >> The other issue is how to handle multiple changes of the same record > >> within the transaction. Should they be stored or not? > > > > In a typical aud

Re: [HACKERS] temporal support patch

2012-08-20 Thread Jeff Davis
having written some data, does the audit log still get updated? > I see the "user" element as having two components: I think this is essentially a good idea, although as I said in my other email, we should be careful how we label the application-supplied information in the audit log.

Re: [HACKERS] temporal support patch

2012-08-20 Thread Jeff Davis
se. That sounds like a good way to start. Actually, even before the tool, how about just some really good examples of triggers for specific kinds of audit logs, and some ways to run queries on them? I think that might settle a lot of these details. Regards, Jeff Davis -- Sent via pgsql

Re: [HACKERS] temporal support patch

2012-08-20 Thread Jeff Davis
sistent with the apparent order of execution Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] NOT NULL constraints in foreign tables

2012-08-20 Thread Jeff Davis
get better performance"? Maybe the user wants to get an error if some important assumption about the remote data source is not as true as when they declared the constraint. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes t

Re: [HACKERS] NOT NULL constraints in foreign tables

2012-08-22 Thread Jeff Davis
want. Normal constraints would be disallowed on foreign tables, but NOT ENFORCED ones would be allowed. That brings up another point: what if someone really, really, doesn't want to pay the overhead of enforcing their constraint on a local table, but wants the planner benefit? Would t

Re: [HACKERS] Audit Logs WAS: temporal support patch

2012-08-22 Thread Jeff Davis
the case you were talking about. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] temporal support patch

2012-08-22 Thread Jeff Davis
interested in the topic you brought up about serializability and system time (transaction time), because it would be a fundamental piece upon which we can build a lot of these other things (including what could be called an audit log). Regards, Jeff Davis -- Sent via pgsql-hack

[HACKERS] Minor comment fixups

2012-08-27 Thread Jeff Davis
I noticed a couple comments that look wrong to me. Patch attached. Regards, Jeff Davis *** a/src/backend/commands/tablecmds.c --- b/src/backend/commands/tablecmds.c *** *** 8784,8792 copy_relation_data(SMgrRelation src, SMgrRelation dst, pfree(buf); /* ! * If

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-06 Thread Jeff Davis
saction block[1], so there are some details to work out, but it might be a workable approach. Regards, Jeff Davis [1]: It seems like the only reason for that is so a multi-table vacuum doesn't hold locks for longer than it needs to, but that's not much of a concern in this case. -

[HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-08 Thread Jeff Davis
ity applies even to aborted transactions (though I haven't spent much time thinking about it), so users accustomed to other truly-serializable implementations might be surprised. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-10 Thread Jeff Davis
rding... I'll write something up. Can I document that you may depend on the results read in aborted subtransactions, or should I leave that undefined for now? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-10 Thread Jeff Davis
g problem because the window between PREPARE TRANSACTION and COMMIT PREPARED is supposed to be narrow (and if it's not, you have bigger problems anyway). As long as the window is narrow, than it's reasonable to retry the transaction T1, and expect it to succeed after a short interval. Regards,

Re: [HACKERS] gistchoose vs. bloat

2012-09-10 Thread Jeff Davis
? I'm inclined to just make it always randomize if it's better. I think it would be hard for a user to guess when it's better and when not. Maybe it's useful to turn randomization off for testing purposes, e.g. to ensure determinism? Regards, Jeff Davis -- Sent via

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-11 Thread Jeff Davis
obvious location: Attached. I thought about putting it as a "note", but it seems like it's easy to go overboard with those. Regards, Jeff Davis ssi_doc.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To m

[HACKERS] WIP checksums patch

2012-09-14 Thread Jeff Davis
files and local buffers * come up with a robust way to determine that all pages in the database are protected * protect uninitialized pages But not all of these will make the first patch. This is just a starting point to reignite the discussion. I welcome any feedback. Regards, Jeff

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-17 Thread Jeff Davis
On Sun, 2012-09-16 at 16:16 -0500, Kevin Grittner wrote: > I'm attaching an alternative proposal, with changes for the following > reasons: Looks good to me, aside from not wrapping the text. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Statistics and selectivity estimation for ranges

2012-09-30 Thread Jeff Davis
f rbound_bsearch_equal is a little confusing because it's looking for the highest index among distinct values, but the lowest index among identical values. Do you see a way to refactor these to be a little easier to understand? Regards, Jeff Davis -- Sent via pgsql-hackers mailing li

Re: [HACKERS] gistchoose vs. bloat

2012-09-30 Thread Jeff Davis
i - FirstOffsetNumber] = i; if (randomization) /* randomize offsets array */ for (i = 0; i < maxoff; i++) { offset = offsets[i]; ... } That's just an idea; if you think it's more readable as-is (or if I am misunderstanding) then let me know. Regards,

Re: [HACKERS] WIP checksums patch

2012-09-30 Thread Jeff Davis
On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: > This is just a rebased version of the patch by Simon here: I just noticed the following note in the docs for this patch: The default is off for backwards compatibility and to allow upgrade. The recommended setting is on though t

Re: [HACKERS] WIP checksums patch

2012-09-30 Thread Jeff Davis
On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: > * we might want to make it slightly easier for external utilities, like > for backup/replication, to verify the pages Ideally, PageVerificationInfoOK should be available to external utilities, so that someone might script a background

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
p/restore > upgrade, which would be fine. It would be interesting to have > pg_upgrade change this setting, or tell the user to change it. I am not > sure enough people are using pg_upgrade to change a default value. I still don't understand why pg_upgrade and page_checksums don

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
no checksum. How is that a reason for defaulting the GUC to off? I'm missing something here. Are we worried about users who turn the GUC on and then expect all of their old data pages to magically be protected? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: > This is just a rebased version of the patch by Simon here: > > http://archives.postgresql.org/message-id/CA > +u5nmkw_gbs6qq_y8-rjgl1v7mvw2hwbhartb8lojhnpfx...@mail.gmail.com Another thing I noticed about the design of this patc

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
d me a link? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] gistchoose vs. bloat

2012-10-21 Thread Jeff Davis
uot;is". Just a nitpick. Regarding my point 4 from the previous email, I mildly disagree with the style, but I don't see a correctness problem there. If the first two items are fixed, then the patch is fine with me. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-10-21 Thread Jeff Davis
On Tue, 2012-09-04 at 17:45 +0400, Alexander Korotkov wrote: > On Mon, Aug 20, 2012 at 12:25 AM, Jeff Davis > wrote: > I am taking a look at this patch now. A few quick comments: > > * It looks like bounds_adjacent modifies it's by-reference

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-11-04 Thread Jeff Davis
end the tree, but it doesn't look like it's used to reconstruct the value (because we already have the value anyway). Since it's being used for a purpose other than what's intended, that should be explained. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (p

[HACKERS] Arguments to foreign tables?

2012-11-04 Thread Jeff Davis
SRFs could just be a simplified special case of foreign tables. I looked for some previous discussions on this topic and nothing turned up. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresq

Re: [HACKERS] Arguments to foreign tables?

2012-11-05 Thread Jeff Davis
On Sun, 2012-11-04 at 15:13 -0500, Tom Lane wrote: > Jeff Davis writes: > > Is there any fundamental or philosophical reason why a foreign table > > can't accept arguments? > > That isn't a table; it's some sort of function. Now that we have > LATERAL, t

<    1   2   3   4   5   6   7   8   9   10   >