Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
is that a reason for defaulting the GUC to off? I'm missing something here. Are we worried about users who turn the GUC on and then expect all of their old data pages to magically be protected? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: This is just a rebased version of the patch by Simon here: http://archives.postgresql.org/message-id/CA +u5nmkw_gbs6qq_y8-rjgl1v7mvw2hwbhartb8lojhnpfx...@mail.gmail.com Another thing I noticed about the design of this patch: It looks like

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Statistics and selectivity estimation for ranges

2012-09-30 Thread Jeff Davis
for the highest index among distinct values, but the lowest index among identical values. Do you see a way to refactor these to be a little easier to understand? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] gistchoose vs. bloat

2012-09-30 Thread Jeff Davis
) /* randomize offsets array */ for (i = 0; i maxoff; i++) { offset = offsets[i]; ... } That's just an idea; if you think it's more readable as-is (or if I am misunderstanding) then let me know. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] WIP checksums patch

2012-09-30 Thread Jeff Davis
On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: This is just a rebased version of the patch by Simon here: I just noticed the following note in the docs for this patch: The default is literaloff/ for backwards compatibility and to allow upgrade. The recommended setting is literalon

Re: [HACKERS] WIP checksums patch

2012-09-30 Thread Jeff Davis
On Fri, 2012-09-14 at 17:58 -0700, Jeff Davis wrote: * we might want to make it slightly easier for external utilities, like for backup/replication, to verify the pages Ideally, PageVerificationInfoOK should be available to external utilities, so that someone might script a background job

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-17 Thread Jeff Davis
On Sun, 2012-09-16 at 16:16 -0500, Kevin Grittner wrote: I'm attaching an alternative proposal, with changes for the following reasons: Looks good to me, aside from not wrapping the text. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

[HACKERS] WIP checksums patch

2012-09-14 Thread Jeff Davis
and local buffers * come up with a robust way to determine that all pages in the database are protected * protect uninitialized pages But not all of these will make the first patch. This is just a starting point to reignite the discussion. I welcome any feedback. Regards, Jeff Davis

Re: [HACKERS] gistchoose vs. bloat

2012-09-11 Thread Jeff Davis
it always randomize if it's better. I think it would be hard for a user to guess when it's better and when not. Maybe it's useful to turn randomization off for testing purposes, e.g. to ensure determinism? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-11 Thread Jeff Davis
: Attached. I thought about putting it as a note, but it seems like it's easy to go overboard with those. Regards, Jeff Davis ssi_doc.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-10 Thread Jeff Davis
truly-serializable implementations might be surprised. That's a fair point. Do you have any suggested wording... I'll write something up. Can I document that you may depend on the results read in aborted subtransactions, or should I leave that undefined for now? Regards, Jeff Davis

Re: [HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-10 Thread Jeff Davis
between PREPARE TRANSACTION and COMMIT PREPARED is supposed to be narrow (and if it's not, you have bigger problems anyway). As long as the window is narrow, than it's reasonable to retry the transaction T1, and expect it to succeed after a short interval. Regards, Jeff Davis -- Sent

[HACKERS] Question about SSI, subxacts, and aborted read-only xacts

2012-09-08 Thread Jeff Davis
haven't spent much time thinking about it), so users accustomed to other truly-serializable implementations might be surprised. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-06 Thread Jeff Davis
to work out, but it might be a workable approach. Regards, Jeff Davis [1]: It seems like the only reason for that is so a multi-table vacuum doesn't hold locks for longer than it needs to, but that's not much of a concern in this case. -- Sent via pgsql-hackers mailing list (pgsql-hackers

[HACKERS] Minor comment fixups

2012-08-27 Thread Jeff Davis
I noticed a couple comments that look wrong to me. Patch attached. Regards, Jeff Davis *** a/src/backend/commands/tablecmds.c --- b/src/backend/commands/tablecmds.c *** *** 8784,8792 copy_relation_data(SMgrRelation src, SMgrRelation dst, pfree(buf

Re: [HACKERS] NOT NULL constraints in foreign tables

2012-08-22 Thread Jeff Davis
their constraint on a local table, but wants the planner benefit? Would they have to make it a remote table to bypass the constraint check? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Audit Logs WAS: temporal support patch

2012-08-22 Thread Jeff Davis
. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] temporal support patch

2012-08-22 Thread Jeff Davis
(transaction time), because it would be a fundamental piece upon which we can build a lot of these other things (including what could be called an audit log). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] NOT NULL constraints in foreign tables

2012-08-20 Thread Jeff Davis
if it's false. Which are you proposing, and how will the user know which kind of constraint they've got? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] temporal support patch

2012-08-20 Thread Jeff Davis
On Mon, 2012-08-20 at 17:04 -0400, Robert Haas wrote: On Sun, Aug 19, 2012 at 6:28 PM, Jeff Davis pg...@j-davis.com wrote: The other issue is how to handle multiple changes of the same record within the transaction. Should they be stored or not? In a typical audit log, I don't see any

Re: [HACKERS] temporal support patch

2012-08-20 Thread Jeff Davis
log still get updated? I see the user element as having two components: I think this is essentially a good idea, although as I said in my other email, we should be careful how we label the application-supplied information in the audit log. Regards, Jeff Davis -- Sent via pgsql-hackers

Re: [HACKERS] temporal support patch

2012-08-20 Thread Jeff Davis
to start. Actually, even before the tool, how about just some really good examples of triggers for specific kinds of audit logs, and some ways to run queries on them? I think that might settle a lot of these details. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] temporal support patch

2012-08-20 Thread Jeff Davis
, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] NOT NULL constraints in foreign tables

2012-08-20 Thread Jeff Davis
if some important assumption about the remote data source is not as true as when they declared the constraint. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-08-19 Thread Jeff Davis
reviewed the spgist code changes in detail though. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-08-19 Thread Jeff Davis
committed for Range Types SP-GiST is already using index-only scans. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] temporal support patch

2012-08-19 Thread Jeff Davis
available without the fancy syntax (though new DDL arrangements might be fine). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] gistchoose vs. bloat

2012-08-19 Thread Jeff Davis
think you need to finish out the inner loop and have a flag that indicates that all attributes are equal before you do the probabilistic replacement. Also, I think you should use random() rather than rand(). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] 16-bit page checksums for 9.2

2012-08-10 Thread Jeff Davis
. Regards, Jeff Davis - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Covering Indexes

2012-07-31 Thread Jeff Davis
constraints anyway, so it just seems duplicative. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Covering Indexes

2012-07-28 Thread Jeff Davis
constraints and the speed of UNIQUE, we should look into it. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Covering Indexes

2012-07-27 Thread Jeff Davis
constraints anyway, so it just seems duplicative. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Using pg_upgrade on log-shipping standby servers

2012-07-26 Thread Jeff Davis
anyway. And if they have played exactly the same WAL, what does this accomplish? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Using pg_upgrade on log-shipping standby servers

2012-07-17 Thread Jeff Davis
this being pretty mechanically intensive, but right now my hands are completely tied as to achieving total continuity of my archives, costing a base-backup's worth of risk window upon upgrade. Does continuity of archives mean avoid downtime or maintain a single WAL sequence. Regards, Jeff

Re: [HACKERS] Using pg_upgrade on log-shipping standby servers

2012-07-16 Thread Jeff Davis
suggestion to run rsync before allowing the standby to rejoin the primary. Do you have plans to change that in the future? If we know that the user data files are identical between primary and replica, it would be nice if we could provide a robust way to avoid copying them. Regards, Jeff Davis

Re: [HACKERS] initdb and fsync

2012-07-13 Thread Jeff Davis
use of sync_file_range? That part of the email was less than conclusive, and I can't remember exactly what point I was trying to make. sync_file_range is a big practical win for the reasons I mentioned above (and also avoids some unpleasant noises coming from the disk drive). Regards, Jeff

Re: [HACKERS] initdb and fsync

2012-07-13 Thread Jeff Davis
it in, and fadvise is essentially a no-op. sync_file_range waits for room in the queue, which is (based on my tests) enough to improve the scheduling a lot. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-07-03 Thread Jeff Davis
, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-07-03 Thread Jeff Davis
On Mon, 2012-07-02 at 23:47 -0700, Jeff Davis wrote: On Thu, 2012-06-14 at 02:56 +0400, Alexander Korotkov wrote: Hackers, attached patch implements quad-tree on ranges. Some performance results in comparison with current GiST indexing. Index creation is slightly slower. Probably

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-07-03 Thread Jeff Davis
On Mon, 2012-07-02 at 23:47 -0700, Jeff Davis wrote: * Perhaps I'm mistaken, but the following code in getQuadrant() looks wrong to me, shouldn't the 1 and 2 be reversed? if (range_cmp_bounds(typcache, upper, centroidUpper) = 0) return 1; else return

Re: [HACKERS] GiST subsplit question

2012-06-26 Thread Jeff Davis
. I'm happy to go demote the message if we have consensus on that, but somebody else is going to need to provide the doc patch. Any takers? I was planning to do that, but it won't be for a few days at least. If someone else wants to do it sooner, feel free. Regards, Jeff Davis -- Sent

Re: [HACKERS] initdb and fsync

2012-06-23 Thread Jeff Davis
to be in a less-controlled pattern, so it might perform more poorly. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-06-22 Thread Jeff Davis
can be confused with or ). I don't have a suggestion yet how to clarify that, but it might be good to use the spatial terminology in more places and avoid lower/upper except where needed. Please excuse the slow review, I'm catching up on the SP-GiST API. Regards, Jeff Davis -- Sent

Re: [HACKERS] initdb and fsync

2012-06-19 Thread Jeff Davis
, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] temporal support patch

2012-06-18 Thread Jeff Davis
Range Types, which would help in a lot of ways (like making the SELECT queries a lot simpler and faster). 2. It's missing a lot of options, like storing the user that modified a row or the changed columns. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] initdb and fsync

2012-06-18 Thread Jeff Davis
to the readability, so I just commented where they came from. If you feel there will be a maintainability problem, I can give it another shot. - I personally would find it way nicer to put USE_PRE_SYNC into pre_sync_fname instead of cluttering the main function with it Done. Regards, Jeff Davis

Re: [HACKERS] initdb and fsync

2012-06-18 Thread Jeff Davis
postgres with fsync off. This is good, and I don't want to have more overhead in the tests. It's not like we already have awesome test coverage for OS interaction. OK, changing back to using -N during regression tests due to performance concerns. Regards, Jeff Davis initdb-fsync

Re: [HACKERS] initdb and fsync

2012-06-18 Thread Jeff Davis
). It would be possible to fallback to posix_fadvise which has been around far longer in that case... Interesting point, but I'm not too worried about it. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] initdb and fsync

2012-06-18 Thread Jeff Davis
into it. There may be performance implications when used one a larger amount of data though. I can do some brief testing. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

2012-06-17 Thread Jeff Davis
and possibly many other types (geometry?) as well. It looks like this is orthogonal from your work, but it does seem like it has potential for confusion in the future. Thoughts? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

2012-06-17 Thread Jeff Davis
, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] temporal support patch

2012-06-15 Thread Jeff Davis
. That's one reason to work on it as an extension first: we can get a better sense of the problem space and various use cases without worrying about violating any standard. Then, as you need specific backend support (e.g. special syntax), we can take the standards more seriously. Regards, Jeff

Re: [HACKERS] initdb and fsync

2012-06-13 Thread Jeff Davis
included in your patch, which would be a suitable place to discuss this briefly. Thank you, I added that. Regards, Jeff Davis initdb-fsync-20120613.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] 9.3: load path to mitigate load penalty for checksums

2012-06-12 Thread Jeff Davis
had in mind (if reads are expensive anyway, maybe people would rather just block). These aren't exclusive. Maybe we can do (c) in READ COMMITTED and above, and (a) in READ UNCOMMITTED. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] 9.3: load path to mitigate load penalty for checksums

2012-06-12 Thread Jeff Davis
many cases where losing atomicity is reasonable (although I'm sure there are some). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.3: load path to mitigate load penalty for checksums

2012-06-12 Thread Jeff Davis
selective index access, which is a substantial use case for the kind of large tables we'd be bulk-loading. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.3: load path to mitigate load penalty for checksums

2012-06-12 Thread Jeff Davis
we have a way to coalesce them later without rewriting. It's too bad filesystems don't allow us to just reallocate some blocks from one file to another. [ OK, I guess I just opened this discussion myself ;) ] Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] 9.3: load path to mitigate load penalty for checksums

2012-06-12 Thread Jeff Davis
, which gives a small performance gain on seqscans. If small means that it's something we can give up, then focusing on HEAP_XMIN_COMMITTED makes sense. But if we can't give it up, then we need to take it into account in the proposal. Regards, Jeff Davis -- Sent via pgsql-hackers mailing

Re: [HACKERS] 16-bit page checksums for 9.2

2012-06-12 Thread Jeff Davis
portions of the merge, and offers a reference point to see where his merge differs from mine. In the meantime, I'm looking at the original v10 patch against master as it existed when Simon posted the patch. I'd like to see checksums take part in the June commitfest. Regards, Jeff Davis

Re: [HACKERS] 9.3: load path to mitigate load penalty for checksums

2012-06-12 Thread Jeff Davis
HEAP_XMIN_COMMITTED (which seems much more costly to ignore). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] initdb and fsync

2012-06-12 Thread Jeff Davis
On Sun, 2012-03-25 at 19:59 -0700, Jeff Davis wrote: On Sat, 2012-03-17 at 17:48 +0100, Cédric Villemain wrote: I agree with Andres. I believe we should use sync_file_range (_before?) with linux. And we can use posix_fadvise_dontneed on other kernels. OK, updated patch attached

Re: [HACKERS] 9.3: load path to mitigate load penalty for checksums

2012-06-11 Thread Jeff Davis
that we might want to push other kinds of optimizations into the load path, and that might upset the interface you've described above. Then again, we'll probably need the normal, load, and transition states regardless, so maybe it's an empty concern. Regards, Jeff Davis -- Sent via pgsql

Re: [HACKERS] 9.3: load path to mitigate load penalty for checksums

2012-06-06 Thread Jeff Davis
On Wed, 2012-06-06 at 15:08 -0400, Robert Haas wrote: On Mon, Jun 4, 2012 at 9:26 PM, Jeff Davis pg...@j-davis.com wrote: Thoughts? Simon already proposed a way of doing this that doesn't require explicit user action, which seems preferable to a method that does require explicit user

[HACKERS] 9.3: load path to mitigate load penalty for checksums

2012-06-04 Thread Jeff Davis
the ShareUpdateExclusiveLock for too long. Also, we want to keep people from using it in the same transaction as the loading xact, because they might not realize that they would get a concurrency of 1 that way (because of the ShareUpdateExclusiveLock). Thoughts? Regards, Jeff Davis -- Sent

Re: [HACKERS] extending relations more efficiently

2012-05-31 Thread Jeff Davis
in larger chunks would probably be necessary. There are some challenges there, but I think it's worth pursuing. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

[HACKERS] GiST subsplit question

2012-05-30 Thread Jeff Davis
to avoid that problem? Or am I misunderstanding something? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Synchronized scans versus relcache reinitialization

2012-05-30 Thread Jeff Davis
when the table was on disk, so if you aren't, we should investigate. One thing we could do is drive the threshold from effective_cache_size rather than shared_buffers, which was discussed during 8.3 development. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] temporal support patch

2012-05-30 Thread Jeff Davis
commitfest ( http://commitfest.postgresql.org ), but this is a relatively large project, so it will most likely take several commitfest cycles. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] WIP: 2d-mapping based indexing for ranges

2012-05-30 Thread Jeff Davis
for the penalty function. In your last patch improving range indexing, it was quite awkward to express the penalty in one dimension. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] pg_upgrade libraries check

2012-05-29 Thread Jeff Davis
I'd bring it up. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Add primary key/unique constraint using prefix columns of an index

2012-05-29 Thread Jeff Davis
a second look. Either way, it seems like the deviations from normal UNIQUE would be better expressed in terms of an exclusion constraint, which offers a little more room in the language. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] RFC: Making TRUNCATE more MVCC-safe

2012-04-21 Thread Jeff Davis
On Sun, 2012-03-04 at 16:39 +, Simon Riggs wrote: v3 attached. Added to next commitfest. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-21 Thread Jeff Davis
here. If we do use WAL for hint bit updates, that has an impact on Hot Standby, because HS can't write WAL. So, it would seem that HS could not set hint bits. Comments? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] Re: 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-21 Thread Jeff Davis
On Sun, 2012-04-22 at 00:08 +0100, Greg Stark wrote: On Sat, Apr 21, 2012 at 10:40 PM, Jeff Davis pg...@j-davis.com wrote: * In addition to detecting random garbage, we also need to be able to detect zeroing of pages. Right now, a zero page is not considered corrupt, so that's a problem

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-19 Thread Jeff Davis
testing it. Right, I will need to come up with a prototype that can at least test the executor piece. I suspect that the plan choice won't be all that different from an ordinary index nestloop versus mergejoin case, but with much worse cardinality estimates to work with. Regards, Jeff Davis

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-19 Thread Jeff Davis
, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-19 Thread Jeff Davis
methods based on partitioning. I'm beginning to think that more than one of these ideas has merit. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-16 Thread Jeff Davis
On Mon, 2012-04-16 at 02:52 -0400, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes: 1. Order the ranges on both sides by the lower bound, then upper bound. Empty ranges can be excluded entirely. 2. Left := first range on left, Right := first range on right 3. If Left or Right

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-16 Thread Jeff Davis
hashtable). We may end up with some kind of combination (as the sweep seems to be; see below). Also there are methods which looks like extension of Jeff Davis proposal to the multidimensional case. It is Plane Sweep and External Plane Sweep methods. However, it might use sophisticated data structures

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-16 Thread Jeff Davis
function, am I right? Per my proposal, the problem statement is that it's slow, so speeding it up is really the entire proposal ;) Extending the syntax might be interesting as well, but I don't have a proposal for that. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-16 Thread Jeff Davis
On Mon, 2012-04-16 at 22:20 +0100, Simon Riggs wrote: On Mon, Apr 16, 2012 at 9:12 PM, Jeff Davis pg...@j-davis.com wrote: That had occurred to me, but I was hesitant to only use temp indexes. It still doesn't really offer a good solution when both sides of the join are relatively large

[HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-15 Thread Jeff Davis
? * The more general problem might be spatial joins which can operate in N dimensions, and I doubt this would work very well in that case. Does someone know of a spatial join algorithm (without IP claims) that would be as good as this one for ranges? * Other thoughts? Regards, Jeff Davis

Re: [HACKERS] Memory usage during sorting

2012-04-12 Thread Jeff Davis
what to do if we start merging a lot more runs and then determine that forcasting doesn't work as well as we'd hoped (e.g. that the data in the runs really is random). But I thought it was worth mentioning. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] bug in fast-path locking

2012-04-10 Thread Jeff Davis
On Mon, 2012-04-09 at 22:47 -0700, Jeff Davis wrote: but other similar paths do: if (!proclock) { AbortStrongLockAcquire(); I don't think it's necessary outside of LockErrorCleanup(), right? I take that back, it's necessary for the dontwait case, too. Regards, Jeff Davis

Re: [HACKERS] Last gasp

2012-04-10 Thread Jeff Davis
should continue to have a mechanism to review patches that aren't ready for commit, though I'm fine if we change it. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] bug in fast-path locking

2012-04-09 Thread Jeff Davis
on the error path (in this case, decrement the count). That was the source of this bug. There may be a way around this problem, but nothing occurs to me right now. Regards, Jeff Davis PS: Oops, I missed this bug in the review, too. PPS: In the README, FastPathStrongRelationLocks

Re: [HACKERS] bug in fast-path locking

2012-04-09 Thread Jeff Davis
decremented it (which is normally at the time the lock is released); which is a cure worse than the disease. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] bug in fast-path locking

2012-04-09 Thread Jeff Davis
there could be some more asserts. There are three sites that decrement FastPathStrongRelationLocks, but in two of them StrongLockInProgress should always be NULL. Other than that, it looks good to me. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] pg_upgrade incorrectly equates pg_default and database tablespace

2012-04-06 Thread Jeff Davis
On Fri, 2012-03-30 at 13:11 -0700, Jeff Davis wrote: I confirmed this bug upgrading 9.1 to master, and that this patch fixes it. Thank you for the report! Patch looks good to me as well, with one very minor nitpick: the added comment is missing an apostrophe. Bruce, can you take a look

Re: [HACKERS] pg_upgrade incorrectly equates pg_default and database tablespace

2012-03-30 Thread Jeff Davis
this is attached. I confirmed this bug upgrading 9.1 to master, and that this patch fixes it. Thank you for the report! Patch looks good to me as well, with one very minor nitpick: the added comment is missing an apostrophe. Bruce, can you take a look at this? Regards, Jeff Davis -- Sent via pgsql

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-26 Thread Jeff Davis
to pg_cancel_backend(), and it might be worthwhile to add an ...and pg_terminate_backend() there. Other than that, it looks good to me. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] initdb and fsync

2012-03-25 Thread Jeff Davis
() is a fallback. Regards, Jeff Davis initdb-fsync-20120325.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-19 Thread Jeff Davis
introduces risk for the table scan. I would tend to agree that for low selectivity fraction and a very low limit (e.g. 1-3 in your example) and a large table, it doesn't seem like a good risk to use a table scan. I don't know how that should be modeled or implemented though. Regards, Jeff

Re: [HACKERS] initdb and fsync

2012-03-16 Thread Jeff Davis
a fairly rare operation for everyone except developers. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-16 Thread Jeff Davis
the safety of the assumption that rows are uniformly distributed. Just trying to follow along. You mean as the selectivity _increases_ the safety of the assumption that the rows are uniformly distributed decreases, right? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Storage Manager crash at mdwrite()

2012-03-16 Thread Jeff Davis
, and attach to the bgwriter process with gdb, and step through it. Then, you can see the exact path which causes the bgwriter to exit, and that will give you a better idea where the problem is. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] initdb and fsync

2012-03-15 Thread Jeff Davis
On Wed, 2012-03-14 at 10:26 +0100, Andres Freund wrote: On Wednesday, March 14, 2012 05:23:03 AM Jeff Davis wrote: On Tue, 2012-03-13 at 09:42 +0100, Andres Freund wrote: for recursively everything in dir: posix_fadvise(fd, POSIX_FADV_DONTNEED); for recursively everything in dir

Re: [HACKERS] Storage Manager crash at mdwrite()

2012-03-15 Thread Jeff Davis
something related to postgreSQL. Any ideas on what's going wrong? What code, exactly, did you change in md.c, and anywhere else in postgres? Are you linking in new libraries/code from somewhere into the postgres backend? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Storage Manager crash at mdwrite()

2012-03-15 Thread Jeff Davis
in the archives for previous discussions around C++. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

<    1   2   3   4   5   6   7   8   9   10   >