Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2013-01-18 Thread Jeff Davis
bad, but it could be better. I am trying to introduce a new way to do spatial joins which will perform better in more circumstances. For instance, we can't use an inner index if the input tables are actually subqueries, because we can't index a subquery. Regards, Jeff Davis -- Sent via

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2013-01-18 Thread Jeff Davis
at the datatype level, and PostGIS only has a couple data types, so I don't think it will be a lot of work. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
, rather than the complex one that exists without the patch. I suppose we could even try to bump the LSN without writing WAL somehow, but it doesn't seem worth reasoning through that (setting a VM bit is rare enough). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 10:39 -0800, Jeff Davis wrote: On Thu, 2013-01-17 at 15:25 +0530, Pavan Deolasee wrote: Now that I look at the patch, I wonder if there is another fundamental issue with the patch. Since the patch removes WAL logging for the VM set operation, this can happen: Thank

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
pinned. I think the bottleneck is elsewhere; although I am keeping the page pinned in this patch to prevent it from becoming a bottleneck. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
if it ever came up. However, the tests I did didn't show any problem there. The tests were somewhat noisy, so perhaps I was doing something wrong, but it didn't appear that looking at the VM page for every update was a bottleneck. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
not a fundamental issue. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-01-16 Thread Jeff Davis
if the PD_ALL_VISIBLE patch is committed first then it will make reviewing this patch easier. Regardless, the second patch to be committed will need to be rebased on top of the first. Regards, Jeff Davis replace-tli-with-checksums-20130116.patch.gz Description: GNU Zip compressed data

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-16 Thread Jeff Davis
Rebased patch attached. No significant changes. Regards, Jeff Davis rm-pd-all-visible-20130116.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] Enabling Checksums

2013-01-16 Thread Jeff Davis
, byte value, and what sort of operation to do: overwrite, AND, OR, XOR. I like XOR here because you can fix it just by running the program again. Oh, good idea. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Enabling Checksums

2013-01-10 Thread Jeff Davis
is optional. Another idea is to make the WAL action for visibilitymap_set have another item in the chain pointing to the heap buffer, and bump the heap LSN. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Enabling Checksums

2013-01-09 Thread Jeff Davis
On Tue, 2012-12-04 at 01:03 -0800, Jeff Davis wrote: For now, I rebased the patches against master, and did some very minor cleanup. I think there is a problem here when setting PD_ALL_VISIBLE. I thought I had analyzed that before, but upon review, it doesn't look right. Setting PD_ALL_VISIBLE

Re: [HACKERS] Enabling Checksums

2012-12-19 Thread Jeff Davis
On Tue, 2012-12-04 at 01:03 -0800, Jeff Davis wrote: 4. We need some general performance testing to show whether this is insane or not. I ran a few tests. Test 1 - find worst-case overhead for the checksum calculation on write: fsync = off bgwriter_lru_maxpages = 0 shared_buffers

Re: [HACKERS] Enabling Checksums

2012-12-18 Thread Jeff Davis
and then hint that it should be rebuilt. Agreed; this applies to any derived data. I don't think it will be very practical to keep a server running in this state forever, but it might give enough time to reach a suitable maintenance window. Regards, Jeff Davis -- Sent via pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-12-18 Thread Jeff Davis
corrupt blocks (of course, that's not implemented yet, so it's still a concern for now). And we can also have ways to do background/offline checksum verification with a separate utility. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-12-17 Thread Jeff Davis
more improvement so I can understand. I found it easier to reason in terms of horizontal and vertical lines, and which quadrants they crossed, and then work out the edge cases. I'm not sure if that reasoning was correct, but it seemed to make more sense to me. Regards, Jeff Davis

Re: [HACKERS] Enabling Checksums

2012-12-17 Thread Jeff Davis
the page if it's corrupt, depending on a GUC. That would at least allow sequential scans to (partially) work, which might be good enough for some data recovery situations. If a catalog index is corrupted, that could just be rebuilt. Haven't thought about the details, though. Regards, Jeff Davis

Re: [HACKERS] Doc patch to note which system catalogs have oids

2012-12-14 Thread Jeff Davis
in the description. Marking ready for committer. Regards, Jeff Davis *** a/doc/src/sgml/catalogs.sgml --- b/doc/src/sgml/catalogs.sgml *** *** 427,432 --- 427,439 tbody row + entrystructfieldoid/structfield/entry + entrytypeoid/type/entry

Re: [HACKERS] gistchoose vs. bloat

2012-12-14 Thread Jeff Davis
, however, and I didn't dig in much further because you've already done significant testing. Marking this one ready again. Regards, Jeff Davis gist_choose_bloat-0.5A.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] gistchoose vs. bloat

2012-12-14 Thread Jeff Davis
randomization. It's not clear what's been randomized. I'd prefer something like distribute_on_equal_penalty, although that's really long. Better ideas? I agree that randomization is vague, but I can't think of anything better. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Enabling Checksums

2012-12-14 Thread Jeff Davis
and then verifies that it's properly detected? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-12-13 Thread Jeff Davis
still haven't reached consensus on the design here. Are we still discussing, or should this be moved to the next CF? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-12-13 Thread Jeff Davis
On Tue, 2012-11-27 at 14:24 -0800, Jeff Davis wrote: On Tue, 2012-11-27 at 15:41 -0500, Robert Haas wrote: I can't quite see how a non-overloaded flag would work, unless we get rid of schemas. It may work to pick the first schema in the search path that has any functions by that name

Re: [HACKERS] Statistics and selectivity estimation for ranges

2012-12-10 Thread Jeff Davis
is the last value? Is that a mistake? * I am still confused by the distinction between rbound_bsearch and rbound_bsearch_bin. What is the intuitive purpose of each? * You use constant value in the comments in several places. Would query value or search key be better? Regards, Jeff Davis

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-10 Thread Jeff Davis
some partial steps toward MVCC-safe access? For instance, how about trying to recheck the xmin of a pg_class tuple when starting a scan? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-12-10 Thread Jeff Davis
://archives.postgresql.org/message-id/1354055056.1766.50.camel@sussancws0025 For what it's worth, I'm glad that people like you are pushing on these usability issues, because it can be hard for insiders to see them sometimes. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-07 Thread Jeff Davis
On Thu, 2012-12-06 at 22:34 -0500, Stephen Frost wrote: * Jeff Davis (pg...@j-davis.com) wrote: That is documented in the committed patch -- it's a trade, basically saying that you lose isolation but avoid extra writes. It seems reasonable that the user gets this behavior if specifically

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-06 Thread Jeff Davis
commits, at which point the rows have committed, too (because this would only kick in when the rows are loaded in the same transaction as the CREATE). So, yes, it's like TRUNCATE in that regard. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-06 Thread Jeff Davis
in the future, we would have a problem if that caused new errors to appear. I think a WARNING might make more sense than a NOTICE, but I don't have a strong opinion about that. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-06 Thread Jeff Davis
lost in the process. So there may yet be some kind of safe protocol to set these even during a load into an existing table... Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] pg_upgrade problem with invalid indexes

2012-12-06 Thread Jeff Davis
indexes at all in --binary-upgrade mode. +1 Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-06 Thread Jeff Davis
, but there may be some more sophisticated approaches, as well. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade problem with invalid indexes

2012-12-06 Thread Jeff Davis
position, and I am fine with either approach. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-06 Thread Jeff Davis
. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Removing PD_ALL_VISIBLE

2012-12-05 Thread Jeff Davis
On Fri, 2012-11-30 at 13:16 -0800, Jeff Davis wrote: I tried for quite a while to show any kind of performance difference between checking the VM and checking PD_ALL_VISIBLE on a 12-core box (24 if you count HT). Three patches in question: 1. Current unpatched master 2. patch

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-05 Thread Jeff Davis
good, and the commit message seems appropriate (essentially, the code got ahead of the discussion). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-12-04 Thread Jeff Davis
for some reason. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-12-04 Thread Jeff Davis
working on tests here, so I will wait for his results. But this looks in good shape for commit otherwise. Great! For now, I rebased the patches against master, and did some very minor cleanup. Regards, Jeff Davis checksums-20121203.patch.gz Description: GNU Zip compressed data

Re: [HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-04 Thread Jeff Davis
On Tue, 2012-12-04 at 10:15 +, Simon Riggs wrote: On 4 December 2012 01:34, Jeff Davis pg...@j-davis.com wrote: I assume that refers to the discussion here: http://archives.postgresql.org/pgsql-hackers/2012-02/msg01177.php But that was quite a while ago, so is there a more recent

Re: [HACKERS] Enabling Checksums

2012-12-04 Thread Jeff Davis
On Tue, 2012-12-04 at 01:03 -0800, Jeff Davis wrote: 3. I think we need an explicit test of this feature (as you describe above), rather than manual testing. corruptiontester? I agree, but I'm not 100% sure how to proceed. I'll look at Kevin's tests for SSI and see if I can do something

[HACKERS] Commits 8de72b and 5457a1 (COPY FREEZE)

2012-12-03 Thread Jeff Davis
. There was some discussion about subtransactions, but those problems only seemed to appear when the CREATE and the INSERT/COPY happened in different subtransactions. So, I guess my question is, why the partial revert? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Removing PD_ALL_VISIBLE

2012-11-30 Thread Jeff Davis
no difference between #1 and #2 at all. I think someone with access to a larger box may need to test this. Or, if someone has a more specific suggestion about how I can create a worst case, then let me know. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-11-27 Thread Jeff Davis
. CREATE {UNIQUE|OVERLOADED} FUNCTION ... I'm not proposing that; in general I am very wary of changes to the type system. I'm just saying that, if we do have special rules, we should have a way to make sure that users know when the rules are changing. Regards, Jeff Davis -- Sent via

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-11-27 Thread Jeff Davis
be careful of these situations and not make them more likely than necessary. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-11-27 Thread Jeff Davis
and which function they intended to call. In such an extensible system, that worries me on several fronts. That being said, I'm not outright in opposition to the idea of making improvements like this, I just think we should do so cautiously. Regards, Jeff Davis -- Sent via pgsql-hackers

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-11-27 Thread Jeff Davis
of the problem cases without requiring any explicit user action. What user action are you concerned about? If we (eventually) made the non-overloaded case the default, would that resolve your concerns? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-11-27 Thread Jeff Davis
so at runtime based on the current contents of the catalog. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Removing PD_ALL_VISIBLE

2012-11-26 Thread Jeff Davis
though, so perhaps someone with a many-core box would be interested to test this out? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Removing PD_ALL_VISIBLE

2012-11-26 Thread Jeff Davis
it was last cleared. Callers that don't have the data buffer yet, such as an index only scan or a VACUUM that is skipping pages, must handle the concurrency as appropriate. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Enabling Checksums

2012-11-25 Thread Jeff Davis
primes, and not seeing any skew), and I didn't see any worrying patterns. Regards, Jeff Davis replace-tli-with-checksums-20121125.patch.gz Description: GNU Zip compressed data checksums-20121125.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Removing PD_ALL_VISIBLE

2012-11-25 Thread Jeff Davis
On Wed, 2012-11-21 at 18:25 -0800, Jeff Davis wrote: Follow up to discussion: http://archives.postgresql.org/pgsql-hackers/2012-11/msg00817.php I worked out a patch that replaces PD_ALL_VISIBLE with calls to visibilitymap_test. It rips out a lot of complexity, with a net drop of about 300

Re: [HACKERS] Do we need so many hint bits?

2012-11-21 Thread Jeff Davis
On Mon, 2012-11-19 at 14:46 -0500, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes: As I said elsewhere in the thread, I'm not planning to introduce any additional locking. There is already precedent in IndexOnlyNext. Of course, that just begs the question of whether the code

[HACKERS] Removing PD_ALL_VISIBLE

2012-11-21 Thread Jeff Davis
, Jeff Davis rm-pd-all-visible-20121121.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-19 Thread Jeff Davis
be quite hard to do properly. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-19 Thread Jeff Davis
, because garbage is more likely to make the LSN too high than too low). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Do we need so many hint bits?

2012-11-19 Thread Jeff Davis
(or mutexes) during a read of the VM bit, because there is concern that it could cause contention. If we lock the entire VM page, that represents many, many data pages, so it's worrisome. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Do we need so many hint bits?

2012-11-19 Thread Jeff Davis
... As I said elsewhere in the thread, I'm not planning to introduce any additional locking. There is already precedent in IndexOnlyNext. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Enabling Checksums

2012-11-19 Thread Jeff Davis
On Mon, 2012-11-19 at 10:35 -0800, Jeff Davis wrote: Yes, the blocks written *after* the checkpoint might have a bad checksum that will be fixed during recovery. But the blocks written *before* the checkpoint should have a valid checksum, but if they don't, then recovery doesn't know about

Re: [HACKERS] Do we need so many hint bits?

2012-11-18 Thread Jeff Davis
On Sun, 2012-11-18 at 15:19 +0100, Andres Freund wrote: On Sunday, November 18, 2012 03:07:01 AM Jeff Davis wrote: Process A (process that clears a VM bit for a data page): 1. Acquires exclusive lock on data buffer 2. Acquires exclusive lock on VM buffer 3. clears VM bit 4

Re: [HACKERS] Enabling Checksums

2012-11-18 Thread Jeff Davis
On Wed, 2012-11-14 at 17:40 -0800, Jeff Davis wrote: I'll do another pass to make sure I update all of the comments, and try to self review it. Updated patches attached (the TLI patch wasn't changed though, only the main checksums patch). Changes: * A lot of cleanup * More testing

Re: [HACKERS] Do we need so many hint bits?

2012-11-17 Thread Jeff Davis
using a buffer that we already have a pin on. I haven't really dug into this yet, but I don't see any obvious problem. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] Do we need so many hint bits?

2012-11-17 Thread Jeff Davis
On Sat, 2012-11-17 at 16:53 -0500, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes: What's the problem with that? If you already have the VM buffer pinned (which should be possible if we keep the VM buffer in a longer-lived structure), then doing the test is almost as cheap as checking

Re: [HACKERS] Do we need so many hint bits?

2012-11-17 Thread Jeff Davis
. If the whole page becomes visible then we can set the VM. Hmm, that's an idea. Maybe we shouldn't bother setting the hints if it's already all-visible in the VM? Something else to think about. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Do we need so many hint bits?

2012-11-16 Thread Jeff Davis
, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Do we need so many hint bits?

2012-11-16 Thread Jeff Davis
like a huge win after I thought about it, anyway. If HEAP_XMIN_INVALID or HEAP_XMAX_COMMITTED are set, it's likely to be dirty anyway, unless it's a bulk delete or something. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Do we need so many hint bits?

2012-11-16 Thread Jeff Davis
without a lot of performance risk. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Do we need so many hint bits?

2012-11-16 Thread Jeff Davis
On Fri, 2012-11-16 at 17:04 -0800, Jeff Janes wrote: On Thu, Nov 15, 2012 at 4:42 PM, Jeff Davis pg...@j-davis.com wrote: Also, I am wondering about PD_ALL_VISIBLE. It was originally introduced in the visibility map patch, apparently as a way to know when to clear the VM bit when doing

Re: [HACKERS] Enabling Checksums

2012-11-15 Thread Jeff Davis
in separate write() calls. So, the hazard of storing the checksum in a different place is not equivalent to the existing hazard of a torn page. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Materialized views WIP patch

2012-11-15 Thread Jeff Davis
a create-table-as-select except that it remembers the query. Would you say that there is a compelling use case for this alone, or is this a building block for more sophisticated materialized view support (e.g. eager updating) later? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Jeff Davis
need to keep those if your patch is accepted? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Do we need so many hint bits?

2012-11-15 Thread Jeff Davis
ways to mitigate those effects. What kind of worst-case tests could I construct to see if there are worrying performance effects to removing these hint bits? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Jeff Davis
in as a function pointer? It could either be an argument to the initialization, or we could make a new global variable that's present inside the server and outside. (Slight complication: palloc is a macro) Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-14 Thread Jeff Davis
a checkpoint. Is this a practical concern or am I borrowing trouble? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-14 Thread Jeff Davis
flag. * Got rid of page_checksums GUC. * Incorporated the page number into the checksum calculation, to detect pages that are transposed. I'll do another pass to make sure I update all of the comments, and try to self review it. So, slightly rough in some places. Regards, Jeff Davis

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Jeff Davis
the simplest approach that doesn't restrict these kinds of ideas if we want to do them later. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Jeff Davis
off a slow background VACUUM CHECKSUM job that might run in pieces. Right now I'm focused on the initial patch and other fairly immediate goals, so I won't address this now. But I don't want to cut off the conversation, either. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Pg_upgrade speed for many tables

2012-11-12 Thread Jeff Davis
(..., POSIX_FADV_DONTNEED) on everything else, and that made subsequent fsyncs more efficient. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-12 Thread Jeff Davis
it as simple as possible. We probably need to decide what to do there before 9.3 is released though. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jeff Davis
this in the future, we should reserve that magic value now. But I can't think of many reasons for it, unless we expect people to be turning checksums on and off repeatedly. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jeff Davis
. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jeff Davis
), or it could do so eagerly and turn it into a fully-protected instance. For the first patch, it might just be an initdb-time option for simplicity. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Enabling Checksums

2012-11-09 Thread Jeff Davis
aren't. We can have a couple bits in the header indicating that a checksum is present, but it's a little disappointing to have only a few bits protecting a 16-bit checksum. Also, I think that people will want to have a way to protect their old data somehow. Regards, Jeff Davis -- Sent via

Re: [HACKERS] Enabling Checksums

2012-11-09 Thread Jeff Davis
it as a system-wide value that's computed as the minimum of all database states (so it stays enabling until all databases have upgraded to on). That's a good point. Maybe this should be done as an offline operation using a command-line utility? Regards, Jeff Davis -- Sent via pgsql

Re: [HACKERS] Enabling Checksums

2012-11-09 Thread Jeff Davis
real foot-guns or other show-stoppers for permanently allowing that in-between-state? The biggest problem that I see is a few bits indicating the presence of a checksum may be vulnerable to more kinds of corruption. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] WIP checksums patch

2012-11-09 Thread Jeff Davis
for verifying the integrity of some of our most critical data (pg_clog) and be a useful building block toward a more complete implementation. That also breaks upgrade, right? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] WIP checksums patch

2012-11-09 Thread Jeff Davis
should focus on an initdb setting for the next commitfest and try for at least an offline migration tool (if not online) later. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] WIP checksums patch

2012-11-09 Thread Jeff Davis
filesystems do atomic writes, so the checksums should always be fine. What am I missing? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Enabling Checksums

2012-11-08 Thread Jeff Davis
of these into VACUUM options. Thoughts? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
side (e.g. twitter feed or something) and parameters are changing. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
On Tue, 2012-11-06 at 09:39 +0100, Dimitri Fontaine wrote: Jeff Davis pg...@j-davis.com writes: Take something as simple as generate_series: right now, it materializes the entire thing if it's in the FROM clause, but it wouldn't need to if it could use the foreign table mechanism. So, my

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
. It would be better if there were a way to pass the argument X to the FDW mechanism. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
remove the user-facing simplicity of SRFs. Granted, this is not very well thought-through from an API standpoint. But I was imagining adding something to the FDW API that allows for arguments to be passed from the FROM clause down the the FDW code. Regards, Jeff Davis -- Sent via pgsql

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
) then you could pass column lists. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Arguments to foreign tables?

2012-11-05 Thread Jeff Davis
On Sun, 2012-11-04 at 15:13 -0500, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes: Is there any fundamental or philosophical reason why a foreign table can't accept arguments? That isn't a table; it's some sort of function. Now that we have LATERAL, there is no good reason

Re: [HACKERS] Statistics and selectivity estimation for ranges

2012-11-05 Thread Jeff Davis
, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-11-04 Thread Jeff Davis
the value (because we already have the value anyway). Since it's being used for a purpose other than what's intended, that should be explained. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

[HACKERS] Arguments to foreign tables?

2012-11-04 Thread Jeff Davis
could just be a simplified special case of foreign tables. I looked for some previous discussions on this topic and nothing turned up. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] gistchoose vs. bloat

2012-10-21 Thread Jeff Davis
. Regarding my point 4 from the previous email, I mildly disagree with the style, but I don't see a correctness problem there. If the first two items are fixed, then the patch is fine with me. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2012-10-21 Thread Jeff Davis
On Tue, 2012-09-04 at 17:45 +0400, Alexander Korotkov wrote: On Mon, Aug 20, 2012 at 12:25 AM, Jeff Davis pg...@j-davis.com wrote: I am taking a look at this patch now. A few quick comments: * It looks like bounds_adjacent modifies it's by-reference arguments

Re: [HACKERS] WIP checksums patch

2012-10-01 Thread Jeff Davis
to have pg_upgrade change this setting, or tell the user to change it. I am not sure enough people are using pg_upgrade to change a default value. I still don't understand why pg_upgrade and page_checksums don't mix. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql

<    1   2   3   4   5   6   7   8   9   10   >