Re: [HACKERS] Cost limited statements RFC

2013-05-23 Thread Greg Smith
. That's a guess though. I'll have to think about this more when I circle back toward usability. Thanks for the implementation idea. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers

[HACKERS] Block write statistics WIP

2013-05-19 Thread Greg Smith
behavior to work properly strikes me as as a bit of a layering violation too. I'd like the read and write paths to have a similar API, but here they don't even operate on the same type of inputs. Addressing that is probably harder than just throwing a hack on the existing code though. -- Greg Smith

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-16 Thread Greg Smith
before its worth adding one for it. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Move unused buffers to freelist

2013-05-14 Thread Greg Smith
of your patch. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-13 Thread Greg Stark
to the application even though there's no need. But media errors can occur any time, even after the initial write so I don't think this should be a blocker. I think posix_fallocate is good enough for us and I would support using it. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-10 Thread Greg Stark
but not synced might have been written to disk and others not. There could be megabytes of correct WAL written with just one block in the middle of the first record not written. If no xlog sync had occurred (or one was in progress but not completed) then that's the correct end of WAL. -- greg

Re: [HACKERS] Concurrent HOT Update interference

2013-05-10 Thread Greg Stark
longer than we've had HOT. As far back as I can remember the recommendation was to run pgbench with fewer sessions than the scale factor. At times some people (Robert and Greg I think?) have used the reverse specifically to test contention but that's something programmers are concerned about

Re: [HACKERS] Concurrent HOT Update interference

2013-05-10 Thread Greg Stark
be an issue and the bloat becomes important. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-10 Thread Greg Stark
result in a physically inconsistent database with index pointers that point to incorrect tuples. Index scans would return tuples that didn't match the index or would miss tuples that should be returned. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-10 Thread Greg Smith
order until that point. All you can rely on is throwing up a stop sign that says tell me when all of them are done. In between those, you have no idea of the ordering. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www

[HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Greg Stark
involved... Neither seems intractable though. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Greg Stark
that would create an inconsistent database would not be missed. I may be missing cases involving checkpoints or the like though. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Greg Stark
massive false positives based on whether vacuum had happened to have run recently? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: [COMMITTERS] pgsql: Fix permission tests for views/tables proven empty by constraint

2013-05-07 Thread Greg Stark
| SELECT 1 + | | | WHERE false; (1 row) -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_dump --snapshot

2013-05-06 Thread Greg Stark
the same thing. But that's like calling pg_dump an incremental backup. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_dump --snapshot

2013-05-06 Thread Greg Stark
at that point in time. That's a pretty clear a parallel to what you propose here. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Commit subject line

2013-05-04 Thread Greg Stark
to implement a new feature to do xxx -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Remaining beta blockers

2013-05-03 Thread Greg Stark
parse the data without needing access to the full catalog for example. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] high io BUT huge amount of free memory

2013-05-03 Thread Greg Stark
in writing the code and has new ideas on how to solve these problems. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Remaining beta blockers

2013-05-03 Thread Greg Stark
transactional and it will make sense to solve that with something general that we can then depend on for lots of things. If I had to guess it would look more like a cached copy of the pg_class row or the whole relcache entry rather than an entirely independent structure. -- greg -- Sent via

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-05-01 Thread Greg Smith
interesting, but it's not necessarily relevant in many cases. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] Re: [BUGS] BUG #8128: pg_dump (= 9.1) failed while dumping a scheme named old from PostgreSQL 8.4

2013-05-01 Thread Greg Stark
a useful level of completeness. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Documentation epub format

2013-05-01 Thread Greg Stark
the Postgres docs are really that big. Surely a big chunk of it is just some reference material like tables of data or something? The other section of the docs that can reasonably be broken out imho is the man pages. But the rest really belong in a single document. -- greg -- Sent via pgsql

Re: [HACKERS] Back branches vs. gcc 4.8.0

2013-04-30 Thread Greg Stark
and structs would need a new flag. Whereas (2) makes the code pretty common traditional code that gcc is going to need to tolerate for the foreseeable future -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-30 Thread Greg Smith
. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com ChecksumMethods.xls Description: MS-Excel spreadsheet -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-30 Thread Greg Smith
without expecting any block differences, that will make both routine quality auditing and forensics of broken clusters so much easier. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql

Re: [HACKERS] Should pg_upgrade use --quote-all-identifiers?

2013-04-30 Thread Greg Stark
thought you were talking about quoting identifiers in an SQL dump. But you're not, you're talking about quoting identifiers in sql being sent to the server during the pg_dump process. Why did pg_dump ever not quote all such identifiers? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Remaining beta blockers

2013-04-30 Thread Greg Stark
generating wal traffic to update the pg_class entry. If they can satisfy the latter that gets a bit trickier. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Should pg_upgrade use --quote-all-identifiers?

2013-04-30 Thread Greg Stark
output one reason it's nice not to quote is that it makes it easier to use the SQL generated by --inserts on a non-postgres database. Mainly I'm thinking of the case issue but also some databases use strange quoting rules. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-26 Thread Greg Smith
considering regardless. The optimizations seem to have a very significant impact on the checksum feature, but I'd like to quantify how they change the code a little bit before even getting into that. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training

Re: [HACKERS] Enabling Checksums

2013-04-18 Thread Greg Stark
is quite likely to pass through checksum detection. I'm not sure about this -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Greg Stark
to avoid it? * 32-bit checksums? * Being able to enable/disable checksums? Anything else? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Greg Smith
have now dug into how exactly that and specific CPU optimization concerns impact the best approach for 8K database pages. This is very clearly a 9.4 project that is just getting started. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Greg Smith
(1), but I wouldn't give it 100% odds of happening either. The user demand that's motivated me to work on this will be happy with any of (1) through (3), and in two of them optimizing the 16 bit checksums now turns out to be premature. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [HACKERS] Enabling Checksums

2013-04-16 Thread Greg Stark
of the record not written. 1/64,000 power losses would then end up with an assertion failure or corrupt database. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PROPOSAL: tracking aggregated numbers from pg_stat_database

2013-04-16 Thread Greg Smith
. It's not a big difference, but thinking in that direction doesn't help your suggestion. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

[HACKERS] Re: [PATCH] Exorcise zero-dimensional arrays (Was: Re: Should array_length() Return NULL)

2013-04-08 Thread Greg Stark
in maintaining huge libraries of backward compatibility code and drastically limits their ability to keep making improvements. -- greg

Re: [HACKERS] unused code in float8_to_char , formatting.c ?

2013-04-08 Thread Greg Jaskiewicz
On 7 Apr 2013, at 05:14, Robert Haas robertmh...@gmail.com wrote: On Thu, Apr 4, 2013 at 6:47 PM, Greg Jaskiewicz gryz...@me.com wrote: Looking around the code Today, one of my helpful tools detected this dead code. As far as I can see, it is actually unused call to strlen() in formatting.c

Re: [HACKERS] Enabling Checksums

2013-04-05 Thread Greg Smith
, along with one of Seagate's new drives with a built-in BBWC. (Their latest SSHD flash hybrid model caches writes with a capacitor for clean shutdown on power loss) -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www

[HACKERS] unused code in float8_to_char , formatting.c ?

2013-04-05 Thread Greg Jaskiewicz
Hi Guys, Looking around the code Today, one of my helpful tools detected this dead code. As far as I can see, it is actually unused call to strlen() in formatting.c, float8_to_char(). Diff attached. formatting_dead_code.diff Description: Binary data -- GJ -- Sent via

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-04-03 Thread Greg Stark
linked list. Perhaps the right thing is to maintain a pool of buffers in some less contention-prone data structure which lets each backend pick buffers out more or less independently of the others. -- greg

Re: [HACKERS] Multi-pass planner

2013-04-03 Thread Greg Stark
but the sample will vary from run to run. (Or vice versa depending on the situation). -- greg

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-04-03 Thread Greg Smith
of these problems available all the time. What I did this week on that front was just go buy a 24 core server with 64GB of RAM that lives in my house. I just need to keep it two floors away if I want to sleep at night. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL

Re: [HACKERS] Patch for checking file parameters to psql before password prompt

2013-04-03 Thread Greg Smith
again later. I'll do the kicking myself now. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-04-02 Thread Greg Stark
I'm confused by this thread. We *used* to maintain an LRU. The whole reason for the clock-sweep algorithm is precisely to avoid maintaining a linked list of least recently used buffers since the head of that list is a point of contention. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-04-02 Thread Greg Smith
to being ready, and I hope you won't get discouraged just because it's probably going to slip to 9.4. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-04-01 Thread Greg Smith
for people who don't use the feature. It's not that much code, but it is going to take a good bit of committer level review to accept due to its risk. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via

[HACKERS] Proof of concept: using genetic algorithm to come up with most optimal PostgreSQL.conf

2013-03-27 Thread Greg Jaskiewicz
(Resending, I think google mail failed delivering it first time). Hi folks, I've always been fascinated with genetic algorithms. Having had a chance to implement it once before, to solve real life issue - I knew they can be brilliant at searching for right solutions in multi dimensional

Re: [HACKERS] Ideas for improving Concurrency Tests

2013-03-26 Thread Greg Stark
. Most bugs are likely to hang out in combinations we don't see in practice -- for instance having a tuple deleted and a new one inserted in the same slot in the time a different transaction was context switched out. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Limiting setting of hint bits by read-only queries; vacuum_delay

2013-03-25 Thread Greg Stark
On Mon, Mar 25, 2013 at 2:50 AM, Greg Smith g...@2ndquadrant.com wrote: The idea I was thinking about is refactoring the background writer's role in hint bit maintenance A good first step might be to separate the dirty bit into two bits. mandatory dirty and optional dirty. (Or maybe hard dirty

[HACKERS] Re: [PATCH] Exorcise zero-dimensional arrays (Was: Re: Should array_length() Return NULL)

2013-03-25 Thread Greg Stark
for 0. If there's a GUC then people need to code defensively without knowing which semantics they'll get. And if they test they need to test twice under both settings just in case the user's database has the other setting. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Limiting setting of hint bits by read-only queries; vacuum_delay

2013-03-25 Thread Greg Stark
more. To save a one-time i/o cost. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Limiting setting of hint bits by read-only queries; vacuum_delay

2013-03-25 Thread Greg Stark
? As you said we've been eyeing this particular logic since 2004, why did it suddenly become more urgent now? Why didn't you work on it 9 months ago at the beginning of the release cycle? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] [proof of concept] Evolving postgresql.conf using genetic algorithm

2013-03-24 Thread Greg Jaskiewicz
Hi folks, I've always been fascinated with genetic algorithms. Having had a chance to implement it once before, to solve real life issue - I knew they can be brilliant at searching for right solutions in multi dimensional space. Thinking about just the postgresql.conf and number of possible

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-03-24 Thread Greg Smith
that any tuning you do for one type of workload might accidentally slow another. Starting with a lot of baseline workloads is the only way to move usefully forward when facing that problem. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services

Re: [HACKERS] Limiting setting of hint bits by read-only queries; vacuum_delay

2013-03-24 Thread Greg Stark
with the hint bit set. And that's the total work caused. After that the record can be read without consulting the clog or anything else. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] Let's invent a function to report lock-wait-blocking PIDs

2013-03-24 Thread Greg Smith
they grabbed in?, that would solve the hardest of the questions I see in the field. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Limiting setting of hint bits by read-only queries; vacuum_delay

2013-03-24 Thread Greg Stark
On Sun, Mar 24, 2013 at 9:02 PM, Simon Riggs si...@2ndquadrant.com wrote: ...and as a result, the rest of your comments don't apply at all to the proposal. Sorry about that confusion. How do you figure that? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Limiting setting of hint bits by read-only queries; vacuum_delay

2013-03-24 Thread Greg Smith
tongue about the design on all these, and return to working on one of the patches already in the CF queue instead. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Strange Windows problem, lock_timeout test request

2013-03-22 Thread Greg Stark
of superscalar execution and multi-level caches things may be weirder than we're imagining. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-03-22 Thread Greg Stark
On Fri, Mar 22, 2013 at 2:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: And we definitely looked at ARC We didn't just look at it. At least one release used it. Then patent issues were raised (and I think the implementation had some contention problems). -- greg -- Sent via pgsql-hackers

Re: [HACKERS] dump, restore, dump yields differences

2013-03-22 Thread Greg Stark
be cool to have make check do this dance and intelligently compare the before and after. There have been more than one patch where you've caught omissions in pg_dump before applying. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-03-21 Thread Greg Smith
nothing changed. It should be as obvious as we can make it to someone that the explicit reload is necessary. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-03-21 Thread Greg Smith
the change. Maybe that's a NOTICE plus a HINT. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Enabling Checksums

2013-03-20 Thread Greg Stark
the cows come home and the fsync latency will hardly change. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Let's invent a function to report lock-wait-blocking PIDs

2013-03-20 Thread Greg Stark
it though. I would have expected something that took whole pg_lock row values or something like that. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
On 3/8/13 4:40 PM, Greg Stark wrote: On Fri, Mar 8, 2013 at 5:46 PM, Josh Berkus j...@agliodbs.com wrote: After some examination of the systems involved, we conculded that the issue was the FreeBSD drivers for the new storage, which were unstable and had custom source patches. However, without

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
to be a multiple of 1K is ever going to be an unreasonable limitation, if that's what it takes to get useful hardware acceleration. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
this efficiently toward the zlib developers, let them drop into assembly to get the best possible Intel acceleration etc. one day. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
CRC32, the zlib CRC32, and the Intel-derived versions Ants hacked on to compare. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
this committed with a safe option and then y'all can discuss the fine merits of each algorithm at leisure. Yes, that's what we're already doing--it just looks like work :) -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
itching to assemble a 24 core AMD box at home anyway, this gives me an excuse to pull the trigger on that. Thanks for the summary of how you view the zlib/libpng project state. I saw 4 releases from zlib in 2012, so it seemed possible development might still move forward there. -- Greg Smith

Re: [HACKERS] Enabling Checksums

2013-03-18 Thread Greg Smith
and manually fixed them, basically go on a hunt for torn pages and the last known good copy via full-page write. Without checksums, there really was nowhere to go with them except dump/reload. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7

Re: [HACKERS] Enabling Checksums

2013-03-18 Thread Greg Smith
summarizing how I use them for that tomorrow, just out of juice for that tonight. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Re: Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-03-17 Thread Greg Smith
way, seemingly just so it's easier to implement. Most of my customers now use tools like Puppet to manage their PostgreSQL configuration. I do not want to have this conversation: Greg: You can use SET PERSISTENT to modify settings instead of changing the postgresql.conf User: That's

Re: [HACKERS] Enabling Checksums

2013-03-17 Thread Greg Smith
like to do that with a CRC16 implementation or two. I'm not sure if it's possible to get a quicker implementation because the target is a CRC16, or whether it's useful to consider truncating a CRC32 into a CRC16. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL

Re: [HACKERS] Enabling Checksums

2013-03-17 Thread Greg Smith
generate during (and beyond) beta. I have my own stress tests I'll keep running too. If the bug rate from the beta adopters is bad and doesn't improve, there's is always the uncomfortable possibility of reverting it before the first RC. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

[HACKERS] Re: Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-03-12 Thread Greg Smith
On 3/11/13 12:19 PM, Greg Stark wrote: Think also about the case where someone wants to change multiple values together and having just some set and not others would be inconsistent. Isn't that an argument for syntax to make an exception though? If starting from a blank slate I would say

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-03-12 Thread Greg Stark
On Tue, Mar 12, 2013 at 9:06 AM, Greg Smith g...@2ndquadrant.com wrote: That's jumping right over a few rounds of simpler ways to do this, and just going right to the approach we know allows adding more such options later with minimal grammar impact. As Craig intimated, the minimal grammar

Re: [HACKERS] Re: Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-03-11 Thread Greg Smith
to be the slowest way, too, that is acceptable. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Btrfs clone WIP patch

2013-03-11 Thread Greg Smith
likely to be accepted if they're demonstrated to work on two OSes. That's much better evidence that the API for the OS specific work is a good one. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

Re: [HACKERS] [v9.3] writable foreign tables

2013-03-11 Thread Greg Stark
and omitted columns mean something else. Perhaps we should have an explicit LOCAL DEFAULT and REMOTE DEFAULT and then have DEFAULT and omitted columns both mean the same thing. This starts getting a bit weird if you start to ask what happens when the remote table is itself an FDW though -- greg

Re: [HACKERS] Using indexes for partial index builds

2013-03-11 Thread Greg Stark
be for concurrent ones. Concurrent index builds block vacuums already. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-03-11 Thread Greg Stark
On Mon, Mar 11, 2013 at 7:39 AM, Greg Smith g...@2ndquadrant.com wrote: I wasn't complaining that the change isn't instant. I understand that can't be done. But I think the signal to reload should be sent. If people execute SET PERSISTENT, and it doesn't actually do anything until

Re: [HACKERS] Re: Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-03-10 Thread Greg Smith
*also* happening at server start. But I could understand that other people might not like that. And having this pop up on every reload, appearing to a client next to another statement altogether, that isn't acceptable though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore

Re: [HACKERS] Btrfs clone WIP patch

2013-03-09 Thread Greg Smith
Regardless, you need to keep most of the structure to copydir anyway. Error handling, handling cancellation, and fsync calls are all vital things. You probably have to make the forked command copy a single file at a time to get the same interrupt handling behavior. -- Greg Smith 2ndQuadrant US

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-09 Thread Greg Stark
On Sat, Mar 9, 2013 at 10:32 AM, Dann Corbit dcor...@connx.com wrote: There is no such thing as a quicksort that never goes quadratic. It was formally proven The median of medians selection of the pivot gives you O(n*log(n)). -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-09 Thread Greg Stark
and then handles fewer cases than it should. Implementing a better introspection that detects all perverse cases and does so with a lower overhead than the current check is a fine idea. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-09 Thread Greg Stark
to make it part of a hybrid algorithm. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-03-08 Thread Greg Stark
round trip to the wal log which can require a sync. Is that right? That seems like a deal breaker to me. I would think an 0-10% i/o bandwidth or cpu bandwidth penalty would be acceptable but an extra rotational latency even just on some transactions would be a real killer. -- greg -- Sent via

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-08 Thread Greg Stark
apart. We do care about the average case as well as the worst-case. There's been a *ton* of research on sorting. I find it hard to believe there isn't a pretty solid consensus on which which of these defense mechanisms is the best trade-off. -- greg -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Enabling Checksums

2013-03-08 Thread Greg Smith
in the database will be valuable. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Request for vote to move forward with recovery.conf overhaul

2013-03-08 Thread Greg Smith
back to facing bit rot and remembering what was going on again, which is what killed the momentum toward committing this the last time. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql

Re: [HACKERS] Materialized views WIP patch

2013-03-06 Thread Greg Stark
the underlying data again. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-03-06 Thread Greg Smith
Linux with filesystem choices skewed toward conservative. Forget about the leading edge--I'd be happy if I could get one large customer to migrate off of ext3... -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www

Re: [HACKERS] Enabling Checksums

2013-03-06 Thread Greg Smith
, a moderately large lookup table. I don't see that there's any advantage to having all that baggage around if you're just going to throw away half of the result anyway. More on CRC32Cs in my next message. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL

Re: [HACKERS] Enabling Checksums

2013-03-06 Thread Greg Smith
as strongly though, as the timetime before we can expect a hardware accelerated version to be available is much further off than a Linux kernel developer's future. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www

Re: [HACKERS] Enabling Checksums

2013-03-06 Thread Greg Stark
for quite a long time. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Enabling Checksums

2013-03-06 Thread Greg Smith
were encountered here though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com attachment: clients-sets.pngattachment: scaling-sets.png Checksum-pgbench.xls Description: MS-Excel spreadsheet -- Sent via

Re: [HACKERS] Re: Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-03-06 Thread Greg Smith
beats me to it. I would find it very useful to me personally if this feature were committed, and we know there's plenty of user demand for it too. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent

<    4   5   6   7   8   9   10   11   12   13   >