Re: [HACKERS] Hard to Use WAS: Hard limit on WAL space

On 15 June 2013 16:18, Craig Ringer wrote: > On 06/15/2013 02:08 PM, Brendan Jurd wrote: >> On 15 June 2013 14:43, Craig Ringer wrote: >>> The #1 question I see on Stack Overflow has to be confusion about >>> pg_hba.conf, mostly from people who have no idea it exists, don't understand >>> how to

Re: [HACKERS] Hard to Use WAS: Hard limit on WAL space

On 06/15/2013 02:08 PM, Brendan Jurd wrote: > On 15 June 2013 14:43, Craig Ringer wrote: >> The #1 question I see on Stack Overflow has to be confusion about >> pg_hba.conf, mostly from people who have no idea it exists, don't understand >> how to configure it, etc. > The totally non-obvious name

Re: [HACKERS] Hard to Use WAS: Hard limit on WAL space

On 15 June 2013 14:43, Craig Ringer wrote: > The #1 question I see on Stack Overflow has to be confusion about > pg_hba.conf, mostly from people who have no idea it exists, don't understand > how to configure it, etc. The totally non-obvious name of the file probably has something to do with that

Re: [HACKERS] [PATCH] Exorcise "zero-dimensional" arrays (Was: Re: Should array_length() Return NULL)

On 14 June 2013 03:53, David E. Wheeler wrote: > Similar things should have dissimilar names. I propose: > > > > Old |New > --+-- > array_dims | array_desc array_bounds? > array_ndims | array_depth > array_length | array_size > array_lower | array_s

Re: [HACKERS] Hard to Use WAS: Hard limit on WAL space

On 06/15/2013 02:16 AM, Josh Berkus wrote: > On 06/12/2013 02:03 PM, Joshua D. Drake wrote: >> What concerns me is we seem to be trying to make this "easy". It isn't >> supposed to be easy. This is hard stuff. Smart people built it and it >> takes a smart person to run it. When did it become a bad

Re: [HACKERS] MD5 aggregate

On 06/13/2013 05:35 PM, Dean Rasheed wrote: > Hi, > > Attached is a patch implementing a new aggregate function md5_agg() to > compute the aggregate MD5 sum across a number of rows. This is > something I've wished for a number of times. I think the primary use > case is to do a quick check that 2 t

Re: [HACKERS] MD5 aggregate

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/14/2013 09:40 PM, Stephen Frost wrote: > Where I'd take this is actually in a completely different direction.. > I'd like the aggregate to be able to match the results of running the > 'md5sum' unix utility on a file that's been COPY'd out. Unti

Re: [HACKERS] [PATCH] Add transforms feature

On 06/14/2013 11:11 AM, Peter Eisentraut wrote: > A transform is an SQL object that supplies to functions for converting > between data types and procedural languages. For example, a transform > could arrange that hstore is converted to an appropriate hash or > dictionary object in PL/Perl or PL/P

Re: [HACKERS] [PATCH] Remove useless USE_PGXS support in contrib

On 06/13/2013 11:16 AM, Peter Eisentraut wrote: > This has served no purpose except to > > 1. take up space > 2. confuse users > 3. produce broken external extension modules that take contrib as an example > 4. break builds of PostgreSQL when users try to fix 3. by exporting USE_PGXS Weak -1 from

Re: [HACKERS] pluggable compression support

On 06/14/2013 06:56 PM, Robert Haas wrote: On Fri, Jun 14, 2013 at 8:45 PM, Andres Freund wrote: On 2013-06-14 17:35:02 -0700, Josh Berkus wrote: No. I think as long as we only have pglz and one new algorithm (even if that is lz4 instead of the current snappy) we should just always use the

Re: [HACKERS] pluggable compression support

On Fri, Jun 14, 2013 at 8:45 PM, Andres Freund wrote: > On 2013-06-14 17:35:02 -0700, Josh Berkus wrote: >> >> > No. I think as long as we only have pglz and one new algorithm (even if >> > that is lz4 instead of the current snappy) we should just always use the >> > new algorithm. Unless I missed

Re: [HACKERS] dynamic background workers

On Sat, Jun 15, 2013 at 6:00 AM, Robert Haas wrote: > The second patch, dynamic-bgworkers-v1.patch, revises the background > worker API to allow background workers to be started dynamically. > This requires some communication channel from ordinary workers to the > postmaster, because it is the po

Re: [HACKERS] pluggable compression support

On 2013-06-14 17:35:02 -0700, Josh Berkus wrote: > > > No. I think as long as we only have pglz and one new algorithm (even if > > that is lz4 instead of the current snappy) we should just always use the > > new algorithm. Unless I missed it nobody seemed to have voiced a > > contrary position? >

Re: [HACKERS] pluggable compression support

> No. I think as long as we only have pglz and one new algorithm (even if > that is lz4 instead of the current snappy) we should just always use the > new algorithm. Unless I missed it nobody seemed to have voiced a > contrary position? > For testing/evaluation the guc seems to be sufficient. The

Re: [HACKERS] [RFC] Minmax indexes

On Fri, Jun 14, 2013 at 11:28 PM, Alvaro Herrera wrote: > Re-summarization is relatively expensive, because the complete page range has > to be scanned. That doesn't sound too bad to me. It just means there's a downside to having larger page ranges. I would expect the page ranges to be something

[HACKERS] [PATCH] Revive line type

Complete the implementations of line_in, line_out, line_recv, line_send. Remove comments and error messages about the line type not being implemented. Add regression tests for existing line operators and functions. --- This just revives existing functionality, doesn't add anything new. One thing

Re: [HACKERS] pluggable compression support

On 2013-06-14 17:12:01 -0700, Josh Berkus wrote: > On 06/14/2013 04:01 PM, Andres Freund wrote: > > It still contains a guc as described in the above message to control the > > algorithm used for compressing new tuples but I think we should remove > > that guc after testing. > > Did you add the st

Re: [HACKERS] pluggable compression support

On 06/14/2013 04:01 PM, Andres Freund wrote: > It still contains a guc as described in the above message to control the > algorithm used for compressing new tuples but I think we should remove > that guc after testing. Did you add the storage attribute? -- Josh Berkus PostgreSQL Experts Inc. htt

Re: [HACKERS] extensible external toast tuple support

On 2013-06-14 19:14:15 -0400, Alvaro Herrera wrote: > Andres Freund escribió: > > > Here's the updated version. It shouldn't contain any obvious WIP pieces > > anymore, although I think it needs some more documentation. I am just > > not sure where to add it yet, postgres.h seems like a bad place

Re: [HACKERS] extensible external toast tuple support

Andres Freund escribió: > Here's the updated version. It shouldn't contain any obvious WIP pieces > anymore, although I think it needs some more documentation. I am just > not sure where to add it yet, postgres.h seems like a bad place :/ How about a new file, say src/include/access/toast.h? --

Re: [HACKERS] [RFC] Minmax indexes

Josh Berkus writes: >> To avoid this, a table having a minmax index would be >> configured so that inserts only go to the page(s) at the end of the table; >> this >> avoids frequent invalidation of ranges in the middle of the table. We >> provide >> a table reloption that tweaks the FSM behavio

Re: [HACKERS] extensible external toast tuple support

On 2013-05-31 23:42:51 -0400, Robert Haas wrote: > On Thu, May 30, 2013 at 7:42 AM, Andres Freund wrote: > > In > > http://archives.postgresql.org/message-id/20130216164231.GA15069%40awork2.anarazel.de > > I presented the need for 'indirect' toast tuples which point into memory > > instead of a to

[HACKERS] GIN improvements part 3: ordering in index

Hackers, attached patch implementing ordering inside GIN index. This is third patch of GIN improvements, see previous two: http://www.postgresql.org/message-id/capphfduxv-il7aedwpw0w5fxrwgakfxijwm63_hzujacrxn...@mail.gmail.com http://www.postgresql.org/message-id/CAPpHfdvftaJq7www381naLw1=4u0h+qpx

Re: [HACKERS] [RFC] Minmax indexes

Alvaro, This sounds really interesting, and I can see the possibilities. However ... > Value changes in columns that are part of a minmax index, and tuple insertion > in summarized pages, would invalidate the stored min/max values. To support > this, each minmax index has a validity map; a range

Re: [HACKERS] stray SIGALRM

Richard Poole writes: > In 9.3beta1, a backend will receive a SIGALRM after authentication_timeout > seconds, even if authentication has been successful. Most of the time > this doesn't hurt anyone, but there are cases, such as when the backend > is doing the open() of a backend copy, when it brea

[HACKERS] GIN improvements part2: fast scan

Hackes, attached patch implementing "fast scan" technique for GIN. This is second patch of GIN improvements, see the 1st one here: http://www.postgresql.org/message-id/capphfduxv-il7aedwpw0w5fxrwgakfxijwm63_hzujacrxn...@mail.gmail.com This patch allow to skip parts of posting trees when their scan

[HACKERS] logical changeset generation v5

Hi! I am rather pleased to announce the next version of the changeset extraction patchset. Thanks to help from a large number of people I think we are slowly getting to the point where it is getting committable. Since the last submitted version (20121115002746.ga7...@awork2.anarazel.de) a large n

[HACKERS] stray SIGALRM

In 9.3beta1, a backend will receive a SIGALRM after authentication_timeout seconds, even if authentication has been successful. Most of the time this doesn't hurt anyone, but there are cases, such as when the backend is doing the open() of a backend copy, when it breaks things and results in an err

Re: [HACKERS] single-user vs standalone in docs and messages

On Thu, Jun 13, 2013 at 6:10 PM, Jeff Janes wrote: > Some places in the docs and elog hints refer to "standalone" backends, while > the official name as used in app-postgres.html is "single-user mode", and in > fact "standalone" does not appear on that page. > > This tries to standardize the other

[HACKERS] [RFC] Minmax indexes

Hi, This is a preliminary proposal for Minmax indexes. I'm experimenting with the code, but it's too crude to post yet, so here's a document explaining what they are and how they work, so that reviewers can poke holes to have the design improved. My intention is to have a patch to show for CF2,

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

I think that the weirdness really comes from the way transactions times are measured, their interactions with throttling, and latent bugs in the code. measurement times, no; interactions with throttling, no. If it was either of those I'd have finished this off days ago. Latent bugs, possibl

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

On 6/14/13 3:50 PM, Fabien COELHO wrote: I think that the weirdness really comes from the way transactions times are measured, their interactions with throttling, and latent bugs in the code. measurement times, no; interactions with throttling, no. If it was either of those I'd have finished

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

Hello Greg, I think that the weirdness really comes from the way transactions times are measured, their interactions with throttling, and latent bugs in the code. One issue is that the throttling time was included in the measure, but not the first time because "txn_begin" is not set at the

Re: [HACKERS] SPGist "triple parity" concept doesn't work

Teodor Sigaev writes: >> Anyway I now think that we might be better off with the other idea of >> abandoning an insertion and retrying if we get a lock conflict. > done, look at the patch. Looks good, committed with some cosmetic adjustments. > We definetly need new idea of locking protocol and

Re: [HACKERS] Hard to Use WAS: Hard limit on WAL space

On 06/14/2013 11:16 AM, Josh Berkus wrote: On 06/12/2013 02:03 PM, Joshua D. Drake wrote: What concerns me is we seem to be trying to make this "easy". It isn't supposed to be easy. This is hard stuff. Smart people built it and it takes a smart person to run it. When did it become a bad thing

Re: [HACKERS] Hard to Use WAS: Hard limit on WAL space

On 06/12/2013 02:03 PM, Joshua D. Drake wrote: > What concerns me is we seem to be trying to make this "easy". It isn't > supposed to be easy. This is hard stuff. Smart people built it and it > takes a smart person to run it. When did it become a bad thing to be > something that smart people need t

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

pgbench already has a "\sleep" command, and the way that delay is handled happens inside threadRun() instead. The pausing of the rate limit throttle needs to operate in the same place. It does operate at the same place. The throttling is performed by inserting a "sleep" first thing when sta

Re: [HACKERS] another error perhaps to be enhanced

On 06/14/2013 11:01 AM, Peter Geoghegan wrote: On Fri, Jun 14, 2013 at 10:54 AM, Joshua D. Drake wrote: Now, with the error previously shown, which one_idx needs to be reindexed? Well, you didn't show an actual error message. ERROR: index "foo_idx" Is not an error message? Granted I d

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Fri, 2013-06-14 at 13:21 -0400, Greg Smith wrote: > I'm planning to duplicate Jon's test program on a few machines here, and > then see if that turns into a useful latency improvement for clients. > I'm trying to get this pgbench rate limit stuff working first though, > because one of the tes

Re: [HACKERS] another error perhaps to be enhanced

On Fri, Jun 14, 2013 at 10:54 AM, Joshua D. Drake wrote: > Now, with the error previously shown, which one_idx needs to be reindexed? Well, you didn't show an actual error message. But if you "\set VERBOSITY verbose" within psql while connected to a 9.3 server, you'll get fully qualified details

Re: [HACKERS] another error perhaps to be enhanced

On 06/14/2013 10:47 AM, Peter Geoghegan wrote: I think you'll need to better describe what you mean here. postgres=# create schema foo; CREATE SCHEMA postgres=# create schema bar; CREATE SCHEMA postgres=# create table foo.foo(id serial); NOTICE: CREATE TABLE will create implicit sequence "f

Re: [HACKERS] another error perhaps to be enhanced

I think you'll need to better describe what you mean here. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] another error perhaps to be enhanced

On Jun 14, 2013, at 13:38, "Joshua D. Drake" wrote: > > ERROR: index "foo_idx" > > We should probably add the schema. I've noticed similar issues with functions. I'd like to see those schema-qualified as well. > > JD > -- > Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

I don't have this resolved yet, but I think I've identified the cause. Updating here mainly so Fabien doesn't duplicate my work trying to track this down. I'm going to keep banging at this until it's resolved now that I got this far. Here's a slow transaction: 1371226017.568515 client 1 exec

Re: [HACKERS] Add visibility map information to pg_freespace.

On Fri, Jun 14, 2013 at 7:23 AM, Andres Freund wrote: > 3). All the others seem to inflict unneccesary pain for not all that > much gain. +1. You might want to add a "historical note" about the name to the pg_freespace documentation, though. -- Peter Geoghegan -- Sent via pgsql-hackers mail

[HACKERS] another error perhaps to be enhanced

ERROR: index "foo_idx" We should probably add the schema. JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579 PostgreSQL Support, Training, Professional Services and Development High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc For my dreams of your image th

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Tue, 2013-06-11 at 12:58 -0400, Stephen Frost wrote: > My main question is really- would this be useful for extending > *relations*? Apologies if it's already been discussed; I do plan to go > back and read the threads about this more fully, but I wanted to voice > my support for using posix_fa

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On 6/14/13 1:06 PM, Jeff Davis wrote: Why have a GUC here at all? Perhaps this was already discussed, and I missed it? Is it just for testing purposes, or did you intend for it to be in the final version? You have guessed correctly! I suggested it stay in there only to make review benchmarkin

Re: [HACKERS] request a new feature in fuzzystrmatch

On Fri, Jun 14, 2013 at 10:14:14AM -0700, Joshua D. Drake wrote: > On 06/14/2013 10:11 AM, David Fetter wrote: > > >>>ok, thanks, I will wait. > >>Hi Joe, > >> > >>Do you have some time in the weekend to help me submit the patch? > >>Thanks, > >> > >>Liming > > > >Liming, > > > >Is your git skill

Re: [HACKERS] request a new feature in fuzzystrmatch

On 06/14/2013 10:11 AM, David Fetter wrote: ok, thanks, I will wait. Hi Joe, Do you have some time in the weekend to help me submit the patch? Thanks, Liming Liming, Is your git skill good enough to create a patch vs. PostgreSQL's git master? If so, send that and once it's hit the mailin

Re: [HACKERS] request a new feature in fuzzystrmatch

On Fri, Jun 14, 2013 at 10:08:24AM -0700, Liming Hu wrote: > On Tue, Jun 11, 2013 at 3:19 PM, Liming Hu wrote: > > On Tue, Jun 11, 2013 at 2:56 PM, Joe Conway wrote: > >> -BEGIN PGP SIGNED MESSAGE- > >> Hash: SHA1 > >> > >> On 06/11/2013 02:23 PM, Liming Hu wrote: > >>> On Tue, Jun 11, 20

Re: [HACKERS] request a new feature in fuzzystrmatch

On Tue, Jun 11, 2013 at 3:19 PM, Liming Hu wrote: > On Tue, Jun 11, 2013 at 2:56 PM, Joe Conway wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> On 06/11/2013 02:23 PM, Liming Hu wrote: >>> On Tue, Jun 11, 2013 at 1:57 PM, Alvaro Herrera >>> wrote: Liming Hu escribió: >

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

On Sat, 2013-05-25 at 13:55 -0500, Jon Nelson wrote: > Ack. I've revised the patch to always have the GUC (for now), default > to false, and if configure can't find posix_fallocate (or the user > disables it by way of pg_config_manual.h) then it remains a GUC that > simply can't be changed. Why h

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

Andres Freund writes: > On 2013-06-14 11:59:04 -0400, Tom Lane wrote: >> Ah, you are right, I forgot the #ifndef CHECKSUM_IMPL_H dance. Will fix >> in a bit. > That won't help against errors if it's included in two different > files/translation units though. Good point, but there's not any real

Re: [HACKERS] MD5 aggregate

On 14 June 2013 16:09, Hannu Krosing wrote: > What skytools/pgq/londiste uses for comparing tables on master > and slave is query like this > > select sum(hashtext(t.*::text)) from t; > > This is non-modulo sum and does not use md5 but relies on > whatever the hashtext() du jour is :) > > So it i

[HACKERS] pg_restore -l with a directory archive

Hi, When I ran pg_restore -l with the directory arhicve input, I found that its format is wrongly reported as UNKNOWN. $ pg_dump -F d -f hoge $ pg_restore -l hoge ; ; Archive created at Sat Jun 15 01:38:14 2013 ; dbname: postgres ; TOC Entries: 9 ; Compression: -1 ; Dump Version:

Re: [HACKERS] Patch for fail-back without fresh backup

On 2013-06-14 09:21:12 -0700, Jeff Davis wrote: > On Fri, 2013-06-14 at 16:10 +0200, Andres Freund wrote: > > Jeff Davis has a patch pending > > (1365493015.7580.3240.camel@sussancws0025) that passes the buffer_std > > flag down to MarkBufferDirtyHint() for exactly that reason. I thought we > > wer

Re: [HACKERS] Patch for fail-back without fresh backup

On Fri, 2013-06-14 at 16:10 +0200, Andres Freund wrote: > Jeff Davis has a patch pending > (1365493015.7580.3240.camel@sussancws0025) that passes the buffer_std > flag down to MarkBufferDirtyHint() for exactly that reason. I thought we > were on track committing that, but rereading the thread it do

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

On 2013-06-14 11:59:04 -0400, Tom Lane wrote: > Jeff Davis writes: > > I have a question about the commit though: shouldn't both functions be > > static if they are in a .h file? Otherwise, it could lead to naming > > conflicts. I suppose it's wrong to include the implementation file > > twice, bu

[HACKERS] refresh materialized view concurrently

Attached is a patch for REFRESH MATERIALIZED VIEW CONCURRENTLY for 9.4 CF1.  The goal of this patch is to allow a refresh without interfering with concurrent reads, using transactional semantics. It is my hope to get this committed during this CF to allow me to focus on incremental maintenance for

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

Jeff Davis writes: > I have a question about the commit though: shouldn't both functions be > static if they are in a .h file? Otherwise, it could lead to naming > conflicts. I suppose it's wrong to include the implementation file > twice, but it still might be confusing if someone tries. Two idea

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

On Thu, 2013-06-13 at 20:09 -0400, Tom Lane wrote: > What I propose we do about this is reduce backend/storage/page/checksum.c > to something like > > #include "postgres.h" > #include "storage/checksum.h" > #include "storage/checksum_impl.h" > > moving all the code currently in the file into a ne

Re: [HACKERS] MD5 aggregate

On 06/14/2013 04:47 PM, Tom Lane wrote: > Dean Rasheed writes: >> On 14 June 2013 14:14, Tom Lane wrote: >>> Personally I'd be a bit inclined to xor the per-row md5's rather than >>> sum them, but that's a small matter. >> But this would be a much riskier thing to do with a single column, >> beca

Re: [HACKERS] MD5 aggregate

On 2013-06-14 15:49:31 +0100, Dean Rasheed wrote: > On 14 June 2013 15:19, Stephen Frost wrote: > > * Andrew Dunstan (and...@dunslane.net) wrote: > >> I'd rather go the other way, processing the records without having > >> to process them otherwise at all. Turning things into text must slow > >> t

Re: [HACKERS] MD5 aggregate

On 14 June 2013 15:19, Stephen Frost wrote: > * Andrew Dunstan (and...@dunslane.net) wrote: >> I'd rather go the other way, processing the records without having >> to process them otherwise at all. Turning things into text must slow >> things down, surely. > > That's certainly an interesting idea

Re: [HACKERS] MD5 aggregate

Dean Rasheed writes: > On 14 June 2013 14:14, Tom Lane wrote: >> Personally I'd be a bit inclined to xor the per-row md5's rather than >> sum them, but that's a small matter. > But this would be a much riskier thing to do with a single column, > because if you updated multiple rows in the same w

Re: [HACKERS] Patch for fail-back without fresh backup

Greg Stark writes: > On Fri, Jun 14, 2013 at 2:21 PM, Tom Lane wrote: >> So it's not that we actually need to log the individual hint bit >> changes, it's that we need to WAL-log a full page image on the first >> update after a checkpoint, so as to recover from torn-page cases. >> Which one are w

Re: [HACKERS] Patch for fail-back without fresh backup

On Fri, Jun 14, 2013 at 2:21 PM, Tom Lane wrote: > So it's not that we actually need to log the individual hint bit > changes, it's that we need to WAL-log a full page image on the first > update after a checkpoint, so as to recover from torn-page cases. > Which one are we doing? Wal logging a fu

Re: [HACKERS] Add visibility map information to pg_freespace.

On 2013-06-14 10:22:19 -0400, Alvaro Herrera wrote: > Kyotaro HORIGUCHI wrote: > > Helle, > > > > I've added visibility map information to pg_freespace for my > > utility. > > This makes sense to me. +1 > I only lament the fact that this makes the > module a misnomer. Do we want to 1) rename t

[HACKERS] Buildfarm client 4.11 released

Version 4.11 of the PostgreSQL Buildfarm client has been released. It can be downloaded from Changes since 4.10: * Turn down module cleanup verbosity * Add check for rogue postmasters. * Add pseudo-branch targets HEAD_PLU

Re: [HACKERS] Add visibility map information to pg_freespace.

Kyotaro HORIGUCHI wrote: > Helle, > > I've added visibility map information to pg_freespace for my > utility. This makes sense to me. I only lament the fact that this makes the module a misnomer. Do we want to 1) rename the module (how inconvenient), 2) create a separate module for this (surely

Re: [HACKERS] MD5 aggregate

On 14 June 2013 14:14, Tom Lane wrote: > Marko Kreen writes: >> On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed >> wrote: >>> Attached is a patch implementing a new aggregate function md5_agg() to >>> compute the aggregate MD5 sum across a number of rows. > >> It's more efficient to calculate pe

Re: [HACKERS] MD5 aggregate

* Andrew Dunstan (and...@dunslane.net) wrote: > I'd rather go the other way, processing the records without having > to process them otherwise at all. Turning things into text must slow > things down, surely. That's certainly an interesting idea also.. Thanks, Stephen s

Re: [HACKERS] [PATCH] pgbench --throttle (submission 7 - with lag measurement)

On 6/12/13 3:19 AM, Fabien COELHO wrote: If you are still worried: if you run the very same command without throttling and measure the same latency, does the same thing happens at the end? My guess is that it should be "yes". If it is no, I'll try out pgbench-tools. It looks like it happens rar

Re: [HACKERS] Patch for fail-back without fresh backup

On 2013-06-14 16:58:38 +0300, Heikki Linnakangas wrote: > On 14.06.2013 16:15, Andres Freund wrote: > >On 2013-06-14 09:08:15 -0400, Tom Lane wrote: > >>I just had my nose in the part of the checksum patch that tediously > >>copies entire pages out of shared buffers to avoid possible instability >

Re: [HACKERS] Patch for fail-back without fresh backup

On 14.06.2013 17:01, Andres Freund wrote: At this point we have a page that has been modified without an FPI. But it's not marked dirty, so it won't be written out without further cause. Which might be fine since there's no cause to write out the page and there probably won't be anyone doing that

Re: [HACKERS] Patch for fail-back without fresh backup

On 2013-06-14 09:21:52 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > On 14.06.2013 16:08, Tom Lane wrote: > >> Refresh my memory as to why we need to WAL-log hints for checksumming? > > > Torn pages: > > So it's not that we actually need to log the individual hint bit > changes, it's t

Re: [HACKERS] MD5 aggregate

On 06/14/2013 09:40 AM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Marko Kreen writes: On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed wrote: Attached is a patch implementing a new aggregate function md5_agg() to compute the aggregate MD5 sum across a number of rows. It's m

Re: [HACKERS] Patch for fail-back without fresh backup

On 14.06.2013 16:15, Andres Freund wrote: On 2013-06-14 09:08:15 -0400, Tom Lane wrote: I just had my nose in the part of the checksum patch that tediously copies entire pages out of shared buffers to avoid possible instability of the hint bits while we checksum and write the page. I am really

Re: [HACKERS] MD5 aggregate

* Tom Lane (t...@sss.pgh.pa.us) wrote: > Marko Kreen writes: > > On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed > > wrote: > >> Attached is a patch implementing a new aggregate function md5_agg() to > >> compute the aggregate MD5 sum across a number of rows. > > > It's more efficient to calcula

Re: [HACKERS] Patch for fail-back without fresh backup

On 2013-06-14 09:21:52 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > On 14.06.2013 16:08, Tom Lane wrote: > >> Refresh my memory as to why we need to WAL-log hints for checksumming? > > > Torn pages: > > So it's not that we actually need to log the individual hint bit > changes, it's t

Re: [HACKERS] Patch for fail-back without fresh backup

On 14.06.2013 16:21, Tom Lane wrote: Heikki Linnakangas writes: On 14.06.2013 16:08, Tom Lane wrote: Refresh my memory as to why we need to WAL-log hints for checksumming? Torn pages: So it's not that we actually need to log the individual hint bit changes, it's that we need to WAL-log a

Re: [HACKERS] [PATCH] Remove useless USE_PGXS support in contrib

On 06/14/2013 08:35 AM, Peter Eisentraut wrote: On 6/13/13 9:20 PM, amul sul wrote: Agree, only if we consider these contrib module is always gonna deployed with the postgresql. But, what if user going to install such module elsewhere i.e. not from contrib directory of pg source. Why would a

Re: [HACKERS] Patch for fail-back without fresh backup

On 2013-06-14 09:08:15 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > Well, time will tell I guess. The biggest overhead with the checksums is > > exactly the WAL-logging of hint bits. > > Refresh my memory as to why we need to WAL-log hints for checksumming? > I just had my nose in the

Re: [HACKERS] MD5 aggregate

On Fri, Jun 14, 2013 at 2:14 PM, Tom Lane wrote: > Marko Kreen writes: > > On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed > wrote: > >> Attached is a patch implementing a new aggregate function md5_agg() to > >> compute the aggregate MD5 sum across a number of rows. > > > It's more efficient to

Re: [HACKERS] Patch for fail-back without fresh backup

Heikki Linnakangas writes: > On 14.06.2013 16:08, Tom Lane wrote: >> Refresh my memory as to why we need to WAL-log hints for checksumming? > Torn pages: So it's not that we actually need to log the individual hint bit changes, it's that we need to WAL-log a full page image on the first update a

Re: [HACKERS] Patch for fail-back without fresh backup

On 14.06.2013 16:08, Tom Lane wrote: Heikki Linnakangas writes: Well, time will tell I guess. The biggest overhead with the checksums is exactly the WAL-logging of hint bits. Refresh my memory as to why we need to WAL-log hints for checksumming? Torn pages: 1. Backend sets a hint bit, dirt

Re: [HACKERS] Patch for fail-back without fresh backup

On Friday, June 14, 2013 2:42 PM Samrat Revagade wrote: > Hello, > We have already started a discussion on pgsql-hackers for the problem of taking fresh backup during the failback operation here is the link for that:   > http://www.postgresql.org/message-id/CAF8Q-Gxg3PQTf71NVECe-6OzRaew5pWhk7yQtb

Re: [HACKERS] MD5 aggregate

Marko Kreen writes: > On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed > wrote: >> Attached is a patch implementing a new aggregate function md5_agg() to >> compute the aggregate MD5 sum across a number of rows. > It's more efficient to calculate per-row md5, and then sum() them. > This avoids th

Re: [HACKERS] Patch for fail-back without fresh backup

Heikki Linnakangas writes: > Well, time will tell I guess. The biggest overhead with the checksums is > exactly the WAL-logging of hint bits. Refresh my memory as to why we need to WAL-log hints for checksumming? I just had my nose in the part of the checksum patch that tediously copies entire p

Re: [HACKERS] [PATCH] Remove useless USE_PGXS support in contrib

On Fri, Jun 14, 2013 at 9:35 PM, Peter Eisentraut wrote: > On 6/13/13 9:20 PM, amul sul wrote: >> Agree, only if we consider these contrib module is always gonna deployed >> with the postgresql. >> But, what if user going to install such module elsewhere i.e. not from >> contrib directory of pg

Re: [HACKERS] [PATCH] Remove useless USE_PGXS support in contrib

On 6/13/13 9:20 PM, amul sul wrote: > Agree, only if we consider these contrib module is always gonna deployed with > the postgresql. > But, what if user going to install such module elsewhere i.e. not from > contrib directory of pg source. Why would anyone do that? -- Sent via pgsql-hackers

Re: [HACKERS] [PATCH] Add transforms feature

On 6/14/13 3:46 AM, Cédric Villemain wrote: > You kept PGXS style makefile... I know, but that's a separate issue that hasn't been decided yet. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-ha

Re: [HACKERS] SPGist "triple parity" concept doesn't work

Anyway I now think that we might be better off with the other idea of abandoning an insertion and retrying if we get a lock conflict. done, look at the patch. I was faced with the fact that my mail is considered spam by postgresql.org, so I repeat some hthoughts from previous mail: I consi

[HACKERS] Issue with PGC_BACKEND parameters

I had observed one problem with PGC_BACKEND parameters while testing patch for ALTER SYSTEM command. Problem statement: If I change PGC_BACKEND parameters directly in postgresql.conf and then do pg_reload_conf() and reconnect, it will still show the old value. Detailed steps 1.

Re: [HACKERS] MD5 aggregate

On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed wrote: > Attached is a patch implementing a new aggregate function md5_agg() to > compute the aggregate MD5 sum across a number of rows. This is > something I've wished for a number of times. I think the primary use > case is to do a quick check that

Re: [HACKERS] Patch for fail-back without fresh backup

On Fri, Jun 14, 2013 at 12:20 PM, Heikki Linnakangas wrote: > For pg_rewind, we wouldn't actually need a full-page image for hint bit > updates, just a small record saying "hey, I touched this page". And you'd > only need to write that the first time a page is touched after a checkpoint. I would

Re: [HACKERS] Patch for fail-back without fresh backup

On 14.06.2013 14:06, Pavan Deolasee wrote: On Fri, Jun 14, 2013 at 4:12 PM, Heikki Linnakangas wrote: Robert Haas pointed out in that thread that it has a problem with hint bits that are not WAL-logged, I liked that tool a lot until Robert pointed out the above problem. I thought this is a

Re: [HACKERS] Patch for fail-back without fresh backup

On Fri, Jun 14, 2013 at 2:51 PM, Benedikt Grundmann < bgrundm...@janestreet.com> wrote: > > A alternative proposal (which will probably just reveal my lack of > understanding about what is or isn't possible with WAL). Provide a way to > restart the master so that it rolls back the WAL changes tha

Re: [HACKERS] Patch for fail-back without fresh backup

On Fri, Jun 14, 2013 at 4:12 PM, Heikki Linnakangas wrote: > Robert Haas pointed out in that thread that it has a problem with hint > bits that are not WAL-logged, > I liked that tool a lot until Robert pointed out the above problem. I thought this is a show stopper because I can't really see a

  1   2   >