Re: [HACKERS] event trigger API documentation?

2013-04-17 Thread Dimitri Fontaine
Peter Eisentraut pete...@gmx.net writes: I'm specifically looking for C API documentation, along the lines of http://www.postgresql.org/docs/devel/static/trigger-interface.html. The current chapter on event triggers might as well be ripped out and folded into the CREATE EVENT TRIGGER

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-17 Thread Amit Kapila
On Monday, April 15, 2013 1:02 PM Florian Pflug wrote: On Apr14, 2013, at 17:56 , Fujii Masao masao.fu...@gmail.com wrote: At fast shutdown, after walsender sends the checkpoint record and closes the replication connection, walreceiver can detect the close of connection before receiving all

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-17 Thread Florian Pflug
On Apr17, 2013, at 12:22 , Amit Kapila amit.kap...@huawei.com wrote: Do you mean to say that as an error has occurred, so it would not be able to flush received WAL, which could result in loss of WAL? I think even if error occurs, it will call flush in WalRcvDie(), before terminating

Re: [HACKERS] TODO links broken?

2013-04-17 Thread Robert Haas
On Wed, Apr 17, 2013 at 12:21 AM, Stephen Scheck singularsyn...@gmail.com wrote: Many of the links in the TODO wiki page result in a page not found error. Is this page up-to-date? Can anything be inferred about the status of these items from the broken link? I think what we can infer is that

Re: [HACKERS] TODO links broken?

2013-04-17 Thread Magnus Hagander
On Wed, Apr 17, 2013 at 2:13 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 17, 2013 at 12:21 AM, Stephen Scheck singularsyn...@gmail.com wrote: Many of the links in the TODO wiki page result in a page not found error. Is this page up-to-date? Can anything be inferred about the

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-17 Thread Amit Kapila
On Wednesday, April 17, 2013 4:19 PM Florian Pflug wrote: On Apr17, 2013, at 12:22 , Amit Kapila amit.kap...@huawei.com wrote: Do you mean to say that as an error has occurred, so it would not be able to flush received WAL, which could result in loss of WAL? I think even if error occurs,

Re: [HACKERS] TODO links broken?

2013-04-17 Thread Robert Haas
On Wed, Apr 17, 2013 at 8:48 AM, Magnus Hagander mag...@hagander.net wrote: Yes. We can infer that. It makes it a whole lot easier to fix something with better bug repors than that, of course, as I'm sure you (Robert in this case, not Stephen) are generally aware of. I've reverted a patch

Re: [HACKERS] TODO links broken?

2013-04-17 Thread Magnus Hagander
On Wed, Apr 17, 2013 at 3:14 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 17, 2013 at 8:48 AM, Magnus Hagander mag...@hagander.net wrote: Yes. We can infer that. It makes it a whole lot easier to fix something with better bug repors than that, of course, as I'm sure you (Robert in

Re: [HACKERS] TODO links broken?

2013-04-17 Thread Robert Haas
On Wed, Apr 17, 2013 at 9:23 AM, Magnus Hagander mag...@hagander.net wrote: Hmm. Sorry for the lack of detail. I assumed the problem was obvious and widespread because I clicked on the first link I saw in the Todo and it didn't work. But after clicking a bunch more links from the Todo, I

Re: [HACKERS] [sepgsql 2/3] Add db_schema:search permission checks

2013-04-17 Thread Robert Haas
On Fri, Apr 12, 2013 at 2:44 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Yes, of course. The attached one replaces the getObjectDescription in sepgsql/proc.c, and relative changes in regression test. Thanks. Committed. I also committed the first two hunks of your cleanup patch but omitted the

Re: [HACKERS] event trigger API documentation?

2013-04-17 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: I'm not sure about ripping it out, it does not sound like a good idea to me. It needs some addition and C level examples yes. The plan was to build a contrib module as an example, that would cancel any (supported) command you try to run by means

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-04-17 Thread Robert Haas
On Tue, Jan 29, 2013 at 6:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 29, 2013 at 2:30 PM, Alvaro Herrera Robert, are you working on this? I wasn't, but I can, if we agree on it. I think we need to do *something* (and accordingly have

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Ants Aasma
On Wed, Apr 17, 2013 at 2:26 AM, Florian Pflug f...@phlo.org wrote: This raises two question. First, why are there two primes? You could just as well using a single prime q and set p=q^64 mod 2^16. You then get S = sum V[i,j] * q^(64*(64-i) + (64-j) = sum V[i,j] * q^(4096 - 64*(i-1) - j)

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Bruce Momjian
On Wed, Apr 17, 2013 at 05:47:55PM +0300, Ants Aasma wrote: The SSE4.1 implementation of this would be as fast as the last pat, generic version will be faster and we avoid the linearity issue. By using different offsets for each of the partial hashes we don't directly suffer from commutativity

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Florian Pflug
On Apr17, 2013, at 17:09 , Bruce Momjian br...@momjian.us wrote: As much as I love the idea of improving the algorithm, it is disturbing we are discussing this so close to beta, with an algorithm that is under analysis, with no (runtime) CPU detection, and in something that is going to be

Re: [HACKERS] event trigger API documentation?

2013-04-17 Thread Peter Eisentraut
On 4/17/13 5:41 AM, Dimitri Fontaine wrote: I'm not sure about ripping it out, it does not sound like a good idea to me. It needs some addition and C level examples yes. The plan was to build a contrib module as an example, that would cancel any (supported) command you try to run by means of

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Florian Pflug
On Apr17, 2013, at 16:47 , Ants Aasma a...@cybertec.at wrote: This made me remember, the issue I had was with high order bits, not with low order ones, somehow I got them confused. The exact issue is that the high order bits don't affect any bit lower than them. It's easy to see that if you

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Greg Stark
On Wed, Apr 17, 2013 at 4:28 PM, Florian Pflug f...@phlo.org wrote: Is there any way we can change the checksum algorithm in 9.4 *without* breaking pg_upgrade? Personally I think we're going to need a solution for page format changes someday eventually What advantages are we postponing now

[HACKERS] Changing schema on the fly

2013-04-17 Thread Daniele Varrazzo
Hello dear -hackers, I'm maintaining pg_reorg/pg_repack, which you may know effectively allow online VACUUM FULL or CLUSTER. It works by installing logging triggers to keep data up-to-date during the migration, creating a copy of the table, and eventually swapping the tables relfilenodes. The

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Bruce Momjian
On Wed, Apr 17, 2013 at 05:28:06PM +0200, Florian Pflug wrote: However, you're right that time's running out. It'd be a shame though if we'd lock ourselves into CRC as the only available algorithm essentially forever. Is there any way we can change the checksum algorithm in 9.4 *without*

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Florian Pflug
On Apr17, 2013, at 18:15 , Bruce Momjian br...@momjian.us wrote: On Wed, Apr 17, 2013 at 05:28:06PM +0200, Florian Pflug wrote: However, you're right that time's running out. It'd be a shame though if we'd lock ourselves into CRC as the only available algorithm essentially forever. Is there

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Bruce Momjian
On Wed, Apr 17, 2013 at 06:33:58PM +0200, Florian Pflug wrote: I was going to ask about the flexibility of pg_upgrade and checksums. Right now you have to match the old and new cluster checksum modes, but it seems it would be possible to allow pg_upgrade use from checksum to no-checksum

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Wed, Apr 17, 2013 at 4:28 PM, Florian Pflug f...@phlo.org wrote: Is there any way we can change the checksum algorithm in 9.4 *without* breaking pg_upgrade? Personally I think we're going to need a solution for page format changes someday eventually

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Bruce Momjian
On Wed, Apr 17, 2013 at 01:22:01PM -0400, Tom Lane wrote: Greg Stark st...@mit.edu writes: On Wed, Apr 17, 2013 at 4:28 PM, Florian Pflug f...@phlo.org wrote: Is there any way we can change the checksum algorithm in 9.4 *without* breaking pg_upgrade? Personally I think we're going to

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Uh, not sure how pg_upgrade would detect that as the version number is not stored in pg_controldata, e.g.: Data page checksums: enabled/disabled That seems pretty shortsighted. The field probably ought to be defined as containing

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-04-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 29, 2013 at 6:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think we need to do *something* (and accordingly have added this to the 9.3 open items page so we don't forget about it). Whether Robert's idea is the best one probably depends in

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Bruce Momjian
On Wed, Apr 17, 2013 at 01:29:18PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Uh, not sure how pg_upgrade would detect that as the version number is not stored in pg_controldata, e.g.: Data page checksums: enabled/disabled That seems pretty

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: On Wed, Apr 17, 2013 at 01:29:18PM -0400, Tom Lane wrote: But having said that, I'm not sure why this would be pg_upgrade's problem. By definition, we do not want pg_upgrade running around looking at individual data pages. Therefore, whatever we might

Re: [HACKERS] event trigger API documentation?

2013-04-17 Thread Dimitri Fontaine
Peter Eisentraut pete...@gmx.net writes: Well, if documentation had been available well before beta, other procedural languages might have gained support for event triggers. If it's not being documented, it might not happen very soon. It's been a moving target for the last two years, and

[HACKERS] Fix typo in contrib/hstore/crc32.c comment

2013-04-17 Thread Fabrízio de Royes Mello
Hi all, The attached patch fix a little typo on contrib/hstore/crc32.c comment. Regards, -- Fabrízio de Royes Mello Consultoria/Coaching PostgreSQL Blog sobre TI: http://fabriziomello.blogspot.com Perfil Linkedin: http://br.linkedin.com/in/fabriziomello Twitter:

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Jeff Davis
On Wed, 2013-04-17 at 12:42 -0400, Bruce Momjian wrote: AFAIK, there's currently no per-page checksum flag. Still, being only able to go from checksummed to not-checksummed probably is for all practical purposes the same as not being able to pg_upgrade at all. Otherwise, why would people

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Jeff Davis
On Wed, 2013-04-17 at 16:58 +0100, Greg Stark wrote: On Wed, Apr 17, 2013 at 4:28 PM, Florian Pflug f...@phlo.org wrote: Is there any way we can change the checksum algorithm in 9.4 *without* breaking pg_upgrade? Personally I think we're going to need a solution for page format changes

Re: [HACKERS] event trigger API documentation?

2013-04-17 Thread Peter Eisentraut
On 4/17/13 3:20 PM, Dimitri Fontaine wrote: It would have been good to have at least one untrusted language with event trigger support, so that you can hook in external auditing or logging systems. With the existing PL/pgSQL support, the possible actions are a bit limited. Well, you do

Re: [HACKERS] [GENERAL] currval and DISCARD ALL

2013-04-17 Thread Marko Kreen
On Tue, Apr 16, 2013 at 05:09:19PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I think his point is why don't we clear currval() on DISCARD ALL? I can't think of a good reason we don't. Because we'd have to invent a new suboperation DISCARD SEQUENCES, for one thing, in

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Bruce Momjian
On Wed, Apr 17, 2013 at 01:59:12PM -0700, Jeff Davis wrote: On Wed, 2013-04-17 at 12:42 -0400, Bruce Momjian wrote: AFAIK, there's currently no per-page checksum flag. Still, being only able to go from checksummed to not-checksummed probably is for all practical purposes the same as not

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Ants Aasma
On Wed, Apr 17, 2013 at 6:54 PM, Florian Pflug f...@phlo.org wrote: On Apr17, 2013, at 16:47 , Ants Aasma a...@cybertec.at wrote: This made me remember, the issue I had was with high order bits, not with low order ones, somehow I got them confused. The exact issue is that the high order bits

Re: [HACKERS] [GENERAL] currval and DISCARD ALL

2013-04-17 Thread Tom Lane
Marko Kreen mark...@gmail.com writes: On Tue, Apr 16, 2013 at 05:09:19PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I think his point is why don't we clear currval() on DISCARD ALL? I can't think of a good reason we don't. Because we'd have to invent a new suboperation

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Tom Lane
Ants Aasma a...@cybertec.at writes: I was thinking about something similar too. The big issue here is that the parallel checksums already hide each other latencies effectively executing one each of movdqu/pmullw/paddw each cycle, that's why the N_SUMS adds up to 128 bytes not 16 bytes. The

Re: [HACKERS] [PATCH] Add \ns command to psql

2013-04-17 Thread Robert Haas
On Tue, Apr 16, 2013 at 5:40 AM, Colin 't Hart co...@sharpheart.org wrote: Here's a new version of a small patch to psql I'm using locally. It adds a command \ns to psql which is a shortcut to set the SEARCH_PATH variable. I've also added tab completion making this command much more useful.

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Florian Pflug
On Apr18, 2013, at 00:32 , Tom Lane t...@sss.pgh.pa.us wrote: Ants Aasma a...@cybertec.at writes: I was thinking about something similar too. The big issue here is that the parallel checksums already hide each other latencies effectively executing one each of movdqu/pmullw/paddw each cycle,

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Florian Pflug
On Apr17, 2013, at 23:44 , Ants Aasma a...@cybertec.at wrote: Performance results: Mul-add checksums: 12.9 bytes/s FNV-1a checksums: 13.5 bytes/s FNV-1a + srl-1: 7.4 bytes/s Detection rates: False positive rates: Add-mul FNV-1a FNV-1a + srl-1 Single bit flip:

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Ants Aasma
On Thu, Apr 18, 2013 at 1:32 AM, Tom Lane t...@sss.pgh.pa.us wrote: Ants Aasma a...@cybertec.at writes: I was thinking about something similar too. The big issue here is that the parallel checksums already hide each other latencies effectively executing one each of movdqu/pmullw/paddw each

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Ants Aasma
On Thu, Apr 18, 2013 at 2:25 AM, Florian Pflug f...@phlo.org wrote: On Apr17, 2013, at 23:44 , Ants Aasma a...@cybertec.at wrote: Performance results: Mul-add checksums: 12.9 bytes/s FNV-1a checksums: 13.5 bytes/s FNV-1a + srl-1: 7.4 bytes/s Detection rates: False positive rates:

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Greg Smith
On 4/17/13 6:32 PM, Tom Lane wrote: The more I read of this thread, the more unhappy I get. It appears that the entire design process is being driven by micro-optimization for CPUs being built by Intel in 2013. And that's not going to get anyone past review, since all the tests I've been

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Ants Aasma
On Thu, Apr 18, 2013 at 3:21 AM, Greg Smith g...@2ndquadrant.com wrote: On 4/17/13 6:32 PM, Tom Lane wrote: The more I read of this thread, the more unhappy I get. It appears that the entire design process is being driven by micro-optimization for CPUs being built by Intel in 2013. And

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Daniel Farina
On Wed, Apr 17, 2013 at 5:21 PM, Greg Smith g...@2ndquadrant.com wrote: Let me see if I can summarize where the messages flying by are at since you'd like to close this topic for now: -Original checksum feature used Fletcher checksums. Its main problems, to quote wikipedia, include that it

[HACKERS] confusing message about archive failures

2013-04-17 Thread Peter Eisentraut
When archive_command fails three times, it prints this message into the logs: transaction log file \%s\ could not be archived: too many failures This leaves it open what happens next. What will actually happen is that it will usually try again after 60 seconds or so, but the message indicates

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Greg Smith
On 4/17/13 8:56 PM, Ants Aasma wrote: Nothing from the two points, but the CRC calculation algorithm can be switched out for slice-by-4 or slice-by-8 variant. Speed up was around factor of 4 if I remember correctly...I can provide you with a patch of the generic version of any of the discussed

Re: [HACKERS] confusing message about archive failures

2013-04-17 Thread Jeff Janes
On Wednesday, April 17, 2013, Peter Eisentraut wrote: When archive_command fails three times, it prints this message into the logs: transaction log file \%s\ could not be archived: too many failures This leaves it open what happens next. What will actually happen is that it will usually

[HACKERS] Word-level bigrams/trigrams in tsvector

2013-04-17 Thread Alan Li
I'm wondering how I can store word-level bigrams/trigrams in a tsvector that I can query against. I was expecting the final query to match the air and return the one tuple to me. For instance: postgres=# create table docs (a tsvector); CREATE TABLE postgres=# insert into docs (a) values