[HACKERS] Fast promotion, loose ends

2013-04-22 Thread Heikki Linnakangas
We never reached a consensus on the user interface of the new 'fast promotion'. We should settle that before beta. The thread died here: http://www.postgresql.org/message-id/CA+U5nMKmDD7hGCYzOo=iFM=eK5OPDXCEzmq79fgLWr0TJk=s...@mail.gmail.com Simon said in that email that he's waiting for furthe

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Simon Riggs
On 22 April 2013 08:13, Heikki Linnakangas wrote: > We never reached a consensus on the user interface of the new 'fast > promotion'. We should settle that before beta. The thread died here: > > http://www.postgresql.org/message-id/CA+U5nMKmDD7hGCYzOo=iFM=eK5OPDXCEzmq79fgLWr0TJk=s...@mail.gmail.co

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Heikki Linnakangas
On 22.04.2013 10:58, Simon Riggs wrote: On 22 April 2013 08:13, Heikki Linnakangas wrote: 2. There is no way to perform 'fast promotion' using the trigger file. That feature is only available using "pg_ctl promote". When "pg_ctl promote" was introduced, it was not meant to replace the trigger f

[HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-22 Thread Jeevan Chalke
Hi, I have observed that following sequence is causing server crash. CREATE MATERIALIZED VIEW temp_class_mv AS SELECT * FROM pg_class WITH NO DATA; CREATE OR REPLACE FUNCTION test_refresh_mv() RETURNS int AS $$ BEGIN REFRESH MATERIALIZED VIEW temp_class_mv; return 1; END; $$ LANGUAGE plpgs

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-22 Thread Andres Freund
On 2013-04-22 18:35:04 +0530, Jeevan Chalke wrote: > Hi, > > I have observed that following sequence is causing server crash. > > CREATE MATERIALIZED VIEW temp_class_mv AS > SELECT * FROM pg_class > WITH NO DATA; > > CREATE OR REPLACE FUNCTION test_refresh_mv() > RETURNS int > AS $$ > BEGIN

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-22 Thread Jeevan Chalke
On Mon, Apr 22, 2013 at 6:41 PM, Andres Freund wrote: > On 2013-04-22 18:35:04 +0530, Jeevan Chalke wrote: > > Hi, > > > > I have observed that following sequence is causing server crash. > > > > CREATE MATERIALIZED VIEW temp_class_mv AS > > SELECT * FROM pg_class > > WITH NO DATA; > > > > CRE

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Shaun Thomas
On 04/22/2013 02:58 AM, Simon Riggs wrote: So, to initiate promotion, you can create a file called $DATADIR/fast_promote or $DATADIR/promote Pardon my naiveté, but could it also be an option to read the method from the promotion file? echo "slow" > /my/promotion/path That would work without

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-04-22 Thread Robert Haas
On Wed, Apr 17, 2013 at 2:06 PM, Tom Lane wrote: > I think this breaks contrib/adminpack, and perhaps other extensions. > They'd not be hard to fix with script changes, but they'd be broken. > > In general, we would now have a situation where relocatable extensions > could never be installed into

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Robert Haas
On Wed, Apr 17, 2013 at 8:21 PM, Greg Smith wrote: >> The more I read of this thread, the more unhappy I get. It appears that >> the entire design process is being driven by micro-optimization for CPUs >> being built by Intel in 2013. > > And that's not going to get anyone past review, since all

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Andres Freund
On 2013-04-22 11:27:25 -0400, Robert Haas wrote: > On Wed, Apr 17, 2013 at 8:21 PM, Greg Smith wrote: > >> The more I read of this thread, the more unhappy I get. It appears that > >> the entire design process is being driven by micro-optimization for CPUs > >> being built by Intel in 2013. > > >

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Simon Riggs
On 22 April 2013 09:29, Heikki Linnakangas wrote: > Hmm. That requires write access to $DATADIR, so that's not quite the same > thing as the trigger_file recovery.conf option. Well, you also (elsewhere) requested that we must keep recovery.conf in $DATADIR, so it needs to be writable. -- Simon

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Simon Riggs
On 22 April 2013 16:09, Shaun Thomas wrote: > On 04/22/2013 02:58 AM, Simon Riggs wrote: > >> So, to initiate promotion, you can create a file called >> $DATADIR/fast_promote or $DATADIR/promote > > > Pardon my naiveté, but could it also be an option to read the method from > the promotion file? >

Re: [HACKERS] event trigger API documentation?

2013-04-22 Thread Robert Haas
On Wed, Apr 17, 2013 at 3:20 PM, Dimitri Fontaine wrote: > Please also note that the first series of patches did include the > support code for all the core PL, but Robert didn't feel like commiting > that and no other commiter did step up. Of course, the chances of that would have been a lot hig

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 6:27 PM, Robert Haas wrote: > On Wed, Apr 17, 2013 at 8:21 PM, Greg Smith wrote: >>> The more I read of this thread, the more unhappy I get. It appears that >>> the entire design process is being driven by micro-optimization for CPUs >>> being built by Intel in 2013. >> >

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 6:33 PM, Andres Freund wrote: > I don't see us changing away from CRCs anymore either by now. But I > think at least changing the polynom to something that > a) has higher error detection properties > b) can noticeably sped up on a a good part of the hardware pg is run on

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Alvaro Herrera
Bruce Momjian wrote: > I am not sure if Tom shared yet, but we are planning to package 9.3 > beta1 on April 29, with a release on May 2. Those dates might change, > but that is the current plan. I have completed a draft 9.3 release > notes, which you can view here: > > http://momjian.us/pg

[HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Hi, We are seeing some overall performance degradation in our application since we installed the security release. Other commits were also done at the same time in the application so we don't know yet if the degradation has any relationship with the security release. While we are digging int

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Josh Berkus
On 04/22/2013 10:38 AM, Anne Rosset wrote: > While we are digging into this, I would like to know if it is possible > that the release has some impact on performance. After reading this > "It was created as a side effect of a refactoring effort to make > establishing new connections to a Pos

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Josh Berkus
On 04/22/2013 10:38 AM, Anne Rosset wrote: > While we are digging into this, I would like to know if it is possible > that the release has some impact on performance. After reading this > "It was created as a side effect of a refactoring effort to make > establishing new connections to a Pos

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Josh Berkus
On 04/22/2013 09:25 AM, Ants Aasma wrote: > This leaves lingering doubts about the quality of the checksum. It's > hard if not impossible to prove absence of interesting patterns that > would trigger collisions. I do know the checksum quality is miles > ahead of the Fletcher sum originally proposed

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Heikki Linnakangas
On 22.04.2013 18:45, Simon Riggs wrote: On 22 April 2013 16:09, Shaun Thomas wrote: On 04/22/2013 02:58 AM, Simon Riggs wrote: So, to initiate promotion, you can create a file called $DATADIR/fast_promote or $DATADIR/promote Pardon my naiveté, but could it also be an option to read the met

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Florian Pflug
On Apr22, 2013, at 18:25 , Ants Aasma wrote: > I was just now writing up a generic C based patch based on the > parallel FNV-1a + shift that we discussed with Florian with an added > round of mixing. Testing the performance in isolation indicates that: > 1) it is about an order of magnitude faster

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Hi Josh, Thanks for your reply. I don't think this is the case since we are using jboss/jdbc driver with a connection pool. Thanks, Anne -Original Message- From: Josh Berkus [mailto:j...@agliodbs.com] Sent: Monday, April 22, 2013 10:58 AM To: Anne Rosset Cc: pgsql-hackers@postgres

Re: [HACKERS] minimizing the target list for foreign data wrappers

2013-04-22 Thread David Gudeman
Re-reading my first email I thought it was a little confusing, so here is some clarification. In GetForeignPlan, tlist seems to be a target list for a basic "select *" from the foreign table. For the ith TargetEntry te in tlist, it seems that te->expr is a var with varattno=i. I was mis-remembering

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 01:54:03PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > I am not sure if Tom shared yet, but we are planning to package 9.3 > > beta1 on April 29, with a release on May 2. Those dates might change, > > but that is the current plan. I have completed a draft 9.3 r

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Simon Riggs
On 22 April 2013 19:04, Heikki Linnakangas wrote: >> We could do that and a similar mechanism existed in pg_standby, but >> was removed in 9.0. > > > That code is still in pg_standby. Maybe you were thinking of some other > feature? It wasn't removed from pg_standby. But since all the features o

[HACKERS] libpq COPY handling

2013-04-22 Thread Robert Haas
Noah Misch pointed out something interesting to me: /* * PQputCopyEnd - send EOF indication to the backend during COPY IN * * After calling this, use PQgetResult() to check command completion status. * * Returns 1 if successful, 0 if data could not be sent (only possible * in nonblock mode),

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 9:04 PM, Florian Pflug wrote: > The one downside of the fnv1+shift approach is that it's built around > the assumption that processing 64-bytes at once is the sweet spot. That > might be true for x86 and x86_64 today, but it won't stay that way for > long, and quite surely

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Heikki Linnakangas
Allow tooling like pg_receivexlog to run on computers with different architectures (Heikki Linnakangas) This probably should be mentioned in the backwards-compatibility section. Any 3rd party tools that speak the streaming replication protocol are affected. E.1.3.2.1. Write-Ahead Log (WAL)

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Jeff Davis
On Mon, 2013-04-22 at 20:04 +0200, Florian Pflug wrote: > The one downside of the fnv1+shift approach is that it's built around > the assumption that processing 64-bytes at once is the sweet spot. That > might be true for x86 and x86_64 today, but it won't stay that way for > long, and quite surely

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Robert Haas
On Mon, Apr 22, 2013 at 3:14 PM, Jeff Davis wrote: > The biggest problem now is getting one of these faster algorithms (FNV > or even a faster CRC) into shape that is acceptable to > reviewers/committers. If we don't do that, we will be missing out on a > lot of potential checksum users for whom t

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Robert Haas
On Mon, Apr 22, 2013 at 2:33 PM, Bruce Momjian wrote: >> In this item >> Add support to event triggers (Dimitri Fontaine, Tom Lane) >> I am not sure why you list Tom. I think Robert should be listed >> instead. > > Tom did a massive fix/cleanup of that code. I have added Robert. I do not thin

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Jeff Davis
On Mon, 2013-04-22 at 19:25 +0300, Ants Aasma wrote: > I was just now writing up a generic C based patch based on the > parallel FNV-1a + shift that we discussed with Florian with an added > round of mixing. Testing the performance in isolation indicates that: > 1) it is about an order of magnitude

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Florian Pflug
On Apr22, 2013, at 21:14 , Jeff Davis wrote: > On Mon, 2013-04-22 at 20:04 +0200, Florian Pflug wrote: >> The one downside of the fnv1+shift approach is that it's built around >> the assumption that processing 64-bytes at once is the sweet spot. That >> might be true for x86 and x86_64 today, but

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Alvaro Herrera
Bruce Momjian wrote: > On Mon, Apr 22, 2013 at 01:54:03PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > I am not sure if Tom shared yet, but we are planning to package 9.3 > > > beta1 on April 29, with a release on May 2. Those dates might change, > > > but that is the current plan.

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Simon Riggs
On 22 April 2013 20:32, Florian Pflug wrote: > Assuming that we only ship a plain C implementation with 9.3, what > are we missing on that front? The C implementation of FNV1+SHIFT is > only a few dozen lines or so. Forgive me, I can't seem to locate the patch for this? Re-post please, just for

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 03:19:36PM -0400, Robert Haas wrote: > On Mon, Apr 22, 2013 at 2:33 PM, Bruce Momjian wrote: > >> In this item > >> Add support to event triggers (Dimitri Fontaine, Tom Lane) > >> I am not sure why you list Tom. I think Robert should be listed > >> instead. > > > > Tom d

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 10:54 PM, Simon Riggs wrote: > On 22 April 2013 20:32, Florian Pflug wrote: > >> Assuming that we only ship a plain C implementation with 9.3, what >> are we missing on that front? The C implementation of FNV1+SHIFT is >> only a few dozen lines or so. > > Forgive me, I can

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Steve Singer
On 13-04-22 01:38 PM, Anne Rosset wrote: Hi, We are seeing some overall performance degradation in our application since we installed the security release. Other commits were also done at the same time in the application so we don't know yet if the degradation has any relationship with the se

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 04:48:58PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Mon, Apr 22, 2013 at 01:54:03PM -0300, Alvaro Herrera wrote: > > > Bruce Momjian wrote: > > > > I am not sure if Tom shared yet, but we are planning to package 9.3 > > > > beta1 on April 29, with a release

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 10:11:48PM +0300, Heikki Linnakangas wrote: > >Allow tooling like pg_receivexlog to run on computers with different > >architectures (Heikki Linnakangas) > > This probably should be mentioned in the backwards-compatibility > section. Any 3rd party tools that speak the stre

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Hi Steve, Thanks for your reply. We are now running 9.0.13. Before it was 9.0.7. How can I find out if we are running into this issue: "ie if statistics are no longer being updated because analyze can't get the exclusive lock for truncation"? I will dig into our logs to see for the query times.

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Steve Singer
On 13-04-22 04:15 PM, Anne Rosset wrote: Hi Steve, Thanks for your reply. We are now running 9.0.13. Before it was 9.0.7. How can I find out if we are running into this issue: "ie if statistics are no longer being updated because analyze can't get the exclusive lock for truncation"? This issu

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Hi Steve, Yes I see these messages in our log. Is there a solution to this? Thanks, Anne -Original Message- From: Steve Singer [mailto:st...@ssinger.info] Sent: Monday, April 22, 2013 1:26 PM To: Anne Rosset Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Performance with the new

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Alvaro Herrera
Some more diacritics .. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services diff --git a/doc/src/sgml/release-9.3.sgml b/doc/src/sgml/release-9.3.sgml index 7c46bd3..68d04a7 100644 --- a/doc/src/sgml/release-9.3.sgml +++ b/doc/sr

[HACKERS] GSOC Student Project Idea

2013-04-22 Thread Michael Schuh
Greetings, Hello, my name is Michael Schuh and I am a PhD student in Computer Science at Montana State University. I have never participated in GSOC before, but I am very excited to propose a project to PostgreSQL that I feel would be a great follow-up to last year's project by Alexander Korotkov

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 05:53:43PM -0300, Alvaro Herrera wrote: > > Some more diacritics .. Thanks, applied. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-ha

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 10:57 PM, Ants Aasma wrote: > On Mon, Apr 22, 2013 at 10:54 PM, Simon Riggs wrote: >> On 22 April 2013 20:32, Florian Pflug wrote: >> >>> Assuming that we only ship a plain C implementation with 9.3, what >>> are we missing on that front? The C implementation of FNV1+SHIF

Re: [HACKERS] high io BUT huge amount of free memory

2013-04-22 Thread Merlin Moncure
On Mon, Apr 22, 2013 at 1:22 PM, Миша Тюрин wrote: > > My first message has been banned for too many latters. > >> > Hi all > There is something wrong and ugly. > > 1) > Intel 32 core = 2*8 *2threads > > Linux avi-sql09 2.6.32-5-amd64 #1 SMP Sun May 6 04:00:17 UTC 2012 x86_64 > GNU/Linux > > Post

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Steve Singer
On 13-04-22 04:41 PM, Anne Rosset wrote: Hi Steve, Yes I see these messages in our log. Is there a solution to this? Thanks, Anne A manual analyze of the effected tables should work and give you updated statistics. If your problem is just statistics then that should help. A manual vacuum will

Re: [HACKERS] high io BUT huge amount of free memory

2013-04-22 Thread Sergey Konoplev
On Mon, Apr 22, 2013 at 11:22 AM, Миша Тюрин wrote: > free -g > total used free sharedbuffers cached > Mem: 378250128 0 0229 > -/+ buffers/cache: 20357 > > and > ! disks usage 100% (free 128GB

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Jeff Davis
On Tue, 2013-04-23 at 01:08 +0300, Ants Aasma wrote: > A slight delay, but here it is. I didn't lift the checksum part into a > separate file as I didn't have a great idea what I would call it. The > code is reasonably compact so I don't see a great need for this right > now. It would be more worth

[HACKERS] boolean abuse in geo_ops.c

2013-04-22 Thread Peter Eisentraut
The function static char * path_encode(bool closed, int npts, Point *pt) is actually occasionally passed -1 as its first parameter to mean something like "neither" or "don't print any delimiters". That doesn't seem like good style. Maybe this should be split off into another bool parameter name

Re: [HACKERS] GSOC Student Project Idea

2013-04-22 Thread Stephen Frost
Michael, * Michael Schuh (schuh.m...@gmail.com) wrote: > In brief, I would like to implement a state-of-the-art indexing algorithm > (named iDistance) directly in PostgreSQL using GiST or SP-GiST trees and > whatever means necessary. For my 2c, this sounds fantastic. Excellent project idea and

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Thanks Steve. I have read that a fix has been put in release 9.2.3 for this issue. Is that right? Thanks, Anne -Original Message- From: Steve Singer [mailto:st...@ssinger.info] Sent: Monday, April 22, 2013 4:35 PM To: Anne Rosset Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Pe

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Alvaro Herrera
Jeff Davis escribió: > I'm not sure what we should call the separate file or where we should > put it, though. How about src/backend/utils/checksum/checksum_fnv.c? Is > there a clean way to override the compiler flags for a single file so we > don't need to put it in its own directory? Sure, see