Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Boszormenyi Zoltan
2013-04-21 15:10 keltezéssel, Bruce Momjian írta: On Sun, Apr 21, 2013 at 09:34:10AM +0200, Boszormenyi Zoltan wrote: 2013-04-21 07:02 keltezéssel, Bruce Momjian írta: I am not sure if Tom shared yet, but we are planning to package 9.3 beta1 on April 29, with a release on May 2. Those dates

[HACKERS] Fast promotion, loose ends

2013-04-22 Thread Heikki Linnakangas
We never reached a consensus on the user interface of the new 'fast promotion'. We should settle that before beta. The thread died here: http://www.postgresql.org/message-id/CA+U5nMKmDD7hGCYzOo=iFM=eK5OPDXCEzmq79fgLWr0TJk=s...@mail.gmail.com Simon said in that email that he's waiting for

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Simon Riggs
On 22 April 2013 08:13, Heikki Linnakangas hlinnakan...@vmware.com wrote: We never reached a consensus on the user interface of the new 'fast promotion'. We should settle that before beta. The thread died here:

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Heikki Linnakangas
On 22.04.2013 10:58, Simon Riggs wrote: On 22 April 2013 08:13, Heikki Linnakangashlinnakan...@vmware.com wrote: 2. There is no way to perform 'fast promotion' using the trigger file. That feature is only available using pg_ctl promote. When pg_ctl promote was introduced, it was not meant to

[HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-22 Thread Jeevan Chalke
Hi, I have observed that following sequence is causing server crash. CREATE MATERIALIZED VIEW temp_class_mv AS SELECT * FROM pg_class WITH NO DATA; CREATE OR REPLACE FUNCTION test_refresh_mv() RETURNS int AS $$ BEGIN REFRESH MATERIALIZED VIEW temp_class_mv; return 1; END; $$ LANGUAGE

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-22 Thread Andres Freund
On 2013-04-22 18:35:04 +0530, Jeevan Chalke wrote: Hi, I have observed that following sequence is causing server crash. CREATE MATERIALIZED VIEW temp_class_mv AS SELECT * FROM pg_class WITH NO DATA; CREATE OR REPLACE FUNCTION test_refresh_mv() RETURNS int AS $$ BEGIN REFRESH

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-22 Thread Jeevan Chalke
On Mon, Apr 22, 2013 at 6:41 PM, Andres Freund and...@2ndquadrant.comwrote: On 2013-04-22 18:35:04 +0530, Jeevan Chalke wrote: Hi, I have observed that following sequence is causing server crash. CREATE MATERIALIZED VIEW temp_class_mv AS SELECT * FROM pg_class WITH NO DATA;

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Shaun Thomas
On 04/22/2013 02:58 AM, Simon Riggs wrote: So, to initiate promotion, you can create a file called $DATADIR/fast_promote or $DATADIR/promote Pardon my naiveté, but could it also be an option to read the method from the promotion file? echo slow /my/promotion/path That would work without

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-04-22 Thread Robert Haas
On Wed, Apr 17, 2013 at 2:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think this breaks contrib/adminpack, and perhaps other extensions. They'd not be hard to fix with script changes, but they'd be broken. In general, we would now have a situation where relocatable extensions could never be

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Robert Haas
On Wed, Apr 17, 2013 at 8:21 PM, Greg Smith g...@2ndquadrant.com wrote: The more I read of this thread, the more unhappy I get. It appears that the entire design process is being driven by micro-optimization for CPUs being built by Intel in 2013. And that's not going to get anyone past

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Andres Freund
On 2013-04-22 11:27:25 -0400, Robert Haas wrote: On Wed, Apr 17, 2013 at 8:21 PM, Greg Smith g...@2ndquadrant.com wrote: The more I read of this thread, the more unhappy I get. It appears that the entire design process is being driven by micro-optimization for CPUs being built by Intel in

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Simon Riggs
On 22 April 2013 09:29, Heikki Linnakangas hlinnakan...@vmware.com wrote: Hmm. That requires write access to $DATADIR, so that's not quite the same thing as the trigger_file recovery.conf option. Well, you also (elsewhere) requested that we must keep recovery.conf in $DATADIR, so it needs to

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Simon Riggs
On 22 April 2013 16:09, Shaun Thomas stho...@optionshouse.com wrote: On 04/22/2013 02:58 AM, Simon Riggs wrote: So, to initiate promotion, you can create a file called $DATADIR/fast_promote or $DATADIR/promote Pardon my naiveté, but could it also be an option to read the method from the

Re: [HACKERS] event trigger API documentation?

2013-04-22 Thread Robert Haas
On Wed, Apr 17, 2013 at 3:20 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Please also note that the first series of patches did include the support code for all the core PL, but Robert didn't feel like commiting that and no other commiter did step up. Of course, the chances of that would

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 6:27 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 17, 2013 at 8:21 PM, Greg Smith g...@2ndquadrant.com wrote: The more I read of this thread, the more unhappy I get. It appears that the entire design process is being driven by micro-optimization for CPUs

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 6:33 PM, Andres Freund and...@2ndquadrant.com wrote: I don't see us changing away from CRCs anymore either by now. But I think at least changing the polynom to something that a) has higher error detection properties b) can noticeably sped up on a a good part of the

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Alvaro Herrera
Bruce Momjian wrote: I am not sure if Tom shared yet, but we are planning to package 9.3 beta1 on April 29, with a release on May 2. Those dates might change, but that is the current plan. I have completed a draft 9.3 release notes, which you can view here:

[HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Hi, We are seeing some overall performance degradation in our application since we installed the security release. Other commits were also done at the same time in the application so we don't know yet if the degradation has any relationship with the security release. While we are digging

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Josh Berkus
On 04/22/2013 10:38 AM, Anne Rosset wrote: While we are digging into this, I would like to know if it is possible that the release has some impact on performance. After reading this It was created as a side effect of a refactoring effort to make establishing new connections to a

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Josh Berkus
On 04/22/2013 10:38 AM, Anne Rosset wrote: While we are digging into this, I would like to know if it is possible that the release has some impact on performance. After reading this It was created as a side effect of a refactoring effort to make establishing new connections to a

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Josh Berkus
On 04/22/2013 09:25 AM, Ants Aasma wrote: This leaves lingering doubts about the quality of the checksum. It's hard if not impossible to prove absence of interesting patterns that would trigger collisions. I do know the checksum quality is miles ahead of the Fletcher sum originally proposed

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Heikki Linnakangas
On 22.04.2013 18:45, Simon Riggs wrote: On 22 April 2013 16:09, Shaun Thomasstho...@optionshouse.com wrote: On 04/22/2013 02:58 AM, Simon Riggs wrote: So, to initiate promotion, you can create a file called $DATADIR/fast_promote or $DATADIR/promote Pardon my naiveté, but could it also be

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Florian Pflug
On Apr22, 2013, at 18:25 , Ants Aasma a...@cybertec.at wrote: I was just now writing up a generic C based patch based on the parallel FNV-1a + shift that we discussed with Florian with an added round of mixing. Testing the performance in isolation indicates that: 1) it is about an order of

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Hi Josh, Thanks for your reply. I don't think this is the case since we are using jboss/jdbc driver with a connection pool. Thanks, Anne -Original Message- From: Josh Berkus [mailto:j...@agliodbs.com] Sent: Monday, April 22, 2013 10:58 AM To: Anne Rosset Cc:

Re: [HACKERS] minimizing the target list for foreign data wrappers

2013-04-22 Thread David Gudeman
Re-reading my first email I thought it was a little confusing, so here is some clarification. In GetForeignPlan, tlist seems to be a target list for a basic select * from the foreign table. For the ith TargetEntry te in tlist, it seems that te-expr is a var with varattno=i. I was mis-remembering

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 01:54:03PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I am not sure if Tom shared yet, but we are planning to package 9.3 beta1 on April 29, with a release on May 2. Those dates might change, but that is the current plan. I have completed a draft 9.3 release

Re: [HACKERS] Fast promotion, loose ends

2013-04-22 Thread Simon Riggs
On 22 April 2013 19:04, Heikki Linnakangas hlinnakan...@vmware.com wrote: We could do that and a similar mechanism existed in pg_standby, but was removed in 9.0. That code is still in pg_standby. Maybe you were thinking of some other feature? It wasn't removed from pg_standby. But since

[HACKERS] libpq COPY handling

2013-04-22 Thread Robert Haas
Noah Misch pointed out something interesting to me: /* * PQputCopyEnd - send EOF indication to the backend during COPY IN * * After calling this, use PQgetResult() to check command completion status. * * Returns 1 if successful, 0 if data could not be sent (only possible * in nonblock

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 9:04 PM, Florian Pflug f...@phlo.org wrote: The one downside of the fnv1+shift approach is that it's built around the assumption that processing 64-bytes at once is the sweet spot. That might be true for x86 and x86_64 today, but it won't stay that way for long, and

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Heikki Linnakangas
Allow tooling like pg_receivexlog to run on computers with different architectures (Heikki Linnakangas) This probably should be mentioned in the backwards-compatibility section. Any 3rd party tools that speak the streaming replication protocol are affected. E.1.3.2.1. Write-Ahead Log

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Jeff Davis
On Mon, 2013-04-22 at 20:04 +0200, Florian Pflug wrote: The one downside of the fnv1+shift approach is that it's built around the assumption that processing 64-bytes at once is the sweet spot. That might be true for x86 and x86_64 today, but it won't stay that way for long, and quite surely

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Robert Haas
On Mon, Apr 22, 2013 at 3:14 PM, Jeff Davis pg...@j-davis.com wrote: The biggest problem now is getting one of these faster algorithms (FNV or even a faster CRC) into shape that is acceptable to reviewers/committers. If we don't do that, we will be missing out on a lot of potential checksum

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Robert Haas
On Mon, Apr 22, 2013 at 2:33 PM, Bruce Momjian br...@momjian.us wrote: In this item Add support to event triggers (Dimitri Fontaine, Tom Lane) I am not sure why you list Tom. I think Robert should be listed instead. Tom did a massive fix/cleanup of that code. I have added Robert. I do

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Jeff Davis
On Mon, 2013-04-22 at 19:25 +0300, Ants Aasma wrote: I was just now writing up a generic C based patch based on the parallel FNV-1a + shift that we discussed with Florian with an added round of mixing. Testing the performance in isolation indicates that: 1) it is about an order of magnitude

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Florian Pflug
On Apr22, 2013, at 21:14 , Jeff Davis pg...@j-davis.com wrote: On Mon, 2013-04-22 at 20:04 +0200, Florian Pflug wrote: The one downside of the fnv1+shift approach is that it's built around the assumption that processing 64-bytes at once is the sweet spot. That might be true for x86 and x86_64

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Alvaro Herrera
Bruce Momjian wrote: On Mon, Apr 22, 2013 at 01:54:03PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I am not sure if Tom shared yet, but we are planning to package 9.3 beta1 on April 29, with a release on May 2. Those dates might change, but that is the current plan. I have

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Simon Riggs
On 22 April 2013 20:32, Florian Pflug f...@phlo.org wrote: Assuming that we only ship a plain C implementation with 9.3, what are we missing on that front? The C implementation of FNV1+SHIFT is only a few dozen lines or so. Forgive me, I can't seem to locate the patch for this? Re-post

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 03:19:36PM -0400, Robert Haas wrote: On Mon, Apr 22, 2013 at 2:33 PM, Bruce Momjian br...@momjian.us wrote: In this item Add support to event triggers (Dimitri Fontaine, Tom Lane) I am not sure why you list Tom. I think Robert should be listed instead. Tom

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 10:54 PM, Simon Riggs si...@2ndquadrant.com wrote: On 22 April 2013 20:32, Florian Pflug f...@phlo.org wrote: Assuming that we only ship a plain C implementation with 9.3, what are we missing on that front? The C implementation of FNV1+SHIFT is only a few dozen lines

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Steve Singer
On 13-04-22 01:38 PM, Anne Rosset wrote: Hi, We are seeing some overall performance degradation in our application since we installed the security release. Other commits were also done at the same time in the application so we don't know yet if the degradation has any relationship with the

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 04:48:58PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: On Mon, Apr 22, 2013 at 01:54:03PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I am not sure if Tom shared yet, but we are planning to package 9.3 beta1 on April 29, with a release on May 2.

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 10:11:48PM +0300, Heikki Linnakangas wrote: Allow tooling like pg_receivexlog to run on computers with different architectures (Heikki Linnakangas) This probably should be mentioned in the backwards-compatibility section. Any 3rd party tools that speak the streaming

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Hi Steve, Thanks for your reply. We are now running 9.0.13. Before it was 9.0.7. How can I find out if we are running into this issue: ie if statistics are no longer being updated because analyze can't get the exclusive lock for truncation? I will dig into our logs to see for the query times.

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Steve Singer
On 13-04-22 04:15 PM, Anne Rosset wrote: Hi Steve, Thanks for your reply. We are now running 9.0.13. Before it was 9.0.7. How can I find out if we are running into this issue: ie if statistics are no longer being updated because analyze can't get the exclusive lock for truncation? This issue

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Hi Steve, Yes I see these messages in our log. Is there a solution to this? Thanks, Anne -Original Message- From: Steve Singer [mailto:st...@ssinger.info] Sent: Monday, April 22, 2013 1:26 PM To: Anne Rosset Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Performance with the new

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Alvaro Herrera
Some more diacritics .. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services diff --git a/doc/src/sgml/release-9.3.sgml b/doc/src/sgml/release-9.3.sgml index 7c46bd3..68d04a7 100644 --- a/doc/src/sgml/release-9.3.sgml +++

[HACKERS] GSOC Student Project Idea

2013-04-22 Thread Michael Schuh
Greetings, Hello, my name is Michael Schuh and I am a PhD student in Computer Science at Montana State University. I have never participated in GSOC before, but I am very excited to propose a project to PostgreSQL that I feel would be a great follow-up to last year's project by Alexander Korotkov

Re: [HACKERS] 9.3 Beta1 status report

2013-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2013 at 05:53:43PM -0300, Alvaro Herrera wrote: Some more diacritics .. Thanks, applied. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + --

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Ants Aasma
On Mon, Apr 22, 2013 at 10:57 PM, Ants Aasma a...@cybertec.at wrote: On Mon, Apr 22, 2013 at 10:54 PM, Simon Riggs si...@2ndquadrant.com wrote: On 22 April 2013 20:32, Florian Pflug f...@phlo.org wrote: Assuming that we only ship a plain C implementation with 9.3, what are we missing on that

Re: [HACKERS] high io BUT huge amount of free memory

2013-04-22 Thread Merlin Moncure
On Mon, Apr 22, 2013 at 1:22 PM, Миша Тюрин tmih...@bk.ru wrote: My first message has been banned for too many latters. Hi all There is something wrong and ugly. 1) Intel 32 core = 2*8 *2threads Linux avi-sql09 2.6.32-5-amd64 #1 SMP Sun May 6 04:00:17 UTC 2012 x86_64 GNU/Linux

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Steve Singer
On 13-04-22 04:41 PM, Anne Rosset wrote: Hi Steve, Yes I see these messages in our log. Is there a solution to this? Thanks, Anne A manual analyze of the effected tables should work and give you updated statistics. If your problem is just statistics then that should help. A manual vacuum

Re: [HACKERS] high io BUT huge amount of free memory

2013-04-22 Thread Sergey Konoplev
On Mon, Apr 22, 2013 at 11:22 AM, Миша Тюрин tmih...@bk.ru wrote: free -g total used free sharedbuffers cached Mem: 378250128 0 0229 -/+ buffers/cache: 20357 and ! disks usage 100%

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Jeff Davis
On Tue, 2013-04-23 at 01:08 +0300, Ants Aasma wrote: A slight delay, but here it is. I didn't lift the checksum part into a separate file as I didn't have a great idea what I would call it. The code is reasonably compact so I don't see a great need for this right now. It would be more worth

[HACKERS] boolean abuse in geo_ops.c

2013-04-22 Thread Peter Eisentraut
The function static char * path_encode(bool closed, int npts, Point *pt) is actually occasionally passed -1 as its first parameter to mean something like neither or don't print any delimiters. That doesn't seem like good style. Maybe this should be split off into another bool parameter named

Re: [HACKERS] GSOC Student Project Idea

2013-04-22 Thread Stephen Frost
Michael, * Michael Schuh (schuh.m...@gmail.com) wrote: In brief, I would like to implement a state-of-the-art indexing algorithm (named iDistance) directly in PostgreSQL using GiST or SP-GiST trees and whatever means necessary. For my 2c, this sounds fantastic. Excellent project idea and it

Re: [HACKERS] Performance with the new security release?

2013-04-22 Thread Anne Rosset
Thanks Steve. I have read that a fix has been put in release 9.2.3 for this issue. Is that right? Thanks, Anne -Original Message- From: Steve Singer [mailto:st...@ssinger.info] Sent: Monday, April 22, 2013 4:35 PM To: Anne Rosset Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS]

Re: [HACKERS] Enabling Checksums

2013-04-22 Thread Alvaro Herrera
Jeff Davis escribió: I'm not sure what we should call the separate file or where we should put it, though. How about src/backend/utils/checksum/checksum_fnv.c? Is there a clean way to override the compiler flags for a single file so we don't need to put it in its own directory? Sure, see