Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-16 Thread Marko Kreen
On Thu, Feb 16, 2012 at 02:24:19PM +0900, Kyotaro HORIGUCHI wrote: As far as I see, on an out-of-memory in getAnotherTuple() makes conn-result-resultStatus = PGRES_FATAL_ERROR and qpParseInputp[23]() skips succeeding 'D' messages consequently. When exception raised within row processor,

Re: [HACKERS] Google Summer of Code? Call for mentors.

2012-02-16 Thread Dave Page
On Wed, Feb 15, 2012 at 10:18 PM, Josh Berkus j...@agliodbs.com wrote: Hackers, The call is now open for Google Summer of Code. If you are interested in being a GSoC mentor this summer, please reply to this email.  I want to gauge whether or not we should participate this summer. I'll

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-02-16 Thread Fujii Masao
On Thu, Feb 16, 2012 at 5:02 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 15.02.2012 18:52, Fujii Masao wrote: On Thu, Feb 16, 2012 at 1:01 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com  wrote: Are you still seeing this failure with the latest patch I

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-02-16 Thread Fujii Masao
On Thu, Feb 16, 2012 at 6:15 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Feb 16, 2012 at 5:02 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 15.02.2012 18:52, Fujii Masao wrote: On Thu, Feb 16, 2012 at 1:01 AM, Heikki Linnakangas

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-16 Thread Alexander Korotkov
On Thu, Feb 16, 2012 at 12:34 AM, Jay Levitt jay.lev...@gmail.com wrote: - But a dimension might be in any domain, not just floats - The distance along each dimension is a domain-specific function What exact domains do you expect? Some domains could appear to be quite hard for index-based

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-16 Thread Albert Cervera i Areny
A Dijous, 16 de febrer de 2012 12:16:31, Simon Riggs va escriure: v8 attached Maybe the docs should include what will happen if the checksum is not correct? -- Albert Cervera i Areny http://www.NaN-tic.com Tel: +34 93 553 18 03 http://twitter.com/albertnan http://www.nan-tic.com/blog

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-16 Thread Robert Haas
On Thu, Feb 16, 2012 at 6:16 AM, Simon Riggs si...@2ndquadrant.com wrote: v8 attached It's hard to believe that this version has been tested terribly thoroughly, because it doesn't compile. + LockBufHdr(buf); + + /* +* Run PageGetLSN while holding header lock. +*/ +

Re: [HACKERS] run GUC check hooks on RESET

2012-02-16 Thread Robert Haas
On Wed, Feb 15, 2012 at 5:36 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Agreed.  I'm not sure we want to change the message text at all in 9.1.  Translations and all that. Agreed. I think we definitely don't want to do that. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Should we add crc32 in libpgport?

2012-02-16 Thread Robert Haas
On Fri, Feb 3, 2012 at 7:33 PM, Daniel Farina dan...@heroku.com wrote: Ah, yes, I think my optimizations were off when building, or something.  I didn't get such verbosity at first, and then I remember doing something slightly different and then getting a lot of output. I didn't pay attention

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-16 Thread Albe Laurenz
Shigeru Hanada wrote: Thanks for the review. Attached patches are revised version, though only fdw_helper_v5.patch is unchanged. Two questions: - Is it on purpose that you can specify all SSL client options except sslcompression? - Since a rescan is done by rewinding the cursor, is it

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-16 Thread Marko Kreen
On Thu, Feb 16, 2012 at 05:49:34PM +0900, Kyotaro HORIGUCHI wrote: I added the function PQskipRemainingResult() and use it in dblink. This reduces the number of executing try-catch block from the number of rows to one per query in dblink. This implementation is wrong - you must not simply

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-16 Thread Kohei KaiGai
I found a strange behavior with v10. Is it available to reproduce? In case of ftbl is declared as follows: postgres=# select * FROM ftbl; a | b ---+- 1 | aaa 2 | bbb 3 | ccc 4 | ddd 5 | eee (5 rows) I tried to raise an error on remote side. postgres=# select *

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-16 Thread Jay Levitt
Alexander Korotkov wrote: On Thu, Feb 16, 2012 at 12:34 AM, Jay Levitt jay.lev...@gmail.com mailto:jay.lev...@gmail.com wrote: - But a dimension might be in any domain, not just floats - The distance along each dimension is a domain-specific function What exact domains do you expect?

[HACKERS] possible new option for wal_sync_method

2012-02-16 Thread Dan Scales
When running Postgres on a single ext3 filesystem on Linux, we find that the attached simple patch gives significant performance benefit (7-8% in numbers below). The patch adds a new option for wal_sync_method, which is open_direct. With this option, the WAL is always opened with O_DIRECT (but

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-16 Thread Kohei KaiGai
2012年2月16日13:41 Shigeru Hanada shigeru.han...@gmail.com: Kaigai-san, Thanks for the review. Attached patches are revised version, though only fdw_helper_v5.patch is unchanged. (2012/02/16 0:09), Kohei KaiGai wrote: [memory context of tuple store] It calls tuplestore_begin_heap() under the

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-16 Thread Jay Levitt
Tom Lane wrote: Jay Levittjay.lev...@gmail.com writes: - I'm not sure how to represent arbitrary column-like features without reinventing the wheel and putting a database in the database. ISTM you could define a composite type and then create operators and an operator class over that type.

Re: [HACKERS] patch for parallel pg_dump

2012-02-16 Thread Robert Haas
On Wed, Feb 8, 2012 at 7:56 PM, Joachim Wieland j...@mcknight.de wrote: So we at least need to press on far enough to get to that point. Sure, let me know if I can help you with something. Alright. I think (hope) that I've pushed this far enough to serve the needs of parallel pg_dump. The

Re: [HACKERS] possible new option for wal_sync_method

2012-02-16 Thread Andres Freund
Hi, On Thursday, February 16, 2012 06:18:23 PM Dan Scales wrote: When running Postgres on a single ext3 filesystem on Linux, we find that the attached simple patch gives significant performance benefit (7-8% in numbers below). The patch adds a new option for wal_sync_method, which is

Re: [HACKERS] [trivial patch] typo in doc/src/sgml/sepgsql.sgml

2012-02-16 Thread Dimitri Fontaine
Alvaro Herrera alvhe...@commandprompt.com writes: If you copied from a pager, those tend to expand tabs to spaces, so the patch gets mangled at that point. At least less does that. OTOH if you :r the patch file, it works fine. You should be able to add an “inline” attachment too, and the

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-16 Thread Tom Lane
Jay Levitt jay.lev...@gmail.com writes: Perfect. Composite types are exactly what I need here; the application can declare its composite type and provide distance functions for each member, and the extension can use those to calculate similarity. How do I introspect the composite type's

Re: [HACKERS] possible new option for wal_sync_method

2012-02-16 Thread Marti Raudsepp
On Thu, Feb 16, 2012 at 19:18, Dan Scales sca...@vmware.com wrote: fsync/fdatasync can be very slow on ext3, because it seems to have to always wait for the current filesystem meta-data transaction to complete, even if that meta-data operation is completely unrelated to the file being

Re: [HACKERS] possible new option for wal_sync_method

2012-02-16 Thread Josh Berkus
On 2/16/12 9:18 AM, Dan Scales wrote: Do folks have any interest in this change, or comments on its usefulness/correctness? It would be just an extra option for wal_sync_method that users can try out and has benefits for certain configurations. Does it have any benefit on Ext4/XFS/Butrfs?

[HACKERS] proposal: copybytea command for psql

2012-02-16 Thread Andrew Dunstan
A while ago I went looking for nice ways to export an unencoded bytea value using psql, see http://people.planetpostgresql.org/andrew/index.php?/archives/196-Clever-trick-challenge.html. Regina Obe is also in want of a solution for this:

Re: [HACKERS] proposal: copybytea command for psql

2012-02-16 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: A while ago I went looking for nice ways to export an unencoded bytea value using psql, see http://people.planetpostgresql.org/andrew/index.php?/archives/196-Clever-trick-challenge.html. Regina Obe is also in want of a solution for this:

Re: [HACKERS] Command Triggers

2012-02-16 Thread Robert Haas
On Thu, Feb 16, 2012 at 12:42 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Fixed, I've spent quite some time refining the API. That is better. What could still be done here is to create a cache (along the lines of attoptcache.c) that stores a big array with one slot for each supported

Re: [HACKERS] proposal: copybytea command for psql

2012-02-16 Thread Andrew Dunstan
On 02/16/2012 03:32 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: A while ago I went looking for nice ways to export an unencoded bytea value using psql, see http://people.planetpostgresql.org/andrew/index.php?/archives/196-Clever-trick-challenge.html. Regina Obe is also in

Re: [HACKERS] Command Triggers

2012-02-16 Thread Alvaro Herrera
Excerpts from Dimitri Fontaine's message of jue feb 16 14:42:26 -0300 2012: Hi, Please find attached version 8 of the patch, which fixes most of your complaints. Hi, I didn't like the new cmdtrigger.h file. It's included by a lot of other headers, and it's also itself including

Re: [HACKERS] Command Triggers

2012-02-16 Thread Dimitri Fontaine
Hi, First answer now, new patch version tomorrow. Robert Haas robertmh...@gmail.com writes: That is better. Cool :) What could still be done here is to create a cache (along the lines of attoptcache.c) that stores a big array with one slot for each supported command type. The first time a

Re: [HACKERS] Command Triggers

2012-02-16 Thread Dimitri Fontaine
Alvaro Herrera alvhe...@commandprompt.com writes: I didn't like the new cmdtrigger.h file. It's included by a lot of other headers, and it's also itself including execnodes.h and parsenodes.h which means practically the whole lot of the source tree is included in turn. If you could split it,

[HACKERS] Bug in intarray?

2012-02-16 Thread Guillaume Lelarge
Hi, On a french PostgreSQL web forum, one of our users asked about a curious behaviour of the intarray extension. This query: SELECT ARRAY[-1,3,1] ARRAY[1, 2]; should give {1} as a result. But, on HEAD (and according to his tests, on 9.0.6 and 9.1.2), it appears to give en empty array.

Re: [HACKERS] run GUC check hooks on RESET

2012-02-16 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: The main thing I would be worried about is whether you're sure that you have separated the RESET-as-a-command case from the cases where we actually are rolling back to a previous state. It looks good to me. I added a few regression tests for that.

Re: [HACKERS] Command Triggers

2012-02-16 Thread Robert Haas
On Thu, Feb 16, 2012 at 4:21 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: We could decide command triggers are superuser only.  Security is not something I'm very strong at, so I'll leave it up to you to decide. That's certainly the easiest option. If you don't feel passionate about

Re: [HACKERS] Command Triggers

2012-02-16 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: On Thu, Feb 16, 2012 at 4:21 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: That's certainly the easiest option. If you don't feel passionate about spending a lot of energy figuring out how to make it secure, then I suggest we just restrict it to

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-16 Thread Jay Levitt
Tom Lane wrote: - Can domains have operators, or are operators defined on types? I think the current state of play is that you can have such things but the system will only consider them for exact type matches, so you might need more explicit casts than you ordinarily would. Turns out it's

[HACKERS] Qual evaluation cost estimates for GIN indexes

2012-02-16 Thread Tom Lane
I looked into the complaint here of poor estimation for GIN indexscans: http://archives.postgresql.org/pgsql-performance/2012-02/msg00028.php At first glance it sounds like a mistake in selectivity estimation, but it isn't: the rowcount estimates are pretty nearly dead on. The problem is in the

Re: [HACKERS] Qual evaluation cost estimates for GIN indexes

2012-02-16 Thread Tom Lane
I wrote: BTW, an entirely different line of thought is why on earth is @@ so frickin expensive, when it's comparing already-processed tsvectors with only a few entries to an already-processed tsquery with only one entry??. This test case suggests to me that there's something unnecessarily

Re: [HACKERS] Designing an extension for feature-space similarity search

2012-02-16 Thread Tom Lane
Jay Levitt jay.lev...@gmail.com writes: Tom Lane wrote: - Can domains have operators, or are operators defined on types? I think the current state of play is that you can have such things but the system will only consider them for exact type matches, so you might need more explicit casts

Re: [HACKERS] possible new option for wal_sync_method

2012-02-16 Thread Dan Scales
Good point, thanks. From the ext3 source code, it looks like ext3_sync_file() does a blkdev_issue_flush(), which issues a flush to the block device, whereas simple direct IO does not. So, that would make this wal_sync_method option less useful, since, as you say, the user would have to know if

Re: [HACKERS] Bug in intarray?

2012-02-16 Thread Tom Lane
Guillaume Lelarge guilla...@lelarge.info writes: This query: SELECT ARRAY[-1,3,1] ARRAY[1, 2]; should give {1} as a result. But, on HEAD (and according to his tests, on 9.0.6 and 9.1.2), it appears to give en empty array. Definitely a bug, and I'll bet it goes all the way back. Digging

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-16 Thread Shigeru Hanada
(2012/02/17 0:15), Albe Laurenz wrote: Shigeru Hanada wrote: Thanks for the review. Attached patches are revised version, though only fdw_helper_v5.patch is unchanged. Two questions: - Is it on purpose that you can specify all SSL client options except sslcompression? No, just an

Re: [HACKERS] Qual evaluation cost estimates for GIN indexes

2012-02-16 Thread Robert Haas
On Thu, Feb 16, 2012 at 6:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: BTW, an entirely different line of thought is why on earth is @@ so frickin expensive, when it's comparing already-processed tsvectors with only a few entries to an already-processed tsquery with only one entry??.  

Re: [HACKERS] Qual evaluation cost estimates for GIN indexes

2012-02-16 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: This issue of detoasting costs comes up a lot, specifically in reference to @@. I wonder if we shouldn't try to apply some quick and dirty hack in time for 9.2, like maybe random_page_cost for every row or every attribute we think will require

Re: [HACKERS] Command Triggers

2012-02-16 Thread Robert Haas
On Thu, Feb 16, 2012 at 5:38 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Another option would be to add a PGC_SUSET boolean GUC that can be used to disable command triggers.  I think that might be more flexible, not to mention useful for recursion prevention. Wait, we already have

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-02-16 Thread Fujii Masao
On Mon, Feb 13, 2012 at 8:37 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 13.02.2012 01:04, Jeff Janes wrote: Attached is my quick and dirty attempt to set XLP_FIRST_IS_CONTRECORD.  I have no idea if I did it correctly, in particular if calling GetXLogBuffer(CurrPos)

[HACKERS] MySQL search query is not executing in Postgres DB

2012-02-16 Thread premanand
In MySQL the below query is executing properly. SELECT * FROM Table-name WHERE (Table.ID LIKE '1%') But when i try to execute the above query in Postgres, i get the following Exception org.postgresql.util.PSQLException: ERROR: operator does not exist: integer ~~ unknown Hint: No operator matches

Re: [HACKERS] Qual evaluation cost estimates for GIN indexes

2012-02-16 Thread Jesper Krogh
Hi. First, thanks for looking at this. Except from GIN indexes and full-text-search being really good in our applications, this also points to those excact places where it can be improved. On 2012-02-17 00:15, Tom Lane wrote: I looked into the complaint here of poor estimation for GIN

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-16 Thread Heikki Linnakangas
On 17.02.2012 07:33, premanand wrote: In MySQL the below query is executing properly. SELECT * FROMTable-name WHERE (Table.ID LIKE '1%') But when i try to execute the above query in Postgres, i get the following Exception org.postgresql.util.PSQLException: ERROR: operator does not exist:

Re: [HACKERS] WIP: Collecting statistics on CSV file data

2012-02-16 Thread Etsuro Fujita
Hi Hanada-san, Sorry for the late response. (2012/02/10 22:05), Shigeru Hanada wrote: (2011/12/15 11:30), Etsuro Fujita wrote: (2011/12/14 15:34), Shigeru Hanada wrote: I think this patch could be marked as Ready for committer with some minor fixes. Please find attached a revised patch