Re: [HACKERS] libpq compression

2012-06-15 Thread Marko Kreen
On Sat, Jun 16, 2012 at 6:39 AM, Magnus Hagander mag...@hagander.net wrote: On Sat, Jun 16, 2012 at 4:04 AM, Euler Taveira eu...@timbira.com wrote: On 15-06-2012 11:39, Magnus Hagander wrote: As long as a free implementation exists, it can be ported to Java/.Net. Sure, it takes more work, but

[HACKERS] [9.2] crash on regex

2012-05-24 Thread Marko Kreen
Following query crashes backend on 9.2: select substring('asd TO foo' from ' TO (([a-z0-9._]+|([^]+|)+)+)'); It is supposed to load potentially quoted table name from CREATE RULE definition. Works fine on 8.3 .. 9.1 Backtrace: Program terminated with signal 11, Segmentation fault. #0

Re: [HACKERS] remove dead ports?

2012-05-04 Thread Marko Kreen
On Fri, May 4, 2012 at 8:45 PM, Peter Eisentraut pete...@gmx.net wrote: On tor, 2012-05-03 at 17:39 +0100, Peter Geoghegan wrote: On 3 May 2012 17:21, Bruce Momjian br...@momjian.us wrote: I think I was the only user left;  I have never heard from a BSD/OS user in the past 5-7 years. I'm

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-04-05 Thread Marko Kreen
On Wed, Apr 04, 2012 at 06:41:00PM -0400, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: On Wed, Apr 4, 2012 at 10:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Given the lack of consensus around the suspension API, maybe the best way to get the underlying libpq patch to a committable

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-04-05 Thread Marko Kreen
On Thu, Apr 05, 2012 at 12:49:37PM -0400, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: Minor cleanups: * Change callback return value to be 'bool': 0 is error. Currently the accepted return codes are 1 and -1, which is weird. No, I did it that way intentionally

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-04-04 Thread Marko Kreen
On Wed, Apr 4, 2012 at 10:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Given the lack of consensus around the suspension API, maybe the best way to get the underlying libpq patch to a committable state is to take it out --- that is, remove the return zero option for row processors. Since we don't

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-04-03 Thread Marko Kreen
On Sun, Apr 01, 2012 at 07:23:06PM -0400, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: So the proper approach would be to have new API call, designed to handle it, and allow early-exit only from there. That would also avoid any breakage of old APIs. Also it would avoid any

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-04-03 Thread Marko Kreen
On Tue, Apr 03, 2012 at 05:32:25PM -0400, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: The fact remains that upper-level code must cooperate with callback. Why is it useful to hijack PQgetResult() to do so? Because that's the defined communications channel. We're not hijacking

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-04-01 Thread Marko Kreen
On Sun, Apr 01, 2012 at 05:51:19PM -0400, Tom Lane wrote: I've been thinking some more about the early-termination cases (where the row processor returns zero or longjmps), and I believe I have got proposals that smooth off most of the rough edges there. First off, returning zero is really

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-30 Thread Marko Kreen
On Thu, Mar 29, 2012 at 06:56:30PM -0400, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: My conclusion is that row-processor API is low-level expert API and quite easy to misuse. It would be preferable to have something more robust as end-user API, the PQgetRow() is my suggestion

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-30 Thread Marko Kreen
On Fri, Mar 30, 2012 at 11:59:12AM -0400, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: On Thu, Mar 29, 2012 at 06:56:30PM -0400, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: Second conclusion is that current dblink row-processor usage is broken when user uses multiple

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-30 Thread Marko Kreen
On Fri, Mar 30, 2012 at 7:04 PM, Marko Kreen mark...@gmail.com wrote: Have you looked at the examples?  PQgetResult() is pretty good hint that one resultset finished... Ok, the demos are around this long thread and hard to find, so here is a summary of links: Original design mail: http

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-30 Thread Marko Kreen
On Fri, Mar 30, 2012 at 05:18:42PM -0400, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: On Wed, Mar 07, 2012 at 03:14:57PM +0900, Kyotaro HORIGUCHI wrote: My suggestion - check in getAnotherTuple whether resultStatus is already error and do nothing then. This allows internal

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-30 Thread Marko Kreen
On Sat, Mar 31, 2012 at 1:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Fri, Mar 30, 2012 at 05:18:42PM -0400, Tom Lane wrote: I'm pretty dissatisfied with the error reporting situation for row processors.  You can't just decide not to solve it, which seems

Re: [HACKERS] Standbys, txid_current_snapshot, wraparound

2012-03-29 Thread Marko Kreen
On Wed, Mar 28, 2012 at 10:54:58PM +0100, Simon Riggs wrote: On Wed, Mar 28, 2012 at 10:24 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Mar 28, 2012 at 9:48 PM, Marko Kreen mark...@gmail.com wrote: On Fri, Mar 23, 2012 at 08:52:40AM +, Simon Riggs wrote: Master pg_controldata

Re: [HACKERS] Standbys, txid_current_snapshot, wraparound

2012-03-29 Thread Marko Kreen
On Thu, Mar 29, 2012 at 10:37:54AM +0100, Simon Riggs wrote: When the standby receives the checkpoint record, it stores the information in 2 places: i) directly into ControlFile-checkPointCopy ii) and then into XLogCtl when a safe restartpoint occurs In RecoveryRestartPoint() I see: -

Re: [HACKERS] Standbys, txid_current_snapshot, wraparound

2012-03-29 Thread Marko Kreen
On Thu, Mar 29, 2012 at 02:46:23PM +0100, Simon Riggs wrote: On Thu, Mar 29, 2012 at 12:06 PM, Simon Riggs si...@2ndquadrant.com wrote: Patch coming in a few hours. This is more straightforward than I was thinking. We just need to initialise XLogCtl at the right place. Looks good to me.

Re: [HACKERS] Standbys, txid_current_snapshot, wraparound

2012-03-29 Thread Marko Kreen
On Thu, Mar 29, 2012 at 03:23:01PM +0100, Simon Riggs wrote: On Thu, Mar 29, 2012 at 3:04 PM, Marko Kreen mark...@gmail.com wrote: Next question: how can flipping archive_mode on and off, with restarts, near wraparound point, break epoch on master?  http://lists.pgfoundry.org/pipermail

Re: [HACKERS] Re: [COMMITTERS] pgsql: pg_test_timing utility, to measure clock monotonicity and timing

2012-03-28 Thread Marko Kreen
On Wed, Mar 28, 2012 at 08:19:37AM -0400, Robert Haas wrote: On Tue, Mar 27, 2012 at 10:10 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Mar 28, 2012 at 5:17 AM, Robert Haas rh...@postgresql.org wrote: pg_test_timing utility, to measure clock monotonicity and timing cost. When I

Re: [HACKERS] Re: [COMMITTERS] pgsql: pg_test_timing utility, to measure clock monotonicity and timing

2012-03-28 Thread Marko Kreen
On Wed, Mar 28, 2012 at 03:46:26PM +0300, Marko Kreen wrote: On Wed, Mar 28, 2012 at 08:19:37AM -0400, Robert Haas wrote: On Tue, Mar 27, 2012 at 10:10 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Mar 28, 2012 at 5:17 AM, Robert Haas rh...@postgresql.org wrote: pg_test_timing

Re: [HACKERS] Re: [COMMITTERS] pgsql: pg_test_timing utility, to measure clock monotonicity and timing

2012-03-28 Thread Marko Kreen
On Wed, Mar 28, 2012 at 08:57:42AM -0400, Robert Haas wrote: On Wed, Mar 28, 2012 at 8:51 AM, Marko Kreen mark...@gmail.com wrote: How about:  .. %10 INT64_FORMAT .. ? Well, it won't work because unlike inttypes.h, Postgres *_FORMAT includes '%' in it. I guess that why inttypes.h

Re: [HACKERS] Standbys, txid_current_snapshot, wraparound

2012-03-28 Thread Marko Kreen
On Fri, Mar 23, 2012 at 08:52:40AM +, Simon Riggs wrote: Master pg_controldata - OK txid_current_snapshot() - OK Standby pg_controldata - OK txid_current_snapshot() - lower value On Skytools list is report about master with slaves, but the lower value appears on master too:

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-27 Thread Marko Kreen
On Sat, Mar 24, 2012 at 02:22:24AM +0200, Marko Kreen wrote: Main advantage of including PQgetRow() together with low-level rowproc API is that it allows de-emphasizing more complex parts of rowproc API (exceptions, early exit, skipresult, custom error msg). And drop/undocument them or simply

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-23 Thread Marko Kreen
I saw Kyotaro already answered, but I give my view as well. On Thu, Mar 22, 2012 at 06:07:16PM -0400, Tom Lane wrote: AFAICT it breaks async processing entirely, because many changes have been made that fail to distinguish insufficient data available as yet from hard error. As an example,

Re: [HACKERS] Another review of URI for libpq, v7 submission

2012-03-17 Thread Marko Kreen
On Thu, Mar 15, 2012 at 11:29:31PM +0200, Alex wrote: https://github.com/a1exsh/postgres/commits/uri The point of the patch is to have one string with all connection options, in standard format, yes? So why does not this work: db = PQconnectdb(postgres://localhost); ? -- marko -- Sent

Re: [HACKERS] pg_crypto failures with llvm on OSX

2012-03-09 Thread Marko Kreen
On Fri, Mar 09, 2012 at 05:28:20PM -0500, Andrew Dunstan wrote: Buildfarm member mussel (OS X 10.7.3, llvm-gcc 4.2.1, x86_64)seems to be getting consistent warnings when running the pgcrypto regression tests, that look like this: WARNING: detected write past chunk end in ExprContext

Re: [HACKERS] [9.2] Confusion over CacheRegisterSyscacheCallback

2012-03-07 Thread Marko Kreen
On Tue, Mar 06, 2012 at 04:27:11PM -0500, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: On Tue, Mar 06, 2012 at 11:10:38AM -0500, Tom Lane wrote: Why would you need to know that? The reason the calculation function is static is that there's no apparent need to expose

Re: [HACKERS] [9.2] Confusion over CacheRegisterSyscacheCallback

2012-03-07 Thread Marko Kreen
On Wed, Mar 07, 2012 at 11:16:06AM -0500, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: On Tue, Mar 06, 2012 at 04:27:11PM -0500, Tom Lane wrote: Or you could do like setrefs.c does, and assume you know how to calculate the hash value for an OID-keyed cache. Ok, the hashoid

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-06 Thread Marko Kreen
On Tue, Mar 06, 2012 at 11:13:45AM +0900, Kyotaro HORIGUCHI wrote: But it's broken in V3 protocol - getAnotherTuple() will be called only if the packet is fully read. If the packet contents do not agree with packet header, it's protocol error. Only valid EOF return in V3

[HACKERS] [9.2] Confusion over CacheRegisterSyscacheCallback

2012-03-06 Thread Marko Kreen
Commit b5282aa893: Revise sinval code to remove no-longer-used tuple TID from inval messages. dropped ItemPointer from callbacks and replaced it with hashValue. There seems to be 2 ways that new backend code calculates it: - hashoid(oid), which seems to assume too much? -

Re: [HACKERS] [9.2] Confusion over CacheRegisterSyscacheCallback

2012-03-06 Thread Marko Kreen
On Tue, Mar 06, 2012 at 11:10:38AM -0500, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: So my question is that after doing generic SearchSysCache() like: tup = SearchSysCache(USERMAPPINGUSERSERVER, ObjectIdGetDatum(user_mapping-userid), ObjectIdGetDatum

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-01 Thread Marko Kreen
On Tue, Feb 28, 2012 at 05:04:44PM +0900, Kyotaro HORIGUCHI wrote: There is still one EOF in v3 getAnotherTuple() - pqGetInt(tupnfields), please turn that one also to protocolerror. pqGetInt() returns EOF only when it wants additional reading from network if the parameter `bytes' is

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-27 Thread Marko Kreen
On Mon, Feb 27, 2012 at 05:20:30PM +0900, Kyotaro HORIGUCHI wrote: Hello, On OOM, the old result is freed to have higher chance that constructing new result succeeds. But if we want to transport error message, we need to keep old PGresult around. Thus two separate paths. Ok, I

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-26 Thread Marko Kreen
On Fri, Feb 24, 2012 at 05:46:16PM +0200, Marko Kreen wrote: - rename to PQrecvRow() and additionally provide PQgetRow() I tried it and it seems to work as API - there is valid behaviour for both sync and async connections. Sync connection - PQgetRow() waits for data from network

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-24 Thread Marko Kreen
On Fri, Feb 24, 2012 at 07:53:14PM +0900, Kyotaro HORIGUCHI wrote: Hello, this is new version of the patch. By the way, I would like to ask you one question. What is the reason why void* should be head or tail of the parameter list? Aesthetical reasons: I got it. Thank you.

[HACKERS] Let's drop V2 protocol

2012-02-24 Thread Marko Kreen
On Fri, Feb 24, 2012 at 02:11:45PM +0200, Marko Kreen wrote: On Fri, Feb 24, 2012 at 07:53:14PM +0900, Kyotaro HORIGUCHI wrote: - I have no idea how to do test for protocol 2... I have a urge to test with rm fe-protocol2.c... Now I tested with 7.3.21 and the non-error case works fine

Re: [HACKERS] Let's drop V2 protocol

2012-02-24 Thread Marko Kreen
On Fri, Feb 24, 2012 at 4:26 PM, Merlin Moncure mmonc...@gmail.com wrote: On Fri, Feb 24, 2012 at 7:52 AM, Marko Kreen mark...@gmail.com wrote: So could we like, uh, not do it and simply drop the V2 code? I think it's plausible that very old client libraries could connect to a modern server

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-24 Thread Marko Kreen
On Tue, Feb 14, 2012 at 01:39:06AM +0200, Marko Kreen wrote: I tried imaging some sort of getFoo() style API for fetching in-flight row data, but I always ended up with rewrite libpq step, so I feel it's not productive to go there. Instead I added simple feature: rowProcessor can return '2

Re: [HACKERS] Let's drop V2 protocol

2012-02-24 Thread Marko Kreen
On Fri, Feb 24, 2012 at 11:46:19AM -0500, Tom Lane wrote: As for taking it out of libpq, I would vote against doing that as long as we have pg_dump support for pre-7.4 servers. Now, I think it would be entirely reasonable to kill pg_dump's support for pre-7.3 servers, because that would

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-23 Thread Marko Kreen
On Thu, Feb 23, 2012 at 07:14:03PM +0900, Kyotaro HORIGUCHI wrote: Hello, this is new version of the patch. Looks good. By the way, I would like to ask you one question. What is the reason why void* should be head or tail of the parameter list? Aesthetical reasons: 1) PGresult and

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-21 Thread Marko Kreen
On Tue, Feb 21, 2012 at 11:42 AM, Kyotaro HORIGUCHI horiguchi.kyot...@oss.ntt.co.jp wrote: Also, as row handler is on connection, then changing it's behavior is connection context, not result. OK, current implement copying the pointer to the row processor from PGconn to PGresult and

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-21 Thread Marko Kreen
On Tue, Feb 21, 2012 at 12:13 PM, Kyotaro HORIGUCHI horiguchi.kyot...@oss.ntt.co.jp wrote: - PQskipResult(conn, true) makes all consequent PQgetResult()'s  to skip all the rows. Well, Is this right? Yes, call getResult() until it returns NULL. If this is right, row processor should stay

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-21 Thread Marko Kreen
On Wed, Feb 22, 2012 at 12:27:57AM +0900, Kyotaro HORIGUCHI wrote: fe-exec.c - PQskipResult() is added instead of PGskipRemainigResults(). It must free the PGresults that PQgetResult() returns. Also, please fix 2 issues mentined here:

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-20 Thread Marko Kreen
On Tue, Feb 21, 2012 at 02:11:35PM +0900, Kyotaro HORIGUCHI wrote: I don't have any attachment to PQskipRemainingResults(), but I think that some formal method to skip until Command-Complete('C') without breaking session is necessary because current libpq does so. We have such function:

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-18 Thread Marko Kreen
Demos/tests of the new API: https://github.com/markokr/libpq-rowproc-demos Comments resulting from that: - PQsetRowProcessorErrMsg() should take const char* - callback API should be (void *, PGresult *, PQrowValue*) or void* at the end, but not in the middle I have not looked yet what

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Marko Kreen
On Sun, Feb 19, 2012 at 1:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: Yeah ... if you *don't* know the difference between a DFA and an NFA, you're likely to find yourself in over your head.  Having said that,

Re: [HACKERS] Command Triggers

2012-02-17 Thread Marko Kreen
On Fri, Feb 17, 2012 at 10:54 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: Wait, we already have ALTER TRIGGER bob ON ANY COMMAND SET DISABLED; Eh, so what happens then if someone sets a command trigger on ALTER TRIGGER? We should remove

Re: [HACKERS] Command Triggers

2012-02-17 Thread Marko Kreen
On Fri, Feb 17, 2012 at 4:04 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Marko Kreen mark...@gmail.com writes: Btw, we already have a GUC for triggers: session_replication_role, how will the command triggers follow that? Note that the replica here in my mind would have been an Hot

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-16 Thread Marko Kreen
On Thu, Feb 16, 2012 at 02:24:19PM +0900, Kyotaro HORIGUCHI wrote: As far as I see, on an out-of-memory in getAnotherTuple() makes conn-result-resultStatus = PGRES_FATAL_ERROR and qpParseInputp[23]() skips succeeding 'D' messages consequently. When exception raised within row processor,

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-16 Thread Marko Kreen
On Thu, Feb 16, 2012 at 05:49:34PM +0900, Kyotaro HORIGUCHI wrote: I added the function PQskipRemainingResult() and use it in dblink. This reduces the number of executing try-catch block from the number of rows to one per query in dblink. This implementation is wrong - you must not simply

Re: [HACKERS] pg_test_fsync performance

2012-02-14 Thread Marko Kreen
On Tue, Feb 14, 2012 at 05:59:06PM -0500, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: On Mon, Feb 13, 2012 at 08:28:03PM -0500, Tom Lane wrote: +1, I was about to suggest the same thing. Running any of these tests for a fixed number of iterations will result in drastic

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-13 Thread Marko Kreen
On Tue, Feb 07, 2012 at 04:44:09PM +0200, Marko Kreen wrote: Although it seems we could allow exceptions, at least when we are speaking of Postgres backend, as the connection and result are internally consistent state when the handler is called, and the partial PGresult is stored under PGconn

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-08 Thread Marko Kreen
On Wed, Feb 08, 2012 at 02:44:13PM +0900, Shigeru Hanada wrote: (2012/02/07 23:44), Marko Kreen wrote: On Tue, Feb 07, 2012 at 07:25:14PM +0900, Shigeru Hanada wrote: - What is the right (or recommended) way to prevent from throwing exceptoin in row-processor callback function? When author

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-07 Thread Marko Kreen
On Tue, Feb 07, 2012 at 07:25:14PM +0900, Shigeru Hanada wrote: (2012/02/02 23:30), Marko Kreen wrote: On Thu, Feb 02, 2012 at 04:51:37PM +0900, Kyotaro HORIGUCHI wrote: Hello, This is new version of dblink.c - Memory is properly freed when realloc returns NULL in storeHandler

Re: [HACKERS] ecpglib use PQconnectdbParams

2012-02-06 Thread Marko Kreen
On Fri, Feb 03, 2012 at 01:15:30PM +0100, Christian Ullrich wrote: * Peter Eisentraut wrote: I noticed ecpglib still uses PQconnectdb() with a craftily assembled connection string. Here is a patch to use PQconnectdbParams() instead. + const char *conn_keywords[6]; + const char

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-02 Thread Marko Kreen
On Tue, Jan 31, 2012 at 8:56 PM, Robert Haas robertmh...@gmail.com wrote: 2012/1/29 Kohei KaiGai kai...@kaigai.gr.jp:     Remote SQL: DECLARE pgsql_fdw_cursor_0 SCROLL CURSOR FOR SELECT a, b FROM public.t1 WHERE (a 2)  (3 rows) Shouldn't we be using protocol-level cursors rather than

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-02 Thread Marko Kreen
On Thu, Feb 02, 2012 at 04:51:37PM +0900, Kyotaro HORIGUCHI wrote: Hello, This is new version of dblink.c - Memory is properly freed when realloc returns NULL in storeHandler(). - The bug that free() in finishStoreInfo() will be fed with garbage pointer when malloc for sinfo-valbuflen

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-01 Thread Marko Kreen
On Wed, Feb 1, 2012 at 10:32 AM, Kyotaro HORIGUCHI horiguchi.kyot...@oss.ntt.co.jp wrote: Another thing: if realloc() fails, the old pointer stays valid. So it needs to be assigned to temp variable first, before overwriting old pointer.  mmm. I've misunderstood of the realloc.. I'll fix there

[HACKERS] libpq: fix sslcompression leak

2012-02-01 Thread Marko Kreen
freePGconn() does not free sslcompression. Fix. Found with valgrind. -- marko diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index 5add143..4605e49 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -2803,6 +2803,8 @@

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-30 Thread Marko Kreen
On Mon, Jan 30, 2012 at 06:06:57PM +0900, Kyotaro HORIGUCHI wrote: The gain of performance is more than expected. Measure script now does query via dblink ten times for stability of measuring, so the figures become about ten times longer than the previous ones. sec

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-30 Thread Marko Kreen
On Tue, Jan 31, 2012 at 4:59 AM, Kyotaro HORIGUCHI horiguchi.kyot...@oss.ntt.co.jp wrote: This is fixed version of dblink.c for row processor. i'll re-send the properly fixed patch for dblink.c later. - malloc error in initStoreInfo throws ERRCODE_OUT_OF_MEMORY. (new error) - storeHandler()

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-27 Thread Marko Kreen
On Fri, Jan 27, 2012 at 05:57:01PM +0900, Kyotaro HORIGUCHI wrote: Hello, This is a new version of the patch formerly known as 'alternative storage for libpq'. - Changed the concept to 'Alternative Row Processor' from 'Storage handler'. Symbol names are also changed. - Callback function

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-27 Thread Marko Kreen
On Fri, Jan 27, 2012 at 09:35:04AM -0600, Merlin Moncure wrote: On Fri, Jan 27, 2012 at 2:57 AM, Kyotaro HORIGUCHI - The meaning of PGresAttValue is changed. The field 'value' now  contains a value withOUT terminating zero. This change seems to  have no effect on any other portion within

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Marko Kreen
On Tue, Jan 24, 2012 at 09:33:52PM -0500, Robert Haas wrote: Furthermore, while we haven't settled the question of exactly what a good negotiation facility would look like, we seem to agree that a GUC isn't it. I think that means this isn't going to happen for 9.2, so we should mark this

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Marko Kreen
On Wed, Jan 25, 2012 at 10:23:14AM -0500, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: On Tue, Jan 24, 2012 at 09:33:52PM -0500, Robert Haas wrote: Furthermore, while we haven't settled the question of exactly what a good negotiation facility would look like, we seem to agree

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Marko Kreen
On Wed, Jan 25, 2012 at 11:40:28AM -0500, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: On Wed, Jan 25, 2012 at 10:23:14AM -0500, Tom Lane wrote: Huh? How can that work? If we decide to change the representation of some other well known type, say numeric, how do we decide whether

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Marko Kreen
On Wed, Jan 25, 2012 at 12:58:15PM -0500, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: On Wed, Jan 25, 2012 at 11:40:28AM -0500, Tom Lane wrote: Then why bother with the bit in the format code? If you've already done some other negotiation to establish what datatype formats you

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Marko Kreen
On Wed, Jan 25, 2012 at 12:54:00PM -0600, Merlin Moncure wrote: On Wed, Jan 25, 2012 at 11:24 AM, Marko Kreen mark...@gmail.com wrote: I specifically want to avoid any sort of per-connection negotation, except the max format version supported, because it will mess up multiplexed usage

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Marko Kreen
On Wed, Jan 25, 2012 at 01:43:03PM -0600, Merlin Moncure wrote: On Wed, Jan 25, 2012 at 1:24 PM, Marko Kreen mark...@gmail.com wrote: On Wed, Jan 25, 2012 at 12:54:00PM -0600, Merlin Moncure wrote: On Wed, Jan 25, 2012 at 11:24 AM, Marko Kreen mark...@gmail.com wrote: I specifically want

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Marko Kreen
On Wed, Jan 25, 2012 at 02:50:09PM -0600, Merlin Moncure wrote: On Wed, Jan 25, 2012 at 2:29 PM, Marko Kreen mark...@gmail.com wrote: well, I see the following cases: 1) Vserver Vapplication: server downgrades wire formats to applications version 2) Vapplication Vlibpq Vserver: since

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-23 Thread Marko Kreen
On Sat, Jan 21, 2012 at 1:52 PM, Marc Mamin m.ma...@intershop.de wrote: c. Refine error handling of dblink.c. I think it preserves the    previous behavior for column number mismatch and type    conversion exception. Hello, I don't know if this cover following issue. I just mention it

GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Marko Kreen
On Sun, Jan 22, 2012 at 11:47 PM, Mikko Tiihonen mikko.tiiho...@nitorcreations.com wrote: * introduced a new GUC variable array_output copying the current  bytea_output type, with values full (old value) and  smallfixed (new default) * added documentation for the new GUC variable If this

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Marko Kreen
On Mon, Jan 23, 2012 at 5:34 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Jan 23, 2012 at 9:36 AM, Florian Weimer fwei...@bfk.de wrote: * Robert Haas: In this particular case, I knew that the change was coming and could push updated Java and Perl client libraries well before the

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Marko Kreen
On Mon, Jan 23, 2012 at 11:20:52AM -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 23, 2012 at 9:59 AM, Marko Kreen mark...@gmail.com wrote: Now that I think about it, same applies to bytea_output? Probably so. But I think we need not introduce quite so many

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Marko Kreen
On Mon, Jan 23, 2012 at 11:06 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Jan 23, 2012 at 11:15 AM, Noah Misch n...@leadboat.com wrote: As I said upthread, and you appeared to agree, the protocol is independent of individual data type send/recv formats.  Even if we were already adding

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-20 Thread Marko Kreen
On Tue, Jan 17, 2012 at 05:53:33PM +0900, Kyotaro HORIGUCHI wrote: Hello, This is revised and rebased version of the patch. a. Old term `Add Tuple Function' is changed to 'Store Handler'. The reason why not `storage' is simply length of the symbols. b. I couldn't find the place to

Re: [HACKERS] [RFC] grants vs. inherited tables

2012-01-03 Thread Marko Kreen
On Mon, Jan 02, 2012 at 12:31:13PM +0100, Dimitri Fontaine wrote: Marko Kreen mark...@gmail.com writes: I tried to generalize a function that creates partitions for a table and found out it's impossible to do it for grants. Basically, what I want is a child table that takes it's grants

Re: [HACKERS] [RFC] grants vs. inherited tables

2011-12-30 Thread Marko Kreen
On Thu, Dec 29, 2011 at 11:11:22PM -0300, Alvaro Herrera wrote: Excerpts from Marko Kreen's message of jue dic 29 15:22:49 -0300 2011: On Thu, Dec 29, 2011 at 03:12:50PM -0300, Alvaro Herrera wrote: Excerpts from Marko Kreen's message of jue dic 29 15:04:49 -0300 2011: 3) Have a way

Re: [HACKERS] [RFC] grants vs. inherited tables

2011-12-29 Thread Marko Kreen
On Thu, Dec 29, 2011 at 03:12:50PM -0300, Alvaro Herrera wrote: Excerpts from Marko Kreen's message of jue dic 29 15:04:49 -0300 2011: 3) Have a way to format aclitem into something that can used to create GRANT statement easily. Eg: pg_get_privilege_info( IN priv

[HACKERS] [RFC] grants vs. inherited tables

2011-12-29 Thread Marko Kreen
Hello, I tried to generalize a function that creates partitions for a table and found out it's impossible to do it for grants. Basically, what I want is a child table that takes it's grants from parent table. IMHO quite reasonable request. But I don't see a way to do it in pl/pgsql. (Writing

Re: [HACKERS] Disable OpenSSL compression

2011-11-10 Thread Marko Kreen
On Thu, Nov 10, 2011 at 5:18 PM, Albe Laurenz laurenz.a...@wien.gv.at wrote: I could go and try to convince Npgsql and JDBC to accept patches to do that on the client side, but that would be more effort than I want to invest.  But then there's still closed source software like Devart

Re: [HACKERS] Disable OpenSSL compression

2011-11-08 Thread Marko Kreen
On Tue, Nov 8, 2011 at 3:59 PM, Albe Laurenz laurenz.a...@wien.gv.at wrote: It is possible that this could cause a performance regression for people who SELECT lots of compressible data over really slow network connections, but is that a realistic scenario? Yes, it's a realistic scenario.

Re: [HACKERS] Disable OpenSSL compression

2011-11-08 Thread Marko Kreen
On Tue, Nov 8, 2011 at 4:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Tue, Nov 8, 2011 at 3:59 PM, Albe Laurenz laurenz.a...@wien.gv.at wrote: It is possible that this could cause a performance regression for people who SELECT lots of compressible data

Re: [HACKERS] Multiple queries in transit

2011-11-03 Thread Marko Kreen
On Mon, Oct 31, 2011 at 7:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 31.10.2011 17:44, Mark Hills wrote: Could libpq be reasonably modified to allow this? I believe it's doable in theory, no-one has just gotten around to it.

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Marko Kreen
On Tue, Oct 25, 2011 at 3:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: 2011/10/25 Shigeru Hanada shigeru.han...@gmail.com: I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a contrib module.  I think that this module would be the

Re: [HACKERS] Cross-compile of 3rd-party extensions fails

2011-09-19 Thread Marko Kreen
On Mon, Sep 19, 2011 at 11:27 AM, Peter Eisentraut pete...@gmx.net wrote: On mån, 2011-09-19 at 10:16 +0200, Boszormenyi Zoltan wrote: [zozo@localhost postgis-1.5.3]$ mingw64-make make -C liblwgeom make[1]: Entering directory `/home/zozo/Schönig-számlák/w64/nsis/9.0/postgis-1.5.3/liblwgeom'

Re: [HACKERS] sha1, sha2 functions into core?

2011-08-12 Thread Marko Kreen
On Thu, Aug 11, 2011 at 5:46 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... which this approach would create, because digest() isn't restricted to just those algorithms.  I think it'd

Re: [HACKERS] sha1, sha2 functions into core?

2011-08-12 Thread Marko Kreen
On Thu, Aug 11, 2011 at 5:46 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... which this approach would create, because digest() isn't restricted to just those algorithms.  I think it'd

Re: [HACKERS] sha1, sha2 functions into core?

2011-08-11 Thread Marko Kreen
On Wed, Aug 10, 2011 at 9:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: I would like to see whether there is support for adding sha1 and sha2 functions into the core. I can't get excited about that, but could put up with it as long as there wasn't scope

Re: [HACKERS] txid_current() forces a real xid

2011-07-11 Thread Marko Kreen
On Mon, Jul 11, 2011 at 5:59 PM, Bruce Momjian br...@momjian.us wrote: Right now, calling txid_current() causes a session to create a non-virtual xid if not already assigned, so observing the xid creates it, which seems kind of odd.  Is that intended?  Here is the C code: Yes, it was

Re: [HACKERS] Please test peer (socket ident) auth on *BSD

2011-06-02 Thread Marko Kreen
On Wed, Jun 1, 2011 at 1:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: My suggestion would be to use getpeereid() everywhere. And just have compat getpeereid() implementation on non-BSD platforms.  This would minimize ifdeffery in core core. Hm, maybe.  I'd

Re: [HACKERS] Please test peer (socket ident) auth on *BSD

2011-06-02 Thread Marko Kreen
On Thu, Jun 2, 2011 at 5:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: Here's my attempt for it.  As conditional port module seems trouble, I set up an unconditional pgGetpeereid() that is always defined. -1 ... why would you think that a conditional

Re: [HACKERS] Please test peer (socket ident) auth on *BSD

2011-06-02 Thread Marko Kreen
On Thu, Jun 2, 2011 at 7:20 PM, Andrew Dunstan and...@dunslane.net wrote: On 06/02/2011 12:04 PM, Marko Kreen wrote: On Thu, Jun 2, 2011 at 6:59 PM, Andrew Dunstanand...@dunslane.net  wrote: On 06/02/2011 11:29 AM, Marko Kreen wrote: As there was no going back now, I even touched msvc.pm

Re: [HACKERS] Please test peer (socket ident) auth on *BSD

2011-06-02 Thread Marko Kreen
On Thu, Jun 2, 2011 at 6:59 PM, Andrew Dunstan and...@dunslane.net wrote: On 06/02/2011 11:29 AM, Marko Kreen wrote: As there was no going back now, I even touched msvc.pm. Why? Windows doesn't have Unix domain sockets at all. Because the function is still referenced in the code. -- marko

Re: [HACKERS] Please test peer (socket ident) auth on *BSD

2011-06-02 Thread Marko Kreen
On Thu, Jun 2, 2011 at 7:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Thu, Jun 2, 2011 at 7:20 PM, Andrew Dunstan and...@dunslane.net wrote: Then maybe we need to use #ifndef WIN32 in those places. That's what we do for similar cases. No, that would

Re: [HACKERS] Please test peer (socket ident) auth on *BSD

2011-06-02 Thread Marko Kreen
On Thu, Jun 2, 2011 at 7:31 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Andrew Dunstan's message of jue jun 02 11:59:02 -0400 2011: On 06/02/2011 11:29 AM, Marko Kreen wrote: As there was no going back now, I even touched msvc.pm. Why? Windows doesn't have Unix domain

Re: [HACKERS] Please test peer (socket ident) auth on *BSD

2011-05-31 Thread Marko Kreen
On Wed, Jun 1, 2011 at 12:22 AM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark gsst...@mit.edu writes: On Tue, May 31, 2011 at 12:38 PM, Peter Eisentraut pete...@gmx.net wrote: Oh yes, no point in having complicated code that doesn't get exercised. This does amount to desupporting old

Re: [HACKERS] Please test peer (socket ident) auth on *BSD

2011-05-31 Thread Marko Kreen
On Wed, Jun 1, 2011 at 1:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: My suggestion would be to use getpeereid() everywhere. And just have compat getpeereid() implementation on non-BSD platforms.  This would minimize ifdeffery in core core. Hm, maybe.  I'd

Re: [HACKERS] Why not install pgstattuple by default?

2011-05-19 Thread Marko Kreen
On Wed, May 18, 2011 at 3:37 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, May 18, 2011 at 15:29, Marko Kreen mark...@gmail.com wrote: On Wed, May 18, 2011 at 2:57 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, May 18, 2011 at 10:25, Greg Smith g...@2ndquadrant.com wrote: Some

Re: [HACKERS] Why not install pgstattuple by default?

2011-05-18 Thread Marko Kreen
On Wed, May 18, 2011 at 2:57 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, May 18, 2011 at 10:25, Greg Smith g...@2ndquadrant.com wrote: Some of my personal discussions of this topic have suggested that some other popular extensions like pgcrypto and hstore get converted too.  I think

<    1   2   3   4   5   6   7   >