Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-04-09 Thread Jan Urbański
Peter Eisentraut writes: On 2/12/15 7:28 AM, Jan Urbański wrote: +#if OPENSSL_VERSION_NUMBER 0x1000 +/* OpenSSL 1.0.0 deprecates the CRYPTO_set_id_callback function and provides a + * default implementation, so there's no need for our own. */ I have some additional concerns about

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-04-03 Thread Jan Urbański
Peter Eisentraut writes: On 4/2/15 4:32 AM, Jan Urbański wrote: Peter Eisentraut writes: I don't think this patch would actually fix the problem that was described after the original bug report (http://www.postgresql.org/message-id/5436991b.5020...@vmware.com), namely that another thread

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-04-02 Thread Jan Urbański
Peter Eisentraut writes: On 2/12/15 7:28 AM, Jan Urbański wrote: For the sake of discussion, here's a patch to prevent stomping on previously-set callbacks, racy as it looks. FWIW, it does fix the Python deadlock and doesn't cause the PHP segfault... I don't think this patch would

Re: [HACKERS] GSoC idea - Simulated annealing to search for query plans

2015-02-27 Thread Jan Urbański
GEQO replacement at PGCon a couple years back. It never got to the point of being a submitted patch though. Yea, it was Jan Urbański (CCed). And the project link: https://github.com/wulczer/saio So what w'ere saying, Grzegorz, is that we would love to see someone pick this up and get

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-12 Thread Jan Urbański
Andres Freund writes: On 2015-02-12 09:31:27 +0100, Jan Urbański wrote: That doesn't solve the problem of the Python deadlock, where you're not at leisure to call a C function at the beginning of your module. We could just never unload the hooks... That's what we did before

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-12 Thread Jan Urbański
Andres Freund writes: On 2015-02-09 18:17:14 +0100, Jan Urbański wrote: First of all, the current behaviour is crazy. We're setting and unsetting the locking callback every time a connection is made/closed, which is not how OpenSSL is supposed to be used. The *application* using libpq should

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-12 Thread Jan Urbański
Jan Urbański writes: Andres Freund writes: On 2015-02-12 09:31:27 +0100, Jan Urbański wrote: That doesn't solve the problem of the Python deadlock, where you're not at leisure to call a C function at the beginning of your module. We could just never unload the hooks... That's what we

Re: [HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-11 Thread Jan Urbański
Jan Urbański writes: I did some more digging on bug http://www.postgresql.org/message-id/cahul3dpwyfnugdgo95ohydq4kugdnbkptjq0mnbtubhcmg4...@mail.gmail.com which describes a deadlock when using libpq with SSL in a multi-threaded environment with other threads doing SSL independently

[HACKERS] libpq's multi-threaded SSL callback handling is busted

2015-02-09 Thread Jan Urbański
I did some more digging on bug http://www.postgresql.org/message-id/cahul3dpwyfnugdgo95ohydq4kugdnbkptjq0mnbtubhcmg4...@mail.gmail.com which describes a deadlock when using libpq with SSL in a multi-threaded environment with other threads doing SSL independently. Attached is a reproducing Python

Re: [HACKERS] Min value for port

2013-06-27 Thread Jan Urbański
On 27/06/13 15:11, Magnus Hagander wrote: On Thu, Jun 27, 2013 at 2:16 PM, Peter Eisentraut pete...@gmx.net wrote: On 6/27/13 6:34 AM, Magnus Hagander wrote: Is there a reason why we have set the min allowed value for port to 1, not 1024? Given that you can't actually start postgres with a

Re: [HACKERS] updated emacs configuration

2013-06-26 Thread Jan Urbański
On 26/06/13 10:51, Dimitri Fontaine wrote: Peter Eisentraut pete...@gmx.net writes: $ git clone git://git.postgresql.org/git/postgresql.git Cloning into 'postgresql'... I can reproduce that here. I don't know why I have those postgres dirs then, and I'm pretty confused about my round of

Re: [HACKERS] storing plpython global pointer

2013-05-28 Thread Jan Urbański
On 28/05/13 14:04, Szymon Guz wrote: Hi, I need to store a global pointer for plpython usage. This is a PyObject* which can be initialized per session I think Where should I keep such a pointer? Hi, you probably could use a global variable, similar to PLy_interp_globals that's defined in

Re: [HACKERS] plpgsql versus SPI plan abstraction

2013-01-30 Thread Jan Urbański
On 30/01/13 22:23, Tom Lane wrote: BTW, I'm also wondering if it's really necessary for plpython/plpy_spi.c to be looking into spi_priv.h ... As far as I can tell, it's not necessary, spi.h would be perfectly fine. Cheers, Jan -- Sent via pgsql-hackers mailing list

Re: [HACKERS] pg_dump --pretty-print-views

2013-01-28 Thread Jan Urbański
On 28/01/13 12:31, Marko Tiikkaja wrote: On 1/28/13 12:14 PM, Jeevan Chalke wrote: I could not apply the patch with git apply, but able to apply it by patch -p1 command. IME that's normal for patches that went through filterdiff. I do: git diff |filterdiff --format=context to re-format the

[HACKERS] fix SQL example syntax in file comment

2013-01-12 Thread Jan Urbański
Hi, Here's a trivial patch that fixes a comment in execProcNode.c For archeological interest, that comment dates back to when it was written in POSTQUEL... The cleanup in a9b1ff4c1d699c8aa615397d47bb3071275c64ef changed RETRIEVE to SELECT, but forgot to add a FROM clause :) Cheers, Jan

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2013-01-09 Thread Jan Urbański
On 12/12/12 20:21, Karl O. Pinc wrote: There were 2 outstanding issue raised: Is this useful enough/proceeding in the right direction to commit now? Should some of the logic be moved out of a subroutine and into the calling code? I can see arguments to be made for both sides. I'm

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-12-12 Thread Jan Urbański
Though not the original patch autor, I did modify and submit it to the CF app, so I'll reply here :) On 10/12/12 19:20, Karl O. Pinc wrote: On 12/09/2012 10:33:59 PM, Karl O. Pinc wrote: I've gone ahead and signed up to review this patch. Thanks! There were 2 outstanding issue raised:

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-11-05 Thread Jan Urbański
On 05/11/12 18:35, Robert Haas wrote: On Wed, Oct 31, 2012 at 5:33 AM, Jan Urbańskiwulc...@wulczer.org wrote: On 30/10/12 22:06, Oskari Saarenmaa wrote: PL/Python maps Python SPIError exceptions with 'spidata' attribute into SQL errors. Here's an alternative patch that takes advantage of

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-11-05 Thread Jan Urbański
On 05/11/12 19:07, Jan Urbański wrote: On 05/11/12 18:35, Robert Haas wrote: You should probably add this to the next CF so we don't forget about it. I will, as soon as I recover my community account. Added as https://commitfest.postgresql.org/action/patch_view?id=971 J -- Sent via

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-10-31 Thread Jan Urbański
On 30/10/12 22:06, Oskari Saarenmaa wrote: PL/Python maps Python SPIError exceptions with 'spidata' attribute into SQL errors. PL/Python also creates classes in plpy.spiexceptions for all known errors but does not initialize their spidata, so when a PL/Python function raises such an exception

[HACKERS] trivial typo in src/tools/RELEASE_CHANGES

2012-09-23 Thread Jan Urbański
There's a typo in src/tools/RELEASE_CHANGES It just ticked off my OCD I guess... Cheers, Jan diff --git a/src/tools/RELEASE_CHANGES b/src/tools/RELEASE_CHANGES new file mode 100644 index 5f1277a..aba1630 *** a/src/tools/RELEASE_CHANGES --- b/src/tools/RELEASE_CHANGES *** Then doing

Re: [HACKERS] PostgreSQL 9.2beta4 ( git HEAD) server crash on creating extension plpython3u

2012-08-22 Thread Jan Urbański
On 21/08/12 20:13, Josh Berkus wrote: No. I get the same backtrace when I try against the 9.1.5 (REL9_1_STABLE) branch. I have reproduced this on Linux, seems like the fix is to to run the postmaster with this env variable exported: PYTHONHOME=/opt/ActivePython-3.2/ (or wherever you

Re: [HACKERS] PostgreSQL 9.2beta4 ( git HEAD) server crash on creating extension plpython3u

2012-08-22 Thread Jan Urbański
On 22/08/12 13:28, Sachin Srivastava wrote: Yes, It worked for me also.. So will this be a workaround? Or do we intend to use something like Py_SetPythonHome() before calling Py_Initialize()/ I think the best we can do is to document that for some installations you might need to set

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-08-09 Thread Jan Urbański
On 06/08/12 13:59, Heikki Linnakangas wrote: On 20.07.2012 10:13, Jan Urbański wrote: On 20/07/12 08:59, Jan Urbański wrote: On 18/07/12 17:17, Heikki Linnakangas wrote: On 14.07.2012 17:50, Jan Urbański wrote: If pg_do_encoding_conversion() throws an error, you don't get a chance to call

Re: [HACKERS] Passing tabular data around using python functions

2012-07-31 Thread Jan Urbański
On 30/07/12 14:33, Achim Domma wrote: Hi, Hi Achim, this list is meant for discussing the development of PostgreSQL, in the future you might want to ask your question on pgsql-general. However, to answer your question: I call the function like this: select * from

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-20 Thread Jan Urbański
On 18/07/12 17:17, Heikki Linnakangas wrote: On 14.07.2012 17:50, Jan Urbański wrote: If pg_do_encoding_conversion() throws an error, you don't get a chance to call Py_DECREF() to release the string. Is that a problem? If an error occurs in PLy_traceback(), after incrementing recursion_depth

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-20 Thread Jan Urbański
On 20/07/12 08:59, Jan Urbański wrote: On 18/07/12 17:17, Heikki Linnakangas wrote: On 14.07.2012 17:50, Jan Urbański wrote: If pg_do_encoding_conversion() throws an error, you don't get a chance to call Py_DECREF() to release the string. Is that a problem? If an error occurs in PLy_traceback

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-14 Thread Jan Urbański
On 13/07/12 13:38, Jan Urbański wrote: On 12/07/12 11:08, Heikki Linnakangas wrote: On 07.07.2012 00:12, Jan Urbański wrote: So you're in favour of doing unicode - bytes by encoding with UTF-8 and then using the server's encoding functions? Sounds reasonable to me. The extra conversion

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-13 Thread Jan Urbański
On 12/07/12 11:08, Heikki Linnakangas wrote: On 07.07.2012 00:12, Jan Urbański wrote: On 06/07/12 22:47, Peter Eisentraut wrote: On fre, 2012-07-06 at 18:53 +0300, Heikki Linnakangas wrote: What shall we do about those? Ignore them? Document that if you're sing one of these encodings then PL

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-06 Thread Jan Urbański
On 06/07/12 10:05, Heikki Linnakangas wrote: On 06.07.2012 00:54, Jan Urbański wrote: On 05/07/12 23:30, Peter Eisentraut wrote: On tor, 2012-07-05 at 22:53 +0200, Jan Urbański wrote: The problem is that PLyUnicode_Bytes is (via an ifdef) used as PyString_ToString on Python3, which means

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-06 Thread Jan Urbański
On 06/07/12 10:14, Jan Urbański wrote: On 06/07/12 10:05, Heikki Linnakangas wrote: In the backend elog routines, there is a global variable 'recursion_depth', which is incremented when an error-handling routine is entered, and decremented afterwards. Can we use a similar mechinism in PLy_elog

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-06 Thread Jan Urbański
On 06/07/12 22:47, Peter Eisentraut wrote: On fre, 2012-07-06 at 18:53 +0300, Heikki Linnakangas wrote: What shall we do about those? Ignore them? Document that if you're sing one of these encodings then PL/Python with Python 2 will be crippled and with Python 3 just won't work? We could

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-05 Thread Jan Urbański
On 05/07/12 21:37, Heikki Linnakangas wrote: Committed. This bug was present in versions = 9.0, so backpatched. Thanks! I used ereport() rather than elog() in the error message. Correct me if that was wrong, but the point was to avoid PLy_elog(), because that might cause recursion, and

[HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-05 Thread Jan Urbański
On 05/07/12 22:37, Heikki Linnakangas wrote: On 05.07.2012 23:31, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@iki.fi writes: Fix mapping of PostgreSQL encodings to Python encodings. The buildfarm doesn't like this --- did you check for side effects on regression test results? Hmm,

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings.

2012-07-05 Thread Jan Urbański
On 05/07/12 23:30, Peter Eisentraut wrote: On tor, 2012-07-05 at 22:53 +0200, Jan Urbański wrote: The problem is that PLyUnicode_Bytes is (via an ifdef) used as PyString_ToString on Python3, which means that there are numerous call sites and new ones might appear in any moment. I'm

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-04 Thread Jan Urbański
On 04/07/12 13:58, Asif Naeem wrote: Patch attached. Asif, could you try a few things on a CP1252 database? First verify if your original test case now works and then try this: I have test the patch on Win64. postgres server is working fine now for WIN1252. Thanks. create function

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-03 Thread Jan Urbański
On 29/06/12 00:36, Jan Urbański wrote: On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. So: I'd add code to translate WINxxx into CPxxx when choosing the Python to use

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-03 Thread Jan Urbański
On 03/07/12 17:45, Jan Urbański wrote: On 29/06/12 00:36, Jan Urbański wrote: On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. So: I'd add code to translate WINxxx

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-28 Thread Jan Urbański
On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. CREATE PROCEDURAL LANGUAGE 'plpython3u'; CREATE OR REPLACE FUNCTION pymax (a integer, b integer) RETURNS integer AS $$ if a b

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-27 Thread Jan Urbański
On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. CREATE PROCEDURAL LANGUAGE 'plpython3u'; CREATE OR REPLACE FUNCTION pymax (a integer, b integer) RETURNS integer AS $$ if a b: return a return b $$

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-11 Thread Jan Urbański
On 10/05/12 19:45, Peter Eisentraut wrote: On lör, 2012-05-05 at 22:45 +0200, Jan Urbański wrote: Apparently once you implement PyMappingMethods.mp_subscript you can drop PySequenceMethods.sq_slice, but I guess there's no harm in keeping it (and I'm not sure it'd work on Python 2.3 with only

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-05 Thread Jan Urbański
On 04/05/12 20:00, Jan Urbański wrote: On 03/05/12 11:04, Jan Urbański wrote: On 02/05/12 20:18, Peter Eisentraut wrote: This doesn't work anymore with Python 3: rv = plpy.execute(...) do_something(rv[0:1]) Sounds ugly. I'll take a look. I found some instructions on how to deal

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-04 Thread Jan Urbański
On 03/05/12 11:04, Jan Urbański wrote: On 02/05/12 20:18, Peter Eisentraut wrote: This doesn't work anymore with Python 3: rv = plpy.execute(...) do_something(rv[0:1]) Apparently, they changed the C API for doing slicing, or rather made one of the two APIs for it silently do nothing. Details

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-03 Thread Jan Urbański
On 02/05/12 20:18, Peter Eisentraut wrote: This doesn't work anymore with Python 3: rv = plpy.execute(...) do_something(rv[0:1]) Apparently, they changed the C API for doing slicing, or rather made one of the two APIs for it silently do nothing. Details are difficult to find, but this email

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-22 Thread Jan Urbański
On 10/04/12 21:47, Jan Urbański wrote: On 10/04/12 21:27, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?=wulc...@wulczer.org writes: Yes, that would be ideal, even though not backwards-compatible. Back-patching is out of the question, but do we want to change trigger functions to receive

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Jan Urbański
On 10/04/12 07:35, Jan Urbański wrote: On 10/04/12 04:20, Tom Lane wrote: Don't know if anybody noticed bug #6559 http://archives.postgresql.org/pgsql-bugs/2012-03/msg00180.php I've confirmed that the given test case works in 9.0 but fails in 9.1 and HEAD. So, I know what's going on, I still

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Jan Urbański
On 10/04/12 20:47, Tom Lane wrote: I wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?=wulc...@wulczer.org writes: Now that I understand what's been going on, I'll try to think of a non-invasive way of fixing that... ISTM that conversion of a composite value to Python ought to produce a dict, now

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Jan Urbański
On 10/04/12 21:27, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?=wulc...@wulczer.org writes: Yes, that would be ideal, even though not backwards-compatible. Back-patching is out of the question, but do we want to change trigger functions to receive dictionaries in NEW? Hm, I was not

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-09 Thread Jan Urbański
On 10/04/12 04:20, Tom Lane wrote: Don't know if anybody noticed bug #6559 http://archives.postgresql.org/pgsql-bugs/2012-03/msg00180.php I've confirmed that the given test case works in 9.0 but fails in 9.1 and HEAD. I find this code pretty unreadable, though, and know nothing to speak of

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-22 Thread Jan Urbański
On 21/02/12 18:28, Jan Urbański wrote: On 21/02/12 18:05, Peter Eisentraut wrote: it might be better to use ereport, to expose the message for translation. After giving it some thought some of these elogs could be changed into PLy_elogs (which is meant to propagate a Python error

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-21 Thread Jan Urbański
On 21/02/12 18:05, Peter Eisentraut wrote: On sön, 2012-02-19 at 22:29 -0500, Tom Lane wrote: My only comment is whether elog(ERROR) is appropriate, ie, do we consider these to be internal errors that users will never see in practice? If there's a significant risk of the error being thrown in

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-20 Thread Jan Urbański
On 20/02/12 04:29, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: On 18/02/12 21:17, Tom Lane wrote: Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some preliminary results for plpython.c:

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-19 Thread Jan Urbański
On 18/02/12 21:18, Jan Urbański wrote: On 18/02/12 21:17, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: On 18/02/12 20:30, Tom Lane wrote: Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some

Re: [HACKERS] pl/python long-lived allocations in datum-dict transformation

2012-02-19 Thread Jan Urbański
On 14/02/12 01:35, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: It's not very comfortable, but I think PLyDict_FromTuple can be allowed to be non-reentrant. I think that's pretty short-sighted. Even if it's safe today (which I am not 100% convinced of),

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-18 Thread Jan Urbański
On 18/02/12 20:30, Tom Lane wrote: Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some preliminary results for plpython.c: https://bugzilla.redhat.com/show_bug.cgi?id=795011 I'm not enough of a Python hacker to evaluate

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-18 Thread Jan Urbański
On 18/02/12 21:17, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: On 18/02/12 20:30, Tom Lane wrote: Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some preliminary results for plpython.c:

Re: [HACKERS] pl/python long-lived allocations in datum-dict transformation

2012-02-13 Thread Jan Urbański
On 12/02/12 00:48, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: This is annoying for functions that plough through large tables, doing some calculation. Attached is a patch that does the conversion of PostgreSQL Datums into Python dict objects in a scratch

[HACKERS] pl/python long-lived allocations in datum-dict transformation

2012-02-05 Thread Jan Urbański
Consider this: create table arrays as select array[random(), random(), random(), random(), random(), random()] as a from generate_series(1, 100); create or replace function plpython_outputfunc() returns void as $$ c = plpy.cursor('select a from arrays') for row in c: pass $$ language

[HACKERS] plpgsql leaking memory when stringifying datums

2012-02-05 Thread Jan Urbański
While chasing a PL/Python memory leak, I did a few tests with PL/pgSQL and I think there are places where memory is not freed sufficiently early. Attached are two functions that when run will make the backend's memory consumption increase until they finish. With both, the cause is

[HACKERS] unfriendly error when accessing NEW in statement-level trigger

2012-01-28 Thread Jan Urbański
When you try to use the NEW variable in a statement-level trigger, you get ERROR: record new is not assigned yet DETAIL: The tuple structure of a not-yet-assigned record is indeterminate. which is not too friendly, given that people sometimes forget to specify FOR EACH sth at all, get the

Re: [HACKERS] PL/Python result metadata

2012-01-11 Thread Jan Urbański
On 11/01/12 22:52, Dimitri Fontaine wrote: Peter Eisentraut pete...@gmx.net writes: .colnames() returns a list of column names (strings) .coltypes() returns a list of type OIDs (integers) I just made that up because there is no guidance in the other standard PLs for this sort of thing,

Re: [HACKERS] pgsphere

2012-01-07 Thread Jan Urbański
- Original message - On Saturday, January 07, 2012 04:43:43 PM Dave Cramer wrote: Well I've sent Teodor a personal email asking him if he was interested and so far no response, so I interpret that as he no longer has interest in the project. I dimly remember him mentioning

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Jan Urbański
On 18/12/11 04:21, Robert Haas wrote: On Sat, Dec 17, 2011 at 7:50 PM, David E. Wheeler da...@kineticode.com wrote: Love having the start here. I forwarded this message to Claes Jakobsson, creator of the jansson-using pg-json extension. He’s a bit less supportive. He gave me permission to

Re: [HACKERS] splitting plpython into smaller parts

2011-12-18 Thread Jan Urbański
On 18/12/11 20:53, Peter Eisentraut wrote: On tis, 2011-12-06 at 00:58 +0100, Jan Urbański wrote: Rebased against master after the SPI cursor patch has been committed. The first patch removes SPI boilerplate from the cursor functions as well and the second patch creates a plpython_cursor.c

Re: [HACKERS] JSON for PG 9.2

2011-12-12 Thread Jan Urbański
- Original message - On Dec 12, 2011, at 4:51 PM, Peter van Hardenberg wrote: Because we haven't heard from him in a while we've been using PL/V8 to validate a JSON datatype simulated by a DOMAIN with a simple acceptance function. (See below.) This is not ideally performant but

Re: [HACKERS] [COMMITTERS] pgsql: plpython: Add SPI cursor support

2011-12-07 Thread Jan Urbański
On 06/12/11 19:33, Jan Urbański wrote: On 06/12/11 19:23, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: plpython: Add SPI cursor support Buildfarm member narwhal does not like this patch. It looks like PyObject_SelfIter is not a compile-time constant on its version of python

Re: [HACKERS] [COMMITTERS] pgsql: plpython: Add SPI cursor support

2011-12-07 Thread Jan Urbański
On 07/12/11 11:16, Jan Urbański wrote: On 06/12/11 19:33, Jan Urbański wrote: On 06/12/11 19:23, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: plpython: Add SPI cursor support Buildfarm member narwhal does not like this patch. It looks like PyObject_SelfIter is not a compile

Re: [HACKERS] [COMMITTERS] pgsql: plpython: Add SPI cursor support

2011-12-06 Thread Jan Urbański
On 06/12/11 19:23, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: plpython: Add SPI cursor support Buildfarm member narwhal does not like this patch. It looks like PyObject_SelfIter is not a compile-time constant on its version of python (2.5, apparently). Hm, I quickly tried

Re: [HACKERS] plpython SPI cursors

2011-12-05 Thread Jan Urbański
On 05/12/11 18:58, Peter Eisentraut wrote: On ons, 2011-11-23 at 19:58 +0100, Jan Urbański wrote: On 20/11/11 19:14, Steve Singer wrote: Responding now to all questions and attaching a revised patch based on your comments. Committed Please refresh the other patch. Great, thanks! I'll

Re: [HACKERS] plpython SPI cursors

2011-12-05 Thread Jan Urbański
On 05/12/11 19:12, Bruce Momjian wrote: Jan Urbański wrote: On 05/12/11 18:58, Peter Eisentraut wrote: On ons, 2011-11-23 at 19:58 +0100, Jan Urba?ski wrote: On 20/11/11 19:14, Steve Singer wrote: Responding now to all questions and attaching a revised patch based on your comments

Re: [HACKERS] Java LISTEN/NOTIFY client library work-around

2011-11-30 Thread Jan Urbański
On 30/11/11 13:07, Joel Jacobson wrote: Hi, As you know, LISTEN/NOTIFY is broken in the Java client library. You have to do a SELECT 1 in a while-loop to receive the notifications. http://jdbc.postgresql.org/documentation/head/listennotify.html Is there some other library with a proper

Re: [HACKERS] splitting plpython into smaller parts

2011-11-28 Thread Jan Urbański
On 28/11/11 11:00, Greg Smith wrote: On 11/13/2011 09:45 AM, Jan Urbański wrote: The second one is the actual split. plpython.c has been split into 11 separate files and one header. Could you comment a bit more about what the goal of this is? We don't have a reviewer for this patch yet, and I

Re: [HACKERS] PL/Python SQL error code pass-through

2011-11-24 Thread Jan Urbański
On 23/11/11 17:24, Mika Eloranta wrote: Hi all, [PL/Python in 9.1 does not preserve SQLSTATE of errors] Oops, you're right, it's a regression from 9.0 behaviour. The fix looks good to me, I changed one place to indent with tabs instead of spaces and added a regression test. I think this

Re: [HACKERS] PL/Python SQL error code pass-through

2011-11-24 Thread Jan Urbański
On 24/11/11 16:15, Heikki Linnakangas wrote: On 24.11.2011 10:07, Jan Urbański wrote: On 23/11/11 17:24, Mika Eloranta wrote: Hi all, [PL/Python in 9.1 does not preserve SQLSTATE of errors] Oops, you're right, it's a regression from 9.0 behaviour. The fix looks good to me, I changed one

Re: [HACKERS] plpython SPI cursors

2011-11-23 Thread Jan Urbański
On 20/11/11 19:14, Steve Singer wrote: On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory (that's what plpy.execute does). J I

Re: [HACKERS] plpython SPI cursors

2011-11-20 Thread Jan Urbański
On 20/11/11 19:14, Steve Singer wrote: On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. I found a few bugs (see my testing section below) that will need fixing + a few questions about the code Hi Steve, thanks a lot

Re: [HACKERS] [9.1] unusable for large views

2011-10-24 Thread Jan Urbański
On 24/10/11 10:57, Omar Bettin wrote: [monster query] I see that your problem is already solved, but incidentially I'm working on a join order planning module and I'm looking for real-life examples of humongous queries like that to benchmark against them. Any chance you could share the

Re: [HACKERS] Monster query

2011-10-24 Thread Jan Urbański
On 24/10/11 19:16, Omar Bettin wrote: Hi, Since the data are of third parties, will prepare a database suitable for the purpose. In any case, the compressed backup will be around 20 MB. If you are able to prepare a database dump that doesn't contain private data, it would be awesome. If

Re: [HACKERS] proposal: set GUC variables for single query

2011-10-17 Thread Jan Urbański
On 17/10/11 02:53, Robert Haas wrote: On Sun, Oct 16, 2011 at 4:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dimitri Fontaine dimi...@2ndquadrant.fr writes: Now that you mention it, the following might actually already work: WITH settings AS ( SELECT set_config('timezone',

[HACKERS] proposal: set GUC variables for single query

2011-10-16 Thread Jan Urbański
Hi, this idea has cropped up last PGCon - the ability to set GUC variables for the duration of a single query. It would work by setting the GUCs for the duration of the query and setting them back to what they were after it has terminated. By setting them back I mean respecting the previously set

Re: [HACKERS] proposal: set GUC variables for single query

2011-10-16 Thread Jan Urbański
On 16/10/11 17:49, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: this idea has cropped up last PGCon - the ability to set GUC variables for the duration of a single query. It would work by setting the GUCs for the duration of the query and setting them back to

[HACKERS] plpython SPI cursors

2011-10-15 Thread Jan Urbański
Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory (that's what plpy.execute does). This patch allows reading the result set in smaller chunks, using a SPI cursor behind the

Re: [HACKERS] Error building v9.1.1 (git) with python 3.2.2

2011-10-05 Thread Jan Urbański
On 05/10/11 15:24, Lou Picciano wrote: Hackers, Have been following with some interest the dialog recently about getting pl/python built within the v9 tree; most recently, I've been trying v9.1.1 (git) and python 3.2.2. Configure and make complete without error, but 'make install' does

Re: [HACKERS] Large C files

2011-09-06 Thread Jan Urbański
On 07/09/11 01:13, Peter Geoghegan wrote: On 6 September 2011 08:29, Peter Eisentraut pete...@gmx.net wrote: I was thinking about splitting up plpython.c, but it's not even on that list. ;-) IIRC the obesity of that file is something that Jan Urbański intends to fix, or is at least

Re: [HACKERS] any results from PL summit?

2011-08-29 Thread Jan Urbański
On 29/08/11 08:21, Pavel Stehule wrote: Hello Hi Pavel, is there some some result, report from PL summit? The notes taken during the summit are here: http://wiki.postgresql.org/wiki/PgCon_2011_PL_Summit I think Selena also sent a summary and next steps mail to the participants, perhaps it

Re: [HACKERS] plpython crash

2011-08-18 Thread Jan Urbański
On 17/08/11 23:10, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: Here are two patches that fix two separate bugs that you found simultaneously. Because they're actually separate issues, it turned out fixing them was a bit more tricky than I expected (fixing one

Re: [HACKERS] Caching Python modules

2011-08-17 Thread Jan Urbański
On 17/08/11 14:09, PostgreSQL - Hans-Jürgen Schönig wrote: CREATE OR REPLACE FUNCTION textprocess.add_to_corpus(lang text, t text) RETURNS float4 AS $$ from SecondCorpus import SecondCorpus from SecondDocument import SecondDocument i am doing some intense text mining

Re: [HACKERS] Caching Python modules

2011-08-17 Thread Jan Urbański
On 17/08/11 14:19, Jan Urbański wrote: On 17/08/11 14:09, PostgreSQL - Hans-Jürgen Schönig wrote: CREATE OR REPLACE FUNCTION textprocess.add_to_corpus(lang text, t text) RETURNS float4 AS $$ from SecondCorpus import SecondCorpus from SecondDocument import SecondDocument i

Re: [HACKERS] plpython crash

2011-08-17 Thread Jan Urbański
On 17/08/11 11:40, Jan Urbański wrote: On 16/08/11 19:12, Jan Urbański wrote: On 16/08/11 19:07, Jean-Baptiste Quenot wrote: [plpython is buggy] I'll have a patch ready soon. Here are two patches that fix two separate bugs that you found simultaneously. Because they're actually separate

Re: [HACKERS] rc1 or beta4?

2011-08-17 Thread Jan Urbański
On 17/08/11 15:00, Dave Page wrote: The current plan (or, the last one I recall) is to push another 9.1 release tomorrow, for Monday release. Are we going with beta4 or rc1? Sorry to butt in, but it would probably be good to include fixes for the two segfault plpython bugs[1] before wrapping up

Re: [HACKERS] rc1 or beta4?

2011-08-17 Thread Jan Urbański
On 17/08/11 17:50, Thom Brown wrote: On 17 August 2011 16:47, Jan Urbański wulc...@wulczer.org wrote: On 17/08/11 15:00, Dave Page wrote: The current plan (or, the last one I recall) is to push another 9.1 release tomorrow, for Monday release. Are we going with beta4 or rc1? Sorry to butt

Re: [HACKERS] plpython crash

2011-08-16 Thread Jan Urbański
On 16/08/11 16:52, Jean-Baptiste Quenot wrote: After backporting plpython.c from HEAD, this is the error message I get: ERROR: key pg.dropped.6 not found in mapping HINT: To return null in a column, add the value None to the mapping with the key named after the column.

Re: [HACKERS] plpython crash

2011-08-16 Thread Jan Urbański
On 16/08/11 17:06, Jan Urbański wrote: On 16/08/11 16:52, Jean-Baptiste Quenot wrote: After backporting plpython.c from HEAD, this is the error message I get: ERROR: key pg.dropped.6 not found in mapping HINT: To return null in a column, add the value None to the mapping

Re: [HACKERS] plpython crash

2011-08-16 Thread Jan Urbański
On 16/08/11 19:07, Jean-Baptiste Quenot wrote: Dear Jan, Sorry I typed the wrong git commands. With latest plpython from branch master I got the same gdb backtrace as reported before. I managed to wrap up a testcase that fails 100% of times on my setup: https://gist.github.com/1149512

Re: [HACKERS] plpython crash

2011-08-12 Thread Jan Urbański
On 11/08/11 18:01, Jean-Baptiste Quenot wrote: Hi there, plpython crashes on me on various 64-bit Ubuntu hosts, see the gdb backtrace at: https://gist.github.com/1140005 Do you believe there was recent bugfixes regarding PLyMapping_ToTuple() ? This is PG 9.0.4 with HEAD of plpython

Re: [HACKERS] plpython crash

2011-08-12 Thread Jan Urbański
On 12/08/11 13:55, Jean-Baptiste Quenot wrote: Here is the same with -O0: https://gist.github.com/1140005 sys.version reports this: INFO: 2.6.6 (r266:84292, Sep 15 2010, 16:41:53) [GCC 4.4.5] I'm still at a loss. Did you reproduce it with git HEAD? I see that the query being execute

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-22 Thread Jan Urbański
On 23/07/11 01:12, Robert Haas wrote: On Fri, Jul 22, 2011 at 6:04 PM, Joey Adams joeyadams3.14...@gmail.com wrote: On another matter, should the JSON type guard against duplicate member keys? The JSON RFC says The names within an object SHOULD be unique, meaning JSON with duplicate members

Re: [HACKERS] PL/Python: No stack trace for an exception

2011-07-21 Thread Jan Urbański
On 21/07/11 15:27, Sushant Sinha wrote: I am using plpythonu on postgres 9.0.2. One of my python functions was throwing a TypeError exception. However, I only see the exception in the database and not the stack trace. It becomes difficult to debug if the stack trace is absent in Python.

Re: [HACKERS] pgbench--new transaction type

2011-06-13 Thread Jan Urbański
On 13/06/11 06:38, Greg Smith wrote: On 06/11/2011 03:21 PM, Jeff Janes wrote: I wouldn't expect IPC chatter to show up in profiling, because it costs wall time, but not CPU time. The time spent might be attributed to the kernel, or to pgbench, or to nothing at all. Profilers aren't

Re: [HACKERS] increasing collapse_limits?

2011-05-06 Thread Jan Urbański
On 01/05/11 21:16, Joshua Berkus wrote: Speaking of which, what happened to replacing GEQO with Simulated Annealing? Where did that project go? It stayed on github (https://github.com/wulczer/saio) and stagnated a bit after I got my degree. It's on the top of my list of things to pick up

  1   2   3   4   >