Re: [HACKERS] [patch] PL/Python is too lossy with floats

2015-06-01 Thread Marko Kreen
On Wed, Mar 11, 2015 at 9:49 PM, Peter Eisentraut pete...@gmx.net wrote: On 3/3/15 9:32 AM, Marko Kreen wrote: PL/Python uses str(v) to convert float data, but is lossy by design. Only repr(v) is guaranteed to have enough precision to make floats roundtrip properly: committed In 9.3

[HACKERS] [patch] PL/Python is too lossy with floats

2015-03-03 Thread Marko Kreen
PL/Python uses str(v) to convert float data, but is lossy by design. Only repr(v) is guaranteed to have enough precision to make floats roundtrip properly: https://docs.python.org/2/library/functions.html#func-repr https://docs.python.org/2/library/functions.html#str Example: $ python

Re: [HACKERS] Supporting Windows SChannel as OpenSSL replacement

2014-06-09 Thread Marko Kreen
On Mon, Jun 09, 2014 at 02:45:08PM +0300, Heikki Linnakangas wrote: Thoughts? While we're at it, we'll probably want to refactor things so that it's easy to support other SSL implementations too, like gnutls. One project that is proud to support several SSL implementations is curl:

[HACKERS] [9.4] Minor SSL/ECDH related doc fixes

2014-05-17 Thread Marko Kreen
(Marko Kreen) /para /listitem -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Problem with txid_snapshot_in/out() functionality

2014-04-14 Thread Marko Kreen
On Sun, Apr 13, 2014 at 05:46:20PM -0400, Jan Wieck wrote: On 04/13/14 14:22, Jan Wieck wrote: On 04/13/14 08:27, Marko Kreen wrote: I think you need to do SET_VARSIZE also here. Alternative is to move SET_VARSIZE after sort_snapshot(). And it seems the drop-double-txid logic should

Re: [HACKERS] Problem with txid_snapshot_in/out() functionality

2014-04-13 Thread Marko Kreen
On Sat, Apr 12, 2014 at 02:10:13PM -0400, Jan Wieck wrote: Since it doesn't seem to produce any side effects, I'd think that making the snapshot unique within txid_current_snapshot() and filtering duplicates on input should be sufficient and eligible for backpatching. Agreed. The attached

Re: [HACKERS] SSL: better default ciphersuite

2014-02-23 Thread Marko Kreen
On Sat, Feb 22, 2014 at 08:31:14PM -0500, Peter Eisentraut wrote: On 2/2/14, 7:16 AM, Marko Kreen wrote: On Thu, Dec 12, 2013 at 04:32:07PM +0200, Marko Kreen wrote: Attached patch changes default ciphersuite to HIGH:MEDIUM:+3DES:!aNULL and also adds documentation about reasoning

Re: [HACKERS] SSL: better default ciphersuite

2014-02-02 Thread Marko Kreen
On Thu, Dec 12, 2013 at 04:32:07PM +0200, Marko Kreen wrote: Attached patch changes default ciphersuite to HIGH:MEDIUM:+3DES:!aNULL and also adds documentation about reasoning for it. This is the last pending SSL cleanup related patch: https://commitfest.postgresql.org/action/patch_view?id

Re: [HACKERS] [COMMITTERS] pgsql: libpq: Support TLS versions beyond TLSv1.

2014-01-31 Thread Marko Kreen
On Sat, Jan 25, 2014 at 12:25:30PM -0500, Tom Lane wrote: Alternatively, given that TLS has been around for a dozen years and openssl versions that old have not gotten security updates for a long time, why don't we just reject SSLv3 on the backend side too? I guess it's barely possible that

Re: [HACKERS] Fix memset usage in pgcrypto

2014-01-21 Thread Marko Kreen
On Mon, Jan 20, 2014 at 06:49:21PM -0300, Alvaro Herrera wrote: Marko Kreen escribió: http://www.viva64.com/en/b/0227/ reported that on-stack memset()s might be optimized away by compilers. Fix it. Just to clarify, this needs to be applied to all branches, right? If so, does the version

Re: [HACKERS] pgcrypto missing header file inclusions

2013-12-30 Thread Marko Kreen
On Sat, Dec 28, 2013 at 10:36:19PM -0500, Peter Eisentraut wrote: While playing around with the pginclude tools, I noticed that pgcrypto header files are failing to include some header files whose symbols they use. This change would fix it: diff --git a/contrib/pgcrypto/pgp.h

[HACKERS] Fix memset usage in pgcrypto

2013-12-26 Thread Marko Kreen
http://www.viva64.com/en/b/0227/ reported that on-stack memset()s might be optimized away by compilers. Fix it. * Replace memset() with px_memset() * Add px_memset to copy_crlf() * ADd px_memset to pgp-s2k.c -- marko diff --git a/contrib/pgcrypto/crypt-blowfish.c

Re: [HACKERS] SSL: better default ciphersuite

2013-12-17 Thread Marko Kreen
On Tue, Dec 17, 2013 at 11:26:13AM -0500, Bruce Momjian wrote: On Tue, Dec 17, 2013 at 09:51:30AM -0500, Robert Haas wrote: I'm starting to think we should just leave this well enough alone. We can't seem to find two people with the same idea of what would be better than what we have now.

Re: [HACKERS] SSL: better default ciphersuite

2013-12-17 Thread Marko Kreen
On Sun, Dec 15, 2013 at 05:10:38PM -0500, James Cloos wrote: MK == Marko Kreen mark...@gmail.com writes: PE == Peter Eisentraut pete...@gmx.net writes: PE Any other opinions on this out there? For reference, see: https://wiki.mozilla.org/Security/Server_Side_TLS for the currently

Re: [HACKERS] SSL: better default ciphersuite

2013-12-13 Thread Marko Kreen
On Thu, Dec 12, 2013 at 09:18:03PM -0500, Peter Eisentraut wrote: On Thu, 2013-12-12 at 12:30 +0200, Marko Kreen wrote: First, if there is explicit wish to keep RC4/SEED in play, I'm fine with HIGH:MEDIUM:!aNULL as new default. Clarity-wise, it's still much better than current value

Re: [HACKERS] SSL: better default ciphersuite

2013-12-12 Thread Marko Kreen
On Wed, Dec 11, 2013 at 10:08:44PM -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Any other opinions on this out there? All instances of other SSL-enabled servers out there, except nginx, default to some variant of DEFAULT:!LOW:... or HIGH:MEDIUM: The proposal here is

Re: [HACKERS] SSL: better default ciphersuite

2013-12-12 Thread Marko Kreen
On Thu, Dec 12, 2013 at 01:33:57PM +0100, Magnus Hagander wrote: On Thu, Dec 12, 2013 at 11:30 AM, Marko Kreen mark...@gmail.com wrote: On Wed, Dec 11, 2013 at 10:08:44PM -0500, Tom Lane wrote: I know that SChannel SSL library in Windows XP (and earlier) is such RC4+3DES only implementation

Re: [HACKERS] Feature request: Logging SSL connections

2013-12-08 Thread Marko Kreen
On Fri, Dec 06, 2013 at 02:53:27PM +0100, Dr. Andreas Kunert wrote: Anything else missing? Functionally it's fine now, but I see few style problems: - if (port-ssl 0) is wrong, -ssl is pointer. So use just if (port-ssl). - Deeper indentation would look nicer with braces. - There are

Re: [HACKERS] Feature request: Logging SSL connections

2013-12-06 Thread Marko Kreen
On Fri, Dec 06, 2013 at 11:43:55AM +0100, Dr. Andreas Kunert wrote: That seems useful. Do we need more information, like whether a client certificate was presented, or what ciphers were used? Yes, please show ciphersuite and TLS version too. Andreas, you can use my recent \conninfo patch

Re: [HACKERS] Feature request: Logging SSL connections

2013-12-05 Thread Marko Kreen
On Thu, Dec 05, 2013 at 09:43:31AM -0500, Peter Eisentraut wrote: On 12/5/13, 8:53 AM, Dr. Andreas Kunert wrote: we were really missing the information in our log files if (and which of) our users are using SSL during their connections. The attached patch is a very simple solution to

[HACKERS] [patch] libpq: Support TLSv1.1+ (was: fe-secure.c and SSL/TLS)

2013-12-03 Thread Marko Kreen
As pointed out by Jeffrey Walton, only SSLv23_method() will negotiate higher TLS versions than immediately requested. All other _method() functions will negotiate only one specific version. Attached are two patches: * libpq.tls11plus.diff Use SSLv23_method() instead TLSv1_method in

Re: [HACKERS] fe-secure.c and SSL/TLS

2013-11-30 Thread Marko Kreen
On Sat, Nov 30, 2013 at 03:46:06AM -0500, Jeffrey Walton wrote: I believe the standard way of achieving TLS1.0 and above is to use the SSLv23_client_method() and then remove the SSL protocols with SSL_OP_NO_SSLv2 and SSL_OP_NO_SSLv3. I have to use handwaiving around standard because I

Re: [HACKERS] [PATCH 1/2] SSL: GUC option to prefer server cipher order

2013-11-29 Thread Marko Kreen
On Fri, Nov 29, 2013 at 09:25:02AM -0500, Peter Eisentraut wrote: On Thu, 2013-11-14 at 11:45 +0100, Magnus Hagander wrote: I think the default behaviour should be the one we recommend (which would be to have the server one be preferred). But I do agree with the requirement to have a GUC to

Re: [HACKERS] SSL: better default ciphersuite

2013-11-29 Thread Marko Kreen
On Fri, Nov 29, 2013 at 09:18:49AM -0500, Peter Eisentraut wrote: On Fri, 2013-11-15 at 01:11 +0200, Marko Kreen wrote: Attached patch changes the default ciphersuite to HIGH:!aNULL instead of old DEFAULT:!LOW:!EXP:!MD5:@STRENGTH where DEFAULT is a shortcut for ALL

Re: [HACKERS] [PATCH 1/2] SSL: GUC option to prefer server cipher order

2013-11-29 Thread Marko Kreen
On Fri, Nov 29, 2013 at 05:51:28PM +0200, Heikki Linnakangas wrote: On 11/29/2013 05:43 PM, Marko Kreen wrote: On Fri, Nov 29, 2013 at 09:25:02AM -0500, Peter Eisentraut wrote: On Thu, 2013-11-14 at 11:45 +0100, Magnus Hagander wrote: I think the default behaviour should be the one we

Re: [HACKERS] fe-secure.c and SSL/TLS

2013-11-29 Thread Marko Kreen
Reply to mails in pgsql-bugs: http://www.postgresql.org/message-id/CAH8yC8mc_2J2UY0Q42WQdWFyaoqT3onG+83Fr=vn46j5+ml...@mail.gmail.com and http://www.postgresql.org/message-id/CAH8yC8nZVUyCQznkQd8=ELMM4k_=uxjrjt8yf9v22cy2x_d...@mail.gmail.com * Default ciphersuite I would argue nothing

Re: [HACKERS] fe-secure.c and SSL/TLS

2013-11-29 Thread Marko Kreen
On Fri, Nov 29, 2013 at 06:01:01PM -0500, Jeffrey Walton wrote: I know of no other ways to check the result of OpenSSL's chain validation. The open question (for me) is where are SSL_get_verify_result/X509_V_OK checked? Neither show up in the Postgres sources. According to SSL_set_verify

Re: [HACKERS] PL/Python: domain over array support

2013-11-27 Thread Marko Kreen
On Tue, Nov 26, 2013 at 07:12:00PM -0200, Rodolfo Campero wrote: 2013/11/26 Heikki Linnakangas hlinnakan...@vmware.com Oops, sorry about that. Fixed. Maybe be you forgot to modify plpython_types_3.out Yes. Heikki, please fix plpython_types_3.out too. See attached patch. -- marko diff

Re: [HACKERS] PL/Python: domain over array support

2013-11-26 Thread Marko Kreen
On Tue, Nov 26, 2013 at 12:23:48AM +0200, Heikki Linnakangas wrote: The new behavior is clearly better, but it is an incompatibility nonetheless. I don't do anything with PL/python myself, so I don't have a good feel of how much that'll break people's applications. Probably not much I guess.

Re: [HACKERS] PL/Python: domain over array support

2013-11-24 Thread Marko Kreen
On Sat, Nov 23, 2013 at 11:09:53AM -0200, Rodolfo Campero wrote: 2013/11/22 Marko Kreen mark...@gmail.com One more thing - please update Python 3 regtests too. The attached patch (version 3) includes the expected results for Python 3 (file plpython_types_3.out). Thanks. Looks good now

Re: [HACKERS] PL/Python: domain over array support

2013-11-22 Thread Marko Kreen
On Sat, Oct 26, 2013 at 11:17:19AM -0200, Rodolfo Campero wrote: The attached patch add support of domains over arrays to PL/Python (eg: CREATE DOMAIN my_domain AS integer[]). Basically it just uses get_base_element_type instead of get_element_type in plpy_typeio.c, and uses domain_check

Re: [HACKERS] PL/Python: domain over array support

2013-11-22 Thread Marko Kreen
On Fri, Nov 22, 2013 at 08:45:56AM -0200, Rodolfo Campero wrote: There are other cosmetic changes in this patch, wrt previous version (not preexistent code): * adjusted alignment of variable name rv in line 12 * reworded comment in line 850, resulting in more than 80 characters, so I

Re: [HACKERS] appendPQExpBufferVA vs appendStringInfoVA

2013-11-18 Thread Marko Kreen
On Mon, Nov 18, 2013 at 06:18:01PM +1300, David Rowley wrote: On Mon, Nov 18, 2013 at 1:01 AM, Marko Kreen mark...@gmail.com wrote: I am bit suspicious of performance impact of this patch, but think that it's still worthwhile as it decreases code style where single string argument is given

Re: [HACKERS] appendPQExpBufferVA vs appendStringInfoVA

2013-11-17 Thread Marko Kreen
On Thu, Nov 14, 2013 at 09:33:59PM +1300, David Rowley wrote: On Sun, Nov 3, 2013 at 3:18 AM, David Rowley dgrowle...@gmail.com wrote: I'm low on ideas on how to improve things much around here for now, but for what it's worth, I did create a patch which changes unnecessary calls to

Re: [HACKERS] Review:Patch: SSL: prefer server cipher order

2013-11-16 Thread Marko Kreen
On Fri, Nov 15, 2013 at 02:16:52PM -0800, Adrian Klaver wrote: On 11/15/2013 11:49 AM, Marko Kreen wrote: On Fri, Nov 15, 2013 at 11:16:25AM -0800, Adrian Klaver wrote: The description of the GUCs show up in the documentation but I am not seeing the GUCs themselves in postgresql.conf, so I

Re: [HACKERS] Review:Patch: SSL: prefer server cipher order

2013-11-16 Thread Marko Kreen
Thanks for testing! On Sat, Nov 16, 2013 at 12:17:40PM -0800, Adrian Klaver wrote: On 11/16/2013 06:24 AM, Marko Kreen wrote: ssl-better-default: SSL should stay working, openssl ciphers -v 'value' should not contain any weak suites (RC4, SEED, DES-CBC, EXP, NULL) and no non

Re: [HACKERS] Review:Patch: SSL: prefer server cipher order

2013-11-16 Thread Marko Kreen
On Sat, Nov 16, 2013 at 01:03:05PM -0800, Adrian Klaver wrote: On 11/16/2013 12:37 PM, Marko Kreen wrote: Thanks for testing! On Sat, Nov 16, 2013 at 12:17:40PM -0800, Adrian Klaver wrote: On 11/16/2013 06:24 AM, Marko Kreen wrote: ssl-better-default: SSL should stay working, openssl

Re: [HACKERS] Review:Patch: SSL: prefer server cipher order

2013-11-16 Thread Marko Kreen
On Sat, Nov 16, 2013 at 02:07:57PM -0800, Adrian Klaver wrote: On 11/16/2013 01:13 PM, Marko Kreen wrote: https://commitfest.postgresql.org/action/patch_view?id=1310 Got it, applied it. Results: openssl ciphers -v 'HIGH:!aNULL'|egrep '(RC4|SEED|DES-CBC|EXP|NULL|ADH|AECDH

Re: [HACKERS] Review:Patch: SSL: prefer server cipher order

2013-11-16 Thread Marko Kreen
On Sat, Nov 16, 2013 at 02:54:22PM -0800, Adrian Klaver wrote: On 11/16/2013 02:41 PM, Marko Kreen wrote: If you don't see any other issues perhaps they are ready for committer? I do not have any other questions/issues at this point. I am new to the review process, so I am not quite sure how

Re: [HACKERS] Review:Patch: SSL: prefer server cipher order

2013-11-16 Thread Marko Kreen
On Sat, Nov 16, 2013 at 03:21:19PM -0800, Adrian Klaver wrote: On 11/16/2013 03:09 PM, Marko Kreen wrote: On Sat, Nov 16, 2013 at 02:54:22PM -0800, Adrian Klaver wrote: On 11/16/2013 02:41 PM, Marko Kreen wrote: If you don't see any other issues perhaps they are ready for committer? I do

Re: [HACKERS] Review:Patch: SSL: prefer server cipher order

2013-11-15 Thread Marko Kreen
On Fri, Nov 15, 2013 at 11:16:25AM -0800, Adrian Klaver wrote: The description of the GUCs show up in the documentation but I am not seeing the GUCs themselves in postgresql.conf, so I could test no further. It is entirely possible I am missing a step and would appreciate enlightenment.

[HACKERS] SSL: better default ciphersuite

2013-11-14 Thread Marko Kreen
Attached patch changes the default ciphersuite to HIGH:!aNULL instead of old DEFAULT:!LOW:!EXP:!MD5:@STRENGTH where DEFAULT is a shortcut for ALL:!aNULL:!eNULL. Main goal is to leave low-level ciphersuite details to OpenSSL guys and give clear impression to Postgres admins what it is

[HACKERS] [PATCH 1/2] SSL: GUC option to prefer server cipher order

2013-11-06 Thread Marko Kreen
By default OpenSSL (and SSL/TLS in general) lets client cipher order take priority. This is OK for browsers where the ciphers were tuned, but few Postgres client libraries make cipher order configurable. So it makes sense to make cipher order in postgresql.conf take priority over client

[HACKERS] [PATCH 2/2] SSL: Support ECDH key excange.

2013-11-06 Thread Marko Kreen
This sets up ECDH key exchange, when compiling against OpenSSL that supports EC. Then ECDHE-RSA and ECDHE-ECDSA ciphersuites can be used for SSL connections. Latter one means that EC keys are now usable. The reason for EC key exchange is that it's faster than DHE and it allows to go to higher

Re: [HACKERS] [PATCH 1/2] SSL: GUC option to prefer server cipher order

2013-11-06 Thread Marko Kreen
On Wed, Nov 06, 2013 at 09:57:32PM -0300, Alvaro Herrera wrote: Marko Kreen escribió: By default OpenSSL (and SSL/TLS in general) lets client cipher order take priority. This is OK for browsers where the ciphers were tuned, but few Postgres client libraries make cipher order

Re: [HACKERS] preserving forensic information when we freeze

2013-07-03 Thread Marko Kreen
On Wed, Jul 3, 2013 at 8:07 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-07-02 16:29:56 -0400, Robert Haas wrote: There's no possibility of getting confused here; if X is still around at all, it's xmax is of the same generation as Y's xmin. Otherwise, we've had an undetected XID

Re: [HACKERS] MD5 aggregate

2013-06-27 Thread Marko Kreen
On Thu, Jun 27, 2013 at 11:28 AM, Dean Rasheed dean.a.rash...@gmail.com wrote: On 26 June 2013 21:46, Peter Eisentraut pete...@gmx.net wrote: On 6/26/13 4:04 PM, Dean Rasheed wrote: A quick google search reveals several people asking for something like this, and people recommending

Re: [HACKERS] MD5 aggregate

2013-06-17 Thread Marko Kreen
On Mon, Jun 17, 2013 at 11:34:52AM +0100, Dean Rasheed wrote: On 15 June 2013 10:22, Dean Rasheed dean.a.rash...@gmail.com wrote: There seem to be 2 separate directions that this could go, which really meet different requirements: 1). Produce an unordered sum for SQL to compare 2 tables

Re: [HACKERS] Add regression tests for DISCARD

2013-06-17 Thread Marko Kreen
On Mon, May 13, 2013 at 2:58 AM, Robins Tharakan thara...@gmail.com wrote: Please find attached a patch that adds basic regression tests for DISCARD command. Any and all feedback is obviously welcome. Perhaps existing tests in guc.sql should be merged into it? -- marko -- Sent via

Re: [HACKERS] Processing long AND/OR lists

2013-06-16 Thread Marko Kreen
On Mon, May 27, 2013 at 5:59 AM, Christopher Browne cbbro...@gmail.com wrote: This situation falls from a problem that we noticed a mighty long time ago in Slony, where the set of XIDs outstanding gets very large, and, attendant to that, the set of action id values by which tuples are being

Re: [HACKERS] MD5 aggregate

2013-06-14 Thread Marko Kreen
On Thu, Jun 13, 2013 at 12:35 PM, Dean Rasheed dean.a.rash...@gmail.com wrote: Attached is a patch implementing a new aggregate function md5_agg() to compute the aggregate MD5 sum across a number of rows. This is something I've wished for a number of times. I think the primary use case is to

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-05-14 Thread Marko Kreen
On Tue, May 14, 2013 at 09:29:38AM +0200, Dimitri Fontaine wrote: Stephen Frost sfr...@snowman.net writes: * Marko Kreen (mark...@gmail.com) wrote: On Sat, May 04, 2013 at 10:57:44PM +0200, Dimitri Fontaine wrote: Other than adminpack, I know of PGQ installing their objects

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-05-13 Thread Marko Kreen
On Sat, May 04, 2013 at 10:57:44PM +0200, Dimitri Fontaine wrote: Other than adminpack, I know of PGQ installing their objects in pg_catalog. They only began doing that when switching to the CREATE EXTENSION facility. And they set relocatable to false. FYI - PgQ and related modules install no

Re: [HACKERS] pgcrypto: Fix RSA password-protected keys

2013-05-10 Thread Marko Kreen
On Fri, May 10, 2013 at 12:52:55PM -0400, Tom Lane wrote: Marko Kreen mark...@gmail.com writes: RSA secret key extraction code uses wrong variable so that decryption is skipped and only secret keys without password work for pgp_pub_decrypt(). Attached patch fixes it and also adds

[HACKERS] pgcrypto: Fix RSA password-protected keys

2013-05-06 Thread Marko Kreen
RSA secret key extraction code uses wrong variable so that decryption is skipped and only secret keys without password work for pgp_pub_decrypt(). Attached patch fixes it and also adds regtest. Please apply to all branches. Reported-by: Keith Fiske ke...@omniti.com -- marko diff --git

Re: [HACKERS] [GENERAL] currval and DISCARD ALL

2013-04-17 Thread Marko Kreen
On Tue, Apr 16, 2013 at 05:09:19PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I think his point is why don't we clear currval() on DISCARD ALL? I can't think of a good reason we don't. Because we'd have to invent a new suboperation DISCARD SEQUENCES, for one thing, in

Re: [HACKERS] pgcrypto seeding problem when ssl=on

2013-01-14 Thread Marko Kreen
On Mon, Jan 14, 2013 at 12:46 AM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Fri, Dec 21, 2012 at 10:27 PM, Noah Misch n...@leadboat.com wrote: How about instead calling RAND_cleanup() after each backend fork? Attached is a patch that adds RAND_cleanup

Re: [HACKERS] pgcrypto seeding problem when ssl=on

2013-01-14 Thread Marko Kreen
On Mon, Jan 14, 2013 at 3:00 PM, Noah Misch n...@leadboat.com wrote: On Mon, Jan 14, 2013 at 02:21:00PM +0200, Marko Kreen wrote: Note: reading from /dev/urandom does not affect /dev/random. Reading from /dev/urandom drains the pool that serves /dev/random: $ cat /proc/sys/kernel/random

Re: [HACKERS] pgcrypto seeding problem when ssl=on

2013-01-13 Thread Marko Kreen
On Fri, Dec 21, 2012 at 10:27 PM, Noah Misch n...@leadboat.com wrote: How about instead calling RAND_cleanup() after each backend fork? Attached is a patch that adds RAND_cleanup() to fork_process(). That way all forked processes start with fresh state. This should make sure the problem does

Re: [HACKERS] allowing multiple PQclear() calls

2013-01-02 Thread Marko Kreen
On Wed, Jan 2, 2013 at 5:11 PM, Boszormenyi Zoltan z...@cybertec.at wrote: 2012-12-11 16:09 keltezéssel, Simon Riggs írta: On 11 December 2012 12:18, Boszormenyi Zoltan z...@cybertec.at wrote: Such mechanism already exist - you just need to set your PGresult pointer to NULL after each

Re: [HACKERS] pgcrypto seeding problem when ssl=on

2012-12-27 Thread Marko Kreen
On Sun, Dec 23, 2012 at 9:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Sat, Dec 22, 2012 at 9:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not thrilled with the suggestion to do RAND_cleanup() after forking though, as that seems like it'll guarantee

Re: [HACKERS] pgcrypto seeding problem when ssl=on

2012-12-23 Thread Marko Kreen
On Sat, Dec 22, 2012 at 9:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not thrilled with the suggestion to do RAND_cleanup() after forking though, as that seems like it'll guarantee that each session starts out with only a minimal amount of entropy. No, that's actually the right fix - that

[HACKERS] pgcrypto seeding problem when ssl=on

2012-12-21 Thread Marko Kreen
When there is 'ssl=on' then postmaster calls SSL_CTX_new(), which asks for random number, thus requiring initialization of randomness pool (RAND_poll). After that all forked backends think pool is already initialized. Thus they proceed with same fixed state they got from postmaster. Output is

Re: [HACKERS] pgcrypto seeding problem when ssl=on

2012-12-21 Thread Marko Kreen
On Fri, Dec 21, 2012 at 10:27 PM, Noah Misch n...@leadboat.com wrote: This should have gone to secur...@postgresql.org, instead. It went and this could have been patched in 9.2.2 but they did not care. On Fri, Dec 21, 2012 at 06:05:10PM +0200, Marko Kreen wrote: When there is 'ssl

Re: [HACKERS] allowing multiple PQclear() calls

2012-12-11 Thread Marko Kreen
On Tue, Dec 11, 2012 at 6:59 AM, Josh Kupershmidt schmi...@gmail.com wrote: Would it be crazy to add an already_freed flag to the pg_result struct which PQclear() would set, or some equivalent safety mechanism, to avoid this hassle for users? Such mechanism already exist - you just need to set

Re: [HACKERS] Successor of MD5 authentication, let's use SCRAM

2012-10-22 Thread Marko Kreen
On Fri, Oct 12, 2012 at 10:47 PM, Stephen Frost sfr...@snowman.net wrote: * Marko Kreen (mark...@gmail.com) wrote: As it works only on connect time, it can actually be secure, unlike user switching with SET ROLE. I'm guessing your issue with SET ROLE is that a RESET ROLE can be issued later

Re: [HACKERS] Successor of MD5 authentication, let's use SCRAM

2012-10-22 Thread Marko Kreen
On Wed, Oct 10, 2012 at 4:24 PM, Marko Kreen mark...@gmail.com wrote: The SCRAM looks good from the quick glance. SCRAM does have weakness - the info necessary to log in as client (ClientKey) is exposed during authentication process. IOW, the stored auth info can be used to log in as client

Re: [HACKERS] Successor of MD5 authentication, let's use SCRAM

2012-10-10 Thread Marko Kreen
On Wed, Oct 10, 2012 at 3:36 PM, Simon Riggs si...@2ndquadrant.com wrote: On 10 October 2012 11:41, Heikki Linnakangas hlinnakan...@vmware.com wrote: Thoughts on that? I think there has been enough discussion of md5 problems elsewhere that we should provide an alternative. If we can agree

Re: [HACKERS] [9.1] 2 bugs with extensions

2012-09-28 Thread Marko Kreen
On Wed, Sep 26, 2012 at 7:15 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Marko Kreen mark...@gmail.com writes: Can we work out a minimal example to reproduce the bug? Yes, the above text or sql/pgq_coop/sql/pgq_coop_init_ext.sql I guess you could replace pgq_coop with any extension

Re: [HACKERS] [9.1] 2 bugs with extensions

2012-09-26 Thread Marko Kreen
On Wed, Sep 26, 2012 at 5:36 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: After much distractions I've at least been able to do something about that bug report. Thanks. 2) If there is schema with functions, but nothing else, create extension pgq_coop from 'unpackaged'; drop extension

Re: [HACKERS] sha1, sha2 functions into core?

2012-08-15 Thread Marko Kreen
On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian br...@momjian.us wrote: Is there a TODO here? There is still open ToDecide here: 1) Status quo - md5() in core, everything else in pgcrypto 2) Start moving other hashes as separate functions into core: eg. sha1() Problems: - sha1 will be obsolete

Re: [HACKERS] sha1, sha2 functions into core?

2012-08-15 Thread Marko Kreen
On Wed, Aug 15, 2012 at 4:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian br...@momjian.us wrote: Is there a TODO here? There is still open ToDecide here: [snip] The argument against moving crypto code into core

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-08-02 Thread Marko Kreen
On Thu, Aug 2, 2012 at 8:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Wed, Aug 1, 2012 at 6:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: So I'm working from the first set of patches in your message 20120721194907.ga28...@gmail.com. Great! Here's

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-08-02 Thread Marko Kreen
On Fri, Aug 3, 2012 at 12:01 AM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Thu, Aug 2, 2012 at 8:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: In principle I suppose we could hack PQconsumeInput enough that it didn't damage the row buffer while still meeting its

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-08-01 Thread Marko Kreen
On Wed, Aug 1, 2012 at 6:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: * Is there better API than PQsetSingleRowMode()? New PQsend* functions is my alternative. After thinking it over, I'm really unexcited about adding new versions of all the PQsend

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 6:18 PM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Jul 23, 2012 at 5:07 PM, Marko Kreen mark...@gmail.com wrote: Does PQgetRowData() break the ability to call PQgetvalue() against the result as well as other functions like PQgetisnull()? If so, I object. I

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 7:08 PM, Marko Kreen mark...@gmail.com wrote: The PQgetRowData() is unimportant as it just exposes the rowBuf to user and thats all. So to get back to the more interesting PQgetRowData() discussion... Did you notice that it's up to app to decide whether it calls

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 7:25 PM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen mark...@gmail.com wrote: I'm arguing that *all* data getting must continue to do so through the result object, and bypassing the result to get at data is breaking the result

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 7:52 PM, Merlin Moncure mmonc...@gmail.com wrote: But, the faster rowbuf method is a generally incompatible way of dealing with data vs current libpq -- this is bad. If it's truly impossible to get those benefits without bypassing result API that then I remove my

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
Hm, I think it's possible to rig the test to do dummy copy of pgresult, thus it's possible to see what kind of speed is possible.. Will try. I added a new method (-x) to rowdump where it asks for row with PQgetRowData() and then tries to emulate super-efficient PGresult copy, then loads data

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 11:34 PM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 2:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: In particular I agree that PQsetSingleRowMode is a bit ugly, so I'm wondering about your thoughts on providing PQgetSingleRowResult instead. I don't

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 10:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: Either way, it looks like there's plausible paths to optimizing repeated result fetch without having expose an alternate data-fetching API and that the proposed implementation doesn't

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Wed, Jul 25, 2012 at 12:35 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 3:52 PM, Marko Kreen mark...@gmail.com wrote: * Is there better API than PQsetSingleRowMode()? New PQsend* functions is my alternative. I would prefer to have PQsetSingleRowMode() over new

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Wed, Jul 25, 2012 at 1:29 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 4:59 PM, Marko Kreen mark...@gmail.com wrote: So if we give only PQgetResult() in 9.2, I do not see that we are locked out from any interesting optimizations. Well, you are locked out of having

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-23 Thread Marko Kreen
Here is a simple test program that takes a SELECT query, reads it and outputs a COPY-formatted stream to standard output, to simulate some activity. It operates on 3 modes, specified by commant-line switches: -f Load full resultset at once - old way. -s Single-Row mode using PQgetResult().

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-23 Thread Marko Kreen
On Tue, Jul 24, 2012 at 12:27 AM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Jul 23, 2012 at 2:05 PM, Marko Kreen mark...@gmail.com wrote: It seems odd (but maybe ok) that you have to set the single row mode on the connection only to have the server reset it whenever you call a send

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-21 Thread Marko Kreen
Here is 2 approaches how to get to state where only PQsetSingleRowMode() is available. Both on top of REL9_2_STABLE branch. a) Remove callback hooks, keep rowBuf, implement single-row-mode on top of that. This was posted before, I just additionally removed the PQgetRowData() function.

[HACKERS] [9.1] 2 bugs with extensions

2012-07-18 Thread Marko Kreen
I converted Skytools modules to extensions and found 2 problems: 1) Dumpable sequences are not supported - if sequence is tagged with pg_catalog.pg_extension_config_dump(), the pg_dump tries to run COPY on it. 2) If there is schema with functions, but nothing else, then when later

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-16 Thread Marko Kreen
On Mon, Jul 16, 2012 at 4:11 AM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: Now, looking at the problem with some perspective, the solution is obvious: when in single-row mode, the PQgetResult() must return proper PGresult for that single row. And everything else

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-16 Thread Marko Kreen
On Mon, Jul 16, 2012 at 4:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Mon, Jul 16, 2012 at 4:11 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm starting to look at this patch now. I think we could drop the PQgetRowData() API: it complicates matters for little

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-16 Thread Marko Kreen
On Mon, Jul 16, 2012 at 6:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Marko Kreen mark...@gmail.com writes: On Mon, Jul 16, 2012 at 4:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Mm. I still think we should drop it, because it's still a dangerous API that's not necessary for the principal benefit

Re: [HACKERS] libpq compression

2012-06-20 Thread Marko Kreen
On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug f...@phlo.org wrote: I'm starting to think that relying on SSL/TLS for compression of unencrypted connections might not be such a good idea after all. We'd be using the protocol in a way it quite clearly never was intended to be used... Maybe,

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-19 Thread Marko Kreen
On Wed, Jun 13, 2012 at 2:28 PM, Andres Freund and...@2ndquadrant.com wrote: +/* + * removes a node from a list + * Attention: O(n) + */ +static inline void ilist_s_remove(ilist_s_head *head, +                                  ilist_s_node *node) +{ +       ilist_s_node *last = head-head;

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-19 Thread Marko Kreen
On Tue, Jun 19, 2012 at 11:02 PM, Andres Freund and...@2ndquadrant.com wrote: On Tuesday, June 19, 2012 09:59:48 PM Marko Kreen wrote: On Wed, Jun 13, 2012 at 2:28 PM, Andres Freund and...@2ndquadrant.com wrote: +/* + * removes a node from a list + * Attention: O(n) + */ +static

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Marko Kreen
On Mon, Jun 18, 2012 at 6:35 PM, Simon Riggs si...@2ndquadrant.com wrote: On 13 June 2012 19:28, Andres Freund and...@2ndquadrant.com wrote: This adds a new configuration parameter multimaster_node_id which determines the id used for wal originating in one cluster. Looks good and it seems

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-06-17 Thread Marko Kreen
On Sat, Jun 16, 2012 at 7:58 PM, Marko Kreen mark...@gmail.com wrote: So my preference would be to simply remove the callback API but keep the processing and provide PQgetRowData() instead. This is implemented in attached patch. It also converts dblink to use single-row API. The patch should

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-06-17 Thread Marko Kreen
On Sun, Jun 17, 2012 at 2:07 PM, Simon Riggs si...@2ndquadrant.com wrote: I prefer the description of Marko's API than the one we have now. Adopting one API in 9.2 and another in 9.3 would be fairly bad. Perhaps we can have both? I see no reason the keep the (public) callback API around,

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-06-16 Thread Marko Kreen
Demos: https://github.com/markokr/libpq-rowproc-demos/blob/master/demo-onerow-sync.c https://github.com/markokr/libpq-rowproc-demos/blob/master/demo-onerow-async.c Few clarifications below. On Fri, Jun 15, 2012 at 9:21 PM, Marko Kreen mark...@gmail.com wrote: Now, looking at the problem

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-06-16 Thread Marko Kreen
On Sat, Jun 16, 2012 at 6:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: I guess this raises the question of whether we ought to revert the row-callback patch entirely and support only this approach.  IMO it is (barely) not too late to do that for 9.2, if we want to. If we don't want to, then this

[HACKERS] [patch] libpq one-row-at-a-time API

2012-06-15 Thread Marko Kreen
(callback exec), but I did not do it yet to avoid affecting existing code too much. -- marko PS. If a squint it seems like fix of exising API instead of new feature, so perhaps it can still fit into 9.2? commit 4114613 (HEAD, single-row) Author: Marko Kreen mark...@gmail.com Date: Sat Apr 7

  1   2   3   4   5   6   7   >