Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread NISHIYAMA Tomoaki
I don't believe it is valid to ignore CJK characters above U+2. If it is used for names, it will be stored in the database. If the behaviour is different from characters below U+, you will get a bug report in meanwhile. see CJK Extension B, C, and D from http://www.unicode.org/charts/

[HACKERS] pg_regress application_name

2012-02-18 Thread Peter Eisentraut
I figured it would be good if pg_regress reported its application_name as pg_regress rather than psql. Any objections to the attached patch? diff --git i/src/test/regress/pg_regress.c w/src/test/regress/pg_regress.c index 2f6b37b..1384223 100644 --- i/src/test/regress/pg_regress.c +++

Re: [HACKERS] pg_regress application_name

2012-02-18 Thread Magnus Hagander
On Sat, Feb 18, 2012 at 11:47, Peter Eisentraut pete...@gmx.net wrote: I figured it would be good if pg_regress reported its application_name as pg_regress rather than psql.  Any objections to the attached patch? Sounds like a good idea to me, +1. --  Magnus Hagander  Me:

[HACKERS] mul_size() overflow check broken in win64 builds?

2012-02-18 Thread Tom Lane
Can anybody with a win64 build reproduce the misbehavior reported in bug #6460? I'm not currently interested in the question of whether my_log2 ought to be changed --- the question rather is why is the system not noticing that his shared_buffers value overflows size_t.

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 The time I got bitten by this was actually with LPAD(), rather than LIKE. +1. This is one of the functions that gave some of our clients real trouble when 8.3 came out. If we really believed that implicit casts any form were evil, we

[HACKERS] pg_restore ignores PGDATABASE

2012-02-18 Thread Erik Rijkers
pg_restore ignores environment variable PGDATABASE. Is this intentional? (perhaps because of the risk of restoring into the wrong db?) I would prefer if it would honor the PGDATABASE variable, but if it does ignore it intentionally, the following (from 9.2devel docs) is obviously incorrect:

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Tom Lane
NISHIYAMA Tomoaki tomoa...@staff.kanazawa-u.ac.jp writes: I don't believe it is valid to ignore CJK characters above U+2. If it is used for names, it will be stored in the database. If the behaviour is different from characters below U+, you will get a bug report in meanwhile. I am

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-02-18 Thread Jeff Janes
On Fri, Feb 17, 2012 at 7:36 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 17.02.2012 07:27, Fujii Masao wrote: Got another problem: when I ran pg_stop_backup to take an online backup, it got stuck until I had generated new WAL record. This happens because, in the

[HACKERS] Future of our regular expression code

2012-02-18 Thread Tom Lane
As those who've been paying attention to it know, our regular expression library is based on code originally developed by Henry Spencer and contributed by him to the Tcl project. We adopted it out of Tcl in 2003. Henry intended to package the code as a standalone library as well, but that never

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Simon Riggs
On Sat, Feb 18, 2012 at 6:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: So I'm feeling that we gotta suck it up and start acting like we are the lead maintainers for this code, not just consumers. By we, I take it you mean you personally? There are many requests I might make for allocations of

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: On Sat, Feb 18, 2012 at 6:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: So I'm feeling that we gotta suck it up and start acting like we are the lead maintainers for this code, not just consumers. By we, I take it you mean you personally? I'm

[HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-18 Thread Tom Lane
Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some preliminary results for plpython.c: https://bugzilla.redhat.com/show_bug.cgi?id=795011 I'm not enough of a Python hacker to evaluate the significance of these issues, but

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-18 Thread Robert Haas
On Tue, Feb 14, 2012 at 3:25 PM, Greg Smith g...@2ndquadrant.com wrote: On 02/14/2012 01:45 PM, Greg Smith wrote: scale=1000, db is 94% of RAM; clients=4 Version TPS 9.0  535 9.1  491 (-8.4% relative to 9.0) 9.2  338 (-31.2% relative to 9.1) A second pass through this data noted that the

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-18 Thread Jan Urbański
On 18/02/12 20:30, Tom Lane wrote: Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some preliminary results for plpython.c: https://bugzilla.redhat.com/show_bug.cgi?id=795011 I'm not enough of a Python hacker to evaluate

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Simon Riggs (si...@2ndquadrant.com) wrote: Do we have volunteers that might save Tom from taking on this task? It's not something that requires too much knowledge and experience of PostgreSQL, so is an easier task for a newcomer. Sure, it doesn't

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-18 Thread Simon Riggs
On Sat, Feb 18, 2012 at 7:35 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Feb 14, 2012 at 3:25 PM, Greg Smith g...@2ndquadrant.com wrote: On 02/14/2012 01:45 PM, Greg Smith wrote: scale=1000, db is 94% of RAM; clients=4 Version TPS 9.0  535 9.1  491 (-8.4% relative to 9.0) 9.2  338

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Simon Riggs
On Sat, Feb 18, 2012 at 7:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: One immediate consequence of deciding that we are lead maintainers and not just consumers is that we should put in some regression tests, instead of taking the attitude that the Tcl guys are in charge of that. I have a head

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Vik Reykja
On Sat, Feb 18, 2012 at 21:04, Simon Riggs si...@2ndquadrant.com wrote: On Sat, Feb 18, 2012 at 7:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: One immediate consequence of deciding that we are lead maintainers and not just consumers is that we should put in some regression tests, instead of

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-18 Thread Tom Lane
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: On 18/02/12 20:30, Tom Lane wrote: Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some preliminary results for plpython.c:

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Andrew Dunstan
On 02/18/2012 02:25 PM, Stephen Frost wrote: Do we have volunteers that might save Tom from taking on this task? It's not something that requires too much knowledge and experience of PostgreSQL, so is an easier task for a newcomer. Sure, it doesn't require knowledge of PG, but I dare say

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-18 Thread Jan Urbański
On 18/02/12 21:17, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: On 18/02/12 20:30, Tom Lane wrote: Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some preliminary results for plpython.c:

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Tom Lane
Vik Reykja vikrey...@gmail.com writes: On Sat, Feb 18, 2012 at 21:04, Simon Riggs si...@2ndquadrant.com wrote: Translating the test cases is a great way in for a volunteer, so please leave a few easy things to get people started on the road to maintaining that. I would be willing to have a

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Rob Wultsch
On Fri, Feb 17, 2012 at 4:12 PM, Josh Berkus j...@agliodbs.com wrote: On 2/17/12 12:04 PM, Robert Haas wrote: The argument isn't about whether the user made the right design choices; it's about whether he should be forced to insert an explicit type cast to get the query to do what it is

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-18 Thread Marko Kreen
Demos/tests of the new API: https://github.com/markokr/libpq-rowproc-demos Comments resulting from that: - PQsetRowProcessorErrMsg() should take const char* - callback API should be (void *, PGresult *, PQrowValue*) or void* at the end, but not in the middle I have not looked yet what

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-18 Thread Kohei KaiGai
2012年2月17日6:08 Shigeru Hanada shigeru.han...@gmail.com: (2012/02/17 2:02), Kohei KaiGai wrote: I found a strange behavior with v10. Is it available to reproduce? snip I tried to raise an error on remote side. postgres=# select * FROM ftbl WHERE 100 / (a - 3) 0; The connection to the

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Don Baccus
On Feb 18, 2012, at 12:57 PM, Rob Wultsch wrote: Where first_name is string the queries above have very different behaviour in MySQL. The first does a full table scan and coerces first_name to an integer (so '5adfs' - 5) Oh my, I can't wait to see someone rise to the defense of *this*

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Christopher Browne
On Sat, Feb 18, 2012 at 4:12 PM, Don Baccus dhog...@pacifier.com wrote: On Feb 18, 2012, at 12:57 PM, Rob Wultsch wrote: Where first_name is string the queries above have very different behaviour in MySQL. The first does a full table scan and coerces first_name to an integer (so '5adfs' - 5)

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Don Baccus
On Feb 18, 2012, at 1:43 PM, Christopher Browne wrote: On Sat, Feb 18, 2012 at 4:12 PM, Don Baccus dhog...@pacifier.com wrote: On Feb 18, 2012, at 12:57 PM, Rob Wultsch wrote: Where first_name is string the queries above have very different behaviour in MySQL. The first does a full table

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Dimitri Fontaine
Don Baccus dhog...@pacifier.com writes: A hierarchy like the following is perfectly logical: - to 0999 :: Cash accounts [1] Your example is actually a good argument for storing account ids as text, because '' like '0%' *will* match. FWIW, I too think that if you want to process your

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Yeah, it's conceivable that we could implement something whereby characters with codes above some cutoff point are handled via runtime calls to iswalpha() and friends, rather than being included in the statically-constructed DFA maps. The cutoff point could

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Andrew Dunstan
On 02/18/2012 05:34 PM, Don Baccus wrote: On Feb 18, 2012, at 1:43 PM, Christopher Browne wrote: On Sat, Feb 18, 2012 at 4:12 PM, Don Baccusdhog...@pacifier.com wrote: On Feb 18, 2012, at 12:57 PM, Rob Wultsch wrote: Where first_name is string the queries above have very different

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Yeah ... if you *don't* know the difference between a DFA and an NFA, you're likely to find yourself in over your head. Having said that, So, here's a paper I found very nice to get started into this subject: http://swtch.com/~rsc/regexp/regexp1.html If

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: Yeah, it's conceivable that we could implement something whereby characters with codes above some cutoff point are handled via runtime calls to iswalpha() and friends, rather than being included in the

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: Yeah ... if you *don't* know the difference between a DFA and an NFA, you're likely to find yourself in over your head. Having said that, So, here's a paper I found very nice to get started into this subject:

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Marko Kreen
On Sun, Feb 19, 2012 at 1:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: Yeah ... if you *don't* know the difference between a DFA and an NFA, you're likely to find yourself in over your head.  Having said that,

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Tom Lane
I wrote: And here's a poorly-tested draft patch for that. I've done some more testing now, and am satisfied that this works as intended. However, some crude performance testing suggests that people might be annoyed with it. As an example, in 9.1 with pl_PL.utf8 locale, I see this:

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Christopher Browne
On Sat, Feb 18, 2012 at 5:34 PM, Don Baccus dhog...@pacifier.com wrote: On Feb 18, 2012, at 1:43 PM, Christopher Browne wrote: A hierarchy like the following is perfectly logical: - to 0999 :: Cash accounts [1] I asked earlier if anyone would expect 01 like '0%' to match … Apparently

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Christopher Browne
On Sat, Feb 18, 2012 at 7:24 PM, Marko Kreen mark...@gmail.com wrote: About our Spencer code - if we don't have resources (not called Tom) Is there anything that would be worth talking about directly with Henry? He's in one of my circles of colleagues; had dinner with a group that included him

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Tom Lane
Christopher Browne cbbro...@gmail.com writes: On Sat, Feb 18, 2012 at 7:24 PM, Marko Kreen mark...@gmail.com wrote: About our Spencer code - if we don't have resources (not called Tom) Is there anything that would be worth talking about directly with Henry? He's in one of my circles of

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Brendan Jurd
On 19 February 2012 06:52, Tom Lane t...@sss.pgh.pa.us wrote: Yeah ... if you *don't* know the difference between a DFA and an NFA, you're likely to find yourself in over your head.  Having said that, this is eminently learnable stuff and pretty self-contained, so somebody who had the time and

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Robert Haas
On Sat, Feb 18, 2012 at 7:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah, it's conceivable that we could implement something whereby characters with codes above some cutoff point are handled via runtime calls to iswalpha() and friends, rather than being included in the statically-constructed

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Vik Reykja
On Sun, Feb 19, 2012 at 04:33, Robert Haas robertmh...@gmail.com wrote: On Sat, Feb 18, 2012 at 7:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah, it's conceivable that we could implement something whereby characters with codes above some cutoff point are handled via runtime calls to

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Robert Haas
On Sat, Feb 18, 2012 at 10:38 PM, Vik Reykja vikrey...@gmail.com wrote: Does it make sense for regexps to have collations? As I understand it, collations determine the sort-ordering of strings. Regular expressions don't care about that. Why do you ask? -- Robert Haas EnterpriseDB:

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Robert Haas
On Sat, Feb 18, 2012 at 4:12 PM, Don Baccus dhog...@pacifier.com wrote: On Feb 18, 2012, at 12:57 PM, Rob Wultsch wrote: Where first_name is string the queries above have very different behaviour in MySQL. The first does a full table scan and coerces first_name to an integer (so '5adfs' - 5)

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-02-18 Thread Robert Haas
On Fri, Feb 17, 2012 at 7:12 PM, Josh Berkus j...@agliodbs.com wrote: On 2/17/12 12:04 PM, Robert Haas wrote: The argument isn't about whether the user made the right design choices; it's about whether he should be forced to insert an explicit type cast to get the query to do what it is

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Vik Reykja
On Sun, Feb 19, 2012 at 05:03, Robert Haas robertmh...@gmail.com wrote: On Sat, Feb 18, 2012 at 10:38 PM, Vik Reykja vikrey...@gmail.com wrote: Does it make sense for regexps to have collations? As I understand it, collations determine the sort-ordering of strings. Regular expressions

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: In theory you can imagine a regular expression engine where these decisions can be postponed until we see the string we're matching against. IOW, your DFA ends up with state transitions for characters specifically named, plus a state transition for

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-18 Thread Robert Haas
On Sat, Feb 18, 2012 at 3:00 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, Feb 18, 2012 at 7:35 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Feb 14, 2012 at 3:25 PM, Greg Smith g...@2ndquadrant.com wrote: On 02/14/2012 01:45 PM, Greg Smith wrote: scale=1000, db is 94% of RAM;

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Tom Lane
Vik Reykja vikrey...@gmail.com writes: On Sun, Feb 19, 2012 at 05:03, Robert Haas robertmh...@gmail.com wrote: On Sat, Feb 18, 2012 at 10:38 PM, Vik Reykja vikrey...@gmail.com wrote: Does it make sense for regexps to have collations? As I understand it, collations determine the sort-ordering

Re: [HACKERS] Future of our regular expression code

2012-02-18 Thread Tom Lane
Brendan Jurd dire...@gmail.com writes: On 19 February 2012 06:52, Tom Lane t...@sss.pgh.pa.us wrote: Yeah ... if you *don't* know the difference between a DFA and an NFA, you're likely to find yourself in over your head.  Having said that, this is eminently learnable stuff and pretty

Re: [HACKERS] Notes about fixing regexes and UTF-8 (yet again)

2012-02-18 Thread Robert Haas
On Sat, Feb 18, 2012 at 11:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: In theory you can imagine a regular expression engine where these decisions can be postponed until we see the string we're matching against.  IOW, your DFA ends up with state

[HACKERS] wal_buffers

2012-02-18 Thread Robert Haas
Just for kicks, I ran two 30-minute pgbench tests at scale factor 300 tonight on Nate Boley's machine, with -n -l -c 32 -j 32. The configurations were identical, except that on one of them, I set wal_buffers=64MB. It seemed to make quite a lot of difference: wal_buffers not set (thus, 16MB):

Re: [HACKERS] pg_restore ignores PGDATABASE

2012-02-18 Thread Robert Haas
On Sat, Feb 18, 2012 at 11:58 AM, Erik Rijkers e...@xs4all.nl wrote: pg_restore ignores environment variable PGDATABASE. What exactly do you mean by ignores? pg_restore prints results to standard output unless a database name is specified. AFAIK, there's no syntax to say I want a

Re: [HACKERS] pg_restore ignores PGDATABASE

2012-02-18 Thread Erik Rijkers
On Sun, February 19, 2012 06:27, Robert Haas wrote: On Sat, Feb 18, 2012 at 11:58 AM, Erik Rijkers e...@xs4all.nl wrote: pg_restore ignores environment variable PGDATABASE. What exactly do you mean by ignores? pg_restore prints results to standard output unless a database name is specified.