Re: [HACKERS] patch: Add JSON datatype to PostgreSQL (GSoC, WIP)

2010-10-24 Thread Terry Laurenzo
It doesn't do particularly well on my previous example of [1,2,3]. It comes out slightly shorter on [a,b,c] and better if the strings need any escaping. I don't think the float4 and float8 formats are very useful; how could you be sure that the output was going to look the same as the

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Stark
On Sat, Oct 23, 2010 at 4:03 PM, Josh Berkus j...@agliodbs.com wrote: I think that such a lock would also be useful for improving the FK deadlock issues we have. I don't see how. I think the problem you're referring to occurs when different plans update rows in different orders and the

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Greg Stark
On Sat, Oct 23, 2010 at 10:11 PM, Robert Haas robertmh...@gmail.com wrote: I dunno if floats have completely consistent representations on all the platforms we support, but with integers it should be quite easy to predict the exact point when you're going to run out of space and what the

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Martijn van Oosterhout
On Sat, Oct 23, 2010 at 03:59:13PM -0700, Greg Stark wrote: I think the blocker with MERGE has always been that the standard is *so* general that it could apply to conditions that *aren't* covered by a unique constraint or exclusion constriant. Personally, I'm fine saying that those cases will

Re: [HACKERS] PostgreSQL and HugePage

2010-10-24 Thread Hsien-Wen Chu
as my known, FreeBSD implements this feature called superpage, it's similar with Solaris, so is it enabled in default? or any default parameter need to be set? Many thank Hsien-Wen On Wed, Oct 20, 2010 at 10:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes:

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Dean Rasheed
On 24 October 2010 05:26, Tom Lane t...@sss.pgh.pa.us wrote: Well, the easy way to read a consistent view of the world is to load the cache using an MVCC snapshot instead of SnapshotNow.  The current code structure isn't amenable to that because it's relying on a syscache to fetch the data for

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 5:50 AM, Martijn van Oosterhout klep...@svana.org wrote: On Sat, Oct 23, 2010 at 03:59:13PM -0700, Greg Stark wrote: I think the blocker with MERGE has always been that the standard is *so* general that it could apply to conditions that *aren't* covered by a unique

Re: [HACKERS] patch: Add JSON datatype to PostgreSQL (GSoC, WIP)

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 2:21 AM, Terry Laurenzo t...@laurenzo.org wrote: It doesn't do particularly well on my previous example of [1,2,3].  It comes out slightly shorter on [a,b,c] and better if the strings need any escaping.  I don't think the float4 and float8 formats are very useful; how

Re: [HACKERS] knngist - 0.8

2010-10-24 Thread Robert Haas
2010/10/22 Teodor Sigaev teo...@sigaev.ru: You can define the additional argument as providing all of the extra info about how the operator is being used, and, if it's being used for ordering, the details of the requested order.  What is your thinking on the matter? Maby be useful, but it

[HACKERS] planner row-estimates for tsvector seems horribly wrong

2010-10-24 Thread Sushant Sinha
I am using gin index on a tsvector and doing basic search. I see the row-estimate of the planner to be horribly wrong. It is returning row-estimate as 4843 for all queries whether it matches zero rows, a medium number of rows (88,000) or a large number of rows (726,000). The table has roughly a

Re: [HACKERS] planner row-estimates for tsvector seems horribly wrong

2010-10-24 Thread Jan Urbański
On 24/10/10 14:44, Sushant Sinha wrote: I am using gin index on a tsvector and doing basic search. I see the row-estimate of the planner to be horribly wrong. It is returning row-estimate as 4843 for all queries whether it matches zero rows, a medium number of rows (88,000) or a large number

Re: [HACKERS] planner row-estimates for tsvector seems horribly wrong

2010-10-24 Thread Jan Urbański
On 24/10/10 14:44, Sushant Sinha wrote: I am using gin index on a tsvector and doing basic search. I see the row-estimate of the planner to be horribly wrong. It is returning row-estimate as 4843 for all queries whether it matches zero rows, a medium number of rows (88,000) or a large number

Re: [HACKERS] planner row-estimates for tsvector seems horribly wrong

2010-10-24 Thread Sushant Sinha
Thanks a ton Jan! It works quite correctly. But many tsearch tutorials ask tsquery to be placed in 'from' statement and that can cause bad plan. Isn't it possible to return the correct number for a join with the query as well? -Sushant. On Sun, 2010-10-24 at 15:07 +0200, Jan Urbański wrote: On

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Smith
Robert Haas wrote: I am also wondering exactly what the semantics are supposed to be under concurrency. We can't assume that it's OK to treat WHEN NOT MATCHED as WHEN MATCHED if a unique-key violation is encountered. The join condition could be something else entirely. I guess we could scan

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: There's nothing magic about the integral types here. If you use a string then you could always split by making the string longer. The entire objective here is to make enum comparisons fast. Somehow, going to a non-primitive data type to represent the sort

Re: [HACKERS] PostgreSQL and HugePage

2010-10-24 Thread Tom Lane
Hsien-Wen Chu chu.hsien@gmail.com writes: as my known, FreeBSD implements this feature called superpage, it's similar with Solaris, so is it enabled in default? or any default parameter need to be set? The Solaris-specific code is just that if SHM_SHARE_MMU is defined (by sys/ipc.h, I

Re: [HACKERS] patch: Add JSON datatype to PostgreSQL (GSoC, WIP)

2010-10-24 Thread Terry Laurenzo
Yeah, my concern is not whether the overhead will be zero; it's whether it will be small, yet allow large gains on other operations. Like, how much slower will it be to pull out a moderately complex 1MB JSON blob (not just a big string) out of a single-row, single-column table? If it's 5%

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 12:20 PM, Tom Lane wrote: With float4 the implementation would fail at somewhere around 2^24 elements in an enum (since even with renumbering, there wouldn't be enough bits to give each element a distinguishable value). I don't see that as a real objection, and anyway if you

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Dean Rasheed
On 24 October 2010 17:20, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark gsst...@mit.edu writes: There's nothing magic about the integral types here. If you use a string then you could always split by making the string longer. The entire objective here is to make enum comparisons fast.  

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 12:15 PM, Greg Smith g...@2ndquadrant.com wrote: Robert Haas wrote: I am also wondering exactly what the semantics are supposed to be under concurrency.  We can't assume that it's OK to treat WHEN NOT MATCHED as WHEN MATCHED if a unique-key violation is encountered.  

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-24 Thread Tallat Mahmood
I'm writing a function that will read data from the buffer in xlog (i.e. from XLogCtl-pages and XLogCtl-xlblocks). I want to make sure that I am doing it correctly. Got an example of what the function might look like? Say something like this: bool ReadLogFromBuffer(char *buf, int len,

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Tom Lane
Dean Rasheed dean.a.rash...@gmail.com writes: The point with an OID array is that you wouldn't need to store the enumsortorder values at all. The sort order would just be the index of the OID in the array. So the comparison code would read the OID array, traverse it building an array of

[HACKERS] typenameTypeId refactoring

2010-10-24 Thread Peter Eisentraut
Here is a first patch to slightly refactor the type and typmod lookups with the aim of making it easier to possibly changing the nature or representation of typmods in the future and making it easier to merge the collation patch. I split the typenameTypeId function into two. Most call sites only

[HACKERS] Segfault in 9.0 inlining SRF

2010-10-24 Thread Brendan Jurd
Hi folks, I have encountered a reproducible segfault in Postgres, and confirmed it in 9.0.1 and HEAD on three separate machines. The bug was not present in 8.4. I've attached a copy of the SQL script I have been using to induce the segfault. With asserts enabled, I get a failed assertion:

Re: [HACKERS] [BUGS] Segfault in 9.0 inlining SRF

2010-10-24 Thread Tom Lane
Brendan Jurd dire...@gmail.com writes: I have encountered a reproducible segfault in Postgres, and confirmed it in 9.0.1 and HEAD on three separate machines. The bug was not present in 8.4. I've attached a copy of the SQL script I have been using to induce the segfault. ... I had a go at

Re: [HACKERS] typenameTypeId refactoring

2010-10-24 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Here is a first patch to slightly refactor the type and typmod lookups with the aim of making it easier to possibly changing the nature or representation of typmods in the future and making it easier to merge the collation patch. The comment for

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 03:33 PM, Tom Lane wrote: Dean Rasheeddean.a.rash...@gmail.com writes: The point with an OID array is that you wouldn't need to store the enumsortorder values at all. The sort order would just be the index of the OID in the array. So the comparison code would read the OID

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 03:28 PM, Tom Lane wrote: This patch isn't committable as-is because I haven't made enum_first, enum_last, enum_range follow these coding rules: they need to stop using the syscache and instead use an indexscan on pg_enum_typid_sortorder_index to locate the relevant rows. That

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 05:09 PM, Andrew Dunstan wrote: select enum_oid, row_number() over () as sort_order from unnest(null::myenum) as enum_oid Of course, I meant: select enum_label, row_number() over () as sort_order from unnest(enum_range(null::myenum)) as enum_label cheers

Re: [HACKERS] [BUGS] Segfault in 9.0 inlining SRF

2010-10-24 Thread Brendan Jurd
On 25 October 2010 07:36, Tom Lane t...@sss.pgh.pa.us wrote: Looks like the invalItems list has been clobbered: (gdb) p *root-glob-invalItems $6 = {type = 2139062143, length = 2139062143, head = 0x7f7f7f7f,  tail = 0x7f7f7f7f} I'm guessing it was modified in the temporary memory context and

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Tom Lane
BTW, I've forgotten --- did anyone publish a test case for checking performance of enum comparisons? Or should I just cons up something privately? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] why does plperl cache functions using just a bool for is_trigger

2010-10-24 Thread Jan Urbański
I see that plperl uses a triple of (function oid, is_trigger flag, user id) as a hash key for caching compiled functions. OTOH pltcl and plpgsql both use (oid, trigger relation oid, user id). Is there any reason why just using a bool as plperl does would be wrong? I'm trying to write a validator

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Smith
Robert Haas wrote: Well, there's no guarantee that any index at all exists on the target table, so any solution based on index locks can't be fully general. Sure, but in the most common use case I think we're targeting at first, no indexes means there's also no unique constraints either.

[HACKERS] Range Types, discrete and/or continuous

2010-10-24 Thread Jeff Davis
I'd like to open the discussion for Range Types again. This is a fairly long email because several issues are intertwined, and I don't think they can be neatly pulled apart. Previous discussions: http://archives.postgresql.org/pgsql-hackers/2009-12/msg01162.php

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 05:34 PM, Tom Lane wrote: BTW, I've forgotten --- did anyone publish a test case for checking performance of enum comparisons? Or should I just cons up something privately? I have been running tests with http://developer.postgresql.org/~adunstan/enumtest.dmp The table

Re: [HACKERS] why does plperl cache functions using just a bool for is_trigger

2010-10-24 Thread Tom Lane
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= wulc...@wulczer.org writes: I see that plperl uses a triple of (function oid, is_trigger flag, user id) as a hash key for caching compiled functions. OTOH pltcl and plpgsql both use (oid, trigger relation oid, user id). Is there any reason why just using a bool

Re: [HACKERS] Range Types, discrete and/or continuous

2010-10-24 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: Last development cycle, one of the questions that was unresolved was whether to handle ranges like a discrete set (that is, [1,5) = [1,4] ) or continuous or both. Put me in the camp that says you need both. I really seriously dislike the idea of

Re: [HACKERS] why does plperl cache functions using just a bool for is_trigger

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 06:44 PM, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?=wulc...@wulczer.org writes: I see that plperl uses a triple of (function oid, is_trigger flag, user id) as a hash key for caching compiled functions. OTOH pltcl and plpgsql both use (oid, trigger relation oid, user id).

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Stark
On Sun, Oct 24, 2010 at 2:50 AM, Martijn van Oosterhout klep...@svana.org wrote: Can we please not get MERGE hung up on trying to make it atomic. The standard requires no such thing and there are plenty of uses of MERGE that don't involve high concurrency updates of the same row by different

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Stark
On Sun, Oct 24, 2010 at 2:39 PM, Greg Smith g...@2ndquadrant.com wrote: Sure, but in the most common use case I think we're targeting at first, no indexes means there's also no unique constraints either.  I don't think people can reasonable expect to MERGE or anything else to guarantee they

Re: [HACKERS] Serializable snapshot isolation patch

2010-10-24 Thread Kevin Grittner
Jeff Davis pg...@j-davis.com wrote: On Mon, 2010-10-18 at 13:26 -0500, Kevin Grittner wrote: 3. Limited shared memory space to hold information about committed transactions that are still interesting. It's a challenging problem, however, and the current solution is less than ideal.

Re: [HACKERS] why does plperl cache functions using just a bool for is_trigger

2010-10-24 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 10/24/2010 06:44 PM, Tom Lane wrote: I'm not certain that plperl is actually correct to do it that way, but that's the basic idea. Why do we need the is_trigger flag at all for the plperl hash key? At first glance it strikes me as unnecessary.

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 10/24/2010 05:34 PM, Tom Lane wrote: BTW, I've forgotten --- did anyone publish a test case for checking performance of enum comparisons? Or should I just cons up something privately? I have been running tests with

Re: [HACKERS] why does plperl cache functions using just a bool for is_trigger

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 07:50 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: On 10/24/2010 06:44 PM, Tom Lane wrote: I'm not certain that plperl is actually correct to do it that way, but that's the basic idea. Why do we need the is_trigger flag at all for the plperl hash key? At

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 08:12 PM, Tom Lane wrote: OK, I did some timing consisting of building a btree index with maintenance_work_mem set reasonably high. This is on a debug-enabled build, so it's not representative of production performance, but it will do for seeing what we're doing to enum

Re: [HACKERS] Extensions, this time with a patch

2010-10-24 Thread Itagaki Takahiro
On Sat, Oct 23, 2010 at 5:26 AM, Josh Berkus j...@agliodbs.com wrote: Yeah - what is the feasibility of cleaning up the things where there are naming inconsistencies right now? Easy.  Heck, the only reason we didn't do it 2 years ago was that we were waiting for extensions before bothering.

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 10/24/2010 08:12 PM, Tom Lane wrote: This shows that the bitmapset optimization really is quite effective, at least for cases where all the enum labels are sorted by OID after all. That motivated me to change the bitmapset setup code to what's

Re: [HACKERS] Extensions, this time with a patch

2010-10-24 Thread Tom Lane
Itagaki Takahiro itagaki.takah...@gmail.com writes: On Sat, Oct 23, 2010 at 5:26 AM, Josh Berkus j...@agliodbs.com wrote: Yeah - what is the feasibility of cleaning up the things where there are naming inconsistencies right now? Easy.  Heck, the only reason we didn't do it 2 years ago was

Re: [HACKERS] why does plperl cache functions using just a bool for is_trigger

2010-10-24 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 10/24/2010 07:50 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: Why do we need the is_trigger flag at all for the plperl hash key? At first glance it strikes me as unnecessary. We might not. Does the presence or absence of the

Re: [HACKERS] WIP: extensible enums

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 09:20 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: On 10/24/2010 08:12 PM, Tom Lane wrote: This shows that the bitmapset optimization really is quite effective, at least for cases where all the enum labels are sorted by OID after all. That motivated me to

Re: [HACKERS] why does plperl cache functions using just a bool for is_trigger

2010-10-24 Thread Andrew Dunstan
On 10/24/2010 09:34 PM, Tom Lane wrote: For both trigger and non-trigger functions, we compile this ahead of the user-set function code: our $_TD; local $_TD=shift; Non-trigger functions get passed undef to correspond to this invisible argument, while trigger functions get passed the

Re: [HACKERS] Range Types, discrete and/or continuous

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 6:59 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Davis pg...@j-davis.com writes: Last development cycle, one of the questions that was unresolved was whether to handle ranges like a discrete set (that is, [1,5) = [1,4] ) or continuous or both. Put me in the camp that

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 7:11 PM, Greg Stark gsst...@mit.edu wrote: On Sun, Oct 24, 2010 at 2:50 AM, Martijn van Oosterhout klep...@svana.org wrote: Can we please not get MERGE hung up on trying to make it atomic. The standard requires no such thing and there are plenty of uses of MERGE that

Re: [HACKERS] Simplifying replication

2010-10-24 Thread Fujii Masao
On Sat, Oct 23, 2010 at 2:34 AM, Josh Berkus j...@agliodbs.com wrote: When we get close enough to max_wal_size (we'll need a couple segments of leeway, I think), we start recycling WAL segments even if they are less that min_wal_time old. What happens if max_wal_size is less than