[HACKERS] chinese parser for text search !

2008-11-02 Thread Oleg Bartunov
Hi there, there is a chinese parser for tsearch2 available from http://code.google.com/p/nlpbamboo/wiki/TSearch2 under BSD license. It'd be nice to have it for chinese text search configuration we are currently completely missing, as well as for japanese language ( any guess ?)

Re: [HACKERS] Well done, Hackers

2008-11-02 Thread Joshua D. Drake
Josh Berkus wrote: Tom, Robert, Simon, What, are people just on edge because of the US election? It looks to me like the commitfest system is going really well. Of course, we'll see how long it takes to close out 8.4. But I think we're in much better shape than we were for 8.3. We're even

Re: [HACKERS] Well done, Hackers

2008-11-02 Thread Dave Page
Sent from my iPhone On 2 Nov 2008, at 06:57, Joshua D. Drake [EMAIL PROTECTED] wrote: Josh Berkus wrote: Tom, Robert, Simon, What, are people just on edge because of the US election? It looks to me like the commitfest system is going really well. Of course, we'll see how long it takes to

Re: [HACKERS] WIP: Column-level Privileges

2008-11-02 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: Stephen Frost [EMAIL PROTECTED] writes: ... A case I just realized might be an issue is doing a 'select 1 from x;' where you have *no* rights on x, or any columns in it, would still get you the rowcount. Well, if you have table-level select on x, I

Re: [HACKERS] Well done, Hackers

2008-11-02 Thread Martijn van Oosterhout
On Sat, Nov 01, 2008 at 08:07:13PM -0400, Andrew Sullivan wrote: On Sat, Nov 01, 2008 at 04:21:30PM -0400, Robert Haas wrote: It looks to me like there are at least half a dozen patches submitted in the last week that are pretty half-baked and fall into the category of Let's submit

Re: [HACKERS] WIP: Column-level Privileges

2008-11-02 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: Stephen Frost [EMAIL PROTECTED] writes: ... A case I just realized might be an issue is doing a 'select 1 from x;' where you have *no* rights on x, or any columns in it, would still get you the rowcount. Well, if you have table-level select on x, I

Re: [HACKERS] Windowing Function Patch Review - Performance Comparison.

2008-11-02 Thread David Rowley
Hitoshi Harada Wrote: Thanks for your test. Didn't post publicly, I've also tested real problems and performed better than I thought. If you can afford it, could you add selfjoin cases? It's like: Ok, did self joins with some. I don't know if it's possible with all. Test Sub query Self join

Re: [HACKERS] PG_PAGE_LAYOUT_VERSION 5 - time for change

2008-11-02 Thread Zdenek Kotala
Tom Lane napsal(a): Heikki Linnakangas [EMAIL PROTECTED] writes: Hmm, you're right. I think it can be made to work by storing the *end* offset of each chunk. To find the chunk containing offset X, search for the first chunk with end_offset X. Yeah, that seems like it would work, and it

[HACKERS] Re: [COMMITTERS] pgsql: Unite ReadBufferWithFork, ReadBufferWithStrategy, and

2008-11-02 Thread Heikki Linnakangas
Simon Riggs wrote: On Sat, 2008-11-01 at 15:18 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Fri, 2008-10-31 at 15:05 +, Heikki Linnakangas wrote: Log Message: --- Unite ReadBufferWithFork, ReadBufferWithStrategy, and ZeroOrReadBuffer functions into one ReadBufferExtended

Re: [HACKERS] don't use MAKE_PTR/OFFSET for shmem pointers

2008-11-02 Thread Tom Lane
Kris Jurka [EMAIL PROTECTED] writes: Since we require every process to map the shared memory region to the same address, we don't need the MAKE_PTR/OFFSET code that was needed when that was not the case. This patch makes shared memory pointers just like regular pointers. Applied with minor

[HACKERS] WIP: Hash Join-Filter Pruning using Bloom Filters

2008-11-02 Thread Jonah H. Harris
All, Attached is an initial patch I've been playing with which uses Bloom filters to reduce unnecessary processing of outer tuples in hash joins. In short, this works by creating a Bloom filter, adding all relevant tuples for the inner relation, and querying the filter (for existence) when

Re: [HACKERS] WIP: Hash Join-Filter Pruning using Bloom Filters

2008-11-02 Thread Hannes Eder
On Sun, Nov 2, 2008 at 10:49 PM, Jonah H. Harris [EMAIL PROTECTED] wrote: Similarly, I created a GUC to enable pruning, named bloom_pruning. I guess calls to bloom_filter_XXX should be surrounded by if (bloom_pruning) ... or a similar construct, i.e. make use of the GUC variable bloom_pruning

Re: [HACKERS] WIP: Hash Join-Filter Pruning using Bloom Filters

2008-11-02 Thread Jonah H. Harris
On Sun, Nov 2, 2008 at 5:36 PM, Hannes Eder [EMAIL PROTECTED] wrote: On Sun, Nov 2, 2008 at 10:49 PM, Jonah H. Harris [EMAIL PROTECTED] wrote: Similarly, I created a GUC to enable pruning, named bloom_pruning. I guess calls to bloom_filter_XXX should be surrounded by if (bloom_pruning) ...

[HACKERS] Hot standby v5 patch assertion failure

2008-11-02 Thread Mark Kirkwood
Trying out a few different scenarios I ran across this: 1/ Setup master and replica with replica using pg_standby 2/ Create a new database (bench in my case) 3/ Initialize pgbench schema size 100 4/ Run with 2 clients and 1 transactions 5/ Replica gets assertion failure This is Postgres

Re: [HACKERS] WIP: Hash Join-Filter Pruning using Bloom Filters

2008-11-02 Thread Greg Stark
I think the k hash functions are actually normally just different slices of bits taken from one actual hash function anyways so it sounds like you've done the right thing. This sounds most interesting for multibatch hash joins if you could build a bloom filter for the future batches to

Re: [HACKERS] Proposed Patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets

2008-11-02 Thread Lawrence, Ramon
Joshua, Thank you for offering to review the patch. The easiest way to test would be to generate your own TPC-H data and load it into a database for testing. I have posted the TPC-H generator at: http://people.ok.ubc.ca/rlawrenc/TPCHSkew.zip The generator can produce skewed data sets. It was

Re: [HACKERS] Proposed Patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets

2008-11-02 Thread Joshua Tolley
On Sun, Nov 2, 2008 at 4:48 PM, Lawrence, Ramon [EMAIL PROTECTED] wrote: Joshua, Thank you for offering to review the patch. The easiest way to test would be to generate your own TPC-H data and load it into a database for testing. I have posted the TPC-H generator at:

Re: [HACKERS] Simple postgresql.conf wizard

2008-11-02 Thread Josh Berkus
Greg Smith wrote: While I've got a pretty clear vision for what I'm doing with this next and will kick off a pgfoundry project real soon, I wanted to throw this out as a WIP for feedback at this point. I was ultimately hoping to one day have something like this shipped as a contrib/ module to

Re: [HACKERS] reducing statistics write overhead

2008-11-02 Thread Tom Lane
Martin Pihlak [EMAIL PROTECTED] writes: Attached is a patch that implements the described signalling. Additionally following non-related changes have been made: 1. fopen/fclose replaced with AllocateFile/FreeFile 2. pgstat_report_stat() now also checks if there are functions to report before

Re: [HACKERS] Proposed Patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets

2008-11-02 Thread Tom Lane
Lawrence, Ramon [EMAIL PROTECTED] writes: The easiest way to test would be to generate your own TPC-H data and load it into a database for testing. I have posted the TPC-H generator at: http://people.ok.ubc.ca/rlawrenc/TPCHSkew.zip The generator can produce skewed data sets. It was produced

Re: [HACKERS] Proposed Patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets

2008-11-02 Thread Lawrence, Ramon
From: Tom Lane [mailto:[EMAIL PROTECTED] What alternatives are there for people who do not run Windows? regards, tom lane The TPC-H generator is a standard code base provided at http://www.tpc.org/tpch/. We have been able to compile this code on Linux. However, we

Re: [HACKERS] array_agg and array_accum (patch)

2008-11-02 Thread Jeff Davis
Here's an updated patch for just array_accum() with some simple docs. If I should document this in more places, let me know. I decided not to include array_agg() in this patch because it doesn't support the standard's ORDER BY clause. My reasoning is that, if someone is using the standard

[HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Tom Lane
http://wiki.postgresql.org/wiki/CommitFest_2008-11 currently says that it's closed to new submissions, but this isn't particularly helpful when the link it gives for where to submit them points right back to itself. So, where should we make CommitFestOpen redirect to? The problem I'm having with

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Josh Berkus
Tom, Thoughts, better ideas? Nope, that's why I ignored CommitFestOpen. I couldn't figure out where to point it. Note that I'm going to propose replacing the wiki with real software come 8.5. The wiki method confuses new reviewers, and makes me spend about 500% as much time managing

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Brendan Jurd
On Mon, Nov 3, 2008 at 2:31 PM, Tom Lane [EMAIL PROTECTED] wrote: The problem I'm having with this is that the current scheme for naming commitfest pages assumes you know the date for each one. It seems way premature to bet on when the first 8.5 fest will start. Wikimedia does seem to

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Note that I'm going to propose replacing the wiki with real software come 8.5. Where is this real software going to come from? I can't imagine that software with more functionality than the wiki is going to spring into existence without a huge investment

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Tom Lane
Brendan Jurd [EMAIL PROTECTED] writes: Note that if you rename a page you also need to update any redirects which point to it. The directions on the move tab imply that this is taken care of automatically? But anyway it'll just be a minor amount of extra work when the time comes.

Re: [HACKERS] [WIP] In-place upgrade

2008-11-02 Thread Robert Haas
I tried to apply this patch to CVS HEAD and it blew up all over the place. It doesn't seem to be intended to apply against CVS HEAD; for example, I don't have backend/access/heap/htup.c at all, so can't apply changes to that file. I was able to clone the GIT repository with the following

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Greg Smith
On Sun, 2 Nov 2008, Tom Lane wrote: Brendan Jurd [EMAIL PROTECTED] writes: Note that if you rename a page you also need to update any redirects which point to it. The directions on the move tab imply that this is taken care of automatically? What you get when you move something is redirect

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Tom Lane
Greg Smith [EMAIL PROTECTED] writes: On Sun, 2 Nov 2008, Tom Lane wrote: The directions on the move tab imply that this is taken care of automatically? What you get when you move something is redirect page that translates the old name into the new one. But if you were already targeting

[HACKERS] How to parse Datum

2008-11-02 Thread Zhe He
I met a problem with parsing Datum, is there any good way to parse this kind of value? Because I want to get the value in an attribute of a table. In TupleTableSlot, there is a tts_values which is Datum type. Does anyone know how to parse it? Best Regards, Zhe HE TEL: (001) 646-789-3008

Re: [HACKERS] Simple postgresql.conf wizard

2008-11-02 Thread Greg Smith
On Sun, 2 Nov 2008, Josh Berkus wrote: I'd start with command-line switches, e.g. config --memory=32GB --type=DW --size=500GB --connections=20 That seems reasonable, I think I'll push a fancier UI on the backburner then and just spec out an options interface like this one. I think in

Re: [HACKERS] Hot standby v5 patch assertion failure

2008-11-02 Thread Simon Riggs
On Mon, 2008-11-03 at 12:16 +1300, Mark Kirkwood wrote: Trying out a few different scenarios I ran across this: CONTEXT: xlog redo update: rel 1663/16384/16397; tid 9614/62; new 158828/59 DEBUG: start recovery xid = 7002 lsn = 0/6F012EE4 CONTEXT: xlog redo update: rel 1663/16384/16397;

Re: [HACKERS] new correlation metric

2008-11-02 Thread Brendan Jurd
Hi Jeff, I've been assigned to do an initial review of your new correlation metric patch. If I'm grokking the thread, it looks like Tom suggested a substantial change in the approach (targetting per-index correlation rather than per-column) [1], and although you agreed with the spirit of his