Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Tom Lane
Markus Schiltknecht [EMAIL PROTECTED] writes: Uh.. can you elaborate on that? AFAICS, you would simply have to query multiple btree indexes and make sure non of them is violated. That only works for the partition-key indexes, ie, ones where you can be sure a-priori that there cannot be

Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-06 Thread ITAGAKI Takahiro
Heikki Linnakangas [EMAIL PROTECTED] wrote: Itagaki, would you like to take a stab at this? Yes, I'll try to fix the mdsync problem. I'll separate this fix from LDC patch. If we need to backport the fix to the back branches, a stand-alone patch would be better. In my understanding from the

Re: [HACKERS] [PATCHES] Optimized pgbench for 8.3

2007-04-06 Thread Tom Lane
Tatsuo Ishii [EMAIL PROTECTED] writes: The patches look good to me. Please commit whatever you think is reasonable. BTW, is anybody working on enabling the fill factor to the tables used by pgbench? 8.3 will introduce HOT, and I think adding the feature will make it easier to test HOT. I'm

Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-06 Thread Tom Lane
ITAGAKI Takahiro [EMAIL PROTECTED] writes: In my understanding from the discussion, we'd better to take cycle ID approach instead of making a copy of pendingOpsTable, because duplicated table is hard to debug and requires us to pay attention not to leak memories. I'll adopt the cycle ID

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Devrim Gündüz
Hi, On Fri, 2007-04-06 at 01:23 -0400, Matthew T. O'Connor wrote: The other thing to consider is that CentOS 5 has Xen built right in, so you should be able run VMs without VMWare on it. ... if the kernel of the OS has Xen support, there will be no performance penalty (only 2%-3%)

[HACKERS] Postgres optimizer

2007-04-06 Thread Suresh
Hi, I want to get the coding details regarding postgres optimizer. Essentially, which files correspond to which functions, how the algo is implemented, the flow etc.. Where can I find this material ? Also, how to I start changing Postgres Code using eclipse platform (with tracing, debug

Re: [HACKERS] Bug in UTF8-Validation Code?

2007-04-06 Thread Albe Laurenz
Martijn van Oosterhout wrote: So your implemntation is simply: 1. Take number and make UTF-8 string 2. Convert it to database encoding. Aah, now I can spot where the misunderstanding is. That's not what I mean. I mean that chr() should simply 'typecast' to char. So when the database encoding

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Marko Kreen
On 4/6/07, Larry Rosenman ler@lerctr.org wrote: I am willing to run any X86 or X64 OS's in VM's as buildfarm clients. What OS's do we need coverage for? Cannot say about OS, but could you run it with Python 2.5? 64bit interface changed there and it would be interesting to see if it still

Re: [HACKERS] Postgres optimizer

2007-04-06 Thread Mark Kirkwood
Suresh wrote: Hi, I want to get the coding details regarding postgres optimizer. Essentially, which files correspond to which functions, how the algo is implemented, the flow etc.. Where can I find this material ? Maybe start with the docs, there a good section on optimization:

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Simon Riggs
On Fri, 2007-04-06 at 01:56 -0400, Tom Lane wrote: Markus Schiltknecht [EMAIL PROTECTED] writes: Uh.. can you elaborate on that? AFAICS, you would simply have to query multiple btree indexes and make sure non of them is violated. That only works for the partition-key indexes, ie, ones

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Gregory Stark
Simon Riggs [EMAIL PROTECTED] writes: If we partition on invoice_date only, there is an implication that people will search for invoices on date range only too, otherwise why not just partition on invoice_id. This still works with the compound key approach. Well there are practical problems

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Matthew O'Connor
Devrim Gündüz wrote: Hi, On Fri, 2007-04-06 at 01:23 -0400, Matthew T. O'Connor wrote: The other thing to consider is that CentOS 5 has Xen built right in, so you should be able run VMs without VMWare on it. ... if the kernel of the OS has Xen support, there will be no performance penalty

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Sander Steffann
Hi, ... if the kernel of the OS has Xen support, there will be no performance penalty (only 2%-3%) (Para-virtualization). Otherwise, there will be full-virtualization, and we should expect a performance loss about 30% for each guest OS (like Windows). I may be wrong but I thought that the

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Zeugswetter Andreas ADI SD
(But that sounds rather like pie in the sky, actually. Which other databases can do that, and how do they do it?) Oracle does it, by building a big index. Few people use it. And others allow a different partitioning strategy for each index, but that has the same problem of how to remove

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Paul Lindner
On Thu, Apr 05, 2007 at 06:28:39PM -0500, Larry Rosenman wrote: I might use that as the base then, since the hardware finishes getting here tomorrow. My question still stands on what OS's we need coverage for. I can provide coverage of SuSE Enterprise 9/10 on i386/x86_64. I just filled out

Re: [HACKERS] [PATCHES] pgbench transaction timestamps

2007-04-06 Thread Tatsuo Ishii
On Fri, 6 Apr 2007, Tatsuo Ishii wrote: 1) latency log file format extention looks usefull (-x option). 2) it seems the cleanup feature (-X option) was withdrawed by the author, but the patches still include the feature. So I'm confused. The patch I sent to the mailing list pulled

Re: [HACKERS] [PATCHES] Optimized pgbench for 8.3

2007-04-06 Thread Tatsuo Ishii
Patch committed. Thanks. -- Tatsuo Ishii SRA OSS, Inc. Japan The attached is a patch to optimize contrib/pgbench using new 8.3 features. - Use DROP IF EXISTS to suppress errors for initial loadings. - Use a combination of TRUNCATE and COPY to reduce WAL on creating the accounts table.

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Simon Riggs
On Fri, 2007-04-06 at 12:47 +0200, Zeugswetter Andreas ADI SD wrote: What I think we would like to have is putting the append nodes into an order that allows removing the sort node whenever that can be done. And maybe a merge node (that replaces the append and sort node) that can merge

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Larry Rosenman
On Fri, 6 Apr 2007, Matthew O'Connor wrote: Devrim G??nd??z wrote: Hi, On Fri, 2007-04-06 at 01:23 -0400, Matthew T. O'Connor wrote: The other thing to consider is that CentOS 5 has Xen built right in, so you should be able run VMs without VMWare on it. ... if the kernel of the OS has Xen

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Markus Schiltknecht
Simon Riggs wrote: i.e. if we have partitions for each year (2001, 2002, 2003 2004, 2005, 2006, 2007) AND we have already proved that 2005 is excluded when we have a WHERE clause saying year = 2006, then we should be able to use the ordering to prove that partitions for 2004 and before are also

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Andrew Dunstan
Larry Rosenman wrote: It doesn't matter as far as MY box is concerned. I use VMWare extensively in my current $DAYJOB, and I want to be able to test/play with things related to that as well. The box I'm building will be using the (free) VMWare Server as it's virtualization platform. I'd

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Stefan Kaltenbrunner
Andrew Dunstan wrote: Larry Rosenman wrote: It doesn't matter as far as MY box is concerned. I use VMWare extensively in my current $DAYJOB, and I want to be able to test/play with things related to that as well. The box I'm building will be using the (free) VMWare Server as it's

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Joshua D. Drake
Zeugswetter Andreas ADI SD wrote: (But that sounds rather like pie in the sky, actually. Which other databases can do that, and how do they do it?) Oracle does it, by building a big index. Few people use it. And others allow a different partitioning strategy for each index, but that has the

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Larry Rosenman
On Fri, 6 Apr 2007, Andrew Dunstan wrote: Larry Rosenman wrote: It doesn't matter as far as MY box is concerned. I use VMWare extensively in my current $DAYJOB, and I want to be able to test/play with things related to that as well. The box I'm building will be using the (free) VMWare

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Larry Rosenman
On Fri, 6 Apr 2007, Stefan Kaltenbrunner wrote: Andrew Dunstan wrote: Larry Rosenman wrote: It doesn't matter as far as MY box is concerned. I use VMWare extensively in my current $DAYJOB, and I want to be able to test/play with things related to that as well. The box I'm building will be

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Joshua D. Drake
VMWare Server is indeed a fine product, which I use extensively. I am not sure what our Windows support is like for x86_64. Magnus has one for MSVC (for which buildfarm support is nearly done, but not quite). But I don't see one for MinGW. OTOH, Windows is not free (in either sense) and

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Dave Page
--- Original Message --- From: Stefan Kaltenbrunner [EMAIL PROTECTED] To: Andrew Dunstan [EMAIL PROTECTED] Sent: 06/04/07, 15:33:20 Subject: Re: [HACKERS] What X86/X64 OS's do we need coverage for? yeah improving windows coverage might be a nice thing - some other I'm awaiting

Re: [HACKERS] elog(FATAL) vs shared memory

2007-04-06 Thread Mark Shuttleworth
Tom Lane wrote: (1) something (still not sure what --- Martin and Mark, I'd really like to know) was issuing random SIGTERMs to various postgres processes including autovacuum. This may be a misfeature in our test harness - I'll ask Stuart Bishop to comment. Mark

Re: [HACKERS] --enable-xml instead of --with-libxml?

2007-04-06 Thread Nikolay Samokhvalov
On 4/5/07, Bruce Momjian [EMAIL PROTECTED] wrote: Nikolay Samokhvalov wrote: [...] If I am wrong and it's better to leave libxml2-free capabilities, then IMHO we need to reflect it explicitly in the docs, what requires libxml2, and what doesn't Agreed, let's do the later and update the

Re: [HACKERS] elog(FATAL) vs shared memory

2007-04-06 Thread Stuart Bishop
Mark Shuttleworth wrote: Tom Lane wrote: (1) something (still not sure what --- Martin and Mark, I'd really like to know) was issuing random SIGTERMs to various postgres processes including autovacuum. This may be a misfeature in our test harness - I'll ask Stuart Bishop to comment.

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Joshua D. Drake
But if we could find a way to represent that it would make a lot of common use cases much more convenient to use. (But that sounds rather like pie in the sky, actually. Which other databases can do that, and how do they do it?) Oracle does it, by building a big index. Few people use it.

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Tom Lane
Larry Rosenman ler@lerctr.org writes: I'd still like to hear from a Tom Lane or someone else on the project with what X86 or X86_64 OS's we need coverage for. FWIW, I think we are more in need of coverage of different configure-option sets than of OS's per se.

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread David Fetter
On Fri, Apr 06, 2007 at 09:22:55AM -0700, Joshua D. Drake wrote: The people that use it are the people stuck by dogmatic rules about every table must have a primary key or every logical constraint must be protected by a database constraint. Ie, database shops run by the CYA principle. Or

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Joshua D. Drake
David Fetter wrote: On Fri, Apr 06, 2007 at 09:22:55AM -0700, Joshua D. Drake wrote: The people that use it are the people stuck by dogmatic rules about every table must have a primary key or every logical constraint must be protected by a database constraint. Ie, database shops run by the CYA

Re: [HACKERS] elog(FATAL) vs shared memory

2007-04-06 Thread Tom Lane
Stuart Bishop [EMAIL PROTECTED] writes: After a test is run, the test harness kills any outstanding connections so we can drop the test database. Without this, a failing test could leave open connections dangling causing the drop database to block. Just to make it perfectly clear: we don't

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Andrew Dunstan
Tom Lane wrote: FWIW, I think we are more in need of coverage of different configure-option sets than of OS's per se. If someone would like to put together a list of gaps we can see what we can do about it. For anyone who wants the data on what is being built currently, the dashboard

Re: [HACKERS] [PATCHES] Fix for large file support

2007-04-06 Thread Tom Lane
[ redirecting to -hackers for wider comment ] Zdenek Kotala [EMAIL PROTECTED] writes: Tom Lane wrote: LET_OS_MANAGE_FILESIZE is good way. I think one problem of this option I fixed. It is size of offset. I went thru the code and did not see any other problem there. However, how you

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Richard Troy
David Fetter wrote: On Fri, Apr 06, 2007 at 09:22:55AM -0700, Joshua D. Drake wrote: The people that use it are the people stuck by dogmatic rules about every table must have a primary key or every logical constraint must be protected by a database constraint. Ie, database shops run

[HACKERS] Fate of pgsnmpd

2007-04-06 Thread Florian G. Pflug
Hi Does anyone know if pgsnmpd is still actively developed? The last version (0.1b1) is about 15 months old. greetings, Florian Pflug ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Fate of pgsnmpd

2007-04-06 Thread Stefan Kaltenbrunner
Florian G. Pflug wrote: Hi Does anyone know if pgsnmpd is still actively developed? The last version (0.1b1) is about 15 months old. there seems to be quite a lot of work going on in the cvs tree: http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgsnmpd/pgsnmpd/ so i would guess it is still

Re: [HACKERS] Auto Partitioning

2007-04-06 Thread Simon Riggs
On Fri, 2007-04-06 at 16:08 +0200, Markus Schiltknecht wrote: Simon Riggs wrote: i.e. if we have partitions for each year (2001, 2002, 2003 2004, 2005, 2006, 2007) AND we have already proved that 2005 is excluded when we have a WHERE clause saying year = 2006, then we should be able to use

Re: [HACKERS] Fate of pgsnmpd

2007-04-06 Thread Magnus Hagander
Florian G. Pflug wrote: Hi Does anyone know if pgsnmpd is still actively developed? The last version (0.1b1) is about 15 months old. It is. There is a team (Josh Tolley, me and Hiroshi Saito) working for RFC 1697 compliance. When that's done, there are some other additions in the pipeline.

Re: [HACKERS] [PATCHES] Fix for large file support

2007-04-06 Thread Zdenek Kotala
Tom Lane wrote: [ redirecting to -hackers for wider comment ] Zdenek Kotala [EMAIL PROTECTED] writes: Tom Lane wrote: LET_OS_MANAGE_FILESIZE is good way. I think one problem of this option I fixed. It is size of offset. I went thru the code and did not see any other problem there. However,

[HACKERS] Eliminating unnecessary left joins

2007-04-06 Thread Ottó Havasvölgyi
Hi, When using views built with left joins, and then querying against these views, there are a lot of join in the plan that are not necessary, because I don't select/use any column of each table in the views every time. Tables that are left joined and never referenced anywhere else in the query

Re: [HACKERS] [PATCHES] Fix for large file support

2007-04-06 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes: It sounds good. There is one think for clarification (for the present). How to handle buffile? It does not currently support non segmented files. I suggest to use same switch to enable/disable segments there. Do you think it really matters?

Re: [HACKERS] Eliminating unnecessary left joins

2007-04-06 Thread Tom Lane
=?ISO-8859-1?Q?Ott=F3_Havasv=F6lgyi?= [EMAIL PROTECTED] writes: When using views built with left joins, and then querying against these views, there are a lot of join in the plan that are not necessary, because I don't select/use any column of each table in the views every time. Tables that

[HACKERS] Problems with the varlena patch in my module

2007-04-06 Thread Tzahi Fadida
Hi, I am having trouble with fixing my code for this recent varlena patch: http://archives.postgresql.org/pgsql-committers/2007-04/msg00081.php My module is fulldisjunctions. I have several problems but first i wish to address the following. This is an excerpt from my code: newtset-tids = (bytea

Re: [HACKERS] Problems with the varlena patch in my module

2007-04-06 Thread Tom Lane
Tzahi Fadida [EMAIL PROTECTED] writes: This is an excerpt from my code: newtset-tids = (bytea *) fastgetattr(tupleTSet, LABELS_ALIGNED, fctx-tupleSetDesc, isnull); It seems that for an empty bytea (only the size of the header), i get that VARSIZE(newtset-tids)==534765440 instead of

Re: [HACKERS] Eliminating unnecessary left joins

2007-04-06 Thread Andreas Pflug
Tom Lane wrote: =?ISO-8859-1?Q?Ott=F3_Havasv=F6lgyi?= [EMAIL PROTECTED] writes: When using views built with left joins, and then querying against these views, there are a lot of join in the plan that are not necessary, because I don't select/use any column of each table in the views every

Re: [PATCHES] [HACKERS] [Fwd: Index Advisor]

2007-04-06 Thread Tom Lane
Gurjeet Singh [EMAIL PROTECTED] writes: Please find attached the latest version of the patch. It applies cleanly on REL8_2_STABLE. The interface to the planner in this seems rather brute-force. To run a plan involving a hypothetical index, you have to make a bunch of catalog entries, run the

[HACKERS] TODO item: Allow the server to be stopped/restarted via an SQL API

2007-04-06 Thread Aleksis Petrov
Hi all, Would it be acceptable to invoke pg_ctl in the API functions to stop and restart the server? thanks

Re: [HACKERS] TODO item: Allow the server to be stopped/restarted via an SQL API

2007-04-06 Thread Tom Lane
Aleksis Petrov [EMAIL PROTECTED] writes: Would it be acceptable to invoke pg_ctl in the API functions to stop and restart the server? No. Not everyone uses pg_ctl for that. regards, tom lane ---(end of broadcast)--- TIP

Re: [HACKERS] TODO item: Allow the server to be stopped/restarted via an SQL API

2007-04-06 Thread Aleksis Petrov
Via signal handling? Use the kill() function (or its pg equivalent) to send the appropriate signal? On 4/6/07, Tom Lane [EMAIL PROTECTED] wrote: Aleksis Petrov [EMAIL PROTECTED] writes: Would it be acceptable to invoke pg_ctl in the API functions to stop and restart the server? No. Not

Re: [HACKERS] Bitmap index thoughts (another segfault)

2007-04-06 Thread Mark Kirkwood
Mark Kirkwood wrote: I'm seeing a segfault on a size TPC-H size 10 database. The patch and code are: - bitmap patch from 12 Mar - 8.3 dev from 27 Mar SELECT count(distinct(o_orderkey)) FROM orders orders_alias WHERE o_orderpriority IN ('1-URGENT', '3-MEDIUM') AND o_orderstatus='P'; (gdb) bt