Re: [HACKERS] slow IN() clause for many cases

2005-10-12 Thread Jonah H. Harris
true dat :) On 10/12/05, Tom Lane [EMAIL PROTECTED] wrote: Ilia Kantor [EMAIL PROTECTED] writes: Bitmap Heap Scan on objects_hier(cost=60.29..179.57 rows=80 width=600) (actual time=0.835..1.115 rows=138 loops=1) vsMerge Join(cost=62.33..576.80 rows=1117 width=600) (actual time=0.542..2.898

Re: [HACKERS] slow IN() clause for many cases

2005-10-12 Thread Andrew - Supernews
On 2005-10-11, Ilia Kantor [EMAIL PROTECTED] wrote: When in clause becomes large enough (20-30 cases), It is much better to use join way of processing.. or even a different way of writing the IN clause. This one is one I've used after considerable research: select * from table where field in

Re: [HACKERS] Socket problem using beta2 on Windows-XP

2005-10-12 Thread Alvaro Herrera
Thomas Hallgren wrote: With great help from Magnus, who advised me to use lspfix from cexx.org to list my lsp's, I found that I had gapsp.dll, Neoteris DNS Provider installed. An uninstall of the Neoteris software made this problem go away. I guess the question is, why is a DNS Provider

Re: [HACKERS] Socket problem using beta2 on Windows-XP

2005-10-12 Thread Magnus Hagander
With great help from Magnus, who advised me to use lspfix from cexx.org to list my lsp's, I found that I had gapsp.dll, Neoteris DNS Provider installed. An uninstall of the Neoteris software made this problem go away. I guess the question is, why is a DNS Provider software blocking

Re: [HACKERS] Beta3 Bundled

2005-10-12 Thread Dave Page
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marc G. Fournier Sent: 12 October 2005 00:50 To: pgsql-hackers@postgresql.org Subject: [HACKERS] Beta3 Bundled Sizes look right compared to beta2 ... please check it over and make sure

Re: [HACKERS] Need A Suggestion

2005-10-12 Thread Alfranio Correia Junior
You can find something to run triggers on commit at http://gorda.di.uminho.pt/community/pgsqlhooks/ It also allows to define triggers on begin, startup and shutdown. It works fine with triggers written in C. A patch will be available soon in order to allow to writte triggers in other languages

Re: [HACKERS] database vacuum from cron hanging

2005-10-12 Thread Tom Lane
I wrote: Kevin Grittner [EMAIL PROTECTED] writes: (gdb) p BufferDescriptors[781] $1 = {tag = {rnode = {spcNode = 1663, dbNode = 16385, relNode = 2666}, blockNum = 1}, flags = 70, usage_count = 5, refcount = 4294967294, wait_backend_pid = 748, buf_hdr_lock = 0 '\0', buf_id = 781, freeNext =

Re: [HACKERS] Socket problem using beta2 on Windows-XP

2005-10-12 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: To reiterate the basic point: The broken LSP breaks a fundamental promise in the sockets API that we absolutely require. The bug is completely within the LSP. ISTM that maybe what we have here is a documentation shortcoming. I'm thinking that our

Re: [HACKERS] Socket problem using beta2 on Windows-XP

2005-10-12 Thread Magnus Hagander
To reiterate the basic point: The broken LSP breaks a fundamental promise in the sockets API that we absolutely require. The bug is completely within the LSP. ISTM that maybe what we have here is a documentation shortcoming. I'm thinking that our Windows FAQ ought to suggest

Re: [HACKERS] slow IN() clause for many cases

2005-10-12 Thread Tom Lane
Andrew - Supernews [EMAIL PROTECTED] writes: As the number of items in the IN clause increases, the planning time grows rather radically. I was looking at this yesterday. There is some O(N^2) behavior in create_bitmap_subplan, stemming from trying to remove duplicated qual conditions. That

Re: [HACKERS] database vacuum from cron hanging

2005-10-12 Thread Kevin Grittner
From an email from the tech who built the box: The hardware is: Server: IBM x346 Model: 8840-42U CPU: (2) 3.6 GHz Xeon processors RAM: 8 GB RAID 5 OS is SUSE Linux Profession 9.3, fully patched. [as of about a week ago] SuSE Linux 9.3 (i586) - Kernel \r (\l). Linux dtr2.wicourts.gov

Re: [HACKERS] database vacuum from cron hanging

2005-10-12 Thread Alvaro Herrera
Kevin Grittner wrote: I'm not sure what you mean regarding pg_config -- could you clarify? The output of pg_config --configure Your email came through as I was trying to figure out where to find the core dump. We restarted the server with cassert, and I find this in the log prior to my

Re: [HACKERS] database vacuum from cron hanging

2005-10-12 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Kevin Grittner wrote: I'm not sure what you mean regarding pg_config -- could you clarify? The output of pg_config --configure Actually I wanted the whole thing, not just --configure (I'm particularly interested in the CFLAGS setting). Your email

Re: [HACKERS] database vacuum from cron hanging

2005-10-12 Thread Kevin Grittner
[EMAIL PROTECTED]:~/postgresql-snapshot /usr/local/pgsql/bin/pg_config BINDIR = /usr/local/pgsql/bin DOCDIR = /usr/local/pgsql/doc INCLUDEDIR = /usr/local/pgsql/include PKGINCLUDEDIR = /usr/local/pgsql/include INCLUDEDIR-SERVER = /usr/local/pgsql/include/server LIBDIR = /usr/local/pgsql/lib

Re: [HACKERS] database vacuum from cron hanging

2005-10-12 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: bufmgr.s file coming in separate (off-list) email. Yup, here is the smoking gun! This code in PinBuffer LockBufHdr_NoHoldoff(buf); buf-refcount++; result = (buf-flags BM_VALID) != 0;

Re: [HACKERS] database vacuum from cron hanging

2005-10-12 Thread Kevin Grittner
Declaring it as volatile gives: movb$1, %al cmpb$0,28(%ebx) jne 1f lock xchgb %al,28(%ebx) 1: testb %al, %al jne .L228 .L221: movl20(%ebx), %eax incl%eax movl%eax, 20(%ebx)

Re: [HACKERS] database vacuum from cron hanging

2005-10-12 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: Declaring it as volatile gives: [ valid code ] Good, so at least your compiler's not broken. I'm busy volatile-izing all the code in bufmgr.c ... should be able to commit a fix soon. regards, tom lane

[HACKERS] 8.1 beta1 - beta2 upgrade question

2005-10-12 Thread Eric Sproul
Hello, I am a systems admin for OmniTI Inc. I'm posting this question at the request of my boss, Theo Schlossnagle. I am new to Postgres, but Theo has asked me to engage some developers for help, so please bear with me and forgive my ignorance. We are testing a 1TB database on 8.1 beta 1.

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-10-12 Thread Simon Riggs
On Wed, 2005-09-14 at 13:32 -0400, Tom Lane wrote: I wrote: Another thought came to mind: maybe the current data layout for LWLocks is bad. Right now, the spinlock that protects each LWLock data struct is itself part of the struct, and since the structs aren't large (circa 20 bytes), the

[HACKERS] definition of evalfunc for execution of join condition

2005-10-12 Thread Esha Palta
ExecQual evaluates join conditions one at a time.It captures one condition and passes it to function ExecEvalExpr which is actually a macro that invokes another function evalfunc( which is a function pointer that is set to the function PostgreSQL wants to use to evaluate the expression. It's

Re: [HACKERS] 8.1 beta1 - beta2 upgrade question

2005-10-12 Thread Eric Sproul
On Wed, 2005-10-12 at 11:20 -0600, Aly S.P Dharshi wrote: Would it be possible to replicate your database to a Slony based beta 2 and then promote this to a master whilst upgrading the old master ? I presume that Slony will do the trick with the 8.1 betas Aly, We don't have the online

Re: [HACKERS] definition of evalfunc for execution of join condition

2005-10-12 Thread Jonah H. Harris
If I understood your question, the following info may be what you're looking for. ExecNestLoop gets the join qualifications and passes them to ExecQual for each outer tuple. Each qualification is checked in ExecQual using the ExecEvalExpr macro which calls the function pointer evalfunc for each

Re: [HACKERS] 8.1 beta1 - beta2 upgrade question

2005-10-12 Thread Alvaro Herrera
Eric Sproul wrote: Hi, We are testing a 1TB database on 8.1 beta 1. The upgrade path to beta 2 (and presumably from beta 2 to the next release, all the way through production) requires a full export and import. Even on our beefy hardware (quad-Opteron, 16GB RAM) we are looking at a 26-hour

Re: [HACKERS] slow IN() clause for many cases

2005-10-12 Thread Ilia Kantor
1) merge join is faster for me then hash join (6 vs 3-4ms): explain analyze select * from objects_hier where id in (select (array[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26, 27,28,29,30])[i] from generate_series(1,30) s(i)); Hash Join (cost=17.50..162.93 rows=223

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-10-12 Thread Emil Briggs
where the number of padding locks is determined by how many lock structures fit within a 128 byte cache line. This isn't exactly elegant coding, but it provides a useful improvement on an 8-way SMP box when run on 8.0 base. OK, lets be brutal: this looks pretty darn stupid. But it does

Re: [HACKERS] slow IN() clause for many cases

2005-10-12 Thread Jonah H. Harris
IMHO, we should first look at whether the O(N^2) behavior is needed.On 10/12/05, Ilia Kantor [EMAIL PROTECTED] wrote:1) merge join is faster for me then hash join (6 vs 3-4ms): explain analyze select * from objects_hier where id in

[HACKERS] Comments on columns in the pg_catalog tables/views

2005-10-12 Thread David Fetter
Folks, Before I dive into this, is there some reason why the pg_catalog.* tables/views should not have comments that match the descriptions in the docs? I can see where this could cause some maintenance issues, and those probably need to be addressed, but it sure would be nice if \d+ pg_depend

[HACKERS] Are cost estimates based on asserts?

2005-10-12 Thread Kevin Grittner
I'm running right now with a dev build which was configured with --enable-cassert (and --enable-debug). Just out of curiosity, I re-ran the query which chose the sort and mergejoin over the (faster) nested index scan. The results seem interesting, although I'd need more tests to consider them

Re: [HACKERS] database vacuum from cron hanging

2005-10-12 Thread Martijn van Oosterhout
On Wed, Oct 12, 2005 at 12:25:11PM -0400, Tom Lane wrote: Kevin Grittner [EMAIL PROTECTED] writes: Declaring it as volatile gives: [ valid code ] Good, so at least your compiler's not broken. I'm busy volatile-izing all the code in bufmgr.c ... should be able to commit a fix soon. Well,

Re: [HACKERS] slow IN() clause for many cases

2005-10-12 Thread Andrew - Supernews
On 2005-10-12, Tom Lane [EMAIL PROTECTED] wrote: Andrew - Supernews [EMAIL PROTECTED] writes: As the number of items in the IN clause increases, the planning time grows rather radically. I was looking at this yesterday. There is some O(N^2) behavior in create_bitmap_subplan, stemming from

Re: [HACKERS] 8.1 beta1 - beta2 upgrade question

2005-10-12 Thread Eric Sproul
On Wed, 2005-10-12 at 13:00 -0400, Eric Sproul wrote: We are interested in finding a way to directly translate the data files from one version to the next. We can devote resources to work on this if the development team can give us guidance. Please excuse the followup to myself... We will,

Re: [HACKERS] How TODO prevent PQfnumber() from lowercasing?

2005-10-12 Thread Volkan YAZICI
On 10/12/05, Bruce Momjian pgman@candle.pha.pa.us wrote: The question mark means we are not sure how to deal with it. I think your idea of using quotes to preserve case is a good one. I think related TODO is added for that gotcha which was written in PQfnumber() comments in fe-exec.c:

Re: [HACKERS] 8.1 beta1 - beta2 upgrade question

2005-10-12 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: There has been talk in the past about creating a tool for in-place upgrades. The most complete description of the needed work I've seen posted to date is here: http://archives.postgresql.org/pgsql-hackers/2003-12/msg00379.php There was another

Re: [HACKERS] Comments on columns in the pg_catalog tables/views

2005-10-12 Thread Tom Lane
David Fetter [EMAIL PROTECTED] writes: Before I dive into this, is there some reason why the pg_catalog.* tables/views should not have comments that match the descriptions in the docs? I can see where this could cause some maintenance issues, Yeah. If you can figure a way to auto-generate

Re: [HACKERS] Are cost estimates based on asserts?

2005-10-12 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: [$subject] No, the planner has no idea about the cost of Asserts. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to

Re: [HACKERS] How TODO prevent PQfnumber() from lowercasing?

2005-10-12 Thread Bruce Momjian
Volkan YAZICI wrote: On 10/12/05, Bruce Momjian pgman@candle.pha.pa.us wrote: The question mark means we are not sure how to deal with it. I think your idea of using quotes to preserve case is a good one. I think related TODO is added for that gotcha which was written in PQfnumber()

Re: [HACKERS] How TODO prevent PQfnumber() from lowercasing?

2005-10-12 Thread Tom Lane
Volkan YAZICI [EMAIL PROTECTED] writes: Returned column names from the backend were lowercased by the server in this or that way. Furthermore, PQfnumber() makes not-quoted strings downcasing on the client side and then performs the compare by using the results returned from the backend. To sum

Re: [HACKERS] Comments on columns in the pg_catalog tables/views

2005-10-12 Thread Tom Lane
I wrote: David Fetter [EMAIL PROTECTED] writes: Before I dive into this, is there some reason why the pg_catalog.* tables/views should not have comments that match the descriptions in the docs? I can see where this could cause some maintenance issues, Yeah. If you can figure a way to

Re: [HACKERS] Are cost estimates based on asserts?

2005-10-12 Thread Kevin Grittner
Hmmm... Poorly phrased question on my part, I guess. When I saw that the cost estimates tracked so much better with the actual execution time when the asserts were being evaluated, it got me wondering whether some significant part of the costing development and testing occured in that

[HACKERS] A costing analysis tool

2005-10-12 Thread Kevin Grittner
I'm looking at trying to fix some clear flaws in costing which cause of our real-world queries to choose sub-optimal plans under PostgreSQL. It's clear that there needs to be a tool to analyze the accuracy of costing for a variety of queries, both to direct any efforts to fix problems and to test

Re: [HACKERS] Comments on columns in the pg_catalog tables/views

2005-10-12 Thread David Fetter
On Wed, Oct 12, 2005 at 07:11:12PM -0400, Tom Lane wrote: I wrote: David Fetter [EMAIL PROTECTED] writes: Before I dive into this, is there some reason why the pg_catalog.* tables/views should not have comments that match the descriptions in the docs? I can see where this could cause

Re: [HACKERS] Comments on columns in the pg_catalog tables/views

2005-10-12 Thread Tom Lane
David Fetter [EMAIL PROTECTED] writes: On Wed, Oct 12, 2005 at 07:11:12PM -0400, Tom Lane wrote: Dept of second thoughts: actually, perhaps see if you can generate the pg_description entries from the C comments in the include/catalog header files. My thought is that by the time somebody is

Re: [HACKERS] GRANT/roles problem: grant is shown as from login role

2005-10-12 Thread Kevin Brown
Tom Lane wrote: So I think we don't have much choice but to implement theory #2; which is essentially the same thing I said earlier, ie, ACLs have to record the grantor of a privilege as being the role actually holding the grant option, not the role-member issuing the GRANT. There are really

Re: [HACKERS] How TODO prevent PQfnumber() from lowercasing?

2005-10-12 Thread Bruce Momjian
Tom Lane wrote: Volkan YAZICI [EMAIL PROTECTED] writes: Returned column names from the backend were lowercased by the server in this or that way. Furthermore, PQfnumber() makes not-quoted strings downcasing on the client side and then performs the compare by using the results returned

Re: [HACKERS] A costing analysis tool

2005-10-12 Thread Bruce Momjian
Have you looked at the TODO list to see our previous ideas on tuning diagnotics? --- Kevin Grittner wrote: I'm looking at trying to fix some clear flaws in costing which cause of our real-world queries to choose

Re: [HACKERS] Comments on columns in the pg_catalog tables/views

2005-10-12 Thread Christopher Kings-Lynne
Yes for the love of god do it :D David Fetter wrote: Folks, Before I dive into this, is there some reason why the pg_catalog.* tables/views should not have comments that match the descriptions in the docs? I can see where this could cause some maintenance issues, and those probably need to be

Re: [HACKERS] [COMMITTERS] pgsql: Do all accesses to shared buffer headers through

2005-10-12 Thread Bruce Momjian
Does any of this need to be backpatched? --- Tom Lane wrote: Log Message: --- Do all accesses to shared buffer headers through volatile-qualified pointers, to ensure that compilers won't rearrange accesses to

Re: [HACKERS] [COMMITTERS] pgsql: Do all accesses to shared buffer headers through

2005-10-12 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: Does any of this need to be backpatched? No --- we didn't have any per-buffer spinlocks before 8.1. It's possible that at some point we'll need to start thinking about applying volatile-pointer coding rules to data structures protected by LWLocks.

Re: [HACKERS] [COMMITTERS] pgsql: Do all accesses to shared buffer

2005-10-12 Thread Neil Conway
On Wed, 2005-12-10 at 22:46 -0400, Tom Lane wrote: No --- we didn't have any per-buffer spinlocks before 8.1. Right. To summarize the problem as I understand it: we need to force $CC not to move loads and stores around spinlock acquire/release. This can't be done by just marking the spinlock

Re: [HACKERS] Minor point about contrib/xml2 functions IMMUTABLE marking

2005-10-12 Thread Bruce Momjian
Agreed. I have changed them both to stable. I think xslt_process() should be stable because it is unlikely you would want a URL's contents to change inside a transaction, but likely you would want it to change between transactions.

Re: [HACKERS] [COMMITTERS] pgsql: Do all accesses to shared buffer headers through

2005-10-12 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: Another way to fix the problem would be to have S_LOCK() and S_UNLOCK() force $CC to not rearrange loads and stores by themselves, without relying upon volatile pointers. That would certainly be better if possible, but AFAIK it's not. (Perhaps there is a

[HACKERS] Darwin compile fixes

2005-10-12 Thread Bruce Momjian
Brian A. Seklecki wrote: On Sun, 2005-10-09 at 10:36, Brian A. Seklecki wrote: Is libpq.dylib getting built? Is it there and just not being found? If so, you may need the LDFLAGS/CFLAGS settings from: http://darwinports.opendarwin.org/darwinports/dports/databases/postgresql8/Portfile

Re: [HACKERS] Minor point about contrib/xml2 functions IMMUTABLE

2005-10-12 Thread Neil Conway
On Wed, 2005-12-10 at 23:46 -0400, Bruce Momjian wrote: Agreed. I have changed them both to stable. I think xslt_process() should be stable because it is unlikely you would want a URL's contents to change inside a transaction Why is it unlikely? If a function's return value for a particular

Re: [HACKERS] Darwin compile fixes

2005-10-12 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: Attached is a cleaned up version of the patch (without wrapping). The change is to use $(CXX) $(CFLAGS) instead of $(COMPILER). Does this change break OS/X? Since our configure doesn't define CXX, I'm having a hard time imagining how it could fail

Re: [HACKERS] A costing analysis tool

2005-10-12 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: Note that I'm talking about a tool strictly to check the accuracy of the estimated costs of plans chosen by the planner, nothing else. We could definitely do with some infrastructure for testing this. I concur with Bruce's suggestion that you should comb