[HACKERS] unsubscribe pgsql-hackers@postgresql.org

2003-11-03 Thread Miguel Biscaia
unsubscribe [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Christopher Browne
The world rejoiced as [EMAIL PROTECTED] (Hannu Krosing) wrote: Christopher Browne kirjutas E, 03.11.2003 kell 02:15: Well, actually, the case where it _would_ be troublesome would be where there was a combination of huge tables needing vacuuming and smaller ones that are _heavily_ updated

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 04:21, Hannu Krosing wrote: Neil Conway kirjutas E, 03.11.2003 kell 08:07: (2) ISTM that we can set POSIX_FADV_RANDOM for *all* indexes, since the vast majority of the accesses to them shouldn't be sequential. Perhaps we could do it for all _leaf_ nodes, the root and

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Andrew Sullivan
On Mon, Nov 03, 2003 at 08:50:00AM -0500, Neil Conway wrote: pool or the kernel's page cache. We can implement the former easily enough, but I don't see any feasible way to do the latter: on a high-end machine with gigabytes of RAM but a relatively small shared_buffers (which is the

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: So what API is desirable for uses 2-4? I'm thinking of adding a new function to the smgr API, smgradvise(). It's a little premature to be inventing APIs when you have no evidence that this will make any useful performance difference. I'd recommend a quick

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Andrew Sullivan
On Sun, Nov 02, 2003 at 01:00:35PM -0500, Tom Lane wrote: real traction we'd have to go back to the take over most of RAM for shared buffers approach, which we already know to have a bunch of severe disadvantages. I know there are severe disadvantages in the current implementation, but are

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Matthew T. O'Connor
Christopher Browne wrote: The world rejoiced as [EMAIL PROTECTED] (Hannu Krosing) wrote: Christopher Browne kirjutas E, 03.11.2003 kell 02:15: Well, actually, the case where it _would_ be troublesome would be where there was a combination of huge tables needing vacuuming and smaller ones

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: POSIX_FADV_RANDOM doesn't effect the page cache, it just determines how aggressive the kernel is when doing readahead (at least on Linux, but I'd expect to see other kernels implement similar behavior). I would expect POSIX_FADV_SEQUENTIAL to reduce the

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Tom Lane
Andrew Sullivan [EMAIL PROTECTED] writes: On Sun, Nov 02, 2003 at 01:00:35PM -0500, Tom Lane wrote: real traction we'd have to go back to the take over most of RAM for shared buffers approach, which we already know to have a bunch of severe disadvantages. I know there are severe

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 10:01, Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: POSIX_FADV_RANDOM doesn't effect the page cache, it just determines how aggressive the kernel is when doing readahead (at least on Linux, but I'd expect to see other kernels implement similar behavior). I

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: On Mon, 2003-11-03 at 10:01, Tom Lane wrote: I would expect POSIX_FADV_SEQUENTIAL to reduce the chance that a page will be kept in buffer cache after it's been used. I don't think that can be reasonably implied from the POSIX text, which is merely:

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Hannu Krosing
Christopher Browne kirjutas E, 03.11.2003 kell 15:22: Can't one just run a _separate_ VACUUM on those smaller tables ? Yes, but that defeats the purpose of having a daemon that tries to manage this all for you. If a dumb deamon can't do its work well, we need smarter daemons ;)

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 11:11, Tom Lane wrote: Why not? The advice says that you're going to access the data sequentially in the forward direction. If you're not going to back up, there is no point in keeping pages in cache after they've been read. The advice says: I'm going to read this data

[HACKERS] RC1 on AIX - working thus far

2003-11-03 Thread Christopher Browne
... much omitted ... alter_table ... ok sequence ... ok polymorphism ... ok stats... ok == shutting down postmaster == == All 93 tests passed. == rm

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Hannu Krosing
Neil Conway kirjutas E, 03.11.2003 kell 18:59: On Mon, 2003-11-03 at 11:11, Tom Lane wrote: Why not? The advice says that you're going to access the data sequentially in the forward direction. If you're not going to back up, there is no point in keeping pages in cache after they've been

[HACKERS] docco on external storage?

2003-11-03 Thread Andrew Dunstan
Perhaps I am missing something, but the documentation on external storage appears remarkably sparse. It says it is better for substring searches, but otherwise doesn't say very much. I have a table with two modest-length bytea fields (around 9K and 2K respectively) of already compressed data.

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 12:17, Hannu Krosing wrote: Can you use combinations like ( FADV_NOREUS | FADV_SEQUENTIAL ) You can do an fadvise() for FADV_SEQUENTIAL, and then another fadvise() for FADV_NOREUSE. -Neil ---(end of broadcast)--- TIP 4:

[HACKERS] Experimental ARC implementation

2003-11-03 Thread Jan Wieck
Attached is a first trial implementation of the Adaptive Replacement Cache (ARC). The patch is against CVS HEAD of 7.4. The algorithm is based on what's described in these papers: http://www.almaden.ibm.com/StorageSystems/autonomic_storage/ARC/arcfast.pdf

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Manfred Spraul
Neil Conway wrote: The present Linux implementation doesn't do this, AFAICS -- all it does it increase the readahead for this file: AFAIK Linux uses a modified LRU that automatically puts pages that were touched only once at a lower priority than frequently accessed pages. Neil: what about

Re: [HACKERS] RC1 on AIX - working thus far

2003-11-03 Thread Peter Eisentraut
Christopher Browne writes: bash-2.05a$ uname -a AIX ibm-db 1 5 000CD13A4C00 We already have a report for AIX. Were you trying to indicate that this is a different variant thereof? -- Peter Eisentraut [EMAIL PROTECTED] ---(end of

Re: Avoiding SIGPIPE (was Re: [HACKERS] OSDL DBT-2 w/ PostgreSQL

2003-11-03 Thread markw
On 1 Nov, Tom Lane wrote: Manfred Spraul [EMAIL PROTECTED] writes: Tom Lane wrote: What we really want is to turn off SIGPIPE delivery on our socket (only), but AFAIK there is no API to do that. Linux has as MSG_NOSIGNAL flag for send(), but that seems to be Linux specific. Hmm ... a

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 09:38, Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: Given a Relation and an advice, this would: (a) propagate the advice for this relation to all the open FDs for the relation All? You cannot affect the FDs being used by other backends. Sorry, I meant

Re: [HACKERS] RC1 on AIX - working thus far

2003-11-03 Thread Christopher Browne
Peter Eisentraut [EMAIL PROTECTED] writes: Christopher Browne writes: bash-2.05a$ uname -a AIX ibm-db 1 5 000CD13A4C00 We already have a report for AIX. Were you trying to indicate that this is a different variant thereof? I'm afraid I hadn't seen another AIX report; this may replicate

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Jan Wieck
Christopher Browne wrote: The world rejoiced as [EMAIL PROTECTED] (Hannu Krosing) wrote: Christopher Browne kirjutas E, 03.11.2003 kell 02:15: Well, actually, the case where it _would_ be troublesome would be where there was a combination of huge tables needing vacuuming and smaller ones that are

Re: [HACKERS] 7.4RC1 tag'd, branched and bundled ...

2003-11-03 Thread Peter Eisentraut
Marc G. Fournier writes: As Tom announced on Friday, Release Candidate 1 has now been tag'd and bundled ... Can you please sometime decide once and for all how you're going to name the tarballs and then communicate that information to the group? Is it rc1 or RC1, beta2 or b2? It's not that

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 14:24, Manfred Spraul wrote: Neil: what about calling posix_fadvise for the whole file immediately after issue_xlog_fsync() in XLogWrite? According to the comment, it's guaranteed that this will happen only once. Or: add an posix_fadvise into issue_xlog_fsync(), for

Re: [HACKERS] RC1 on AIX - working thus far

2003-11-03 Thread Peter Eisentraut
Christopher Browne writes: I'm afraid I hadn't seen another AIX report; this may replicate other reports... See http://developer.postgresql.org/docs/postgres/supported-platforms.html for a list of platforms that have been verified with 7.4. (Linux/Playstation, Linux/hppa, and UnixWare will be

Re: [HACKERS] 7.4RC1 tag'd, branched and bundled ...

2003-11-03 Thread Marc G. Fournier
On Mon, 3 Nov 2003, Peter Eisentraut wrote: Marc G. Fournier writes: As Tom announced on Friday, Release Candidate 1 has now been tag'd and bundled ... Can you please sometime decide once and for all how you're going to name the tarballs and then communicate that information to the

Re: Avoiding SIGPIPE (was Re: [HACKERS] OSDL DBT-2 w/ PostgreSQL

2003-11-03 Thread Manfred Spraul
Tom Lane wrote: It strikes me that sigpipe handling will be a global affair in any particular application --- it's unlikely that it would be correct for some PG connections and wrong for others. So one possibility is to make the control variable be global (static) and thus it could be set before

Re: Avoiding SIGPIPE (was Re: [HACKERS] OSDL DBT-2 w/ PostgreSQL

2003-11-03 Thread Tom Lane
[EMAIL PROTECTED] writes: I'm a bit unfamiliar with this stuff, so I wanted to ask if this was something that Linux appears to be handling differently than other OS's, or if this was a platform specific issue with postgresql. It's generic to all Unixen. regards, tom

Re: [HACKERS] 7.4RC1 tag'd, branched and bundled ...

2003-11-03 Thread Peter Eisentraut
Marc G. Fournier writes: Why does it scare you? *raised eyebrow* Because release-making should be a predictable process. But if every other release brings a new surprise in the naming scheme, it's not. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of

Re: [HACKERS] Experimental ARC implementation

2003-11-03 Thread Jan Wieck
Jan Wieck wrote: I will follow up shortly with an approach that integrates Tom's delay mechanism plus my first READ_BY_VACUUM hack into one combined experiement. Okay, the attached patch contains the 3 already discussed and one additional change. I also made a few changes. 1) ARC policy. Has

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: On Mon, 2003-11-03 at 11:11, Tom Lane wrote: Why not? The advice says that you're going to access the data sequentially in the forward direction. If you're not going to back up, there is no point in keeping pages in cache after they've been read. The

Re: [HACKERS] 7.4RC1 tag'd, branched and bundled ...

2003-11-03 Thread Gaetano Mendola
Marc G. Fournier wrote: As Tom announced on Friday, Release Candidate 1 has now been tag'd and bundled ... please test to confirm that nothing seems off with this build Is in the last Tom's patch about Vacuum sleep between pages ? Regards Gaetano Mendola ---(end of

Re: [HACKERS] 7.4RC1 tag'd, branched and bundled ...

2003-11-03 Thread Marc G. Fournier
On Tue, 4 Nov 2003, Gaetano Mendola wrote: Marc G. Fournier wrote: As Tom announced on Friday, Release Candidate 1 has now been tag'd and bundled ... please test to confirm that nothing seems off with this build Is in the last Tom's patch about Vacuum sleep between pages ? that won't

[HACKERS] 7.4RC1 failed to build on Linux

2003-11-03 Thread Kenji Sugita
Failed to build on Linux. $ uname -mrp 2.2.18-0ph1smp i686 unknown $ uname -mr 2.2.18-0ph1smp i686 $ cat /etc/redhat-release Red Hat Linux release 6.2 (Zoot) $ gcc --version 2.95.3 $ cat ../myconf #!/bin/sh CFLAGS=-O2 ./configure \ --with-tcl

Re: [HACKERS] 7.4RC1 failed to build on Linux

2003-11-03 Thread Marc G. Fournier
what version of tcl do you have installed? On Tue, 4 Nov 2003, Kenji Sugita wrote: Failed to build on Linux. $ uname -mrp 2.2.18-0ph1smp i686 unknown $ uname -mr 2.2.18-0ph1smp i686 $ cat /etc/redhat-release Red Hat Linux release 6.2 (Zoot) $ gcc --version 2.95.3 $ cat ../myconf

Re: [HACKERS] 7.4RC1 tag'd, branched and bundled ...

2003-11-03 Thread Tom Lane
Marc G. Fournier [EMAIL PROTECTED] writes: On Tue, 4 Nov 2003, Gaetano Mendola wrote: Is in the last Tom's patch about Vacuum sleep between pages ? that won't be in v7.4, to the best of my knowledge ... Definitely not. It's a very experimental patch. regards, tom

Re: [HACKERS] 7.4RC1 failed to build on Linux

2003-11-03 Thread Tom Lane
Marc G. Fournier [EMAIL PROTECTED] writes: On Tue, 4 Nov 2003, Kenji Sugita wrote: ../../../src/interfaces/libpgtcl/libpgtcl.so: undefined reference to `Tcl_NewByteArrayObj' ../../../src/interfaces/libpgtcl/libpgtcl.so: undefined reference to `Tcl_GetByteArrayFromObj' what version of tcl

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Ang Chin Han
Christopher Browne wrote: Centuries ago, Nostradamus foresaw when Stephen [EMAIL PROTECTED] would write: As it turns out. With vacuum_page_delay = 0, VACUUM took 1m20s (80s) to complete, with vacuum_page_delay = 1 and vacuum_page_delay = 10, both VACUUMs completed in 18m3s (1080 sec). A factor of

Re: [HACKERS] 7.4RC1 failed to build on Linux

2003-11-03 Thread Jan Wieck
Marc G. Fournier wrote: what version of tcl do you have installed? Looks like 8.0. As per recent discussion we are actually require 8.1 exactly because of those missing functions in all versions prior to that. Jan On Tue, 4 Nov 2003, Kenji Sugita wrote: Failed to build on Linux. $ uname

[HACKERS] bufmgr code question

2003-11-03 Thread Neil Conway
In the BufferDesc struct, there seem to be two ways to mark a buffer page as dirty: setting the BM_DIRTY bit mask in the 'flags' field of the struct, and setting the 'cntxDirty' field to true. What is the difference between these two indications of a page's dirtiness? Or, more to the point, is

Re: [HACKERS] Experimental patch for inter-page delay in VACUUM

2003-11-03 Thread Jan Wieck
Ang Chin Han wrote: Christopher Browne wrote: Centuries ago, Nostradamus foresaw when Stephen [EMAIL PROTECTED] would write: As it turns out. With vacuum_page_delay = 0, VACUUM took 1m20s (80s) to complete, with vacuum_page_delay = 1 and vacuum_page_delay = 10, both VACUUMs completed in 18m3s

Re: [HACKERS] [DOCS] Annotated release notes

2003-11-03 Thread Josh Berkus
Ms. Carroll, I'm hoping one of you an answer a question for me. I am accessing Service Center data via an Oracle ODBC driver for reporting and queries. How can I get the resolution field using these methods. Howdy from one of your customers! I'm not certain how, exactly, you got the set of