Re: [HACKERS] Batch update of indexes on data loading

2008-02-28 Thread Tom Lane
ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> wrote: >>> Can we do REINDEX >>> holding only shared lock on the index? >> >> No. When you commit the reindex, the old copy of the index will >> instantaneously disappear; it will not do for someone to be actively >> scan

Re: [HACKERS] Batch update of indexes on data loading

2008-02-28 Thread ITAGAKI Takahiro
Tom Lane <[EMAIL PROTECTED]> wrote: > > Can we do REINDEX > > holding only shared lock on the index? > > No. When you commit the reindex, the old copy of the index will > instantaneously disappear; it will not do for someone to be actively > scanning that copy. Hmm... Is it ok if the index wil

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Tom Lane
"Tom Dunstan" <[EMAIL PROTECTED]> writes: > This seems like a good enough opportunity to mention an idea that I > had while/after doing the enum patch. The patch was fairly intrusive > for something that was just adding a type because postgresql isn't > really set up for parameterized types other t

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Tom Dunstan
On Fri, Feb 29, 2008 at 9:26 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > Andrew Sullivan <[EMAIL PROTECTED]> writes: > > "Be conservative in what you send, and liberal in what you accept." > > Yeah, I was about to quote that same maxim myself. I don't have a big > problem with allowing uuid_in to

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Mark Mielke
James Mansion wrote: Mark Mielke wrote: I recall there being a measurable performance difference between the most liberal parser, and the most optimized parser, back when I wrote one for PostgreSQL. I don't know how good the one in use for PostgreSQL 8.3 is. As to whether the cost is noticeabl

Re: [HACKERS] Batch update of indexes on data loading

2008-02-28 Thread Tom Lane
"Markus Bertheau" <[EMAIL PROTECTED]> writes: > 2008/2/29, Tom Lane <[EMAIL PROTECTED]>: >> No. When you commit the reindex, the old copy of the index will >> instantaneously disappear; it will not do for someone to be actively >> scanning that copy. > Can a shared lock be taken at first, and whe

Re: [HACKERS] Batch update of indexes on data loading

2008-02-28 Thread Markus Bertheau
2008/2/29, Tom Lane <[EMAIL PROTECTED]>: > ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > > BTW, why REINDEX requires access exclusive lock? Read-only queries > > are forbidden during the operation now, but I feel they are ok > > because REINDEX only reads existing tuples. Can we do REINDEX > >

Re: [HACKERS] Silly Newbie question

2008-02-28 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Aaron Spiteri wrote: I wish to create a an aggregate function and have read and re-read the 34.10 section of the 8.3 documentation as well as scanned through the fmgr header file. This is not the correct forum for

Re: [HACKERS] Silly Newbie question

2008-02-28 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Aaron Spiteri wrote: >> I wish to create a an aggregate function and have read and re-read the >> 34.10 section of the 8.3 documentation as well as scanned through the >> fmgr header file. > This is not the correct forum for such questions. Please ask

Re: [HACKERS] Silly Newbie question

2008-02-28 Thread Tom Lane
Aaron Spiteri <[EMAIL PROTECTED]> writes: > I wish to create a an aggregate function and have read and re-read the > 34.10 section of the 8.3 documentation as well as scanned through the > fmgr header file. My problem is that I know certain structures are > created with the idea of retaining infor

Re: [HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread Pavan Deolasee
On Fri, Feb 29, 2008 at 9:15 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > > > We've heard that idea before, and it's just as bad as it was when > proposed before. "Pre-frozen" tuples eliminate any possibility of > tracking when a tuple was inserted; which is extremely important to know > when you

Re: [HACKERS] OSSP can be used in the windows environment now!

2008-02-28 Thread Tom Lane
"Dave Page" <[EMAIL PROTECTED]> writes: > On Thu, Feb 28, 2008 at 12:22 PM, Magnus Hagander <[EMAIL PROTECTED]> wrote: >>> If not, what do other people think about adding this Makefile and a >>> README file to our contrib directory? >> >> Comments from others? Objections? > If it's well documente

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Tom Lane
Andrew Sullivan <[EMAIL PROTECTED]> writes: > "Be conservative in what you send, and liberal in what you accept." Yeah, I was about to quote that same maxim myself. I don't have a big problem with allowing uuid_in to accept known format variants. (I'm not sure about allowing a hyphen *anywhere*,

Re: [HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread Tom Lane
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > In a typical scenario, user might create a table and load data in the > table as part of a single transaction (e.g pg_restore). In this case, > it would help if we create the tuples in the *frozen* state to avoid > any wrap-around related issues with t

Re: [HACKERS] Batch update of indexes on data loading

2008-02-28 Thread Tom Lane
ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > BTW, why REINDEX requires access exclusive lock? Read-only queries > are forbidden during the operation now, but I feel they are ok > because REINDEX only reads existing tuples. Can we do REINDEX > holding only shared lock on the index? No. When you

[HACKERS] A couple of PG schedule reminders

2008-02-28 Thread Tom Lane
Don't forget that our first "commit fest" for 8.4 development starts Saturday March 1. If you've got a patch that you'd like reviewed in this fest, it's time to send it in. (But there's plenty more fests to come, so don't panic about it.) Also, 8.3.0 has been out for almost a month, so it's time

[HACKERS] "could not open relation 1663/16384/16584: No such file or directory" in a specific combination of transactions with temp tables

2008-02-28 Thread John Smith
Architecture: Intel Core 2 Duo OS: linux-2.6.20-gentoo-r8 Filesystem: ext3 Postgres v8.2.3 compiled with gcc 4.1.1-r3 RAM - 2GB Shared buffers - 24MB [All other Postgres configuration parameters are default values] Problem description: COPY into temp table fails using a specific combination of cre

Re: [HACKERS] Proposal: wildcards in pg_service.conf

2008-02-28 Thread Dawid Kuroczko
On Thu, Feb 28, 2008 at 11:27 AM, Albe Laurenz <[EMAIL PROTECTED]> wrote: > The LDAP case is the main motivation why I would like to have > wildcards, so that all our databases could be handled with one > entry in the service file. Currently we have to add an entry to the > file for every new d

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Sam Mason
On Thu, Feb 28, 2008 at 06:45:18PM -0500, Mark Mielke wrote: > My personal opinion is that this is entirely a philosophical issue, and > that both sides have merits. I think it depends on what you're optimising for: initial development time, maintaince time or run time. > There is no reason for

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread James Mansion
Mark Mielke wrote: I recall there being a measurable performance difference between the most liberal parser, and the most optimized parser, back when I wrote one for PostgreSQL. I don't know how good the one in use for PostgreSQL 8.3 is. As to whether the cost is noticeable to people or not -

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Mark Mielke
James Mansion wrote: Kenneth Marshall wrote: conversion process themselves. Accepting random input puts a performance hit on everybody following the standard. Why is that necessarily the case? Why not have a liberal parser and a configurable switch that determines whether non-standard forms a

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread James Mansion
Kenneth Marshall wrote: conversion process themselves. Accepting random input puts a performance hit on everybody following the standard. Why is that necessarily the case? Why not have a liberal parser and a configurable switch that determines whether non-standard forms are liberally accepted

Re: [HACKERS] Need Mentors for Google Summer of Code!

2008-02-28 Thread Jonah H. Harris
On Thu, Feb 28, 2008 at 4:48 PM, Josh Berkus <[EMAIL PROTECTED]> wrote: > Mentors must be patient, friendly, easy to stay in touch with, > knowledgeable about their project areas, and able to commit to spending > several hours a week helping their student(s) from May to August. Unlike last time

Re: [HACKERS] Google SoC 2008?

2008-02-28 Thread Kevin Grittner
>>> On Thu, Feb 28, 2008 at 3:43 PM, in message <[EMAIL PROTECTED]>, Josh Berkus <[EMAIL PROTECTED]> wrote: > http://www.postgresql.org/developer/summerofcode That page contains a link which is dead. This line: More information on these projects can be found on Google's PostgreSQL SoC page.

Re: [HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread Kevin Grittner
>>> On Thu, Feb 28, 2008 at 3:08 AM, in message <[EMAIL PROTECTED]>, "Pavan Deolasee" <[EMAIL PROTECTED]> wrote: > I had this idea sometime back. Not sure if this has been discussed before There was a thread discussing the problems you're looking to address: http://archives.postgresql.org/pgs

[HACKERS] Need Mentors for Google Summer of Code!

2008-02-28 Thread Josh Berkus
Hackers, Over the last 2 years, Google Summer of Code has been responsible for several new features of PostgreSQL: XML, ghost XID, phpPgAdmin upgrade, and more, as well as some new long-term contributors to the project. We want to have an equally good summer this year. So: we need PostgreSQL

Re: [HACKERS] Google SoC 2008?

2008-02-28 Thread Josh Berkus
Joey, > Where can I see a list of projects? You can look at last year's. We're updating for this year. http://www.postgresql.org/developer/summerofcode -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco ---(end of broadcast)--- TIP 3: Have yo

Re: [HACKERS] Producer/Consumer Issues in the COPY across network

2008-02-28 Thread Simon Riggs
On Thu, 2008-02-28 at 15:39 +0100, Martijn van Oosterhout wrote: > That's why I was wondering about the rows per packet. Sending bigger > packets reduces overall overhead. > > (The malloc/free per row doesn't seem too efficient.) I guess neither of us know then. Oh well. That's good 'cos it soun

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Kenneth Marshall
On Thu, Feb 28, 2008 at 08:06:46PM +0100, Zeugswetter Andreas ADI SD wrote: > > > > Well, I guess the question is: if we don't offer some builtin way to > render > > > non-standard formats built into company products, will those > companies fix > > > their format or just not use PostgreSQL? > >

Re: [HACKERS] OSSP can be used in the windows environment now!

2008-02-28 Thread Hiroshi Saito
Hi. - Original Message - From: "Dave Page" <[EMAIL PROTECTED]> On Thu, Feb 28, 2008 at 12:22 PM, Magnus Hagander <[EMAIL PROTECTED]> wrote: > >If not, what do other people think about adding this Makefile and a > >README file to our contrib directory? > > If there is no contrary

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Zeugswetter Andreas ADI SD
> > Well, I guess the question is: if we don't offer some builtin way to render > > non-standard formats built into company products, will those companies fix > > their format or just not use PostgreSQL? > > Well, there is an advantage that Postgres has that some others don't: you > can extend

[HACKERS] Google SoC 2008?

2008-02-28 Thread Joey K.
Where can I see a list of projects? Thanks! Joey

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Andrew Sullivan
On Thu, Feb 28, 2008 at 08:58:01AM -0800, Josh Berkus wrote: > Well, I guess the question is: if we don't offer some builtin way to render > non-standard formats built into company products, will those companies fix > their format or just not use PostgreSQL? Well, there is an advantage that Pos

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Josh Berkus
Tom, > I think the question we have to answer is whether we want to be > complicit in the spreading of a nonstandard UUID format. Even if > we answer "yes" for this HP case, it doesn't follow that we should > create a mechanism for anybody to do anything with 'em. That way > lies the madness peo

Re: [HACKERS] Producer/Consumer Issues in the COPY across network

2008-02-28 Thread Martijn van Oosterhout
On Thu, Feb 28, 2008 at 01:57:49AM +, Simon Riggs wrote: > > > > AIUI the server merely streams the rows to you, the client doesn't get > > to say how many :) > > Right, but presumably we generate a new message per PQgetCopyData() > request? So my presumption is we need to wait for that to be

Re: [HACKERS] Silly Newbie question

2008-02-28 Thread Andrew Dunstan
Aaron Spiteri wrote: To all, I am brand new to writing functions for PostgreSQL and understand that this question has been asked one hundred and one times. Please be patient with me, I do learn quickly. My question is: I wish to create a an aggregate function and have read and re-read th

Re: [HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread Florian Pflug
ITAGAKI Takahiro wrote: Without this, very large read-only tables would require one round of complete freezing if there are lot of transactional activities in the other parts of the database. And when that happens, it would generate lots of unnecessary IOs on these large tables. To make things

Re: [HACKERS] Buildfarm member gypsy_moth seems not to like alignment patch

2008-02-28 Thread Jorgen Austvik - Sun Norway
Tom Lane wrote: It looks like gypsy_moth has been failing like this: creating directory /export/home/tmp/pg-test/build-suncc/HEAD/pgsql.21325/src/test/regress/./tmp_check/data ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers/max_f

Re: [HACKERS] Buildfarm member gypsy_moth seems not to like alignment patch

2008-02-28 Thread Jorgen Austvik - Sun Norway
Tom Lane wrote: This is unfortunate and surprising, since that patch was intended to prevent compilers from making unsafe alignment assumptions, but it sure looks like this compiler has instead added a new one. Could you poke into it --- at least get a stack trace from the core dump? Forgot so

Re: [HACKERS] Logging conflicted queries on deadlocks

2008-02-28 Thread Alvaro Herrera
ITAGAKI Takahiro wrote: > There are some open issues. One of the issues is that the killed query is > logged by log_min_error_statement, but conflicted query is logged in DETAIL. > Killed query is logged only on server but conflicted queries are > sent to both server and client. In addition, if lo

[HACKERS] Silly Newbie question

2008-02-28 Thread Aaron Spiteri
To all, I am brand new to writing functions for PostgreSQL and understand that this question has been asked one hundred and one times. Please be patient with me, I do learn quickly. My question is: I wish to create a an aggregate function and have read and re-read the 34.10 section of the 8

Re: [HACKERS] Logging conflicted queries on deadlocks

2008-02-28 Thread Alvaro Herrera
ITAGAKI Takahiro wrote: > I wrote a small patch to log conflicted queries. > With my patch, deadlock message will change as following: > > | ERROR: deadlock detected > | DETAIL: Process 3468 waits for ShareLock on transaction 451; blocked by > process 4068. > | STATEMENT: UPDATE test S

Re: [HACKERS] OSSP can be used in the windows environment now!

2008-02-28 Thread Dave Page
On Thu, Feb 28, 2008 at 12:22 PM, Magnus Hagander <[EMAIL PROTECTED]> wrote: > > >If not, what do other people think about adding this Makefile and a > > >README file to our contrib directory? > > > > If there is no contrary opinion, I believe that it is help to many people's > > Comments from

Re: [HACKERS] OSSP can be used in the windows environment now!

2008-02-28 Thread Magnus Hagander
On Thu, Feb 28, 2008 at 09:38:00AM +0900, Hiroshi Saito wrote: > Hi. > > - Original Message - > From: "Magnus Hagander" <[EMAIL PROTECTED]> > > > >I take it you are in contact with them, since you helped them with the > >port? Can you ask them if they are interested in distributing tha

Re: [HACKERS] Proposal: wildcards in pg_service.conf

2008-02-28 Thread Albe Laurenz
Tom Lane wrote: >> I'd like to extend the libpq service file by allowing >> wildcards, e.g. like this: > >> [%] >> host=dbhost.mycompany.com >> dbname=% > >> Such an entry would match all service parameters, >> and all ocurrences of the wildcard right of a = would >> be replaced with the service

Re: [HACKERS] UUID data format 4x-4x-4x-4x-4x-4x-4x-4x

2008-02-28 Thread Jochem van Dieten
On Thu, Feb 28, 2008 at 1:19 AM, Tom Lane wrote: > I think the question we have to answer is whether we want to be > complicit in the spreading of a nonstandard UUID format. I don't. I have patched the UUID input and output functions to be compatible with Adobe ColdFusion (http://adobe.com/prod

Re: [HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread Heikki Linnakangas
Pavan Deolasee wrote: On Thu, Feb 28, 2008 at 3:05 PM, Florian G. Pflug <[EMAIL PROTECTED]> wrote: If that works, then we might also want to set the visibility hint bits. Oh yes. Especially because random time-scattered index scans on the table can actually generate multiple writes of a page

Re: [HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread Pavan Deolasee
On Thu, Feb 28, 2008 at 3:25 PM, ITAGAKI Takahiro <[EMAIL PROTECTED]> wrote: > > > Sounds cool. I recommended users to do VACUUM FREEZE just after initial > loading, but we can avoid it with your method. > Yeah, and the additional step of VACUUM FREEZE adds up to the restore time. > > To make

Re: [HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread Pavan Deolasee
On Thu, Feb 28, 2008 at 3:05 PM, Florian G. Pflug <[EMAIL PROTECTED]> wrote: > > If that works, then we might also want to set the visibility hint bits. Oh yes. Especially because random time-scattered index scans on the table can actually generate multiple writes of a page of a read-only table.

Re: [HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread ITAGAKI Takahiro
"Pavan Deolasee" <[EMAIL PROTECTED]> wrote: > In a typical scenario, user might create a table and load data in the table as > part of a single transaction (e.g pg_restore). In this case, it would help if > we > create the tuples in the *frozen* state to avoid any wrap-around related > issues >

Re: [HACKERS] Varlena Type Creation

2008-02-28 Thread Dimitri Fontaine
Le mercredi 27 février 2008, Martijn van Oosterhout a écrit : > I see no-one responded to this: a varlena has no fixed header size, so > you can't fit it in a structure anyway. Once you're passed a pointer > you use the LEN/PTR macros to extract what you want. Once the type exists and the code get

Re: [HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread Florian G. Pflug
Pavan Deolasee wrote: In a typical scenario, user might create a table and load data in the table as part of a single transaction (e.g pg_restore). In this case, it would help if we create the tuples in the *frozen* state to avoid any wrap-around related issues with the table. Without this, very

[HACKERS] CREATE TABLE, load and freezing

2008-02-28 Thread Pavan Deolasee
I had this idea sometime back. Not sure if this has been discussed before In a typical scenario, user might create a table and load data in the table as part of a single transaction (e.g pg_restore). In this case, it would help if we create the tuples in the *frozen* state to avoid any wrap-around

[HACKERS] Logging conflicted queries on deadlocks

2008-02-28 Thread ITAGAKI Takahiro
Hello, We see error logs something like following on deadlocks: | ERROR: deadlock detected | DETAIL: Process 1612 waits for ShareLock on transaction 407; blocked by process 3388. | Process 3388 waits for ShareLock on transaction 406; blocked by process 1612. | STATEMENT: UPDATE test