Re: [PERFORM] long transaction

2008-08-12 Thread Sabin Coanda
long running transactions can be evil. is there a reason why this has to run in a single transaction? This single transaction is used to import new information in a database. I need it because the database cannot be disconected from the users, and the whole new data has to be consistently.

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Mario Weilguni
Valentin Bogdanov schrieb: --- On Mon, 11/8/08, Gregory Stark [EMAIL PROTECTED] wrote: From: Gregory Stark [EMAIL PROTECTED] Subject: Re: [PERFORM] Using PK value as a String To: Jay [EMAIL PROTECTED] Cc: pgsql-performance@postgresql.org Date: Monday, 11 August, 2008, 10:30 AM Jay [EMAIL

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Jay D. Kang
You guys totally rock! I guess, bottom line, we should take that extra day to convert our PK and FK to a numerical value, using BIG INT to be on the save side. (Even though Wikipedia's UserID uses just an integer as data type) To Gregory: Thank you for you valuable statement. But the real

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Gregory Stark
Mario Weilguni [EMAIL PROTECTED] writes: UUID is already a surrogate key not a natural key, in no aspect better than a numeric key, just taking a lot more space. So why not use int4/int8? The main reason to use UUID instead of sequences is if you want to be able to generate unique values

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Bill Moran
In response to Gregory Stark [EMAIL PROTECTED]: Mario Weilguni [EMAIL PROTECTED] writes: UUID is already a surrogate key not a natural key, in no aspect better than a numeric key, just taking a lot more space. So why not use int4/int8? The main reason to use UUID instead of

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Mark Mielke
Bill Moran wrote: The main reason to use UUID instead of sequences is if you want to be able to generate unique values across multiple systems. So, for example, if you want to be able to send these userids to another system which is taking registrations from lots of places. Of course that only

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Gregory Stark
Mark Mielke [EMAIL PROTECTED] writes: - Increased keyspace. Even if keyspace allocation is performed, an int4 only has 32-bit of keyspace to allocate. The IPv4 address space is already over 85% allocated as an example of how this can happen. 128-bits has a LOT more keyspace than 32-bits

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Mathias Stjernström
Hi! We use normal sequences to generate id's across multiple nodes. We use the increment parameter for the sequence and we specify each node to increment its sequence with for example 10 and the the first node to start the sequence at 1 and the second at 2 and so on. In that way you get

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Mark Mielke
Gregory Stark wrote: Mark Mielke [EMAIL PROTECTED] writes: - Increased keyspace. Even if keyspace allocation is performed, an int4 only has 32-bit of keyspace to allocate. The IPv4 address space is already over 85% allocated as an example of how this can happen. 128-bits has a LOT more

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Merlin Moncure
On Tue, Aug 12, 2008 at 9:46 AM, Gregory Stark [EMAIL PROTECTED] wrote: Mark Mielke [EMAIL PROTECTED] writes: - Increased keyspace. Even if keyspace allocation is performed, an int4 only has 32-bit of keyspace to allocate. The IPv4 address space is already over 85% allocated as an

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Moritz Onken
We chose UUID as PK because there is still some information in an integer key. You can see if a user has registered before someone else (user1.id user2.id) or you can see how many new users registered in a specific period of time (compare the id of the newest user to the id a week ago).

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Bill Moran
In response to Moritz Onken [EMAIL PROTECTED]: We chose UUID as PK because there is still some information in an integer key. You can see if a user has registered before someone else (user1.id user2.id) or you can see how many new users registered in a specific period of time

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Moritz Onken
Am 12.08.2008 um 17:04 schrieb Bill Moran: In response to Moritz Onken [EMAIL PROTECTED]: We chose UUID as PK because there is still some information in an integer key. You can see if a user has registered before someone else (user1.id user2.id) or you can see how many new users registered

Re: [PERFORM] Distant mirroring

2008-08-12 Thread dforums
Tx to all. I reach the same reflection on partitionning the data to those tables. And postgresql is giving very good tools for that with the rules features. I got the SAS server for immediate fix. But I'm looking for buying a machine that will handle my needs for more long time. Regarding

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Bill Moran
In response to Moritz Onken [EMAIL PROTECTED]: Am 12.08.2008 um 17:04 schrieb Bill Moran: In response to Moritz Onken [EMAIL PROTECTED]: We chose UUID as PK because there is still some information in an integer key. You can see if a user has registered before someone else (user1.id

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Moritz Onken
Am 12.08.2008 um 17:21 schrieb Bill Moran: In response to Moritz Onken [EMAIL PROTECTED]: Am 12.08.2008 um 17:04 schrieb Bill Moran: In response to Moritz Onken [EMAIL PROTECTED]: We chose UUID as PK because there is still some information in an integer key. You can see if a user has

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Bill Moran
In response to Steve Atkins [EMAIL PROTECTED]: On Aug 12, 2008, at 8:21 AM, Bill Moran wrote: In response to Moritz Onken [EMAIL PROTECTED]: Am 12.08.2008 um 17:04 schrieb Bill Moran: In response to Moritz Onken [EMAIL PROTECTED]: We chose UUID as PK because there is still

Re: [PERFORM] Using PK value as a String

2008-08-12 Thread Mark Mielke
Bill Moran wrote: We chose UUID as PK because there is still some information in an integer key. You can see if a user has registered before someone else (user1.id user2.id) or you can see how many new users registered in a specific period of time (compare the id of the newest user to the

Re: [PERFORM] long transaction

2008-08-12 Thread Merlin Moncure
On Tue, Aug 12, 2008 at 4:17 AM, Sabin Coanda [EMAIL PROTECTED] wrote: long running transactions can be evil. is there a reason why this has to run in a single transaction? This single transaction is used to import new information in a database. I need it because the database cannot be

Re: [PERFORM] Filesystem benchmarking for pg 8.3.3 server

2008-08-12 Thread Henrik
Hi again all, Just wanted to give you an update. Talked to Dell tech support and they recommended using write- through(!) caching in RAID10 configuration. Well, it didn't work and got even worse performance. Anyone have an estimated what a RAID10 on 4 15k SAS disks should generate in

Re: [PERFORM] Filesystem benchmarking for pg 8.3.3 server

2008-08-12 Thread Scott Marlowe
On Tue, Aug 12, 2008 at 1:40 PM, Henrik [EMAIL PROTECTED] wrote: Hi again all, Just wanted to give you an update. Talked to Dell tech support and they recommended using write-through(!) caching in RAID10 configuration. Well, it didn't work and got even worse performance. Someone at Dell

Re: [PERFORM] Filesystem benchmarking for pg 8.3.3 server

2008-08-12 Thread Ron Mayer
Greg Smith wrote: some write cache in the SATA disks...Since all non-battery backed caches need to get turned off for reliable database use, you might want to double-check that on the controller that's driving the SATA disks. Is this really true? Doesn't the ATA FLUSH CACHE command (say,

[PERFORM] Incorrect estimates on correlated filters

2008-08-12 Thread Chris Kratz
Hello All, Ran into a re-occuring performance problem with some report queries again today. In a nutshell, we have filters on either multiple joined tables, or multiple columns on a single table that are highly correlated. So, the estimates come out grossly incorrect (the planner has no way to

Re: [PERFORM] Filesystem benchmarking for pg 8.3.3 server

2008-08-12 Thread Scott Carey
Some SATA drives were known to not flush their cache when told to. Some file systems don't know about this (UFS, older linux kernels, etc). So yes, if your OS / File System / Controller card combo properly sends the write cache flush command, and the drive is not a flawed one, all is well. Most

Re: [PERFORM] Filesystem benchmarking for pg 8.3.3 server

2008-08-12 Thread Ron Mayer
Scott Carey wrote: Some SATA drives were known to not flush their cache when told to. Can you name one? The ATA commands seem pretty clear on the matter, and ISTM most of the reports of these issues came from before Linux had write-barrier support. I've yet to hear of a drive with the

Re: [PERFORM] Filesystem benchmarking for pg 8.3.3 server

2008-08-12 Thread Scott Marlowe
On Tue, Aug 12, 2008 at 6:23 PM, Scott Carey [EMAIL PROTECTED] wrote: Some SATA drives were known to not flush their cache when told to. Some file systems don't know about this (UFS, older linux kernels, etc). So yes, if your OS / File System / Controller card combo properly sends the write

Re: [PERFORM] Filesystem benchmarking for pg 8.3.3 server

2008-08-12 Thread david
On Tue, 12 Aug 2008, Ron Mayer wrote: Scott Carey wrote: Some SATA drives were known to not flush their cache when told to. Can you name one? The ATA commands seem pretty clear on the matter, and ISTM most of the reports of these issues came from before Linux had write-barrier support. I

Re: [PERFORM] Filesystem benchmarking for pg 8.3.3 server

2008-08-12 Thread Scott Carey
I'm not an expert on which and where -- its been a while since I was exposed to the issue. From what I've read in a few places over time ( storagereview.com, linux and windows patches or knowledge base articles), it happens from time to time. Drives usually get firmware updates quickly. Drivers

Re: [PERFORM] Filesystem benchmarking for pg 8.3.3 server

2008-08-12 Thread Ron Mayer
Scott Marlowe wrote: I can attest to the 2.4 kernel not being able to guarantee fsync on IDE drives. Sure. But note that it won't for SCSI either; since AFAICT the write barrier support was implemented at the same time for both. -- Sent via pgsql-performance mailing list