Re: [PERFORM] Distributed/Parallel Computing

Hi Jeff, These are bulk updates of GIS data and OLTP. For example, we are running some sqls to remove specific POIs those are intersecting with others, for such exercise we need to compare and remove the data form diffrent tables including the 20M data tables. Apart form these there are bulk sele

Re: [PERFORM] Distributed/Parallel Computing

On Mon, Oct 5, 2009 at 12:11 PM, Viji V Nair wrote: > Hi Team, > > This question may have asked many times previously also, but I could not > find a solution for this in any post. any help on the following will be > greatly appreciated. > > We have a PG DB with PostGIS functions. There are around

Re: Maybe OT, not sure Re: [PERFORM] Best suiting OS

On Mon, Oct 5, 2009 at 6:35 AM, Adam Tauno Williams wrote: >> Maybe - if the only thing the server is running is PostgreSQL. Show of >> hands - how many users who ONLY install PostgreSQL, and use a bare >> minimum OS install, choosing to not run any other software? Now, how >> many people ALSO run

Re: [PERFORM] Best suiting OS

Axel Rau wrote: > > Am 05.10.2009 um 20:06 schrieb Karl Denninger: > >> gjournal, no. ZFS has potential stability issues - I am VERY interested >> in it when those are resolved. It looks good on a test platform but I'm >> unwilling to run it in production; there are both reports of crashes and >>

Re: [PERFORM] Speed / Server

On Mon, Oct 5, 2009 at 7:30 AM, Nikolas Everett wrote: > >> But you should plan on partitioning to multiple db servers up front >> and save pain of conversion later on.  A dual socket motherboard with >> 16 to 32 SAS drives and a fast RAID controller is WAY cheaper than a >> similar machine with 4

Re: [PERFORM] Best suiting OS

> Claus Guttesen wrote: > > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=13001+0+current/freebsd-performance > > Not being particularly passionate about any OS, I've been intrigued by > the FreeBSD benchmarks.  However, management is reluctant to use boxes > which don't have heavily-advertised dec

Re: [PERFORM] Best suiting OS

Am 05.10.2009 um 20:06 schrieb Karl Denninger: gjournal, no. ZFS has potential stability issues - I am VERY interested in it when those are resolved. It looks good on a test platform but I'm unwilling to run it in production; there are both reports of crashes and I have been able to cra

Re: [PERFORM] Best suiting OS

Claus Guttesen wrote: > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=13001+0+current/freebsd-performance Not being particularly passionate about any OS, I've been intrigued by the FreeBSD benchmarks. However, management is reluctant to use boxes which don't have heavily-advertised decals on t

Re: [PERFORM] Best suiting OS

Scott Carey wrote: > On 10/5/09 11:15 AM, "Karl Denninger" wrote: > > >> I'm running the 9650s in most of my "busier" machines. Haven't tried a >> PERC card yet - its on my list. Most of my stuff is configured as RAID >> 1 although I have a couple of RAID 10 arrays in service; depending on >>

Re: [PERFORM] Query plan for NOT IN

Grzegorz Jaśkiewicz wrote: well, as a rule of thumb - unless you can't think of a default value of column - don't use nulls. So using nulls as default 'idunno' - is a bad practice, but everybody's opinion on that differ. I don't understand this point of view. The concept of null was introd

[PERFORM] Distributed/Parallel Computing

Hi Team, This question may have asked many times previously also, but I could not find a solution for this in any post. any help on the following will be greatly appreciated. We have a PG DB with PostGIS functions. There are around 100 tables in the DB and almost all the tables contains 1 million

Re: [PERFORM] Speed while runnning large transactions.

On Fri, 2 Oct 2009, Tom Lane wrote: The people who hollered loudest about this seemed to often have long-running read-only transactions in parallel with lots of short read-write transactions. Which makes sense if you think about it. Long-running read-only reports are quite common in DBA lan

Re: [PERFORM] Best suiting OS

On 10/5/09 11:15 AM, "Karl Denninger" wrote: > Scott Carey wrote: >> >> On 10/5/09 10:27 AM, "Karl Denninger" wrote: >> >> >>> I don't run the 3x series 3ware boards. If I recall correctly they're not >>> true coprocessor boards and rely on the host CPU. Those are always going to >>> be a

Re: [PERFORM] Best suiting OS

On Sun, 4 Oct 2009, Mark Mielke wrote: I can show you tickets where RedHat has specifically state they *will not* update the kernel to better support new hardware, for fear of breaking support for older hardware. There are two reasonable paths you'll find in the Open Source world, which mirr

Re: [PERFORM] Best suiting OS

Scott Carey wrote: > > On 10/5/09 10:27 AM, "Karl Denninger" wrote: > > >> I don't run the 3x series 3ware boards. If I recall correctly they're not >> true coprocessor boards and rely on the host CPU. Those are always going to >> be a lose compared to a true coprocessor with dedicated cache

Re: [PERFORM] Best suiting OS

Am 05.10.2009 um 19:42 schrieb Karl Denninger: I have not yet benchmarked FreeBSD 8.x - my production systems are all on FreeBSD 7.x at present. The improvement going there from 6.x was MASSIVE. 8.x is on my plate to start playing with in the next couple of months. Did you ever try gjou

Re: [PERFORM] Best suiting OS

Axel Rau wrote: > Am 05.10.2009 um 19:42 schrieb Karl Denninger: > >> I have not yet benchmarked FreeBSD 8.x - my production systems are >> all on FreeBSD 7.x at present. The improvement going there from 6.x >> was MASSIVE. 8.x is on my plate to start playing with in the next >> couple of months.

Re: [PERFORM] Best suiting OS

Claus Guttesen wrote: >> However, I have certainly seen some inefficiencies with Linux and large use >> of shared memory -- and I wouldn't be surprised if these problems don't >> exist on FreeBSD or OpenSolaris. >> > > This came on the freebsd-performance-list a few days ago. > > http://docs.

Re: [PERFORM] Best suiting OS

On 10/5/09 10:27 AM, "Karl Denninger" wrote: > Scott Carey wrote: >> >> >> On 10/3/09 7:35 PM, "Karl Denninger" >> wrote: >> >> >> >>> >>> I am a particular fan of FreeBSD, and in some benchmarking I did between it >>> and CentOS FreeBSD 7.x literally

Re: [PERFORM] Best suiting OS

> However, I have certainly seen some inefficiencies with Linux and large use > of shared memory -- and I wouldn't be surprised if these problems don't > exist on FreeBSD or OpenSolaris. This came on the freebsd-performance-list a few days ago. http://docs.freebsd.org/cgi/getmsg.cgi?fetch=13001+0

Re: [PERFORM] Best suiting OS

Scott Carey wrote: > On 10/3/09 7:35 PM, "Karl Denninger" wrote: > > >> I am a particular fan of FreeBSD, and in some benchmarking I did between it >> and CentOS FreeBSD 7.x literally wiped the floor with the CentOS release I >> tried on IDENTICAL hardware. >> I also like the 3ware raid copro

Re: [PERFORM] Best suiting OS

On 10/3/09 7:35 PM, "Karl Denninger" wrote: > Denis Lussier wrote: >> I'm a BSD license fan, but, I don't know much about *BSD otherwise (except >> that many advocates say it runs PG very nicely). >> >> >> >> On the Linux side, unless your a dweeb, go with a newer, popular & well >> suppo

Re: [PERFORM] Performance RAID 0

> -Mensaje original- > De: mange > > Hi, > I have a pretty small database on my home computer (~25Gb). I > have three 250Gb HDDs. > > My setup was 1 HDD for OS (Windows XP) and the other 2 HDD > in RAID 0 for postgre database. > Will I see any performance improvement if I instead h

Re: [PERFORM] Best suiting OS

Devrim GÜNDÜZ wrote: On Mon, 2009-10-05 at 12:07 +0200, Jean-Michel Pouré wrote: Go for Debian: * It is a free community, very active. Well, we need to state that this is not a unique feature. * It is guaranteed to be upgradable. Depends. I had lots of issues with upgrade process in the pa

Re: [PERFORM] Best suiting OS

Stefan Kaltenbrunner wrote: Devrim GÜNDÜZ wrote: On Mon, 2009-10-05 at 12:07 +0200, Jean-Michel Pouré wrote: Go for Debian: * It is a free community, very active. Well, we need to state that this is not a unique feature. * It is guaranteed to be upgradable. Depends. I had lots of issues w

Re: [PERFORM] Query plan for NOT IN

Matthew Wakeling writes: > Yes, that does work, but only because id is NOT NULL. I thought Postgres > 8.4 had had a load of these join types unified to make it less important > how the query is written? NOT IN is not easily optimizable because of its odd behavior in the presence of nulls. Use

Re: [PERFORM] Query plan for NOT IN

2009/10/5 Matthew Wakeling > > Yes, that does work, but only because id is NOT NULL. I thought Postgres > 8.4 had had a load of these join types unified to make it less important how > the query is written? > well, as a rule of thumb - unless you can't think of a default value of column - don't

Re: [PERFORM] Query plan for NOT IN

On Mon, 5 Oct 2009, Grzegorz Jaśkiewicz wrote: On Mon, Oct 5, 2009 at 2:52 PM, Matthew Wakeling wrote: Table "public.intermineobject"  Column |  Type   | Modifiers +-+---  object | text    |  id     | integer | not null  class  | text  

Re: [PERFORM] Query plan for NOT IN

On Mon, Oct 5, 2009 at 2:52 PM, Matthew Wakeling wrote: > > mnw21-modmine-r13features-copy=# select count(*) from project; > count > --- >10 > (1 row) > > mnw21-modmine-r13features-copy=# select count(*) from intermineobject; > count > -- > 26344616 > (1 row) > > mnw21-modmine-r

[PERFORM] Query plan for NOT IN

mnw21-modmine-r13features-copy=# select count(*) from project; count --- 10 (1 row) mnw21-modmine-r13features-copy=# select count(*) from intermineobject; count -- 26344616 (1 row) mnw21-modmine-r13features-copy=# \d intermineobject; Table "public.intermineobject" Column |

Re: [PERFORM][OT] Best suiting OS

Hi Jean-David, On Mon, 2009-10-05 at 15:37 +0200, Jean-David Beyer wrote: > Robert Haas wrote (in part): > > > Also, I'd just like to mention that vi is a much better editor than > > emacs. > > > That is not my impression. I have used vi from when it first came out (I > used ed before that) unti

Re: [PERFORM] Best suiting OS

On Mon, 2009-10-05 at 09:37 -0400, Jean-David Beyer wrote: > Robert Haas wrote (in part): > > Also, I'd just like to mention that vi is a much better editor than > > emacs. > That is not my impression. I have used vi from when it first came out (I > used ed before that) until about 1998 when I firs

Re: [PERFORM] Best suiting OS

Robert Haas wrote (in part): Also, I'd just like to mention that vi is a much better editor than emacs. That is not my impression. I have used vi from when it first came out (I used ed before that) until about 1998 when I first installed Linux on one of my machines and started using emacs. I f

Re: [PERFORM] Speed / Server

> But you should plan on partitioning to multiple db servers up front > and save pain of conversion later on. A dual socket motherboard with > 16 to 32 SAS drives and a fast RAID controller is WAY cheaper than a > similar machine with 4 to 8 sockets is gonna be. And if you gotta go > there anyway

Re: [PERFORM] Best suiting OS

On Mon, Oct 5, 2009 at 2:00 AM, Craig James wrote: > Fedora is a very nice project, but it's not suitable for production database > servers. The trick is to write such a kick-ass application that before the Fedora support window ends, the load has increased enough that it's time to upgrade the ha

Re: Maybe OT, not sure Re: [PERFORM] Best suiting OS

> Maybe - if the only thing the server is running is PostgreSQL. Show of > hands - how many users who ONLY install PostgreSQL, and use a bare > minimum OS install, choosing to not run any other software? Now, how > many people ALSO run things like PHP, and require software more > up-to-date tha

Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

On Mon, Oct 5, 2009 at 1:24 PM, Omar Kilani wrote: > > > I'm not really sure what the alternatives are -- it never really makes > sense to get the selectivity for thousands of items in the IN clause. > I've never seen a different plan for the same query against a DB with > that patch vs without -

Re: [PERFORM] Best suiting OS

On Sun, 2009-10-04 at 10:05 -0400, Mark Mielke wrote: > On 10/01/2009 03:44 PM, Denis Lussier wrote: > > I'm a BSD license fan, but, I don't know much about *BSD otherwise > > (except that many advocates say it runs PG very nicely). > > On the Linux side, unless your a dweeb, go with a newer, popu

Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

Robert, On Mon, Oct 5, 2009 at 11:01 PM, Robert Haas wrote: > On Sun, Oct 4, 2009 at 9:58 PM, Omar Kilani wrote: >> Hi Xia, >> >> Try this patch: >> >> http://treehou.se/~omar/postgresql-8.4.1-array_sel_hack.patch >> >> It's a hack, but it works for us. I think you're probably spending >> most o

Re: [PERFORM] Bad performance of SELECT ... where id IN (...)

On Sun, Oct 4, 2009 at 9:58 PM, Omar Kilani wrote: > Hi Xia, > > Try this patch: > > http://treehou.se/~omar/postgresql-8.4.1-array_sel_hack.patch > > It's a hack, but it works for us. I think you're probably spending > most of your query time planning, and this patch helps speed things up > 10x o

Re: [PERFORM] updating a row in a table with only one row

On Mon, Oct 5, 2009 at 5:17 AM, Michal Vitecek wrote: >  Could the problem be the HW RAID card? There's ServerRAID 8k with 256MB >  with write-back enabled. Could it be that its internal cache becomes >  full and all disk I/O operations are delayed until it writes all >  changes to hard drives?

Re: [PERFORM] Best suiting OS

On Mon, 2009-10-05 at 12:07 +0200, Jean-Michel Pouré wrote: > Go for Debian: > * It is a free community, very active. Well, we need to state that this is not a unique feature. > * It is guaranteed to be upgradable. Depends. I had lots of issues with upgrade process in the past -- but yeah, it is

Re: [PERFORM] Best suiting OS

On Sun, 2009-10-04 at 15:51 -0400, Mark Mielke wrote: > If somebody were to come to you with a *new* deployment request, what > would you recommend? Would you really recommend RHEL 5 *today*? Well, "I" would, and I do recommend people. RHEL5 is well-tested, and stable. Many hardware vendors supp

Re: [PERFORM] Best suiting OS

On Thu, 2009-10-01 at 15:16 +0530, S Arvind wrote: > What is the best Linux flavor for server which runs postgres > alone. The postgres must handle greater number of database around 200 > +. Performance on speed is the vital factor. > Is it FreeBSD, CentOS, Fedora, Redhat xxx?? Go for Debia

Re: [PERFORM] Best suiting OS

On Sun, 2009-10-04 at 15:51 -0400, Mark Mielke wrote: > How do you provide effective support for a kernel that has 3000 back > ported patches against it? This is again nonsense. Red Hat employs top kernel hackers. They do maintain vanilla kernel. It is not hard for Red Hat to maintain their "own

Re: [PERFORM] updating a row in a table with only one row

Robert Haas wrote: >On Fri, Oct 2, 2009 at 9:54 AM, Merlin Moncure wrote: >> On Fri, Oct 2, 2009 at 4:18 AM, Michal Vitecek wrote: >>>  Hello everyone, >>> >>>  I'm using PostgreSQL 8.3.8 running on a server with 2 Xeon CPUs, 4GB >>>  RAM, 4+2 disks in RAID 5 and CentOS 5.3. There's only one data

Re: [PERFORM] Best suiting OS

Scott Marlowe wrote: Personally, I use Fedora, and my servers have been quite stable. One of our main web servers running Fedora: It's not that there can't be stable releases of FC, it's that it's not the focus of that project. So, if you get lucky, great! I can't imagine running a production