Re: [GENERAL] Large index operation crashes postgres

2010-05-02 Thread strk
On Mon, Mar 29, 2010 at 09:39:35AM +0200, Frans Hals wrote: > Paul, > > took your advice and installed Geos 3.2.0. > Index is now running for 14 hrs, postmaster is taking all the RAM. > Sadly it looks like the Geos update didn't save me. Frans, could you try installing GEOS 3.2.2 ? A leak was fix

Re: [GENERAL] Large index operation crashes postgres

2010-03-29 Thread Frans Hals
Paul, I have checked the different kinds of data in the table for their memory usage. ST_LineSting is the one that's leaking, the other types complete indexing without leakage. Update to Geos 3.2.0 didn't improve the operation. Kind regards Frans 2010/3/28 Paul Ramsey : > MIght be random, might

Re: [GENERAL] Large index operation crashes postgres

2010-03-29 Thread Frans Hals
Paul, took your advice and installed Geos 3.2.0. Index is now running for 14 hrs, postmaster is taking all the RAM. Sadly it looks like the Geos update didn't save me. Regards Frans 2010/3/28 Paul Ramsey : > GEOS 3.2 is backwards compatible, so you can install it overtop of 3.1 > and things will

Re: [GENERAL] Large index operation crashes postgres

2010-03-27 Thread Frans Hals
Paul, do you know, if Postgis 1.3.6 will survive, if I install Geos 3.2 over it? Currently, there's Geos 3.1.1 installed. I remember, I had to downgrade Postgis/Geos to be compatible with Postgres 8.3.9. As I "herited" the database from a 8.3.9 dump, I needed to fit my installation before restori

Re: [GENERAL] Large index operation crashes postgres

2010-03-27 Thread Frans Hals
Paul, I kindly received the information about the table data (quoting here): >It changes as it goes down the table, it's a right mixture. ST_LineString |2 | 5398548 ST_LineString |3 | 2877681 ST_LineString |4 | 2160809 ST_LineString |

Re: [GENERAL] Large index operation crashes postgres

2010-03-26 Thread Frans Hals
The index mentioned below has been created in some minutes without problems. Dropped it and created it again. Uses around 36 % of memorywhile creating, after completion postmaster stays at 26 %. > I'm not sure, what you're thinking about generating a self-contained > test that exhibits similar bl

Re: [GENERAL] Large index operation crashes postgres

2010-03-26 Thread Frans Hals
Tom, I'm pretty new to memory debugging, so please be patient if I'm not as precise as you suppose me to be. For the records I have run a valgrind postmaster session, starting my initial indexing routine until it crashes postgres. If you think this might be enlightning for you, I'll send you the t

Re: [GENERAL] Large index operation crashes postgres

2010-03-26 Thread Paul Ramsey
Occams razor says it's PostGIS. However, I'm concerned about how old the code being run is. In particular, the library underneath PostGIS, GEOS, had a *lot* of memory work done on it over the last year. I'd like to see if things improve if you upgrade to GEOS 3.2. On Fri, Mar 26, 2010 at 9:04 AM,

Re: [GENERAL] Large index operation crashes postgres

2010-03-26 Thread Tom Lane
Frans Hals writes: > Operation is now running for around 13 hrs. > Two postmaster processes above 1% memory usage are running. > One of them uses constantly 26.5% of memory. > The other one is growing: > After 1 hr25% > After 9 hrs 59% > After 13 hrs64% Well, it's pretty clear t

Re: [GENERAL] Large index operation crashes postgres

2010-03-26 Thread Frans Hals
Operation is now running for around 13 hrs. Two postmaster processes above 1% memory usage are running. One of them uses constantly 26.5% of memory. The other one is growing: After 1 hr25% After 9 hrs 59% After 13 hrs64% Thanks & regards Frans 2010/3/25 Frans Hals : > Paul, > >

Re: [GENERAL] Large index operation crashes postgres

2010-03-25 Thread Frans Hals
Paul, I have started the operation right now after a fresh reboot of the machine. Please give me some time to collect the results... Thanks & regards Frans 2010/3/25 Paul Ramsey : > If you build an index, or try to update a column, using just the > > ST_X(ST_Centroid()) > > without all the ext

Re: [GENERAL] Large index operation crashes postgres

2010-03-25 Thread Frans Hals
Tom, ran a CREATE INDEX to the gdb operated postmaster. Nothing new due to missing debugging libraries, so this might not help: Reading symbols from /usr/bin/postmaster...(no debugging symbols found)...done. Missing separate debuginfos, use: debuginfo-install postgresql-server-8.3.9-1PGDG.f12.x86

Re: [GENERAL] Large index operation crashes postgres

2010-03-24 Thread Frans Hals
5 x postmaster taking memory: 93.3 % 18.7 % 0.3 % 0.2 % 0.0 % 112.5% Looks like there is someone living beyond its means? Frans 2010/3/24 Tom Lane : > Hm.  I wonder about a memory leak in there somewhere.  Have you checked > the process size while this is going on? > >            

Re: [GENERAL] Large index operation crashes postgres

2010-03-24 Thread Paul Ramsey
Can you do? alter table placex add column geometry_sector integer; update placex set geometry_sector = geometry_sector(geometry); P. On Wed, Mar 24, 2010 at 1:15 PM, Frans Hals wrote: > Hi, > > running a geo-database from a dump restore where still one of the most > important indexes is missing

Re: [GENERAL] Large index operation crashes postgres

2010-03-24 Thread Tom Lane
Frans Hals writes: > Whenever I try to add the follwing index to the table "placex", one of > the postmaster processes dies and the server restarts. Can you provide a stack trace from the crash? > I try: > CREATE INDEX idx_placex_sector ON placex USING btree > (geometry_sector(geometry), rank_ad

[GENERAL] Large index operation crashes postgres

2010-03-24 Thread Frans Hals
Hi, running a geo-database from a dump restore where still one of the most important indexes is missing and so the search is slow. Whenever I try to add the follwing index to the table "placex", one of the postmaster processes dies and the server restarts. I try: CREATE INDEX idx_placex_sector ON