Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-05-03 Thread Markus Schaber
Hi, Josh, Josh Berkus wrote: > Yes, actually. We need 3 different estimation methods: > 1 for tables where we can sample a large % of pages (say, >= 0.1) > 1 for tables where we sample a small % of pages but are "easily estimated" > 1 for tables which are not easily estimated by we can't afford

Re: [PERFORM] batch inserts are "slow"

2005-05-03 Thread Markus Schaber
Hi, all, David Parker wrote: > We ran into the need to use COPY, but our application is also in Java. > We wrote a JNI bridge to a C++ routine that uses the libpq library to do > the COPY. The coding is a little bit weird, but not too complicated - > the biggest pain in the neck is probably gettin

Re: [PERFORM] [postgis-users] Is my query planner failing me, or vice versa?

2005-12-15 Thread Markus Schaber
Hi, Gregory, Gregory S. Williamson wrote: > Forgive the cross-posting, but I found myself wondering if might not > be some way future way of telling the planner that a given table > (column ?) has a high likelyhood of being TOASTed. Similar to the > random_page_cost in spirit. We've got a lot of i

[PERFORM] Read only transactions - Commit or Rollback

2005-12-20 Thread Markus Schaber
performance-wise, if it does make any difference at all: Having autocommit on or off? (I presume "off") Using commit or rollback? Committing / rolling back occasionally (e. G. when returning the connection to the pool) or not at all (until the pool closes the connection)? Thanks, Markus

Re: [PERFORM] Read only transactions - Commit or Rollback

2005-12-20 Thread Markus Schaber
might make a difference: Having ACID-Level SERIALIZABLE or READ COMMITED? Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org -

Re: [PERFORM] Read only transactions - Commit or Rollback

2005-12-20 Thread Markus Schaber
eemed to be 30% faster, which surprised us. That's why I ask this questions, and mainly because we currently cannot perform a large bunch of benchmarking. Thanks, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fi

Re: [PERFORM] Read only transactions - Commit or Rollback

2005-12-20 Thread Markus Schaber
VACUUM can't reclaim dead rows). Okay, so I'll stick with my current behaviour (Autocommit off and ROLLBACK after each bunch of work). Thanks, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against

Re: [PERFORM] MySQL is faster than PgSQL but a large margin in

2005-12-22 Thread Markus Schaber
database concurrently. Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)--- TIP 6: explain ana

Re: [PERFORM] Invulnerable VACUUM process thrashing everything

2006-01-05 Thread Markus Schaber
tions to propagate to the disks. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [PERFORM] What's the best hardver for PostgreSQL 8.1?

2006-01-05 Thread Markus Schaber
d and two writes. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)--- TIP 2: Don&#x

Re: [PERFORM] [SQL] two queryes in a single tablescan

2007-10-20 Thread Markus Schaber
esults in a faster way. PostgreSQL 8.3 contains great improvements in this area, you can simply start the selects from concurrent connections, and the backend will synchronize the scans. Regards, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Softwar

Re: [PERFORM] new to postgres (and db management) and performance

2006-01-17 Thread Markus Schaber
AND leaving the db fully operable and accessible? > > You're not doing regular vacuums often enough. It may also help to increase the max_fsm_pages setting, so postmaster has more memory to remember freed pages between VACUUMs. HTH, Markus -- Markus Schaber | Logical Tracking

Re: [PERFORM] new to postgres (and db management) and performance

2006-01-17 Thread Markus Schaber
s indices as well as the fact that a table can consist of multiple files. Also, I believe that newer PostgreSQL versions allow VACUUM to truncate files when free pages happen to appear at the very end of the file.) HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl

Re: [PERFORM] Autovacuum / full vacuum

2006-01-17 Thread Markus Schaber
hen allocates new pages for the rest. Then VACUUM comes and refills the FSM, however, the FSM does not contain enough free space for the next large bulk update. The same is for deletes and large bulk inserts, btw. So your table keeps growing steadily, until VACUUM FULL or CLUSTER comes along to clean

Re: [PERFORM] Autovacuum / full vacuum

2006-01-17 Thread Markus Schaber
s get slower when files get bigger or there are more files, but this effect should not really be noticeable here. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.

Re: [PERFORM] Use of Stored Procedures and

2006-01-19 Thread Markus Schaber
gt; understand how make it. Which interface are you using to connect to PostgreSQL? libpq, libpqxx, pgjdbc, python-popy? E. G. PGJDBC handles prepared plans transparently by using the PreparedStatement class. If you use command line PSQL, you can use the PREPARE commands. Markus -- Markus Schaber

Re: [PERFORM] Use of Stored Procedures and

2006-01-19 Thread Markus Schaber
you can use the PREPARE commands. > > I'm using the adodb to call the stored procedure (plpgsql). So your statements are inside a plpgsql stored procedure, important to know that. AFAIK, plpgsql uses prepared statements internally, so it should not be necessary to use them explicitly.

Re: [PERFORM] [PERFORMANCE] Stored Procedures

2006-01-23 Thread Markus Schaber
ly should consider using a connection pool (most web application servers provide pooling facilities) or some other means to keep the connection between several http requests. Worried, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS

Re: [PERFORM] Where is my bottleneck?

2006-02-02 Thread Markus Schaber
ur > cpu is idle, your i/o isn't fast enough. Or the query is misoptimized (low work_mem, missing indices) and cause much more I/O than necessary. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS F

Re: [PERFORM] partitioning and locking problems

2006-02-02 Thread Markus Schaber
n't start any #1 that won't finish before it's time for #3. You could also use the LOCK command on an empty lock table at the beginning of each #1 or #3 transaction to prevent #3 from getting the view lock before #1 is finished. HTH, Markus -- Markus Schaber | Logical Trackin

Re: [PERFORM] Default autovacuum settings too conservative

2006-02-07 Thread Markus Schaber
are a lot of code to hack, but although I'm not a postgresql core developer, I am keen enough to invite you to send patches. :-) Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www

Re: [PERFORM] Default autovacuum settings too conservative

2006-02-08 Thread Markus Schaber
hould be possible to divide this full table sweep into smaller chunks, each of them in its own transaction. It will certainly be necessary to block e. G. simultaneous VACUUMs, CLUSTERs or other maintainance commands for the whole VACUUM run, but normal SELECT, INSERT and UPDATE statement shoul

Re: [PERFORM] Default autovacuum settings too conservative

2006-02-08 Thread Markus Schaber
your often do range queries on an index that does not correspond to the insertion order, you may benefit from CLUSTERing on that index from time to time. Hth, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against so

Re: [PERFORM] Default autovacuum settings too conservative

2006-02-08 Thread Markus Schaber
Hi, Tim, Tim Allen schrieb: >> I don't know what "70 lac" means. > One lac (also spelt "lakh") is one hundred thousand. And one crore is > ten million. Indians count differently from the rest of the world :-). Okay, so he talks about 7 million rows. Thank you. Markus --

Re: [PERFORM] Large Database Design Help

2006-02-10 Thread Markus Schaber
very slow, so temporarily dropping the indices will not be useful if you merge the tables. Btw, I don't know whether PostgreSQL can make use of partial indices when building other partial indices. If yes, you could temporarily drop all but one of the partial indices for a specific client.

Re: [PERFORM] Large Database Design Help

2006-02-10 Thread Markus Schaber
IMHO. > ( the current database is PG 7.4 - I intend to upgrade it to 8.1 if and > when I do this refactoring ) This is a very good idea, 8.1 is miles ahead of 7.4 in many aspects. > ( the current OS is Debian Unstable but I intend to be running RHEL 4.0 > if and when I do this refac

Re: [PERFORM] Basic Database Performance

2006-02-10 Thread Markus Schaber
al performance tuning tips here...) > We are running PG 7.4 on a Debian Sarge server, and will be upgrading to > pg8.0 on a new server, but have some migration issues (that’s for > another list!) Ignore 8.0 and go to 8.1 directly. HTH, Markus -- Markus Schaber | Logical Tracking&a

Re: [PERFORM] 10+hrs vs 15min because of just one index

2006-02-12 Thread Markus Schaber
umeric or decimal may be even better. If most of your data is different in the first 8 bytes, it may also make sense to duplicate them into a bigint, and create the bigint on them. Then you can use AND in your query to test for the 8 bytes (uses index) and the bytea. Ugly, but may work. HTH, Markus -

Re: [PERFORM] help required in design of database

2006-02-13 Thread Markus Schaber
sulation, no external app should see the change inside their transaction. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(

Re: [PERFORM] copy and postgresql.conf

2006-02-15 Thread Markus Schaber
t this time can be improved by changing postgresql configuration > file. > But which parameters i need to manipulate and with which values ? Increase the size of the wal. If its just a develpoment environment, or you don't mind data inconsistency in case of a crash, disable fsync. HTH

Re: qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Markus Schaber
you give us some pointers (names, URLs, papers) to such algorithms? Thanks a lot, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org --

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Markus Schaber
Hi, Mark, Mark Lewis schrieb: > It seems that instead of maintaining a different sorting code path for > each data type, you could get away with one generic path and one > (hopefully faster) path if you allowed data types to optionally support > a 'sortKey' interface by providing a function f whi

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Markus Schaber
Hi, PFC, PFC schrieb: > By the way, I'd like to declare my zipcode columns as SQL_ASCII > while the rest of my database is in UNICODE, so they are faster to > index and sort. Come on, MySQL does it... Another use case for parametric column definitions - charset definitions - and the first

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-17 Thread Markus Schaber
Hi, David, David Lang schrieb: >> In SQL_ASCII, just take the first 4 characters (or 8, if using a 64-bit >> sortKey as elsewhere suggested). The sorting key doesn't need to be a >> one-to-one mapping. > that would violate your second contraint ( f(a)==f(b) iff (a==b) ) no, it doesn't. When b

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-17 Thread Markus Schaber
Hi, Ron, Ron schrieb: > OK, so here's _a_ way (there are others) to obtain a mapping such that > if a < b then f(a) < f (b) and > if a == b then f(a) == f(b) > > Pretend each row is a integer of row size (so a 2KB row becomes a 16Kb > integer; a 4KB row becomes a 32Kb integer; etc) > Since eve

Re: [PERFORM] Created Index is not used

2006-02-23 Thread Markus Schaber
ion(); > "PostgreSQL 7.4.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.3.3 > (SuSE Linux)" Btw, you should update to 7.4.12, there are importand bug fixes and it is upgradable "in place", without dumping and reloading the database. HTH, Markus -- Markus S

Re: [PERFORM] Looking for a tool to "*" pg tables as ERDs

2006-02-23 Thread Markus Schaber
Hi, Ron, Ron Peacetree wrote: > Where "*" == > {print | save to PDF | save to format | display on screen} > > Anyone know of one? psql with fancy output formatting comes to my mind, or "COPY table TO file" SQL command. HTH, Markus -- Markus Schaber | Logica

Re: [PERFORM] count(*) performance

2006-03-28 Thread Markus Schaber
Warning if FSM is not high enough, maybe you can find useful hints in the log file. HTH Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoft

Re: [PERFORM] simple join uses indexes, very slow

2006-03-28 Thread Markus Schaber
Hi, George, george young wrote: >>Looks like a hash join might be faster. What is your work_mem set to? > > work_mem= 1024 This is 1 Megabyte. By all means, increase it, if possible. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf

Re: [PERFORM] Massive Inserts Strategies

2006-03-28 Thread Markus Schaber
atabase. INSERT INTO original_table (col1, col2, col3) SELECT col1, col2, col3 FROM temp_table WHERE ... HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org

Re: [PERFORM] Slow performance on Windows .NET and OleDb

2006-03-29 Thread Markus Schaber
not, that's the reason why it's so slow, every transaction sync()s through to the disk. Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org

Re: [PERFORM] Decide between Postgresql and Mysql (help of

2006-03-30 Thread Markus Schaber
es static content, or http://www.tntnet.org/ Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end

Re: [PERFORM] Index scan startup time

2006-03-30 Thread Markus Schaber
uery more often) a conditional index on activity_id where state in (1,10001). Btw, PostgreSQL 8.1 could AND two bitmap index scans on the activity and state indices, and get the result faster (i presume). Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | So

Re: [PERFORM] Sun Fire T2000 and PostgreSQL 8.1.3

2006-04-06 Thread Markus Schaber
customers, the more CPUs are utilized. But beware, if you have floating point maths, it will be very slow. All 8 CPUs / 32 Threads share a single FPU. So if you need floating point (e. G. Mapserver, PostGIS geoprocessing, Java2D chart drawing or something), T2000 is not the right thing for you.

Re: [PERFORM] Indexes with descending date columns

2006-04-11 Thread Markus Schaber
columns. Just a thought, Markus [1] It may make sense to implement the mixed specifiers on indices as well, to allow CLUSTERing on mixed search order. [2] But I admit that I currently don't have enough knowledge in PostgreSQL index scan internals to know whether it really is easy to imple

Re: [PERFORM] Better index stategy for many fields with few values

2006-04-12 Thread Markus Schaber
expect query planning time to get rather significant for so much columns, so gequo tuning, tuning work_mem (for the bitmap scans) and prepared statements will pay off. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software

Re: [PERFORM] FOREIGN KEYS vs PERFORMANCE

2006-04-12 Thread Markus Schaber
ome connector that connects / synchronizes to another software, or a data import tool), database based constraints cannot be broken opposed to application based ones. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Developmen

Re: [PERFORM] Better index stategy for many fields with few values

2006-04-13 Thread Markus Schaber
ter than a multi-column index on 20 columns of character(10), if only because it is approx. 1/25th in size. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.f

Re: [PERFORM] Better index stategy for many fields with few values

2006-04-13 Thread Markus Schaber
A functional index may also make sense to CLUSTER the table to optimize the locality of search results (and so reducing disk I/O). In case your table has low write activity, but high read-only activity, the overhead that comes with the additional index is neglible compared to the performance

Re: [PERFORM] Better index stategy for many fields with few values

2006-04-13 Thread Markus Schaber
indices on different disks / raid arrays, to parallelize writes. Btw, I guess you have multiple, concurrent users? Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii

Re: [PERFORM] pgmemcache

2006-04-13 Thread Markus Schaber
it infrastructure for them. Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)--- T

Re: [PERFORM] Inserts optimization?

2006-04-18 Thread Markus Schaber
at has successfully finished a commit can be shure their data is on the platters.[1] HTH, Markus [1] As long as the platters don't lie, but that's another subject. -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against so

Re: [PERFORM] merge>hash>loop

2006-04-18 Thread Markus Schaber
s a crude way of partially compensating for this basic problem. I totall agree with this, it's just what we did here from time to time. :-) Hmm, how does effective_cach_size correspond with it? Shouldn't a high effective_cache_size have a similar effect? Thanks, Markus -- Markus Schaber |

Re: [PERFORM] Inserts optimization?

2006-04-19 Thread Markus Schaber
g extra delays. Well, if you read the documentation, you will see that it will only wait if there are at least commit_siblings other transactions active. So when Bacula serializes access, there will be no delays, as there is only a single transaction alive. HTH Markus -- Markus Schaber | Lo

Re: [PERFORM] Inserts optimization?

2006-04-19 Thread Markus Schaber
oject-du-jour are > "dumbed down" to the mysql featureset. (And not just mysql, but > mysql-lowest-common-factors, which means myisam etc) Well, most of those projects don't need a database, they need a bunch of tables and a lock. Heck, they even use client-side SELECT-loops

Re: [PERFORM] Quick Performance Poll

2006-04-20 Thread Markus Schaber
to a read-only table for now. Are they capable to index custom datatypes like the PostGIS geometries that use the GIST mechanism? This could probably speed up our Geo Databases for Map rendering, containing static data that is updated approx. 2 times per year. Markus -- Markus Schaber | Logical Tr

Re: [PERFORM] Introducing a new linux readahead framework

2006-04-21 Thread Markus Schaber
hink that this is an easy, understandable and useful interpretation of posix_fadvise() hints. Are there any rough estimates when this will get into mainline kernel (if you intend to submit)? Thanks, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. |

Re: [PERFORM] Recovery will take 10 hours

2006-04-24 Thread Markus Schaber
will try to access a not-completely-restored wal file. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---

Re: [PERFORM] how unsafe (or worst scenarios) when setting fsync

2006-04-28 Thread Markus Schaber
g with commit_delay and commit_siblins may improve your situation. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of

[PERFORM] Arrays and index scan

2006-04-28 Thread Markus Schaber
ter".flatten_array = streets.link_id) (4 rows) Currently, we're planning to use the array flattening approach, but are there any plans to enhance the query planner for the direct ARRAY approach? Thanks, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. I

Re: [PERFORM] Why so slow?

2006-05-02 Thread Markus Schaber
Additonally, the "free_space_map" setting has to be high enough, it has to cover enough space to put in all pages that get dead rows between two vacuum runs. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight

Re: [PERFORM] extremely slow when execute select/delete for certain

2006-05-08 Thread Markus Schaber
ur indices using REINDEX command. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)-

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-10 Thread Markus Schaber
e estimator functions have an implicit return parameter of int8. Parameters may be NULL when they are not known at query planning time. What do you think about this idea? The same scheme could be used to add a CPUCOST_ESTIMATOR to expensive functions. HTH, Markus -- Markus Schaber | Logical Track

Re: [PERFORM] [GENERAL] Arguments Pro/Contra Software Raid

2006-05-10 Thread Markus Schaber
SB/FireWire to IDE case, and an older linux cryptoloop implementations, IIRC). If you're interested, I can dig for the C source... HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software

Re: [PERFORM] [GENERAL] Arguments Pro/Contra Software Raid

2006-05-10 Thread Markus Schaber
), but not syncs/write rate. Both of them can be captured by the other script, but not by my tool. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatent

Re: [PERFORM] [GENERAL] Arguments Pro/Contra Software Raid

2006-05-10 Thread Markus Schaber
Hi, Bruce, Markus Schaber wrote: >>>It does not find as much liers as the script above, but it is less >>Why does it find fewer liers? > > It won't find liers that have a small "lie-queue-length" so their > internal buffers get full so they have to bloc

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-10 Thread Markus Schaber
E. G. for generate_series we return ($2-$1)/$3, and for some functions even constant estimates will be good enough. > - please execute me and store my results in a temporary storage, > count the rows returned, and plan the outer query accordingly That's an interesting idea.

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-10 Thread Markus Schaber
aps a better solution would be to cache the > result of the estimator function. Sophisticated estimator functions are free to use the pg_statistics views for their row count estimation. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Dev

Re: [PERFORM] Lot'sa joins - performance tip-up, please?

2006-05-18 Thread Markus Schaber
Hi, Mario, Mario Splivalo wrote: > This helps also. I don't get sequential scans any more. I'd like a tip > on how to set 'enable_nestloop = off' trough JDBC? statement.execute("SET enable_nestloop TO off"); should do. HTH, Markus -- Markus Schaber | Logic

Re: [PERFORM] How can I make this query faster (resend)

2006-05-22 Thread Markus Schaber
ctions, and then give them at least 30 megs of RAM each. This should also cut down the connection creation overhead. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! ww

Re: [PERFORM] Getting even more insert performance (250m+rows/day)

2006-05-29 Thread Markus Schaber
and input file to different spindles won't help you much. HTH Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---

Re: [PERFORM] lowering priority automatically at connection

2006-06-20 Thread Markus Schaber
a, most Datasource implementations (e. G. the JBoss one) allow to specify SQL statements that are executed on connection init. Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www

Re: [PERFORM] scaling up postgres

2006-06-21 Thread Markus Schaber
atabase is only 58M it's a read only DB and will lasts only for a > month. I guess it is a simple table with a single PK (some subscription numer) - no joins or other things. For this cases, a special non-RDBMS like MySQL, SQLite, or even some hancrafted thingy may give you better resu

Re: [PERFORM] scaling up postgres

2006-06-21 Thread Markus Schaber
2 G. This sounds like you want to have one postgresql backend per apache frontend. Did you try running pgpool on the Apache machine, and have only a few (hundred) connections to the backend? Maybe http://en.wikipedia.org/wiki/Memcached could be helpful, too. Markus -- Markus Schaber | Lo

Re: [PERFORM] Some queries starting to hang

2006-06-21 Thread Markus Schaber
le. Most of the cases when we had database bloat despite running autovacuum, it was due to a low free_space_map setting. Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosof

Re: [PERFORM] SAN performance mystery

2006-06-23 Thread Markus Schaber
e, and increase the WAL size / checkpoint segments. When most of the restore time was spent in index creation, increase the sort mem / maintainance work mem settings. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fig

Re: [PERFORM] SAN performance mystery

2006-06-23 Thread Markus Schaber
Hi, Tim, Seems I sent my message to fast, cut in middle of a sencence: Markus Schaber wrote: >> A pg_dump/pg_restore cycle reduced the total >> database size from 81G to 36G. > If you still have the original database around, ... can you check wether VACUUM FULL and REINDEX a

Re: [PERFORM] Help tuning autovacuum - seeing lots of relationbloat

2006-06-23 Thread Markus Schaber
when leaving it open while having lunch... Some older JDBC driver versions had the bug that they always had an open transaction, thus an application server having some pooled connections lingering around could block vacuum forever. Markus -- Markus Schaber | Logical Tracking&Tracing Inte

Re: [PERFORM] Help tuning autovacuum - seeing lots of relationbloat

2006-06-23 Thread Markus Schaber
s is pretty clearly in the 1% > it can't handle. Maybe your free space map is configured to small, can you watch out for log messages telling to increase it? HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight ag

Re: [PERFORM] Index Being Ignored?

2006-06-30 Thread Markus Schaber
dx ON purchase_order_items (expected_quantity) WHERE expected_quantity > 0; HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---

Re: [PERFORM] Is postgresql ca do the job for software deployed in

2006-07-05 Thread Markus Schaber
d from listing all schemas (or even all databases in the same user), regardless whether he/she has access rights. But it is not always acceptable that a customer knows which other customers one has. This forces the use of the "one cluster per customer" paradigm. Thanks, Markus -- Markus S

Re: [PERFORM] Is postgresql ca do the job for software deployed in

2006-07-05 Thread Markus Schaber
PostgreSQL) we use the one schema per customer paradigm quite successfully. > My $0.02 (not worth what it was) Oh, I think the're at least $0.03 cents worth. :-) Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight again

Re: [PERFORM] getting better performance

2006-07-07 Thread Markus Schaber
dropping and recreation. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)---

Re: [PERFORM] need vacuum after insert/truncate/insert?

2006-07-07 Thread Markus Schaber
ble, to have current statistics. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)-

Re: [PERFORM] Update INSERT RULE while running for Partitioning

2006-07-07 Thread Markus Schaber
the affected-row count > on the last rule which gives Hibernate problems. This could be considered a PostgreSQL bug - maybe you should discuss this on the appropriate list (general, hackers)? HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. |

Re: [PERFORM] Kill a session

2006-07-14 Thread Markus Schaber
is is for plpgsql, pljava and plpython? HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)

Re: [PERFORM] RAID stripe size question

2006-07-17 Thread Markus Schaber
e system.) HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [PERFORM] RAID stripe size question

2006-07-17 Thread Markus Schaber
'll definitely fiddle with that in the coming > tests as well. How many parallel transactions do you have? Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU!

Re: [PERFORM] BUG #2543: Performance delay acrros the same day

2006-07-24 Thread Markus Schaber
gh and/or don't have your FSM > setting high enough. Depending on the PostgreSQL version, it might also be that he suffers from index bloat. He might look into the manual pages about REINDEX for a description. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing Inter

Re: [PERFORM] loading increase into huge table with 50.000.000 records

2006-07-26 Thread Markus Schaber
Hi, Larry, Hi, Sven, Sven Geisler wrote: > You can increase wal_buffers, checkpoint_segments and checkpoint_timeout > much higher. You also should increase the free space map settings, it must be large enough to cope with your weekly bunch. Markus -- Markus Schaber | Logical Tr

Re: [PERFORM] Disk writes

2006-08-07 Thread Markus Schaber
[1] We once had such a problem because an ill-compiled kernel having USB verbose logging on... -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org --

Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig

2006-08-07 Thread Markus Schaber
'm used to with the Adaptec SCSI junk. Well, for sequential reading, you should be able to get double drive speed on a 2-disk mirror with a good controller, as it can balance the reads among the drives. Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf.

Re: [PERFORM] PostgreSQL scalability on Sun UltraSparc T1

2006-08-07 Thread Markus Schaber
se results for low concurrency situations). Different fsync method settings can also make a difference (I presume that syncing was enabled). HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against so

Re: [PERFORM] sub select performance due to seq scans

2006-08-07 Thread Markus Schaber
Hi, Scott and Hale, Scott Marlowe wrote: > Make sure analyze has been run and that the statistics are fairly > accurate. It might also help to increase the statistics_target on the column in question. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG

Re: [NOVICE] [PERFORM] 7.3.2 pg_restore very slow

2006-08-07 Thread Markus Schaber
g the kernel. As stated on the http://www.postgresql.org/docs/8.1/interactive/kernel-resources.html page, those values can be changed via sysctl or echoing values into /proc, under linux at least. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Sof

Re: [PERFORM] Beginner optimization questions, esp. regarding Tsearch2

2006-08-10 Thread Markus Schaber
ound writer, stats collector or autovacuum, depending on your version and configuration. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org

Re: [PERFORM] [BUGS] BUG #2567: High IOWAIT

2006-08-11 Thread Markus Schaber
settings might help you out. As far as I understand, this will only help for concurrent inserts by different clients, dealing throughput for latency. Please correct me if I'm wrong. HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Deve

Re: [PERFORM] Migrating data from DB2 to SQL Server

2006-08-11 Thread Markus Schaber
that. Thank you. It seems that you, by accident, hit the wrong list with your question. But, as you're here, why don't you migrate to PostgreSQL instead? Have a nice day, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development G

Re: [PERFORM] slow transfer speeds with PostgreSQL

2006-08-11 Thread Markus Schaber
rying to estimate future usage patterns). HTH, Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)-

Re: [PERFORM] Slow access to PostgreSQL server

2006-08-11 Thread Markus Schaber
ewall or antivirus network filtering software. HTH, Marku -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)--- TIP 6: explain analyze is your friend

  1   2   >