Re: [fpc-pascal] Firebird: bulk insert performance: suggestions?

2012-09-08 Thread patspiper
On 07/09/12 14:03, Reinier Olislagers wrote: For my Dutch postcode program https://bitbucket.org/reiniero/postcode with an embedded Firebird 2.5 database, I allow users to read in a CSV file with new or updated postcode data. I use sqldb, FPC x86. I'd like to get your suggestions on speed

Re: [fpc-pascal] Firebird: bulk insert performance: suggestions?

2012-09-08 Thread Reinier Olislagers
On 8-9-2012 11:04, patspiper wrote: On 07/09/12 14:03, Reinier Olislagers wrote: - Turning Forced Writes off while doing the bulk inserts may help (at a higher risk). But make sure you turn it back on afterwards. - If my memory serves well, the Firebird SP can directly read from the CSV

Re: [fpc-pascal] Firebird: bulk insert performance: suggestions?

2012-09-08 Thread Graeme Geldenhuys
On 08/09/12 06:31, Martin Schreiber wrote: It depends on the reaction time of the network because AFAIK there is a roundtrip for every inserted record... I didn't commit the transaction after every record. I first tried in a batch of 1 records per transaction. Later I even bumped that up

[fpc-pascal] Firebird: bulk insert performance: suggestions?

2012-09-07 Thread Reinier Olislagers
For my Dutch postcode program https://bitbucket.org/reiniero/postcode with an embedded Firebird 2.5 database, I allow users to read in a CSV file with new or updated postcode data. I use sqldb, FPC x86. I'd like to get your suggestions on speed improvements. I try to get the data into a

Re: [fpc-pascal] Firebird: bulk insert performance: suggestions?

2012-09-07 Thread michael . vancanneyt
On Fri, 7 Sep 2012, Reinier Olislagers wrote: For my Dutch postcode program https://bitbucket.org/reiniero/postcode with an embedded Firebird 2.5 database, I allow users to read in a CSV file with new or updated postcode data. I use sqldb, FPC x86. I'd like to get your suggestions on speed

Re: [fpc-pascal] Firebird: bulk insert performance: suggestions?

2012-09-07 Thread Ludo Brands
For my Dutch postcode program https://bitbucket.org/reiniero/postcode with an embedded Firebird 2.5 database, I allow users to read in a CSV file with new or updated postcode data. I use sqldb, FPC x86. I'd like to get your suggestions on speed improvements. Turn of indices when inserting

Re: [fpc-pascal] Firebird: bulk insert performance: suggestions?

2012-09-07 Thread Graeme Geldenhuys
On 07/09/12 12:12, michael.vancann...@wisa.be wrote: I once did tests with that (600.000 records) and did not notice any influence of the transaction control. Same here... I've imported 100's of thousands of records with SqlDB+Firebird with no serious speed issues. Also from CSV files.

Re: [fpc-pascal] Firebird: bulk insert performance: suggestions?

2012-09-07 Thread Martin Schreiber
On Saturday 08 September 2012 01:05:28 Graeme Geldenhuys wrote: On 07/09/12 12:12, michael.vancann...@wisa.be wrote: I once did tests with that (600.000 records) and did not notice any influence of the transaction control. Same here... I've imported 100's of thousands of records with