Hi all, First i've tryed to set the size of primaryKeys fetched per statement in _FrontBasePlugin to 500. I've gained abound 5%. Next i've tryed to use ERXJDBCAdopter with batchsize 1000, but got no benefit. Then I tryed connectionpooling, but ended with a lot of exceptions, because the initialisation of Connections got broken. I switched to ObjectStoreCoordinator and the performance got worse.
Multithreading did also slow down the application (With 2 Threads i got a better performance by about 3-5%, but with more threads the performance slows down by 5% with 10 Threads and 15% with 20 Threads). My solution: 1. Return the hole multithreading, ObjectStoreCoordinator. 2. Configure Testinstance on Productionsystem only to import the data. Database near Applikation got me to speed up. the 5 hours import is done in 5min. Its not the best for me, but with the latency and slow connection issues it's the best i can get. Thanks for all your hints. regards Tobias 2014-05-06 10:40 GMT+02:00 Patrick Middleton <[email protected]>: > We encountered this recently against MySQL 5.6.12. > (Not using Project Wonder) > The default primary key generation was invoking a stored procedure once > per row. Once per second, some sort of garbage collection would occur > inside the RDBMS and the stored procedure would take about 0.2s to complete > instead of about 0.005s. > > The fix is to create and cache all your new primary keys at once in > advance using something like > er.extensions.jdbc.ERXJDBCAdaptor.Channel.primaryKeysForNewRowsWithEntity(int > cnt, com.webobjects.eoaccess.EOEntity entity) > > then use something like > databaseContextNewPrimaryKey(EODatabaseContext dbCtxt, Object object, > EOEntity entity) > http://wocommunity.org/documents/javadoc/WebObjects/ > 5.4.2/com/webobjects/eoaccess/EODatabaseContext.Delegate. > html?is-external=true > > to fetch a candidate new key from your cache rather than make a database > round-trip to make one. > > > On 6 May 2014, at 08:20, Tobias Janz wrote: > > I forgott to replay to the list. >> >> the database is close to the server. I did not find the way to copy the >> Database from Mac OS Frontbase to CentOS Frontbase or the other way. I >> tryed to recover me lokal Database with a Backup created with >> CentOS-Frontbase, but it fails. >> >> So I think the best way for me will be the WRITE ALL export. >> >> I will take a look at this. >> >> Thanks >> >> Tobias >> >> >> 2014-05-06 7:58 GMT+02:00 Chuck Hill <[email protected]>: >> Hi Tobias, >> >> I missed the "Frontbase in the cloud on centOS” before. Is the app >> server not “close” to the database server? With latency like this, you are >> going to face production issues, too. That would be my first concern. >> >> Some ideas to consider: >> Import it locally, do a WRITE ALL export and import on the server >> Change to the binary UUID primary keys that are generated in the >> application >> ERXLongPrimaryKeyFactory >> Chuck >> >> >> On 2014-05-05, 10:28 PM, "Tobias Janz" wrote: >> >> Hi Chuck, >> >> thanks for your advice, but this approch i have allready tryed. I create >> a new editing context for each 100. Also i've tryed multiple threads with >> relevant success. >> >> When i do the import on my lokal database its much much faster. I think >> it's because of the latency. Is there a way to savely create primary key >> with multiple DB-connections in multiple Threads? And how do i get multiple >> database connections? >> >> regards Tobias >> >> >> 2014-05-06 1:32 GMT+02:00 Chuck Hill <[email protected]>: >> Are you creating the objects and saving them all at once? If so, that is >> your problem. Save them in batches of 500 or 1000 and it will go much >> quicker. You might want to try making a new editing context for each batch. >> >> >> Chuck >> >> >> On 2014-05-05, 4:29 PM, "Tobias Janz" wrote: >> >> Hi all, >> >> i need to import a csv-file with about 58.000 Lines und 60 Columns. I >> have 8 Entitys where the values will be stored. The conversion of a line to >> entitys takes almost no time. But the full prozess, from conversion until >> saveChanges is done, takes several hours (5-6h). After profiling it points >> me to the generating of primary keys. This part uses half of the time. >> >> I need to do the import several times. >> Do you see a way to improve the speed? >> I'm using WebObject 5.4.3 (with Wonder). My Development is on Mac. >> Database is a Frontbase in the cloud on centOS. >> >> Thanks for any advice. >> >> Tobias >> >> >> _______________________________________________ >> Do not post admin requests to the list. They will be ignored. >> Webobjects-dev mailing list ([email protected]) >> Help/Unsubscribe/Update your Subscription: >> https://lists.apple.com/mailman/options/webobjects- >> dev/patrick%40onestep.co.uk >> >> This email sent to [email protected] >> > > > This email, including any attachments, is confidential and intended solely > for the person or organisation to whom it is addressed. If you are not the > intended recipient you must not disseminate, distribute or copy any partof > this email nor take any action in reliance on it. > > If you have received this in error please notify the sender immediately by > email or phone +44 (0)1702 426400 and delete this email and any > attachments > from your system. > > Email transmission cannot be guaranteed to be secure or error-free as > information could be intercepted, corrupted, lost, destroyed, arrive > late or incomplete, or contain viruses. The sender therefore does not > accept liability for any errors or omissions in the contents of this > message which arise as a result of email transmission. If verification > is required please request a hard-copy version. > > OneStep Solutions LLP is registered in England and Wales under registration > number OC337173 and has its registered office at 44 The Pantiles, Tunbridge > Wells, Kent, TN2 5TN. > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Webobjects-dev mailing list ([email protected]) > Help/Unsubscribe/Update your Subscription: > https://lists.apple.com/mailman/options/webobjects- > dev/janz.tobias%40gmail.com > > This email sent to [email protected]
_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list ([email protected]) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [email protected]
