Hello, I'm trying to perform massive insertion of data into my database (v. 1.7.2). I have a class Location and one of it's properties locationId. Firstly I've added something about 50k vertexes of Location class (Lets call them 'CITIES'). Then I've added a unique index for this class: CREATE INDEX Location.locationId ON Location (locationId) unique
After that I'm starting adding 600k new Location class vertexes (lets call them ZIP_CODES). As we have rather limited system resources, we decided to add these postal codes by 50k batches one after another, each in separate java process. Zip codes depends from cities. There should be an edge between them, so index for locationId is required. Each 50k Locations also have 500k dependent vertexes and edges total linked to them. Each batch make orient db to consume additional 300-400mb memory which it doesn't seem to return :(. When memory consumption is near to it's limit (I allocated 3GB for OrientDB), I start getting a very surprising error (sometimes it is a batch 200-250k or 250-300k): APPLOG: 2014-07-01 16:07:31,885 ERROR [com.efinancialcareers.locations. export.job.ExportLocationsJob] - <Exception in thread "main" com. orientechnologies.orient.core.exception.OConcurrentModificationException: Cannot UPDATE the record #12:289185 because the version is not the latest. Probably you are updating an old record or it has been modified by another user (db=v2 your=v0)> APPLOG: 2014-07-01 16:07:31,885 ERROR [com.efinancialcareers.locations. export.job.ExportLocationsJob] - < at com.orientechnologies.orient.core. storage.impl.local.paginated.OLocalPaginatedStorage.updateRecord( OLocalPaginatedStorage.java:818)> APPLOG: 2014-07-01 16:07:31,885 ERROR [com.efinancialcareers.locations. export.job.ExportLocationsJob] - < at com.orientechnologies.orient.core. storage.impl.local.paginated.OLocalPaginatedStorage.commitEntry( OLocalPaginatedStorage.java:2110)> APPLOG: 2014-07-01 16:07:31,885 ERROR [com.efinancialcareers.locations. export.job.ExportLocationsJob] - < at com.orientechnologies.orient.core. storage.impl.local.paginated.OLocalPaginatedStorage.commit( OLocalPaginatedStorage.java:1099)> APPLOG: 2014-07-01 16:07:31,885 ERROR [com.efinancialcareers.locations. export.job.ExportLocationsJob] - < at com.orientechnologies.orient.core. tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:132)> APPLOG: 2014-07-01 16:07:31,885 ERROR [com.efinancialcareers.locations. export.job.ExportLocationsJob] - < at com.orientechnologies.orient.core. tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:105)> *There is no other thread which could change any record. Moreover I'm doing this all in scope of a single transaction. So I have no idea why this error appears. Please advise me something...* I've looked through the https://code.google.com/p/orient/wiki/TroubleshootingJava and https://code.google.com/p/orient/wiki/GraphDatabaseRaw#ConcurrencyGraphDB and before accessing the graph I'm disabling mvcc. I'm also trying to improve memory consumption and speedup the initialization process using next command: graph.getRawGraph() .setValidationEnabled(false) .setRetainRecords(false) .setMVCC(false) .declareIntent(new OIntentMassiveInsert()); However I can't even be sure that it helps! Should these settings be applied to client side app to the server itself? -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
