Hi Michael!! I am encountering the same problems while loading a large file into the orientdb, I am very tempted to try the no indexing option now as well, I am using version 2.2.1 of community. The indexes are automatically made when I load the data, can you please tell me how you disabled the automatic indexes to load the data?
this is the error message i got: : com.orientechnologies.common.directmemory.ODirectMemoryViolationException: Negative offset was provided On Sunday, 20 October 2013 05:03:20 UTC+2, Michael Campbell wrote: > > With no indexes the test passes to completion to 384000 records. > > So again, this appears to be an index problem (and not, apparently, an > Object API one). > > On Sat, Oct 19, 2013 at 7:44 PM, Michael Campbell > <[email protected] <javascript:>> wrote: > > With the latest code (from your link), it still dies in the same spot, > > but with yet another error. Going to try with NO indexes and the > > latest code. > > > > I'm just not having any luck with automatic indexes =( > > > > 52000, 982.725, 52.914, 53.282, 6274.32, 13.54 > > Exception in thread "main" > > > com.orientechnologies.common.directmemory.ODirectMemoryViolationException: > > Negative offset was provided > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > > > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > > > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > > at > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:508) > > > > at > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:461) > > > > at > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:146) > > > > at > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:57) > > > > at > com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1894) > > > > at > com.orientechnologies.orient.client.remote.OStorageRemote.createRecord(OStorageRemote.java:328) > > > > at > com.orientechnologies.orient.client.remote.OStorageRemoteThread.createRecord(OStorageRemoteThread.java:224) > > > > at > com.orientechnologies.orient.core.db.raw.ODatabaseRaw.save(ODatabaseRaw.java:275) > > > > at > com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeSaveRecord(ODatabaseRecordAbstract.java:801) > > > > at > com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:77) > > > > at > com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:270) > > > > at > com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:38) > > > > at > com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.save(ODatabaseRecordWrapperAbstract.java:285) > > > > at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:277) > > > > at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:226) > > > > at com.example.spike.orientdb.Driver.save(Driver.java:213) > > at com.example.spike.orientdb.Driver.run(Driver.java:179) > > at com.example.spike.orientdb.Driver.main(Driver.java:67) > > > > On Sat, Oct 19, 2013 at 7:18 PM, Michael Campbell > > <[email protected] <javascript:>> wrote: > >> I am using the 1.6.0-SNAPSHOT you sent me yesterday, not the maven > version. > >> > >> That said... this isn't unique to the Object API. I have rewritten my > >> test using a contrived DAO which creates a document of my object (at > >> all nested levels), and I get an error at the same approximate > >> location, although this code does run much faster than the Object API > >> method. > >> > >> This seems to be an index issue. I am going to run without indexes > >> and see what happens there. > >> > >> 52000, 973.103, 53.437, 54.621, 6212.89, 13.54 > >> Exception in thread "main" > >> com.orientechnologies.orient.core.exception.ODatabaseException: Error > >> on saving record in cluster #14 > >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > >> at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > > >> at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > > >> at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > >> at > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:506) > > > >> at > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:461) > > > >> at > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:146) > > > >> at > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:57) > > > >> at > com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1894) > > > >> at > com.orientechnologies.orient.client.remote.OStorageRemote.createRecord(OStorageRemote.java:328) > > > >> at > com.orientechnologies.orient.client.remote.OStorageRemoteThread.createRecord(OStorageRemoteThread.java:224) > > > >> at > com.orientechnologies.orient.core.db.raw.ODatabaseRaw.save(ODatabaseRaw.java:275) > > > >> at > com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeSaveRecord(ODatabaseRecordAbstract.java:801) > > > >> at > com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:77) > > > >> at > com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:270) > > > >> at > com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:38) > > > >> at > com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.save(ODatabaseRecordWrapperAbstract.java:285) > > > >> at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:277) > > > >> at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:226) > > > >> at com.example.spike.orientdb.Driver.save(Driver.java:213) > >> at com.example.spike.orientdb.Driver.run(Driver.java:179) > >> at com.example.spike.orientdb.Driver.main(Driver.java:67) > >> Caused by: java.lang.ClassCastException: > >> com.orientechnologies.orient.core.type.tree.OMVRBTreeRIDSet cannot be > >> cast to > com.orientechnologies.orient.core.db.record.ridset.sbtree.OSBTreeIndexRIDContainer > > > >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > >> at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > > >> at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > > >> at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > >> at > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:508) > > > >> ... 17 more > >> > >> > >> > >> On Sat, Oct 19, 2013 at 3:13 PM, Andrey Lomakin > >> <[email protected] <javascript:>> wrote: > >>> Hi Michael, > >>> We do not know ). > >>> > >>> But could you use latest distribution from CI server > >>> > http://helios.orientechnologies.com/job/orient-git-maven/lastStableBuild/com.orientechnologies$orientdb/artifact/com.orientechnologies/orientdb/1.6.0-SNAPSHOT/orientdb-1.6.0-SNAPSHOT-distribution.zip > > >>> that is link to last stable build. > >>> > >>> Maven repository is bit out of dated because of caching of artefacts. > >>> > >>> > >>> On Sat, Oct 19, 2013 at 8:00 PM, Michael Campbell > >>> <[email protected] <javascript:>> wrote: > >>>> > >>>> Before you spend a LOT of time on this, is this all related to the > >>>> ObjectAPI or do we know? I may go with a document API and see what > >>>> that does. > >>>> > >>>> On Sat, Oct 19, 2013 at 12:59 PM, Michael Campbell > >>>> <[email protected] <javascript:>> wrote: > >>>> > The same as you sent - I ran with the parameters 100, 100, 20. > >>>> > Although I mentioned in a later mail that when I ran again, I got a > >>>> > different error but in the same (approximate) place. I delete and > >>>> > re-create the database between each test. > >>>> > > >>>> > On Sat, Oct 19, 2013 at 11:29 AM, Andrey Lomakin > >>>> > <[email protected] <javascript:>> wrote: > >>>> >> Hi Michael, > >>>> >> Can I reproduce all errors by the test which you sent me or I > need > >>>> >> another > >>>> >> one ? > >>>> >> > >>>> >> > >>>> >> On Sat, Oct 19, 2013 at 5:34 PM, Michael Campbell > >>>> >> <[email protected] <javascript:>> wrote: > >>>> >>> > >>>> >>> And now a different error yet! > >>>> >>> > >>>> >>> 52000, 2353.358, 22.096, 22.030, 15025.29, 13.54 > >>>> >>> Exception in thread "main" > >>>> >>> com.orientechnologies.orient.core.exception.ODatabaseException: > Error > >>>> >>> on saving record in cluster #14 > >>>> >>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>>> >>> Method) > >>>> >>> at > >>>> >>> > >>>> >>> > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > > >>>> >>> at > >>>> >>> > >>>> >>> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > > >>>> >>> at > java.lang.reflect.Constructor.newInstance(Constructor.java:513) > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:506) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:461) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:146) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:57) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1894) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.client.remote.OStorageRemote.createRecord(OStorageRemote.java:328) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.client.remote.OStorageRemoteThread.createRecord(OStorageRemoteThread.java:224) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.core.db.raw.ODatabaseRaw.save(ODatabaseRaw.java:275) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeSaveRecord(ODatabaseRecordAbstract.java:801) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:77) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:270) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:38) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.save(ODatabaseRecordWrapperAbstract.java:285) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:400) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:425) > > > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:337) > > > >>>> >>> at com.example.spike.orientdb.Driver.save(Driver.java:202) > >>>> >>> at com.example.spike.orientdb.Driver.run(Driver.java:176) > >>>> >>> at com.example.spike.orientdb.Driver.main(Driver.java:68) > >>>> >>> Caused by: java.lang.ArrayIndexOutOfBoundsException: 2490368 > >>>> >>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>>> >>> Method) > >>>> >>> at > >>>> >>> > >>>> >>> > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > > >>>> >>> at > >>>> >>> > >>>> >>> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > > >>>> >>> at > java.lang.reflect.Constructor.newInstance(Constructor.java:513) > >>>> >>> at > >>>> >>> > >>>> >>> > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:508) > > > >>>> >>> ... 18 more > >>>> >>> > >>>> >>> On Sat, Oct 19, 2013 at 9:41 AM, Michael Campbell > >>>> >>> <[email protected] <javascript:>> wrote: > >>>> >>> > Still working on my big insert testing, and this time I have a > >>>> >>> > rather > >>>> >>> > large document with 3 levels of embedded lists. Reducing my > indexes > >>>> >>> > from 3 to 1 has seemed to fix the index issue, but now I get a > >>>> >>> > different error. > >>>> >>> > > >>>> >>> > For Andrey, this is using the test code I sent you with the > >>>> >>> > parameters: 100 100 20. On my machine it took about 40 minutes > to > >>>> >>> > get > >>>> >>> > to this point (2386 seconds) > >>>> >>> > > >>>> >>> > > >>>> >>> > 52000, 2386.152, 21.792, 21.857, 15234.66, 13.54 > >>>> >>> > > >>>> >>> > Exception in thread "main" > >>>> >>> > com.orientechnologies.orient.core.exception.ODatabaseException: > >>>> >>> > Error > >>>> >>> > on saving record in cluster #14 > >>>> >>> > at > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>>> >>> > Method) > >>>> >>> > at > >>>> >>> > > >>>> >>> > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > > >>>> >>> > at > java.lang.reflect.Constructor.newInstance(Constructor.java:513) > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:506) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:461) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:146) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:57) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1894) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.client.remote.OStorageRemote.createRecord(OStorageRemote.java:328) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.client.remote.OStorageRemoteThread.createRecord(OStorageRemoteThread.java:224) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.core.db.raw.ODatabaseRaw.save(ODatabaseRaw.java:275) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeSaveRecord(ODatabaseRecordAbstract.java:801) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:77) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:270) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:38) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.save(ODatabaseRecordWrapperAbstract.java:285) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:400) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:425) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:337) > > > >>>> >>> > at com.example.spike.orientdb.Driver.save(Driver.java:202) > >>>> >>> > at com.example.spike.orientdb.Driver.run(Driver.java:176) > >>>> >>> > at com.example.spike.orientdb.Driver.main(Driver.java:68) > >>>> >>> > Caused by: java.lang.ClassCastException: > >>>> >>> > com.orientechnologies.orient.core.type.tree.OMVRBTreeRIDSet > cannot > >>>> >>> > be > >>>> >>> > cast to > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.core.db.record.ridset.sbtree.OSBTreeIndexRIDContainer > > > >>>> >>> > at > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>>> >>> > Method) > >>>> >>> > at > >>>> >>> > > >>>> >>> > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > > >>>> >>> > at > >>>> >>> > > >>>> >>> > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > > >>>> >>> > at > java.lang.reflect.Constructor.newInstance(Constructor.java:513) > >>>> >>> > at > >>>> >>> > > >>>> >>> > > com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:508) > > > >>>> >>> > ... 18 more > >>>> >>> > >>>> >>> -- > >>>> >>> > >>>> >>> --- > >>>> >>> You received this message because you are subscribed to the > Google > >>>> >>> Groups > >>>> >>> "OrientDB" group. > >>>> >>> To unsubscribe from this group and stop receiving emails from it, > send > >>>> >>> an > >>>> >>> email to [email protected] <javascript:>. > >>>> >>> For more options, visit https://groups.google.com/groups/opt_out. > > >>>> >> > >>>> >> > >>>> >> > >>>> >> > >>>> >> -- > >>>> >> Best regards, > >>>> >> Andrey Lomakin. > >>>> >> > >>>> >> Orient Technologies > >>>> >> the Company behind OrientDB > >>>> >> > >>>> >> -- > >>>> >> > >>>> >> --- > >>>> >> You received this message because you are subscribed to the Google > >>>> >> Groups > >>>> >> "OrientDB" group. > >>>> >> To unsubscribe from this group and stop receiving emails from it, > send > >>>> >> an > >>>> >> email to [email protected] <javascript:>. > >>>> >> For more options, visit https://groups.google.com/groups/opt_out. > >>>> > >>>> -- > >>>> > >>>> --- > >>>> You received this message because you are subscribed to the Google > Groups > >>>> "OrientDB" group. > >>>> To unsubscribe from this group and stop receiving emails from it, > send an > >>>> email to [email protected] <javascript:>. > >>>> For more options, visit https://groups.google.com/groups/opt_out. > >>> > >>> > >>> > >>> > >>> -- > >>> Best regards, > >>> Andrey Lomakin. > >>> > >>> Orient Technologies > >>> the Company behind OrientDB > >>> > >>> -- > >>> > >>> --- > >>> You received this message because you are subscribed to the Google > Groups > >>> "OrientDB" group. > >>> To unsubscribe from this group and stop receiving emails from it, send > an > >>> email to [email protected] <javascript:>. > >>> For more options, visit https://groups.google.com/groups/opt_out. > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
