Hi Rich, The problem with Tree Bonsai is that is not ready to run on distributed architecture yet (we have an open issue on that). So you should set "ridBag.embeddedToSbtreeBonsaiThreshold" to Integer.MAX_VALUE to avoid using it.
Best Regards, Luca Garulli Founder & CEO OrientDB <http://orientdb.com/> On 18 August 2015 at 11:06, Richard Jones <[email protected]> wrote: > Hello, > > Quite a long problem, apologies. > > I've got a two-node distribution setup, my database has data (imported > through ETL offline and synched to the other node when ODB is started), but > I need to import daily updates (deltas) to that data, and can't get any way > to work. Here are the ways I've tried to import a delta of 273 changes > (each of which will update multiple vertices and edges). The import is a > bit messy, there are less than 100 changes that actually need to be made, > with lots of duplicates and changes being made and undone sequentially, so > it's quite understandable that there's a transactional problem, but I would > hope there's a way to overcome it. > > Firstly, I tried an ETL import to a remote:localhost:2424/database URL > (online), writeQuorum 2, tx true and batchCommit 1000 (defaults) - I get a > writeQuorum failure in the ETL output, and the server log for the node that > I'm not running ETL on has this : > > Error on updating record #11:390 (cluster: > com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster@365e3fb7 > ) > java.lang.NullPointerException > at > com.orientechnologies.orient.core.index.sbtreebonsai.local.OSBTreeBonsaiLocal.load(OSBTreeBonsaiLocal.java:455) > at > com.orientechnologies.orient.core.db.record.ridbag.sbtree.OSBTreeCollectionManagerShared.loadTree(OSBTreeCollectionManagerShared.java:95) > at > com.orientechnologies.orient.core.db.record.ridbag.sbtree.OSBTreeCollectionManagerAbstract.loadSBTree(OSBTreeCollectionManagerAbstract.java:98) > at > com.orientechnologies.orient.core.db.record.ridbag.sbtree.OSBTreeCollectionManagerProxy.loadSBTree(OSBTreeCollectionManagerProxy.java:52) > at > com.orientechnologies.orient.core.storage.impl.local.paginated.ORidBagUpdateSerializationOperation.loadTree(ORidBagUpdateSerializationOperation.java:73) > at > com.orientechnologies.orient.core.storage.impl.local.paginated.ORidBagUpdateSerializationOperation.execute(ORidBagUpdateSerializationOperation.java:54) > at > com.orientechnologies.orient.core.storage.impl.local.paginated.ORecordSerializationContext.executeOperations(ORecordSerializationContext.java:99) > at > com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doUpdateRecord(OAbstractPaginatedStorage.java:1567) > at > com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.updateRecord(OAbstractPaginatedStorage.java:728) > at > com.orientechnologies.orient.server.distributed.ODistributedStorage.updateRecord(ODistributedStorage.java:599) > at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.executeSaveRecord(ODatabaseDocumentTx.java:1757) > at > com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:102) > at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:2316) > at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:2150) > at > com.orientechnologies.orient.server.distributed.task.OUpdateRecordTask.execute(OUpdateRecordTask.java:95) > at > com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.executeOnLocalNode(OHazelcastPlugin.java:753) > at > com.orientechnologies.orient.server.hazelcast.ODistributedWorker.onMessage(ODistributedWorker.java:298) > at > com.orientechnologies.orient.server.hazelcast.ODistributedWorker.run(ODistributedWorker.java:121) > > Which appears to be trying to get a page out of cache. Any suggestions as > to what might be the problem here? I was going to set the embedded/OSB > threshold very high to force everything to be embedded edges, but according > to the various warnings (e.g. > http://orientdb.com/docs/last/Concurrency.html#concurrency-on-adding-edges) > when in distributed mode the edges are always embedded so this has no > effect. I have to wonder why the OSBTreeBonsai code is being executed, but > I must misunderstand what it's being used for. Or maybe it's because I > imported the initial dataset offline, so the edges were created as OSBTree > before the database was somehow flagged as distributed? This may be the > case because if I import the delta into an empty database (so there's been > no offline ETL import), then I get a different exception > (OWOWCache.java:865, see below). > > The end result of this import is that I have one vertex (11:390 mentioned > in the error) with different versions (out by one) on each node, so the > database is unusable and I don't know how to fix it. For each different > method of importing the DB I reset to the offline-import before trying > another way. > > I then tried to shut down one ODB server and perform an offline ETL of the > delta. This works fine, but when I start up the server with the new data, > it then receives a copy of the (old) data from the other node, presumably > because the offline node is rejoining an active cluster and so needs to get > new data? NB - there are no inserts/updates (or even selects) happening > other then my ETL imports, so the node that was online has not received any > changes. How can I make the node that receives the offline ETL import the > 'master' so it syncs its database to the other node? > > One way I've managed to import the data is turn off one node, run the > online ETL on the remaining node, then bring the other node back online. > This causes the node that was offline to receive a copy of the updated > database. But this isn't ideal, as it's difficult to orchestrate and > automate, and it means I'll only have one DB node running (which will be > busy) for the period of the import. > > If I try to import with writeQuorum 1, then the import completes but on > the node I'm not running ETL on, the log contains amny errors like the > stack trace above, and also this : > > Error on updating record #55:10 (cluster: > com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster@6d66a58c > ) > java.lang.NullPointerException > at > com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache.fileNameById(OWOWCache.java:865) > at > com.orientechnologies.orient.core.index.hashindex.local.cache.OReadWriteDiskCache.fileNameById(OReadWriteDiskCache.java:236) > at > com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperation.fileNameById(OAtomicOperation.java:281) > at > com.orientechnologies.orient.core.db.record.ridbag.sbtree.OSBTreeCollectionManagerShared.loadTree(OSBTreeCollectionManagerShared.java:89) > at > com.orientechnologies.orient.core.db.record.ridbag.sbtree.OSBTreeCollectionManagerAbstract.loadSBTree(OSBTreeCollectionManagerAbstract.java:98) > at > com.orientechnologies.orient.core.db.record.ridbag.sbtree.OSBTreeCollectionManagerProxy.loadSBTree(OSBTreeCollectionManagerProxy.java:52) > at > com.orientechnologies.orient.core.storage.impl.local.paginated.ORidBagUpdateSerializationOperation.loadTree(ORidBagUpdateSerializationOperation.java:73) > at > com.orientechnologies.orient.core.storage.impl.local.paginated.ORidBagUpdateSerializationOperation.execute(ORidBagUpdateSerializationOperation.java:54) > at > com.orientechnologies.orient.core.storage.impl.local.paginated.ORecordSerializationContext.executeOperations(ORecordSerializationContext.java:99) > at > com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doUpdateRecord(OAbstractPaginatedStorage.java:1567) > at > com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.updateRecord(OAbstractPaginatedStorage.java:728) > at > com.orientechnologies.orient.server.distributed.ODistributedStorage.updateRecord(ODistributedStorage.java:599) > at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.executeSaveRecord(ODatabaseDocumentTx.java:1757) > at > com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:102) > at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:2316) > at > com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:2150) > at > com.orientechnologies.orient.server.distributed.task.OUpdateRecordTask.execute(OUpdateRecordTask.java:95) > at > com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.executeOnLocalNode(OHazelcastPlugin.java:753) > at > com.orientechnologies.orient.server.hazelcast.ODistributedWorker.onMessage(ODistributedWorker.java:298) > at > com.orientechnologies.orient.server.hazelcast.ODistributedWorker.run(ODistributedWorker.java:121) > > The database is then (understandably) completely out of sync and unusable. > > I also get the OSBTreeBonsaiLocal.java:455 stack trace if I have a > database with writeQuorum 2, and an ETL import with tx true and batchCommit > 1, which I would expect to be slow but safe. > > I'm a bit stuck at the moment, so unless anyone has any suggestions on the > NullPointerExceptions or on how to allow the node I have run the offline > ETL import on to sync its data to the other node, then I can't really see a > way past this problem. Is ETL the wrong tool for this problem? Should I be > batching up inserts and running them via another client API? > > I can try to provide a minimal test case if anyone is interested and the > stack traces are not enough. > > Thanks, > Rich > > -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
