Hi guys, As Mateusz pointed the quorum settings should be to 2 and failureAvailableNodesLessQuorum = true to maintain consistency. Then I suggest you to try 1.7-SNAPSHOT where we fixed a few issues against distributed configuration.
About your database is it a plocal? The fact your logs show OMVRBTreeRIDSet usage let me think this is an old database created with older version of OrientDB, maybe with local? Lvc@ On 17 April 2014 18:10, Andrey Lomakin <[email protected]> wrote: > Hi guys, > Could you create issue about it so we will not forget to look on it ? > > > On Thu, Apr 17, 2014 at 6:10 PM, Daniel Hanelt [STARGAST Systems GmbH] < > [email protected]> wrote: > >> Hey, >> >> we are using the 1.7 RC2 but we've had the same problems with 1.6 >> >> You are right with the numbers of nodes, three would make more sense. >> But I doubt that this would solve the problems we've experienced, the >> exceptions started right away and there was no network issue at that time. >> >> Thanks >> Daniel >> >> Am 17.04.2014 um 03:34 schrieb "Mateusz Dymczyk" <[email protected]>: >> >> Which version are you using? I remember similar exceptions before >> switching to 1.7, seems they had some bugs. >> >> Also I don't know if Luca suggested exactly that setup but running such >> a DB with 2 nodes doesn't seem like a great idea - should you have a link >> (or well any sort of network error) between them or if one of them should >> go down for some time you will have inconsistent data (since both quorums >> are set to only 1). An odd number of servers with quorum (N/2)+1 is a >> better idea. >> >> Mateusz >> >> On Wednesday, April 16, 2014 9:22:31 PM UTC+9, Daniel wrote: >>> >>> Because we've had daily outages of 15 minutes due to the backup of the >>> DB, we follwed the advice of Luca to set up a distributed environment >>> with two nodes. >>> >>> Here is the config: >>> >>> >>> default-distributed-db-config.json >>> >>> { >>> "replication": true, >>> "autoDeploy": true, >>> "hotAlignment": true, >>> "resyncEvery": 15, >>> "clusters": { >>> "internal": { >>> "replication": false >>> }, >>> "index": { >>> "replication": false >>> }, >>> "ODistributedConflict": { >>> "replication": false >>> }, >>> "*": { >>> "replication": true, >>> "readQuorum": 1, >>> "writeQuorum": 1, >>> "failureAvailableNodesLessQuorum": false, >>> "readYourWrites": true, >>> "partitioning": { >>> "strategy": "round-robin", >>> "default": 0, >>> "partitions": [ >>> [ "<NEW_NODE>" ] >>> ] >>> } >>> } >>> } >>> } >>> >>> >>> So now here is what happended and finally led to a corrupted DB: >>> >>> >>> - Stopped the application server >>> - Setup and configured replication with the settings above >>> - Two nodes: node01 and node02 >>> - node02 had no existing database, so node01 exported and zipped the db >>> and sent it to node02 >>> - node02 extracted the db successfully, log message: INFO [node02] >>> installed database [OHazelcastPlugin] >>> - We started the application server >>> - After some time the following exception (sometimes) appeares in the >>> orient-server.log: >>> >>> Cannot route TX operation against distributed node >>> Error on committing distributed transaction >>> -> com.orientechnologies.orient.server.distributed. >>> ODistributedStorage.commit(ODistributedStorage.java:502) >>> -> com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit( >>> OTransactionOptimistic.java:109) >>> -> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.commit( >>> ODatabaseRecordTx.java:146) >>> -> com.orientechnologies.orient.core.db.document. >>> ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:440) >>> -> com.orientechnologies.orient.core.db.document. >>> ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:435) >>> -> com.orientechnologies.orient.server.network.protocol. >>> binary.ONetworkProtocolBinary.commit(ONetworkProtocolBinary.java:1253) >>> -> com.orientechnologies.orient.server.network.protocol. >>> binary.ONetworkProtocolBinary.executeRequest( >>> ONetworkProtocolBinary.java:325) >>> -> com.orientechnologies.orient.server.network.protocol.binary. >>> OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract >>> .java:126) >>> -> com.orientechnologies.common.thread.OSoftThread.run( >>> OSoftThread.java:45) >>> >>> >>> - After a while the following exception appeared in our application >>> with increased frequency: >>> >>> Caused by: >>> com.orientechnologies.orient.core.exception.OTransactionException: >>> Cannot insert item in mvrb-tree because the transactional item was not >>> found. >>> at com.orientechnologies.orient.core.type.tree.OMVRBTreeRID. >>> internalPut(OMVRBTreeRID.java:156) >>> at com.orientechnologies.orient.core.type.tree.OMVRBTreeRID. >>> internalPut(OMVRBTreeRID.java:57) >>> at com.orientechnologies.orient.core.type.tree. >>> OMVRBTreePersistent.put(OMVRBTreePersistent.java:468) >>> at com.orientechnologies.orient.core.type.tree.provider. >>> OMVRBTreeRIDProvider.lazyUnmarshall(OMVRBTreeRIDProvider.java:227) >>> at com.orientechnologies.orient.core.type.tree.OMVRBTreeRID. >>> getTreeSize(OMVRBTreeRID.java:332) >>> at com.orientechnologies.orient.core.type.tree.OMVRBTreeRID. >>> size(OMVRBTreeRID.java:318) >>> at com.orientechnologies.orient.core.type.tree. >>> OMVRBTreeRIDSet.size(OMVRBTreeRIDSet.java:91) >>> at com.orientechnologies.common.collection.OMultiValue. >>> getSize(OMultiValue.java:82) >>> at com.orientechnologies.orient.core.serialization.serializer. >>> record.string.ORecordSerializerSchemaAware2CSV.toString( >>> ORecordSerializerSchemaAware2CSV.java:165) >>> at com.orientechnologies.orient.core.serialization.serializer. >>> record.string.ORecordSerializerStringAbstract.toStream( >>> ORecordSerializerStringAbstract.java:92) >>> at com.orientechnologies.orient.core.serialization.serializer. >>> record.string.ORecordSerializerSchemaAware2CSV.toStream( >>> ORecordSerializerSchemaAware2CSV.java:518) >>> at com.orientechnologies.orient.core.record. >>> ORecordSchemaAwareAbstract.toStream(ORecordSchemaAwareAbstract.java:127) >>> at com.orientechnologies.orient.core.record. >>> ORecordSchemaAwareAbstract.toStream(ORecordSchemaAwareAbstract.java:122) >>> at com.orientechnologies.orient.core.record.impl.ODocument. >>> toStream(ODocument.java:391) >>> at com.orientechnologies.orient.client.remote.OStorageRemote. >>> commitEntry(OStorageRemote.java:1919) >>> ... 75 more >>> >>> >>> - Then we took a look at the corresponding dataset to those exception >>> via the orientdb console, i.e.: >>> >>> select from #12:155580 >>> >>> Error: >>> com.orientechnologies.orient.core.exception.OTransactionException: >>> Cannot insert item in mvrb-tree because the transactional item was not >>> found. >>> >>> >>> - Simple properties could be selected without problems, i.e.: >>> >>> select email from #12:155580 >>> ----+-----+--------- >>> # |@RID |email >>> ----+-----+--------- >>> 0 |#-2:1|<removed> >>> ----+-----+--------- >>> >>> >>> - Selections of linked Edges resulted sometimes in errors, i.e.: >>> >>> select out_Friend from #12:155580 >>> Error: >>> com.orientechnologies.orient.core.exception.OTransactionException: >>> Cannot insert item in mvrb-tree because the transactional item was not >>> found. >>> >>> >>> - Whereas others worked >>> >>> select in_Friend from #12:155580 >>> ----+-----+--------- >>> # |@RID |in_Friend >>> ----+-----+--------- >>> 0 |#-2:1|[63] >>> ----+-----+--------- >>> 1 item(s) found. Query executed in 0.004 sec(s). >>> >>> >>> Any idea or hint for us? >>> >>> Thanks a million >>> Daniel >>> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Best regards, > Andrey Lomakin. > > Orient Technologies > the Company behind OrientDB > > -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
