UPDATE can't be faster, because it load the current version of record to restore in case the quorum is not reached.
Lvc@ ᐧ On 25 September 2014 11:01, Luca Garulli <[email protected]> wrote: > Hey Ben, > This is also because your issue and reports. I'm looking what's wrong with > UPDATE right now... > > Lvc@ > > ᐧ > > On 24 September 2014 23:41, <[email protected]> wrote: > >> Many thanks to Luca for seriously looking into this matter. >> >> I can report that inserts and deletes are blazingly fast compared to what >> was previously. >> >> Only connects and updates seem to still lag behind. >> >> Following is an overview issuing commands over the binary interface: >> >> *dserver.sh* with *one* node up: >> >> connect: 0.002266 s >> insert: 0.000607 s >> update:0.000972 s >> delete: 0.000280 s >> >> *dserver.sh* with *two* nodes up: >> >> connect:* 0.118694 s* >> insert: *0.000548 s* <--- MASSIVE IMPROVEMENT >> update: *1.292313 s* >> delete: *0.000533 s* <--- MASSIVE IMPROVEMENT >> >> >> With these results I look very forward to the final release of OrientDB >> 2.0. Hopefully the connects and updates can be improved by then too. >> >> Regards, >> - Ben >> >> >> >> On Wednesday, September 24, 2014 8:08:38 PM UTC+2, Lvc@ wrote: >>> >>> Hi Stuart, >>> You're right. I've applied your fix, pushed, deployed on sonatype as >>> 2.0-M2-SNAPSHOT. >>> >>> Lvc@ >>> >>> ᐧ >>> >>> On 24 September 2014 18:17, Stuart McCulloch <[email protected]> wrote: >>> >>>> On 24 Sep 2014, at 16:31, Luca Garulli <[email protected]> wrote: >>>> >>>> Hi guys, >>>> I've some updates. Using an queue in the middle improved performance. >>>> Furthermore I've extended the asynchronous also to transactions. >>>> >>>> Please could you re-test last version 2.0-M2-SNAPSHOT? >>>> >>>> >>>> FYI, I get the following error using the latest snapshot in embedded >>>> mode with Java 7: >>>> >>>> java.lang.NoSuchMethodError: java.util.concurrent. >>>> ConcurrentHashMap.keySet()Ljava/util/concurrent/ >>>> ConcurrentHashMap$KeySetView; >>>> >>>> I assume this is because the snapshot was built using Java 8, which >>>> changed the return type of keySet for ConcurrentHashMap: >>>> >>>> http://docs.oracle.com/javase/8/docs/api/java/util/ >>>> concurrent/ConcurrentHashMap.html#keySet-- >>>> >>>> whereas in Java 7 it’s still the plain Set: >>>> >>>> http://docs.oracle.com/javase/7/docs/api/java/util/ >>>> concurrent/ConcurrentHashMap.html#keySet() >>>> >>>> Possible workarounds (assuming the plan is still to support Java 7) is >>>> either build with a version of Java before 8, or use Map or ConcurrentMap >>>> for the field rather than the concrete type: >>>> >>>> diff --git a/core/src/main/java/com/orientechnologies/orient/core/ >>>> config/OContextConfiguration.java b/core/src/main/java/com/ >>>> orientechnologies/orient/core/config/OContextConfiguration.java >>>> index d0dee2b..52c28b4 100644 >>>> --- a/core/src/main/java/com/orientechnologies/orient/core/ >>>> config/OContextConfiguration.java >>>> +++ b/core/src/main/java/com/orientechnologies/orient/core/ >>>> config/OContextConfiguration.java >>>> @@ -27,7 +27,7 @@ import java.util.concurrent.ConcurrentHashMap; >>>> * >>>> */ >>>> public class OContextConfiguration implements Serializable { >>>> - private final ConcurrentHashMap<String, Object> config = new >>>> ConcurrentHashMap<String, Object>(); >>>> + private final Map<String, Object> config = new >>>> ConcurrentHashMap<String, Object>(); >>>> >>>> /** >>>> * Empty constructor to create just a proxy for the >>>> OGlobalConfiguration. No values are setted. >>>> >>>> ( this is the place where the exception originates, there may be a >>>> couple of other fields using ConcurrentHashMap where they could just as >>>> well use Map or ConcurrentMap... ) >>>> >>>> — >>>> Cheers, Stuart >>>> >>>> Lvc@ >>>> >>>> On 24 September 2014 11:31, Luca Garulli <[email protected]> wrote: >>>> >>>>> Hi Ben, >>>>> I found a bottleneck in Hazelcast: https://github.com/ >>>>> hazelcast/hazelcast/issues/3661. >>>>> >>>>> I could put a queue between ODistributedStorage and HZ. Let me try it >>>>> now. Soon a new push. >>>>> >>>>> Lvc@ >>>>> >>>>> On 24 September 2014 10:59, <[email protected]> wrote: >>>>> >>>>>> Hi Luca, >>>>>> >>>>>> I tested this and now get 0.24sec average inserts, which is the >>>>>> fastest distributed performance I've experienced yet. >>>>>> >>>>>> There still seems to be communication with the remote node before >>>>>> releasing the client though. >>>>>> >>>>>> Regards, >>>>>> - Ben >>>>>> >>>>>> On Wednesday, September 24, 2014 1:24:23 AM UTC+2, Lvc@ wrote: >>>>>>> >>>>>>> Hi Ben, >>>>>>> I've pushed a new version that reads the configuration from file >>>>>>> *default-distributed-db-config.json*, by adding: >>>>>>> >>>>>>> * "executionMode": "asynchronous",* >>>>>>> >>>>>>> Lvc@ >>>>>>> >>>>>>> On 23 September 2014 21:53, <[email protected]> wrote: >>>>>>> >>>>>>>> Hi Luca, >>>>>>>> >>>>>>>> Yes, I'm aware of that... >>>>>>>> >>>>>>>> My reference to 1.7.9 was only concerning Christian's suggestion. >>>>>>>> >>>>>>>> When I tested 2.0-M2, i still have no improvement. Does it need to >>>>>>>> be activated for use in the console or binary interface? >>>>>>>> >>>>>>>> I do not have Java apps for creating documents. >>>>>>>> >>>>>>>> Regards, >>>>>>>> - Ben >>>>>>>> >>>>>>>> On Tuesday, September 23, 2014 9:14:57 PM UTC+2, Lvc@ wrote: >>>>>>>>> >>>>>>>>> Hi Ben, >>>>>>>>> This is supported only in 2.0-M2. >>>>>>>>> >>>>>>>>> Lvc@ >>>>>>>>> >>>>>>>>> >>>>>>>>> On 23 September 2014 21:03, <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Christian, >>>>>>>>>> >>>>>>>>>> Tried your suggestion too on 1.7.9, unfortunately it hasn't had >>>>>>>>>> any influence even though it looked promising. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> - Ben >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tuesday, September 23, 2014 8:48:00 AM UTC+2, Christian >>>>>>>>>> Laakmann wrote: >>>>>>>>>>> >>>>>>>>>>> Hi all, Luca, >>>>>>>>>>> >>>>>>>>>>> it seems as if you wait for synchronous responses from _all_ >>>>>>>>>>> available Nodes, even if quorum is set to 0. In >>>>>>>>>>> ODistributedResponseManager# >>>>>>>>>>> collectResponse, this code handles the notification of waiting >>>>>>>>>>> threads after all expected responses have been received. >>>>>>>>>>> >>>>>>>>>>> if (receivedResponses >= expectedSynchronousResponses && >>>>>>>>>>> (!waitForLocalNode || receivedCurrentNode)) { >>>>>>>>>>> if (completed || isMinimumQuorumReached(false)) { >>>>>>>>>>> // NOTIFY TO THE WAITER THE RESPONSE IS COMPLETE NOW >>>>>>>>>>> notifyWaiters(); >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> My expectation would be, that #isMinimumQuorumReached() were >>>>>>>>>>> called as part of the first IF, i.e.: >>>>>>>>>>> >>>>>>>>>>> if (completed >>>>>>>>>>> || isMinimumQuorumReached(false) >>>>>>>>>>> || (receivedResponses >= expectedSynchronousResponses && >>>>>>>>>>> (!waitForLocalNode || receivedCurrentNode))) { >>>>>>>>>>> // NOTIFY TO THE WAITER THE RESPONSE IS COMPLETE NOW >>>>>>>>>>> notifyWaiters(); >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> From my understanding, this would notify the waiting thread that >>>>>>>>>>> pushed the task as soon as the quorum is reached or all responses >>>>>>>>>>> from the >>>>>>>>>>> available nodes have been collected. >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Christian >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tuesday, 23 September 2014 00:58:55 UTC+2, [email protected] >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Luca, >>>>>>>>>>>> >>>>>>>>>>>> Since I've been hung out to dry, I've toyed around with the >>>>>>>>>>>> orientdb sources. I was able to establish that the bottlenecks are >>>>>>>>>>>> indeed >>>>>>>>>>>> sendRequest and send2Nodes. Removing sleeps in the latest 1.7.9 >>>>>>>>>>>> release >>>>>>>>>>>> already provided a 50% improvement (From ~1.2sec down to >>>>>>>>>>>> ~0.55sec.). >>>>>>>>>>>> Clearly we have different views on what is asynchronous. In your >>>>>>>>>>>> methods, >>>>>>>>>>>> you inarguably wait for a response from the nodes before giving a >>>>>>>>>>>> response >>>>>>>>>>>> to the client. IMHO asynchronous replication should update the >>>>>>>>>>>> local >>>>>>>>>>>> database, respond to the client and care about the rest later. I >>>>>>>>>>>> believe >>>>>>>>>>>> the replication methodology needs a rethink. Whilst in may be >>>>>>>>>>>> suitable to >>>>>>>>>>>> LAN environments, it is certainly not suitable for WAN. I still >>>>>>>>>>>> wonder if >>>>>>>>>>>> this is intended or even if it has ever been tested. I see >>>>>>>>>>>> examples with >>>>>>>>>>>> europe and usa nodes with sharding, etc.. but I can't believe such >>>>>>>>>>>> delays >>>>>>>>>>>> are accepted or the norm. Other technologies can do it, so should >>>>>>>>>>>> OrientDB. >>>>>>>>>>>> >>>>>>>>>>>> Anyway, enough of my ramblings.... Have a good evening. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> - Ben >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "OrientDB" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> --- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "OrientDB" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> >>>>>> -- >>>>>> >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "OrientDB" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> >>>> >>>> -- >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "OrientDB" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "OrientDB" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
