Hey Ben, This is also because your issue and reports. I'm looking what's wrong with UPDATE right now...
Lvc@ ᐧ On 24 September 2014 23:41, <[email protected]> wrote: > Many thanks to Luca for seriously looking into this matter. > > I can report that inserts and deletes are blazingly fast compared to what > was previously. > > Only connects and updates seem to still lag behind. > > Following is an overview issuing commands over the binary interface: > > *dserver.sh* with *one* node up: > > connect: 0.002266 s > insert: 0.000607 s > update:0.000972 s > delete: 0.000280 s > > *dserver.sh* with *two* nodes up: > > connect:* 0.118694 s* > insert: *0.000548 s* <--- MASSIVE IMPROVEMENT > update: *1.292313 s* > delete: *0.000533 s* <--- MASSIVE IMPROVEMENT > > > With these results I look very forward to the final release of OrientDB > 2.0. Hopefully the connects and updates can be improved by then too. > > Regards, > - Ben > > > > On Wednesday, September 24, 2014 8:08:38 PM UTC+2, Lvc@ wrote: >> >> Hi Stuart, >> You're right. I've applied your fix, pushed, deployed on sonatype as >> 2.0-M2-SNAPSHOT. >> >> Lvc@ >> >> ᐧ >> >> On 24 September 2014 18:17, Stuart McCulloch <[email protected]> wrote: >> >>> On 24 Sep 2014, at 16:31, Luca Garulli <[email protected]> wrote: >>> >>> Hi guys, >>> I've some updates. Using an queue in the middle improved performance. >>> Furthermore I've extended the asynchronous also to transactions. >>> >>> Please could you re-test last version 2.0-M2-SNAPSHOT? >>> >>> >>> FYI, I get the following error using the latest snapshot in embedded >>> mode with Java 7: >>> >>> java.lang.NoSuchMethodError: java.util.concurrent. >>> ConcurrentHashMap.keySet()Ljava/util/concurrent/ >>> ConcurrentHashMap$KeySetView; >>> >>> I assume this is because the snapshot was built using Java 8, which >>> changed the return type of keySet for ConcurrentHashMap: >>> >>> http://docs.oracle.com/javase/8/docs/api/java/util/ >>> concurrent/ConcurrentHashMap.html#keySet-- >>> >>> whereas in Java 7 it’s still the plain Set: >>> >>> http://docs.oracle.com/javase/7/docs/api/java/util/ >>> concurrent/ConcurrentHashMap.html#keySet() >>> >>> Possible workarounds (assuming the plan is still to support Java 7) is >>> either build with a version of Java before 8, or use Map or ConcurrentMap >>> for the field rather than the concrete type: >>> >>> diff --git a/core/src/main/java/com/orientechnologies/orient/core/ >>> config/OContextConfiguration.java b/core/src/main/java/com/ >>> orientechnologies/orient/core/config/OContextConfiguration.java >>> index d0dee2b..52c28b4 100644 >>> --- a/core/src/main/java/com/orientechnologies/orient/core/ >>> config/OContextConfiguration.java >>> +++ b/core/src/main/java/com/orientechnologies/orient/core/ >>> config/OContextConfiguration.java >>> @@ -27,7 +27,7 @@ import java.util.concurrent.ConcurrentHashMap; >>> * >>> */ >>> public class OContextConfiguration implements Serializable { >>> - private final ConcurrentHashMap<String, Object> config = new >>> ConcurrentHashMap<String, Object>(); >>> + private final Map<String, Object> config = new >>> ConcurrentHashMap<String, Object>(); >>> >>> /** >>> * Empty constructor to create just a proxy for the >>> OGlobalConfiguration. No values are setted. >>> >>> ( this is the place where the exception originates, there may be a >>> couple of other fields using ConcurrentHashMap where they could just as >>> well use Map or ConcurrentMap... ) >>> >>> — >>> Cheers, Stuart >>> >>> Lvc@ >>> >>> On 24 September 2014 11:31, Luca Garulli <[email protected]> wrote: >>> >>>> Hi Ben, >>>> I found a bottleneck in Hazelcast: https://github.com/ >>>> hazelcast/hazelcast/issues/3661. >>>> >>>> I could put a queue between ODistributedStorage and HZ. Let me try it >>>> now. Soon a new push. >>>> >>>> Lvc@ >>>> >>>> On 24 September 2014 10:59, <[email protected]> wrote: >>>> >>>>> Hi Luca, >>>>> >>>>> I tested this and now get 0.24sec average inserts, which is the >>>>> fastest distributed performance I've experienced yet. >>>>> >>>>> There still seems to be communication with the remote node before >>>>> releasing the client though. >>>>> >>>>> Regards, >>>>> - Ben >>>>> >>>>> On Wednesday, September 24, 2014 1:24:23 AM UTC+2, Lvc@ wrote: >>>>>> >>>>>> Hi Ben, >>>>>> I've pushed a new version that reads the configuration from file >>>>>> *default-distributed-db-config.json*, by adding: >>>>>> >>>>>> * "executionMode": "asynchronous",* >>>>>> >>>>>> Lvc@ >>>>>> >>>>>> On 23 September 2014 21:53, <[email protected]> wrote: >>>>>> >>>>>>> Hi Luca, >>>>>>> >>>>>>> Yes, I'm aware of that... >>>>>>> >>>>>>> My reference to 1.7.9 was only concerning Christian's suggestion. >>>>>>> >>>>>>> When I tested 2.0-M2, i still have no improvement. Does it need to >>>>>>> be activated for use in the console or binary interface? >>>>>>> >>>>>>> I do not have Java apps for creating documents. >>>>>>> >>>>>>> Regards, >>>>>>> - Ben >>>>>>> >>>>>>> On Tuesday, September 23, 2014 9:14:57 PM UTC+2, Lvc@ wrote: >>>>>>>> >>>>>>>> Hi Ben, >>>>>>>> This is supported only in 2.0-M2. >>>>>>>> >>>>>>>> Lvc@ >>>>>>>> >>>>>>>> >>>>>>>> On 23 September 2014 21:03, <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Christian, >>>>>>>>> >>>>>>>>> Tried your suggestion too on 1.7.9, unfortunately it hasn't had >>>>>>>>> any influence even though it looked promising. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> - Ben >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tuesday, September 23, 2014 8:48:00 AM UTC+2, Christian >>>>>>>>> Laakmann wrote: >>>>>>>>>> >>>>>>>>>> Hi all, Luca, >>>>>>>>>> >>>>>>>>>> it seems as if you wait for synchronous responses from _all_ >>>>>>>>>> available Nodes, even if quorum is set to 0. In >>>>>>>>>> ODistributedResponseManager# >>>>>>>>>> collectResponse, this code handles the notification of waiting >>>>>>>>>> threads after all expected responses have been received. >>>>>>>>>> >>>>>>>>>> if (receivedResponses >= expectedSynchronousResponses && >>>>>>>>>> (!waitForLocalNode || receivedCurrentNode)) { >>>>>>>>>> if (completed || isMinimumQuorumReached(false)) { >>>>>>>>>> // NOTIFY TO THE WAITER THE RESPONSE IS COMPLETE NOW >>>>>>>>>> notifyWaiters(); >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> My expectation would be, that #isMinimumQuorumReached() were >>>>>>>>>> called as part of the first IF, i.e.: >>>>>>>>>> >>>>>>>>>> if (completed >>>>>>>>>> || isMinimumQuorumReached(false) >>>>>>>>>> || (receivedResponses >= expectedSynchronousResponses && >>>>>>>>>> (!waitForLocalNode || receivedCurrentNode))) { >>>>>>>>>> // NOTIFY TO THE WAITER THE RESPONSE IS COMPLETE NOW >>>>>>>>>> notifyWaiters(); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> From my understanding, this would notify the waiting thread that >>>>>>>>>> pushed the task as soon as the quorum is reached or all responses >>>>>>>>>> from the >>>>>>>>>> available nodes have been collected. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Christian >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tuesday, 23 September 2014 00:58:55 UTC+2, [email protected] >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Luca, >>>>>>>>>>> >>>>>>>>>>> Since I've been hung out to dry, I've toyed around with the >>>>>>>>>>> orientdb sources. I was able to establish that the bottlenecks are >>>>>>>>>>> indeed >>>>>>>>>>> sendRequest and send2Nodes. Removing sleeps in the latest 1.7.9 >>>>>>>>>>> release >>>>>>>>>>> already provided a 50% improvement (From ~1.2sec down to ~0.55sec.). >>>>>>>>>>> Clearly we have different views on what is asynchronous. In your >>>>>>>>>>> methods, >>>>>>>>>>> you inarguably wait for a response from the nodes before giving a >>>>>>>>>>> response >>>>>>>>>>> to the client. IMHO asynchronous replication should update the local >>>>>>>>>>> database, respond to the client and care about the rest later. I >>>>>>>>>>> believe >>>>>>>>>>> the replication methodology needs a rethink. Whilst in may be >>>>>>>>>>> suitable to >>>>>>>>>>> LAN environments, it is certainly not suitable for WAN. I still >>>>>>>>>>> wonder if >>>>>>>>>>> this is intended or even if it has ever been tested. I see examples >>>>>>>>>>> with >>>>>>>>>>> europe and usa nodes with sharding, etc.. but I can't believe such >>>>>>>>>>> delays >>>>>>>>>>> are accepted or the norm. Other technologies can do it, so should >>>>>>>>>>> OrientDB. >>>>>>>>>>> >>>>>>>>>>> Anyway, enough of my ramblings.... Have a good evening. >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> - Ben >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> --- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "OrientDB" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "OrientDB" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> >>>>> -- >>>>> >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "OrientDB" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>> >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OrientDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OrientDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
