Murtadha Hubail has posted comments on this change. Change subject: Add Global Resource Id Factory ......................................................................
Patch Set 1: (8 comments) @Till, a) What do we use the ResourceIds for and why do the NCs need to agree on them now? Hopefully this was answered here: https://cwiki.apache.org/confluence/display/ASTERIXDB/Fault+Tolerance b) Why do the ResourceIds need to be a top-level Hyracks concept (especially outside of storage). Maybe the resourceId naming confused things. The idea is to have a global unique id generator among the NCs, whether it is used for storage or something else, which is coordinate by the CC. In order to avoid storing this value on disk during CC startup/shutdown, each NC when it registers it passes its local maximum and the CC updates the max global value on each NC registration. If the cluster is not yet active (not all NCs reported their values), the CC id generator throws an exception. This value could be moved to Zookeeper on a later stage to support CC fault tolerance. why is it not sufficient to keep them in AsterixDB? Where do you suggest to keep them in Asterix? Asterix is using ClusterControllerService and NodeControllerService that Hyracks provides and the CC/NC communication is already provided there. This also could be a useful feature that Hyracks provides. Let's have a chat about it when you have the time. I might be missing something. https://asterix-gerrit.ics.uci.edu/#/c/485/1//COMMIT_MSG Commit Message: Line 7: Add Global Resource Id Factory > Could you add a few words about the change after the headline? (Separated b Few words will be added in the next patch set. https://asterix-gerrit.ics.uci.edu/#/c/485/1/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/ICCApplicationEntryPoint.java File hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/ICCApplicationEntryPoint.java: Line 27: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/485/1/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/INCApplicationEntryPoint.java File hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/INCApplicationEntryPoint.java: Line 28: public long getMaxLocalResourceId() throws Exception; > That seems to be a pretty low-level method on a pretty high-level interface would renaming it to getNCMaxLocalResourceID() help? :) https://asterix-gerrit.ics.uci.edu/#/c/485/1/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/context/IHyracksRootContext.java File hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/context/IHyracksRootContext.java: Line 31: public long getResourceId() throws Exception; > What is the resourceId of a HyracksRootContext? It's not a resource id. The method name could be changed to requestAndGetGlobalResourceID(). It delegates the call to NodeControllerService which actually sends the request to the CC and waits for the response. https://asterix-gerrit.ics.uci.edu/#/c/485/1/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java File hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java: Line 660: public synchronized long getResourceId() throws Exception { > If getting the resourceId give a different one every time, the method name generateAndGetGlobalResourceId? https://asterix-gerrit.ics.uci.edu/#/c/485/1/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/CCNCFunctions.java File hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/CCNCFunctions.java: Line 1260: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/485/1/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/ClusterControllerRemoteProxy.java File hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/ipc/ClusterControllerRemoteProxy.java: Line 157: > WS Done https://asterix-gerrit.ics.uci.edu/#/c/485/1/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/work/WorkQueue.java File hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/work/WorkQueue.java: Line 125: + dequeueCount.incrementAndGet() + "/" + enqueueCount); > If there's no change to the semantics of this file, could you revert it? Sure. -- To view, visit https://asterix-gerrit.ics.uci.edu/485 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib9f49234eebe912c48e7f71980433a9b42595741 Gerrit-PatchSet: 1 Gerrit-Project: hyracks Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail <hubail...@gmail.com> Gerrit-Reviewer: Ian Maxon <ima...@apache.org> Gerrit-Reviewer: Jenkins <jenk...@fulliautomatix.ics.uci.edu> Gerrit-Reviewer: Murtadha Hubail <hubail...@gmail.com> Gerrit-Reviewer: Till Westmann <ti...@apache.org> Gerrit-Reviewer: abdullah alamoudi <bamou...@gmail.com> Gerrit-HasComments: Yes
