> On July 9, 2013, 2:13 a.m., Mona Chitnis wrote: > > Did a first-pass of the review. For HA to work with secure Zookeeper, we'll > > need to add another server setting to indicate ZK to accept SASL based > > authentication, not IP based, and another one for the associated Kerberos > > principal. I see currently you are going with hostname based auth, but for > > dynamically resizing the "oozies" lot, this will require a ZK restart no? > > > > Yet to review the unit tests..
I was kinda thinking that it may make sense to do the secure HA stuff as a second patch to get the basics in first. Unless it would make more sense to do it now with this patch? I'm not sure what you mean by the "oozies lot"; nothing should require a ZK restart as far as I know. Oozie servers can join and leave without any issues. > On July 9, 2013, 2:13 a.m., Mona Chitnis wrote: > > /trunk/core/src/main/java/org/apache/oozie/service/ZKJobsConcurrencyService.java, > > line 117 > > <https://reviews.apache.org/r/11922/diff/3/?file=309728#file309728line117> > > > > Is there a way to save 'myIndex' as a service level variable, so no > > need to query ZK for each invocation of 'jobIdBelongsToMe'? For this > > however, ZK will need to notify when it discovers when one of the Oozie > > servers goes down/comes up, and all the servers then recalculate their > > indexes just once again. ZK has "watches" where you can subscribe to updates and Curator has some stuff to cache znode info. I'll look into these some more; my concern with them before was that they can be out of date for a short period of time, so even if we used them, we might still have to query ZK everytime anyway depending on how important something is to be up-to-date. I suppose the necessity of this would depend on how heavy a load the Oozie servers are under, and how heavy a load they put on ZooKeeper as a result (which we need more testing to figure out). > On July 9, 2013, 2:13 a.m., Mona Chitnis wrote: > > /trunk/core/src/main/java/org/apache/oozie/service/ZKXLogStreamingService.java, > > line 243 > > <https://reviews.apache.org/r/11922/diff/3/?file=309730#file309730line243> > > > > what does this put do, when line 255 also puts the log lines against > > the currentTimestamp? This is to catch the edge case and would miss the last message from a server. > On July 9, 2013, 2:13 a.m., Mona Chitnis wrote: > > /trunk/core/src/test/java/org/apache/oozie/service/TestZKLocksService.java, > > line 194 > > <https://reviews.apache.org/r/11922/diff/3/?file=309752#file309752line194> > > > > What does a timeout of '0' signify for the lock here? It means that if it can't acquire the lock immediately, it will timeout (i.e. it won't wait at all). And -1 means that it will never timeout (i.e. it will wait forever). - Robert ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11922/#review22878 ----------------------------------------------------------- On June 20, 2013, 6:20 p.m., Robert Kanter wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11922/ > ----------------------------------------------------------- > > (Updated June 20, 2013, 6:20 p.m.) > > > Review request for oozie. > > > Bugs: OOZIE-615 > https://issues.apache.org/jira/browse/OOZIE-615 > > > Repository: oozie > > > Description > ------- > > See this comment for details: > https://issues.apache.org/jira/browse/OOZIE-615?focusedCommentId=13686181&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13686181 > > > Diffs > ----- > > /trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1495113 > /trunk/client/src/main/java/org/apache/oozie/client/OozieClient.java > 1495113 > /trunk/client/src/main/java/org/apache/oozie/client/rest/RestConstants.java > 1495113 > /trunk/core/pom.xml 1495113 > /trunk/core/src/main/java/org/apache/oozie/BaseEngine.java 1495113 > /trunk/core/src/main/java/org/apache/oozie/BundleEngine.java 1495113 > /trunk/core/src/main/java/org/apache/oozie/CoordinatorEngine.java 1495113 > /trunk/core/src/main/java/org/apache/oozie/DagEngine.java 1495113 > /trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1495113 > /trunk/core/src/main/java/org/apache/oozie/command/Command.java 1495113 > /trunk/core/src/main/java/org/apache/oozie/command/XCommand.java 1495113 > > /trunk/core/src/main/java/org/apache/oozie/service/ActionCheckerService.java > 1495113 > > /trunk/core/src/main/java/org/apache/oozie/service/JobsConcurrencyService.java > PRE-CREATION > /trunk/core/src/main/java/org/apache/oozie/service/MemoryLocksService.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/service/PauseTransitService.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/service/PurgeService.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/service/RecoveryService.java > 1495113 > > /trunk/core/src/main/java/org/apache/oozie/service/StatusTransitService.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/service/XLogService.java 1495113 > > /trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java > PRE-CREATION > > /trunk/core/src/main/java/org/apache/oozie/service/ZKJobsConcurrencyService.java > PRE-CREATION > /trunk/core/src/main/java/org/apache/oozie/service/ZKLocksService.java > PRE-CREATION > > /trunk/core/src/main/java/org/apache/oozie/service/ZKXLogStreamingService.java > PRE-CREATION > /trunk/core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/servlet/ServicesLoader.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/servlet/V0JobServlet.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/servlet/V2AdminServlet.java > 1495113 > /trunk/core/src/main/java/org/apache/oozie/util/ConfigUtils.java 1495113 > > /trunk/core/src/main/java/org/apache/oozie/util/FixedJsonInstanceSerializer.java > PRE-CREATION > /trunk/core/src/main/java/org/apache/oozie/util/LockToken.java PRE-CREATION > /trunk/core/src/main/java/org/apache/oozie/util/MemoryLocks.java 1495113 > /trunk/core/src/main/java/org/apache/oozie/util/XLogStreamer.java 1495113 > /trunk/core/src/main/java/org/apache/oozie/util/ZKUtils.java PRE-CREATION > /trunk/core/src/main/resources/oozie-default.xml 1495113 > > /trunk/core/src/test/java/org/apache/oozie/TestCoordinatorEngineStreamLog.java > 1495113 > /trunk/core/src/test/java/org/apache/oozie/command/TestXCommand.java > 1495113 > > /trunk/core/src/test/java/org/apache/oozie/service/DummyLogStreamingServlet.java > PRE-CREATION > > /trunk/core/src/test/java/org/apache/oozie/service/TestJobsConcurrencyService.java > PRE-CREATION > /trunk/core/src/test/java/org/apache/oozie/service/TestXLogService.java > 1495113 > > /trunk/core/src/test/java/org/apache/oozie/service/TestXLogStreamingService.java > PRE-CREATION > > /trunk/core/src/test/java/org/apache/oozie/service/TestZKJobsConcurrencyService.java > PRE-CREATION > /trunk/core/src/test/java/org/apache/oozie/service/TestZKLocksService.java > PRE-CREATION > > /trunk/core/src/test/java/org/apache/oozie/service/TestZKXLogStreamingService.java > PRE-CREATION > > /trunk/core/src/test/java/org/apache/oozie/servlet/MockCoordinatorEngineService.java > 1495113 > > /trunk/core/src/test/java/org/apache/oozie/servlet/MockDagEngineService.java > 1495113 > /trunk/core/src/test/java/org/apache/oozie/test/ZKXTestCase.java > PRE-CREATION > /trunk/core/src/test/java/org/apache/oozie/util/TestMemoryLocks.java > 1495113 > /trunk/core/src/test/java/org/apache/oozie/util/TestXLogFilter.java 1495113 > /trunk/core/src/test/java/org/apache/oozie/util/TestZKUtils.java > PRE-CREATION > /trunk/distro/src/main/tomcat/ssl-web.xml 1495113 > /trunk/docs/src/site/twiki/AG_Install.twiki 1495113 > /trunk/docs/src/site/twiki/DG_CommandLineTool.twiki 1495113 > /trunk/docs/src/site/twiki/WebServicesAPI.twiki 1495113 > /trunk/pom.xml 1495113 > > Diff: https://reviews.apache.org/r/11922/diff/ > > > Testing > ------- > > > Thanks, > > Robert Kanter > >
