> On April 15, 2016, 6:31 p.m., Suma Shivaprasad wrote: > > webapp/src/main/java/org/apache/atlas/web/setup/SetupSteps.java, line 59 > > <https://reviews.apache.org/r/45929/diff/3/?file=1344552#file1344552line59> > > > > what is the expected behaviour in unclean shutdown case when lock is > > not cleared? Is this an ephemeral node that will be cleared automatically? > > Hemanth Yamijala wrote: > I verified using Zookeeper Shell. The Lock nodes created by Curator are > ephemeral nodes. Hence, they are automatically cleared when the client > connection to Zookeeper breaks. The more important question though is what > happens if the setup is halfway through when this happens. To handle this > case, I took this approach: > > * It is critical to not have setup run multiple times in inconsistent > state as the results could be pretty much undefined. > * Given the above, it is safer, although very inconvenient to refuse to > run setup if something is detected as wrong. Manual fixing will be required > in this extreme scenario. > > To handle this, I have introduced a marker znode in Zookeeper to indicate > setup is in progress. This is deleted only when all setup steps are done. If > this znode is found on entering the critical section, Setup will fail with > SetupException and a message stating there was an unclean shutdown of setup > and this needs to be fixed manually. > > Like I said, in cases where the node failed to be cleared even when all > setup is done, this would be inconvenient. But the approach leans on safety > rather than convenience. Please let me know if this is OK. Leaving the issue > open until review is done. > > Suma Shivaprasad wrote: > Yes this makes sense since the user would get to know that there was an > unclean shutdown during setup which could have caused other issues like > disabling the hbase 'titan' table which means the next setup could fail. > Should we add this as as a known issue in documentation - the hbase setup ?
Added documentation in InstallationSteps.twiki - Hemanth ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45929/#review129171 ----------------------------------------------------------- On April 20, 2016, 3:28 a.m., Hemanth Yamijala wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45929/ > ----------------------------------------------------------- > > (Updated April 20, 2016, 3:28 a.m.) > > > Review request for atlas. > > > Bugs: ATLAS-628 > https://issues.apache.org/jira/browse/ATLAS-628 > > > Repository: atlas > > > Description > ------- > > Implements capability in atlas to run a set of registered setup steps. > Currently the only step is setting up the backend graph repository. The > intent is to extend this later for setting up Kafka at least and maybe other > things like Solr. > > > Diffs > ----- > > distro/src/bin/atlas_start.py 7db74c0 > distro/src/conf/atlas-application.properties 4131240 > docs/src/site/twiki/Configuration.twiki 023f5a0 > docs/src/site/twiki/InstallationSteps.twiki 0851d53 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphSchemaInitializer.java > PRE-CREATION > server-api/src/main/java/org/apache/atlas/setup/SetupException.java > PRE-CREATION > server-api/src/main/java/org/apache/atlas/setup/SetupStep.java PRE-CREATION > webapp/src/main/java/org/apache/atlas/Atlas.java 58c386d > webapp/src/main/java/org/apache/atlas/web/service/ActiveInstanceState.java > 6b39970 > > webapp/src/main/java/org/apache/atlas/web/service/AtlasZookeeperSecurityProperties.java > 40a6030 > webapp/src/main/java/org/apache/atlas/web/service/CuratorFactory.java > 773e04e > webapp/src/main/java/org/apache/atlas/web/setup/AtlasSetup.java > PRE-CREATION > webapp/src/main/java/org/apache/atlas/web/setup/AtlasSetupModule.java > PRE-CREATION > webapp/src/main/java/org/apache/atlas/web/setup/SetupSteps.java > PRE-CREATION > > webapp/src/test/java/org/apache/atlas/web/service/AtlasZookeeperSecurityPropertiesTest.java > 118587a > webapp/src/test/java/org/apache/atlas/web/setup/SetupStepsTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/45929/diff/ > > > Testing > ------- > > Existing ITs, UTs pass. > > * Manually verified setting up with both embedded HBase setup and external > HBase setup. > * Also ran atlas HA instances at exactly same time and verified the noted > exception does not happen. > > > Thanks, > > Hemanth Yamijala > >
