Not sure what is going on, I did not encounter the problem on the VM after changing the zoo.cfg. The other file you might need to change is Hadoop's hdfs-site.xml, to change the value of the dfs.datanode.data.dir.
However, changing the hadoop defaults will not fix the problem with Accumulo. My suggestion is to re-start everything: 1. stop Accumulo (which is probably not running anyway) 2. clean up Hadoop's /accumulo directory (see Christopher's message earlier in the thread) 3. stop zookeeper 4. change hadoop's hdfs-site.xml - make sure permission on the new folder are permissive enough 5. double check zoo.cfg and permissions for the dataDir 6. restart computer 7. start zookeeper 8. try initialize Accumulo 9. try start Accumulo On Sat, May 14, 2016 at 1:34 PM, Amila Wijayarathna < [email protected]> wrote: > Hi Adina, > Zookeeper is successfully started, i got imok back. But Accumulo is failed > to initialize. i got the following, > > 2016-05-14 22:56:07,267 [init.Initialize] INFO : Hadoop Filesystem is > hdfs://localhost:9000 > 2016-05-14 22:56:07,268 [init.Initialize] INFO : Accumulo data dirs are > [hdfs://localhost:9000/ACCUMULO5] > 2016-05-14 22:56:07,268 [init.Initialize] INFO : Zookeeper server is > localhost:2181 > 2016-05-14 22:56:07,268 [init.Initialize] INFO : Checking if Zookeeper is > available. If this hangs, then you need to make sure zookeeper is running > 2016-05-14 22:57:07,640 [init.Initialize] FATAL: > java.lang.RuntimeException: Failed to connect to zookeeper (localhost:2181) > within 2x zookeeper timeout period 30000 > java.lang.RuntimeException: Failed to connect to zookeeper (localhost:2181) > within 2x zookeeper timeout period 30000 > at > org.apache.accumulo.fate.zookeeper.ZooSession.connect(ZooSession.java:123) > at > > org.apache.accumulo.fate.zookeeper.ZooSession.getSession(ZooSession.java:167) > at > org.apache.accumulo.fate.zookeeper.ZooReader.getSession(ZooReader.java:39) > at > > org.apache.accumulo.fate.zookeeper.ZooReaderWriter.getZooKeeper(ZooReaderWriter.java:50) > at > org.apache.accumulo.fate.zookeeper.ZooReader.exists(ZooReader.java:166) > at > > org.apache.accumulo.server.init.Initialize.zookeeperAvailable(Initialize.java:277) > at > org.apache.accumulo.server.init.Initialize.checkInit(Initialize.java:159) > at > org.apache.accumulo.server.init.Initialize.doInit(Initialize.java:213) > at org.apache.accumulo.server.init.Initialize.main(Initialize.java:590) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.accumulo.start.Main$1.run(Main.java:141) > at java.lang.Thread.run(Thread.java:745) > Thread "init" died java.lang.RuntimeException: Failed to connect to > zookeeper (localhost:2181) within 2x zookeeper timeout period 30000 > java.lang.RuntimeException: java.lang.RuntimeException: Failed to connect > to zookeeper (localhost:2181) within 2x zookeeper timeout period 30000 > at org.apache.accumulo.server.init.Initialize.main(Initialize.java:594) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.accumulo.start.Main$1.run(Main.java:141) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException: Failed to connect to zookeeper > (localhost:2181) within 2x zookeeper timeout period 30000 > at > org.apache.accumulo.fate.zookeeper.ZooSession.connect(ZooSession.java:123) > at > > org.apache.accumulo.fate.zookeeper.ZooSession.getSession(ZooSession.java:167) > at > org.apache.accumulo.fate.zookeeper.ZooReader.getSession(ZooReader.java:39) > at > > org.apache.accumulo.fate.zookeeper.ZooReaderWriter.getZooKeeper(ZooReaderWriter.java:50) > at > org.apache.accumulo.fate.zookeeper.ZooReader.exists(ZooReader.java:166) > at > > org.apache.accumulo.server.init.Initialize.zookeeperAvailable(Initialize.java:277) > at > org.apache.accumulo.server.init.Initialize.checkInit(Initialize.java:159) > at > org.apache.accumulo.server.init.Initialize.doInit(Initialize.java:213) > at org.apache.accumulo.server.init.Initialize.main(Initialize.java:590) > ... 6 more > > Are there any configurations need to change other than zoo.cfg? > > Thank you! > > On Sat, May 14, 2016 at 9:02 PM, Adina Crainiceanu <[email protected]> wrote: > > > Amila, > > > > > > 2 ideas: > > > > 1) Are you sure that Zookeeper started successfully? Maybe it did not. > Make > > sure the permissions on that local dir you used for dirData are correct. > > > > Try the > > sudo echo ruok | nc 127.0.0.1 2181 > > > > if you don;t get imok back, start Zookeeper > > > > 2) Did you re-initialized Accumulo? Since you started Zookeeper in a > > different directory, Accumulo needs to be initialized before it starts > > > > accumulo-${ACCUMULO_VERSION}/bin/accumulo init --instance-name dev > > --password root > > accumulo-${ACCUMULO_VERSION}/bin/start-all.sh > > > > Adina > > > > On Sat, May 14, 2016 at 7:34 AM, Amila Wijayarathna < > > [email protected]> wrote: > > > > > Hi Adina, > > > > > > After a restart I used *accumulo-1.6.4/bin/stop-all.sh* and after that > I > > > start accumulo again with *accumulo-1.6.4/bin/start-all.sh * then I can > > > successfully start accumulo without initializing problem. > > > > > > But after I changed the Datadir value to a local, I can successfully > > start > > > Zookeeper. Then I tried to start accumulo using > > > *accumulo-1.6.4/bin/start-all.sh > > > *but I got the following, > > > > > > > > > amila@amila-X550LD:~$ ~/Installs/accumulo-1.6.4/bin/start-all.sh > > > > > > Starting monitor on localhost > > > > > > WARN : Max open files on localhost is 1024, recommend 32768 > > > > > > Starting tablet servers .... done > > > > > > 2016-05-14 16:34:54,379 [server.Accumulo] INFO : Attempting to talk to > > > zookeeper > > > > > > Starting tablet server on localhost > > > > > > WARN : Max open files on localhost is 1024, recommend 32768 > > > > > > Thread "org.apache.accumulo.master.state.SetGoalState" died Failed to > > > connect to zookeeper (localhost:2181) within 2x zookeeper timeout > period > > > 30000 > > > > > > java.lang.RuntimeException: Failed to connect to zookeeper > > (localhost:2181) > > > within 2x zookeeper timeout period 30000 > > > > > > at > > > > > > org.apache.accumulo.fate.zookeeper.ZooSession.connect(ZooSession.java:123) > > > > > > at > > > > > > > > > org.apache.accumulo.fate.zookeeper.ZooSession.getSession(ZooSession.java:167) > > > > > > > > > at > > > > > > org.apache.accumulo.fate.zookeeper.ZooReader.getSession(ZooReader.java:39) > > > > > > at > > > > > > > > > org.apache.accumulo.fate.zookeeper.ZooReaderWriter.getZooKeeper(ZooReaderWriter.java:50) > > > > > > > > > at > > > > > > > > > org.apache.accumulo.fate.zookeeper.ZooReader.getChildren(ZooReader.java:128) > > > > > > > > > at > > > > > > > > > org.apache.accumulo.server.Accumulo.waitForZookeeperAndHdfs(Accumulo.java:269) > > > > > > > > > at > > org.apache.accumulo.master.state.SetGoalState.main(SetGoalState.java:46) > > > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > > > at > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > > > > > > > > at > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > > > > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > > > > at org.apache.accumulo.start.Main$1.run(Main.java:141) > > > > > > at java.lang.Thread.run(Thread.java:745) > > > > > > Starting master on localhost > > > > > > WARN : Max open files on localhost is 1024, recommend 32768 > > > > > > Starting garbage collector on localhost > > > > > > WARN : Max open files on localhost is 1024, recommend 32768 > > > > > > Starting tracer on localhost > > > > > > WARN : Max open files on localhost is 1024, recommend 32768 > > > > > > > > > Any idea about this is appreciated. > > > > > > Thank you. > > > > > > On Fri, May 13, 2016 at 8:28 PM, Adina Crainiceanu <[email protected]> > > wrote: > > > > > > > Amila, > > > > > > > > As Christopher said, make sure that Zookeeper and Hadoop store the > data > > > > outside the /tmp folder. For Zookeeper, the default configuration > > > specifies > > > > the /tmp/zookeeper folder, so make sure to change that (go to > > > > /pathtoZookeeper/conf/zoo.cfg and look at the dataDir value; it > should > > > not > > > > start with /tmp. If it does, change that to some local, non-tmp > folder, > > > for > > > > example /data/zookeeper). > > > > > > > > That is a problem for the Rya Vagrant VM too - Zookeeper stores the > > data > > > in > > > > /tmp/zookeeper so a restart creates problems with Accumulo, including > > > > a "Waiting > > > > for accumulo to be initialized." message > > > > > > > > Adina > > > > > > > > On Thu, Apr 21, 2016 at 2:17 PM, Amila Wijayarathna < > > > > [email protected]> wrote: > > > > > > > > > Hi Christoper, > > > > > Thank you for your prompt reply. I successfully re-corrected that > > > warning > > > > > messages from your guidelines and still working on that > initializing > > > > > problem.Any further ideas about that are thoroughly appreciated. > > > > > > > > > > Thank you! > > > > > > > > > > On Tue, Apr 19, 2016 at 11:06 PM, Christopher <[email protected] > > > > > > wrote: > > > > > > > > > > > The warning about the ZooKeeper version is probably due to a > > trailing > > > > > slash > > > > > > at the end of your environment variable for ZOOKEEPER_HOME. > > > > > > > > > > > > The max open files warning is regarding what is being discussed > > here: > > > > > > > > > > > > > > > > > > > > > > > > > > > http://askubuntu.com/questions/162229/how-do-i-increase-the-open-files-limit-for-a-non-root-user > > > > > > (this > > > > > > is just the first link I found that covered the topic... but > there > > > > might > > > > > be > > > > > > better discussions of system security limits and ulimit > elsewhere) > > > > > > > > > > > > The synconclose warning can be addressed by setting the > recommended > > > > value > > > > > > to true in your hadoop configuration (hdfs-site.xml) > > > > > > > > > > > > As for the initialization problem, it sounds like you're trying > to > > > > > > re-initialize Accumulo in the same location that it was > previously > > > > > > initialized. You only need to do initialization once, but you do > > need > > > > to > > > > > > make sure that wherever Hadoop and ZooKeeper are storing their > data > > > in > > > > > the > > > > > > local filesystem, that they aren't in a temporary directory or > > > > something > > > > > > like that. > > > > > > > > > > > > If you actually do wish to re-initialize Accumulo, you can delete > > the > > > > > > previous contents of Hadoop's /accumulo directory first (make > sure > > > you > > > > > want > > > > > > to do this... existing data will be deleted), using something > like > > > > > `hadoop > > > > > > fs -rmr /accumulo` (exact command may depend on your version of > > > > Hadoop). > > > > > > > > > > > > On Tue, Apr 19, 2016 at 1:10 PM Amila Wijayarathna < > > > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I'm trying to work with RYA in my Ubuntu laptop. And it needs > > > Hadoop, > > > > > > > Zookeeper and Accumulo servers As prerequisites. > > > > > > > > > > > > > > After a shutdown, I need to restart hadoop, zookeeper and > > accumulo. > > > > > When > > > > > > I > > > > > > > do, Hadoop and Zookeeper servers are starting successfully but > > > when I > > > > > > start > > > > > > > Accumulo I get the following, > > > > > > > > > > > > > > WARN : Using Zookeeper > > > > > > > /home/amila/Installs/zookeeper-3.4.8/zookeeper-3.4.8. Use > version > > > > 3.3.0 > > > > > > or > > > > > > > greater to avoid zookeeper deadlock bug. > > > > > > > > > > > > > > Starting monitor on localhost > > > > > > > > > > > > > > WARN : Max open files on localhost is 1024, recommend 32768 > > > > > > > > > > > > > > Starting tablet servers .... done > > > > > > > > > > > > > > 2016-04-19 17:40:43,303 [fs.VolumeManagerImpl] WARN : > > > > > > > dfs.datanode.synconclose set to false in hdfs-site.xml: data > loss > > > is > > > > > > > possible on hard system reset or power loss > > > > > > > > > > > > > > 2016-04-19 17:40:43,308 [server.Accumulo] INFO : Attempting to > > talk > > > > to > > > > > > > zookeeper > > > > > > > > > > > > > > 2016-04-19 17:40:43,522 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:44,534 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:45,536 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:46,538 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:47,541 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:48,543 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:49,545 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:50,547 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:51,549 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:52,550 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > Starting tablet server on localhost > > > > > > > > > > > > > > WARN : Max open files on localhost is 1024, recommend 32768 > > > > > > > > > > > > > > 2016-04-19 17:40:53,551 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:54,552 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:55,554 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:56,555 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:57,556 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:58,558 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:40:59,559 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > 2016-04-19 17:41:00,561 [server.Accumulo] INFO : Waiting for > > > accumulo > > > > > to > > > > > > be > > > > > > > initialized > > > > > > > > > > > > > > Then I tried to initialize Accumulo and I got following, > > > > > > > > > > > > > > 2016-04-19 16:56:06,984 [fs.VolumeManagerImpl] WARN : > > > > > > > dfs.datanode.synconclose set to false in hdfs-site.xml: data > loss > > > is > > > > > > > possible on hard system reset or power loss > > > > > > > > > > > > > > 2016-04-19 16:56:06,985 [init.Initialize] INFO : Hadoop > > Filesystem > > > is > > > > > > > hdfs://localhost:9000 > > > > > > > > > > > > > > 2016-04-19 16:56:06,986 [init.Initialize] INFO : Accumulo data > > dirs > > > > are > > > > > > > [hdfs://localhost:9000/accumulo] > > > > > > > > > > > > > > 2016-04-19 16:56:06,986 [init.Initialize] INFO : Zookeeper > server > > > is > > > > > > > localhost:2181 > > > > > > > > > > > > > > 2016-04-19 16:56:06,986 [init.Initialize] INFO : Checking if > > > > Zookeeper > > > > > is > > > > > > > available. If this hangs, then you need to make sure zookeeper > is > > > > > running > > > > > > > > > > > > > > 2016-04-19 16:56:07,250 [init.Initialize] FATAL: It appears the > > > > > > directories > > > > > > > [hdfs://localhost:9000/accumulo] were previously initialized. > > > > > > > > > > > > > > 2016-04-19 16:56:07,250 [init.Initialize] FATAL: Change the > > > property > > > > > > > instance.volumes to use different filesystems, > > > > > > > > > > > > > > 2016-04-19 16:56:07,250 [init.Initialize] FATAL: or change the > > > > property > > > > > > > instance.dfs.dir to use a different directory. > > > > > > > > > > > > > > 2016-04-19 16:56:07,250 [init.Initialize] FATAL: The current > > value > > > of > > > > > > > instance.dfs.uri is || > > > > > > > > > > > > > > 2016-04-19 16:56:07,250 [init.Initialize] FATAL: The current > > value > > > of > > > > > > > instance.dfs.dir is |/accumulo| > > > > > > > > > > > > > > 2016-04-19 16:56:07,250 [init.Initialize] FATAL: The current > > value > > > of > > > > > > > instance.volumes is |hdfs://localhost:9000/accumulo| > > > > > > > > > > > > > > > > > > > > > Can anyone point me how to do this correctly? > > > > > > > > > > > > > > Thank you! > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > *Amila Wijayarathna* > > > > > > > Undergraduate, > > > > > > > Faculty of Information Technology, > > > > > > > University of Moratuwa, > > > > > > > Sri Lanka. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > *Amila Wijayarathna* > > > > > Undergraduate, > > > > > Faculty of Information Technology, > > > > > University of Moratuwa, > > > > > Sri Lanka. > > > > > > > > > > > > > > > > > > > > > -- > > > > Dr. Adina Crainiceanu > > > > Associate Professor, Computer Science Department > > > > United States Naval Academy > > > > 410-293-6822 > > > > [email protected] > > > > http://www.usna.edu/Users/cs/adina/ > > > > > > > > > > > > > > > > -- > > > > > > *Amila Wijayarathna* > > > Undergraduate, > > > Faculty of Information Technology, > > > University of Moratuwa, > > > Sri Lanka. > > > > > > > > > > > -- > > Dr. Adina Crainiceanu > > Associate Professor, Computer Science Department > > United States Naval Academy > > 410-293-6822 > > [email protected] > > http://www.usna.edu/Users/cs/adina/ > > > > > > -- > > *Amila Wijayarathna* > Undergraduate, > Faculty of Information Technology, > University of Moratuwa, > Sri Lanka. > -- Dr. Adina Crainiceanu Associate Professor, Computer Science Department United States Naval Academy 410-293-6822 [email protected] http://www.usna.edu/Users/cs/adina/
