That is a good point John, thanks. Kevin, can you share your configs for Accumulo, Hadoop and Zookeeper?
On Fri, Aug 5, 2016 at 12:42 PM, John Vines <[email protected]> wrote: > How did you configure hdfs? Did you map the on disk directories away from > tmp? The apache versions of hadoop point to /tmp (or at least used to) so > when you reboot they get clobbered. > > > On Fri, Aug 5, 2016 at 12:16 PM Michael Wall <[email protected]> wrote: > >> Ok, great. One node. In a production cluster, services are distributed >> onto many nodes, so a random reboot has less impact. In your case, a >> reboot means everything will die at the same time. Let's try this. >> >> - Get to a good state and insert some data into accumulo. Create table, >> and run the insert command. Then call flush from the shell to persist the >> data to disk. >> - Scan the metadata table to see where the files are located in hdfs. >> Maybe even hdfs df ls to see the files >> - Reboot the VM >> - Start up hdfs and zookeeper. Look for you files in hdfs. Assuming >> they are there, start up Accumulo >> - Where do you see "Waiting for accumulo to be initialized" >> - Go to the accumulo monitor webpage, something like >> http://localhost:50095. Click on the recent logs and let's see what >> accumulo is saying. >> - Log files will be $ACCUMULO_HOME/logs unless you reconfigured. There >> should be log for the master process. What does that show? >> >> On Fri, Aug 5, 2016 at 12:06 PM, Kevin Cho <[email protected]> wrote: >> >>> Yes, I meant VM Reboot. >>> >>> On Fri, Aug 5, 2016 at 4:05 PM, Kevin Cho <[email protected]> wrote: >>> >>>> Hi Mike, >>>> >>>> Thanks for helping. >>>> - What versions are you using? >>>> Accumulo 1.7.2 >>>> Hadoop 2.7.2 >>>> ZooKeeper 3.4.8 >>>> >>>> - By reboot, do you mean rebooted the box? How many nodes in your >>>> cluster? >>>> Just a single box. Single node. >>>> >>>> - How did you determine the init was successful? >>>> I just run the 'accumulo init' after removing the /accumulo directory >>>> in hdfs. Then I can start/stop the accumulo w/o any issue. It's only when >>>> I start again after hard VM reboot. >>>> >>>> - What is currently happening on the master? >>>> I'm not sure how to answer this. I'm very new to accumulo. >>>> >>>> - What messages are showing up on the monitor? >>>> I'm not sure how to answer this. I'm very new to accumulo. >>>> >>>> - Can you send logs and jstacks? >>>> I'm not sure how to answer this. I'm very new to accumulo. >>>> >>>> I just freshed installed those versions above on Ubuntu 14.04. I've >>>> reinstalled it couple of times and still had the same result. It seems >>>> unless I execute stop-all.sh before reboot, I'm not able to start accumulo >>>> again after the reboot. >>>> >>>> >>>> >>>> On Fri, Aug 5, 2016 at 3:44 PM, Michael Wall <[email protected]> wrote: >>>> >>>>> Kevin, >>>>> >>>>> We are going to need more info. Here are some things I can thing of. >>>>> >>>>> - What versions are you using? >>>>> - By reboot, do you mean rebooted the box? How many nodes in your >>>>> cluster? >>>>> - How did you determine the init was successful? >>>>> - What is currently happening on the master? >>>>> - What messages are showing up on the monitor? >>>>> - Can you send logs and jstacks? >>>>> >>>>> Thanks, >>>>> >>>>> Mike >>>>> >>>>> On Fri, Aug 5, 2016 at 11:20 AM, Kevin Cho <[email protected]> wrote: >>>>> >>>>>> I was able to init accumulo and able to start/stop. However, if I >>>>>> run the command 'reboot' and tried to start again.. I keep getting this >>>>>> >>>>>> Waiting for accumulo to be initialized >>>>>> >>>>>> It just keeps going and I'm not sure how to fix that. Of course, I >>>>>> can re-init but I don't want to do that. The reason why I'm testing with >>>>>> 'reboot' is that sometimes power gets rebooted or anyone w/ access can >>>>>> reboot as well. Thanks for the help! >>>>>> >>>>> >>>>> >>>> >>> >>
