Hi Harshada, I'm confused about what state your setup is in. Can you try the following. 1. cap cleandb 2. Run ps and make sure all Hypertable related procs (Hyperspace, RangeServer, Master, DFSBroker, ThroftBroker, any dangling start/stop scripts) have been stopped. If not kill them manually. 3. just to be sure run cap cleandb again 4. cap start
Also are you running on a single node? If not is your capfile updated with all the servers in your cluster? -Sanjit On Fri, May 28, 2010 at 1:43 PM, Harshada <[email protected]> wrote: > Looks like this issue is related to the Hyperspace @master. > > When I checked RangServer's log at slave, I found this > > "1275077211 ERROR Hypertable.RangeServer : (/opt/hypertable/ > hypertable-0.9.2.8-alpha/src/cc/Hypertable/RangeServer/main.cc:86) > Unable to connect to hyperspace, exiting..." > > I checked Hyperspace's log at master for the error: > > 1275076989 ERROR Hyperspace.Master : run (/opt/hypertable/ > hypertable-0.9.2.8-alpha/src/cc/Hyperspace/RequestHandlerOpen.cc:60): > Hypertable::Exception: node: '/hypertable/master' parent node: '/ > hypertable' - HYPERSPACE file not found > at void > Hyperspace::Master::open(Hyperspace::ResponseCallbackOpen*, uint64_t, > const char*, uint32_t, uint32_t, std::vector<Hyperspace::Attribute, > std::allocator<Hyperspace::Attribute> >&) (/opt/hypertable/ > hypertable-0.9.2.8-alpha/src/cc/Hyperspace/Master.cc:877) > > What does HYPERSPACE file not found mean? > > There exists a directory /hypertable at HDFS at the master with 777 > permission. > > Please help. > > On May 29, 12:00 am, Harshada <[email protected]> wrote: > > Thanks for your reply Sanjit. > > > > 'cap cleandb' gives following output: (with one error at the end) > > > > * executing `stop' > > ** transaction: start > > * executing `stop_slaves' > > * executing "/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/ > > stop-servers.sh --no-hyperspace" > > servers: ["master", "slave"] > > [master] executing command > > ** [out :: master] Sending shutdown command > > ** [out :: master] Unable to establish connection to range server > > [slave] executing command > > ** [out :: slave] Sending shutdown command > > ** [out :: slave] Unable to establish connection to range server > > ** [out :: master] Shutdown range server complete > > ** [out :: slave] Shutdown range server complete > > ** [out :: master] Shutdown DFS broker complete > > ** [out :: master] Shutdown thrift broker complete > > ** [out :: master] Shutdown hypertable master complete > > ** [out :: slave] Shutdown thrift broker complete > > ** [out :: slave] Shutdown DFS broker complete > > ** [out :: slave] Shutdown hypertable master complete > > command finished > > * executing `stop_master' > > * executing "/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/ > > stop-servers.sh --no-hyperspace" > > servers: ["master"] > > [master] executing command > > ** [out :: master] Sending shutdown command > > ** [out :: master] Unable to establish connection to range server > > ** [out :: master] Shutdown range server complete > > ** [out :: master] Shutdown DFS broker complete > > ** [out :: master] Shutdown thrift broker complete > > ** [out :: master] Shutdown hypertable master complete > > command finished > > * executing `stop_hyperspace' > > * executing "/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/ > > stop-hyperspace.sh" > > servers: ["master"] > > [master] executing command > > ** [out :: master] Killing Hyperspace.pid 14096 > > *** [err :: master] /opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/ > > bin/ht-env.sh: line 68: kill: (14096) - No such process > > ** [out :: master] Shutdown hyperspace complete > > command finished > > ** transaction: commit > > harsh...@erts-server:/opt/hypertable/hypertable-0.9.2.8-alpha/conf$ / > > var/lib/gems/1.8/bin/cap cleandb > > * executing `cleandb' > > ** transaction: start > > * executing `clean_ranges' > > * executing "/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/ > > start-dfsbroker.sh hadoop --config=/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/conf/hypertable.cfg && /opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/clean-database.sh;" > > servers: ["master", "slave"] > > [master] executing command > > [slave] executing command > > ** [out :: master] DFS broker: available file descriptors: 1024 > > ** [out :: slave] DFS broker: available file descriptors: 1024 > > ** [out :: slave] Started DFS Broker (hadoop) > > ** [out :: slave] Removed /hypertable/servers in DFS > > ** [out :: slave] Removed /hypertable/tables in DFS > > ** [out :: slave] Cleared hyperspace > > ** [out :: slave] Killing DfsBroker.hadoop.pid 24587 > > ** [out :: slave] Shutdown hyperspace complete > > ** [out :: slave] Shutdown range server complete > > ** [out :: slave] Shutdown thrift broker complete > > ** [out :: slave] Shutdown hypertable master complete > > ** [out :: slave] Shutdown DFS broker complete > > ** [out :: master] Waiting for DFS Broker (hadoop) to come up... > > ** [out :: master] Waiting for DFS Broker (hadoop) to come up... > > ** [out :: master] Waiting for DFS Broker (hadoop) to come up... > > ** [out :: master] Waiting for DFS Broker (hadoop) to come up... > > ** [out :: master] Waiting for DFS Broker (hadoop) to come up... > > ** [out :: master] Waiting for DFS Broker (hadoop) to come up... > > ** [out :: master] Waiting for DFS Broker (hadoop) to come up... > > ** [out :: master] Waiting for DFS Broker (hadoop) to come up... > > ** [out :: master] ERROR: DFS Broker (hadoop) did not come up > > ** [out :: master] DfsBroker.hadoop appears to be running (19475): > > ** [out :: master] harshada 19475 19394 0 May28 ? 00:00:00 java - > > classpath /opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8:/opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/*.jar:/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/commons-logging-1.0.4.jar:/ > > opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/hadoop-0.20.1- > > core.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/ > > hypertable-0.9.2.8.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/ > > 0.9.2.8/lib/java/junit-4.3.1.jar:/opt/hypertable/hypertable-0.9.2.8- > > alpha/0.9.2.8/lib/java/libthrift-0.2.0.jar:/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/log4j-1.2.13.jar:/opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/slf4j- > > api-1.5.8.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/ > > java/slf4j-log4j12-1.5.8.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/ > > 0.9.2.8/lib/jetty-ext/*.jar org.hypertable.DfsBroker.hadoop.main -- > > verbose --config=/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/conf/ > > hypertable.cfg > > command finished > > failed: "sh -c '/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/ > > start-dfsbroker.sh hadoop --config=/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/conf/hypertable.cfg && /opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/clean-database.sh;'" > > on master > > > > But going ahead when I run 'cap start', everything else starts up > > except slaves's RangeServer. Here's the output: > > > > * executing `start' > > ** transaction: start > > * executing `start_hyperspace' > > * executing "/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/ > > start-hyperspace.sh --config=/opt/hypertable/hypertable-0.9.2.8- > > alpha/0.9.2.8/conf/hypertable.cfg" > > servers: ["master"] > > [master] executing command > > ** [out :: master] Started Hyperspace > > command finished > > * executing `start_master' > > * executing "/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/ > > start-dfsbroker.sh hadoop --config=/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/conf/hypertable.cfg &&\\\n /opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/start-master.sh -- > > config=/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/conf/ > > hypertable.cfg" > > servers: ["master"] > > [master] executing command > > ** [out :: master] DFS broker: available file descriptors: 1024 > > ** [out :: master] DfsBroker.hadoop appears to be running (19475): > > ** [out :: master] harshada 19475 1 0 May28 ? 00:00:01 java - > > classpath /opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8:/opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/*.jar:/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/commons-logging-1.0.4.jar:/ > > opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/hadoop-0.20.1- > > core.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/ > > hypertable-0.9.2.8.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/ > > 0.9.2.8/lib/java/junit-4.3.1.jar:/opt/hypertable/hypertable-0.9.2.8- > > alpha/0.9.2.8/lib/java/libthrift-0.2.0.jar:/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/log4j-1.2.13.jar:/opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/slf4j- > > api-1.5.8.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/ > > java/slf4j-log4j12-1.5.8.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/ > > 0.9.2.8/lib/jetty-ext/*.jar org.hypertable.DfsBroker.hadoop.main -- > > verbose --config=/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/conf/ > > hypertable.cfg > > ** [out :: master] Started Hypertable.Master > > command finished > > * executing `start_slaves' > > * executing "/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/ > > random-wait.sh 5 &&\\\n /opt/hypertable/hypertable-0.9.2.8-alpha/ > > 0.9.2.8/bin/start-dfsbroker.sh hadoop --config=/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/conf/hypertable.cfg &&\\\n /opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/bin/start- > > rangeserver.sh --config=/opt/hypertable/hypertable-0.9.2.8-alpha/ > > 0.9.2.8/conf/hypertable.cfg &&\\\n /opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/bin/start-thriftbroker.sh -- > > config=/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/conf/ > > hypertable.cfg" > > servers: ["master", "slave"] > > [master] executing command > > [slave] executing command > > ** [out :: master] DFS broker: available file descriptors: 1024 > > ** [out :: master] DfsBroker.hadoop appears to be running (19475): > > ** [out :: master] harshada 19475 1 0 May28 ? 00:00:01 java - > > classpath /opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8:/opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/*.jar:/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/commons-logging-1.0.4.jar:/ > > opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/hadoop-0.20.1- > > core.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/ > > hypertable-0.9.2.8.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/ > > 0.9.2.8/lib/java/junit-4.3.1.jar:/opt/hypertable/hypertable-0.9.2.8- > > alpha/0.9.2.8/lib/java/libthrift-0.2.0.jar:/opt/hypertable/ > > hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/log4j-1.2.13.jar:/opt/ > > hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/java/slf4j- > > api-1.5.8.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/lib/ > > java/slf4j-log4j12-1.5.8.jar:/opt/hypertable/hypertable-0.9.2.8-alpha/ > > 0.9.2.8/lib/jetty-ext/*.jar org.hypertable.DfsBroker.hadoop.main -- > > verbose --config=/opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/conf/ > > hypertable.cfg > > ** [out :: slave] DFS broker: available file descriptors: 1024 > > ** [out :: master] Started Hypertable.RangeServer > > ** [out :: slave] Started DFS Broker (hadoop) > > ** [out :: master] Started ThriftBroker > > ** [out :: slave] Waiting for Hypertable.RangeServer to come up... > > ** [out :: slave] Waiting for Hypertable.RangeServer to come up... > > *** [err :: slave] /opt/hypertable/hypertable-0.9.2.8-alpha/0.9.2.8/ > > bin/ht-env.sh: line 95: 25904 Segmentation fault (core dumped) > > $HEAPCHECK $VALGRIND $HYPERTABLE_HOME/bin/$servercmd --pidfile > > $pidfile "$@" >&$logfile > > ** [out :: slave] Waiting for Hypertable.RangeServer to come up... > > ** [out :: slave] Waiting for > > ... > > > > read more ยป > > -- > You received this message because you are subscribed to the Google Groups > "Hypertable Development" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<hypertable-dev%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/hypertable-dev?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
