I restarted my zookeeper to get rid of all the accumulo entries in the
zookeeper. Now I am getting this error in my monitor debug log:
2016-10-07 10:07:46,548 [impl.ThriftScanner] DEBUG: Failed to locate
tablet for table : !0 row : ~err_\00
2016-10-07 10:07:49,781 [monitor.Monitor] INFO : Failed to obtain problem
reports
java.lang.RuntimeException:
org.apache.accumulo.core.client.impl.ThriftScanner$ScanTimedOutException
at
org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:161)
at
org.apache.accumulo.server.problems.ProblemReports$3.hasNext(ProblemReports.java:252)
at
org.apache.accumulo.server.problems.ProblemReports.summarize(ProblemReports.java:310)
at org.apache.accumulo.monitor.Monitor.fetchData(Monitor.java:371)
at org.apache.accumulo.monitor.Monitor$2.run(Monitor.java:511)
at
org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.accumulo.core.client.impl.ThriftScanner$ScanTimedOutException
at
org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:230)
at
org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:80)
at
org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:151)
... 6 more
Best regards,
Yamini Joshi
On Fri, Oct 7, 2016 at 10:08 AM, Sean Busbey <[email protected]> wrote:
> tracers used to be under the instance id in 1.6. I can confirm that in
> 1.7.2 it is definitely at /tracers. I just ran into this issue last
> week.
>
> @Yamini where do you get the Node does not exist error? Could you
> copy/paste the whole stacktrace? When I followed the same workaround
> Josh mentioned, restarting the Trace servers was sufficient to have
> the znode recreated.
>
> On Fri, Oct 7, 2016 at 10:01 AM, Josh Elser <[email protected]> wrote:
> > I had pulled up 1.7.2 and I thought /tracers was at the root of ZooKeeper
> > (not within the /accumulo/$instance.id znode). Maybe I looked at the
> wrong
> > code though?
> >
> > Michael Wall wrote:
> >>
> >> Maybe this is already known, so apologies in advance. But don't you
> >> need to do
> >>
> >> $ $ZOOKEEPER_HOME/bin/zkCli.sh -server m4:2181,m5:2181,m6:2181
> >>>
> >>> rmr accumulo/WHATEVER-INSTANCE-ID/tracers
> >>
> >>
> >> instead of rmr /tracers
> >>
> >> If you don't know the instance id, I you can do the following in
> zookeeper
> >>
> >> get /accumulo/instances/INSTANCE_NAME
> >>
> >>
> >>
> >> On Thu, Oct 6, 2016 at 6:26 PM, Josh Elser <[email protected]
> >> <mailto:[email protected]>> wrote:
> >>
> >> Uh, I am confused now. /tracers/ is the default parent znode that
> >> the Accumulo Tracer process registers itself in. We can also see
> >> this is what was happening with your snippet.
> >>
> >> I don't understand how that would not exist. Have you properly
> >> configured ZooKeeper to act as a quorom? (e.g. if you zkCli.sh
> >> -server to your ZK servers individually, do you see a consistent
> >> view of ZK?)
> >>
> >> You can try to remove the directory in HDFS you configured Accumulo
> >> to use, and re `accumulo init` to get back to an initial state. I'm
> >> not sure how you got into this situation to begin with.
> >>
> >> Yamini Joshi wrote:
> >>
> >> 1.7.2
> >>
> >>
> >> Best regards,
> >> Yamini Joshi
> >>
> >> On Thu, Oct 6, 2016 at 4:17 PM, Josh Elser <
> [email protected]
> >> <mailto:[email protected]>
> >> <mailto:[email protected] <mailto:[email protected]>>>
> >> wrote:
> >>
> >> Hrm, maybe I am looking at a newer version of Accumulo than
> >> what
> >> you're using. What version are you on?
> >>
> >> Yamini Joshi wrote:
> >>
> >> Thank you for replying Josh.
> >> I am getting this error:
> >> Node does not exist: /tracers
> >>
> >> I did ls and saw accumulo there. Should I delete that?
> >> When I try to delete accumulo, it gives me a
> >> Authentication is
> >> not valid
> >> error.
> >>
> >> Best regards,
> >> Yamini Joshi
> >>
> >> On Thu, Oct 6, 2016 at 2:57 PM, Josh Elser
> >> <[email protected] <mailto:[email protected]>
> >> <mailto:[email protected] <mailto:[email protected]>>
> >> <mailto:[email protected] <mailto:[email protected]>
> >> <mailto:[email protected] <mailto:[email protected]>>>>
> >> wrote:
> >>
> >> Hi Yamini,
> >>
> >> The /tracers node in ZooKeeper was likely
> >> initialized
> >> previously
> >> using an Accumulo configuration that has a
> >> different value for
> >> instance.secret. You can delete this znode to move
> >> forward
> >> if you
> >> have no other Accumulo instance you intend to run
> >> on this
> >> ZK instance.
> >>
> >> $ $ZOOKEEPER_HOME/bin/zkCli.sh -server
> >> m4:2181,m5:2181,m6:2181
> >> > rmr /tracers
> >>
> >> You can also set a different value for
> >> trace.zookeeper.path in
> >> accumulo-site.xml. After doing this, you can try
> >> running a
> >> start-all.sh again to start the tracer.
> >>
> >> You also mentioned that the Monitor did not start
> >> but did not
> >> include an exception for that service. The error
> you
> >> included only
> >> shows that the Accumulo Master/TabletServer did
> not
> >> correctly start
> >> and assign the system tables (accumulo.root and
> >> accumulo.metadata).
> >>
> >>
> >> Yamini Joshi wrote:
> >>
> >> Hello all
> >>
> >> I am trying to configure a multinode cluster
> >> with
> >> accumulo. I am
> >> using 3
> >> machines with the config as follows:
> >>
> >> M4: Hadoop Master/Acc Slave
> >> M5: Accumulo Master/Monitor/Gc
> >> M6: Slave
> >>
> >> I have configured zookeeper ensemble on the
> >> cluster
> >> with all the 3
> >> nodes. The Hadoop part seems to work fine, I
> >> can see
> >> the console
> >> and run
> >> MR jobs easily. I can also see the zookeeper
> >> cluster
> >> with M2 as the
> >> leader. My problem is with accumulo, after
> >> accumulo init:
> >> bin/accumulo init
> >> which seems to run without any error, after
> >> I isuue
> >> the command
> >> bin/start-all.sh
> >> the Monitor process can't start. I am
> >> getting this
> >> error in
> >> my tracer log:
> >> [start.Main] ERROR: Thread 'tracer' died.
> >>
> >> org.apache.zookeeper.KeeperException$NoAuthException:
> >> KeeperErrorCode =
> >> NoAuth for /tracers/trace-
> >> at
> >>
> >>
> >>
> >> org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> >> at
> >>
> >>
> >>
> >> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> >> at
> >> org.apache.zookeeper.ZooKeeper.create(ZooKeeper.
> java:783)
> >> at
> >>
> >>
> >>
> >> org.apache.accumulo.fate.zookeeper.ZooUtil.putEphemeralSequential(
> ZooUtil.java:464)
> >> at
> >>
> >>
> >>
> >> org.apache.accumulo.fate.zookeeper.ZooReaderWriter.
> putEphemeralSequential(ZooReaderWriter.java:99)
> >> at
> >>
> >>
> >>
> >> org.apache.accumulo.tracer.TraceServer.registerInZooKeeper(
> TraceServer.java:304)
> >> at
> >>
> >>
> >>
> >> org.apache.accumulo.tracer.TraceServer.<init>(TraceServer.java:241)
> >> at
> >>
> >>
> >> org.apache.accumulo.tracer.TraceServer.main(TraceServer.
> java:346)
> >> at
> >>
> >>
> >>
> >> org.apache.accumulo.tracer.TracerExecutable.execute(
> TracerExecutable.java:33)
> >> at
> >> org.apache.accumulo.start.Main$1.run(Main.java:120)
> >> at java.lang.Thread.run(Thread.
> java:745)
> >>
> >> And this in my monitor debug log:
> >> Failed to locate tablet for table : !0 row :
> >> ~err_\100
> >>
> >> I can see that the accumulo dir has been
> >> created in the
> >> HDFS.
> >> Can anyone
> >> please guide me set this up? I am attaching my
> >> config
> >> files here
> >> (Rest
> >> all the same generated as a result of
> >> bin_config file).
> >>
> >> Best regards,
> >> Yamini Joshi
> >>
> >>
> >>
> >>
> >
>
>
>
> --
> busbey
>
