Hello,
I was trying to run hbase 1.2.4 (prerequisite for OpenTSDB) and I found
that, contrary to my previous experience with hbase on OS X, it does not
run out of the box (+ basic config). Instead, the java process only eats
all the CPU.
I tried to get a stack trace as suggested in the hbase mailing list (see
the discussion at the end of this email), but jstack does not seems to work.
This is the output (as also pasted in the link below):
$ /usr/java/bin/jstack -m 16177
Attaching to process ID 16177, please wait...
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.tools.jstack.JStack.runJStackTool(JStack.java:136)
at sun.tools.jstack.JStack.main(JStack.java:102)
Caused by: java.lang.UnsatisfiedLinkError: no saproc in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1889)
at java.lang.Runtime.loadLibrary0(Runtime.java:849)
at java.lang.System.loadLibrary(System.java:1088)
at
sun.jvm.hotspot.debugger.proc.ProcDebuggerLocal.<clinit>(ProcDebuggerLocal.java:660)
at
sun.jvm.hotspot.bugspot.BugSpotAgent.setupDebuggerSolaris(BugSpotAgent.java:634)
at
sun.jvm.hotspot.bugspot.BugSpotAgent.setupDebugger(BugSpotAgent.java:513)
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:492)
at
sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:331)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
... 6 more
Both java and jdk packages are installed from main OmniOS repo and
compiled for r151020, the OS version I'm running right now.
I would appreciate help in the solution of this problem so that the guys
from the hbase mailing list can pinpoint the issue about hbase under OmniOS.
Of course, if anyone has experience with hbase on OmniOS and has seen
similar issues, or is successfully running previous versions, feel free
to let me know. I haven't checked any previous version, I only tested
the current stable.
Thanks
Olaf
-------- Forwarded Message --------
Subject: Re: "Quickstart" on OmniOS/illumos not working, java eats CPU
Date: Sun, 13 Nov 2016 08:28:42 -0800
From: Ted Yu <[email protected]>
Reply-To: [email protected]
To: [email protected] <[email protected]>
bq. Caused by: java.lang.UnsatisfiedLinkError: no saproc in
java.library.path
Was the jstack of version 1.7.0_101 ?
I used google and found:
http://umchee.blogspot.com/p/i-recently-needed-to-investigate.html
but it was about Java 1.5
You can use this command to find the commandline for hbase process
(replace 16059
with the new process Id):
ps aux | grep 16059
Then use the jstack from same path as java to capture stack trace:
Thanks
On Sun, Nov 13, 2016 at 8:16 AM, Olaf Marzocchi <[email protected]> wrote:
I'm not 100% sure about the stack trace (not a developer myself), but I
searched online and used jstack:
http://pastebin.com/AT6pScBv
Additional info: after turning on DEBUG, the java process did not
immediately jump to 100% cpu as it used to do, it ramped up and it took 3
minutes to get there. It may however be a coincidence.
Hope this can help pinpointing the issue.
Olaf
On 13/11/2016 01:18, Ted Yu wrote:
Can you turn on DEBUG logging ?
In log4j.properties :
log4j.logger.org.apache.hadoop.hbase=DEBUG
Since this is reproducible, can you capture one or two stack traces before
killing the server ?
Thanks
On Sat, Nov 12, 2016 at 3:09 PM, Olaf Marzocchi <[email protected]>
wrote:
Hello,
I tested HBase with OpenTSDB some time ago on OS X locally and the
quickstart guide worked perfectly, merged with
http://opentsdb.net/setup-hbase.html where necessary.
I tried yesterday to do the same on my OmniOS/illumos server and I
encountered issues with HBase.
First of all I set "JAVA_HOME=/usr/java" in "conf/hbase-env.sh".
$ /usr/java/bin/java -version
openjdk version "1.7.0_101"
OpenJDK Runtime Environment (build 1.7.0_101-b00)
OpenJDK Server VM (build 24.101-b00, mixed mode)
I edited also "hbase-site.xml":
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///export/home/olaf/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/export/home/olaf/zookeeper</value>
</property>
</configuration>
I then launched "bin/start-hbase.sh" and it quits apparently normally. A
directory "/export/home/olaf/zookeeper" appeared, but no
/export/home/olaf/hbase.
$ ls -l /export/home/olaf/
drwxrwxr-x 3 olaf olaf 3 Nov 12 01:16 zookeeper
I also checked with prstat ("top") and I see that java is taking a full
CPU. After some minutes it's still crunching something.
If I launch "./bin/hbase shell" it never gets to "hbase(main):001:0>".
At this point the only way is to kill -9 both java processes: the stop
script doesn't work, it outputs dots forever.
I uploaded the two log files (.log and .out) to Pastebin:
http://pastebin.com/aFMXmWYU
I'm not sure where to begin with the debugging, since I did no
customisation at all and I don't see any error logged.
I would be grateful for some help.
Thanks
Olaf
_______________________________________________
OmniOS-discuss mailing list
[email protected]
http://lists.omniti.com/mailman/listinfo/omnios-discuss
