Also, I should point out that if you cannot (or do not want to) correct the problem in HBase, you can instead run HBase in a separate process, and communicate to it using RMI. Both the FileNet and Documentum connectors use this strategy. It's usually reserved for situations where jar dependencies are incompatible with the rest of ManifoldCF, or when there are native code dependencies, but this would also be a valid technique if you find you cannot get HBase to work properly in anything other than a very specific environment.
Karl On Mon, Apr 8, 2013 at 5:39 AM, Karl Wright <[email protected]> wrote: > Hi Minoru, > > The connector-lib jars are loaded using a different classloader than the > core jars in MCF. If hbase is coded properly, it should have no trouble > finding resources in those jars. But if the code in hbase tries to do > something tricky, I can imagine that it might not work. Or, hbase may just > not be coded correctly in this regard. Either way, if you understand how > the resource is being loaded we can either file a ticket/patch with hbase, > or change manifoldcf code, whichever is appropriate. > > I suggest that you try to find the part of the hbase code that reads the > configuration file as a resource. Compare this against code that works. > The SharePoint connector, for instance, has a wsdd resource that it loads > which is included in the connector jar file which loads just fine. > > Thanks! > Karl > > > On Mon, Apr 8, 2013 at 5:17 AM, Minoru Osuka <[email protected]>wrote: > >> Hi, >> >> I'm trying to make HBase Connector for ManiforldCF. >> I had put hbase.jar to connector-lib directory, so I can use HBase's >> classes in ManifoldCF. >> But, It seems that org.apache.hadoop.hbase.HBaseConfiguration.create() >> can't read hbase-default.xml in hbase.jar. >> It can't get hbase.defaults.for.version in hbase-default.xml, So >> org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(Configuration >> conf) throws java.lang.RuntimeException: hbase-default.xml file seems to be >> for and old version of HBase (null), this version is 0.94.3. >> >> I had made CLI application and Jetty Servlet using >> org.apache.hadoop.hbase.HBaseConfiguration.create() in the same way. It >> works. >> >> How should I load hbase.jar to read inner hbase-default.xml? >> Please see following sample code and exception, and give me some advice. >> >> >> >> SOURCE CODE: >> >> public class HBaseRepositoryConnector extends BaseRepositoryConnector { >> : >> protected Configuration config = null; >> : >> public void connect(ConfigParams configParams) { >> >> super.connect(configParams); >> >> config = HBaseConfiguration.create(); >> : >> } >> : >> } >> >> >> >> EXCEPTION: >> >> 43577 [qtp1940931807-189] WARN org.eclipse.jetty.servlet.ServletHandler - >> /mcf-crawler-ui/execute.jsp >> org.apache.jasper.JasperException: An exception occurred processing JSP >> page /viewconnection.jsp at line 101 >> >> 98: String connectionStatus; >> 99: try >> 100: { >> 101: IRepositoryConnector c = >> RepositoryConnectorFactory.grab(threadContext,className,parameters,maxCount); >> 102: if (c == null) >> 103: connectionStatus = >> Messages.getString(pageContext.getRequest().getLocale(),"viewconnection.Connectorisnotinstalled"); >> 104: else >> >> >> Stacktrace: >> at >> org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:521) >> at >> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:430) >> at >> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313) >> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:547) >> at >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:480) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) >> at >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:520) >> at >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:941) >> at >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:409) >> at >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >> at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:288) >> at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115) >> at >> org.apache.jasper.runtime.PageContextImpl.doForward(PageContextImpl.java:709) >> at >> org.apache.jasper.runtime.PageContextImpl.forward(PageContextImpl.java:680) >> at org.apache.jsp.execute_jsp._jspService(execute_jsp.java:382) >> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at >> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388) >> at >> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313) >> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:547) >> at >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:480) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) >> at >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:520) >> at >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:941) >> at >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:409) >> at >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >> at >> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110) >> at org.eclipse.jetty.server.Server.handle(Server.java:349) >> at >> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:441) >> at >> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:936) >> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:801) >> at >> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:224) >> at >> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:51) >> at >> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:586) >> at >> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:44) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533) >> at java.lang.Thread.run(Thread.java:680) >> Caused by: java.lang.RuntimeException: hbase-default.xml file seems to be >> for and old version of HBase (null), this version is 0.94.3 >> at >> org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfiguration.java:68) >> at >> org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfiguration.java:100) >> at >> org.apache.hadoop.hbase.HBaseConfiguration.create(HBaseConfiguration.java:111) >> at >> org.apache.manifoldcf.crawler.connectors.hbase.HBaseRepositoryConnector.connect(HBaseRepositoryConnector.java:142) >> at >> org.apache.manifoldcf.crawler.interfaces.RepositoryConnectorFactory$Pool.getConnector(RepositoryConnectorFactory.java:619) >> at >> org.apache.manifoldcf.crawler.interfaces.RepositoryConnectorFactory.grab(RepositoryConnectorFactory.java:384) >> at >> org.apache.jsp.viewconnection_jsp._jspService(viewconnection_jsp.java:318) >> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at >> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388) >> ... 47 more >> >> >> Thanks, >> Minoru >> >> >> >> Minoru Osuka >> [email protected] >> >> >> >> >> >
