Hari Sankar Sivarama Subramaniyan created HIVE-5796:
-------------------------------------------------------
Summary: percentjobcomplete returned by webhcat is null
Key: HIVE-5796
URL: https://issues.apache.org/jira/browse/HIVE-5796
Project: Hive
Issue Type: Bug
Components: WebHCat
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
Currently the WebHCat E2E test TestMapReduce_1 fails when comparing the job
status field percentComplete which is returned as null, the expected value is
"map 100% reduce 100%".
In the templeton.log I see the following message:
{noformat}
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob | Using Hadoop
Version: 0.23
INFO | 07 Nov 2013 17:14:08,106 |
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob |
WEBHCAT_CONF_DIR=null
WARN | 07 Nov 2013 17:14:08,106 |
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob | Could not find
D:\hdp\hcatalog-0.12.0.2.0.6.0-1007\bin\null\override-container-log4j.properties.
Monitoring of Hadoop jobs submitted through WebHCat may be affected.
{noformat}
To get past this problem I explicitly set the WEBHCAT_CONF_DIR environment
variable to HCATALOG_HOME/etc/webhcat where override-container-log4j.properties
is present.
After fixing the above and rerunning, I still see the following error:
{noformat}
INFO | 07 Nov 2013 18:29:39,517 |
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob |
WEBHCAT_CONF_DIR=D:\hdp\\hcatalog-0.12.0.2.0.6.0-1007\etc\webhcat
INFO | 07 Nov 2013 18:29:39,517 |
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob | Found
D:\hdp\hcatalog-0.12.0.2.0.6.0-1007\etc\webhcat\override-container-log4j.properties
to use for job submission.
ERROR | 07 Nov 2013 18:29:39,519 |
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob |
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob is not properly
initialized. Monitoring of Hadoop jobs submitted through WebHCat may be
affected.
java.lang.IllegalArgumentException: Pathname /d:/hadoop/hdfs/tmp from
d:/hadoop/hdfs/tmp is not a valid DFS filename.
at
org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:184)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:92)
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1106)
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1102)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1102)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1397)
at
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob$1.run(TempletonControllerJob.java:93)
at
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob$1.run(TempletonControllerJob.java:82)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob.copyLog4JtoFileSystem(TempletonControllerJob.java:82)
at
org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob.<clinit>(TempletonControllerJob.java:126)
at
org.apache.hive.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:104)
at
org.apache.hive.hcatalog.templeton.LauncherDelegator$1.run(LauncherDelegator.java:101)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.hive.hcatalog.templeton.LauncherDelegator.queueAsUser(LauncherDelegator.java:101)
at
org.apache.hive.hcatalog.templeton.LauncherDelegator.enqueueController(LauncherDelegator.java:82)
at
org.apache.hive.hcatalog.templeton.JarDelegator.run(JarDelegator.java:55)
at
org.apache.hive.hcatalog.templeton.Server.mapReduceJar(Server.java:690)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1480)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1411)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1360)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1350)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1360)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:384)
at org.apache.hadoop.hdfs.web.AuthFilter.doFilter(AuthFilter.java:85)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:477)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:349)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:449)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:925)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
at java.lang.Thread.run(Unknown Source)
{noformat}
I think the problem is due to the reliance on the value of hadoop.tmp.dir as a
temporary HDFS directory which is a temporary file location and applies to both
local file system and HDFS. There are some derived parameters that build on top
of this location assuming its local (eg. mapred.local.dir) and sometimes
assuming its on hdfs (eg. mapred.system.dir), in short it is very hairy to rely
on hadoop.tmp.dir.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
