Hey
I think its more of a classpath issue.
Can u pl check whether u r hadoop config (hadoop-site.xml , etc) files
are present in classpath when u start from webserver.
I dont think u wud have errors in hadoop log files, but its good to chk
Jani Arvonen wrote:
Oh yes, of course. Our web application throws this exception when trying
to use copyFromLocalFile() through DistributedFileSystem. The local file
im trying to copy to the HDFS do exists...
And we have also tried to use LocalFileSystem which works fine but that
is not exactly what we are trying to achieve :)
stacktrace:
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:155)
at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:119)
at
org.apache.hadoop.fs.ChecksumFileSystem.copyFromLocalFile(ChecksumFileSystem.java:576)
at
com.floobs.service.media.DefaultMediaStorage.copyFromLocalFile(DefaultMediaStorage.java:23)
at
com.floobs.service.media.DefaultMediaStorage.copyFromLocalFile(DefaultMediaStorage.java:18)
at
com.floobs.web.media.MediaUploadFormController.storeFile(MediaUploadFormController.java:155)
at com.floobs.web.media.MediaUploadFormController.access
$500(MediaUploadFormController.java:36)
at com.floobs.web.media.MediaUploadFormController
$2.processFile(MediaUploadFormController.java:120)
at com.floobs.web.media.MediaUploadFormController
$UploadFileProcessor.execute(MediaUploadFormController.java:144)
at
com.floobs.web.media.MediaUploadFormController.processFileUploads(MediaUploadFormController.java:114)
at
com.floobs.web.media.MediaUploadFormController.onSubmit(MediaUploadFormController.java:65)
at
org.springframework.web.servlet.mvc.SimpleFormController.processFormSubmission(SimpleFormController.java:267)
at
org.springframework.web.servlet.mvc.AbstractFormController.handleRequestInternal(AbstractFormController.java:250)
at
org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at
org.riotfamily.cachius.spring.CacheableControllerHandlerAdapter.handle(CacheableControllerHandlerAdapter.java:118)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:839)
at
org.riotfamily.common.web.servlet.ReloadableDispatcherServlet.doDispatch(ReloadableDispatcherServlet.java:107)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:425)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:691)
at
org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:594)
at
org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:505)
at
org.riotfamily.pages.component.impl.IncludeComponent.renderInternal(IncludeComponent.java:66)
at
org.riotfamily.pages.component.impl.AbstractComponent.render(AbstractComponent.java:98)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponent(AbstractRenderStrategy.java:243)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponentVersion(AbstractRenderStrategy.java:207)
at
org.riotfamily.pages.component.render.EditModeRenderStrategy.renderContainer(EditModeRenderStrategy.java:169)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponents(AbstractRenderStrategy.java:156)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponentList(AbstractRenderStrategy.java:137)
at
org.riotfamily.pages.component.render.EditModeRenderStrategy.renderComponentList(EditModeRenderStrategy.java:93)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.render(AbstractRenderStrategy.java:87)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.render(AbstractRenderStrategy.java:81)
at org.riotfamily.pages.component.ComponentListController
$1.doInTransactionWithoutResult(ComponentListController.java:202)
at
org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
at
org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:127)
at
org.riotfamily.pages.component.ComponentListController.handleRequest(ComponentListController.java:199)
at
org.riotfamily.cachius.spring.CacheableControllerHandlerAdapter.handle(CacheableControllerHandlerAdapter.java:118)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:839)
at
org.riotfamily.common.web.servlet.ReloadableDispatcherServlet.doDispatch(ReloadableDispatcherServlet.java:107)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:425)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:691)
at
org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:594)
at
org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:505)
at
org.riotfamily.common.web.view.freemarker.IncludeMethod.exec(IncludeMethod.java:37)
at
org.riotfamily.common.web.view.freemarker.AbstractSimpleMethod.exec(AbstractSimpleMethod.java:38)
at freemarker.core.MethodCall._getAsTemplateModel(MethodCall.java:93)
at freemarker.core.Expression.getAsTemplateModel(Expression.java:89)
at freemarker.core.Expression.getStringValue(Expression.java:93)
at freemarker.core.DollarVariable.accept(DollarVariable.java:76)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.Macro$Context.runMacro(Macro.java:164)
at freemarker.core.Environment.visit(Environment.java:537)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:128)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.Environment.process(Environment.java:176)
at freemarker.template.Template.process(Template.java:232)
at
org.springframework.web.servlet.view.freemarker.FreeMarkerView.processTemplate(FreeMarkerView.java:333)
at
org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView.doRender(RiotFreeMarkerView.java:113)
at org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView
$1.doInContext(RiotFreeMarkerView.java:93)
at
org.riotfamily.common.web.view.ViewContext.execute(ViewContext.java:45)
at
org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView.renderMergedTemplateModel(RiotFreeMarkerView.java:91)
at
org.springframework.web.servlet.view.AbstractTemplateView.renderMergedOutputModel(AbstractTemplateView.java:179)
at
org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:247)
at
org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView.render(RiotFreeMarkerView.java:75)
at
org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1123)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:860)
at
org.riotfamily.common.web.servlet.ReloadableDispatcherServlet.doDispatch(ReloadableDispatcherServlet.java:107)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:425)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.riotfamily.common.web.filter.PluginChain.doFilter(PluginChain.java:60)
at
org.riotfamily.pages.page.support.FolderFilterPlugin.doFilter(FolderFilterPlugin.java:82)
at
org.riotfamily.common.web.filter.PluginChain.doFilter(PluginChain.java:57)
at
org.riotfamily.riot.security.AccessControlFilterPlugin.doFilter(AccessControlFilterPlugin.java:58)
at
org.riotfamily.common.web.filter.PluginChain.doFilter(PluginChain.java:57)
at
org.riotfamily.common.web.filter.PluginFilter.doFilterInternal(PluginFilter.java:123)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.riotfamily.common.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:95)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
at org.apache.coyote.http11.Http11BaseProtocol
$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:619)
And our hadoop server logfile (datanode log) contains this several times
with different timestamp:
2007-08-21 00:00:23,393 INFO org.apache.hadoop.dfs.DataNode: using
BLOCKREPORT_INTERVAL of 3356962msec
2007-08-21 00:00:23,407 ERROR org.apache.hadoop.dfs.DataNode: Exception:
java.lang.NullPointerException
at org.apache.hadoop.dfs.FSDataset
$FSDir.getBlockInfo(FSDataset.java:148)
at org.apache.hadoop.dfs.FSDataset
$FSVolume.getBlockInfo(FSDataset.java:353)
at org.apache.hadoop.dfs.FSDataset
$FSVolumeSet.getBlockInfo(FSDataset.java:419)
at
org.apache.hadoop.dfs.FSDataset.getBlockReport(FSDataset.java:636)
at
org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:523)
at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1310)
at java.lang.Thread.run(Thread.java:619)
but im not sure if this has anything to do with this particular problem.
Any help would be really appreciated!
Best regards,
Jani
On Tue, 2007-08-21 at 11:53 -0500, Sagar Naik wrote:
Hello
Can u provide some more information.
Like a stackstrace from log files
- Sagar
Jani Arvonen wrote:
Hello everybody!
I've been trying to use hadoop distributed file system from my java
spring web application but without any good results :). We have one
server where hadoop namenode are datanode are succesfully running (so
they all are running on a single node). I managed to configure it with
the help of documentation page found from hadoop website.
Now i need to make my web application to use this distributed file
system running on the other server. I've tried to place the same
hadoop-site.xml found from the hadoop master server in the programs
classpath and using DistributedFileSystem but without no luck. Always
when trying to use for example copyFromLocalFile it throws a
NullPointerException. Im getting really lost with this issue because i
havent found any examples or tutorials related to this :)
I would really appreciate if somebody could help me so that i can
connect to this hadoop master server from my java spring web application
and to use distributed file system.
best regards,
Jani
--
This message has been scanned for viruses and
dangerous content and is believed to be clean.
