dtgateway does not use the native libraries. I just tried a clean install of RTS 3.5.0 community edition on plain vanilla hadoop 2.7.3 on ubuntu 14.04 and was able to launch the pi demo from the web interface. I was able to do the same even with the native libraries removed.
What is your fs.defaultFS property in hadoop? Can you verify that it points to your hdfs (hdfs://namenode:9000/) ? Unfortunately I don't have access to an ARM64 so any finding you post here would be helpful. David On Wed, Sep 28, 2016 at 11:04 AM, <[email protected]> wrote: > I had a thought of what might be the problem: > the hadoop native library doesn't build on SUSE/SLES ARM64, > so I'm running with the pure Java library. > > Is it possible that the dtgateway is trying to use the native library > directly? > > Thanks, > -david > > > On 2016-09-26 14:45, David Yan wrote: > >> Do you see any exception stacktrace in the log when this error >> occurred: >> >> Wrong FS: hdfs://namenode:9000/user/dtadmin/datatorrent, expected: >> file:/// >> >> David >> >> On Mon, Sep 26, 2016 at 2:39 PM, <[email protected]> wrote: >> >> From David Yan <[email protected]> >>>> Date Sat 00:12 >>>> >>>> Can you please provide any exception stacktrace in the >>>> dtgateway.log file when that happens? >>>> >>> >>> I reran the dtgateway installation wizard, which failed (same as >>> before) with error msg: >>> >>> | DFS directory cannot be written to with error message "Mkdirs >>> failed to create /user/dtadmin/datatorrent (exists=false, >>> cwd=file:/opt/datatorrent/releases/3.4.0)" >>> >>> The corresponding exception in dtgateway.log is: >>> >>> | 2016-09-26 13:43:54,767 ERROR com.datatorrent.gateway.I: DFS >>> Directory cannot be written to with exception: >>> | java.io.IOException: Mkdirs failed to create >>> /user/dtadmin/datatorrent (exists=false, >>> cwd=file:/opt/datatorrent/releases/3.4.0) >>> | at >>> >>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileS >> ystem.java:455) >> >>> | at >>> >>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileS >> ystem.java:440) >> >>> | at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911) >>> | at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892) >>> | at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:789) >>> | at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:778) >>> | at >>> com.datatorrent.stram.client.FSAgent.createFile(FSAgent.java:77) >>> | at com.datatorrent.gateway.I.h(gc:324) >>> | at com.datatorrent.gateway.I.h(gc:284) >>> | at >>> com.datatorrent.gateway.resources.ws.v2.ConfigResource.h(fc:136) >>> | at >>> com.datatorrent.gateway.resources.ws.v2.ConfigResource.h(fc:171) >>> | at >>> >>> com.datatorrent.gateway.resources.ws.v2.ConfigResource.setCo >> nfigProperty(fc:34) >> >>> | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> | at >>> >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >> ssorImpl.java:62) >> >>> | at >>> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >> thodAccessorImpl.java:43) >> >>> | at java.lang.reflect.Method.invoke(Method.java:498) >>> | at >>> >>> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invo >> ke(JavaMethodInvokerFactory.java:60) >> >>> | at >>> >>> com.sun.jersey.server.impl.model.method.dispatch.AbstractRes >> ourceMethodDispatchProvider$ResponseOutInvoker._dispatch(A >> bstractResourceMethodDispatchProvider.java:205) >> >>> | at >>> >>> com.sun.jersey.server.impl.model.method.dispatch.ResourceJav >> aMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) >> >>> | at >>> >>> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept( >> HttpMethodRule.java:288) >> >>> | at >>> >>> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accep >> t(RightHandPathRule.java:147) >> >>> | at >>> >>> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept( >> SubLocatorRule.java:134) >> >>> | at >>> >>> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accep >> t(RightHandPathRule.java:147) >> >>> | at >>> >>> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept( >> SubLocatorRule.java:134) >> >>> | at >>> >>> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accep >> t(RightHandPathRule.java:147) >> >>> | at >>> >>> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accep >> t(ResourceClassRule.java:108) >> >>> | at >>> >>> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accep >> t(RightHandPathRule.java:147) >> >>> | at >>> >>> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule >> .accept(RootResourceClassesRule.java:84) >> >>> | at >>> >>> com.sun.jersey.server.impl.application.WebApplicationImpl._h >> andleRequest(WebApplicationImpl.java:1469) >> >>> | at >>> >>> com.sun.jersey.server.impl.application.WebApplicationImpl._h >> andleRequest(WebApplicationImpl.java:1400) >> >>> | at >>> >>> com.sun.jersey.server.impl.application.WebApplicationImpl.ha >> ndleRequest(WebApplicationImpl.java:1349) >> >>> | at >>> >>> com.sun.jersey.server.impl.application.WebApplicationImpl.ha >> ndleRequest(WebApplicationImpl.java:1339) >> >>> | at >>> >>> com.sun.jersey.spi.container.servlet.WebComponent.service(We >> bComponent.java:416) >> >>> | at >>> >>> com.sun.jersey.spi.container.servlet.ServletContainer.servic >> e(ServletContainer.java:537) >> >>> | at >>> >>> com.sun.jersey.spi.container.servlet.ServletContainer.servic >> e(ServletContainer.java:699) >> >>> | at >>> javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >>> | at >>> >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669) >> >>> | at >>> >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan >> dler.java:457) >> >>> | at >>> >>> org.eclipse.jetty.server.session.SessionHandler.doHandle( >> SessionHandler.java:229) >> >>> | at >>> >>> org.eclipse.jetty.server.handler.ContextHandler.doHandle( >> ContextHandler.java:1075) >> >>> | at >>> >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand >> ler.java:384) >> >>> | at >>> >>> org.eclipse.jetty.server.session.SessionHandler.doScope( >> SessionHandler.java:193) >> >>> | at >>> >>> org.eclipse.jetty.server.handler.ContextHandler.doScope( >> ContextHandler.java:1009) >> >>> | at >>> >>> org.eclipse.jetty.server.handler.ScopedHandler.handle(Scoped >> Handler.java:135) >> >>> | at >>> >>> org.eclipse.jetty.server.handler.HandlerCollection.handle( >> HandlerCollection.java:154) >> >>> | at >>> >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(Handl >> erWrapper.java:116) >> >>> | at org.eclipse.jetty.server.Server.handle(Server.java:368) >>> | at >>> >>> org.eclipse.jetty.server.AbstractHttpConnection.handleReques >> t(AbstractHttpConnection.java:489) >> >>> | at >>> >>> org.eclipse.jetty.server.AbstractHttpConnection.content(Abst >> ractHttpConnection.java:953) >> >>> | at >>> >>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandl >> er.content(AbstractHttpConnection.java:1014) >> >>> | at >>> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861) >>> | at >>> >>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) >> >>> | at >>> >>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHtt >> pConnection.java:82) >> >>> | at >>> >>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(Select >> ChannelEndPoint.java:628) >> >>> | at >>> >>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectC >> hannelEndPoint.java:52) >> >>> | at >>> >>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Queued >> ThreadPool.java:608) >> >>> | at >>> >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedT >> hreadPool.java:543) >> >>> | at java.lang.Thread.run(Thread.java:745) >>> | 2016-09-26 13:43:54,772 INFO >>> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception >>> in processing web service: >>> com.datatorrent.stram.client.DTConfiguration$ConfigException: DFS >>> directory cannot be written to with error message "Mkdirs failed to >>> create /user/dtadmin/datatorrent (exists=false, >>> cwd=file:/opt/datatorrent/releases/3.4.0)" >>> >>> Also, when I shutdown the gateway, I get a different exception in >>> dtgateway.log: >>> >>> | 2016-09-26 13:40:44,366 INFO com.datatorrent.gateway.DTGateway: >>> Shutting down >>> | 2016-09-26 13:40:44,427 ERROR com.datatorrent.gateway.I: DFS >>> Directory cannot be written to with exception: >>> | java.io.IOException: Filesystem closed >>> | at >>> org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:808) >>> | at >>> org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:2041) >>> | at >>> >>> org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall( >> DistributedFileSystem.java:707) >> >>> | at >>> >>> org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall( >> DistributedFileSystem.java:703) >> >>> | at >>> >>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSyst >> emLinkResolver.java:81) >> >>> | at >>> >>> org.apache.hadoop.hdfs.DistributedFileSystem.delete(Distribu >> tedFileSystem.java:714) >> >>> | at >>> com.datatorrent.stram.client.FSAgent.deleteFile(FSAgent.java:94) >>> | at com.datatorrent.gateway.I.h(gc:30) >>> | at com.datatorrent.gateway.l.K(pc:158) >>> | at com.datatorrent.gateway.l.h(pc:65) >>> | at com.datatorrent.gateway.K.run(yc:156) >>> | at java.lang.Thread.run(Thread.java:745) >>> | 2016-09-26 13:40:44,444 INFO com.datatorrent.gateway.DTGateway: >>> Shutdown complete >>> >>> Thanks much. >>> -david >>> >>> On Tue, Sep 13, 2016 at 4:06 PM, <[email protected]> wrote: >>>> >>>> I have the dtgateway running (community edition) with >>>>> >>>> apex/malhar 3.4 and hadoop 2.7.4 on SuSE SLES 12.1 on an ARM64 >>>> server (aarch64). >>>> >>>>> >>>>> In the Installation Wizard, I set the DFS location to: >>>>> >>>>> hdfs://namenode:9000/user/dtadmin/datatorrent >>>>> >>>>> and the gateway saves the hadoop configuration and restarts >>>>> >>>> successfully, but I get an error: >>>> >>>>> >>>>> Wrong FS: hdfs://namenode:9000/user/dtadmin/datatorrent, >>>>> >>>> expected: file:/// >>>> >>>>> >>>>> the 'service dtgateway status' command says it's running. >>>>> The log file has the same error as above, but nothing else >>>>> >>>> useful. >>>> >>>>> >>>>> Is there a manual way to move the demo apps into the >>>>> >>>> gateway? >>>> >>>>> >>>>> If I ignore this error and try to upload or import a demo >>>>> >>>> app, I get the same error. >>>> >>>>> >>>>> What's odd is that the gateway does write into the HDFS >>>>> >>>> directory: >>>> >>>>> >>>>> Found 6 items >>>>> drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35 >>>>> >>>> /user/dtadmin/datatorrent/appPackages >>>> >>>>> drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35 >>>>> >>>> /user/dtadmin/datatorrent/apps >>>> >>>>> drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35 >>>>> >>>> /user/dtadmin/datatorrent/conf >>>> >>>>> drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35 >>>>> >>>> /user/dtadmin/datatorrent/dashboards >>>> >>>>> drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35 >>>>> >>>> /user/dtadmin/datatorrent/licenses >>>> >>>>> drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35 >>>>> >>>> /user/dtadmin/datatorrent/systemAlerts >>>> >>>>> >>>>> >>>>> I've tried playing with different ways to configure the DFS >>>>> >>>> location, but nothing works. >>>> >>>>> If I use an actual local filesystem directory, I get a >>>>> >>>> different errro from the installation wizard (Non-DFS file system >>>> is used: org.apache.hadoop.fs.LocalFileSystem), which makes sense. >>>> But if I ignore this error, the app wizard successsfully writes >>>> packages into the local directory, but I can't launch them (no >>>> surprise there). >>>> >>>>> >>>>> Anybody have any ideas what I can do? >>>>> >>>>> >>>>> thanks much. >>>>> -david >>>>> >>>> >
