My guess is there's actually a need to have the MapR hadoop common jar (the one I would assume has an updated file system implementation) be pre-pended to the class path. I don't have an installation in front of me, but I'd look into modifying slider-env or the like to prepend at least that one jar to the class path.
On Fri, Feb 17, 2017 at 12:19 PM, Nicolás Velásquez O. <[email protected]> wrote: > Hey, > It's great to initiate MapR to slider! > Unfortunately, following the suggestions did not work (remove the classpath > modification on slider-env.sh and copied mapr's jars to slider's lib > folder). > > I'm guessing support for parsing the fs.defaultFS string ("maprfs:///") has > to be added to Slider's code. Something similar to > https://issues.apache.org/jira/browse/FLINK-1006. > > Should I just open an incident on Jira? > > Thanks! > Nicolas > > > > On Fri, Feb 17, 2017 at 4:40 PM, Billie Rinaldi <[email protected]> > wrote: > > > I don't think anyone has tried running Slider on MapR yet, so you are > > breaking new ground. I would not try to change CLASSPATH in > slider-env.sh. > > Slider has a standalone lib directory, meaning that all of its required > > jars are in the lib dir (including hadoop jars). It might be the case > that > > more jars are needed from the mapr lib directory. I would try copying all > > the jars from the mapr lib directory to the slider lib directory and see > if > > that makes a difference. If it works, then you might be able to remove > some > > of the jars. > > > > On Fri, Feb 17, 2017 at 3:32 AM, Nicolás Velásquez O. < > [email protected]> > > wrote: > > > > > Hi, > > > > > > I'm having problems to run Solr using Apache Slider on MapR. > > Specifically, > > > when I try to install the slider package it fails because slider > doesn't > > > have support for maprfs: > > > $ slider install-package --replacepkg --name solr --package > > > ~/inst/solr/solr-slider/solr-on-yarn.zip > > > 2017-02-17 12:08:28,635 [main] INFO service.AbstractService - Service > > > Slider Client failed in state INITED; cause: java.io.IOException: No > > > FileSystem for scheme: maprfs > > > java.io.IOException: No FileSystem for scheme: maprfs > > > at > > > org.apache.hadoop.fs.FileSystem.getFileSystemClass( > FileSystem.java:2644) > > > at > > > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651) > > > at org.apache.hadoop.fs.FileSystem.access$200( > > FileSystem.java:92) > > > at > > > org.apache.hadoop.fs.FileSystem$Cache.getInternal( > FileSystem.java:2687) > > > at org.apache.hadoop.fs.FileSystem$Cache.get( > > FileSystem.java:2669) > > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) > > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170) > > > at > > > org.apache.slider.common.tools.CoreFileSystem.<init>( > > > CoreFileSystem.java:82) > > > at > > > org.apache.slider.common.tools.SliderFileSystem.<init>( > > > SliderFileSystem.java:38) > > > at > > > org.apache.slider.client.SliderClient.initHadoopBinding( > > > SliderClient.java:498) > > > at > > > org.apache.slider.client.SliderClient.serviceInit( > SliderClient.java:318) > > > at > > > org.apache.hadoop.service.AbstractService.init( > AbstractService.java:163) > > > at > > > org.apache.slider.core.main.ServiceLauncher.launchService( > > > ServiceLauncher.java:182) > > > at > > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly( > > > ServiceLauncher.java:475) > > > at > > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit( > > > ServiceLauncher.java:403) > > > at > > > org.apache.slider.core.main.ServiceLauncher.serviceMain( > > > ServiceLauncher.java:630) > > > at org.apache.slider.Slider.main(Slider.java:49) > > > Exception: java.io.IOException: No FileSystem for scheme: maprfs > > > 2017-02-17 12:08:28,637 [main] ERROR main.ServiceLauncher - Exception: > > > java.io.IOException: No FileSystem for scheme: maprfs > > > org.apache.hadoop.service.ServiceStateException: java.io.IOException: > No > > > FileSystem for scheme: maprfs > > > at > > > org.apache.hadoop.service.ServiceStateException.convert( > > > ServiceStateException.java:59) > > > at > > > org.apache.hadoop.service.AbstractService.init( > AbstractService.java:172) > > > at > > > org.apache.slider.core.main.ServiceLauncher.launchService( > > > ServiceLauncher.java:182) > > > at > > > org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly( > > > ServiceLauncher.java:475) > > > at > > > org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit( > > > ServiceLauncher.java:403) > > > at > > > org.apache.slider.core.main.ServiceLauncher.serviceMain( > > > ServiceLauncher.java:630) > > > at org.apache.slider.Slider.main(Slider.java:49) > > > Caused by: java.io.IOException: No FileSystem for scheme: maprfs > > > at > > > org.apache.hadoop.fs.FileSystem.getFileSystemClass( > FileSystem.java:2644) > > > at > > > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651) > > > at org.apache.hadoop.fs.FileSystem.access$200( > > FileSystem.java:92) > > > at > > > org.apache.hadoop.fs.FileSystem$Cache.getInternal( > FileSystem.java:2687) > > > at org.apache.hadoop.fs.FileSystem$Cache.get( > > FileSystem.java:2669) > > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) > > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170) > > > at > > > org.apache.slider.common.tools.CoreFileSystem.<init>( > > > CoreFileSystem.java:82) > > > at > > > org.apache.slider.common.tools.SliderFileSystem.<init>( > > > SliderFileSystem.java:38) > > > at > > > org.apache.slider.client.SliderClient.initHadoopBinding( > > > SliderClient.java:498) > > > at > > > org.apache.slider.client.SliderClient.serviceInit( > SliderClient.java:318) > > > at > > > org.apache.hadoop.service.AbstractService.init( > AbstractService.java:163) > > > ... 5 more > > > 2017-02-17 12:08:28,639 [main] INFO util.ExitUtil - Exiting with > status > > 56 > > > > > > I have tried (with no success) to copy the maprfs jar to the slider lib > > > path, and to set the CLASSPATH on slider-env.sh as: > > > export CLASSPATH=/opt/mapr/lib/maprfs-5.2.0-mapr.jar > > > and > > > export CLASSPATH=`hadoop classpath` > > > > > > Has anybody got apache slider to work on top of mapr? > > > > > > Thanks! > > > Nicolas > > > > > > > > > -- > Nicolás Velásquez O. > Mobile +41.767189348 > -- Sent from some random computer....
