Hi Laurent Yes crail://localhost:9060/mypath would be the path you want to use in your Spark program to access Crail. Did you check if your Crail deployment is up and running and can be accessed using the crail hdfs client (./bin/crail fs)? If the crail hdfs client works access via Spark should be a matter of setting the extra class path.
Please also take a look at https://incubator-crail.readthedocs.io/en/latest/config.html#configuration for examples how to configure Crail on a single machine or on a cluster. -Patrick On Mon, Apr 19, 2021 at 6:18 PM Laurent d'Orazio < laurent.dora...@univ-rennes1.fr> wrote: > Dear all, > My deepest apologies for a probably naive question :( > I want to test Crail instead of HDFS with a basic Spark test (in Python) > reading a very small (3 lines of 3 columns) cvs file (textFile) and saving > it in another file (with saveAsTextFile). > I have thus changed the path from hdfs://localhost:9000/<mypath> to > hdfs://localhost:9060/<mypath> > I have the enclosed error. > Please note that I tried using crail://localhost:9060/<mypath>, but in > that case I have a Java.io.IOException: No FileSystem for Scheme : crail. > Can you let me know what is my mistake? > Thanks in advance. > Best, > Laurent >
