We use the mapr rpm and have successfully read and written hdfs data. Are you using custom readers/writers? Maybe the relevant stacktrace might help.
Maybe also try a standard text reader and writer to see if there is a basic issue with accessing mfs? -Suren On Mon, May 26, 2014 at 11:31 AM, nelson <nelson.verd...@ysance.com> wrote: > thanks for replying guys. > > Mayur: > Indeed i tried the local mode (sparkmaster: local[5]) before and the > application runs well, no serialization problem. The problem arises as soon > as i try to run the app over the cluster. > > Surendranauth: > I just double checked my spark_classpath from spark_env.sh and we have a > similar one. > Is your team running spark with the mapr rpm [1]? Or did you compile it > yourself? > > [1]: http://doc.mapr.com/display/MapR/Installing+Spark+and+Shark > <http://doc.mapr.com/display/MapR/Installing+Spark+and+Shark> > > Thanks > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/maprfs-and-spark-libraries-tp6392p6400.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > -- SUREN HIRAMAN, VP TECHNOLOGY Velos Accelerating Machine Learning 440 NINTH AVENUE, 11TH FLOOR NEW YORK, NY 10001 O: (917) 525-2466 ext. 105 F: 646.349.4063 E: suren.hiraman@v <suren.hira...@sociocast.com>elos.io W: www.velos.io
