It expects an iterable, and if you iterate over a string, you get the individual characters. Use a list instead: pyfiles=['/path/to/file']
On Mon, Jul 27, 2015 at 2:40 PM, Naveen Madhire <[email protected]> wrote: > Hi, > > I am running pyspark in windows and I am seeing an error while adding > pyfiles to the sparkcontext. below is the example, > > sc = SparkContext("local","Sample",pyFiles="C:/sample/yattag.zip") > > this fails with no file found error for "C" > > > The below logic is treating the path as individual files like "C", ":" "/" > etc. > > https://github.com/apache/spark/blob/master/python/pyspark/context.py#l195 > > > It works if I use Spark Conf, > > sparkConf.set("spark.submit.pyFiles",*"**C:/sample/yattag.zip**"*) > sc = SparkContext("local","Sample",conf=sparkConf) > > > Is this an existing issue or I am not including the files in correct way in > Spark Context? > > > Thanks. > > > > > > when I run this, I am getting > > -- www.skrasser.com <http://www.skrasser.com/?utm_source=sig>
