Re: Using Spark on Azure Blob Storage

2015-06-25 Thread Daniel Haviv
:19 PM > To: Silvio Fiorito > Cc: user@spark.apache.org > Subject: Re: Using Spark on Azure Blob Storage > > Thank you guys for the helpful answers. > > Daniel > > On 25 ביוני 2015, at 21:23, Silvio Fiorito > wrote: > > Hi Daniel, > > As Peter pointe

RE: Using Spark on Azure Blob Storage

2015-06-25 Thread Jacob Kim
Fiorito Cc: user@spark.apache.org Subject: Re: Using Spark on Azure Blob Storage Thank you guys for the helpful answers. Daniel On 25 ביוני 2015, at 21:23, Silvio Fiorito mailto:silvio.fior...@granturing.com>> wrote: Hi Daniel, As Peter pointed out you need the hadoop-azure JAR as well as the

Re: Using Spark on Azure Blob Storage

2015-06-25 Thread Daniel Haviv
ell. The Power BI library is up > at http://spark-packages.org/package/granturing/spark-power-bi the Event Hubs > library should be up soon. > > Thanks, > Silvio > > From: Daniel Haviv > Date: Thursday, June 25, 2015 at 1:37 PM > To: "user@spark.apache.org" &g

Re: Using Spark on Azure Blob Storage

2015-06-25 Thread Silvio Fiorito
:37 PM To: "user@spark.apache.org<mailto:user@spark.apache.org>" Subject: Using Spark on Azure Blob Storage Hi, I'm trying to use spark over Azure's HDInsight but the spark-shell fails when starting: java.io.IOException: No FileSystem f

Re: Using Spark on Azure Blob Storage

2015-06-25 Thread Peter Rudenko
Hi Daniel, yes it supported, however you need to add hadoop-azure.jar to classpath of spark shell (http://search.maven.org/#search%7Cga%7C1%7Chadoop-azure - it's available only for hadoop-2.7.0). Try to find it on your node and run: export CLASSPATH=$CLASSPATH:hadoop-azure.jar && spark-shell

Using Spark on Azure Blob Storage

2015-06-25 Thread Daniel Haviv
Hi, I'm trying to use spark over Azure's HDInsight but the spark-shell fails when starting: java.io.IOException: No FileSystem for scheme: wasb at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.