RE: Getting Spark SQL talking to Sql Server

[Ashic Mahtab]

Thanks. Didn't know about jdbcrdd...should do nicely for now. The foreign data 
source api looks interesting...


Date: Tue, 21 Oct 2014 20:33:03 +0800
From: lian.cs@gmail.com
To: as...@live.com; user@spark.apache.org
Subject: Re: Getting Spark SQL talking to Sql Server


  

  
  
Instead of using Spark SQL, you can use JdbcRDD to extract data from
SQL server. Currently Spark SQL can't run queries against SQL
server. The foreign data source API planned in Spark 1.2 can make
this possible.



On 10/21/14 6:26 PM, Ashic Mahtab
  wrote:



  
  Hi,

Is there a simple way to run spark sql queries against Sql
Server databases? Or are we limited to running sql and doing
sc.Parallelize()? Being able to query small amounts of lookup
info directly from spark can save a bunch of annoying etl, and
I'd expect Spark Sql to have some way of doing this.



Cheers,

Ashic.

  


  

Re: Getting Spark SQL talking to Sql Server

[Cheng Lian]

Instead of using Spark SQL, you can use JdbcRDD to extract data from SQL 
server. Currently Spark SQL can't run queries against SQL server. The 
foreign data source API planned in Spark 1.2 can make this possible.


On 10/21/14 6:26 PM, Ashic Mahtab wrote:

Hi,
Is there a simple way to run spark sql queries against Sql Server 
databases? Or are we limited to running sql and doing 
sc.Parallelize()? Being able to query small amounts of lookup info 
directly from spark can save a bunch of annoying etl, and I'd expect 
Spark Sql to have some way of doing this.


Cheers,
Ashic.

Getting Spark SQL talking to Sql Server

[Ashic Mahtab]

Hi,
Is there a simple way to run spark sql queries against Sql Server databases? Or 
are we limited to running sql and doing sc.Parallelize()? Being able to query 
small amounts of lookup info directly from spark can save a bunch of annoying 
etl, and I'd expect Spark Sql to have some way of doing this.

Cheers,
Ashic.