Hi Jerry, We are in the process of creating connectors between Spark and Trafodion. Right now we have an example using JdbcRdd that Ming mentioned. Could you please share more about how you would like to use Spark & Trafodion together. Will you be interested in accessing Spark data from Trafodion or vice versa?
Ming, Spark SQL could complement Trafodion in many ways since it can (a) talk to wider variety of data sources than Trafodion can currently (b) has an API in Scala and (c) has a large set of ML and other functions which Trafodion currently does not. On the other hand Trafodion can provide transactions, can likely handle OLTP type queries better and possibly other benefits. I have not made a comparison, so this is a guess. Thanks Suresh On Thu, Jan 14, 2016 at 7:14 PM, Liu, Ming (Ming) <[email protected]> wrote: > Hi, Jerry, > > > > We used to have a Spark program using jdbcRDD to access Trafodion tables. > It is a standard way of Spark to access any other relational database as > well. But I never tried with Spark SQL. > > I know very little about Spark, so I think others in the list will give > you better advices. > > > > I don’t know your requirements, so maybe not make any sense to you : I > feel that is not necessary to run Spark SQL. Trafodion itself is a good SQL > database. it supports more SQL standard than Spark SQL, even with full ACID > transaction protection. So you can use SQL directly on Trafodion without a > need to run Spark SQL on Trafodion. > > > > Thanks, > > Ming > > > > *发件人:* Jerry He [mailto:[email protected]] > *发送时间:* 2016年1月15日 7:14 > *收件人:* [email protected] > *主题:* Spark integration with Trafodion > > > > Hi, > > > > Is there a Spark/Trafodion or SparkSQL/Trafodion connector available? How > do I use Trafodion with Spark SQL? > > > > Thanks. > > > > Jerry >
