Depending on what you mean by "save," you might be able to use the Twitter Storehaus package to do this. There was a nice talk about this at a Spark meetup -- "Stores, Monoids and Dependency Injection - Abstractions for Spark Streaming Jobs." Video here: https://www.youtube.com/watch?v=C7gWtxelYNM&feature=youtu.be.
Jim Donahue Adobe -----Original Message----- From: Ron Gonzalez [mailto:zlgonza...@yahoo.com.INVALID] Sent: Wednesday, August 06, 2014 7:18 AM To: Vida Ha Cc: u...@spark.incubator.apache.org Subject: Re: Save an RDD to a SQL Database Hi Vida, It's possible to save an RDD as a hadoop file using hadoop output formats. It might be worthwhile to investigate using DBOutputFormat and see if this will work for you. I haven't personally written to a db, but I'd imagine this would be one way to do it. Thanks, Ron Sent from my iPhone > On Aug 5, 2014, at 8:29 PM, Vida Ha <vid...@gmail.com> wrote: > > > Hi, > > I would like to save an RDD to a SQL database. It seems like this would be a > common enough use case. Are there any built in libraries to do it? > > Otherwise, I'm just planning on mapping my RDD, and having that call a method > to write to the database. Given that a lot of records are going to be > written, the code would need to be smart and do a batch insert after enough > records have collected. Does that sound like a reasonable approach? > > > -Vida > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
