!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Save-an-RDD-to-a-SQL-Database-tp11516p12921.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e
.youtube.com/watch?v=C7gWtxelYNM&feature=youtu.be.
Jim Donahue
Adobe
-Original Message-
From: Ron Gonzalez [mailto:zlgonza...@yahoo.com.INVALID]
Sent: Wednesday, August 06, 2014 7:18 AM
To: Vida Ha
Cc: u...@spark.incubator.apache.org
Subject: Re: Save an RDD to a SQL Database
Hi Vida,
That's a good idea - to write to files first and then load. Thanks.
On Thu, Aug 7, 2014 at 11:26 AM, Flavio Pompermaier
wrote:
> Isn't sqoop export meant for that?
>
>
> http://hadooped.blogspot.it/2013/06/apache-sqoop-part-3-data-transfer.html?m=1
> On Aug 7, 2014 7:59 PM, "Nicholas Chammas"
Isn't sqoop export meant for that?
http://hadooped.blogspot.it/2013/06/apache-sqoop-part-3-data-transfer.html?m=1
On Aug 7, 2014 7:59 PM, "Nicholas Chammas"
wrote:
> Vida,
>
> What kind of database are you trying to write to?
>
> For example, I found that for loading into Redshift, by far the ea
Vida,
What kind of database are you trying to write to?
For example, I found that for loading into Redshift, by far the easiest
thing to do was to save my output from Spark as a CSV to S3, and then load
it from there into Redshift. This is not a slow as you think, because Spark
can write the outp
The use case I was thinking of was outputting calculations made in Spark
into a SQL database for the presentation layer to access. So in other
words, having a Spark backend in Java that writes to a SQL database and
then having a Rails front-end that can display the data nicely.
On Thu, Aug 7, 20
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Save-an-RDD-to-a-SQL-Database-tp11516p11672.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e
On Thu, Aug 7, 2014 at 11:25 AM, Cheng Lian wrote:
> Maybe a little off topic, but would you mind to share your motivation of
> saving the RDD into an SQL DB?
Many possible reasons (Vida, please chime in with yours!):
- You have an existing database you want to load new data into so
ever
On Thu, Aug 7, 2014 at 11:08 AM, 诺铁 wrote:
> what if network broken in half of the process? should we drop all data in
> database and restart from beginning?
The best way to deal with this -- which, unfortunately, is not commonly
supported -- is with a two-phase commit that can span connection
”)
// Torture them until they tell you the truth :)
sql(“SELECT fieldA FROM myTable WHERE fieldB > 10”)
On Aug 6, 2014, at 11:29 AM, Vida Ha wrote:
>
> Hi,
>
> I would like to save an RDD to a SQL database. It seems like this would be a
> common enough use case. Are
re than 1 insert at a time.
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Save-an-RDD-to-a-SQL-Database-tp11516p11549.html
>> Sent from the Apache Spark User List mailing list arch
o open as few connections as
> possible but write more than 1 insert at a time.
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Save-an-RDD-to-a-SQL-Database-tp11
https://groups.google.com/forum/#!topic/spark-users/LUb7ZysYp2k
Basically the short story is that you want to open as few connections as
possible but write more than 1 insert at a time.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Save-an-RDD-to-a-SQL-Dat
n
Sent from my iPhone
> On Aug 5, 2014, at 8:29 PM, Vida Ha wrote:
>
>
> Hi,
>
> I would like to save an RDD to a SQL database. It seems like this would be a
> common enough use case. Are there any built in libraries to do it?
>
> Otherwise, I'm just pla
Hi,
I would like to save an RDD to a SQL database. It seems like this would be
a common enough use case. Are there any built in libraries to do it?
Otherwise, I'm just planning on mapping my RDD, and having that call a
method to write to the database. Given that a lot of records are goi
15 matches
Mail list logo
