Hi,
I did some experiment with loading data from s3 into spark. I loaded data
from s3 using sc.textFile(....). Have a look at the following code snippet:
val csv = sc.textFile("s3n://mybucket/myfile.csv")
val rdd = csv.map(line => line.split(",").map(elem => elem.trim)) // my
data format is in CSV format, comma separated
.map (r => MyIbject(r(3), r(4).toLong, r(5).toLong, r(6))) //just map
it to the target object format
hope this helps,
best,
/Shahab
On Wed, Jul 8, 2015 at 12:57 AM, spark user <[email protected]>
wrote:
> Hi
> Can you help me how to load data from s3 bucket to redshift , if you gave
> sample code can you pls send me
>
> Thanks
> su
>
