[
https://issues.apache.org/jira/browse/BEAM-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406520#comment-16406520
]
Jacob Marble commented on BEAM-3032:
------------------------------------
I have implemented DoFns Copy and Unload, which wrap the COPY and UNLOAD SQL
commands.
I have also implemented PTransforms Redshift.Read and Redshift.Write.
For example, Read<T> extends PTransform<PBegin, PCollection<T>> steps:
1 invokes Unload (Redshift to S3)
2 reads and unmarshals the S3 objects into a PCollection
3 deletes S3 objects
This code was working months ago when I first wrote it, now trying to add unit
tests.
> Add RedshiftIO
> --------------
>
> Key: BEAM-3032
> URL: https://issues.apache.org/jira/browse/BEAM-3032
> Project: Beam
> Issue Type: New Feature
> Components: io-ideas
> Environment: AWS Redshift
> Reporter: Jacob Marble
> Assignee: Jacob Marble
> Priority: Minor
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> I would like to add a RedshiftIO Java extension to perform bulk read/write
> to/from AWS Redshift via the UNLOAD and COPY Redshift SQL commands. This
> requires S3, which is the subject of BEAM-2500.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
