[
https://issues.apache.org/jira/browse/SQOOP-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14215790#comment-14215790
]
Hudson commented on SQOOP-1588:
-------------------------------
SUCCESS: Integrated in Sqoop2-hadoop200 #611 (See
[https://builds.apache.org/job/Sqoop2-hadoop200/611/])
SQOOP-1588: Sqoop2: TO-side: Write data to HDFS (abraham:
https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=59ffd880258e63775d58905e6059da1c4287aa44)
*
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/JarUtil.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/LinkConfiguration.java
* connector/connector-kite/src/main/resources/sqoopconnector.properties
* test/pom.xml
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/ToJobConfiguration.java
*
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToDestroyer.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/LinkConfig.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteConnector.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToDestroyer.java
* connector/connector-kite/src/test/resources/log4j.properties
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteConnectorConstants.java
*
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteExecutor.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/util/KiteDataTypeUtil.java
*
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToInitializer.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteDatasetExecutor.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToInitializer.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteLoader.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/ToJobConfig.java
* connector/connector-kite/src/main/resources/connector-configs.properties
* server/pom.xml
*
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteLoader.java
* connector/connector-kite/src/main/resources/kite-connector-config.properties
* connector/pom.xml
* connector/connector-kite/pom.xml
* pom.xml
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/util/InputValidation.java
*
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteConnectorError.java
*
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/FileFormat.java
> Sqoop2: TO-side: Write data to HDFS
> -----------------------------------
>
> Key: SQOOP-1588
> URL: https://issues.apache.org/jira/browse/SQOOP-1588
> Project: Sqoop
> Issue Type: Sub-task
> Components: connectors
> Reporter: Qian Xu
> Assignee: Qian Xu
> Fix For: 1.99.5
>
> Attachments: SQOOP-1588.5.patch
>
>
> Create a basic Kite connector that can write data (i.e. from a jdbc
> connection) to HDFS.
> The scope is defined as follows:
> - Destination: HDFS
> - File Format: Avro Parquet and CSV.
> - Compression Codec: Use default
> - Partitioner Strategy: Not supported
> - Column Mapping: Not supported
> Exposed Configuration:
> - [Link] File Format (Enum)
> - [To] Dataset URI (String, has a validation check)
> Workflow:
> - Create a link to Kite Connector
> - Create a job with valid configuration (see above)
> - Start a job {{KiteToInitializer}} will check dataset existence
> - Sqoop will create N {{KiteLoader}} instances.
> - Kite requires an Avro schema for data manipulation, {{KiteLoader}} will
> create an Avro schema from Sqoop schema provided by {{LoaderContext}}. As
> Sqoop schema types are not identical to Avro types, some types will be
> mapped. The original Sqoop type information will be kept as {{SqoopType}} in
> schema field, which can be used for a reversed type mapping.
> - {{KiteLoader}} will create a temporary dataset and writes data records into
> it. If any error occurs, the dataset will be deleted.
> - {{KiteToDestroy}} will merge all temporary datasets as one dataset.
> Further features will be implemented in follow-up JIRAs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
