-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12936/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: SQOOP-777
https://issues.apache.org/jira/browse/SQOOP-777
Repository: sqoop-sqoop2
Description
-------
Implemented a pluggable intermediate data format that decouples the internal
representation of the data from the connector and the output formats.
Connectors can choose to implement and support a format that is more efficient
for them. Also separated the SqoopWritable so that we can use the intermediate
data format independent of (current) Hadoop.
I ran a full build - all tests including integration tests pass. I have not
added any new tests, yet. I will add unit tests for the new classes. Also, I
have not tried running this on an actual cluster - so things may be broken. I'd
like some initial feedback based on the current patch.
I also implemented escaping of characters. There is some work remaining to
support binary format, but it is mostly integration, the basic implementation
is in place.
Diffs
-----
common/pom.xml db11b5b
common/src/main/java/org/apache/sqoop/common/SqoopException.java 6caed13
common/src/main/java/org/apache/sqoop/etl/io/CSVIntermediateDataFormat.java
PRE-CREATION
common/src/main/java/org/apache/sqoop/etl/io/DataReader.java 3e1adc7
common/src/main/java/org/apache/sqoop/etl/io/DataWriter.java d81364e
common/src/main/java/org/apache/sqoop/etl/io/IntermediateDataFormat.java
PRE-CREATION
common/src/main/java/org/apache/sqoop/schema/type/Column.java 8b630b2
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcConnector.java
e0da80f
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportInitializer.java
7212843
connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestExportLoader.java
aa1c4ff
connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportExtractor.java
a7ed6ba
core/src/main/java/org/apache/sqoop/framework/FrameworkConstants.java 4293dce
core/src/main/java/org/apache/sqoop/framework/JobManager.java 9f09982
core/src/main/java/org/apache/sqoop/framework/SubmissionRequest.java 53d0039
execution/mapreduce/pom.xml f9a2a0e
execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java
767080c
execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportExtractor.java
1978ec6
execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsSequenceImportLoader.java
a07c511
execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsTextImportLoader.java
4621942
execution/mapreduce/src/main/java/org/apache/sqoop/job/io/SqoopWritable.java
PRE-CREATION
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopFileOutputFormat.java
356ae8a
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java
59cf391
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopNullOutputFormat.java
90de6ef
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java
739eb17
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopReducer.java
b31161c
execution/mapreduce/src/test/java/org/apache/sqoop/job/JobUtils.java e21f15b
execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsExtract.java
b7079dd
execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java
f849aae
execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java
7b264c6
execution/mapreduce/src/test/java/org/apache/sqoop/job/mr/TestSqoopOutputFormatLoadExecutor.java
bee8ab7
pom.xml 520c107
spi/src/main/java/org/apache/sqoop/connector/spi/SqoopConnector.java 2becc56
submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java
6fc485b
Diff: https://reviews.apache.org/r/12936/diff/
Testing
-------
Thanks,
Hari Shreedharan
