----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39240/#review104142 -----------------------------------------------------------
Thanks for this patch. Did a quick pass and have some comments. Will do a deeper review this week src/java/org/apache/sqoop/concurrency/HiveMergeTableLock.java (line 31) <https://reviews.apache.org/r/39240/#comment162412> No author tags in apache projects src/java/org/apache/sqoop/concurrency/HiveMergeTableLock.java (line 33) <https://reviews.apache.org/r/39240/#comment162413> This is not a ? you probably meant "This is a " src/java/org/apache/sqoop/hive/merge/record/strategy/AbstractMergeRecord.java (line 33) <https://reviews.apache.org/r/39240/#comment162414> No @author tags src/java/org/apache/sqoop/hive/merge/record/strategy/MergeRecordFactory.java (line 25) <https://reviews.apache.org/r/39240/#comment162415> Same as others - No @author tags - will not repeat for other occurrences. Please remove all @author tags src/test/org/apache/sqoop/util/clusters/SqoopMiniDFSCluster.java (line 137) <https://reviews.apache.org/r/39240/#comment162416> Please remove all commented out code - Venkat Ranganathan On Oct. 12, 2015, 10:44 a.m., Rakesh Sharma wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/39240/ > ----------------------------------------------------------- > > (Updated Oct. 12, 2015, 10:44 a.m.) > > > Review request for Sqoop and Venkat Ranganathan. > > > Repository: sqoop-trunk > > > Description > ------- > > Sqoop currently doesn't support merging of two hive tables. Implement a new > tool for Sqoop which : > 1. merges two hive tables. (This will be most commonly used to merge new > incremental data into an existing hive table) > 2. should support both partitioned and non partitioned tables > 3. supports merge on composite keys > 4. For partitioned tables, merges old partitions and also add new partitions. > 5. Supports Text/RC/ORC/Seq file formats > 6. Should make sure that only process is performing merge. > 7. should be an atomic operation. If it fails at any point , it should revert > the target hive table to its original state. > > > Diffs > ----- > > ivy.xml d84b88f > ivy/libraries.properties 2e3d884 > src/docs/user/SqoopUserGuide.txt 8d9c12d > src/docs/user/hive-merge-purpose.txt PRE-CREATION > src/docs/user/hive-merge.txt PRE-CREATION > src/java/org/apache/sqoop/concurrency/HiveMergeTableLock.java PRE-CREATION > > src/java/org/apache/sqoop/hive/merge/record/strategy/AbstractMergeRecord.java > PRE-CREATION > > src/java/org/apache/sqoop/hive/merge/record/strategy/MergeRecordFactory.java > PRE-CREATION > src/java/org/apache/sqoop/hive/merge/record/strategy/ORCMergeRecord.java > PRE-CREATION > src/java/org/apache/sqoop/hive/merge/record/strategy/RCMergeRecord.java > PRE-CREATION > src/java/org/apache/sqoop/hive/merge/record/strategy/RecordInspector.java > PRE-CREATION > src/java/org/apache/sqoop/hive/merge/record/strategy/TaggedMergeRecord.java > PRE-CREATION > > src/java/org/apache/sqoop/hive/merge/record/strategy/TextSequenceMergeRecord.java > PRE-CREATION > src/java/org/apache/sqoop/hive/partition/util/NewPartitionsHandler.java > PRE-CREATION > src/java/org/apache/sqoop/hive/partition/util/NonPartitionTableHandler.java > PRE-CREATION > src/java/org/apache/sqoop/hive/partition/util/PartitionFilter.java > PRE-CREATION > src/java/org/apache/sqoop/hive/partition/util/PartitionHandler.java > PRE-CREATION > src/java/org/apache/sqoop/hive/partition/util/UpdatedPartitionsHandler.java > PRE-CREATION > src/java/org/apache/sqoop/hooks/ShutdownCleanupHook.java PRE-CREATION > src/java/org/apache/sqoop/io/CodecMap.java cec9358 > src/java/org/apache/sqoop/io/FILE_FORMAT.java PRE-CREATION > src/java/org/apache/sqoop/io/OriginalStateRestorer.java PRE-CREATION > src/java/org/apache/sqoop/io/VersionHandler.java PRE-CREATION > src/java/org/apache/sqoop/mapreduce/hivemerge/AbstractHiveMergeMapper.java > PRE-CREATION > src/java/org/apache/sqoop/mapreduce/hivemerge/ConfigurationConstants.java > PRE-CREATION > src/java/org/apache/sqoop/mapreduce/hivemerge/HiveMergeJob.java > PRE-CREATION > src/java/org/apache/sqoop/mapreduce/hivemerge/HiveMergeReducer.java > PRE-CREATION > src/java/org/apache/sqoop/mapreduce/hivemerge/ORCHiveMergeMapper.java > PRE-CREATION > src/java/org/apache/sqoop/mapreduce/hivemerge/RCHiveMergeMapper.java > PRE-CREATION > src/java/org/apache/sqoop/mapreduce/hivemerge/SerdeFactory.java > PRE-CREATION > > src/java/org/apache/sqoop/mapreduce/hivemerge/TextSequenceHiveMergeMapper.java > PRE-CREATION > src/java/org/apache/sqoop/tool/HiveMergeException.java PRE-CREATION > src/java/org/apache/sqoop/tool/HiveMergeTool.java PRE-CREATION > src/java/org/apache/sqoop/tool/SqoopTool.java 5b8453d > src/java/org/apache/sqoop/util/HDFSUtil.java PRE-CREATION > src/java/org/apache/sqoop/util/HiveUtil.java PRE-CREATION > src/java/org/apache/sqoop/util/IOStreamUtils.java PRE-CREATION > src/java/org/apache/sqoop/util/JSONUtil.java PRE-CREATION > > src/test/org/apache/sqoop/hive/merge/record/strategy/TestAbstractMergeRecord.java > PRE-CREATION > > src/test/org/apache/sqoop/hive/merge/record/strategy/TestORCMergeRecord.java > PRE-CREATION > src/test/org/apache/sqoop/hive/merge/record/strategy/TestRCMergeRecord.java > PRE-CREATION > > src/test/org/apache/sqoop/hive/merge/record/strategy/TestTaggedMergeRecord.java > PRE-CREATION > > src/test/org/apache/sqoop/hive/merge/record/strategy/TestTextSequenceMergeRecord.java > PRE-CREATION > src/test/org/apache/sqoop/hive/partition/util/TestNewPartitionHandler.java > PRE-CREATION > > src/test/org/apache/sqoop/hive/partition/util/TestNonPartitionTableHandler.java > PRE-CREATION > > src/test/org/apache/sqoop/hive/partition/util/TestUpdatedPartitionsHandler.java > PRE-CREATION > src/test/org/apache/sqoop/io/TestVersionHandler.java PRE-CREATION > src/test/org/apache/sqoop/mapreduce/hivemerge/TestHiveMergeJob.java > PRE-CREATION > src/test/org/apache/sqoop/mapreduce/hivemerge/TestORCHiveMergeMapper.java > PRE-CREATION > src/test/org/apache/sqoop/mapreduce/hivemerge/TestRCHiveMergeMapper.java > PRE-CREATION > > src/test/org/apache/sqoop/mapreduce/hivemerge/TestTextSequenceHiveMergeMapper.java > PRE-CREATION > src/test/org/apache/sqoop/util/TestHDFSUtil.java PRE-CREATION > src/test/org/apache/sqoop/util/TestJSONUtil.java PRE-CREATION > src/test/org/apache/sqoop/util/clusters/SqoopMiniDFSCluster.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/39240/diff/ > > > Testing > ------- > > YES > > > Thanks, > > Rakesh Sharma > >
