-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35833/
-----------------------------------------------------------
(Updated Aug. 9, 2015, 10:46 a.m.)
Review request for Falcon and Srikanth Sundarrajan.
Changes
-------
Updated patch to use Hadoop 2.5.0. Building Oozie for Hive 1.2.0 and Hive DR
instructions need more change - will follow up in a separate JIRA
Bugs: FALCON-1188
https://issues.apache.org/jira/browse/FALCON-1188
Repository: falcon-git
Description
-------
Falcon support for HiveDR
As referenced in the document attached to FALCON-1188, this patch introduces
Falcon support for HiveDR
It is implemented as recipe and added to Falcon addons package and can be
included into the base product using the maven profile hivedr.
This supports HiveDR in both secure and unsecure clusters.
This is a large code base and we have repackage it to be in the addons so that
we can quickly integrate with falcon master branch and continue further
refinements/development in the master branch instead of creating a separate
branch introducing merge issues
This work is the result of contribution of Balu Vellanki, Peeyush Bishnoi,
Sowmya Ramesh and Venkat Ranganathan with help from Hive folks
Diffs (updated)
-----
Installation-steps.txt bb92a85
addons/hivedr/README PRE-CREATION
addons/hivedr/pom.xml PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/DefaultPartitioner.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/EventSourcer.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/HiveDRArgs.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/HiveDROptions.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/HiveDRTool.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/LastReplicatedEvents.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/MetaStoreEventSourcer.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/Partitioner.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/ReplicationEventMetadata.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/exception/HiveReplicationException.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/mapreduce/CopyCommitter.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/mapreduce/CopyMapper.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/mapreduce/CopyReducer.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/DBReplicationStatus.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/DRStatusStore.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/DelimiterUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/EventSourcerUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/EventUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/FileUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/HiveDRStatusStore.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/HiveDRUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/HiveMetastoreUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/ReplicationStatus.java
PRE-CREATION
addons/hivedr/src/main/resources/log4j.xml PRE-CREATION
addons/hivedr/src/test/java/org/apache/falcon/hive/DBReplicationStatusTest.java
PRE-CREATION
addons/hivedr/src/test/java/org/apache/falcon/hive/DRTest.java PRE-CREATION
addons/hivedr/src/test/java/org/apache/falcon/hive/HiveDRStatusStoreTest.java
PRE-CREATION
addons/hivedr/src/test/java/org/apache/falcon/hive/ReplicationStatusTest.java
PRE-CREATION
addons/recipes/hdfs-replication/src/main/resources/hdfs-replication-template.xml
824e6f5
addons/recipes/hdfs-replication/src/main/resources/hdfs-replication-workflow.xml
145d489
addons/recipes/hdfs-replication/src/main/resources/hdfs-replication.properties
19b8459
addons/recipes/hive-disaster-recovery/README.txt PRE-CREATION
addons/recipes/hive-disaster-recovery/pom.xml PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-secure-template.xml
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-secure-workflow.xml
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-secure.properties
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-template.xml
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-workflow.xml
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery.properties
PRE-CREATION
client/pom.xml afa91c9
client/src/main/java/org/apache/falcon/cli/FalconCLI.java 148f789
client/src/main/java/org/apache/falcon/client/FalconClient.java 9649e10
client/src/main/java/org/apache/falcon/recipe/HdfsReplicationRecipeTool.java
PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/HdfsReplicationRecipeToolOptions.java
PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/HiveReplicationRecipeTool.java
PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/HiveReplicationRecipeToolOptions.java
PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/Recipe.java PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/RecipeFactory.java PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/RecipeTool.java 069db9f
client/src/main/java/org/apache/falcon/recipe/RecipeToolArgs.java baa4846
client/src/main/java/org/apache/falcon/recipe/RecipeToolOptions.java a1c29cd
client/src/main/java/org/apache/falcon/recipe/util/RecipeProcessBuilderUtils.java
PRE-CREATION
docs/src/site/twiki/InstallationSteps.twiki 3dd034b
oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java
e3de6a4
pom.xml 34a5471
replication/pom.xml 8c4d6b4
src/main/assemblies/distributed-package.xml 1e48c94
src/main/assemblies/standalone-package.xml 682d52f
test-tools/hadoop-webapp/pom.xml cadc7f5
webapp/pom.xml 063d42c
webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java 90acb59
webapp/src/test/resources/process.properties 1eb282c
Diff: https://reviews.apache.org/r/35833/diff/
Testing
-------
HiveDR has been validated both in secure and unsecure cluster configurations.
Checkstyle and RAT checks pass. There is an associated UI enhancement in
Falcon UI that is provided independently.
Please see the README on component version requirements.
Thanks,
Venkat Ranganathan
