-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35833/
-----------------------------------------------------------
(Updated June 26, 2015, 10:56 a.m.)
Review request for Falcon and Srikanth Sundarrajan.
Changes
-------
Updated the patch to include the followin changes.
HiveDR deletes source of root staging path after job completion
HiveDR secure workflow tmeplate is missing the DB information in the Hive2
credential URL
Bugs: FALCON-1188
https://issues.apache.org/jira/browse/FALCON-1188
Repository: falcon-git
Description
-------
Falcon support for HiveDR
As referenced in the document attached to FALCON-1188, this patch introduces
Falcon support for HiveDR
It is implemented as recipe and added to Falcon addons package and can be
included into the base product using the maven profile hivedr.
This supports HiveDR in both secure and unsecure clusters.
This is a large code base and we have repackage it to be in the addons so that
we can quickly integrate with falcon master branch and continue further
refinements/development in the master branch instead of creating a separate
branch introducing merge issues
This work is the result of contribution of Balu Vellanki, Peeyush Bishnoi,
Sowmya Ramesh and Venkat Ranganathan with help from Hive folks
Diffs (updated)
-----
addons/hivedr/README PRE-CREATION
addons/hivedr/pom.xml PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/DefaultPartitioner.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/EventSourcer.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/HiveDRArgs.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/HiveDROptions.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/HiveDRTool.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/LastEvents.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/MetaStoreEventSourcer.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/Partitioner.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/ReplicationEventMetadata.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/exception/HiveReplicationException.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/mapreduce/CopyCommitter.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/mapreduce/CopyMapper.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/mapreduce/CopyReducer.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/DBReplicationStatus.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/DRStatusStore.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/DelimiterUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/EventSourcerUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/EventUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/FileUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/HiveDRStatusStore.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/HiveDRUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/HiveMetastoreUtils.java
PRE-CREATION
addons/hivedr/src/main/java/org/apache/falcon/hive/util/ReplicationStatus.java
PRE-CREATION
addons/hivedr/src/main/resources/log4j.xml PRE-CREATION
addons/hivedr/src/test/java/org/apache/falcon/hive/DBReplicationStatusTest.java
PRE-CREATION
addons/hivedr/src/test/java/org/apache/falcon/hive/DRTest.java PRE-CREATION
addons/hivedr/src/test/java/org/apache/falcon/hive/HiveDRStatusStoreTest.java
PRE-CREATION
addons/hivedr/src/test/java/org/apache/falcon/hive/ReplicationStatusTest.java
PRE-CREATION
addons/recipes/hdfs-replication/src/main/resources/hdfs-replication-template.xml
824e6f5
addons/recipes/hdfs-replication/src/main/resources/hdfs-replication-workflow.xml
145d489
addons/recipes/hdfs-replication/src/main/resources/hdfs-replication.properties
19b8459
addons/recipes/hive-disaster-recovery/README.txt PRE-CREATION
addons/recipes/hive-disaster-recovery/pom.xml PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-secure-template.xml
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-secure-workflow.xml
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-secure.properties
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-template.xml
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery-workflow.xml
PRE-CREATION
addons/recipes/hive-disaster-recovery/src/main/resources/hive-disaster-recovery.properties
PRE-CREATION
client/pom.xml afa91c9
client/src/main/java/org/apache/falcon/cli/FalconCLI.java f169917
client/src/main/java/org/apache/falcon/client/FalconClient.java 20c32e4
client/src/main/java/org/apache/falcon/recipe/HdfsReplicationRecipeTool.java
PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/HdfsReplicationRecipeToolOptions.java
PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/HiveReplicationRecipeTool.java
PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/HiveReplicationRecipeToolOptions.java
PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/Recipe.java PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/RecipeFactory.java PRE-CREATION
client/src/main/java/org/apache/falcon/recipe/RecipeTool.java 069db9f
client/src/main/java/org/apache/falcon/recipe/RecipeToolArgs.java baa4846
client/src/main/java/org/apache/falcon/recipe/RecipeToolOptions.java a1c29cd
client/src/main/java/org/apache/falcon/recipe/util/RecipeProcessBuilderUtils.java
PRE-CREATION
common/pom.xml 36de1f5
docs/src/site/twiki/InstallationSteps.twiki 1dd242a
oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java
e3de6a4
pom.xml 31997e8
replication/pom.xml 8c4d6b4
src/main/assemblies/distributed-package.xml 1e48c94
src/main/assemblies/standalone-package.xml 682d52f
test-tools/hadoop-webapp/pom.xml cadc7f5
webapp/pom.xml 063d42c
webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java 90acb59
webapp/src/test/resources/process.properties 1eb282c
Diff: https://reviews.apache.org/r/35833/diff/
Testing
-------
HiveDR has been validated both in secure and unsecure cluster configurations.
Checkstyle and RAT checks pass. There is an associated UI enhancement in
Falcon UI that is provided independently.
Please see the README on component version requirements.
Thanks,
Venkat Ranganathan
