Sowmya Ramesh created FALCON-1105:
-------------------------------------
Summary: Recipe repository management API's and CLI support
Key: FALCON-1105
URL: https://issues.apache.org/jira/browse/FALCON-1105
Project: Falcon
Issue Type: Sub-task
Affects Versions: 0.6
Reporter: Sowmya Ramesh
Assignee: Sowmya Ramesh
Fix For: 0.7
*Recipe Listing*
A GET method may be added to the Prism server on a new Jersey resource to list
recipes and their corresponding root location on recipe repository.
Corresponding CLI methods to be present
{noformat}
[hrt_qa@node-1 recipe]$ falcon recipe -list
List of recipes
{
"totalSize": 2,
"results": {
"hdfs-replication":
"hdfs://node-1.example.com:8020/apps/falcon/recipe/hdfs-replication",
"hive-disaster-recovery":
"hdfs://node-1.example.com:8020/apps/falcon/recipe/hive-disaster-recovery"
}
}
{noformat}
*Get resources of a given recipe*
List all the resources of a given recipe
{noformat}
[hrt_qa@node-1 recipe]$ falcon recipe -getResources -name hdfs-replication
{
"totalSize": 3,
"results": {
"hdfs-replication-template.xml":
"hdfs://node-1.example.com:8020/apps/falcon/recipe/hdfs-replication/resources/build/hdfs-replication-template.xml",
"hdfs-replication-workflow.xml":
"hdfs://node-1.example.com:8020/apps/falcon/recipe/hdfs-replication/resources/runtime/hdfs-replication-workflow.xml",
"hdfs-replication.properties":
"hdfs://node-1.example.com:8020/apps/falcon/recipe/hdfs-replication/resources/build/hdfs-replication.properties"
}
}
{noformat}
*Recipe Description*
A GET method may be added to the Prism server to echo the README as
documentation for the users. This may contain brief on the functionality
offered by the recipe and any operability notes of importance
{noformat}
[hrt_qa@node-1 recipe]$ falcon recipe -describe -name hdfs-replication
HDFS Directory Replication Recipe
Overview
This recipe implements replicating arbitrary directories on HDFS from one
Hadoop cluster to another Hadoop cluster.
This piggy backs on replication solution in Falcon which uses the DistCp tool.
Use Case
* Copy directories between HDFS clusters with out dated partitions
* Archive directories from HDFS to Cloud. Ex: S3, Azure WASB
Limitations
As the data volume and number of files grow, this can get inefficient.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
