[
https://issues.apache.org/jira/browse/FALCON-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14485859#comment-14485859
]
Peeyush Bishnoi commented on FALCON-1146:
-----------------------------------------
I am able to replicate this issue in which entire feed path deleted up to the
root, if path is empty. Currently feedBasePath (/adis/test) is not fully
qualified when passed to deleteInstance and to deleteParentIfEmpty function.
But instance directory path is fully qualified
(hdfs://champlain.hortonworks.local:8020/adis/test/2015/04/02/09/28) which has
to be deleted. When check was performed whether feedBasePath (/adis/test)
matches to parent feed instance directory path
(hdfs://champlain.hortonworks.local:8020/adis/test) to stop deletion, it did
not match and thus deletion up to the root level happened (if path is empty) as
there is recursive call for deleteParentIfEmpty function.
{code:java}
private void deleteParentIfEmpty(FileSystem fs, Path parent, Path feedBasePath)
throws IOException {
if (feedBasePath.equals(parent)) {
LOG.info("Not deleting feed base path: {}", parent);
} else {
FileStatus[] files = fs.listStatus(parent);
if (files != null && files.length == 0) {
LOG.info("Parent path: {} is empty, deleting path", parent);
if (fs.delete(parent, true)) {
LOG.info("Deleted empty dir: {}", parent);
} else {
throw new IOException("Unable to delete parent path:" +
parent);
}
deleteParentIfEmpty(fs, parent.getParent(), feedBasePath);
}
}
}
{code}
After feedBasePath has been made fully qualified, it matches to the parent
directory path appropriately, thus it stop deletion up to the feedBasePath
directory only. [~sriksun] I will provide the patch for this issue so as to fix
in 0.6.* itself.
> feed retention policy deleted everything all the way up to the root
> -------------------------------------------------------------------
>
> Key: FALCON-1146
> URL: https://issues.apache.org/jira/browse/FALCON-1146
> Project: Falcon
> Issue Type: Bug
> Components: common, retention
> Affects Versions: 0.6
> Reporter: Peeyush Bishnoi
> Assignee: Peeyush Bishnoi
> Priority: Critical
> Fix For: 0.6.1
>
>
> For the below feed definition xml, feed is set to delete every minute but
> once it run it deleted everything all the way up and even the root directory.
> {code}
> <?xml version="1.0" encoding="UTF-8"?>
> <feed description="ALU EVDO retention feed" name="alu-evdo-retention"
> xmlns="uri:falcon:feed:0.1">
> <frequency>minutes(30)</frequency>
> <clusters>
> <cluster name="primary-cluster" type="source">
> <validity start="2014-03-01T00:00Z" end="2016-01-01T00:00Z"/>
> <retention limit="minutes(1)" action="delete"/>
> </cluster>
> </clusters>
> <locations>
> <location type="data"
> path="/adis/test/${YEAR}-${MONTH}-${DAY}/${HOUR}/${MINUTE}"/>
> </locations>
> <ACL owner="flume" group="landing" permission="0x777"/>
> <schema location="/none" provider="none" />
> </feed>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
