[
https://issues.apache.org/jira/browse/FALCON-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pragya Mittal updated FALCON-1675:
----------------------------------
Description:
In case of feed lifecycle :
If we define lifecycle at cluster level and global level lifecycle is not
defined, instead of honouring lifecycle at cluster level it falls back to old
retention level.
Feed definition is :
{noformat}
<feed name="RetentionLifecycleTest-inPath-eeb77a18" description="clicks log"
xmlns="uri:falcon:feed:0.1">
<frequency>hours(1)</frequency>
<timezone>UTC</timezone>
<late-arrival cut-off="hours(1)"/>
<clusters>
<cluster name="RetentionLifecycleTest-corp-02f1746f" type="source">
<validity start="2015-12-18T18:48Z" end="2015-12-18T20:48Z"/>
<retention limit="hours(10)" action="delete"/>
<lifecycle>
<retention-stage>
<frequency>hours(3)</frequency>
<queue>default</queue>
<priority>LOW</priority>
<properties>
<property name="retention.policy.agebaseddelete.limit"
value="hours(4)"/>
</properties>
</retention-stage>
</lifecycle>
</cluster>
</clusters>
<locations>
<location type="data"
path="/tmp/falcon-regression/RetentionLifecycleTest/testFolders/${YEAR}/${MONTH}/${DAY}/${HOUR}"/>
<location type="stats" path="/projects/falcon/clicksStats"/>
<location type="meta" path="/projects/falcon/clicksMetaData"/>
</locations>
<ACL owner="pragya.mittal" group="dataqa" permission="*"/>
<schema location="/schema/clicks" provider="protobuf"/>
<properties/>
</feed>
{noformat}
Job has
retention frequency = 360min (6 hours)
priority = NORMAL
Intsnace retains last 10 instances instead of 4 (as is defined in cluster level
lifecycle)
was:
In case of feed lifecycle :
If we define lifecycle at cluster level and global level lifecycle is not
defined, instead of honouring lifecycle at cluster level it falls back to old
retention level.
> Lifecycle stage is not honoured when only cluster level lifecycle is
> implemented.
> ---------------------------------------------------------------------------------
>
> Key: FALCON-1675
> URL: https://issues.apache.org/jira/browse/FALCON-1675
> Project: Falcon
> Issue Type: Bug
> Components: feed
> Affects Versions: 0.9
> Environment: QA
> Reporter: Pragya Mittal
>
> In case of feed lifecycle :
> If we define lifecycle at cluster level and global level lifecycle is not
> defined, instead of honouring lifecycle at cluster level it falls back to old
> retention level.
> Feed definition is :
> {noformat}
> <feed name="RetentionLifecycleTest-inPath-eeb77a18" description="clicks log"
> xmlns="uri:falcon:feed:0.1">
> <frequency>hours(1)</frequency>
> <timezone>UTC</timezone>
> <late-arrival cut-off="hours(1)"/>
> <clusters>
> <cluster name="RetentionLifecycleTest-corp-02f1746f" type="source">
> <validity start="2015-12-18T18:48Z" end="2015-12-18T20:48Z"/>
> <retention limit="hours(10)" action="delete"/>
> <lifecycle>
> <retention-stage>
> <frequency>hours(3)</frequency>
> <queue>default</queue>
> <priority>LOW</priority>
> <properties>
> <property
> name="retention.policy.agebaseddelete.limit" value="hours(4)"/>
> </properties>
> </retention-stage>
> </lifecycle>
> </cluster>
> </clusters>
> <locations>
> <location type="data"
> path="/tmp/falcon-regression/RetentionLifecycleTest/testFolders/${YEAR}/${MONTH}/${DAY}/${HOUR}"/>
> <location type="stats" path="/projects/falcon/clicksStats"/>
> <location type="meta" path="/projects/falcon/clicksMetaData"/>
> </locations>
> <ACL owner="pragya.mittal" group="dataqa" permission="*"/>
> <schema location="/schema/clicks" provider="protobuf"/>
> <properties/>
> </feed>
> {noformat}
> Job has
> retention frequency = 360min (6 hours)
> priority = NORMAL
> Intsnace retains last 10 instances instead of 4 (as is defined in cluster
> level lifecycle)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
