[
https://issues.apache.org/jira/browse/FALCON-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954481#comment-14954481
]
Ajay Yadava commented on FALCON-1524:
-------------------------------------
Thanks for reviewing [~bvellanki].
Frequency can not be fraction. This is ensured by checks at xsd level. Here is
the relevant section from feed xsd.
{code}
<xs:simpleType name="frequency-type">
<xs:annotation>
<xs:appinfo>
<jaxb:javaType name="org.apache.falcon.entity.v0.Frequency"
parseMethod="org.apache.falcon.entity.v0.Frequency.fromString"
printMethod="org.apache.falcon.entity.v0.Frequency.toString"/>
</xs:appinfo>
</xs:annotation>
<xs:restriction base="xs:string">
<xs:pattern value="(minutes|hours|days|months)\([1-9]\d*\)"/>
</xs:restriction>
</xs:simpleType>
{code}
You are spot on in saying that I don't need to check for hours (I thought I
deleted that but somehow the patch doesn't contain it :) Thanks for catching
it.
The reason I have not put the limit as a property as it is specific to Oozie
Scheduler only e.g. for native scheduler this won't(shouldn't be as minutely
frequencies are a strong requirement for native scheduler) be the case.
> Improve Lifecycle Retention validation checks
> ---------------------------------------------
>
> Key: FALCON-1524
> URL: https://issues.apache.org/jira/browse/FALCON-1524
> Project: Falcon
> Issue Type: Improvement
> Reporter: Ajay Yadava
> Assignee: Ajay Yadava
> Fix For: 0.8
>
> Attachments: FALCON-1524.patch
>
>
> 2 enhancements:
> 1. Check that retention should not be more frequent than the data
> availability.
> 2. Oozie doesn't behave appropriately for retention for high frequencies like
> minutes(3) etc. Retention follows LAST_ONLY execution order and for high
> frequencies and default configuration retention might not execute. To make it
> work you will need to override
> oozie.service.CoordMaterializeTriggerService.materialization.window
> and
> oozie.service.CoordMaterializeTriggerService.lookup.interval
> which might affect the Oozie system performance.
> Hence we should set a limit like hours(1) for retention frequency and
> retention more frequent than that should throw validation error.
> Originally reported by [~pragya.mittal]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
