[ https://issues.apache.org/jira/browse/FALCON-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804728#comment-15804728 ]
Ajay Yadava commented on FALCON-2234: ------------------------------------- Hello Prashant, Thank you for reporting this issue. If I recall correctly, the original plan was to add support for [incremental, append] as another patch which never came through and hence has landed the documentation and validation in an inconsistent state. I believe the documentation needs to be fixed. Would you like to provide a patch? > Falcon feed fails for RDBMS import if extract type is incremental > ----------------------------------------------------------------- > > Key: FALCON-2234 > URL: https://issues.apache.org/jira/browse/FALCON-2234 > Project: Falcon > Issue Type: Bug > Components: falcon-ui > Affects Versions: 0.10 > Reporter: Prashant > Priority: Critical > > Falcon feed is failing with > org.apache.falcon.entity.parser.ValidationException > when extract type is incremental. > As per the Apache doc Falcon supports > Import policy > The valid combinations are: > [full,snapshot] - data is extracted in full and dumped into the feed instance > location. > [incremental, append] - data is extracted incrementally using the key > specified in the deltacolumn > As per the below code , it appears we only check [full,snapshot] and > [incremental, append] goes to exception > https://github.com/apache/falcon/blob/0.10/common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java > Snippet > {code} > /** > * Validate extraction and merge type combination. Currently supported > combo: > * > * ExtractionType = FULL and MergeType = SNAPSHOT. > * ExtractionType = INCREMENTAL and MergeType = APPEND. > * > * @param feed Feed entity > * @param cluster Cluster referenced in the Feed definition > * @throws FalconException > */ > private void validateFeedExtractionType(Feed feed, Cluster cluster) > throws FalconException { > Extract extract = cluster.getImport().getSource().getExtract(); > if (ExtractMethod.FULL == extract.getType()) { > if ((MergeType.SNAPSHOT != extract.getMergepolicy()) > || (extract.getDeltacolumn() != null)) { > throw new ValidationException(String.format("Feed %s is using > FULL " > + "extract method but specifies either a superfluous " > + "deltacolumn or a mergepolicy other than > snapshot", feed.getName())); > } > } else { > throw new ValidationException(String.format("Feed %s is using > unsupported " > + "extraction mechanism %s", feed.getName(), > extract.getType().value())); > } > } > {code} > It must have a clause to also check [incremental, append] . -- This message was sent by Atlassian JIRA (v6.3.4#6332)