[ https://issues.apache.org/jira/browse/FALCON-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant updated FALCON-2234: ----------------------------- Description: Falcon feed is failing with org.apache.falcon.entity.parser.ValidationException when extract type is incremental. As per the Apache doc Falcon supports Import policy The valid combinations are: [full,snapshot] - data is extracted in full and dumped into the feed instance location. [incremental, append] - data is extracted incrementally using the key specified in the deltacolumn As per the below code , it appears we only check [full,snapshot] and [incremental, append] goes to exception https://github.com/apache/falcon/blob/0.10/common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java Snippet {code} /** * Validate extraction and merge type combination. Currently supported combo: * * ExtractionType = FULL and MergeType = SNAPSHOT. * ExtractionType = INCREMENTAL and MergeType = APPEND. * * @param feed Feed entity * @param cluster Cluster referenced in the Feed definition * @throws FalconException */ private void validateFeedExtractionType(Feed feed, Cluster cluster) throws FalconException { Extract extract = cluster.getImport().getSource().getExtract(); if (ExtractMethod.FULL == extract.getType()) { if ((MergeType.SNAPSHOT != extract.getMergepolicy()) || (extract.getDeltacolumn() != null)) { throw new ValidationException(String.format("Feed %s is using FULL " + "extract method but specifies either a superfluous " + "deltacolumn or a mergepolicy other than snapshot", feed.getName())); } } else { throw new ValidationException(String.format("Feed %s is using unsupported " + "extraction mechanism %s", feed.getName(), extract.getType().value())); } } {code} It must have a clause to also check [incremental, append] . was: Falcon feed is failing with org.apache.falcon.entity.parser.ValidationException when extract type is incremental. As per the Apache doc Falcon supports Import policy The valid combinations are: [full,snapshot] - data is extracted in full and dumped into the feed instance location. [incremental, append] - data is extracted incrementally using the key specified in the deltacolumn As per the below code , it appears we only check [full,snapshot] and [incremental, append] goes to exception https://github.com/apache/falcon/blob/0.10/common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java Snippet /* private void validateFeedExtractionType(Feed feed, Cluster cluster) throws FalconException { Extract extract = cluster.getImport().getSource().getExtract(); if (ExtractMethod.FULL == extract.getType()) { if ((MergeType.SNAPSHOT != extract.getMergepolicy()) || (extract.getDeltacolumn() != null)) { throw new ValidationException(String.format("Feed %s is using FULL " + "extract method but specifies either a superfluous " + "deltacolumn or a mergepolicy other than snapshot", feed.getName())); } } else { throw new ValidationException(String.format("Feed %s is using unsupported " + "extraction mechanism %s", feed.getName(), extract.getType().value())); } } */ It must have a clause to also check [incremental, append] . > Falcon feed fails for RDBMS import if extract type is incremental > ----------------------------------------------------------------- > > Key: FALCON-2234 > URL: https://issues.apache.org/jira/browse/FALCON-2234 > Project: Falcon > Issue Type: Bug > Components: falcon-ui > Affects Versions: 0.10 > Reporter: Prashant > Priority: Critical > > Falcon feed is failing with > org.apache.falcon.entity.parser.ValidationException > when extract type is incremental. > As per the Apache doc Falcon supports > Import policy > The valid combinations are: > [full,snapshot] - data is extracted in full and dumped into the feed instance > location. > [incremental, append] - data is extracted incrementally using the key > specified in the deltacolumn > As per the below code , it appears we only check [full,snapshot] and > [incremental, append] goes to exception > https://github.com/apache/falcon/blob/0.10/common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java > Snippet > {code} > /** > * Validate extraction and merge type combination. Currently supported > combo: > * > * ExtractionType = FULL and MergeType = SNAPSHOT. > * ExtractionType = INCREMENTAL and MergeType = APPEND. > * > * @param feed Feed entity > * @param cluster Cluster referenced in the Feed definition > * @throws FalconException > */ > private void validateFeedExtractionType(Feed feed, Cluster cluster) > throws FalconException { > Extract extract = cluster.getImport().getSource().getExtract(); > if (ExtractMethod.FULL == extract.getType()) { > if ((MergeType.SNAPSHOT != extract.getMergepolicy()) > || (extract.getDeltacolumn() != null)) { > throw new ValidationException(String.format("Feed %s is using > FULL " > + "extract method but specifies either a superfluous " > + "deltacolumn or a mergepolicy other than > snapshot", feed.getName())); > } > } else { > throw new ValidationException(String.format("Feed %s is using > unsupported " > + "extraction mechanism %s", feed.getName(), > extract.getType().value())); > } > } > {code} > It must have a clause to also check [incremental, append] . -- This message was sent by Atlassian JIRA (v6.3.4#6332)