[
https://issues.apache.org/jira/browse/APEXCORE-283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062706#comment-15062706
]
ASF GitHub Bot commented on APEXCORE-283:
-----------------------------------------
Github user ashishtadose commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/184#issuecomment-165569141
@tweise @gauravgopi123 commits are squashed with appropriate changes.
Thanks for reviewing.
> Operator checkpointing in distributed in-memory store
> -----------------------------------------------------
>
> Key: APEXCORE-283
> URL: https://issues.apache.org/jira/browse/APEXCORE-283
> Project: Apache Apex Core
> Issue Type: New Feature
> Affects Versions: 3.2.0
> Reporter: Ashish Tadose
> Assignee: Ashish Tadose
> Priority: Minor
> Fix For: 3.3.0
>
>
> Currently Apex engine provides operator checkpointing in Hdfs ( with Hdfs
> backed StorageAgents i.e. FSStorageAgent & AsyncFSStorageAgent )
> As operator check-pointing is critical functionality of Apex streaming
> platform to ensure fault tolerant behavior, platform should also provide
> alternate StorageAgents which will work seamlessly with large applications
> that requires Exactly once semantics.
> HDFS read/write latency is limited and doesn't improve beyond certain point
> because of disk io & staging writes. Having alternate strategy to this
> check-pointing in fault tolerant distributed in-memory grid would ensure
> application stability and performance is not impacted by checkpointing
> *This feature will add below functionalities to core*
> - A KeyValue store interface which is used by In-memory checkpointing storage
> agent.
> - Abstract implementation of KeyValue storage agent which can be configured
> with concrete implementation of KeyValue store for checkpointing.
> - Concrete implementation of In memory storage agent for Apache Geode
> - Interface for storage agent to provide application id
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
