[
https://issues.apache.org/jira/browse/APEXCORE-283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062713#comment-15062713
]
ASF GitHub Bot commented on APEXCORE-283:
-----------------------------------------
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/184#discussion_r47955345
--- Diff: engine/src/main/java/com/datatorrent/stram/StramClient.java ---
@@ -455,6 +476,12 @@ public void startApplication() throws YarnException,
IOException
}
dag.getAttributes().put(LogicalPlan.APPLICATION_PATH,
appPath.toString());
+ StorageAgent agent =
dag.getAttributes().get(OperatorContext.STORAGE_AGENT);
+ if (agent != null && agent instanceof
StorageAgent.ApplicationAwareStorageAgent) {
+ StorageAgent.ApplicationAwareStorageAgent ap =
(StorageAgent.ApplicationAwareStorageAgent)dag.getAttributes().get(OperatorContext.STORAGE_AGENT);
--- End diff --
This call is not needed
> Operator checkpointing in distributed in-memory store
> -----------------------------------------------------
>
> Key: APEXCORE-283
> URL: https://issues.apache.org/jira/browse/APEXCORE-283
> Project: Apache Apex Core
> Issue Type: New Feature
> Affects Versions: 3.2.0
> Reporter: Ashish Tadose
> Assignee: Ashish Tadose
> Priority: Minor
> Fix For: 3.3.0
>
>
> Currently Apex engine provides operator checkpointing in Hdfs ( with Hdfs
> backed StorageAgents i.e. FSStorageAgent & AsyncFSStorageAgent )
> As operator check-pointing is critical functionality of Apex streaming
> platform to ensure fault tolerant behavior, platform should also provide
> alternate StorageAgents which will work seamlessly with large applications
> that requires Exactly once semantics.
> HDFS read/write latency is limited and doesn't improve beyond certain point
> because of disk io & staging writes. Having alternate strategy to this
> check-pointing in fault tolerant distributed in-memory grid would ensure
> application stability and performance is not impacted by checkpointing
> *This feature will add below functionalities to core*
> - A KeyValue store interface which is used by In-memory checkpointing storage
> agent.
> - Abstract implementation of KeyValue storage agent which can be configured
> with concrete implementation of KeyValue store for checkpointing.
> - Concrete implementation of In memory storage agent for Apache Geode
> - Interface for storage agent to provide application id
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
