[
https://issues.apache.org/jira/browse/AURORA-942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220127#comment-14220127
]
Bill Farner commented on AURORA-942:
------------------------------------
We already chunk large log entries \[1\], so we have control over this.
\[1\]
https://github.com/apache/incubator-aurora/blob/66bd6fec9340eade40a61784ed464c3e94b9d784/src/main/java/org/apache/aurora/scheduler/storage/log/EntrySerializer.java#L73-L83
> Explore using a replicated log on top of ZooKeeper
> --------------------------------------------------
>
> Key: AURORA-942
> URL: https://issues.apache.org/jira/browse/AURORA-942
> Project: Aurora
> Issue Type: Task
> Components: Scheduler
> Reporter: Bill Farner
> Priority: Minor
>
> The scheduler uses the replicated log implementation provided by mesos
> (native libmesos.so). It would be interesting to compare this against a
> replacement that sllows us to:
> - shed code to implement backups and recovery
> - remove one use of a dynamically-linked native library
> - use a store that allows non-leaders to read, for faster recovery and
> serving from non-active members
> - avoid the need for periodic failover (we currently have to do this to
> induce compaction in LevelDB and minimize log replay time)
> At first glance, it seems like it would be relatively straightforward to come
> up with a Log implementation \[1\] that persists transactions as nodes in
> ZooKeeper. This would enable all the above results.
> \[1\]
> https://github.com/apache/incubator-aurora/blob/10da38a3a0ad6ebbee055c26adc3ed3437ec3930/src/main/java/org/apache/aurora/scheduler/log/Log.java#L26
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
