[
https://issues.apache.org/jira/browse/HDDS-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16847788#comment-16847788
]
Hudson commented on HDDS-1512:
------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16602 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/16602/])
HDDS-1512. Implement DoubleBuffer in OzoneManager. (#810) (github: rev
6d0e79c1216f342b4463131f6f997fea8e3ce247)
* (add)
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java
* (add)
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/helpers/DoubleBufferEntry.java
* (add)
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/OMBucketDeleteResponse.java
* (add)
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ratis/TestOzoneManagerDoubleBufferWithOMResponse.java
* (add)
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/OMVolumeDeleteResponse.java
* (add)
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ratis/TestOzoneManagerDoubleBufferWithDummyResponse.java
* (add)
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/OMClientResponse.java
* (add)
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/package-info.java
* (add)
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/OMBucketCreateResponse.java
* (add)
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/helpers/package-info.java
* (add)
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/OMVolumeCreateResponse.java
> Implement DoubleBuffer in OzoneManager
> --------------------------------------
>
> Key: HDDS-1512
> URL: https://issues.apache.org/jira/browse/HDDS-1512
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: Ozone Manager
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.5.0
>
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> This Jira is created to implement DoubleBuffer in OzoneManager to flush
> transactions to OM DB.
>
> h2. Flushing Transactions to RocksDB:
> We propose using an implementation similar to the HDFS EditsDoubleBuffer. We
> shall flush RocksDB transactions in batches, instead of current way of using
> rocksdb.put() after every operation. At a given time only one batch will be
> outstanding for flush while newer transactions are accumulated in memory to
> be flushed later.
>
> In DoubleBuffer it will have 2 buffers one is currentBuffer, and the other is
> readyBuffer. We add entry to current buffer, and we check if another flush
> call is outstanding. If not, we flush to disk Otherwise we add entries to
> otherBuffer while sync is happening.
>
> In this if sync is happening, we shall add new requests to other buffer and
> when we can sync we use *RocksDB batch commit to sync to disk, instead of
> rocksdb put.*
>
> Note: If flush to disk is failed on any OM, we shall terminate the
> OzoneManager, so that OM DB’s will not diverge. Flush failure should be
> considered as catastrophic failure.
>
> Scope of this Jira is to add DoubleBuffer implementation, integrating to
> current OM will be done in further jira's.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
