Maxwell-Guo commented on code in PR #2777:
URL: https://github.com/apache/cassandra/pull/2777#discussion_r1373087987
##########
src/java/org/apache/cassandra/config/Config.java:
##########
@@ -1147,6 +1148,8 @@ public enum DiskAccessMode
mmap,
mmap_index_only,
standard,
+ legacy,
Review Comment:
what does legacy in diskaccess mode stand for ? @jacek-lewandowski
@amitdpawar ,seems lack of description at least.
##########
src/java/org/apache/cassandra/config/DatabaseDescriptor.java:
##########
@@ -511,6 +511,41 @@ else if (conf.commitlog_sync_period.toMilliseconds() != 0)
logger.debug("Syncing log with a period of {}",
conf.commitlog_sync_period.toString());
}
+ // Only mmap & Direct-I/O access modes are supported. Standard access
mode is preassumed if Compressed or Encrypted segment is used.
+ // This ensures default behavior is unchanged.
+ if (conf.commitlog_disk_access_mode != Config.DiskAccessMode.auto
+ && conf.commitlog_disk_access_mode != Config.DiskAccessMode.legacy
Review Comment:
I'm a little confused about the logic of this code. What do you want to do
here ?
if commitlog access mode is direct or mmap then exception throw?
##########
src/java/org/apache/cassandra/db/commitlog/DirectIOSegment.java:
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.commitlog;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.file.Files;
+
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.io.FSWriteError;
+import org.apache.cassandra.io.util.FileUtils;
+
+/*
+ * Direct-IO segment. Allocates ByteBuffer using ByteBuffer.allocateDirect and
align
+ * ByteBuffer.position, ByteBuffer.limit and FileChannel.position to page size
(4K).
+ * Java-11 forces minimum page size to be written to disk with Direct-IO.
+ */
+public class DirectIOSegment extends CommitLogSegment
+{
+ ByteBuffer original;
+ static int minimumAllowedAlign;
+
+ // Needed to track number of bytes written to disk in multiple of page
size.
+ long lastWritten = 0;
+
+ /**
+ * Constructs a new segment file.
+ *
+ * @param commitLog the commit log it will be used with.
+ */
+ DirectIOSegment(CommitLog commitLog, AbstractCommitLogSegmentManager
manager)
+ {
+ super(commitLog, manager);
+
+ // mark the initial sync marker as uninitialised
+ int firstSync = buffer.position();
+ buffer.putInt(firstSync + 0, 0);
+ buffer.putInt(firstSync + 4, 0);
+
+ // Testing shows writing initial bytes takes some time. During peak
load, it helps
+ // lot and Syncer thread can actually do flush activity. Making this
initial
+ // slow operation to be executed by Allocator thread here.
+ int oldLastSyncedOffset = lastSyncedOffset;
+ // 8 bytes are written above.
+ lastSyncedOffset = 8;
+ flush(0, lastSyncedOffset);
+ lastSyncedOffset = oldLastSyncedOffset;
+ }
+
+ ByteBuffer createBuffer(CommitLog commitLog)
Review Comment:
Hello , a new question here, and that is what I want to know for a long
time: how can we ensure the correctness of this code, does it need to be
tested， I mean the class of this Segment ? @blambov , @jacek-lewandowski ，wdyt
？
##########
src/java/org/apache/cassandra/db/commitlog/DirectIOSegment.java:
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.commitlog;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.file.Files;
+
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.io.FSWriteError;
+import org.apache.cassandra.io.util.FileUtils;
+
+/*
+ * Direct-IO segment. Allocates ByteBuffer using ByteBuffer.allocateDirect and
align
+ * ByteBuffer.position, ByteBuffer.limit and FileChannel.position to page size
(4K).
+ * Java-11 forces minimum page size to be written to disk with Direct-IO.
+ */
+public class DirectIOSegment extends CommitLogSegment
+{
+ ByteBuffer original;
+ static int minimumAllowedAlign;
+
+ // Needed to track number of bytes written to disk in multiple of page
size.
+ long lastWritten = 0;
+
+ /**
+ * Constructs a new segment file.
+ *
+ * @param commitLog the commit log it will be used with.
+ */
+ DirectIOSegment(CommitLog commitLog, AbstractCommitLogSegmentManager
manager)
+ {
+ super(commitLog, manager);
+
+ // mark the initial sync marker as uninitialised
+ int firstSync = buffer.position();
+ buffer.putInt(firstSync + 0, 0);
+ buffer.putInt(firstSync + 4, 0);
+
+ // Testing shows writing initial bytes takes some time. During peak
load, it helps
+ // lot and Syncer thread can actually do flush activity. Making this
initial
+ // slow operation to be executed by Allocator thread here.
+ int oldLastSyncedOffset = lastSyncedOffset;
+ // 8 bytes are written above.
+ lastSyncedOffset = 8;
+ flush(0, lastSyncedOffset);
+ lastSyncedOffset = oldLastSyncedOffset;
+ }
+
+ ByteBuffer createBuffer(CommitLog commitLog)
Review Comment:
Hello , a new question here, and that is what I want to know for a long
time: how can we ensure the correctness of this code, does it need to be
tested， I mean the class of this Segment ? @blambov , @jacek-lewandowski ，wdyt
？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
