[GitHub] [kafka] novosibman commented on a diff in pull request #13782: Suggest for performance fix: KAFKA-9693 Kafka latency spikes caused by log segment flush on roll - trunk version
novosibman commented on code in PR #13782:
URL: https://github.com/apache/kafka/pull/13782#discussion_r1230002264
##
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##
@@ -976,6 +976,30 @@ public static void flushDir(Path path) throws IOException {
}
}
+/**
+ * Flushes dirty file to guarantee crash consistency.
+ *
+ * @throws IOException if flushing the file fails.
+ */
+public static void flushFile(Path path) throws IOException {
+if (path != null) {
+try (FileChannel fileChannel = FileChannel.open(path,
StandardOpenOption.READ)) {
+fileChannel.force(true);
+}
+}
+}
+
+/**
+ * Flushes dirty file quietly, logs warning when exception happens.
+ */
+public static void flushFileQuietly(Path path, String name) {
+try {
+flushFile(path);
+} catch (IOException e) {
+log.warn("Failed to flush {} at path {} with exception {}", name,
path, e);
Review Comment:
Third parameter removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] novosibman commented on a diff in pull request #13782: Suggest for performance fix: KAFKA-9693 Kafka latency spikes caused by log segment flush on roll - trunk version
novosibman commented on code in PR #13782:
URL: https://github.com/apache/kafka/pull/13782#discussion_r1223374923
##
storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java:
##
@@ -681,7 +687,12 @@ private static void writeSnapshot(File file, Map entri
try (FileChannel fileChannel = FileChannel.open(file.toPath(),
StandardOpenOption.CREATE, StandardOpenOption.WRITE)) {
fileChannel.write(buffer);
-fileChannel.force(true);
+}
+
+if (scheduler != null) {
+scheduler.scheduleOnce("flush-producer-snapshot", () ->
Utils.flushFileQuietly(file.toPath(), "producer-snapshot"));
+} else {
+Utils.flushFileQuietly(file.toPath(), "producer-snapshot");
Review Comment:
Open/close changes done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] novosibman commented on a diff in pull request #13782: Suggest for performance fix: KAFKA-9693 Kafka latency spikes caused by log segment flush on roll - trunk version
novosibman commented on code in PR #13782:
URL: https://github.com/apache/kafka/pull/13782#discussion_r1223373702
##
storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java:
##
@@ -430,11 +428,19 @@ public Optional lastEntry(long
producerId) {
* Take a snapshot at the current end offset if one does not already exist.
*/
public void takeSnapshot() throws IOException {
+takeSnapshot(null);
+}
+
+/**
+ * Take a snapshot at the current end offset if one does not already exist.
+ * Flush the snapshot asynchronously if scheduler != null
+ */
+public void takeSnapshot(Scheduler scheduler) throws IOException {
Review Comment:
IOException still will be thrown on open/write/close operations. Force
(flush) operation running by scheduler in a separate thread will write log
warning only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
