Jira (PDB-2780) Consider making stockpile durable on OS X

Rob Browning (JIRA) Tue, 14 Jun 2016 12:57:07 -0700

Title: Message Title

Rob Browning created an issue

PDB-2780

Issue Type:	Improvement
Affects Versions:	PDB 4.2.0
Assignee:	Unassigned
Created:	2016/06/14 12:55 PM
Priority:	Normal
Reporter:	Rob Browning

On OS X, fsync() doesn't actually push the data all the way to disk. For that you have to call fcntl(fd, F_FULLSYNC), and the JVM (currently) doesn't.

So if we want stockpile to provide durability on OS X, we'll need to do that, or perhaps just wait for JDK 9: https://bugs.openjdk.java.net/browse/JDK-8080589

If we do want to pursue this, it should be possible via JNA. Vague hackish sketch:

;; project.clj

(def dependencies

  (remove

   nil?

   `[[org.clojure/clojure "1.8.0"]

     [org.apache.commons/commons-lang3 "3.4"]

     ~(when (.contains (System/getProperty "os.name") "OS X")

        '[net.n01se/clojure-jna "1.0.0"])]))

(defproject stockpile "0.0.1-SNAPSHOT"

  :description "Simple, durable Clojure queuing library"

  ;; :url "http://example.com/FIXME"

  :license {:name "Apache License Version 2.0"

            :url "http://www.apache.org/licenses/LICENSE-2.0"}

  :dependencies ~dependencies)

;; stockpile.clj

;; Looks like F_FULLFSYNC may be 51.

(def F_FULLFSYNC 51)  ;; Hard-coded.  Really?

(def fsync

  (if-not osx?

    (fn [x metadata?]

      (with-open [fc (FileChannel/open (as-path x)

                                       (into-array OpenOption []))]

        (.force fc metadata?)))

    (fn [f metadata?]

      (let [pathstr (-> (as-path x) .toAbsolutePath .toString)

            fd (jna/open ...)]

        (if-not (= -1 fd)

          (try

            (let [rc (jna/fcntl fd F_FULLFSYNC)]

              (when-not (= -1 rc)

                (error errno etc...)))

            (finally

              (check-call (jna/close fd))))

          (else error errno etc...))))))

Add Comment

This message was sent by Atlassian JIRA

--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PDB-2780) Consider making stockpile durable on OS X

Reply via email to