Github user ahgittin commented on the issue:
https://github.com/apache/brooklyn-server/pull/791
@geomacy @aledsage I hadn't realized this either; javadocs and online
searches suggest the `fd.sync()` may be needed.
i'm curious whether this really is the issue -- at least on Mac -- as i've
done some testing and the `sync` seems to be very fast whereas `write` is what
takes the time. this is true even under contention: `write`s contend with
each other (8 files written concurrently take 8+ times as long as writing 1
file; if the OS cache is in play i'd expect CPU contention which with 4 cores
we'd expect 8 files to take twice as long, which is what happens if we do a
`System.arraycopy` instead of a `write`) and if there's background disk
activity it also slows down substantially. numbers and code below.
worth having this i agree, based on what i've read, though i have one worry
-- if `sync` is done _after_ the `close`, i always get the
`SyncFailedException`; and it looks like the code in this PR is synching after
close. we might want to understand that before merging?
i also came across the code at
https://github.com/apache/activemq-apollo/blob/trunk/apollo-util/src/main/scala/org/apache/activemq/apollo/util/IOHelper.java#L272
: here to get sync they included some custom JNI calls. this makes me wonder
whether -- although things acted nicely in my tests -- there aren't actually
any strong guarantees with java+mac, and you need that type of low-level
library?
although another part of me wonders if
https://issues.apache.org/jira/browse/BROOKLYN-526 was in fact caused by
something completely different (disk corruption?).
---
one thread:
```
t0 took 1.390 seconds to write 1,048,576,000 bytes to file
t0 took 0.010 seconds to sync
t0 took 0.000 seconds to close
```
8 threads:
```
t7 took 13.180 seconds to write 1,048,576,000 bytes to file
t7 took 0.099 seconds to sync
t7 took 0.000 seconds to close
```
code:
```
public class FileWrite {
public static void main(String[] args) throws Exception {
write("warmup");
int NUM_THREADS = 8;
for (int x=0; x<NUM_THREADS; x++) {
final String id = "t"+x;
new Thread() {
public void run() {
try {
write(id);
} catch (Exception e) {
e.printStackTrace();
}
}
}.start();
}
}
public static void write(String id) throws Exception {
Stopwatch t = Stopwatch.createStarted();
File file = File.createTempFile("deleteme", ".dat"); // put your
file here.
System.out.printf("%s took %.3f seconds to create %s%n", id,
t.elapsed(TimeUnit.NANOSECONDS) / 1e9, file.getAbsolutePath()); t.reset();
t.start();
FileOutputStream fos = new FileOutputStream(file);
byte[] bytes = new byte[32 * 1024];
// byte[] bytes2 = new byte[32 * 1024];
for (long l = 0; l < 1000 * 1024 * 1024; l += bytes.length) {
fos.write(bytes);
// System.arraycopy(bytes, 0, bytes2, 0, 32 * 1024);
}
System.out.printf("%s took %.3f seconds to write %,d bytes to
file%n", id, t.elapsed(TimeUnit.NANOSECONDS) / 1e9, file.length()); t.reset();
t.start();
fos.getFD().sync();
System.out.printf("%s took %.3f seconds to sync%n", id,
t.elapsed(TimeUnit.NANOSECONDS) / 1e9); t.reset(); t.start();
fos.close();
System.out.printf("%s took %.3f seconds to close%n", id,
t.elapsed(TimeUnit.NANOSECONDS) / 1e9); t.reset(); t.start();
File file2 = File.createTempFile("deleteme", ".dat"); // put your
file here.
java.nio.file.Files.move(file.toPath(), file2.toPath(),
StandardCopyOption.ATOMIC_MOVE);
System.out.printf("%s took %.3f seconds to move to %s%n", id,
t.elapsed(TimeUnit.NANOSECONDS) / 1e9, file2.getAbsolutePath()); t.reset();
t.start();
}
}
```
---
