[
https://issues.apache.org/jira/browse/HDFS-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111284#comment-14111284
]
Colin Patrick McCabe commented on HDFS-6912:
--------------------------------------------
bq. Hey Colin. Did you verify that tmpfs supports fallocate going back to old
versions? Looking at the kernel git history, it was only added in mid 2012
(e2d12e22c59ce714008aa5266d769f8568d74eac) corresponding to version 3.5. So,
I'm not sure if it would be supported on el6 for example (maybe they backported
it, maybe not).
I believe the glibc {{posix_fallocate}} wrapper falls back to using {{write()}}
calls when {{fallocate}} itself is not supported by the kernel. There is some
discussion here:
https://lists.gnu.org/archive/html/bug-coreutils/2009-05/msg00207.html which
talks about:
bq. i.e. fall back to using write() as the glibc posix_fallocate()
implementation does.
But, I think it's simpler to just use {{write}} here. Any performance
advantage to using {{ftruncate}} + {{fallocate}} is going to be extremely tiny
(or nonexistent) since this file is only 8192 bytes. And {{write}} is much
more portable. So here is a new version that does that.
> SharedFileDescriptorFactory should not allocate sparse files
> ------------------------------------------------------------
>
> Key: HDFS-6912
> URL: https://issues.apache.org/jira/browse/HDFS-6912
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: caching
> Affects Versions: 2.5.0
> Environment: HDFS Data node, with 8 gb tmpfs in /dev/shm
> Reporter: Gopal V
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HDFS-6912.001.patch
>
>
> SharedFileDescriptor factory should not allocate sparse files. Sparse files
> can lead to a SIGBUS later in the short-circuit reader when we try to read
> from the sparse file and memory is not available.
> Note that if swap is enabled, we can still get a SIGBUS even with a
> non-sparse file, since the JVM uses MAP_NORESERVE in mmap.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
