[
https://issues.apache.org/jira/browse/HDFS-9957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176651#comment-16176651
]
Jan Kunigk commented on HDFS-9957:
----------------------------------
The way I understand this manpage entry is:
Any user appliaction (C-binary, JVM) that calls mlock under Linux will get a
page aligned buffer.
POSIX says that it is valid for other implementations of mlock (e.g. UNIX,
etc.) to not do that automatically and leave it as a responsibility of the
application.
But, given that the Linux implementation's alignment feature will most likely
not go away and that HDFS is very unlikely to officially run under any other OS
than Linux in the near future, I don't really see a problem here.
Also, I don't understand how the mlocked region of a file used by the JVM
affects the JVM itself... The JVM process itself, including the heap lives in
anonymous user space memory, no? I.e. it is not backed by a file (it may be
swapped, but not paged)... Libraries used by the JVM may indeed also be mmaped,
but those are different invocations of mmaps/mlocks.
That's the way I understand it...
> HDFS's use of mlock() is not portable
> -------------------------------------
>
> Key: HDFS-9957
> URL: https://issues.apache.org/jira/browse/HDFS-9957
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: native
> Affects Versions: 2.7.2
> Environment: Any UNIX system other than Linux
> Reporter: Alan Burlison
> Assignee: Alan Burlison
>
> HDFS uses mlock() to lock in the memory used to back java.nio.Buffer.
> Unfortunately the way it is done is not standards-compliant. As the Linux
> manpage for mlock() says:
> {quote}
> Under Linux, mlock(), mlock2(), and munlock() automatically round
> addr down to the nearest page boundary. However, the POSIX.1
> specification of mlock() and munlock() allows an implementation to
> require that addr is page aligned, so portable applications should
> ensure this.
> {quote}
> The HDFS code does not do any such alignment, nor is it true that the backing
> buffers for java.nio.Buffer are necessarily page aligned. And even if the
> address was aligned by the code, it would end up calling mlock() on other
> random JVM data structures that shared the same page. That seems potentially
> dangerous.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
