[
https://issues.apache.org/jira/browse/HDFS-14251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758281#comment-16758281
]
Kevin Risden commented on HDFS-14251:
-------------------------------------
JDK9+ changed the behavior of ForkJoinPool so that threads by default get no
permissions under the Java SecurityManager with the default thread factory.
This wasn't the case in JDK8.
[https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ForkJoinPool.html]
{code:java}
If a SecurityManager is present and no factory is specified, then the default
pool uses a factory supplying threads that have no Permissions enabled. The
system class loader is used to load these classes. Upon any error in
establishing these settings, default parameters are used. It is possible to
disable or limit the use of threads in the common pool by setting the
parallelism property to zero, and/or using a factory that may return null.
However doing so may cause unjoined tasks to never be executed.
{code}
[https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/concurrent/ForkJoinPool.html]
{code:java}
If no thread factory is supplied via a system property, then the common pool
uses a factory that uses the system class loader as the thread context class
loader. In addition, if a SecurityManager is present, then the common pool uses
a factory supplying threads that have no Permissions enabled. Upon any error in
establishing these settings, default parameters are used. It is possible to
disable or limit the use of threads in the common pool by setting the
parallelism property to zero, and/or using a factory that may return null.
However doing so may cause unjoined tasks to never be executed.{code}
From
[https://github.com/apache/hadoop/blob/branch-3.2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java#L213]
Based on discussion in SOLR-9515, doing any IO within the ForkJoinPool can
cause issues and should most likely not be used for this purpose. If the
ForkJoinPool must be used, then a way to set the custom thread factory would be
useful so the security policy can be passed in.
> BlockPoolSlice ForkJoinPool introduced by HDFS-13768 breaks under Java
> SecurityManager
> ---------------------------------------------------------------------------------------
>
> Key: HDFS-14251
> URL: https://issues.apache.org/jira/browse/HDFS-14251
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.10.0, 3.2.0, 3.1.2
> Reporter: Kevin Risden
> Priority: Major
>
> HDFS-13768 introduced ForkJoinPool to improve performance of BlockPoolSlice.
> The created ForkJoinPool doesn't pass in a custom thread factory and by
> default the thread factory prevents all privileges under the Java
> SecurityManager.
> This broke when integrating Hadoop 3.2.0 with Apache Solr in SOLR-9515.
> It isn't clear that the ForkJoinPool is necessary when a regular normal
> executor could work instead?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
