[
https://issues.apache.org/jira/browse/HDFS-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-4491:
--------------------------------
Attachment: HDFS-4491--n12.patch
bq. Does MiniCluster work with ports in a different way when run on Windows?
I can repro the problem consistently on Mac, so the problem is not specific to
Windows.
bq. Definitely NetUtils.getFreeSocketPort which is used by MiniDFSCluster to
find ports for data nodes is a source of problems as it finds an available
socket, the closes it, then returns the port number to the caller.
Yes, this could be a race, but I'm seeing a different problem. I'm seeing that
the test consistently uses the same static ports: 50020 for RPC and 50075 for
HTTP.
I traced the problem to {{MiniDFSCluster#setupDatanodeAddress}}. This method
doesn't set ephemeral ports for HTTP and RPC in certain conditions. I'm
attaching version 12 of the patch with a small change in that method to correct
it. I've had multiple successful parallel test runs with this patch, even
while using netcat to hold ports 50020 and 50075. Does this change look good?
I think this is the last issue we would need to resolve before committing.
> Parallel testing HDFS
> ---------------------
>
> Key: HDFS-4491
> URL: https://issues.apache.org/jira/browse/HDFS-4491
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
> Affects Versions: 3.0.0
> Reporter: Tsuyoshi OZAWA
> Assignee: Andrey Klochkov
> Attachments: HDFS-4491--n10.patch, HDFS-4491--n10.patch,
> HDFS-4491--n11.patch, HDFS-4491--n11.patch, HDFS-4491--n12.patch,
> HDFS-4491--n2.patch, HDFS-4491--n3.patch, HDFS-4491--n4.patch,
> HDFS-4491--n5.patch, HDFS-4491--n6.patch, HDFS-4491--n7.patch,
> HDFS-4491--n8.patch, HDFS-4491--n9.patch, HDFS-4491.patch
>
>
> Parallel execution of HDFS tests in multiple forks. See HADOOP-9287 for
> details.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
