[
https://issues.apache.org/jira/browse/HADOOP-13351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15367098#comment-15367098
]
Mingliang Liu edited comment on HADOOP-13351 at 7/8/16 2:12 AM:
----------------------------------------------------------------
{quote}
the OS declining to set a large send buffer size for a socket.
{quote}
Thanks [~fabbri] for the explanation. In {{StandardSocketOptions}}, an attempt
to set the socket send buffer to larger than its maximum size causes it to be
set to its maximum size (refer to [link
here|https://docs.oracle.com/javase/7/docs/api/java/net/StandardSocketOptions.html#SO_SNDBUF]).
I think the value for {{Socket}} would work in the same way. This way, a
larger value like 512KB would not cause problems for set operation. However,
the test may still fail since the maximum size, as the default size, is system
dependent.
That being said, testing the specified value is indeed tricky. To make the test
assertion covers this case, how about comparing the hinted buffer size? I see
very limited chance that the test fails, but it does confirm that, the config
key works as expected.
Sample code as following:
{code:java}
@Test
public void testSpecifiedSendBufferSize() throws IOException {
conf.setInt(DFS_CLIENT_SOCKET_SEND_BUFFER_SIZE_KEY, 256 * 1024); // a big
hint
final Socket s1 = createSocket();
conf.setInt(DFS_CLIENT_SOCKET_SEND_BUFFER_SIZE_KEY, 1024); // a
small hint
final Socket s2 = createSocket();
LOG.info("The real big buffer size is {}, small buffer size is {}",
s1.getSendBufferSize(), s2.getSendBufferSize());
assertTrue(s1.getSendBufferSize() > s2.getSendBufferSize());
}
{code}
What's your thought?
was (Author: liuml07):
{quote}
the OS declining to set a large send buffer size for a socket.
{quote}
Thanks [~fabbri] for the explanation. In {{StandardSocketOptions}}, an attempt
to set the socket send buffer to larger than its maximum size causes it to be
set to its maximum size (refer to [link
here|https://docs.oracle.com/javase/7/docs/api/java/net/StandardSocketOptions.html#SO_SNDBUF]).
I think the value for {{Socket}} would work in the same way for socket. This
way, a larger value like 512KB would not cause problems for set operation.
However, the test may still fail as the maximum size, as the default size, is
system dependent.
That being said, testing the specified value is indeed tricky. To make the test
assertion covers this case, how about comparing the hinted buffer size? I see
very limited chance that the test fails, but it does confirm that, the config
key works as expected.
Sample code as following:
{code:java}
@Test
public void testSpecifiedSendBufferSize() throws IOException {
conf.setInt(DFS_CLIENT_SOCKET_SEND_BUFFER_SIZE_KEY, 256 * 1024);
final Socket s1 = createSocket();
conf.setInt(DFS_CLIENT_SOCKET_SEND_BUFFER_SIZE_KEY, 1024);
final Socket s2 = createSocket();
LOG.info("The real big buffer size is {}, small buffer size is {}",
s1.getSendBufferSize(), s2.getSendBufferSize());
assertTrue(s1.getSendBufferSize() > s2.getSendBufferSize());
}
{code}
What's your thought?
> TestDFSClientSocketSize buffer size tests are flaky
> ---------------------------------------------------
>
> Key: HADOOP-13351
> URL: https://issues.apache.org/jira/browse/HADOOP-13351
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.8.0, 3.0.0-alpha1
> Reporter: Aaron Fabbri
> Assignee: Aaron Fabbri
> Attachments: HADOOP-13551.001.patch
>
>
> {{TestDFSClientSocketSize}} has two tests that assert that a value that was
> set via {{java.net.Socket#setSendBufferSize}} is equal to the value
> subsequently returned by {{java.net.Socket#getSendBufferSize}}.
> These tests are flaky when we run them. The occasionally fail.
> This is expected behavior, actually, because
> {{Socket#setSendBufferSize()}}[is only a
> hint|https://docs.oracle.com/javase/7/docs/api/java/net/Socket.html#setSendBufferSize(int)].
> (Similar to how the underlying libc {{setsockopt(SO_SNDBUF)}} works).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
